gauging online community health

September 12, 2009 6:12 PM

What are the five metrics you would use to measure the health* of an online community?

That pretty much sums it up. I'm open to more or less than five, but I'm trying to keep it the list of measurements short and the descriptions simple.

*how 'health' is defined is yours to decide.

posted by A Terrible Llama to computers & internet (10 comments total)

-Total participating members (logging in, say, at least once a week) divided by total registered members.
Number of comments in a day / total registered members.
-The inverse of insults per comment.
posted by dunkadunc at 6:36 PM on September 12

Rate of growth (high is good)
rate of participation (high is good)
enthusiasm (high is good)
rancor (low is good)
existence of and detonation of trigger subjects (low is good)
posted by Chocolate Pickle at 6:37 PM on September 12

A body of shared references, in-jokes, shared experiences. A definable "culture", if you will.

A reasonable, but not overwhelming, influx of newcomers.

Old-timers who are open to the newcomers

An awareness of the community's biases, in terms of whose voices are heard louder, which viewpoints aren't respected, as well as a willingness to work on changing those biases so that the community is more open.

A sense of commitment to the community, but again, not allowing that commitment to calcify into cliqueishness.

Yeah, I'm utopian. I own it.
posted by jason's_planet at 6:53 PM on September 12

Firstly, I'm going to assume by online community you mean something akin to WWW forums, and not IRC or instant message services. If I were going to describe the 'health' as separate from other metrics, I'd measure:

* the number of subpoenas the site administration receives
* the willingness of people to share personal information like location, occupation or names with others in the community
* the number of bans implemented
* a ratio of spam:ham
* the number threads without replies from anyone else (self replies don't count)
posted by pwnguin at 8:12 PM on September 12

One metric, for me, is how much the site spills into its members' "real" lives. Has anyone gotten a job because of it? Has anyone met on it and started dating? Gotten married? How far have members traveled to see each other? If so-and-so is a member and his wife isn't, does she still know any of the other members' names? Do the other members know his wife's name? Can social capital from the site be "spent" in the real world? (Simple test: If someone is a trusted regular on the site, are you willing invite them into your home?) And so on.

A website that crosses over into its users' daily lives is a community. A website that people stop talking or thinking about when they sign off, that's just crowdsourced television.
posted by nebulawindphone at 8:33 PM on September 12 [2 favorites has favorites]

I'm not sure rate of growth is a good metric. For one thing, a lot of people prefer it when there isn't an influx of new members on an online community they're part of, because it makes the community less personal and you get a lot of trolls and single-posters and it kills the feeling of "community". Also, some communities are growing at a steady and slow rate but are very active and the members are trusted people, and good friends.

My five are:

1. Active mods that ensure the quality of the posts (I'm talking about a forum-style community).

2. A common culture that both draws people originally to the grew and develops within the group. For example, on a forum I frequent, people are drawn to it originally because it's one of the best gay youth forums, so that gives some common ground, and we have our jokes, our shared experiences, etc.

3. The number of people that maintain contact through other means. For example, emails, facebook, instant messaging, or even face-to-face contact.

4. The discussion on the threads. Is it "yeah man that's so cool," just so that you can reply to a topic and show your support/like, or does it develop into an actual discussion?

5. Number of very frequent members divided by number of occasional members. For example, if you have 10 people posting all the time and another 200 coming and going that's not much of a community, wheras if you have 60 frequent posters and another 150 coming and going you get a much different feeling.

Also, in sociology there are a few different criteria for sorting groups into categories:

1. Size - the smaller the group, the higher the commitment of each individual member, and interpersonal relationships are generally tighter.
2. Criteria for acceptance into the group, called closed vs. open groups. A closed group is one which is difficult to get into. There are two other ways to divide this: if the criteria is related to someone's attributes or to his accomplishments. Again, if related to accomplishments the commitment will be higher.
3. The nature of the relations in the group - primary vs. secondary groups. The goal of primary groups is mainly to maintain the group, the well-being of the group, while a secondary group has outer goals, for example a laborer's union which is interested in the well-being of workers. Also, primary groups more often answer to the emotional needs of the individuals in the group.
4. The structure of communication in the group. Does it all go through one person? Does everyone have direct contact with everyone? etc.
5. The decision-making process in the group. Is the process conducted democraticaly or does one person make the decisions? Does everyone get an equal say?
6. The level of formality of the group. A formal group was created by an outside factor and is managed by it, like a class of schoolchildren, wheras an informal group could be a group of kids who get together to play monopoly every friday.
7. The nature of belonging to the group - voluntary vs. involuntary. A voluntary group is a group of friends or a band or a soccer team, and an involuntary group is a class of schoolchildren - they have to be in school.
8. The method of social supervision of the group - groups where there are formal sanctions and punishments administered by a leader or higher power vs. informal sanctions and punishments such as distrust, criticism, etc.
9. Level of social unity - different factors can affect this, like an outer threat, a shared goal, criteria of acceptance, size of the group, etc.


note: direct translation from my high school sociology textbook
posted by alona at 10:07 PM on September 12 [1 favorite has favorites]

Size: Number of members
Growth: The rate at which the community is growing
Stickiness: Number of members logging in weekly
Intervention: Threads to moderation ratio
Google: How well you rank against your competitors and where you place for your keywords

Google Analytics can also give you another set of metrics to work with that are more traditional. Bounce rates and time per visit are interesting and there's a lot of informative numbers buried in there.

I also like jason's_planet's list but I would reject openness as a defacto requirement or metric. I ran an online community with 30,000 members for several years. It said on the registration form "XYZ is a pro-choice site." We would occasionally suffer an onslaught of fundies and wielded the band hammer very liberally around that topic.
posted by DarlingBri at 5:33 AM on September 13

Thanks everyone!
posted by A Terrible Llama at 2:21 AM on September 14

out of interest, can I ask what this is for?
posted by alona at 5:43 PM on September 14

I have to evaluate a bunch of websites, and I wanted to clarify for myself what criteria I wanted to use, and sort of compare it against what other people would use. I'm still finalizing the evaluation, but this was helpful.

If it were just one, I'd probably just write out my thoughts at a high level, but for many, I need some kind of evaluation framework.
posted by A Terrible Llama at 3:48 PM on September 15