Whatever it is called, we still need a nice simple single number that is a rough measure of how much good content someone has contributed, as judged by all others on that site. "Reputation" is a reasonable name that captures this concept well enough, but there are probably others out there that would do too. Maybe "awesomeness score", "godliness status", "total worth as a human being", or something ;-)

The core value of any site is its ability to provide good answers. That means you need to have domain experts willing to spend their free time providing this value for you. We can't pay them, but what we can do it give them public recognition. The system doesn't have to be perfect (what system is?), but these experts providing the core value of your site need something to keep them engaged. The awesomeness score reputation number is a good way to do that.

Reputation has to actually mean something, else it is just a useless "atta boy". Some of the abilities should be dependent on reputation. It's fine for someone to get edit privileges after having enough edits approved. It's quite another to give someone policy-setting powers that isn't vested in the site and hasn't proven it by providing a lot of content.

These sites are all still young with small numbers of users. By and large, most people here actually care about the sites. That will change if we are successful. That's OK by itself, but the fatal error is giving the future masses any real power.

These sites must NOT be democracies. The masses are their own worst enemy. It is to any one user's advantage to get their question answered regardless of the effect on the site. However in the long run, the health of the site is far more important than any one question.

When a site gets larger, there is clamor from the masses for more leniency, resistance to closing bad questions, accusations of being "unwelcoming", mistreating newbies, and the like. If these are acted upon, then site quality declines, the experts get disillusioned, feel unappreciated, and eventually leave.

This is exactly what has happened to several SE sites when SE started valuing clicks over quality, probably because they associated clicks with revenue. They forgot that it was that quality that attracted the clicks in the first place. No matter how much the masses bitch and moan about actions to keep quality high, they are still ultimately there because of that quality.

To solve this, you give the core of resident experts the power to keep the place clean and set the policy. Sure, others can help with things like edits to fix bad wording or spellings, deleting inappropriate comments, etc, but the policy-setting powers need to be in the relatively few hands of the core group that has shown a commitment to the place by consistently providing good content. This isn't as much a reward for the service, as a recognition that these are the people that "make" the place, and need to be given the freedom to run it according to their vision.

That means there needs to be a way to identify the top committed contributors. This is the other aspect to "reputation". Means of figuring reputation need to be designed for this purpose. When done right, it will serve the public recognition purpose of reputation well enough, even if just as a byproduct.

Reputation must therefore measure "amount of good content contributed, as judged by the consumers of that content". SE got this largely right, although a few tweaks would make it better. Reputation should be:

1. Gained from upvotes on answers to questions in domain-content categories. This means you don't earn rep by asking questions, nor by anything in meta. You earn rep by providing good answers. How much others agreed or disagreed with things you said on meta shouldn't matter.
2. Lost by some types of downvotes on answers in domain-content categories. This is the reverse of #1, largely. One problem on SE they never really addressed was malicious and vandalous downvotes. I have felt for a long time that the best way to prevent that, and get honest downvotes in general is to make them all public. However, there is strong resistance to that. A good compromise would be to allows two types of downvotes, anonymous and "signed". Anonymous downvotes affect answer score, and therefore sort order, but nothing more. Signed downvotes also affect rep, but require a comment explaining the reason, or a "me too" on someone else's downvote comment.
3. Can be different per category. For example, votes on meta generally shouldn't affect rep at all, although that is settable as with other categories. On EE we have a Papers category. Writing a paper is more work, and it's held to a higher standard than questions and answers in the Q&A category. In that case, votes on the question (there are no answers in Papers) should mean more, both up and down. Overall, each category has mod-selectable numbers for how much upvotes and (signed) downvotes mean on questions and answers, for a total of 4 tweak factors. No special case then needs to be made for meta or any other category.

Reputation must also be simple. Just a single number is best. After all, we're really just trying to measure one thing, which is roughly "appreciated contribution level". Multiple numbers just confuse and/or dilute that.

Another tweak relative to SE would be having rep be based on "recent" contributions. Put another way, the rep from old contributions slowly decreases over time, eventually falling to 0. The point of this is to avoid making someone "eternal moderator for life". If someone hasn't been active on the site for a few years, they should no longer have a say in how it is run. This also gets around the problem of it being impossible for new active contributors to become part of the core group. If you provide a lot of good content over two years or so, you should be "in".

The exact parameters need to be tweakable, in no small part because it will take experience to find good values. A good start might be everything in the last year (or 2 years?) counts full, then decays to half its value every year after that. Or, maybe just a linear downgrade to 0 over 5 years after the initial year or two. The exact mechanism isn't important, at least not right now.

posted over 3 years ago

CC BY-SA 4.0

Olin Lathrop‭

45 112 324 110