Communities

Writing
Writing
Codidact Meta
Codidact Meta
The Great Outdoors
The Great Outdoors
Photography & Video
Photography & Video
Scientific Speculation
Scientific Speculation
Cooking
Cooking
Electrical Engineering
Electrical Engineering
Judaism
Judaism
Languages & Linguistics
Languages & Linguistics
Software Development
Software Development
Mathematics
Mathematics
Christianity
Christianity
Code Golf
Code Golf
Music
Music
Physics
Physics

Dashboard
Notifications
Mark all as read
Q&A

Welcome to Codidact Meta!

Codidact Meta is the meta-discussion site for the Codidact community network and the Codidact software. Whether you have bug reports or feature requests, support questions or rule discussions that touch the whole network – this is the site for you.

What will become of reputation?

+5
−0

For a while now, the reputation topic has (re-)surfaced repeatedly with the the Codidact team taking a slight stance to remove reputation in the long run. Everytime this was mentioned, there were at least some commenters who argued against this motion.

@luap42 announced a discussion thread about it three months ago in the news for the abilities update:

We will no longer base your abilities on reputation, but we're not taking it away (yet?). We've heard you loud and clear: some communities and participants want a quick "score" number. Our plan is still to remove it, but we’ll be thinking of reasonable replacements that suit all communities. You can expect a separate discussion thread in a few days.

With the recent header design change, it was mentioned again, this time even in bold:

This is part of our move away from reputation (although we will not remove the option to have reputation for now).

At least for me (and I think for some other users as well), there are now several questions but the biggest one is: What will become of reputation? Will it be removed? What is the plan right now for it? When and how can we discuss on that matter? (I would like to participate in a discussion regarding the future of reputation as I also lean to the side favouring it.)

Why does this post require moderator attention?
You might want to add some details to your flag.
Why should this post be closed?

0 comments

3 answers

+6
−0

(Personal opinion, not official anything.)

I'm one of the people who, from the beginning, wanted to change how we handle reputation, because while Someplace Else got some things right with it, their setup also produced some degenerate results that I don't want to replicate.

The biggest problem, IMO, was tying privileges to raw reputation without looking deeper. On one of the sites I used to moderate, we fairly often saw poor answers on popular questions shoot up in (net) votes, and one-time posters suddenly had privileges they didn't really know how to use. This created extra work for community curators. We were keen to not repeat that mistake here at Codidact, and we now have an abilities system that ignores reputation and instead looks at the underlying actions -- your posts, suggested edits, and flags. For the purpose of abilities, an answer at net +5 counts the same as one at net +500; we're looking for "positive" and aren't measuring how positive. We think that someone who contributes ten positive answers has shown more fluency than someone with a single very popular answer.

So great, we've done that -- abilities are decoupled from reputation. What, then, of reputation?

I'm not opposed to having a single "bragging rights" number, whether it's called reputation or score or karma or something else. Communities that want to display it can; those that don't can omit it or even turn it off entirely. And I hear the requests for different weightings; not all post types in all categories are created equal. We currently allow communities to set the rep gain/loss for votes on each post type, without regard to category. That's an artifact of the original architecture, before we added categories, and I grant that it is not ideal. I'd like to be able to allow better, but still simple, configuration of this bragging-rights number. I'd like to replace the current rep configuration with a matrix: categories on one axis, post types on the other, rep for upvotes and downvotes within. This would allow a community to say that papers are worth more than blog posts even though they're both articles, remove downvote penalties on meta because working out policies and such inherently involves posting unpopular options for voting, remove rep from a sandbox entirely, and so on.

If we make rep more configurable in this way, I would want us to agree that this is the only way to gain or lose rep. It's about bragging rights from the posts you contribute, as another answer argues. All of the other valuable contributions people make to a community (edits, curation, etc) fall outside the rep system, to avoid diluting its signal and to avoid making the code even more complex.

Why does this post require moderator attention?
You might want to add some details to your flag.

1 comment

I agree, except that the resulting rep is useful for more than just "bragging rights" as you put it. That's one of the two things it's for. The other is to identify the few key core people that put serious effort into the site. For that, a great answer is better and should count more than one that simply was positive. However, the thresholds should be so that a few 10s or even 100s (for an active mature site) shouldn't make much difference. In SE context, 1000 rep should be a small bump. Olin Lathrop‭ 4 months ago

+5
−0

Back in the real old days, we had a real long discussion about reputation and privileges. (exhibit A, B, C and likely many other discussions in other threads)

What emerged from those discussions was mainly that:

  • People want something to feel recognized for their contributions.
  • People want to see, how helpful they are to the community.
  • Some people like reputation because they know it.
  • Reputation isn't good to decide, who's trustworthy to do moderation actions (editing, closing, deleting...), because it's focused on posts and expertize
  • Reputation can produce negative behaviors

Based on that, we collectively concluded, that "classical reputation" has a lot of flaws. It combines aspects of recognition, feedback and privilege. And while it is an acceptable compromise of these three factors; we thought we might be able to do better by splitting them up.

So we did.

We started thinking of replacements of reputation for purposes of determining trust and privileges. At first, we looked at Discourse's trust level model. Instead of an unbound, continuous reputation, you would only have ca. 5 levels, which would indicate how much you could be trusted. They would be calculated based on all kind of things: number of posts, number of flags, number of edits, number of votes and their qualitiy.

However, that system was -- in our opinion -- still flawed: You only have one path of progression. You couldn't distinguish users who were good at editing to users who were good at detecting off-topic posts. We started thinking about providing multiple ways to get one trust level (AKA X edits or X successful flags), but things would just get more complicated and it'd still not solve the core problem:

We can only trust you to do things you show to be trustworthy at.

Finally, we developed the Abilities system. The core idea: dead simple. If you show that you are continuously making good edit suggestions to post, you'll eventually be trusted to make edits without supervision. Likewise for moderation.

But that leaves us with recognition and feedback. This is actually hard. Many people feel appreciated by different stuff.

Ultimately, we came to the conclusion, that we should show different statistical details about a user on their "user card", so that other users can see, whether they are experienced contributors or not. (One thing we also considered, but which hasn't been worked on so far and which hasn't experienced more thought recently, is to show something like a "tag expert" badge. When you're making helpful contributions to the Python tag, you'll eventually get a "Python expert" badge under your user card. I still like this idea, but it needs a lot more consideration.)

This is, what the recent update introduced. Reputation-only display was amended by more specific profile details, such as number of posts or received votes.

But, especially in this area, we've received a lot of criticism and objections. People just like having a small number going up when they make useful posts. Hence, we are not going to "remove reputation" for now. Instead, we are deprioritizing it and experiment with other options, such as the one introduced yesterday. Also, in our long tradition of empowering the communities, we wanted to make these details as customizable as possible. Therefore, communities are able to decide, which details to show. If a community wants rep and only rep, that's fine. If they desperately want to get rid of rep, now they finally can.

I'm seeing this change as an experiment. But I also think, that the current solution (possibly combined with the expert-badge) is a good one and could be, what we ultimately end up with. Only time will be able to tell.

This leaves us, last but not least, with feedback. Reputation is a nice feedback, because it goes up when you do nice things and it goes down when you do bad things. Reputation going up also has the advantage of making you feel nice.

I know that this has been removed right now without replacement. I know that this isn't optimal. I'm also a bit annoyed by it. But, we are going to add some feedback mechanism (whether something fancy or just a list of recent votes). It's high priority on our TODO list, but that still means, that it'll be a while, because we are all only volunteeres here. Maybe, I'll add the reputation number in the header again, at least for the time being.

So, don't worry, the feed will be back again soon. (:P)

Why does this post require moderator attention?
You might want to add some details to your flag.

3 comments

I agree that putting the rep number back in the header until we have those other feedback mechanisms would be a good idea. That number changing is currently the only indicator people have that people are voting on their posts. And particularly if there are downvotes, you want to know so you can try to fix whatever the problem is. We don't make it easy to find those votes (which is part of that high-priority task to fix), and now we've made it harder to even notice they happened. Monica Cellio‭ 4 months ago

I completely agree with this, except for the "Reputation is a nice feedback, because it goes up when you do nice things and it goes down when you do bad things." part. That it does so makes it a form of extrinsic motivation, which isn't actually a good thing in the long term - it brings people in, but on average, decreases long term contribution Mithrandir24601‭ 4 months ago

@Mith: That doesn't make sense. Having a score that keeps going up when you do good things incentivizes you to keep doing good things, not stop doing them. Olin Lathrop‭ 4 months ago

+1
−2

Whatever it is called, we still need a nice simple single number that is a rough measure of how much good content someone has contributed, as judged by all others on that site. "Reputation" is a reasonable name that captures this concept well enough, but there are probably others out there that would do too. Maybe "awesomeness score", "godliness status", "total worth as a human being", or something ;-)

The core value of any site is its ability to provide good answers. That means you need to have domain experts willing to spend their free time providing this value for you. We can't pay them, but what we can do it give them public recognition. The system doesn't have to be perfect (what system is?), but these experts providing the core value of your site need something to keep them engaged. The awesomeness score reputation number is a good way to do that.

Reputation has to actually mean something, else it is just a useless "atta boy". Some of the abilities should be dependent on reputation. It's fine for someone to get edit privileges after having enough edits approved. It's quite another to give someone policy-setting powers that isn't vested in the site and hasn't proven it by providing a lot of content.

These sites are all still young with small numbers of users. By and large, most people here actually care about the sites. That will change if we are successful. That's OK by itself, but the fatal error is giving the future masses any real power.

These sites must NOT be democracies. The masses are their own worst enemy. It is to any one user's advantage to get their question answered regardless of the effect on the site. However in the long run, the health of the site is far more important than any one question.

When a site gets larger, there is clamor from the masses for more leniency, resistance to closing bad questions, accusations of being "unwelcoming", mistreating newbies, and the like. If these are acted upon, then site quality declines, the experts get disillusioned, feel unappreciated, and eventually leave.

This is exactly what has happened to several SE sites when SE started valuing clicks over quality, probably because they associated clicks with revenue. They forgot that it was that quality that attracted the clicks in the first place. No matter how much the masses bitch and moan about actions to keep quality high, they are still ultimately there because of that quality.

To solve this, you give the core of resident experts the power to keep the place clean and set the policy. Sure, others can help with things like edits to fix bad wording or spellings, deleting inappropriate comments, etc, but the policy-setting powers need to be in the relatively few hands of the core group that has shown a commitment to the place by consistently providing good content. This isn't as much a reward for the service, as a recognition that these are the people that "make" the place, and need to be given the freedom to run it according to their vision.

That means there needs to be a way to identify the top committed contributors. This is the other aspect to "reputation". Means of figuring reputation need to be designed for this purpose. When done right, it will serve the public recognition purpose of reputation well enough, even if just as a byproduct.

Reputation must therefore measure "amount of good content contributed, as judged by the consumers of that content". SE got this largely right, although a few tweaks would make it better. Reputation should be:

  1. Gained from upvotes on answers to questions in domain-content categories. This means you don't earn rep by asking questions, nor by anything in meta. You earn rep by providing good answers. How much others agreed or disagreed with things you said on meta shouldn't matter.
  2. Lost by some types of downvotes on answers in domain-content categories. This is the reverse of #1, largely. One problem on SE they never really addressed was malicious and vandalous downvotes. I have felt for a long time that the best way to prevent that, and get honest downvotes in general is to make them all public. However, there is strong resistance to that. A good compromise would be to allows two types of downvotes, anonymous and "signed". Anonymous downvotes affect answer score, and therefore sort order, but nothing more. Signed downvotes also affect rep, but require a comment explaining the reason, or a "me too" on someone else's downvote comment.
  3. Can be different per category. For example, votes on meta generally shouldn't affect rep at all, although that is settable as with other categories. On EE we have a Papers category. Writing a paper is more work, and it's held to a higher standard than questions and answers in the Q&A category. In that case, votes on the question (there are no answers in Papers) should mean more, both up and down. Overall, each category has mod-selectable numbers for how much upvotes and (signed) downvotes mean on questions and answers, for a total of 4 tweak factors. No special case then needs to be made for meta or any other category.

Reputation must also be simple. Just a single number is best. After all, we're really just trying to measure one thing, which is roughly "appreciated contribution level". Multiple numbers just confuse and/or dilute that.

Another tweak relative to SE would be having rep be based on "recent" contributions. Put another way, the rep from old contributions slowly decreases over time, eventually falling to 0. The point of this is to avoid making someone "eternal moderator for life". If someone hasn't been active on the site for a few years, they should no longer have a say in how it is run. This also gets around the problem of it being impossible for new active contributors to become part of the core group. If you provide a lot of good content over two years or so, you should be "in".

The exact parameters need to be tweakable, in no small part because it will take experience to find good values. A good start might be everything in the last year (or 2 years?) counts full, then decays to half its value every year after that. Or, maybe just a linear downgrade to 0 over 5 years after the initial year or two. The exact mechanism isn't important, at least not right now.

Why does this post require moderator attention?
You might want to add some details to your flag.

2 comments

There are definitely parts of this answer that I am not at all sure about on a network-wide scale (which is not to say that they might not be reasonable choices for individual communities to make, and therefore would be beneficial to support somehow as options), but the idea of having the bragging rights value (call it "rep" or whatever) of a user's past contributions decline over time is an interesting one, as if done right, it could incentivize continued quality activity. Canina‭ 4 months ago

@Canina: Everything I talked about would be configurable per site. Olin Lathrop‭ 4 months ago

Sign up to answer this question »

This community is part of the Codidact network. We have other communities too — take a look!

You can also join us in chat!

Want to advertise this community? Use our templates!