Welcome to Codidact Meta!
Codidact Meta is the meta-discussion site for the Codidact community network and the Codidact software. Whether you have bug reports or feature requests, support questions or rule discussions that touch the whole network – this is the site for you.
Comments on Support subjective scoring
Parent
Support subjective scoring
Codidact, like many other sites, relies on the wisdom of the majority. The approach has some well known flaws. One is "tyranny of the majority" which is really just a special case of "what if the majority votes wrong?"
Early on communities are small and tight knit. They're obscure and a certain type of person is preselected to join them. The small nature encourages cohesion and people naturally try to get along with each other. The votes work beautifully. Those of us with SO accounts (and reddit, and...) older than 2010 have all seen it.
As the community grows, all sorts of different people start arriving, and there is no longer homogeneity. The community's mass increases inertia, inertia makes individuals take interactions for granted. They stop trying to get along and instead want to disrupt, rebel against or control the community. Principled voting declines and voting blocks based on ideology begin to form. This greatly undervalues voting to users who try to leverage the wisdom of the crowd to find good and bad answers.
What if:
- The site analyzes past voting patterns to find which users have voted the same way on the same types of questions
- A vote similarity score is calculated for each pair of users (this can be done more efficiently than
O(N^N)) - The "subjective score" is the same vote tally, but weighted by similarity of each vote's users
This would give users a low impact, peaceful way to handle controversy in the community. People are mad about homework questions, but you love helping with homework? No problem, you'll see homework questions because you vote them up, they won't see them because they don't.
The technical challenges to implementing this aside, what if such a system was in place? Would it help improve the overall user experience?
Post
This is an interesting proposal. Although I think it would be detrimental in its current form, I'd like to emphasise that I would like to see more discussion of potential modifications to the way the site is curated and presented. I don't think this should ever be regarded as "finished" - improvements should always be sought.
As such I've added an upvote to the question to indicate that this is an important discussion to have.
My concerns
Seeing less
If you see mostly posts that others with similar voting patterns like, then this may lead to an echo chamber effect where some posts that would benefit you are hidden from you.
Curating less
Votes on Codidact are not just about expressing what you like and want to see, they are indicating which posts are beneficial to the community.
Since the community is responsible for measuring the quality of a post, showing people only what they are expected to upvote may have a cost in quality. If all of the people who might downvote a post are never shown it, this means that the people best placed to recognise a problem are prevented from seeing it.
This is not just a voting bias. The people who would comment to help explain how a post can be improved will also be kept away from it. I suspect this may also lead to a decrease in suggested edits. Every post is only presented to those who like it, so everything gets upvotes and the critical analysis of a community of experts is removed.
Testing
My concerns have not been measured - they are just my own opinions. Personally I would prefer not to experiment with this on codidact.com unless there is a strong reason to believe there will be a benefit, but that is not my decision.
If a Codidact community expresses interest in trying out this approach or one like it, it would just need someone to make the required modifications to the software.
The QPixel software which runs Codidact is open source and available for anyone to modify and set up their own instance to test this if they wish, even if there isn't an existing Codidact community that wants to run a pilot.
2 comment threads
2 comment threads