Welcome to Codidact Meta!
Codidact Meta is the meta-discussion site for the Codidact community network and the Codidact software. Whether you have bug reports or feature requests, support questions or rule discussions that touch the whole network – this is the site for you.
Hobbling of users who consistently post low-quality content
There are, unfortunately, a few users on Codidact who relatively consistently make low-quality contributions. These posts often come in bursts and tend to be downvoted fairly quickly, but that doesn't slow them down.
I propose that Codidact should implement some manner in which to slow down such users. They shouldn't be prevented entirely from posting, but there should be limits in place to ensure that their posts don't drown out other content.
I am posting a self-answer with a suggestion for how this can be done, but alternative suggestions (or arguments why this is a bad idea) are certainly welcome!
It sounds like we want to temporarily revoke a person's ability to make lots of posts when recent quality is an issue. …
3y ago
The problem has gotten even worse in the four months since this question was asked. I had forgotten about this question …
3y ago
This is my suggestion for how to implement hobbling of users who consistently post low-quality content. For a final d …
3y ago
This is a proposal for a refinement of Canina's proposal, or any proposal substantially like it, in response to one of c …
3y ago
Maybe I am the one who got downvotes (talking about the CD Meta) rather than anyone else. So do I have some idea what a …
3y ago
5 answers
You are accessing this answer with a direct link, so it's being shown above all other answers regardless of its score. You can return to the normal view.
Maybe I am the one who got downvotes (talking about the CD Meta) rather than anyone else. So do I have some idea what a downvote represents? Even sometimes I ignore the person who had downvoted (but not his texts).
In your answer:
First, look at their most recent post of the same type (question, answer, article, ...). If that post has a score >0.5 (that is, is positively received by the community), then allow the new post. (Note that this means that if their most recent post has not yet been voted on at all, the user does not get a free pass.)
Look, everyone has their opinion. And you can't understand a person by his/her past. I may write bad posts yesterday and today I may write good posts.
A person isn't always same. Earlier Stack Overflow was directly blocking people from asking without any warning. But nowadays they are showing a warning message above the question field.
I know what had gone up to me.
Someone said,"another person can never realize what had happened to a person until they are in their place".
There's several reason of downvoting.
- bad English (too hard to read) (this answer might be harder to read also :P (but I don't care of downvote :D))
- no research effort.
- don't have idea of basic things but doing advance things.
- Suggestion isn't good (talking about Meta posts)
Actually, when I was completely new to the community a user (I am not mentioning his name) had told me, "It's very hard to read your answer, and that's why I think other had downvoted". But that doesn't mean the post is bad. My English might be bad, but sometimes I may write good content also. bring me back
Actually, the suggestion may not good to you, but it may be good to some others. But if you judge a person by voting then it won't be good. As I said earlier, the last suggestion was bad, but you can't say future suggestion is bad also. bring me back
Some people may not have idea of everything. If you go to learn something new then there will be a lot of things that you don't know. But before asking a question, they should search on the Internet if there's something relevant. That represents a bad post. And I remark them as bad post either. We should restrict them. My suggestion or communities suggestion
Someone had mentioned in a post that, there's ability in your profile. New users don't get the ability. But since we are small right now, that's why we are giving that ability to every user. I had asked on Discord once, "when we remove the 'participate everywhere' ability. I had shown them that some was writing questions too poorly and their reputation is negative (>50). I got no reply, so I didn't bring that question again".
If anyone think that we should discuss about that then create a new question. I don't have time to do that now.
I care of the question when you say that a single user has written a lot of poor questions. But I don't want someone to be permanently banned. Because if you ban a user then they will go to create another account and he/she will keep doing it. So permanently banning isn't a good idea. My suggestion is to deal with a user's ability.
5 comment threads
It sounds like we want to temporarily revoke a person's ability to make lots of posts when recent quality is an issue.
The Participate Everywhere ability allows one to post without limit. Nominally, the requirement to earn it is:
To earn this ability, you need to have roughly 75% of your posts be positively received, with a minimum of 5 positively-recieved posts (these numbers may vary from site to site).
I said "nominally", because we've started communities here in "new" mode, so that people trying to build and expand our small communities aren't, um, hobbled.
Without "new" mode, people start at Participate:
This ability allows you to posts 3 [configurable] top-level posts (questions and articles) a day, and to post 20 answers a day.
This ability also allows you to raise 15 flags on posts a day.
Three posts a day is hobbling, not a block, so people can lift themselves out of it. That number is configurable per community.
Moderators can suspend or revoke individual abilities. I know this question is about automatic measures, but there is a manual option for individual cases that are disrupting a community.
The system automatically checks qualifications periodically (I think a few times a day) and grants new abilities when people have earned them. It does not currently revoke abilities (and in some cases that would be hard to even test).
Question 1: Should we build a "hobbling" system on the same tools? If the criteria (TBD) are met to tell a user to slow down, the script could suspend Participate Everywhere. Later runs of the script would check whether conditions have improved and restore the ability if so.
Question 2: If so, what should the criteria be to hobble a user? Abilities are based on all activity, but it seems like the problem that motivated this question arises from recent activity. If somebody had a bad start a year ago, fixed it, and then has one bad post now, that shouldn't hobble the person, I don't think. But if several more bad posts follow, that's different.
No matter what approach we take, I think it's important to give a user some warning at posting time -- something to the effect of "hey, be careful -- several of your recent posts haven't been well-received and you might get rate-limited".
If we take the approach I'm describing, then the only timing problem would be if the script happens to run while the person is posting. We can probably catch and handle that so the person doesn't lose work. (It'd stink to write up a long answer and only then have it rejected.) Otherwise, we know when you click "ask question" or start to type into the answer box whether you're allowed to post right now, and we can intercept you if not (like we do with existing rate limits).
By using the abilities system, we reduce the chances of creating weird conflicts with the abilities system by having two different, parallel ways of deciding what you're allowed to do. That seems more resilient.
A final question: Should we lift "new" mode from any network communities and recalculate abilities? I'll have to check with devs to find out what other effects there are; I think it's mainly that in "new" mode everyone starts with Participate Everywhere, but if there are other effects too, that would factor into the discussion.
2 comment threads
The problem has gotten even worse in the four months since this question was asked. I had forgotten about this question, and posted a new one "Need to throttle users with consistently bad posts". Please consider that an "answer" here.
In summary, I looked around on all the sites and found it surprisingly easy to spot the few problem users. You only need to look down the question list for negative scores, then look at who posted those question. The same users appear consistently, in any one site and even across all sites. A couple of users have gotten below -50 rep, but still keep doing the same thing. Take a look at the Outdoors, Math, and Power Users sites for the most egregious examples.
Suggested mechanism
The simple solution I propose is to force a wait time between questions based on your rep. One day (24 hours) for each negative count sounds about right, and is easy for everyone to understand. Someone with -7 rep wouldn't be able to post more than one question per week, for example.
It's time to do something
This question got overwhelming support and the problem has gotten worse since then, but the issue seems to have been forgotten. The duplicate I wrote has also been heavily upvoted despite being closed. I know the developers are volunteers, but at least this should go fairly high up on whatever list of priorities there is.
In the mean time, moderators need to take the initiative and suspend problem users, with the length of the suspension being related to rep or up/down vote totals.
This is my suggestion for how to implement hobbling of users who consistently post low-quality content.
For a final decision, when a user saves a post:
- First, look at their most recent post of the same type (question, answer, article, ...). If that post has a score >0.5 (that is, is positively received by the community), then allow the new post. (Note that this means that if their most recent post has not yet been voted on at all, the user does not get a free pass.)
- If their most recent post has a score of 0.5 or lower (negatively or neutrally received by the community, including not yet having been voted on at all), then look at the user's sum of received votes. If that value is 0 or higher, then allow the new post.
- If the user's sum of received votes is negative, then block creation of the new post unless, since the user's most recent post of the same type, the count of posts by other users is at least the absolute value of the user's sum of received votes. (So if a user's sum of received votes is, say, -3, then at least 3 posts must have been posted by other users.)
- As an escape hatch to allow users to redeem themselves in a reasonable amount of time, cap the number of posts that must have been posted by other users in the interim at 10.
For clarity's sake, because it seems I wasn't clear enough on this part, these are meant to be applied from top to bottom as listed above until an actual decision point is reached. If one criteria doesn't result in an "allow the post" decision, that fact doesn't mean that the final outcome will necessarily be a "reject the post" decision; it means nothing more and nothing less than that the particular criteria didn't result in an "allow" decision. (And similarly for a "reject" decision.) Only if a criteria results in an actual "accept the post" or "reject the post" decision will that be the outcome without considering later criteria. There is no falling through the end of the set of criteria (if there is, that's a bug in this suggestion, so please point it out in a comment); every situation will result in either an "allow the post" or a "reject the post" decision no later than by the time the criteria expressed in the last bullet point has been evaluated.
Looking at the user's most recent previous post as a shortcut means that if a user posts, say, a well-received question, then they can post another question fairly quickly even if they have a previous history of low-quality content, but that after doing so they may (depending on their updated sum of received votes) have to wait for one of their existing posts to be voted on before they can post a third question. This allows a user who has a history of previously having posted low-quality content to, if they put in the effort to write good posts, relatively quickly start working their way back up. Once the user's sum of received votes indicates that their contributions are overall positively received, they will be able to post repeatedly based instead on that record.
Restricting to posts of the same type ensures that for example self-answers aren't hobbled just because they are self-answers to a not-yet-voted-on question.
We might want to look only at the sum of received votes for posts of the same type, or the sum of received votes for posts over a limited time period, or the sum of received votes for the user's most recent some number of posts, but I'm not sure how reasonable any of that would be from a performance point of view. Also, the total sum of received votes is already readily available in the UI, so it's something the user can look at and, knowing the criteria, can predict the effect of. If considering all posts by a user turns out to have unintended side effects, it may be worth considering some variation along these lines.
For UX reasons it might well make sense to also evaluate these criteria early, but the ultimate decision needs to be made by the time the post is saved because we can't know that the client is well-behaved.
Ideally, the thresholds (vote score threshold 0.5, sum of received votes threshold 0, interim post count cap 10, possible number of past posts considered) should be configurable per community and possibly even per category (it makes sense to allow more controversial content in the Meta category, for example, since hashing out policies will by its very nature involve making suggestions that some people disagree with), but the values I mention above seem to me like a reasonable starting point that should work reasonably well for low-traffic and higher-traffic communities alike.
4 comment threads
This is a proposal for a refinement of Canina's proposal, or any proposal substantially like it, in response to one of celtschk's objections in the comments.
Canina assumed, and I agree, that the software should execute an algorithm when a post is submitted, using as input any of the data available at the time, with the output determining whether or not the post is allowed at that time. The software could also use the same algorithm to preemptively disable posting UI for users who would not be allowed to submit. celtschk's objection is that any changes to the inputs of the algorithm between the time that a user starts writing a post and the time that the post is submitted could cause a post to get rejected, and that the experience of seemingly being allowed to write a post but then having it be rejected due to causes outside one's control is an unnecessarily demoralizing one, to be avoided.
My proposal is to use an additional bit in the database for each user to store whether a user is currently being ‘hobbled’. The ‘hobbled bit’, or ‘hobbit’ if I may, is checked before a page render to determine whether to disable posting UI, and also checked when a submission is posted to determine whether to allow the post. When a post is allowed, the software additionally executes whatever underlying algorithm is used to determine if a user is allowed to post, on all the data available at the time. If the algorithm yields a negative result, the hobbit is set, and a notice is displayed to the user that subsequent posts won't be accepted until the community sees more activity, or whatever message is appropriate given the underlying algorithm.
A user's own post would be the only way to set their hobbit, so there is no concern about some action outside the user's control causing the user's work to be discarded. The only way for a user to submit a post that would be rejected, assuming normal use of the UI, would be to start two posts in parallel, submit one, ignore the notice that would be displayed after the submission is accepted, and then submit the other.
(Note that the hobbit is not a mutex and is not client-side or connected to session state or cookies; if a user starts writing a post and then closes the tab, their hobbit is not affected nor are any other post drafts in any other tabs. Only submitting a post can set the hobbit, and it is set server-side, checked by any subsequent GETs or POSTs.)
Hobbits should be reset once the underlying algorithm no longer forbids posts, based on time, votes, other user activity, or whatever else the algorithm takes as an input. Implementation of the reset depends on the specific algorithm chosen and on performance concerns; I would think it'd be acceptable to only reset hobbits hourly or even daily, if it came to that. Again, this functionality would only affect set hobbits; the only way to change a user's hobbit from unset to set should be for that user to submit an accepted post, while failing to meet the criteria specified in the algorithm.
Some final notes: throughout, I talk about one hobbit per user, but if the algorithm is written (as Canina's is currently) to consider different post types differently, there would be a hobbit for each post type per user, or a hobbit for each post type and category per user, or however many hobbits are logically needed. Also, if it would make sense to implement hobbits as abilities that are granted and revoked instead of a custom bit (or bitset) column, that might have advantages—I don't really know enough about the ability system to say.
4 comment threads