Communities

Writing
Writing
Codidact Meta
Codidact Meta
The Great Outdoors
The Great Outdoors
Photography & Video
Photography & Video
Scientific Speculation
Scientific Speculation
Cooking
Cooking
Electrical Engineering
Electrical Engineering
Judaism
Judaism
Languages & Linguistics
Languages & Linguistics
Software Development
Software Development
Mathematics
Mathematics
Christianity
Christianity
Code Golf
Code Golf
Music
Music
Physics
Physics
Linux Systems
Linux Systems
Power Users
Power Users
Tabletop RPGs
Tabletop RPGs
Community Proposals
Community Proposals
tag:snake search within a tag
answers:0 unanswered questions
user:xxxx search by author id
score:0.5 posts with 0.5+ score
"snake oil" exact phrase
votes:4 posts with 4+ votes
created:<1w created < 1 week ago
post_type:xxxx type of post
Search help
Notifications
Mark all as read See all your notifications »
Q&A

Welcome to Codidact Meta!

Codidact Meta is the meta-discussion site for the Codidact community network and the Codidact software. Whether you have bug reports or feature requests, support questions or rule discussions that touch the whole network – this is the site for you.

Comments on We need to talk about Abilities

Parent

We need to talk about Abilities

+8
−1

The problem

Almost three years ago we introduced Abilities to Codidact as an improved system for granting activity-based permissions, such as allowing users who have made consistently great edit suggestions to edit posts without requiring approval. I'll defer to the linked posts on how Abilities function in detail.

I do still believe that coupling permissions with specific experience in the area of the permission is a good idea and should be kept as the basic premise of the permission system.

However, there are a three major issues I see with how our abilities system works currently:

  1. The Ability thresholds are calculated using the Wilson score formula we use for posts. This means, that the minimum number of (positive) actions needed and the minimum percentage of positive actions are directly coupled.

    In some cases this coupling can be inappropriate for the intended purposes. For example, a community might want a user to have suggested many edits to demonstrate experience but want to require only an average approved percentage of like 75%. In our current system this would not be possible.

    In some cases communities might want the opposite. They would want a high percentage of 95% edits approved, but because they have low activity they only want a small sample of 5 edits (which would of course mean that once one edit is rejected your amount of edits that need to be approved would rise drastically). This would also not be possible with the current system.

  2. Currently abilities are directly coupled with permissions. This means, that the system will check whether or not you have a certain ability and will infer your permission from this. This becomes a problem once a permission should be inferred from multiple abilities, because then the every time a permission is checked the code would have to keep in mind how exactly abilities bind to permissions, and if multiple permissions are coupled into one ability, because they are then affixed together as an all-or-nothing thing. However one might want to separate permissions from another for moderation purposes. A user might need to be banned from reviewing edits while their power to edit posts should not be restricted.

  3. The current system is based on specific existing abilities hard-coded in the application. However communities might want to change the ability progression to better match their individual needs.

A solution?

Therefore I am proposing some tweaks to how abilities are implemented:

  1. There are two factors: "privileges" and "abilities" (or "trust levels").

  2. Privileges are technical allowances, such as "may edit other people's posts without review", "may review suggested edits" or "may vote to close". A user can have or not have a privilege at any time.

  3. Abilities are steps on the progression ladder. Technically an ability consists of one or multiple privileges and a "win condition". If you earn an ability, you will be granted the privileges it contains.

  4. Privileges can be individually suspended by moderators (as abilities can be suspended for time/infinity right now). Abilities cannot be suspended because the only thing that goes beyond suspendable privileges are "bragging rights".

  5. Abilities are reached when its win condition is reached. A win condition may consist of multiple requirements, each of which must be completed. Requirements could be, for example,

    • minimum user age
    • obtaining a specific ability
    • minimum post count
    • minimum positively received post count
    • minimum percentage of posts positively received
    • minimum (suggested) edits
    • minimum approved/applied edits
    • minimum percentage of suggested edits approved
    • minimum flag count
    • minimum helpful flag count
    • minimum percentage of helpful flags
    • minimum close/delete votes
    • minimum effective close/delete votes
    • manual appointment (i.e. this ability can only be awarded manually)

As before, moderators would have all such privileges.

Something similar to the current ability progression should be set as standard, so that we have an ability trees not unlike this:

  • everyone (general path)
    • established user (general path)
      • editing posts (edit path)
        • editing tags (edit path)
      • closing posts (mod path)
        • deleting posts (mod path)
    • moderators

However communities would be free to change this progression as they like.

I think that these tweaks would allow better bootstrapping of communities and would allow it to better react to community size with configuring the ability system.

Discussion

What do you think? Do you have any other ideas on how to improve abilities?

History
Why does this post require attention from curators or moderators?
You might want to add some details to your flag.
Why should this post be closed?

0 comment threads

Post
+3
−0

I've been thinking about abilities again lately (and for a while), because they're too hard to earn but just tweaking the Wilson numbers won't fix it. The fundamental problem is that a single negative event sticks to you forever and requires many more positive events to compensate. I sometimes look at active users, wondering "why doesn't this person have such-and-such ability yet?", and the answer is one declined flag two years ago or something like that, despite dozens of accepted ones since.

What problems would arise if abilities, instead of using Wilson scores, were formulated as:

  • At least N positive events within timeframe T1
  • And no more than M negative events within timeframe T2

The events would be the same as what we use now: posts, suggested edits, flags. Maybe those should be changed too, but I'd like to see if we can just adjust how we count the things we already have for the abilities we already have and if that would make things run more smoothly for our communities.

I originally wrote this as a single timeframe, but I think for some things we'll want to consider the positive record for longer than the negative record. Maybe the timeframes will end up being the same in some cases. If we're going to do this at all, having two values rather than one doesn't seem too complicated.

I'm thinking this as the logic for granting an ability, not logic for doing ongoing assessment to grant or revoke. Currently, abilities are not auto-revoked if you do something that would have caused you to no longer qualify. Moderators can suspend or revoke abilities if needed.

History
Why does this post require attention from curators or moderators?
You might want to add some details to your flag.

2 comment threads

Considering both timeframe and last N events (7 comments)
What happens when a user becomes less active? (3 comments)
What happens when a user becomes less active?
Olin Lathrop‭ wrote 2 months ago

You need to think thru what happens as the actions that granted an ability decay over time. Old actions decaying by becoming less relevant to current calculations makes sense, but what should happen when a user becomes less active? At the least, enabling abilities needs to have some hysteresis. Anyone who is granted an ability will be right on the threshold. You don't want them to loose the ability due to a single negative event.

I'm not sure what the answer should be, but this needs to be thought out carefully.

Michael‭ wrote 2 months ago

Agreed with Olin.

Maybe one could combine timeframe with last N events. Something like "Last M years' positive events, filling backward to X positive events if you don't have X events in the last M years" and then supplement with "Any negative events from last N years' total events, backfilling to negatives within Y total events, if Y events hadn't occurred in N years."

Sorry it's a complicated description. And I'd have to see it mapped against (real?) user patterns to see whether it makes sense or solves the downsides described in current algorithm.

Monica Cellio‭ wrote 2 months ago

I wasn't thinking about losing abilities over time. That's not how the current system works either: once you earn an ability, you have it forever unless a moderator suspends or revokes it. We certainly don't want to create yo-yo situations where someone just earns an ability, goes on vacation, and comes back to find it gone because one flag aged out.

We could add logic for revoking abilities after long periods of inactivity, but if someone leaves for a year, comes back, and starts misusing an ability because norms changed or the person changed or whatever, moderators can already handle that. I'd rather a human be involved when removing abilities.