The technical design behind Abilities

So... as promised, here's a post on how Abilites are designed. If you haven't already, read the official announcement for them, before continuing to read this post to get a high-level understanding of what Abilities are and how they are supposed to work from a user perspective. You might also want to read the awesome blog post from Mithical about the community aspect of Abilities.

Ability Thresholds

The principle of Abilities is that you should earn one by showing that you have the specific related skill. So if you want to edit posts without review, show us that you can suggest good edits to posts.

Based on that thinking, we decided to have three different threshold types:

• Post Activity: For this threshold we look at how many mostly upvoted vs mostly downvoted posts you have. A post counts as "good" if it has a Wilson score of more than 0.5 and counts as "bad" if the score is below 0.5. This means that posts without any vote (or with an exact tie) don't matter for Abilities.
• Edit Activity: This threshold is based on approved and rejected edit suggestions. We might change this in the future and somehow include direct edits if we see problems with some Abilities being unreachable.
• Moderation Activity: This threshold is based on how well you are able to detect "bad posts". We look at the amount of approved ("good") and rejected ("bad") flags.

As you can see, all of those thresholds generate a pair of numbers, namely the number of good and of bad items. Based on those numbers, we calculate a score. For that, we use the same Wilson score formula that is also used for posts, with the exception that the constant cannot be changed but is fixed to 2.0.

Any ability can require a minimum value for one, two or all three of those.

While we looked at the calculations and played with the numbers to find reasonable default limits, Mithrandir24601 found an simple and great explanation for how we can explain the Wilson score:

In order to achieve a score s, you need a proportion of upvotes/accepted flags/edits equal to s, with an additional 'threshold' number of upvotes/accepted flags/edits, given by 4*(s-0.5)/(1-s).

We calculate those scores on demand, but they are heavily cached, because they are quite expensive.

The Database Structure

The Abilities update adds two new tables to the database: abilities and user_abilities,

The abilities table contains all earnable Abilities with their thresholds. We have one copy of them per community, so that communities can change the thresholds. The table has these columns:

• id: as primary key
• community_id which is the reference to the communities table
• name, icon, summary and description which are mostly used for display purposes and for the /abilities help center
• internal_id which is used internally to access a specific ability
• post_score_threshold for the post activity threshold
• edit_score_threshold for the activity threshold related to suggested edits
• flag_score_threshold for the moderation and flagging threshold

One thing to note about the thresholds is that we consider an Ability "only manually earnable" when all thresholds are set to NULL values. This is used for the moderator Ability, which you can't earn through reaching specific activity quotas, but by being appointed or elected as a moderator.

Then we have the user_abilities table storing who has earned which Ability. That one looks like this:

• id: as primary key
• community_id which is the reference to the communities table
• community_user_id which is the reference to the community user table
• ability_id is the reference to the abilities table
• is_suspended indicates whether the Ability is currently suspended (by a moderator)
• suspension_end can contain the end date of an Ability suspenion (or NULL if permanent)
• suspension_message is a message shown to the user given by the mod to explain the reasons for the suspension.

Awarding Abilities

Abilities are awarded by a script that will run regularly and that recalculates everyone's thresholds and checks, whether they are now eligible for a new Ability. Right now, the script just recalcs everyone's Abilities, but as this would bring down our server to its knees, this is going to be changed to only recalc the Abilities of users with relevant recent activity.

I can't say for sure what the future implementation will look like, but it's likely that we'll add the user to a schedule list, whenever something happens that might require an Ability recalc. Then we regularly check that list and recalculate Abilities for all users on it.

The actual recalculation process works like this for all possible Abilities:

1. We check whether the user already has the Ability. If they do, we won't do anything beyond this point and call it a day.
2. Then we discard Abilities that can only be granted manually.
3. If we are in new site mode, we'll skip the following checks for the Participate Everywhere Ability.
4. We'll then abort if any of the thresholds is not NULL and hasn't been reached.
5. If all prior checks succeeded, we'll now grant the Ability.

Besides that, we'll also always look if a user has old-style moderator rights and will also award them the moderator Ability to, make sure that both systems are in sync.

Every user account has the Participate Ability. Therefore we recalculate the abilities upon community user creation. Because the Ability used to be called Everyone and such a person without that Ability would be "Nobody", I thought it would be funny to name the method for that prevent_ulysses_case.

One note about suspending Abilities. This is a tool given to moderators to prevent destructive behavior by users who have lost the trust indicated by the Ability. Moderators can suspend an Ability permanently or for a specific time in days. They also have the option to delete an Ability, but this is less effective, because it will be re-granted upon the next recalculation. The option mostly exists, because moderators can also manually grant Abilities and to allow them to easily fix misclicks made.

Rate Limits

The Abilities change will also include the addition of rate limits. All rate limits work in a rolling 24h window. This not only has the advantage of being easier to calculate 😉, but is also more user-friendly, because the limits will now adapt to your usage and timezone.

We have the following limits:

• Top Level Posts (Questions and Articles) are limited to 20 per day (3 for new users)
• Second Level Posts (Answers) are limited to 30 per day (10 for new users)
• Votes are limited to 30 per day (5 for new users)
• Suggested Edits are limited to 20 per day (3 for new users)
• Flags are limited to 30 per day (10 for new users)
• Comments are limited to 50 per day (0 for new users)

For those limits, verified positive actions won't count (except for posts). This means that helpful flags and approved suggested edits won't matter for the rate limit. Also, votes and comments to own posts and to answers thereto are also ignored.

We believe that these rate limits will allow everyone to participate well in the community, while preventing being flooded (whether as spam or because of good intentions) by a few users. They also act as a safeguard for new users for us (against spam and trolling) and for them (to have fewer chances to make mistakes).

Convention over Configuration

While we tried to set reasonable default values for all Ability thresholds and rate limits, we can't -- of course -- say that they are going to work exactly as we want. We therefore expect to change those values over time as we gain more experience about the usage of our communities.

"One size doesn't fit all" is also our philosophy, hence all values will be able to be changed per-community. Moderators will be able to change the values for the Abilities (with care!) and they'll be able reach out to the Codidact staff to change the rate limits.

Thanks for reading and if you have questions or suggestions, feel free to comment here or to ping me in the Codidact chatrooms.

posted about 4 years ago

CC BY-SA 4.0

3y ago by sau226‭

luap42‭ staff

17 68 568 98