Let's improve how we handle duplicates

As a momentary foreword, I would like to express my appreciation to you and all of the members of the community which have worked to make this site possible. I wish it the best in its growth and development, and hope to help as a part of it. Now, here's my proposal on this topic:

Duplicate proposals should be separated from closure-as-duplicate voting.

I think you've made a very good point about question closure (and the accompanying feeling of judgement—or, as I would put it, dismissal, perhaps even exclusion), and I think this insight should be applied to duplicates a step further than you suggest.

Again, as you mention, a question being marked as a duplicate (of an answered question, at least) should be a good experience for the question author if all goes well, who should then gladly close the question as a duplicate. However, I think there's a fundamental problem with a single proposal-closure duplicate system: the tendency of users viewing a question marked as a duplicate is to judge whether the question as asked is a duplicate for themselves. There's the added issue of the voting process starting immediately, not necessarily giving the author time to review the question before it's closed—and once that happens, it can easily become a fight against the perception that a question is closed for good reason.

In either case, there is an important step which is being missed in a system which treats the question, not the author: the primary purpose of this site is to answer the questions of its users, whether directly posed or through the provision of existing answered questions, and in the case of a disputed duplicate proposal, the author feels that the linked question does not answer the question the author is trying to ask. This mismatch is key to resolving good-faith duplicate disputes, and should be the primary focus of a user viewing a question with a duplicate suggestion that has not yet been closed.

This might seem like a benign issue, in which I'm putting too little faith on the users judging the question duplicity, but there's a deeper principle here: we should be maximising the ease of use for individuals acting in good faith (i.e. trying their best to follow its rules and principles), but an author who feels that a question has been wrongfully closed as a duplicate is quickly put in an uphill battle, being required to convince and get the attention of enough people to reach a position where the question that the author is trying to ask can be asked instead. This can be exhausting, and worse, can drive users away from trying to ask their questions, and I think there should be some safeguard against this kind of situation.

My proposal for the procedure of duplicate resolution, in full, then, is this:

1. A user marks the question as a possible duplicate, linking to the question(s) it is believed to be a duplicate of.^[1] An explanation of why the question is thought to be a duplicate should be provided such that the author has immediately actionable information.
2. The author has 72 hours to respond to the proposal, either by accepting the proposal (closing the question as a duplicate), or by disputing the duplicate.^[2] Any activity on the question by the author reduces the window to 1 hour, since its purpose is to give the author time to edit the question or otherwise address the proposal.^[3]
3. While a question is proposed as a duplicate, it is marked as such to direct user attention to comments where the proposal is hopefully being discussed between disputer, author, and other involved users. This period is designed to help the author and community resolve the misalignemnt between (author) intent and (community) perception of the question.
4. Once a duplicate proposal is disputed, or the window expires^[4], voting opens for closure on the basis of being a duplicate question. The question is closed as a duplicate if the net vote passes a certain positive threshold determined by the proportion of votes (e.g. +25%; i.e. at least 5:4 pro:versus^[5]), with some minimum (e.g. +3 votes net) for small questions. This is important on its own (and something which I don't think I've seen discussed) because the judgement of a question should scale correctly with the popularity of the question: proportional judgement is how we do things in a democratic society, except the threshold is usually 50:50 (or 51:49)! This should also mean that cycles in closure and reopening will be less likely.
5. As in your suggested procedure, a user (including the author) can vote to reopen a question, starting a new vote, and this vote may have a different threshold—perhaps the inverse of the threshold to close. A user with the Edit ability may reopen a question automatically with an edit of the question (presumably with the intent to differentiate it from the linked (duplicate) question). Each time a question is reopened, the window for the author to respond and the threshold to close the question (again) decrease slightly.^[6] This prevents questions from being endlessly reopened, and means that an author should still be careful about the edits made before reopening the question, even with the ability to do so without approval.

I believe this proposal satisifes (almost^[7]) all of the goals proposed in the question:

• The author is notified of a duplicate proposal, which can be made as soon as the question is posted. While a duplicate proposal is awaiting resolution, a notice informs users viewing the question of the possible duplicate, meaning that they can check the linked question(s) and each make an assessment before spending any (potentially redundant) time and effort answering the question: the question should only receive answers from users who believe it not to be a duplicate.
• Authors are provided with a question to check for duplicity (and answers to their original questions), and so can immediately make the assessment of whether to approve or dispute the duplicate proposal.
• Feedback of either kind is collected accordingly via suggestions in comments or even other duplicate proposals.
• Presenting duplicate proposals as something which authors are at their leisure to make the choice to accept, I believe, would do a massive deal of good for the way that this would feel. An author can also feel safer knowing that the question has a period of time in which it can be judged by a reasonable sample of people before it is threatened with closure by a small sample size.

Addendum - a personal note

I understand and appreciate that this proposal may be seen as too extreme. I think that it's appropriate for me to mention that part of the reason I wanted to write this answer is because I had a pretty unpleasant time recently at Somewhere Else regarding this topic (and closure in general), and I felt, quite frankly, bullied and powerless in the way that the system was set up. I would like to ask only that you consider what cost this proposal (in its whole or as its separable components, if any) would have to the good operation of this site.

Thank you again for your role in this site, Monica. I hope that my first post on this site is received well. :)

posted over 3 years ago

CC BY-SA 4.0

3y ago

Fie‭

0 1 6 1

Question closure can leave people feeling judged (as we learned Somewhere Else)

...counter the impression that duplicates are bad

To solve that problem, one needs to address the source. Somewhere Else would instantly make a conclusion like "aha it's the evil community being rude again" and then come up with some misguided system to counter that. But by applying a slight bit of empathy, we can get to the root source:

The people who cast duplicate close votes Somewhere Else are fed up with endless duplicates. Newbies asking the same question over and over again, with little to no research effort made. Therefore the regulars get tired of that behavior and just close the posts without providing much feedback to the person who asked the question.

There exists a somewhat rare phenomenon though: sometimes when a high quality question that is a dupe gets asked, it is left open long enough for good answers to pop up. And when this happens, this new question might actually turn into the "canonical" target for duplicates. And then the old, present duplicates get closed with the new post as target. It's a very good thing when this happens. The old duplicates of diverse quality are not necessarily the best ones.

But most of the time, new questions that are duplicates just get closed with an old post as target, because that's how the system was designed.

Somewhere Else is suffering from the results of this: there's a lot of old posts with canonical status but so-so quality. Also such posts tend to attract a whole lot of answers over time, where everyone and their mother feels inclined to contribute even though they aren't adding anything new. Or in case they are adding something new, they only add that part and not a complete answer. So over time the canonical post "fragments" into several answers and the result as whole is not very good.

It would be better if these old posts were recompiled into complete answers and one natural way to do that is to close them when something better and more complete shows up. But the duplicate system often doesn't let that happen.

We shouldn't close posts as duplicates unless there exists a high quality duplicate target. If a question has been asked before, then that alone should not be a reason to close it.

posted over 3 years ago

CC BY-SA 4.0

Lundin‭

30 57 412 51

Update based on feedback

I really like the ideas proposed in this answer, which at the time of this posting has the same vote count as this question (+6, -0). The proposal in that answer creates what I think is a better experience for the asker and the community, and is less complicated from a behavior and UI perspective. Win! Some additional points came up in a discussion, and this other answer also raises important points, so I'm going to try to bring this all together here.

New proposed workflow:

1. A user with the Participate Everywhere ability marks a question as either similar to or a duplicate of another linked question.

• You can only propose a duplicate if the other question has at least one well-received answer (thanks Lundin for raising that). "Duplicate" is about guiding people to answers; if a bunch of similar questions exist but none are answered, then perhaps what we have been waiting for is the right framing, and we shouldn't shut the new question down. (Maybe the others will end up as duplicates of this one!)

• Similar is just informational; we find a place to show the links with the question. The question remains open to receive answers. The rest of this workflow is specifically about duplicates.

2. The author has 72 hours to respond, as described in the first answer I linked.

• If the author has the Edit ability, we (once) treat the edit as resolving the duplicate suggestion. Those links are moved into the "similar to" list, and somebody might re-nominate a duplicate.

• If the author does not have the Edit ability, or if the author has previously resolved a duplicate suggestion here with an edit, the "suggested duplicate" notice on the question is updated to add that the author edited after the suggestion was made. Anybody else with the Edit ability can confirm that the edit resolves the issue and dismiss the duplicate notice (moving it to the "similar" list as above).

3. While there's a duplicate suggestion pending, we show a notice and alert people starting to answer.

4. When the author disputes the duplicate suggestion or the window expires, the community can vote as described in the feature proposal. Instead of a net score, we're looking for a ratio, as suggested by Fie, with a minimum number -- something like "at least 3 dupe votes, and at least 60% of all votes are yes". As Fie pointed out, this allows voting to scale with question popularity, which I think will help prevent the ping-pong effects of groups of three users alternately voting yes and no.

5. Removing a duplicate linkage ("reopening") is possible; see step 5 in Fie's answer.

posted about 3 years ago

CC BY-SA 4.0

Monica Cellio‭ staff

64 377 2269 838

Psychology

In sum: I think the social problems here have more to do with communication than they do with policy, and I'm extremely wary of softening policy in order to avoid hurting anyone's feelings.

To be fair, sometimes people should feel judged for posting a duplicate question - they clearly haven't made any attempt to look for the already-existing information within the site. However, it can also easily happen that someone thinks of the same problem in different terms, and thus there is a new title or description of the same problem that is also useful to have.

In most such cases, there is either no clear advantage or disadvantage to the new phrasing; the OP should simply be pointed at the old question with a note that has a positive tone: "good news, your question appears to have an answer here already:" etc. etc.

Marking duplicates, and other closures

Somewhere Else, marking something as a duplicate is treated by the system as a form of closure. Insofar as "closed" means "the state in which a question is publicly visible but may not receive new answers", I see no reason to diverge from that model. Recognizing a question as a duplicate absolutely should function as an injunction on answers. It is crucial to the value of a Q&A site that answers for a question are centralized:

• so that someone who searches for those answers can get them all at once;

• so that when those ideas are considered in the "marketplace", there is no friction in that system;

• so that answers are given equal exposure, phrased in terms of the same motivating example (if there's actually a need to refer to a specific motivating example), and otherwise compared on an apples-to-apples basis.

If people react poorly to a prohibition on "their" question receiving new answers, but "their" question is in fact the same as one that does qualify for new answers, then there is a need to take ego out of the equation in some form. This is entirely a social problem, not a technical problem.

If people react poorly to "their" question being "closed", regardless of the implications of "closure", then evidently the problem is that terminology, not the implications. We should rephrase this, and I think that would be a good idea anyway. The other experimental phrasing for this idea is "On Hold"; I don't think this is really any better.

I don't have a clear-cut proposal here, but I do want to note that this phrasing interacts with policy for closing questions. If we ever reached a level of popularity, for example, where it makes sense to start out questions in a closed state just to maintain basic QC, then it would in turn make sense to use terminology like "under review", "being workshopped" etc.

Disputing closures

The procedure laid out for us by our forbears is pretty straightforward: just as the question was nominated and confirmed for closure by a certain number of votes, it can be nominated and confirmed for reopening in the same way. While the closed question cannot receive new answers, it can be edited, including unilaterally by the OP.

My contention here is that this system mostly works fine. To the extent that it fails, this is a technical problem with discoverability of the reopening process.

Perhaps OP should be able to add a privileged note explaining why the identification as a duplicate is contested - sometimes someone in OP's position won't have an easy time editing the question, but this information can make it possible for someone else to apply the edit. In other cases, it can help commenters to explain why the target question really is a duplicate.

There definitely should be a way that curation-minded users - whether or not they have an actual Curate ability yet - can easily search for recently closed questions, and specifically for recently duplicate-marked questions, in order to review those decisions.

The system definitely needs to communicate clearly to users, in the aftermath of a closure, about what they can and should do. Ideally this process is interactive, like the initial guided tour, asking the user questions like "did you understand the linked question and did it solve your problem?" (At this point we could even softly suggest adding a "worked for me" reaction...) "Is your actual question fundamentally different? If so, how?" etc. Designing the interface for something like this requires considerable thought, and should definitely not be done "by committee" through Meta.

The "help desk" question

Communities need to decide for themselves whether people who come in with a problem to solve and get directed to existing Q&A, are responsible for understanding how that Q&A applies to their personal situation.

If a community sees itself as offering a "help desk" to new users, then it is useful to be able to write an "answer" that references an existing Q&A and briefly explains how the existing Q&A applies to the current circumstance, perhaps showing the result of applying advice from the answers to the specific circumstances. There shouldn't ever be a need for more than one such answer, and technical measures should be used to prevent competition in that regard. Once the question is identified as, let's say, an example of a common problem, it could have an automatically generated, templated, community-wiki answer that cites the would-be "duplicate" and which can then be edited to add explanation (of the sort that, Somewhere Else, would be given in the comments if at all).

On the other hand, if such questions aren't accepted (i.e. not even in a separate category, as I've been suggesting in some other Meta Q&A), then there is no particular need for such "mercy" - just close the question as a duplicate.

posted over 1 year ago

CC BY-SA 4.0

Karl Knechtel‭

23 36 149 28