Difference between revisions of "Team Fortress Wiki talk:Discussion/Wiki Cap"

From Team Fortress Wiki
Jump to: navigation, search
(Three-Phase System Proposal: Updated the proposal, and merged Noble's comments into it (with permission).)
m (Removed duplicate of Noble's section (it's at the top of my proposal).)
Line 309: Line 309:
 
::Indeed, it is overly complex and just seems too difficult to understand. Perhaps integrating that idea of Pros/Cons and one reason per nomination per person in to the existing system might mix things up enough? [[File:User Wingless Winged Signature.png|150px|link=User:Wingless]] 17:54, 31 August 2012 (PDT)
 
::Indeed, it is overly complex and just seems too difficult to understand. Perhaps integrating that idea of Pros/Cons and one reason per nomination per person in to the existing system might mix things up enough? [[File:User Wingless Winged Signature.png|150px|link=User:Wingless]] 17:54, 31 August 2012 (PDT)
 
::The jist of the system seems to crowdsource (someone's gonna hate me for using that word) the "mechanical" verification tasks to a more general public, as well formalizing each step with precise criteria to meet. The crowdsourcing aspect is probably a good thing, as it partially addresses the staff willingness/laziness issue (part of the lack of motivation is, I guess, the amount of work that goes into it. When a wiki cap discussion begins, it's often for a good couple of hours, and no one like to dive into that on a nice sunday). It also has some openness benefits, with the bigger community participating closer. The formalization aspect however is, as it stands, way too complicated for it to be practical. It adds a lot of bureaucractic overhead, of extra enforcement work (when people don't follow the rules), and it would probably put off a large number of people (who would otherwise have contributed) merely by being complex like this. This being said, it's also impossible to crowdsource work without having it framed in a formal system, so some level of formalization has to happen (this is why you have upvotes/downvotes on Reddit or YouTube or Steam Greenlight etc.), but I think this proposal is overdoing it. I do appreciate the level of detail, thought, and research that has gone into this though. Perhaps one day under some distribution systems, that may just warrant a wiki cap :3 — [[User:WindPower|<span style="font-weight:bold;">Wind</span>]] 14:27, 1 September 2012 (PDT)
 
::The jist of the system seems to crowdsource (someone's gonna hate me for using that word) the "mechanical" verification tasks to a more general public, as well formalizing each step with precise criteria to meet. The crowdsourcing aspect is probably a good thing, as it partially addresses the staff willingness/laziness issue (part of the lack of motivation is, I guess, the amount of work that goes into it. When a wiki cap discussion begins, it's often for a good couple of hours, and no one like to dive into that on a nice sunday). It also has some openness benefits, with the bigger community participating closer. The formalization aspect however is, as it stands, way too complicated for it to be practical. It adds a lot of bureaucractic overhead, of extra enforcement work (when people don't follow the rules), and it would probably put off a large number of people (who would otherwise have contributed) merely by being complex like this. This being said, it's also impossible to crowdsource work without having it framed in a formal system, so some level of formalization has to happen (this is why you have upvotes/downvotes on Reddit or YouTube or Steam Greenlight etc.), but I think this proposal is overdoing it. I do appreciate the level of detail, thought, and research that has gone into this though. Perhaps one day under some distribution systems, that may just warrant a wiki cap :3 — [[User:WindPower|<span style="font-weight:bold;">Wind</span>]] 14:27, 1 September 2012 (PDT)
 
==== Short Said: ====
 
 
To make this proposal abit more easier to understand, then [[User:Mainman|Mainman]] has a short explanation for you all:
 
 
Wave 1: Someone awards the 'Nomination' wikichievement to another user. People leave feedback on the user's Talk page. The user passes Wave 1 if enough people agree with the nomination.
 
 
Wave 2: A team of moderators and 'trusted' users pick a nominee, gather evidence, do a background check and make a simple review. Fellow reviewers check the facts and leave their own review. Wave 2 is passed if the majority of the reviews agree with the nomination.
 
 
Wave 3: Staff members look over the evidence and post their reviews to a private forum. Wave 3 is passed if most of the participating staff say "Yes".
 
 
''Written by Mainman himself''
 
[[File:User Ravecrib9t4 Signature Logo.JPG |35px]] [[User:Ravecrib9t4|<span style="font-family:TF2 Build; color:#141414 ">TNS</span>]] 15:24, 1 September 2012 (PDT)
 

Revision as of 22:56, 1 September 2012

Round-the-Clock Moderation

Whilst more of a combination of two of the existing ones, has there been a consideration of removing moderators who are inactive (through a discussions of course), then re-hiring around 5-6 moderators who are in different time zones? For instance, 5 mods from UK/West EU, 5 from USA, and 5 from Eest EU/Asia/Oceania. This kind of a set-up should give a strong moderator presence around the clock.

Most other solutions seem like they might not work in a democratic and fair system: non-staff members could easily bias votes if they think un/friendly things about the candidate, static discussions would mean people would put their views and generally not bother to look at it again, and less staff members mean less discussion.

Just some of my views ^^ User Wingless Winged Signature.png 15:14, 29 August 2012 (PDT)

New nomination system idea

Maybe it looks cold on the first look but i think this shortens it, but also makes it more clear what a person does on the wiki Pick 3-5 main points and after that what the person which is nominated also does. Here my suggestions:

Pictogram plus.png  Makes redirects
Pictogram plus.png  Creates new articles in "language"
Pictogram plus.png  Makes grammar corrections
Pictogram plus.png  Deletes bad edits and/or trivia and/or bugs
Pictogram plus.png  Uploads new pictures
Pictogram plus.png  Creates new pictures (paintable hats f.e.)
Pictogram plus.png  Helps new user to improve with informations about a wiki problem
Pictogram plus.png  Creates needed templates
Pictogram plus.png  Updates articles which are outdated in "language"
Pictogram plus.png  Works on the STS for TF2 translations (discussable)

δ³Σx² 08:01, 30 August 2012 (PDT)

That summarizes most of what can be done on the wiki, I'm against relating the wiki cap to the STS though, it's two completely different things. Working on STS shouldn't in any way be related to obtaining a Wiki Cap. The Wiki Cap is a reward for contributing and improving the TF Wiki, not the STS. Tturbo Killicon ambassadorhs unused.png (Speech voice.png / Intel neutral pickedup.png) 08:16, 30 August 2012 (PDT)
I was under the idea that most/all of this was already considered. That and being somewhat active in general. But I agree, the STS stuff doesn't really seem relevant to the wiki. ~Sven~ 08:18, 30 August 2012 (PDT)
So add some more points, which are missing? If people add a person for nomination they often dont insert certain informations, and only some "could be argument". So i think with these we could shorten the nominations but also show what makes this user good(special). δ³Σx² 08:23, 30 August 2012 (PDT)
To be very honest, nomination systems should work by giving examples and not simply saying "oh he does this, and he does that~". There should be very clear and very strong examples as to what the user has contributed to the Wiki, be they overhauls, new pages and a lot of content, or pictures they've uploaded (etc). Small things like redirects and tags (i.e. improveimage, delete, move, etc) should not count towards the main nomination, but definitely can be considered in the final decision-making process as a way of seeing what users do - if it's constant redirects and silly small edits to boost edit count then no, it shouldn't be considered. But if it's all a part of several things then it can be considered if the people behind the final decision wish it to be. Also, STS workers should be a catalyst behind decisions and not a reason for a nomination. User Wingless Winged Signature.png 08:33, 30 August 2012 (PDT)

Three-Phase System Proposal

It's quite sad to see the current system isn't working anymore, especially after I helped to revamp its presentation about a year ago. Helping to fix the mechanics of the process seems like next logical step.

This proposal includes ideas already presented on the staff and user Talk pages; credit is given where credit is due.


In a Nutshell...

  • Wave 1: Someone awards a 'Nomination' wikichievement to another user. People leave feedback on the user's Talk page. The nominee passes Wave 1 if enough people agree with the nomination.
  • Wave 2: A team of moderators and 'trusted' users pick a nominee, gather evidence, do a background check and make a simple review. Other trusted "reviewers" check the facts and leave their own review. Wave 2 is passed if the majority of the reviews agree with the nomination.
  • Wave 3: Staff members look over the evidence and post their own reviews to a private forum. Wave 3 is passed and a Wiki Cap is awarded if most of the participating staff members say "Yes".


Phase 1: Nominations (All Wiki Users)

Firstly, the existing Nominations page would become protected from normal user edits, and mostly update on its own via templates as described below.

The new Nomination process would start on users' Talk pages. After deciding to nominate someone for a Wiki Cap, a user would create a dedicated section on the nominee's Talk page to award them a new 'Nomination' Wikichievement.

  • The "Reasons for Nomination" in the old process would find their home here. Users who 'agree' or 'disagree' with the nomination will cite whatever reasons they think are relevant. Linking to evidence will not be necessary at this point in the process.


MOCKUP:

Wikichievement Nominated By Reasons
Tf scared stiff.png

Cap Nominee
Somebody out there thinks you've got what it takes to wear a Wiki Cap! Do the other wiki users agree? Let's find out...

User:Person_Z I like zis weapo... person. Iz good.
Username Agree/Disagree Reasons
User:Person_B Pictogram plus.png Yes! We have nothing to fear from trusting this man. Our wiki, and our country, are in good hands.
User:Person_C Pictogram minus.png Nah Ah always edit drunk, and I c'n still spell beh'er than 'im!
User:Person_D Pictogram plus.png Yeh Left his dispenser on the barracks for me last week. He's a mate, fer deffo.


The existing Nominations page would have a simple table called Phase 1: Nominations, automatically listing any users with the Wikichievement on their talk page, along with an Agree/Disagree 'points' counter. The counter would be a template that updates its 'Points' scores based on the following:

  • Points are invalid if they don't begin with Pictogram plus.png + / Pictogram minus.png - and end with a user signature.
  • Only one point can be given per user.
    • Pictogram question.png Discussion Point: Excluding the proposed 'Nomination' Wikichievement, should there be an additional restriction where users must have at least one Wikichievement to their name for their "Point" to count?

After reaching a minimum number of 'points', while also maintaining a good Agree/Disagree points ratio, the user would automatically move to a table called Phase 1: Candidates.

  • Pictogram question.png Discussion Point: What would be a suitable minimum number of points?
  • Pictogram question.png Discussion Point: What would be a suitable ratio of Agree/Disagree points?

If the user has a bad Agree/Disagree ratio, then they will automatically be moved to a different table called Phase 1: Non-Candidates until their ratio improves.


MOCKUPS:
Phase 1: Candidates

Nominee Total Points Agree/Disagree Count Ratio
User:Person_A 4 Pictogram plus.png 3 / Pictogram minus.png 1 75%

Phase 1: Non-Candidates

Nominee Total Points Agree/Disagree Count Ratio
User:Person_E 8 Pictogram plus.png 1 / Pictogram minus.png 7 12.5%


Phase 2: Filtering (Trusted Users)

A staff-selected team of users, along with any moderators and staff members who want to help, would hold responsibility for the next step: Manually filtering through the Phase 1 Candidates for those who deserve to reach Phase 3.

  • Pictogram question.png Discussion Point: How many users should the staff grant the 'trusted' responsibility to?

Firstly, a Trusted User marks a Phase 1 Candidate as 'selected', which removes the nominee's info from that table and adds them to an invisible one called Selected Users (all users that have Wiki Caps will be in this list too, to prevent re-nomination). The information is manually transferred across to a table called Phase 2: Filtered Nominations. The table would require the following information:

  • A check-list of Yes/No answers to various questions about the user's contributions (as originally suggested by TheDoctor):
    • Q1 Has the user significantly improved an existing article, or made extensive valuable changes to a page that was already good?
    • Q2 Has the user contributed good 'new' pages? (Excluding patch-day additions)
    • Q3 Has the user contributed good 'new' pages on a TF2 patch day, related to items or features the patch added?
    • Q4 Does the user have consistently good spelling and grammar, and/or do they often correct the bad grammar and spelling of other users?
    • Q5 Has the user made good, quick decisions on correcting and reverting other users' edits? This includes patrolling Recent Changes and deleting bad Trivia/Bug contributions.
    • Q6 (If the user is fluent in a non-English language) Has the user made highly accurate, localised translations of English articles into other languages?
      • Since this question cannot be answered by most of the Trusted Users group, the staff will need to try and ensure there is at least one person on the Trusted team, per language, who can assess this honestly (a 'Loc Mod', as Epic Eric calls it).
    • Q7 Has the user made significant 'technical' edits? This includes creating or fixing templates, renaming pages, redirects and categories.
    • Q8 Excluding anything uploaded for personal use, does the user upload files of sufficient quality and usefulness?
    • Q9 Has the user made a recognised contribution to one of the wiki's various Projects?
    • Q10 Does the user spend time putting the Welcome template on new accounts' Talk pages, or otherwise teach new users about editing techniques and etiquette?
    • Q11 Can the user be considered "a friendly regular" in the IRC channel, on the wiki's TF2 server, or somewhere else with a high population of fellow editors?
  • Alongside each answer in the above section, one or more links to evidence, whenever possible or appropriate.
  • A link to the Nomination section on the nominee's Talk Page.
  • An overall 'Yes/No' judgement on whether they personally think the user should reach Phase 3.
    • Any comments the 'reviewer' wants to make should be left on the user's Talk Page.

After the nominee is added to the table, the other Trusted Users can Pictogram plus.png Agree, Pictogram minus.png Disagree and/or Pictogram info.png Make A Correction for each answer on the check-list, and make their own overall 'Yes/No' judgement on the candidate's eligibility for Phase 3.

  • If the Trusted Users want to go in-depth on their reasons for Agreeing/Disagreeing with particular points, these comments should be left on the nominee's Talk Page.
  • The table will be set up so that only a set number of Trusted Users can reply to a given nomination.
    • Pictogram question.png Discussion Point: How many Trusted Users should review a nominee? It has to be an odd number (like 9) to prevent tiebreaker situations.
    • Pictogram question.png Discussion Point (proposed by Noble Scout: Should there be two different types of "reviewer"? One group that does all the preliminary reviews, and another group that only responds to existing reviews?
    • Pictogram question.png Discussion Point (proposed by Noble Scout: Should the Trusted Users have to follow an "average number of reviews per week" rule to keep them active and responsible?


MOCKUP:

Nominee Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10 Q11 Ratio Discussion Reviewed By Verdict
User:Person_A Pictogram plus.png Y Pictogram plus.png Y Pictogram plus.png Y Pictogram plus.png Y Pictogram minus.png N Pictogram neutral.png ? Pictogram plus.png Y Pictogram minus.png N Pictogram plus.png Y Pictogram minus.png N Pictogram plus.png Y 75% User talk:Person_A/Wiki Cap Nomination User:Person_F Pictogram plus.png Mmmhmm
Reviewer Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10 Q11 Notes Verdict
User:Person_G Pictogram plus.png Y Pictogram plus.png Y Pictogram plus.png Y Pictogram plus.png Y Pictogram minus.png N Pictogram neutral.png N/A Pictogram plus.png Y Pictogram minus.png N Pictogram plus.png Y Pictogram minus.png N Pictogram plus.png Y Fully agree; hard-hat's one reliable dude. Pictogram plus.png Yeah
User:Person_H Pictogram plus.png Y Pictogram plus.png Y Pictogram plus.png Y Pictogram plus.png Y Pictogram info.png Y Pictogram neutral.png N/A Pictogram plus.png Y Pictogram minus.png N Pictogram plus.png Y Pictogram minus.png N Pictogram plus.png Y Correction: He's űber-fast at fixing ze bad coding. Evidence here unt here. Pictogram plus.png Yah


Once all the spaces are filled, the ratio of overall Agreement/Disagreement is checked and the nominee's information will be moved to one of two tables:

  • Phase 2: Candidates if the majority voted 'Yes.
  • Phase 2: Non-Candidates if the majority voted 'No'
    • Pictogram question.png Discussion Point: How long should a Phase 2 Non-Candidate be made to wait before they can return to the "Filtered Nominations" table?


Phase 3: Decision (Staff)

Based on points that Epic Eric and Keisari discuss on the staff page, the Staff portion of the process must avoid a few existing problems: Time-Zones, Responsibility, and IRC Participation. This proposal will provide possible solutions to the first and third problems, and possible reduce the second problem.

The wiki staff could host an off-site private voting and discussion system for filtering the Phase 2 Candidates, eventually deciding who earns a Wiki Cap and who doesn't, based on the evidence presented (which should be very easy to read through at this point, having been collected and collated by the Trusted Editors).

Presumably, the staff will want to use the Phase 2 system for themselves (table of information with yes/no questions, linked evidence and a final yes/no judgement), but with room to add comments that the other staff members could read. This would be a system where time-zones are irrelevant; the staff member simply visits a webpage and contributes to it in their own time. It's ultimately up to them how they want to handle this Phase (as they might want to include live discussions in the process somewhere), but something is practically certain: A minimum number of staff members have to vote on a nominee before a final decision is reached.

  • Pictogram question.png Discussion Point: How many staff members should review a nominee before a final decision is made?

Anybody selected for a Wiki Cap would have their information moved to the (invisible) Selected Users table. If the staff turn somebody down, they'll get moved to the Phase 2: Non-Candidates table and be subject to the table's "waiting period" before being re-considered for nomination.

Diagram

User Mainman Phase Syatem Diagram.png

...And that should be everything! Please leave some feedback! Mainman (TalkContribs.) 16:52, 31 August 2012 (PDT)

 –  UPDATE #1: Added the "In a Nutshell..." section, added discussion points suggested by Noble Scout. Mainman (TalkContribs.) 15:43, 1 September 2012 (PDT)


Discussion Points

Please make all your replies in this section, so that the proposal stays in one piece.

Phase 1

Should "have a Wikichievement" be a requirement for the Points system?
How many Points should you need to reach Phase 2?
What kind of % Ratio between Agrees/Disagrees would be appropriate?

Phase 2

How many Trusted Users should there be?
How many Trusted Users should review a nominee?
Should there be a dedicated team of "Preliminary Reviewers"?
Should there be a "minimum activity" rule to keep reviewers active?
After failing Phase 2, how long should users have to wait for a retry?

Phase 3

How many staff members should review a nominee?

Any Other Business

For anything else you want to bring up that isn't listed as a Discussion Point.

Uhh... well, this seems like a lot of work, but I have several questions, specially about Phases 1 and 2.
  • Wouldn't a single vote be big enough for a high ratio?
  • How would this incentivate users to have a better review on candidates?
  • Since I don't expect people to vote negatively (I can imagine several reasons why), this ratio system wouldn't change a thing.
  • How are Trusted Users selected as Trusted Users?
  • Why do people have to be jacks-of-all-trades to win a Wiki Cap?
  • Don't you think only one Trusted User can be too one-sided for a decent filtering?
 –  Epic Eric (T | C) 17:24, 31 August 2012 (PDT)
  • There is also a minimum number of votes. Having one positive vote alone would make your ratio 100%, but you'd remain in Phase 1 until enough votes had been made. Ratios could also be removed from the process entirely (see point #3).
  • The unfortunate truth is, if we allow users to give freeform reasons for their nominations, it's going to produce similar results to the old system. Freeform feedback is valuable in most cases, and so I made sure to include it in the process while also making sure none of it appears beyond Wave 1 (By the time a nomination gets through Wave 2, the Trusted Users have verified and linked to all the evidence you need to make a good judgement on the nominee).
  • In retrospect, I agree; people seem to have no trouble leaving negative feedback about nominees on the old Nominations page, but leaving the same negative comments on a user's Talk page (where they'll almost definitely read it) is risky and will probably result in arguments. The "ratio" system could be removed and Wave 1 could rely purely on "minimum number of 'Agree' votes" instead.
  • There's already a 'Trusted Users' system visible in the IRC channel; when Spacenet posts an edit, the username is always either green or purple. Green names belong to administrators, moderators, and a select few 'normal' users who have demonstrated that they can be trusted with extra responsibility and powers (editing the IRC channel welcome message, for example). This nomination system would be done the same way; a Trusted User is someone the staff feel can be trusted with higher responsibility, including the fair assessment of wiki cap candidates.
  • The Yes/No questions are only there for reference, to know at-a-glance what the user is good at. The only thing that ultimately matters is their Yes/No "Verdict" votes; if only two of the questions can be answered with "Yes", but the nominee is exceptionally good and valuable in these areas of activity, then they could still get an overall "Yes" verdict.
  • One Trusted User takes it upon themselves to research the evidence and present it in the appropriate table. The job of the other Trusted Users is to verify this evidence, make corrections or additions, and ultimately give a balanced opinion and verdict.
 –  Mainman (TalkContribs.) 11:02, 1 September 2012 (PDT)

Verdict

Put your overall Agree/Disagree points for this proposal here.

@Mainman Very complex this system and detailed, but it makes it even more hard to nominate people. It is not that they dont care about the person who they nominate but they dont ask themselfs these questions. Most people have 1-4 core elements which makres them nominate the person and that's it. I would prepare a clean approach, with fixed terms which are just added. Positive as negative stated, but shorter and more precise, see above my approach. Maybe we can combine that. δ³Σx² 17:50, 31 August 2012 (PDT)
Indeed, it is overly complex and just seems too difficult to understand. Perhaps integrating that idea of Pros/Cons and one reason per nomination per person in to the existing system might mix things up enough? User Wingless Winged Signature.png 17:54, 31 August 2012 (PDT)
The jist of the system seems to crowdsource (someone's gonna hate me for using that word) the "mechanical" verification tasks to a more general public, as well formalizing each step with precise criteria to meet. The crowdsourcing aspect is probably a good thing, as it partially addresses the staff willingness/laziness issue (part of the lack of motivation is, I guess, the amount of work that goes into it. When a wiki cap discussion begins, it's often for a good couple of hours, and no one like to dive into that on a nice sunday). It also has some openness benefits, with the bigger community participating closer. The formalization aspect however is, as it stands, way too complicated for it to be practical. It adds a lot of bureaucractic overhead, of extra enforcement work (when people don't follow the rules), and it would probably put off a large number of people (who would otherwise have contributed) merely by being complex like this. This being said, it's also impossible to crowdsource work without having it framed in a formal system, so some level of formalization has to happen (this is why you have upvotes/downvotes on Reddit or YouTube or Steam Greenlight etc.), but I think this proposal is overdoing it. I do appreciate the level of detail, thought, and research that has gone into this though. Perhaps one day under some distribution systems, that may just warrant a wiki cap :3 — Wind 14:27, 1 September 2012 (PDT)