Team Fortress Wiki:Discussion/Wiki Cap

It has been a year

One year (and a month) ago, we were finally stablishing the new system that would properly reward editors. While it did serve for quite a long time, new issues started to emerge and currently make it impossible for any Wiki Cap discussion to take place. There have been seven opportunities in a row where a Cap vote could have taken place -- but there were no discussions on candidates. Plus, the list of nominations has more than doubled since. The number of staff members active for cap votes slowly decreased from 7 to 4, even getting as low as 2 occasionally, while our minimum for any decisions was 5. The way things are going, this system won't be of any help.

Of course, going back to the previous vote system is completely out of the question. But there are several flaws that make the current system unoperable. I've created specific sections below for issue-identifying and solution-suggesting; please add to them, or create new ones. Certainly, some of these may seem pretty harsh, but unless we decide something soon, they may integrate the Wiki Cap discussions system.

Also, if you want to make any comments regarding something else, feel free to add it below.  –  Epic Eric (T | C) 14:51, 29 August 2012 (PDT)

Identified issues

Staff willingness

I get this feeling that some staff members don't want to participate in cap votes. That's okay if there were enough mods for Wiki Cap discussions -- except there aren't. Sure, no-one is made of steel and may feel like not-voting one day or another, or are even busy doing something elsewhere, but members of the staff could at least create a list with their opinions on each candidate, plus a verdict, so that other moderators may add it to the discussion. There are also moderators that "idle" in the channel doing nothing, even when they are warned by a stalk word.  –  Epic Eric (T | C) 14:51, 29 August 2012 (PDT)

Attendance rate is determined by two things:

• Not being able to attend due to having other obligations or being on vacations, etc.
• Not wanting to attend

The former varies from mod to mod. As such, forcing a general "attendance rate" won't really work without an explicit "I won't be there for the next n weeks" acknowledgement. Some people have done that (such as myself, when I went on vacations this summer), others have not (for which I won't give names here because everyone who needs to know who they are know who they are).

The latter is a different matter, and those who adhere to this stance have given various excuses for it. Things like not accepting the responsibility and the burden that the Wiki Cap causes; that it shouldn't be just because someone other than them thought a Wiki Cap would be a good idea that they should be the one to suffer the consequences of that decision today (if that run-on sentence didn't make sense, re-read it slowly). Things like not having done their work the day before and now they have to do it that day. Things like just idling during the vote. Things like being lazy. Things like just not showing up anywhere at all, perhaps assuming we forgot about them. Some stances are more reasonable than others, but the net effect is the same: those who do accept to bear the responsibility get to bear a lot more than their fair share. It's psychologically exhausting to keep doing a burdensome task when others (who are on the same level as you) have exempted themselves from that task. — Wind 18:14, 29 August 2012 (PDT)

Lack of loc mods

There have been several situations where a loc mod was necessary to evaluate a certain candidate. One of the most recent examples involved a user for a language that does have a moderator, but he isn't active -- which is a different discussion. For these cases, we have been contacting Steam Translation Server moderators in order to get an accurate description of each candidate's quality. While it helps clearing some boulders, this isn't a practical solution. More localization mods for more languages means more readiness and confiabilityness when evaluating a candidate.  –  Epic Eric (T | C) 14:51, 29 August 2012 (PDT)

While there is a shortage of loc moderators, and shortage of activeness of the existing ones, I think it is fair to say that there has also been a shortage of good candidates for replacement in some languages. Some languages don't have a loc mod at all and never had one because there never really was a candidate. Hiring more loc mods where possible is a good thing and will help things around the Wiki, but by itself it is not a complete solution to the cap problem. I think another thing which would help is purging the inactive ones. — Wind 18:14, 29 August 2012 (PDT)

Time zones

Being a world-wide website, we expect people from all around the globe to be part of this project. But since the Earth is spherical, people have different timezones and can't all be online at the same time, which means discussions sometimes have to be interrupted so that one mod or another may leave their opinion, specially when it is tie-breaking. Plus, we're probably abusing the European mods, making them stay awake for way too long before Mondays, which is a work day for most people. I know I'm just stating the obvious here, but this an important issue to take into account.  –  Epic Eric (T | C) 14:51, 29 August 2012 (PDT)

Solutions

Vote enforcement

The solution I hate the most, but still a solution: forcing staff members to leave a opinion on candidates. There could be a minimum of how many candidates a moderator should review (75%, dunno), or else they'd be punished. I personally think it's too hard and a bad solution, but it may be our last-resort plan if all else fails, unfortunately.  –  Epic Eric (T | C) 14:51, 29 August 2012 (PDT)

c.f. my reply in the "Staff willingness" section. Additionally, this is a solution that is likely not to pass due to those who don't want to participate in it, because obviously that wouldn't benefit them. — Wind 18:14, 29 August 2012 (PDT)

New discussion members

Straightforwardly, adding new staff members as we need them. Another option is to let certain non-staff users from the Wiki community partake in the process. I also don't like this idea, but it can help populate Wiki Cap discussions.  –  Epic Eric (T | C) 14:51, 29 August 2012 (PDT)

We supposedly already have this community feedback from the nominations page. Sadly, it has turned into something political; users encouraging other users to nominate them, nomination reviews that sound like templated professional recommendation blurbs instead of getting to the meat of things and giving examples, etc. Perhaps that last bit could be alleviated a little by changing the wording on that page, but it won't help the backroom nomination deals problem.

This is also probably why the actual text of the nominations is very seldomly used during the voting process. It is generally useless, and I feel bad for the people who wrote truthful nominations because they are muffled under the noise. — Wind 18:14, 29 August 2012 (PDT)

Static discussions

Instead of quick IRC discussions, we should have them at slower rates, allowing mods from anywhere to add their comments and verdicts in no rush. The IRC channel would probably be scrapped (or used for quick discussions about one addition or another), while the votes would happen in a shared doc or a shared page or something of that sort. Discussions would take approximately days, but at least more staff members could add into it, plus giving us more time to think about a certain candidate.  –  Epic Eric (T | C) 14:51, 29 August 2012 (PDT)

This is doable but requires some sort of dedicated system with a proper authorization wall and, most importantly, a push to remember people to go back to it every so often. If things take place in some sort of forum-style thing, then it won't really work because people will "forget" to come back to it, etc. I think things like discussions-with-deadlines and regular email reminders or something may be necessary. This way it will be easier to tell who is participating and who is not. — Wind 18:14, 29 August 2012 (PDT)

Whenever there's an opportuniy

At any time 5 or more moderators are available for a Wiki Cap vote, a discussion will take place, regardless of the time or day. Of course some mods willing to participate won't be able to, but at least there'll be discussions. The problem here is, of course, getting these mods over for a vote.  –  Epic Eric (T | C) 14:51, 29 August 2012 (PDT)

 No. This would add entropy to the decision process, something which I, if I were nominated, would not want to deal with. I wouldn't want the decision to be based on some randomness over which I have no control. The process should be deterministic, or at least keep enough predictability for people to trust it. A system can only be trusted if it is transparent and not too unpredictable. — Wind 18:14, 29 August 2012 (PDT)

Less staff members for a decision

A less drastic solution would be to decrease the number of necessary staff for a vote accordingly. I don't think that less than 5 voters is good for any serious decision, but if there were only (say) 3 moderators discussing, they would speak for the entire staff in their decision. Somewhat painful, but a solution nonetheless.  –  Epic Eric (T | C) 14:51, 29 August 2012 (PDT)

This would be workable because while it's sometimes hard to get 5 or 6 people, it not hard at all to get 3. This is a plausible solution to continue with the current system unmodified, but only if those who don't participate accept that the decision should depend on fewer people. — Wind 18:14, 29 August 2012 (PDT)

The problem with all of these proposals

Is that no matter what is proposed, it's not going to solve the problem of people becoming frustrated with how the nominations keep ending up filled with people who do lots of redirects, lots of translations, and overall do vast quantities of editing without a large measurable gain from it. Proceeding to nominate the same people that nominated them as a back scratching favor. Then have it coming down to: "Well, it looks like person A works the hardest and have been here for X days. They deserve it more.". It's fairly obvious to me and others at this point that even the nominators themselves can't figure out a good reason for their nomination so they fall back to "he works pretty hard and has over a million billion edits". You're looking in the wrong place if you want to fix this. Instead of starting with the staff attitude being the problem, try considering why they feel this way in the first place. -- Lagg 18:16, 29 August 2012 (PDT)

I understand your point. I agree there are several instances where personal relations are used as a basis for a nomination (not that we should be cold machines). Yes, our evaluation process should be fully reviewed, taking in consideration how quality has nothing to do with quantity; and yes, the nomination process should also be filtered better. What I'm saying is that one of our biggest problems to actually get a discussion going is how we have enough staff to do them, but none that actually "move". There may be problems with their stance on nominations, and I don't know if they avoid joining the channel for this sole purpose, but it won't hurt to give your opinion at least once a week. Anyone is free to create sections detailing their issues here, and we will look into it.

Also, if the nomination is not fair, because the user still doesn't deserve the hat, then the user will be rejected in a discussion. Simple as that. As for staff attitude, I see little movement from certain people (whose names I won't say, for the reasons Wind stated in Staff willingness, and because pointing fingers is bad). The entire Wiki Cap selection process is a duty of the staff -- not one member or another, but the entire staff. Of course, when they choose not to partake in discussions, it's their choice, but in this volume, it is enough to entirely break this structure.

If these staff members think that an user was overrated for personal reasons, instead of hiding away and let it solve itself, they should give their opinion on the discussion on why that editor shouldn't be awarded a Wiki Cap. If we don't like it, we should do something about it, not ignore it.  –  Epic Eric (T | C) 19:02, 29 August 2012 (PDT)

It's not a matter of them ignoring it so much as it is giving up on it. Some staff regret the hat entirely for the reasons we've been observing lately, so I honestly understand their reasoning if they don't want to bother trying to rationalize giving it to people for reasons it was never designed for in the first place. I can't speak for them but I'm guessing more staff would mobilize if they didn't feel like it was futile. That's why you can't fall back to "they'll just get rejected anyway". They probably will, but maybe not. Especially if someone goes with the admittedly bad idea that giving the decisive power to 3 people is. The fact that it keeps happening time after time indicates a problem that is definitely not on the staff's end. It's our responsibility to give the hat to people that deserve it, but it's the other editor's responsibility to alert us to who they think deserves it properly. Now I might be the exception to what I said before, I'm tired of it but I'm still there if someone reminds me. However some things said make my blood boil and no matter how many times I reiterate these things it passes through the ears. Yet I'll keep doing it because I'm the type that knows repetition is learning, but for the same reasons if other staff members don't have the energy to do this I honestly don't blame them. So again, start at the correct problem. -- Lagg 19:12, 29 August 2012 (PDT)

Certainly, a better selection process for nominations would be great. But how should we "educate" editors for this? The only current pre-requisites for nominations are the eligibility factors. Do you think a better detailing on these would help? Or maybe letting the staff do it, even though they'll probably overlook some editors that deserve a cap? Or a periodic cleaning by mods on bad nominations, deleting those without a strong argument?

Another question: what would be the reason the Wiki Cap was designed for and why is it not being followed to its full extent in recent discussions?  –  Epic Eric (T | C) 19:32, 29 August 2012 (PDT)