Team Fortress Wiki:Discussion/Wiki Cap

It has been a year

One year (and a month) ago, we were finally stablishing the new system that would properly reward editors. While it did serve for quite a long time, new issues started to emerge and currently make it impossible for any Wiki Cap discussion to take place. There have been seven opportunities in a row where a Cap vote could have taken place -- but there were no discussions on candidates. Plus, the list of nominations has more than doubled since. The number of staff members active for cap votes slowly decreased from 7 to 4, even getting as low as 2 occasionally, while our minimum for any decisions was 5. The way things are going, this system won't be of any help.

Of course, going back to the previous vote system is completely out of the question. But there are several flaws that make the current system unoperable. I've created specific sections below for issue-identifying and solution-suggesting; please add to them, or create new ones. Certainly, some of these may seem pretty harsh, but unless we decide something soon, they may integrate the Wiki Cap discussions system.

Also, if you want to make any comments regarding something else, feel free to add it below.  –  Epic Eric (T | C) 14:51, 29 August 2012 (PDT)

Identified issues

Staff willingness

I get this feeling that some staff members don't want to participate in cap votes. That's okay if there were enough mods for Wiki Cap discussions -- except there aren't. Sure, no-one is made of steel and may feel like not-voting one day or another, or are even busy doing something elsewhere, but members of the staff could at least create a list with their opinions on each candidate, plus a verdict, so that other moderators may add it to the discussion. There are also moderators that "idle" in the channel doing nothing, even when they are warned by a stalk word.  –  Epic Eric (T | C) 14:51, 29 August 2012 (PDT)

Attendance rate is determined by two things:

• Not being able to attend due to having other obligations or being on vacations, etc.
• Not wanting to attend

The former varies from mod to mod. As such, forcing a general "attendance rate" won't really work without an explicit "I won't be there for the next n weeks" acknowledgement. Some people have done that (such as myself, when I went on vacations this summer), others have not (for which I won't give names here because everyone who needs to know who they are know who they are).

The latter is a different matter, and those who adhere to this stance have given various excuses for it. Things like not accepting the responsibility and the burden that the Wiki Cap causes; that it shouldn't be just because someone other than them thought a Wiki Cap would be a good idea that they should be the one to suffer the consequences of that decision today (if that run-on sentence didn't make sense, re-read it slowly). Things like not having done their work the day before and now they have to do it that day. Things like just idling during the vote. Things like being lazy. Things like just not showing up anywhere at all, perhaps assuming we forgot about them. Some stances are more reasonable than others, but the net effect is the same: those who do accept to bear the responsibility get to bear a lot more than their fair share. It's psychologically exhausting to keep doing a burdensome task when others (who are on the same level as you) have exempted themselves from that task. — Wind 18:14, 29 August 2012 (PDT)

I want to condense all my points in one paragraph so I shall ride off this comment. For me, this is the whole root of the problem. It does not matter how the nominees are selected, the fact is some people are not bothered/unable to attend cap votes. It takes a while for a vote to start up and we all have stuff to do. As you said, the fact that some staff members (myself included) exclude themselves from votes for whatever reason causes resentment in the other staff members. I truly believe if we changed the system, there would be a period of renewed hope only for staff members to once again get tired of the monotony and fall into the same old routine. I really do not think it matters in the slightest how the nominees are bought to our attention or who they are. Even if we somehow removed unworthy cap nominees there are still many editors who are deserving of one, and trying to separate them is an almost impossible task for anyone. So when staff can spend hours debating the merits of nominees this only adds to the huge timesuck that is the capvote, hence the resentment of those participating to those who aren't. I can only see one solution to this issue. Nominate a small number of staff members who are willing to do the cap vote regularly. Instead of spending several hours waiting for 6/7 members to turn up at the same time, reduce it to 3 or even 2. Allow other staff members to leave comments in the cap channel during the week if they wish to point something out about a nominee, but at the end it is decided by 2 or 3 people.

Regarding any system that involves editors filtering out nominees (including the current system), it can only end badly as editors take up a 'scratch my back' attitude. Another benefit of having less staff members vote is that they can decide quicker on nominees they don't feel deserve one yet. —Moussekateer·talk 16:00, 7 September 2012 (PDT)

Lack of loc mods

There have been several situations where a loc mod was necessary to evaluate a certain candidate. One of the most recent examples involved a user for a language that does have a moderator, but he isn't active -- which is a different discussion. For these cases, we have been contacting Steam Translation Server moderators in order to get an accurate description of each candidate's quality. While it helps clearing some boulders, this isn't a practical solution. More localization mods for more languages means more readiness and confiabilityness when evaluating a candidate.  –  Epic Eric (T | C) 14:51, 29 August 2012 (PDT)

While there is a shortage of loc moderators, and shortage of activeness of the existing ones, I think it is fair to say that there has also been a shortage of good candidates for replacement in some languages. Some languages don't have a loc mod at all and never had one because there never really was a candidate. Hiring more loc mods where possible is a good thing and will help things around the Wiki, but by itself it is not a complete solution to the cap problem. I think another thing which would help is purging the inactive ones. — Wind 18:14, 29 August 2012 (PDT)

Occasionally we have seen the need for STS mod to check out the translations. Sometimes this process have delayed the result for weeks, after all its not their work and all we can do, is politely ask them to review the candidates. -- Keisari 04:59, 30 August 2012 (PDT)

Oh that reminds me, Tturbo suggested Stache as a replacement for him. Though I see he didn't note that on this talk page like I suggested. I can't remember if he said he did STS strings or not, but does it really matter as long as they know the language? STS strings aren't set in stone until Valve pushes them. -- Lagg 05:06, 30 August 2012 (PDT)

It's not on the talk page because it's not official it should be done later today... STS strings aren't set in stone but they are quite solid I can assure you. When a token gets on the site you know it's going to end up in the game. Eventhough I'm not sure 'Stache is on STS, I'll still be there and accessible if you need me, what's important in my opinion is to know the language and how the wiki works. Tturbo _{( / )} 07:48, 30 August 2012 (PDT)

(on a side-note, no, he's not in STS. Still, that usually doesn't mean anything in discussions).  –  Epic Eric (T | C) 09:38, 30 August 2012 (PDT)

Time zones

Being a world-wide website, we expect people from all around the globe to be part of this project. But since the Earth is spherical, people have different timezones and can't all be online at the same time, which means discussions sometimes have to be interrupted so that one mod or another may leave their opinion, specially when it is tie-breaking. Plus, we're probably abusing the European mods, making them stay awake for way too long before Mondays, which is a work day for most people. I know I'm just stating the obvious here, but this an important issue to take into account.  –  Epic Eric (T | C) 14:51, 29 August 2012 (PDT)

The timezones indeed put some pressure on the european mods, especially those from eastern europe and western russia, making them regularly stay up after midnight. -- Keisari 04:56, 30 August 2012 (PDT)

Oh quit being a bunch of whiners. I stay awake for days at a time for the sake of people in Europe at my job and online alike. -- Lagg 05:03, 30 August 2012 (PDT)

Lack of IRC

There are some moderators among us, who dont use IRC or are not online regularly. Even if these moderators would be active, its absolutely no help if they are not in where the discussion process is held. -- Keisari 05:31, 30 August 2012 (PDT)

Solutions

Vote enforcement

The solution I hate the most, but still a solution: forcing staff members to leave a opinion on candidates. There could be a minimum of how many candidates a moderator should review (75%, dunno), or else they'd be punished. I personally think it's too hard and a bad solution, but it may be our last-resort plan if all else fails, unfortunately.  –  Epic Eric (T | C) 14:51, 29 August 2012 (PDT)

c.f. my reply in the "Staff willingness" section. Additionally, this is a solution that is likely not to pass due to those who don't want to participate in it, because obviously that wouldn't benefit them. — Wind 18:14, 29 August 2012 (PDT)

New discussion members

Straightforwardly, adding new staff members as we need them. Another option is to let certain non-staff users from the Wiki community partake in the process. I also don't like this idea, but it can help populate Wiki Cap discussions.  –  Epic Eric (T | C) 14:51, 29 August 2012 (PDT)

We supposedly already have this community feedback from the nominations page. Sadly, it has turned into something political; users encouraging other users to nominate them, nomination reviews that sound like templated professional recommendation blurbs instead of getting to the meat of things and giving examples, etc. Perhaps that last bit could be alleviated a little by changing the wording on that page, but it won't help the backroom nomination deals problem.

This is also probably why the actual text of the nominations is very seldomly used during the voting process. It is generally useless, and I feel bad for the people who wrote truthful nominations because they are muffled under the noise. — Wind 18:14, 29 August 2012 (PDT)

Having more moderators could help the discussions go on for some time, but it does not fix the core problem. As Wind pointed out the nominations have gone political and i see possible problem in larger community based discussions where one can give "yes", just because nominee is their friend, or gives other goods to them for example (the wiki caps are sought for collector items after all!). Not to say this cant happen with staff members, but i belive it is more under control and easier to monitor. -- Keisari 05:31, 30 August 2012 (PDT)

Static discussions

Instead of quick IRC discussions, we should have them at slower rates, allowing mods from anywhere to add their comments and verdicts in no rush. The IRC channel would probably be scrapped (or used for quick discussions about one addition or another), while the votes would happen in a shared doc or a shared page or something of that sort. Discussions would take approximately days, but at least more staff members could add into it, plus giving us more time to think about a certain candidate.  –  Epic Eric (T | C) 14:51, 29 August 2012 (PDT)

This is doable but requires some sort of dedicated system with a proper authorization wall and, most importantly, a push to remember people to go back to it every so often. If things take place in some sort of forum-style thing, then it won't really work because people will "forget" to come back to it, etc. I think things like discussions-with-deadlines and regular email reminders or something may be necessary. This way it will be easier to tell who is participating and who is not. — Wind 18:14, 29 August 2012 (PDT)

 Support  Suggestion My favourite solution so far, after reading the page. I think it would actually entice me to start getting involved in the cap votes again - being able to contribute when I find the time, instead of in and instant-message environment when I would much rather be spending my time relaxing all cool hexing some nuker outside the tier 2. For proper authorization, Google Docs with staff members' Google accounts hooked up could be a workable solution; a doc could easily be formatted just like a Wiki discussion (or many other ways) - while remaining staff-only and also allowing nested comments - allowing easy comments and discussions on a variety of aspects about a candidate, something which forums' would make difficult. Google Docs also allows mass-emailing all collaborators, so poking everybody reminding them to put their comments down is pretty simple too. -RJ 13:36, 31 August 2012 (PDT)

Whenever there's an opportuniy

At any time 5 or more moderators are available for a Wiki Cap vote, a discussion will take place, regardless of the time or day. Of course some mods willing to participate won't be able to, but at least there'll be discussions. The problem here is, of course, getting these mods over for a vote.  –  Epic Eric (T | C) 14:51, 29 August 2012 (PDT)

 No. This would add entropy to the decision process, something which I, if I were nominated, would not want to deal with. I wouldn't want the decision to be based on some randomness over which I have no control. The process should be deterministic, or at least keep enough predictability for people to trust it. A system can only be trusted if it is transparent and not too unpredictable. — Wind 18:14, 29 August 2012 (PDT)

Less staff members for a decision

A less drastic solution would be to decrease the number of necessary staff for a vote accordingly. I don't think that less than 5 voters is good for any serious decision, but if there were only (say) 3 moderators discussing, they would speak for the entire staff in their decision. Somewhat painful, but a solution nonetheless.  –  Epic Eric (T | C) 14:51, 29 August 2012 (PDT)

This would be workable because while it's sometimes hard to get 5 or 6 people, it not hard at all to get 3. This is a plausible solution to continue with the current system unmodified, but only if those who don't participate accept that the decision should depend on fewer people. — Wind 18:14, 29 August 2012 (PDT)

While this is a solution, the idea of 2 staff members needed to make decision is troubling, as it makes subjective view to rule the discussion much more likely than with more members. -- Keisari 06:28, 30 August 2012 (PDT)

The problem with all of these proposals

Is that no matter what is proposed, it's not going to solve the problem of people becoming frustrated with how the nominations keep ending up filled with people who do lots of redirects, lots of translations, and overall do vast quantities of editing without a large measurable gain from it. Proceeding to nominate the same people that nominated them as a back scratching favor. Then have it coming down to: "Well, it looks like person A works the hardest and have been here for X days. They deserve it more.". It's fairly obvious to me and others at this point that even the nominators themselves can't figure out a good reason for their nomination so they fall back to "he works pretty hard and has over a million billion edits". You're looking in the wrong place if you want to fix this. Instead of starting with the staff attitude being the problem, try considering why they feel this way in the first place. -- Lagg 18:16, 29 August 2012 (PDT)

I understand your point. I agree there are several instances where personal relations are used as a basis for a nomination (not that we should be cold machines). Yes, our evaluation process should be fully reviewed, taking in consideration how quality has nothing to do with quantity; and yes, the nomination process should also be filtered better. What I'm saying is that one of our biggest problems to actually get a discussion going is how we have enough staff to do them, but none that actually "move". There may be problems with their stance on nominations, and I don't know if they avoid joining the channel for this sole purpose, but it won't hurt to give your opinion at least once a week. Anyone is free to create sections detailing their issues here, and we will look into it.

Also, if the nomination is not fair, because the user still doesn't deserve the hat, then the user will be rejected in a discussion. Simple as that. As for staff attitude, I see little movement from certain people (whose names I won't say, for the reasons Wind stated in Staff willingness, and because pointing fingers is bad). The entire Wiki Cap selection process is a duty of the staff -- not one member or another, but the entire staff. Of course, when they choose not to partake in discussions, it's their choice, but in this volume, it is enough to entirely break this structure.

If these staff members think that an user was overrated for personal reasons, instead of hiding away and let it solve itself, they should give their opinion on the discussion on why that editor shouldn't be awarded a Wiki Cap. If we don't like it, we should do something about it, not ignore it.  –  Epic Eric (T | C) 19:02, 29 August 2012 (PDT)

It's not a matter of them ignoring it so much as it is giving up on it. Some staff regret the hat entirely for the reasons we've been observing lately, so I honestly understand their reasoning if they don't want to bother trying to rationalize giving it to people for reasons it was never designed for in the first place. I can't speak for them but I'm guessing more staff would mobilize if they didn't feel like it was futile. That's why you can't fall back to "they'll just get rejected anyway". They probably will, but maybe not. Especially if someone goes with the admittedly bad idea that giving the decisive power to 3 people is. The fact that it keeps happening time after time indicates a problem that is definitely not on the staff's end. It's our responsibility to give the hat to people that deserve it, but it's the other editor's responsibility to alert us to who they think deserves it properly. Now I might be the exception to what I said before, I'm tired of it but I'm still there if someone reminds me. However some things said make my blood boil and no matter how many times I reiterate these things it passes through the ears. Yet I'll keep doing it because I'm the type that knows repetition is learning, but for the same reasons if other staff members don't have the energy to do this I honestly don't blame them. So again, start at the correct problem. -- Lagg 19:12, 29 August 2012 (PDT)

Certainly, a better selection process for nominations would be great. But how should we "educate" editors for this? The only current pre-requisites for nominations are the eligibility factors. Do you think a better detailing on these would help? Or maybe letting the staff do it, even though they'll probably overlook some editors that deserve a cap? Or a periodic cleaning by mods on bad nominations, deleting those without a strong argument?

Another question: what would be the reason the Wiki Cap was designed for and why is it not being followed to its full extent in recent discussions?  –  Epic Eric (T | C) 19:32, 29 August 2012 (PDT)

Well I already attempted to start by putting a bold note near the nominee table noting the redundancy of giving edit counts as a reason, and I think (I haven't looked through it carefully yet) it's being followed. I don't think staff can do it themselves because it defeats the point of open nomination. Cleaning the ones out that don't have sufficient reasons would be an acceptable place to start. Ideally people would only put there what can be used in the discussions themselves, as it being so bad that it is disregarded in the staff discussion indicates that people aren't using it what it's meant to be used for. Namely to stop people from being gung-ho and slapping every person they think of up there or trying to do a "I scratch your back" sort of deal, and giving us something to start at in discussions. The cleanings would probably be done as soon as possible instead of periodically for maximum discouragement. As for what the hat was designed for, it was always meant to be a reward for useful and major contributions (and by extension an incentive to get people to join and contribute). The only thing it has done is cause an influx of bad edits that we spend most of the day cleaning up. Then we have idiots (who will not be named) that have been found trying to get it (or in a few cases actually getting it) via pretty anti-social methods to put it lightly. Not to mention it being used as a token for people that are "working hard", which means very little since all editors work hard. Making putting in effort seem like the exception rather than the rule is pretty discouraging to people who wonder why they didn't get the hat when they were working equally hard but didn't have a friend to play musical nominations with. -- Lagg 20:10, 29 August 2012 (PDT)

The Wiki Cap was created to be given to "valuable contributors" to the Wiki; that's what the item description says. We always have a problem on agreeing on what is valuable and what isn't, which is part of the problem. If an editor works hard to the point that their total contribution to the Wiki is significant, does the fact that other editors put in the same amount of work make their contribution any less valuable? — Wind 13:09, 30 August 2012 (PDT)

New proposal table

Simplify the nominations page. Put qualification-standards on top of page. One can only add nominees name on the page and their own name as a sign who added them. No multiple backers, no reasons, no dates. Now the page has names on them and every staff member has a right (but no need to) to pick anyone from the list to be discussed. The page would be cleared in set period of time, and it starts all over again. This is pretty raw idea and can be refined more, but the main bonus from this would be that there would not be pointless discussions anymore as only staff pick the ones for the review. -- Keisari 06:28, 30 August 2012 (PDT)

It already was like this more or less before. People need to give a reason for nominating someone, if they don't they have no business doing it in the first place. As I said above it serves both to give us somewhere to start a discussion at and to discourage people from adding others without a second thought or as a favor. There will still be pointless discussion because I can guarantee you that we'll spend most of the time trying to figure out both why the person chose the nominee, and then dismissing them because of uselessness. We could ask the nominator directly for their rationale but then that is the point of the reason column. -- Lagg 06:42, 30 August 2012 (PDT)

Sidenote: The Kritzkast case

I'd like to point out that something has changed since last time we discussed this: the Lo-Fi Longwave. We are no longer the only community who is able to create items. There's been an asston of negative feedback to the very restrictive original release mechanism of the Lo-Fi Longwave (being in a Steam group preemptively and correctly filling out an actually-pretty-tough quiz), and after that Kritzkast changed to a simple hey-this-guy-deserves-it-boom-hat model, with occasional community events with those hats as reward, and with a one hat per week for the "Player of the Week" in their Steam group. I'm not saying that their model is superior, but you'll notice that all of those methods are all under staff control and there is no community involvement in deciding who gets the hat; there is only community participation caused by the promise of the hat, there is no community participation to determine who or how someone gets the hat. I think this is kind of key here: If one lets the community decide who gets the hat, then there will be competitive behavior and backroom deals and all that crap. I'm not suggesting that we change to a staff-only nomination system, I simply want to point out why I think the Kritzkast guys are probably having less trouble than we are while getting away with it. — Wind 13:09, 30 August 2012 (PDT)

During the last month...

... we actually made a Wiki Cap discussion. The good part is that, even though we only had 4 mods discussing, it "worked", and a Wiki Cap was eventually handed out. The bad part is that there was only one discussion and 2 candidates reviewed, while the list of candidates up for review has increased in 3 in 30 days (which means in this rate, the list will slowly fill up until we cannot handle it). What has become obvious is that lowering the number of staff for a discussion to 4 wasn't much of a trouble, and I guess even at least 3 mods could be enough for "clearer" verdicts, but as it was also noted, there are issues with "being in the mood" for this. Now that we've experimentated with some points mentioned in this page, the question is: how to tackle this issue, and how to probably implement other solutions for everyone's sake.  –  Epic Eric (T | C) 17:30, 1 October 2012 (PDT)

Having it at 4 people was already pushing it. So it really shouldn't go lower than that if we're going to keep objectiveness. The number of people reviewed was an increase as it was and I think it could get better with time, I'm personally making sure that there are only sane nominations now so if it fills up that fast it indicates that I'm either not doing my job or the people doing the nominating aren't doing their's. There's no reliable long term solution besides giving people a reason to be interested in it, which as far as I'm concerned means putting a stop to the nonsense. It's entirely possible we're too late on jumping on this, but if that's the case then that's how it's going to be. It seems more common now that people are saying how tired they are with the hat and regret it ever happened. In the worst case scenario we're just going to be a learning lesson for other community projects. I said quite some time ago that we should stop with the scheduled voting and discussion and do it only on a per-exceptional-person basis. That's the only other solution I can think of. -- Lagg 12:59, 8 October 2012 (PDT)

Current situation & vote

For a few weeks now, we haven't been able to get 5 people at IRC same time, not on Sunday nor on Saturday. Which means that we probably aren't going to get 5 people there at the same time for a long time. Thus, I'm proposing 2 solutions, and both have been proposed before:

• "Hiring" more moderators, because amount of active moderators (in IRC) is so low
• Moving to use static discussions.

I recommend you people to give your vote on this; we need to do something, last verdict was weeks ago, and some people have waited quite long for their nomination to be reviewed.

The main problems/questions are:

In first proposal: Finding time that everyone can agree with (speaking of time when we do the verdict).

In second proposal: Finding site/method/medium where to do this, amount of time we give people to give their vote, and way to remind them to visit the "voting site".

So, think about this, and please, give your vote. Current situation isn't good, and something needs to be done in my opinion.

-Asplode 11:53, 15 December 2012 (PST)

Option 1

 Disagree The current mods are enough to do all the (other) tasks in the Wiki. Hiring mods solely for getting these discussions going seems like too much for too little. Besides, we have enough staff members for discussions; the real problem is how to get these discussions going. Also, since we are all in different time zones, there's no best time where all of us can be (at least there's one for most of us, but then it doesn't get past 4 mods...).  –  Epic Eric (T | C) 12:27, 15 December 2012 (PST)

 Disagree Hiring mods just so that they can vote is bad idea. However,  Agree hiring mods (for both voting and regular mod business) is good idea. — Wind 12:48, 15 December 2012 (PST)

Option 2

 Support Continuing my line of thought on my opinion for the first option, the problem is not the number of mods, but how to get all the mods from an international Wiki to communicate with each other. The best solution, then, would be using static discussions. This way, mods do not only have the chance to give their opinion on a certain candidate ragrdless of where they are, they can also take more time to get info on acertain candidate to further improve their decisions. Plus, it's a lot tidier than IRC or any instant-communication stuph. Anyway, if we are to move to static discussions, I'd propose they'd be on Google Docs, as 1) forums may not be suitable for this (as we may want to separate opinions in topics and in a single thread, this would be harder to see than in a doc) and 2) when users request our discussions on them, it'll be easier to share it with them that way. And these discussions could go like: 3/4 users a week, during weekdays we give our opinions and show good/bad points about the candidates, on the weekend we'll say our verdicts and drop Caps (if any). It'll give everyone enough time to give their thoughts on each user. If it seems too slow, we can hurry it up or increase user count.  –  Epic Eric (T | C) 12:27, 15 December 2012 (PST)

 Neutral I still think it won't work and that over time, people will either forget them, either "forget" them. Maybe they should come with an inactivity threshold after which the person gets de-mod'd if they don't respond, unless they say that they're on vacations or something beforehand — Wind 12:48, 15 December 2012 (PST)

Yeah, that brings us back to #Vote enforcement. People who are willing to participate won't get harmed, but people who don't might be in trouble. As you said in #Staff willingness, there's a huge difference between not being able to and not wanting to attend discussions. This option would eliminate the first case, but as Mousse mentioned in that section, (some) people will get bothered by this eventually. Which brings us to a new point: would these staff members that do not want to participate in discussions, but were obliged to, really contribute or it'd decrease the discussions' overall quality? I really hope it doesn't, and that this only becomes a solution in a last-resort case if this solution still isn't enough to get at least 5 people for a discussion.  –  Epic Eric (T | C) 13:09, 15 December 2012 (PST)

If people are made to come to discussion, it will lower the quality. It should be something like "optional, but mandatory". Participating in cap votes is one of the things mod should do, and many people seem to forgot or "forgot" that. -Asplode 13:14, 15 December 2012 (PST)