Idézet: ImLittleJon - 2010.04.07. 19:48:39 I've been thinking about this issue, and the way I see it, there are 4 issues:
1) How long it takes to get a question moderated. My first question took 6 days. My other ones have been about 3 days. If it usually took only 1 day to get a response, it would be less of a big deal to have to re-submit. Solution: get more moderators. Effort on the part of the game developers: extremely low.
Difference is quality vs. quantity. Granted, the quality is pretty poor right now - but do we want it getting worse?!?
2) Lack of communication between user and moderator. It's frustrating to have a question rejected and not understand why. Solution: there should be an additional field for the moderator to be able to type a (short) explanation, in addition to the broad category of spelling / too specific / etc. In addition, there should be an extra field on the page where you submit the question to add a message to the moderator that is not part of the question, such as urls of supporting documents. Now, this would make the moderation process take longer, so even more moderators would be needed to handle the additional load. Effort on the part of the game developers: minimal.
This is a great idea, and would actually lessen the workload on the Mods, as the research link is already there - as long as it's a reputable source.
3) False positives: seeing quiz questions that were accepted which should have been rejected. Solution: on every quiz question, there should be a button to allow users to flag it as bad. Ideally, they could then pick from the same list of rejection categories as the moderators do and fill in the (short) explanation for why it should be rejected. The question should then be sent into the moderation queue, with the values pre-filled out. If the moderator agrees, they should be able to delete the question using the reporter's text with the click of a single button. If the moderator agrees but doesn't like the reporter's text, they can edit the text first and then delete the question. If the moderator disagrees, they should be able to reject the report by clicking a different button. In that situation, it might make sense to give the reporter 1 bad question on their tally. Effort on the part of the game developers: low to medium.
Ideally good, but practically, a nightmare in waiting. Imagine if a guy flagged every question they didn't know? Repeat by 500? There is a fundamental difference between a difficult-but-fair question, and a question from out of left-field (a retarded question). The Mods often can't keep the two separate - would you really expect all the users to be able to draw that line? Hell, even I would flag some of those more obscure Lit questions and Hungry-centric ones, just out of spite. It would just create major issues in a game that is already struggling with fair geographic representation.
4) False negatives: getting your question rejected by a moderator, when other moderators would approve it. There is no way to fully avoid this issue other than doing away with judgment calls by moderators entirely. Which has been proposed a few times by different people lately. But those proposals would basically put the game back into the state where everyone was a moderator, which the game designers tried and didn't like. So I think those proposals have very little chance of being implemented. So within the current system, what could be done to alleviate the issue? Well, if the response time was quicker, there was some explanation as to why it got rejected, and there were fewer false positives, that would go a long way. Maybe that's all we can hope for.
This is the biggest singular problem with the quiz - zero accountability by the Mods. They can decline a question without cause or reason. They can approve a question without cause or reason. Yet there is no way to get more information as to why that decision happened.
Short of using a Multi-Mod system, the only fair way to do it is to give the question asker an opportunity to
challenge a Mod decision - likely by writing their reason in a box on the "Rejection" frame, and sending it for re-analysis by a
different Mod. Will this happen? Not likely. What this would do is allow tracking of "reversed rejections", and if a Mod falls below a threshold, they get cut. This game is in desperate need of some accountability!!!
Here's one other, more radical, idea (though I expect it also has very little chance of being implemented). Maybe the moderators could give a question a status of provisional, in addition to accepting or rejecting it. Provisional questions would then go into a state where everyone was a moderator, either with a yes/no control or simply by how many answered it correctly. Then, after a specified time period, it would either attain permanent status or be deleted.
That sounds like my proposal of all-questions-pass (Mod verifies correct spelling & answer - ensuring that an answer CAN be found) for 10 reviews, if 7+ of 10 get it wrong, it goes to a Mod for difficulty review. I'm still in favour of it.