Felhasználónév: Jelszó:
Oldal: 1234567
Szerző
Topic neve: The Quiz Evaluation Discussion/Debate
averageS10MATT
Végzetúr poronty
6 hozzászólás
Profil megtekintése
Re: The Quiz Evaluation Discussion/Debate
41. hozzászólás - 2010.04.04. 03:01:32
I have now had 2 questions denied, both of which were a hell of a lot easier than some of the ridiculous questions they let go through.
To get my answers i either lready know it, or use google in seperate window, which is easy enough to do in the 3 minutes that i have.
one of them was to ambiguous.
What is the official name of the symbol ...
, which can be answered simply by copying and pasting into google.
and the other was 'only a special group of people might know the answer to this', which again could have been answered by simply copying and pasting
What is Homer J. Simpsons middle name?


Both eay to find out the answer to, compared with some other quetions which cant even be found in search engines because they are too specific, and unless someone had taught them these obsurd facts, no-one else would know.
So i would like to know who these people evaluating the questions are, because, quite frankly they are morons, and deserve to be replaced
Pontszám: 5
much likedstekkos
Végzetúr poronty
1356 hozzászólás
Profil megtekintése
Re: The Quiz Evaluation Discussion/Debate
42. hozzászólás - 2010.04.04. 09:50:18 (Válasz S10MATT #41 hozzászólására.)
Once again i will post that being easy to google does not make a question good. Middle name of a character in a cartoon that might or might not be popular in parts of the world is not a good question. The first name, the number of kids in Homers family, that might pass as good since is general trivia on a famous cartoon. But middle name goes a bit too into details
Pontszám: 5
everyone's favoriteCursed Templar Barg
Végzetúr poronty
10 hozzászólás
Profil megtekintése
Re: The Quiz Evaluation Discussion/Debate
43. hozzászólás - 2010.04.04. 11:03:26
It is just disheartening to have to be quizzed on what is John Wayne's real name? When you follow the guidelines to the very word for word of what is required to only be rejected. I don't take the rejection as personal but I find the questioning very:

1. Subjectivity of the evaluator(s). How do we know that our question is really being properly evaluated?

2. The invisible evaluators. Our hand are tied so to speak as we 'flail in air' at 'invisible evaluators'. The one in particular makes me uneasy. The evaluators are very biased as they themselves are filled with their own ideas of what is a good or bad question.

3. How do we know that every question is really being looked at properly?

4. Why do we even have need of this in the game?

This game has been played in Hungary and you have brought this to another culture. I wonder if these same ideas have already been covered by others? There are too many questions that are unanswerable.

There are too many answers that leave gaps in logic and proper thinking processes. I am grasping for whatever I can as an avid gamer for any reason to continue on playing this game.

Right now I am 'rejecting' this game as a 'bad question or to vague'.

Rise up gamers and revolt !!

Down with tyranny

Seriously consider getting rid of the bonuses for 'accepted questions'. Take the subjective application out of the game.

Thanks for listening,

Templar
Pontszám: 5
averageViridel
Végzetúr poronty
796 hozzászólás
Profil megtekintése
Re: The Quiz Evaluation Discussion/Debate
44. hozzászólás - 2010.04.04. 18:25:15 (Válasz stekkos #42 hozzászólására.)
Idézet: stekkos - 2010.04.04. 09:50:18
Once again i will post that being easy to google does not make a question good. Middle name of a character in a cartoon that might or might not be popular in parts of the world is not a good question. The first name, the number of kids in Homers family, that might pass as good since is general trivia on a famous cartoon. But middle name goes a bit too into details


I would suggest that almost ANY question regarding The Simpsons would be more universally well known than 2/3rds of the Literature questions floating around, and more well known than the VAST majority of historical questions (especially those involving dates). Pop Culture is as valid as any other topic, and because of the "currentness" of it, the answers are VERY available online. You're right - just because it's on Google doesn't make it a good question. But NOT having it readily available on Google definitely makes it a BAD question - and there are a lot of those in the quiz.

That said, the answer to Homer J. Simpson was only said in one episode, which I would think makes it too trivial for this game.
Pontszám: 5
much likedLydonB
Végzetúr poronty
266 hozzászólás
Profil megtekintése
Re: The Quiz Evaluation Discussion/Debate
45. hozzászólás - 2010.04.04. 20:22:35
What is really needed is the clarification of the "general knowledge" issue.

Somehow, a clear line must be drawn; and the whole evaluation process needs to be more transparent.

At the moment, the "line" is extremely fuzzy and there's no transparency in the matter.


Perhaps we should move away from appointed evaluators and adopt a different system. Something like: a new question is put up for 100 times for random players; depending on the percentage of correct answers. If the absolute majority answer it correctly then whatever it is has to be considered "general knowledge" and correct. If it fails to meet the percentage criteria the player is informed and is given the chance to edit the question or re-submit another one.

As for the Goggle issue. Well, is there anyone here who never "cheated" via Goggle? If there is I'll say bullocks! It just ain't possible... not unless the "person" in question has access to the database or has some sort of computer instead of a brain.


Then there is the whole issue about the code. Well, the code will only be written once, IF the whole /ranting about the quiz is to be solved; the only way to accomplish it is to act on it. If the game owners simply "don't care" about the whole issue, then threads concerning the issue should be banned. Simply having threads upon threads of complains and suggestions/discussions on the issue will not somehow fix itself.
Pontszám: 7.50
much likedstekkos
Végzetúr poronty
1356 hozzászólás
Profil megtekintése
Re: The Quiz Evaluation Discussion/Debate
46. hozzászólás - 2010.04.05. 05:20:20 (Válasz LydonB #45 hozzászólására.)
Google is not cheating, its a helpfull tool. Just dont think that it is the basis for evaluation. If a question is good and you can google it easily, even better. But just because you can google it easy, doesnt make it good, so first the question is evaluated and then comes google.

As for the rest of your points i just want to add my support and say evaluators have to agree on a clear guidelne and POST IT! So after that no excuse if someone does not follow it. But now its not a mess, but quite close
Pontszám: 5
averageViridel
Végzetúr poronty
796 hozzászólás
Profil megtekintése
Re: The Quiz Evaluation Discussion/Debate
47. hozzászólás - 2010.04.06. 18:41:37
Duplicate post with the below - sorry.
A hozzászólást Viridel módosította 2010.04.06. 18:47:51-kor
Pontszám: 5
averageViridel
Végzetúr poronty
796 hozzászólás
Profil megtekintése
Re: The Quiz Evaluation Discussion/Debate
48. hozzászólás - 2010.04.06. 18:47:09
National Hockey League (NHL) - New Jersey ______? [Note to Mods - This question has been APPROVED by Game Bosses - DO NOT REJECT]?
Devils
Demons
Angels
Saints

The question is valued as "Bad, incorrect question"!
According to evaluators your question is ambiguous.


SERIOUSLY! WHAT THE FUCK!!!!!!!!!!! I PRE-cleared the question with the Game BOSSES!


"I checked your problem. I would have accepted this question, but it's always at the quiz admin's discretion...Btw, ambiguous is not "too special", but rather, when the answers are wrong (but I don't see it either)."

I am really about to lose my mind unless we get some TRANSPARENCY here.
Pontszám: 5
averageImmortalDragonFérfi
Végzetúr poronty
2 hozzászólás
Profil megtekintése
Re: The Quiz Evaluation Discussion/Debate
49. hozzászólás - 2010.04.06. 22:52:28
The answer to a question is wrong.
The question is :Albumin is a substance you can find in: ............ ?
The answer it gave is egg whites.

That however is Albumen not Albumin.
Pontszám: 5
much likedImLittleJon
Végzetúr poronty
395 hozzászólás
Profil megtekintése
Re: The Quiz Evaluation Discussion/Debate
50. hozzászólás - 2010.04.07. 00:06:36 (Válasz Viridel #48 hozzászólására.)
Idézet: Viridel - 2010.04.06. 18:47:09
National Hockey League (NHL) - New Jersey ______?


My guess is they wanted more of a sentence. Like "The team name of the National Hockey League (NHL) franchise located in New Jersey is _____?" I understood your question, but I had to look twice.
Pontszám: 5
averageViridel
Végzetúr poronty
796 hozzászólás
Profil megtekintése
Re: The Quiz Evaluation Discussion/Debate
51. hozzászólás - 2010.04.07. 02:26:55 (Válasz ImLittleJon #50 hozzászólására.)
Idézet: ImLittleJon - 2010.04.07. 00:06:36
Idézet: Viridel - 2010.04.06. 18:47:09
National Hockey League (NHL) - New Jersey ______?


My guess is they wanted more of a sentence. Like "The team name of the National Hockey League (NHL) franchise located in New Jersey is _____?" I understood your question, but I had to look twice.



It's not HOW the question is phrased - it's the fact that Euro Mods don't want to pass a North American sports question. Like I said above - the question was APPROVED by a GAME ADMIN! So what the hell is wrong with these Mods?
Pontszám: 5
averageViridel
Végzetúr poronty
796 hozzászólás
Profil megtekintése
Re: The Quiz Evaluation Discussion/Debate
52. hozzászólás - 2010.04.07. 16:42:10
So here we write and wait, and apparently still nobody gives a shit from the Game Admin side. Is anything going to happen, or is the quiz just going to get worse and worse - both in content and in frustration.


Oh, and I posted that same NHL Team question, and now there is some magical "Spelling Error" - just another excuse to block the question. There's no legitimate reason, it just gets blocked.

Does ANYONE have the ear of the Game Admins?
Pontszám: 5
much likedChaoticGeminiNő
Végzetúr poronty
583 hozzászólás
Profil megtekintése
Re: The Quiz Evaluation Discussion/Debate
53. hozzászólás - 2010.04.07. 16:51:18 (Válasz Viridel #52 hozzászólására.)
I got your question this time Viridel. There was more written than just your question. Be happy I gave you a spelling error with the opportunity to fix it.
Pontszám: 5
much likedImLittleJon
Végzetúr poronty
395 hozzászólás
Profil megtekintése
Re: The Quiz Evaluation Discussion/Debate
54. hozzászólás - 2010.04.07. 19:48:39
I've been thinking about this issue, and the way I see it, there are 4 issues:

1) How long it takes to get a question moderated. My first question took 6 days. My other ones have been about 3 days. If it usually took only 1 day to get a response, it would be less of a big deal to have to re-submit. Solution: get more moderators. Effort on the part of the game developers: extremely low.

2) Lack of communication between user and moderator. It's frustrating to have a question rejected and not understand why. Solution: there should be an additional field for the moderator to be able to type a (short) explanation, in addition to the broad category of spelling / too specific / etc. In addition, there should be an extra field on the page where you submit the question to add a message to the moderator that is not part of the question, such as urls of supporting documents. Now, this would make the moderation process take longer, so even more moderators would be needed to handle the additional load. Effort on the part of the game developers: minimal.

3) False positives: seeing quiz questions that were accepted which should have been rejected. Solution: on every quiz question, there should be a button to allow users to flag it as bad. Ideally, they could then pick from the same list of rejection categories as the moderators do and fill in the (short) explanation for why it should be rejected. The question should then be sent into the moderation queue, with the values pre-filled out. If the moderator agrees, they should be able to delete the question using the reporter's text with the click of a single button. If the moderator agrees but doesn't like the reporter's text, they can edit the text first and then delete the question. If the moderator disagrees, they should be able to reject the report by clicking a different button. In that situation, it might make sense to give the reporter 1 bad question on their tally. Effort on the part of the game developers: low to medium.

4) False negatives: getting your question rejected by a moderator, when other moderators would approve it. There is no way to fully avoid this issue other than doing away with judgment calls by moderators entirely. Which has been proposed a few times by different people lately. But those proposals would basically put the game back into the state where everyone was a moderator, which the game designers tried and didn't like. So I think those proposals have very little chance of being implemented. So within the current system, what could be done to alleviate the issue? Well, if the response time was quicker, there was some explanation as to why it got rejected, and there were fewer false positives, that would go a long way. Maybe that's all we can hope for.

Here's one other, more radical, idea (though I expect it also has very little chance of being implemented). Maybe the moderators could give a question a status of provisional, in addition to accepting or rejecting it. Provisional questions would then go into a state where everyone was a moderator, either with a yes/no control or simply by how many answered it correctly. Then, after a specified time period, it would either attain permanent status or be deleted.
Pontszám: 5
averageViridel
Végzetúr poronty
796 hozzászólás
Profil megtekintése
Re: The Quiz Evaluation Discussion/Debate
55. hozzászólás - 2010.04.08. 05:50:48 (Válasz ChaoticGemini #53 hozzászólására.)
Idézet: ChaoticGemini - 2010.04.07. 16:51:18
I got your question this time Viridel. There was more written than just your question. Be happy I gave you a spelling error with the opportunity to fix it.


And why couldn't you just take out what was in the quotes (Saying that it had been approved by the game bosses) and pass it??? Why would you make me go thru the resubmission process which has been a complete clusterfuck and is what has caused much of my frustration in the first place? Why create a problem instead of solving one?
Pontszám: 5
averageViridel
Végzetúr poronty
796 hozzászólás
Profil megtekintése
Re: The Quiz Evaluation Discussion/Debate
56. hozzászólás - 2010.04.08. 06:26:08 (Válasz ImLittleJon #54 hozzászólására.)
Idézet: ImLittleJon - 2010.04.07. 19:48:39
I've been thinking about this issue, and the way I see it, there are 4 issues:

1) How long it takes to get a question moderated. My first question took 6 days. My other ones have been about 3 days. If it usually took only 1 day to get a response, it would be less of a big deal to have to re-submit. Solution: get more moderators. Effort on the part of the game developers: extremely low.


Difference is quality vs. quantity. Granted, the quality is pretty poor right now - but do we want it getting worse?!?

2) Lack of communication between user and moderator. It's frustrating to have a question rejected and not understand why. Solution: there should be an additional field for the moderator to be able to type a (short) explanation, in addition to the broad category of spelling / too specific / etc. In addition, there should be an extra field on the page where you submit the question to add a message to the moderator that is not part of the question, such as urls of supporting documents. Now, this would make the moderation process take longer, so even more moderators would be needed to handle the additional load. Effort on the part of the game developers: minimal.


This is a great idea, and would actually lessen the workload on the Mods, as the research link is already there - as long as it's a reputable source.

3) False positives: seeing quiz questions that were accepted which should have been rejected. Solution: on every quiz question, there should be a button to allow users to flag it as bad. Ideally, they could then pick from the same list of rejection categories as the moderators do and fill in the (short) explanation for why it should be rejected. The question should then be sent into the moderation queue, with the values pre-filled out. If the moderator agrees, they should be able to delete the question using the reporter's text with the click of a single button. If the moderator agrees but doesn't like the reporter's text, they can edit the text first and then delete the question. If the moderator disagrees, they should be able to reject the report by clicking a different button. In that situation, it might make sense to give the reporter 1 bad question on their tally. Effort on the part of the game developers: low to medium.


Ideally good, but practically, a nightmare in waiting. Imagine if a guy flagged every question they didn't know? Repeat by 500? There is a fundamental difference between a difficult-but-fair question, and a question from out of left-field (a retarded question). The Mods often can't keep the two separate - would you really expect all the users to be able to draw that line? Hell, even I would flag some of those more obscure Lit questions and Hungry-centric ones, just out of spite. It would just create major issues in a game that is already struggling with fair geographic representation.

4) False negatives: getting your question rejected by a moderator, when other moderators would approve it. There is no way to fully avoid this issue other than doing away with judgment calls by moderators entirely. Which has been proposed a few times by different people lately. But those proposals would basically put the game back into the state where everyone was a moderator, which the game designers tried and didn't like. So I think those proposals have very little chance of being implemented. So within the current system, what could be done to alleviate the issue? Well, if the response time was quicker, there was some explanation as to why it got rejected, and there were fewer false positives, that would go a long way. Maybe that's all we can hope for.


This is the biggest singular problem with the quiz - zero accountability by the Mods. They can decline a question without cause or reason. They can approve a question without cause or reason. Yet there is no way to get more information as to why that decision happened.

Short of using a Multi-Mod system, the only fair way to do it is to give the question asker an opportunity to challenge a Mod decision - likely by writing their reason in a box on the "Rejection" frame, and sending it for re-analysis by a different Mod. Will this happen? Not likely. What this would do is allow tracking of "reversed rejections", and if a Mod falls below a threshold, they get cut. This game is in desperate need of some accountability!!!

Here's one other, more radical, idea (though I expect it also has very little chance of being implemented). Maybe the moderators could give a question a status of provisional, in addition to accepting or rejecting it. Provisional questions would then go into a state where everyone was a moderator, either with a yes/no control or simply by how many answered it correctly. Then, after a specified time period, it would either attain permanent status or be deleted.


That sounds like my proposal of all-questions-pass (Mod verifies correct spelling & answer - ensuring that an answer CAN be found) for 10 reviews, if 7+ of 10 get it wrong, it goes to a Mod for difficulty review. I'm still in favour of it.
A hozzászólást Viridel módosította 2010.04.08. 06:27:41-kor
Pontszám: 5
much likedChaoticGeminiNő
Végzetúr poronty
583 hozzászólás
Profil megtekintése
Re: The Quiz Evaluation Discussion/Debate
57. hozzászólás - 2010.04.08. 08:08:12 (Válasz Viridel #55 hozzászólására.)
Evaluators can't change the text. Only a moderator can.
Pontszám: 5
everyone's favoriteCursed Templar Barg
Végzetúr poronty
10 hozzászólás
Profil megtekintése
Deletion was inevitable
58. hozzászólás - 2010.04.18. 21:24:42




Deletion was inevitable. I guess this is my puny little protest. Keep making that money....


Just a 'bye' to my friends that I have made here. I hope you all have continued fun and success. The question thing is just not relating at all to the game.


I just can't make the connection of asking, answering, or making up questions as a major part of a game were your character wins or loses. Further I can't connect with the subjectiveness of how questions are accepted and rejected.



The boundaries described by the game are not compatible with how the questions are evaluated. Too much 'wiggle room'. I make up a question that has clarity only to be rejected, then I see a misspelled word in a question that is bogus, I see no reason to continue.



I am not one to protest and still play the game when nothing is being done to make the game fair for everyone to play. To say that the questions are only a part of the game is to only be fooling yourself.



I am just one person with one opinion and with one vote. I am certain there are others here that love the game and I wish you all well.



Templar
Pontszám: 7.50
averageViridel
Végzetúr poronty
796 hozzászólás
Profil megtekintése
Re: Deletion was inevitable
59. hozzászólás - 2010.04.19. 01:49:20 (Válasz Cursed Templar Barg #58 hozzászólására.)
I quit World 1 for this exact reason. The Admins do NOT care about the users - that much is obvious. I keep hoping that things will get better now that a year has passed, but that's surely too much to ask.
Pontszám: 5
averageViridel
Végzetúr poronty
796 hozzászólás
Profil megtekintése
Re: The Quiz Evaluation Discussion/Debate
60. hozzászólás - 2010.04.30. 22:02:54
So I just got a 72 hour Quiz Ban for posting the following question:


In order to have your question approved, you have to:?
a) Make the question or answer about Hungary
b) Write about boring literature
c) Don't write about sports, and especially North American sports
d) All of the above

Ok, yes. This was not a valid attempt at a legitimate question. But it underlines a point that I and MANY others are getting REALLY PISSED OFF at the double standards (and the complete mystery standards) in the Quiz system.
Pontszám: 5
Oldal: 1234567
Dobókocka - a sokoldalú kiegészítőAz Idő Kereke tovább forog! Megjelent a befejező trilógia utolsó része!