Username: Password:
Page: 1234567
Author
Name of topic: The Quiz Evaluation Discussion/Debate
popularBannedMale

73 post
View profile
The Quiz Evaluation Discussion/Debate
1. post - 2010.03.14. 09:15:30
I felt we were... overbearing the "My quiz question was rejected" section with this...so...

Let's continue this here... so that we can keep that open for others when they have a problem that needs to be helped....

As I stated before:
How many people of the world know what the Hungarian currency is?
Or what was the only thing that could inflict a wound upon Baldr of the Norse mythology?
and how about
What does Thalassa Mean In Greek?

All three of these questions .... are...well.... let's face it...

Questions that "only a special group of people might know the answer"...
UNLESS of course they used a search engine...

Just like Viridel's questions...which even I...an over the hill American... would probaly not be able to answer without using Google first.

The point I am trying to make is this.....

Just because the evaluators believe the world does not know the answer, does not mean that the answer cannot be found just as easily as ones that they feel are known.

This is the internet age, with information literally at our fingertips.

I see your point stekkos.... but do you not see ours?

One question took me three minutes to find the answer to...but I did find the correct answer.

If an evaluator cannot find the answer in five minutes using search engines...then I could understand denying the question to be allowed...

But to deny certain questions on the premise of the belief that a player cannot answer it, when you have never put the question to the test.

Well...that is like not selling a pork roast and throwing it away just because an Arabic family shops at your deli.


Furthermore...

Let's take the Quiz Olympics...

In order to complete it, you would have to spend no more than 18 seconds per question.

I assure you that anyone that has completed it, had to research half of those questions at some point prior to answering them...
Or...was in a clan that logged each Quiz Question and Answer.

Either way....they did not know the answer to ALL of them the first time they read the questions.
.
.
.

Okay...rebuttal?
This post was modified by Banned at 2010.03.14. 09:36:27
Score: 6
much likedstekkos

1356 post
View profile
Re: The Quiz Evaluation Discussion/Debate
2. post - 2010.03.14. 12:19:19
First let me congratulate your idea for this topic, it was time to be removed from the quiz area

As for the questions you list, Hungarian currency is not a well known fact but a general knowledge question.
The Baldr one falls in the debate if mythology of one area of the world should be accepted or not. If Greek gods and Egyptian believes can be acepted then also that question is ok. So the fact is, you either reject the whole mythology category or you accept the fact that some mythologies are less known but still part of a cultures history, therefore accepted.

The thalassa one is not a valid question, even if i do speak Greek. That is an evaluation mistake as far as i am concerned, but does not justify other questions of similar specific knowledge. It should be removed.

As i explained to Vidriel, the general idea is to ask things that are considered worldwide knowledge, even if you do not know the answer, when you get it it adds something to your facts. Think of it like a quiz show on tv. At any part of the world, the currency of a country, the author of a book, chemistry, physics, geography, etc might pop up as a valid trivia. Those types are the ones that should pass here also. Not all that pass should, i grand you that, but is no excuse to add more.

And to finish with the NFL debate, asking how many teams play in NFL should be ok. You learn a nice fact about a game that is local to USA. But going in furhter like the one we discuss, does not add anything to a person outside US. even if Google can help or not. The internet helps find answers but should not define which are the good questions
This post was modified by stekkos at 2010.03.14. 12:19:58
Score: 6
popularBannedMale

73 post
View profile
Re: The Quiz Evaluation Discussion/Debate
3. post - 2010.03.14. 13:41:29 (Reply to post #2 of stekkos.)
Actually.... I am surprised Viridel did not do this first... but I appreciate the compliment.

Ok...I can see your point.
Some facts...even though others of the same category (in this case, Sports) tend to be... too specific.

So, I agree...that a question such as "who lead the all time rushing ....blah blah in 1997?" or something to that effect is a tad...too much.

But, I still hold my opinion that the Evaluation System is flawed.

I am catching a few questions that are quite simply ... wrong and I can only hope that are omitted because of the 75% rule.

This tends to screw up the whole Quiz system as a whole.

I did poorly on the Quiz Olympics... and a few I know I answered correctly, but the people that created...and approved those questions..failed to do the fact research to ensure the reliability of the "correct answer".

Yet, I see many people have their questions rejected on spelling or grammar....but it's ok to spell Brazil with an "s", wolverine with only one "e" and so on....

Personally...I see a good trivia question as one that educates, yet challenges the players.

Yes, I think that the question "What goaltender holds the NHL record for Career Wins?" is a specific question and the answer could also change in a year..making it a poor one.

While "Prior to the 2010 Winter Olympics, which host country had never won a gold metal?" should be valid, and I think is a good question.

But..I also feel that fact checking using Wiki (a user submitted encyclopedia) is also a bad idea.
Because it is not always correct, and at times,can be very misleading.

We are allowed "to ask questions about Doomlord as well, but only ask questions that do not require the character to be high level and any player can answer"
But, I submitted a question taking the answer right from a page that even a level one could find...yet it was rejected on the basis of "According to evaluators only a special group of people might know the answer."

So, I might just need to rephrase it.

Like any new system....it has flaws. I suppose some get "all fired up" more than others when their questions are repeatedly denied...
Which is why the system needs to be tweaked because allowing random players to evaluate questions obviously is not working.

I hope...that a better system is revised soon, because some questions are getting ridiculous.
Score: 5
averageViridel

796 post
View profile
Re: The Quiz Evaluation Discussion/Debate
4. post - 2010.03.15. 00:15:36 (Reply to post #3 of Banned.)
I was borderline to quitting on this game once and for all... But let's see if something good can come out of behaviour I consider no less than borderline racist.

I've agreed with most of what you say, but to draw attention to one:
Yes, I think that the question "What goaltender holds the NHL record for Career Wins?" is a specific question and the answer could also change in a year..making it a poor one.

This is my question, and here's the reasoning why it is a safe question (at least as safe as asking who the career scoring leader in the NBA is, which IS in the quiz) - Martin Brodeur set the record last year... Now this page hasn't been updated really recently, but it makes my point:
http://en.wikipedia.org/wiki/List_of_NHL_statistical_leaders#Regular_season_wins

Brodeur has a 170 win lead over the second place active player, and 250 wins over the #3 active guy - almost double the 3rd active goaltender. And by the end of this year, it will likely be double the #3, and the #2 guy is now a backup, and won't break 450 wins. Meanwhile Brodeur will likely hit 600 wins this year, putting him 50 (8%) higher than the 2nd guy in NHL history, and 120 (20%) higher than guy #3 in NHL history. We are seeing a shattering of a record to a degree possibly unparalleled in any sport since Wilt Chamberlin.

It's like asking who has the Consecutive Games record in Baseball (Ripken) or Consecutive Games with a Hit (Gherig). These records CAN be broken - but it is phenomenally unlikely that the game will still be alive if/when that were to happen (and quite likely that it won't happen in our lifetimes). If we eliminate any questions that an answer could become obsolete, this turns the game into a history challenge - which would be boring and tedious.

Regarding Wiki - you're right, it's user input, which is inherently fallible. But if the questions are not debatable, and all you need is straight fact (not opinion) then Wiki is fine. Wiki's not going to say that Juventis plays in Argentina or that a Lemur is a reptile.


Just to set the record straight, I'm not pissed because MY questions get booted... I'm pissed because Amerocentric questions get booted while Eurocentric questions get approved - in spite of poor spelling / grammar / obscurity / or even being correct at all.
This post was modified by Viridel at 2010.03.15. 00:20:13
Score: 7.50
averageViridel

796 post
View profile
Re: The Quiz Evaluation Discussion/Debate
5. post - 2010.03.15. 00:22:26
I posted this several months ago, and I think it still is the right way to go:

Please create an international team of players to review these questions. 9 people to review every question... 2 North American, 2 European, 2 Asian, 1 each from S. America, Africa and Australia. If 7 to 9 people accept the question, it's added, if 0 thru 4 approve, then gone. If it's 5 or 6, then there can be QUICK discussion from both sides.

Right now, you have absurdly obscure questions that were accepted (and questions with wrong answers), yet you have also rejected questions based on criteria that you accepted for previous questions.

The Quiz portion of the game is one of the best parts of any online game... However the method under which the questions are evaluated and passed/failed is horribly poor and inaccurate. Therefore, I suggest moving to the group-system outlined above... Let an international pool of PLAYERS tell you which are good or bad questions.
Score: 5
popularBannedMale

73 post
View profile
Re: The Quiz Evaluation Discussion/Debate
6. post - 2010.03.15. 00:50:11 (Reply to post #5 of Viridel.)
Indeed..it must have been your post that I had seen when I posted this:

I had a "brainfart" from another section....
so I will just copy and post the relevant parts here as well; in case it gets drowned out there.

In reference to those that evaluate the quiz questions that are submitted by the players.
(This is not a "gripe and moan", though after reading it, I can see how some might think that it was.)

How about appointing select active players from W1 and W2 to a "Quiz Evaluation Committee"?

Hold a test for members who would like to be a part of this committee to take; which require them to investigate the answers.
If they pass and are still active a certain amount of time after taking this test.... they are allowed access to a database of the submitted questions to revue.

Inactivity of said players that have been appointed, or those that fail at doing the research once they have been appointed (aka: their JOB), are ejected from The Committee.

Also, have it so that they could not revue their own personal questions that they themselves submitted.

I think I may have seen this idea mentioned once before, but I do not believe anything came about becuase of it.


I think it is a valid suggestion...from both of us.

I have seen some Amerocentric questions in the quiz... but I agree concerning misspelling.

Actually I have a quiz question that I accidently used the wrong word myself....and it was approved. (sorry, ya'll)


I can see your point concerning statistics... and in a small way feel those types of questions should be allowed provided they are accurate.

And if 75% percent get it wrong...then so be it.


A larger part of me can understand why those statistical questions are revoked.
As for the others that have been approved, perhaps they were done so before things became more restricted?

Afterall.... math questions that are above basic skills... have also been approved, and now are rejected.
(seriously..I had to really think about what the cube sqaure root of 8 was in the quiz olympics.....and probably spent waaaay too much time doing so.)

But...I agree in your logic that eliminating questions that an answer could become obsolete is not a sound basis for rejection.

And I am sure that the evaluation process is being reviewed... especailly when so many wrong answers have been found, and the fact that a few of us actually care enough about it to question the process as a whole.
Score: 5
averageViridel

796 post
View profile
Re: The Quiz Evaluation Discussion/Debate
7. post - 2010.03.15. 01:45:32 (Reply to post #6 of Banned.)
And I am sure that the evaluation process is being reviewed... especailly when so many wrong answers have been found, and the fact that a few of us actually care enough about it to question the process as a whole.


You, sir, have far more faith than I. The only incentive for change is usually in the pocketbook. So until they are damaged there, I fear little will change.
Score: 5
popularBannedMale

73 post
View profile
Re: The Quiz Evaluation Discussion/Debate
8. post - 2010.03.15. 01:49:47 (Reply to post #7 of Viridel.)
Perhaps....

But.. considering it's the players that put the change into their pocketbooks...

It would be wise for any game developer to be open to suggestions concerning flaws within their product.
Even if it is considered a minigame within the game itself.
Score: 5
averageViridel

796 post
View profile
Re: The Quiz Evaluation Discussion/Debate
9. post - 2010.03.15. 03:16:13 (Reply to post #8 of Banned.)
Quote: Banned - 2010.03.15. 01:49:47
Perhaps....

But.. considering it's the players that put the change into their pocketbooks...

It would be wise for any game developer to be open to suggestions concerning flaws within their product.
Even if it is considered a minigame within the game itself.


Let me know when an Admin or Mod pops into this topic
Score: 5
much likedstekkos

1356 post
View profile
Re: The Quiz Evaluation Discussion/Debate
10. post - 2010.03.15. 20:39:44
I have been with this game almost from its start. I got really frustrated with the way they evaluated, stop posting questions for a while and i thought of quitting because the quiz was the reason i joined. Then i realised that you actually play a game the way it is made and enjoy it, or just stop since it is no fun. Games are for fun, if a game failes to deliver that TO ME PERSONALLY, then why am i still here? And i relised, even so i was enjoying this game. I adapted my questions to the way the evaluators think and i got 110 good, 14 bad (most bad were at first, when i got angry). I still posted suggestions, since that is a must if the game will evolve and be better. One of them being that the special group evaluation should be removed, similar to what a lot suggest now.

But since it is still there, getting angry about it doesnt solve anything. Keep posting suggestions, wrong qustions for correction, etc is constructive and it does help. A lot of questions running now in the quiz were corrected due to players reporting them, and many more removed as invalid after players pointed them out.

Bottom line:
The game has flaws and will keep having flaws like any other game
Suggestions are good, they help even if you thing they are ignored, admins do not inform of every detail they change especially about the quiz
Getting mad at a game you play voluntarily for fun is realy useless. If you like it as it is then play it and if it gets better great, if you cannot play it as it is but want to force it to change, even if you are 100% right, then what are you doing here?

So keep trying to improve but do not make this a debate site just for proving who is right. Its a game, enjoy it if you can, if you cannot stop playing
Score: 8
much likedCiphasMale

109 post
View profile
Re: The Quiz Evaluation Discussion/Debate
11. post - 2010.03.15. 21:12:12 (Reply to post #10 of stekkos.)
Great post Stekkos !!!
Score: 5
averageViridel

796 post
View profile
Re: The Quiz Evaluation Discussion/Debate
12. post - 2010.03.15. 23:43:23 (Reply to post #10 of stekkos.)
It is a great post, but it fails to address the inherent (lack of) equality in the game. The game itself IS fun - that's why I'm back here for World 2 after quitting World 1. But the quiz functionally makes up, at minimum, 10% of the game (the the "fun factor" of the quiz is FAR higher, of course). So you basically have to participate in it to be competitive in the game.

So when I have to answer crap about literature that I couldn't give two piles of horse manure about - I deal with it, because obviously that topic is important to someone. I don't look up the answer, I just guess, get it wrong, and possibly remember for next time.

But why should I have to answer stuff that is important to someone else that I don't care about at all - when someone else doesn't have to take the time or effort (or do what I do, and just guess) to answer what is important to me?

What makes someone's question about Stephen King more important than my question about the NHL? I'm Canadian - there is nothing more important to us than Hockey. I'm not saying you have to LIKE it - but if I have to answer your "crap", then you should have to answer mine.

As has been said many times, the ONLY questions that should get kicked out are the ones that are nonsensical, get complaints or are flat-out wrong. Everything else is important to someone, and their questions should be honored in that manner.
Score: 5
much likedstekkos

1356 post
View profile
Re: The Quiz Evaluation Discussion/Debate
13. post - 2010.03.16. 06:41:16 (Reply to post #12 of Viridel.)
It does adress the problem, if frustration goes away then its going to be fun if they reject your question or not. Is not a matter who has their interests included or not, i love star trek but i cant get one question to pass. I find it annoying, i complained about it, finally they agreed that at least star trek movie ones can pass. Even if they did not i dont feel that i have to include my interests for it to be fair, i just go another way and post on science, geography, history, etc that are more likely to pass

Again let me stress out i agree with letting all but the silliest questions pass. And ofcourse there is nothing wrong with NHL, i have personally seen quite a few in the quiz, and i would be happy to answer any question about it even if it is totally unknown where i live. After all, i learn something about a new subject and i like that Just go a different way, ask about the game for example, not the teams. Ask about intresting, unique rules, about famous records, world championship results etc. Even about teams, ask general stuff like which team was top for last year, but dont ask which was top in 1912. I cant say if they will pass for sure, but they will have a much higher chance. That way we all learn about that sport without you getting the bad evaluation and ruining your fun
Score: 5
irritatingHyannisMale

48 post
View profile
Re: The Quiz Evaluation Discussion/Debate
14. post - 2010.03.16. 13:56:33
Viridel has asked that I join this group and I am happy do so. I have read some of your comments and feel they are all valid points. Surely, all these questions are vetted before being released upon our poor minds to answer. One question that went through the system unchecked: What was the nationality of Albert Einstein? Please check my reasoning why this question should be removed under Quiz Errors page 34.

I find that the questions are poorly written and this leads to all kinds of confusion for those tasked to answer the question, and losing valuable soul-energy!! Another thing I find, is that a question could already be asked, but HOW do I know what question I can ask, if I do not have access to the database?

The other thing we need to be mindful of is the diversity of the people playing this game. I am not privy to the number of players, but if this is advertised on the WWW, then you can guarantee that quite a number of different nationalities play Doom Lord. You cannot expect a British person to answer a question on American History and an American person to answer a question on British History without being tolerant of their country. The WWW is a worthwhile tool to find answers to different countries of the World.

Questions should only be banned/removed if the question contains errors and/or the answer is wrong/more than one answer can be right.
Score: 5
much likedstekkos

1356 post
View profile
Re: The Quiz Evaluation Discussion/Debate
15. post - 2010.03.16. 20:53:35
I really hope the quiz administrators post their thoughts about it here, this whole discussion is good but i would like to hear the ones in charge after so many days of debate. I am sure they read the forum but its time they chip in to the dicussion
Score: 5
popularBannedMale

73 post
View profile
Re: The Quiz Evaluation Discussion/Debate
16. post - 2010.03.16. 23:27:56 (Reply to post #15 of stekkos.)
I agree... 100% on that stekkos.

We need to know something, even if it is to say "Your point is moot, but feel free to continue the debate."
Score: 5
much likedChaoticGeminiFemale

583 post
View profile
Re: The Quiz Evaluation Discussion/Debate
17. post - 2010.03.17. 05:58:13
Since no one else is speaking up, I will. I am only one of several evaluators, but here are my opinions. . .

I get just as frustrated at see how terrible some of the questions are that people try to submit. They affect our game too. There are some categories I am just as tired of answering as you all are, but those are the questions submitted. We have all noticed plenty of bad questions in the database. That does not mean we need more and I am sure the other evaluators are working just as hard to see that we have a better database of questions.

Personally, I try to avoid the 'special knowledge' evaluation unless it is something ridiculous or impossible to find an answer to. I feel the 25% rule will filter out the "special knowledge" the same way it filters bad questions.

Spelling and grammar are the biggest issues I see in submitted questions. There are many that cannot write a basic sentence. To address a spelling complaint I have seen: I know many of the differences between english spellings and when in doubt, I look them up to see if that is correct by Brit standards.

I see far more European-based questions than American questions submitted by players. It has nothing to do with bias. Also, remember when writing american questions that not everyone will be from America. I recently rejected a question that asked how many states touch the Gulf of Mexico. It didn't say US states, so the answers were incorrect because Mexican states also touch the Gulf.

I happened to be on the phone with a friend while he was answering a couple hunt questions and when he finished, he started to complain about a cricket record question. He was tired of all the sports that are not popular in the US. I told him I actually let it in the system. I don't know the first thing about cricket, but when I researched the question, the answer was easy to find. The record seemed like something impressive that would not easily be broken. (It was a record that appeared as important as the NFL player with the most touchdowns in a season.) It was spelled correct and written well. No one can be an expert in everything or know what other players will find easy or interesting.

As I said at the beginning, I am only one evaluator and I am fairly new to the process. I see some of the "my question was rejected" posts and wonder myself what in the world the evaluator was thinking. Evaluators are not perfect. I have had to message an administrator to correct a couple of questions that I let in the system. Since you all care enough to complain, you can help the game by posting the errors that slip by. They do get corrected by an administrator.

Just to address a few other complaints. . . We do not evaluate our own questions. The database is the same for both worlds. Since you don't know what is in the database, when you get that evaluation you do not loose the right to ask another question.

If I missed any of your thoughts or comments, let me know. Just keep in mind, I can only speak for my personal opinion, not the thoughts of the other evaluators, admins or developers.
Score: 5
averageViridel

796 post
View profile
Re: The Quiz Evaluation Discussion/Debate
18. post - 2010.03.17. 06:36:19 (Reply to post #17 of ChaoticGemini.)
Thank you Gemeni for an excellent post. You are obviously one of "The Good Ones" - one of those who see ignorance as an opportunity to learn, rather than an opportunity to refuse.

I was thinking about this while I was driving today, and what would be tremendously fair and easy to implement is a 40% rule. ALL properly worded and correctly answered questions get approved - completely remove the "Special Knowledge" refusal. Instead have a 40% review. If the accuracy drops below 40% after 10 answers, it gets re-evaluated in a special category - is this question too difficult / obscure. 25% is merely chance with 4 questions, so luck alone should keep the most foolish question alive - but 40% seems like a fair standard of expectation. This would get rid of those horrible Dragonlance questions (amongst others), or questions that are TOO region specific, while allowing everything else that is fair and reasonable.

By the same token, a player who cannot achieve 40% is a liability to the quiz system, as they are either just guessing, intentionally giving wrong answers, or are just dumbasses. These players could/should have their Quiz rights suspended for 2 days, then a week, then permanently.

Lastly, and I know this is a touchy subject for many - and that's the exact reason for the suggestion - get rid of any question(s) dealing in religion. People take it extremely seriously, and there is nothing that damages harmony like ideological differences.
Score: 5
popularBannedMale

73 post
View profile
Re: The Quiz Evaluation Discussion/Debate
19. post - 2010.03.17. 07:49:38 (Reply to post #17 of ChaoticGemini.)
Ok....ok. Fair.
And we do indeed appreciate that someone that actually knows what is going on replied.

So, if I may ask... do you, ChaoticGemini; know what happens if and when an evaluator does his or her job poorly?
As in, allowing obviously bad questions that are wrong or constantly does not do the simple research needed to ensure that the answer is valid with a correct answer that was given...

Do they just get a slap on the hand.... or are they no longer allowed to be an evaluator?
This post was modified by Banned at 2010.03.17. 07:51:25
Score: 5
much likedChaoticGeminiFemale

583 post
View profile
Re: The Quiz Evaluation Discussion/Debate
20. post - 2010.03.17. 08:51:06 (Reply to post #19 of Banned.)
Quote: Banned - 2010.03.17. 07:49:38

So, if I may ask... do you, ChaoticGemini; know what happens if and when an evaluator does his or her job poorly?

As in, allowing obviously bad questions that are wrong or constantly does not do the simple research needed to ensure that the answer is valid with a correct answer that was given...

Do they just get a slap on the hand.... or are they no longer allowed to be an evaluator?


Those are good questions. I do not know. That would be something an admin would have to answer.
Score: 5
Page: 1234567