As for your idea about 40%, is not the percentage that will prove a good question because even if the question is fine, maybe a lot of people happened to answer it wrong because they were just bored to think or find the answer. Good idea though, get rid of the special group but make a rule that after 30 answers, the 25% rule applies. Then the question for sure is too hard if between 30 random persons not even 8 got it right.

Ok now my turn. I will post this even on suggestions in the hope that an admin or dev will see it.
So here goes nothing: Yeah i see allot of bad questions: some badly written, some with a wrong answer marked as good, some that don't make any sense, some that require special knowledge and the examples go on.

What i believe that this system needs?
I tried to find a good solution and this is the best i can come up with: A log where we can see the last let's say 20 question since 20 is the maximum number of Hunts we can do at once, and a reporting system. This way we can see the question we answered and if we find a wrong question we can report it and if it was marked as good by mistake it can be removed.

ChaoticGemini - I trust that you have the posts from Banned, Stekkos, Viridel, Ciphas and myself about the number of poor questions, questions that were banned and shouldn't have been banned, questions that weren't banned and should have been banned and each of us of a great understanding of how to improve the game. Is there any reason why we cannot become Evaluators?

I also have a suggestion to be considered regarding the soul-energy points. Why not grade the questions between 1-5. 1 is easy and 5 extremely hard to answer. The soul-energy is then phased between the severity of the question. ie 1-5 energy points for questions in bracket 1; 6-10 for questions in bracket 2 and so on til bracket 5 will offer you 21 -25 soul energy. The Evaluators can judge whether or not a uestion is easy to hard. Only a suggestion on improving the questions

Idézet: Hyannis - 2010.03.17. 20:40:29

While this might seem a good idea at the first looking at it and analyzing it makes it bad. Why?
Because this will make people write harder question hunting for more SE and will increase the difficulty of them. This will not only lead to allot of bad question, but let's take as an example a Quiz Olympics where you have 15 minutes to answer 50 questions. It might happen to a player to get let's say 30 HARD questions while another one might get 50 easy ones. To prevent this they must write a new routine that will have to check the difficulty and that will be extremely difficult.

Also remember that what is easy for one is not necessarily easy for another. NHL/NFL team names, for example, I would rate as a 1 or 2 - because the answer is extremely easy to find even if you don't know it. Most of those Literature questions are probably pretty easy - but since it's not an area of interest for me, I usually get them wrong.

This could create bigger problems than it solves because it creates a new friction point - people looking for easy questions and not getting ones in an area that they know about - meaning hard for them.

As for the auto-kick, I don't like 25% for the simple reason that a monkey can get 25% on 4-choice multiple choice. 40% is "better than chance", making it a viable number to send a question for review.

Lastly, as for us becoming moderators - that doesn't solve the problem, only dilutes it. There will still be Mods out there who kick questions that shouldn't be kicked... Which is why there needs to be a system change, not just more Mods.

First, I would like a few points to be made clear.

I and I am pretty sure Viridel and stekkos are not here, in this topic to try and become evaluators.
However, if it was asked of me to do so, I and I am sure others, would agree to the task.

BUT... this is not why we are here. We simply see flaws in a system that needs to be fixed. Period.

The first issue was, that the evaluatorse need to be players that are active in the game.
Now, we know that this is indeed the case.. as Chaotic has proven.. this issue is no longer present.

The second issue is... some of those players that are evaluating questions are making mistakes.
We are human, and this is an inevitable happenstance.

So, the main question with the second issue is: How is this handled?

Are they warned about the mistake and given a chance to not make any more?
Omitted from being an evaluater after several bad questions were approved regardless of warnings?
Or is nothing being done and allowing bad evaluators to continue passing bad questions?

Okay... now...Grammar.

Yes, this is a pet peeve for some... but, it shouldn't be the MAIN reason for complaing about quiz questions, perhaps we should introduce a new sub section for "Quiz grammar mistakes" which would include mispelled or misused words...
I admit... my grammar at times is not 100%...in fact, I used the wrong word in one of my questions myself. ("know" instead of what should have been "known")

The majority of us are just tired and upset when we see questions that are blatantly incorrect... and the fact that some types of questions are being allowed, while others that fit in the same category of triva; are not allowed.

Some questions are blatantly ambigiuous... becuase the were obviously poorly researched,or... are based on knowledge that might be out of date.
Afterall....it wasn't too long ago when we believed that lead and asbestos based products wasn't a health concern.
This knowledge has not been amended to "serious health threat"

Information changes.

So, again..to make this clear: this discussion is more about the flawed system when it concerns how the questions are evaluated, not a discussion of who wants to be an evaluator.

I think the majority of questions would end up for review if we had a 40% rule.

I kid you not, in desperation at my early questions having below 50% I asked what I assumed even a 5 year-old could answer. "Where do wild hippopotamus live?" The answers were listed by continent and for at least 2 weeks it was below 40%.

I have seen the posts by all of you, but I simply evaluate questions. I have no say in how things run or who becomes evaluators. I was asked if I was interested in the position and chose to accept it.

I do like Dakaan's idea for us to see the questions we were asked. I like a log of your last hunting session, but even if it listed the question above where it currently lists "correct" or "wrong answer" it would be an improvement. It would allow us to copy and paste the questions on the forum for errors. We wouldn't have the time to do it during the olympics, but during a hunt there is usually plenty of time.

I have had conflicting thoughts on evaluation since before I started doing it. Complaints pop-up that questions are not being approved fast enough, so I hate to slow approval down more. On the other hand, if every question required two evaluators to accept or reject a question we may have a better database. I'm not sure which would upset more players.

The only problem with the current evaluation system (time wise) is that you can't ask another question until the previous one has been decided on. If it went to a 2-to-pass / 2-to-fail / tie decided by a 3rd (the consistently "Best") reviewer - as long as this was communicated to the masses that it was to get/keep the garbage out of the system, AND it didn't restrict the asking of the next question - 99.5% of the people would be in favour.

I think it would be a huge improvement over the one person pass/fail.

In fact, it should be 1 Euro and one N.A. paired up to review...

In fact, this is starting to sound like the suggestion on Page 1 of this thread...

In fact, that's the suggestion I made like 8 months ago

In fact, I think I need a thesaurus.

In fact, I think "What is a thesaurus?" would be a good Quiz Question.

I'll be back; in fact.

Idézet: Viridel - 2010.03.18. 03:06:38


I can't ask it yet because my previous question has not been approved... Oh, the irony.

Actually.....
Not being allowed to create more than one question at a time is a good thing...

Why?
Can you imagine 100 players creating more than 1 question each at the same time?

That would bog the evaluators down even more...and quite possibly allowing them to, in haste, make more mistakes by approving incorrect/inacurate questions.

I would rather have 1 question take 5 days to be approved by solid research...

Then have 5 questions to be approved in 1 day with the chance that they might be inaccurate due to the fact the evaluators simply "breezed" by them.

Simply put:
Quality is better than quantity.

Sure, but then it takes 5 days to have a question get rejected, and the player has enough "credits" to write another 4 questions, and that pisses them (me) off even more!

There aren't THAT many active players on a server that the Mods should be overwhelmed. Instead of a question every 30 correct, make it 40 correct. And if people are as dumb as Gemeni says, that 40 Correct will take 10 days to hit

After reading all these i think the best solutionn would be to:


1. Discard the special group and keep only the percentage 25% rejection by the system (but i still feel it should count only after 20-30 people got a chance at the question.)

2. Questions that are silly, wrong or clearly no effort went into them still be kicked ofcourse.

3. If you get the evaluation äs incorrect (any reason) do not loose the chance to post, so no frustration due to lost se or question opportunity.

4. After a player gets a lot of question rejected, compared to good ones he post even with this system, remove his right to post for a while.

5. If he/she comes back with the same kind of silly, wrong or bad questions remove the right permanently.

6. Finally, evaluators should know each other and form a comitte to lay down the rules that ALL shall follow. Also the admin should monitor the complains and if they come a lot from the rejections due to the same evaluator, internal action sould be taken to conclude if he has strayed from the rules set.

Idézet: stekkos - 2010.03.18. 11:34:18

1) Half agree. I still don't like 25%, since I could answer a) to every question I don't know, and still average 25%

2) Of course.

3) No, there needs to be a punishment for lack of effort. If it's a simple spelling / grammar, then allow a re-write (remember, not everyone here is First Language English). But if it's a freakin' retarded question, or one with a blatantly wrong answer, then a punishment must be handed out, as it's wasting Mod time.

4) And this is the danger with the 1-reviewer system. If a Mod doesn't like sports, and kicks 5 of my questions in a row, I shouldn't be punished for that. If, in a multi-reviewer system, the questions are repeatedly wrong, then suspensions are appropriate.

5) Agreed, with the caveat that 4) is considered.

6) Agreed overall. I really think that better geographical representation is also needed... In a multi-reviewer setup.

National Hockey League (NHL) - New Jersey ______?
Devils
Demons
Angels
Saints

The question is valued as "Bad, incorrect question"!
According to evaluators your question is ambiguous.

Wonderful! They have taken out "Special Knowledge" as a refusal criteria... But it's then been replaced my "ambiguous". Ummm, who here knows what ambiguous means - it means multiple possible answers / interpretations. Last time I checked, a team name had an absolute, definite answer. So not only is this a continued example of geographical racism, a continued example of reviewers being shit at their jobs, but now they are just making up reasons to decline a question.

THIS IS WHY I DON'T SPEND MONEY TO SUPPORT THIS GAME!

Idézet: Rasputin - 2010.03.29. 14:26:57

This is atrocious, and complete proof that the Quiz evaluation system needs a significant overhaul.

I think it is time I decided to add my thoughts as well, at first I was angry like most people but as I was reading a question came to mind!

What happens when most of the questions have already been asked?

In other words all the questions defined in the rule, people have filled the database with the valid ones and are only left with invalid questions!

I know we are not close to that at the moment but I think that might want the be looked into because there are lots of people on doomlord and in the end everyone is going to run out of questions!

Also this my seem like a stupid question but, why were questions about religion allowed to be asked?

I am concerned for the problems that can arise with such a touchy subject!

In fact there have already been problems in the past about religion (WW2 for instance)!

I am just curious, and yes I don't believe in God or Jesus if you were to ask, but that doesn't mean I don't care about other peoples beliefs!

I think another issue that hasn't been discussed is the fact that we are not actually notified exactly what is wrong with our questions, which normally raises blood pressure even more!

If you could add another section to the questions marked incorrect to say 'incorrect spelling or incorrectly worded', it might help people to understand a bit better that their question isn't exactly bad, just incorrectly worded or has spelling mistakes!

But I have to agree that some of my questions haven't been the best either and I understand that I need to generalize my questions a bit more!

I will do my best from here, I just thought that might help matters a bit as well!

Idézet: champon7 - 2010.03.31. 00:20:03

I don't think this can ever really happen. Trivial Pursuit has approx. 20,000 question PER BOX (and there are at least a dozen versions in North America)




And I've said this same thing in the past - there is nothing more divisive than religion, and in a global game, philosophical differences are very dangerous. But the same thing happened when I mentioned it before as will happen now.

Nothing.




The quiz component is both the best and worst part of the game. It certainly has the potential to become "awesome", but the administration of it currently is rather atrocious.

I sent this straight to the guy running the game, and not surprisingly, have not heard back:

It should simply be that NO question should be kicked out for being "too specific". But after 10 answers, if 3 or fewer got it right (less than 40%), then it goes to Moderation to determine if it's a fair/reasonable question; and it should be auto-kicked if it's less than 40% accuracy after 50 answers.

The first moderation should only verify the accuracy of the answer and FIX any spelling & grammar errors. That way nobody can get mad (or wonder why) their question got kicked, and if it gets removed later, it's because the user-base of Doomlords said it was too difficult. And really, regardless of the Moderator opinion, the only people who matter are the users - the current system simply does not represent that.

Also, Quiz Olympics questions should only be taken from the pool of 50+ answers. There is nothing more frustrating than the game having wrong (or impossible to find) answers in the Quiz Olympics.

No SE would be given until the question has passed the 10-answer test (and it should be increased to 3x to compensate for the time), and we should not be blocked from posting another question while the previous one is under "peer-review".

Ultimately the "hunt quiz" will get more difficult as there WILL be more obscure questions... But ultimately, I don't think anyone will care as much about the several SE lost on the Hunt Bonus when it means that the Quiz Olympics don't have any debatable questions, and that nobody will have to worry about (and get mad when) their questions are rejected for no legitimate reason other than a Mod not liking it. There won't be any point in complaining about a question, as it'll get kicked after 10 answers if it is too difficult!


This would solve ALL complaints (except where the Mod doesn't verify the correct answer), ultimately lessen the workload for the Mods, and the users of the game can dictate when a question is too "specific".


Now, the guy(s) running the game have come right out and told me that they don't monitor the Forums, and they have "lesser admins" for that. This saddens me greatly, as how are the creators of the game supposed to know what us users are thinking if they don't review the Forum, and actually SEE the problems that exist???

As such, if you agree with the above concept, I would suggest you copy it, add your thoughts, and e-mail it to info@doomlord.net

Maybe if they get people e-mailing them directly, something productive may come of it, the Forum clearly isn't the answer for change.