Username: Password:
Page: 123456789
Author
Name of topic: Quiz: the aspects of evaluation
much likedImLittleJon

395 post
View profile
Re: Quiz: the aspects of evaluation
161. post - 2010.08.21. 18:29:41 (Reply to post #160 of stekkos.)
Quote: stekkos - 2010.08.21. 07:25:04
More mods would mean a lot of them will be sitting idle hoping to get a question evaluated by chance. We are currently 4, and most of the time there are only 3-4 questions waiting that did not get eveluated by someone else.


But in another part of her list, she said each question would have to be checked by 5 evaluators, so that would mean a lot more evaluations to do.
Score: 5
averageViridel

796 post
View profile
Re: Quiz: the aspects of evaluation
162. post - 2010.08.21. 20:08:38 (Reply to post #160 of stekkos.)
Quote: stekkos - 2010.08.21. 07:25:04
More mods would mean a lot of them will be sitting idle hoping to get a question evaluated by chance. We are currently 4, and most of the time there are only 3-4 questions waiting that did not get eveluated by someone else.


But if a question got evaluated up to 5 times before being processed, then things will likely queue up again. Of course, if we don't need 100 Mods where 10 would do, then so be it.

Also, I guess I wasn't specific about this... You would "log in" to the Moderation Area (another box in the Misc area, for example) and evaluate from that queue... That way, there is no "chance" factor - you Mod when you want to Mod. If there's nothing there, then so be it.

Also i would love to see a space for comments, it would make my job a lot easier since i would not have to post every rejection in the forum. But about spelling i feel the player should know what the mistake was and correct it himself. That way will have to pay attention to grammar. If they know we will accept the question and fix the spell errors then why would they bother double checking before they post?


To say that people need to pay attention to grammar & spelling somewhat dismisses the fact that a lot of people are English second language. Beyond that, mistakes happen. In this post alone, you misspelled "eveluated" and "Separetely". Rejecting a question over something like this certainly seems unnecessary.

Separetely from hunt i also a nice way of avoiding rush decisions, i would like to hear the opinion of other evaluators on that. I will ask them personally if they do not post here
This post was modified by Viridel at 2010.08.21. 20:13:17
Score: 5
averagefiranx

2 post
View profile
Re: Quiz: the aspects of evaluation
163. post - 2010.11.04. 15:43:33
How long does question evaluation usually take?
Score: 5
averageViridel

796 post
View profile
Re: Quiz: the aspects of evaluation
164. post - 2010.11.04. 21:32:20 (Reply to post #163 of firanx.)
Quote: firanx - 2010.11.04. 15:43:33
How long does question evaluation usually take?


Usually between 30 minutes and a week.
Score: 5
much likedstekkos

1356 post
View profile
Re: Quiz: the aspects of evaluation
165. post - 2010.11.05. 09:55:00 (Reply to post #164 of Viridel.)
come on Viridel, i think between 5min and a week is much more accurate
Score: 5
much likedstekkos

1356 post
View profile
Re: Quiz: the aspects of evaluation
166. post - 2010.11.05. 09:57:19
Anw, seriously now, if an evaluator is hunting when you post a question it might be evaluated right away. If not, then USUALLY someone will hunt and evaluate within 2-3 days. Ofcourse it can happen that all of us did not hunt for the same period of time which makes the questions pile up, leading to the week Viridel mentioned. I do think that is rare, am i wrong?
Score: 5
averageViridel

796 post
View profile
Re: Quiz: the aspects of evaluation
167. post - 2010.11.05. 19:17:59 (Reply to post #166 of stekkos.)
Quote: stekkos - 2010.11.05. 09:57:19
Anw, seriously now, if an evaluator is hunting when you post a question it might be evaluated right away. If not, then USUALLY someone will hunt and evaluate within 2-3 days. Ofcourse it can happen that all of us did not hunt for the same period of time which makes the questions pile up, leading to the week Viridel mentioned. I do think that is rare, am i wrong?


1-3 days is certainly a realistic range, 80%+ of the time. I haven't written a question in a while (and my mental health thanks me), so I can't comment on the current turnaround
Score: 5
averageDuke

3 post
View profile
Re: Quiz: the aspects of evaluation
168. post - 2011.06.26. 01:34:20
i just wrote a question with correct answer and spelling and it was said, According to evaluators only a special group of people might know the answer.
I am confused to be honest, not complaining just confused and would like to make my point on the question. if there is another place to do this please let me know.
Thank you
Malick
Score: 5
much likedstekkos

1356 post
View profile
Re: Quiz: the aspects of evaluation
169. post - 2011.06.26. 10:41:30 (Reply to post #168 of Duke.)
Please post which was your question
Score: 5
popularSbartMale

871 post
View profile
Re: Quiz: the aspects of evaluation
170. post - 2011.07.07. 14:04:22
According to the guidelines given in this post odd-one out questions should be filtered. Yet there is an ever increasing amount of these (odd-one out) questions in the database. Could we get a final answer to how odd-one out questions are suppose to be handled by the evaluators? Are they ok or not?
Score: 5
much likedstekkos

1356 post
View profile
Re: Quiz: the aspects of evaluation
171. post - 2011.07.07. 21:42:57 (Reply to post #170 of Sbart.)
Is not an easy thing to discuss with all evaluators and the final of from the admin. Please be patient
Score: 5
popularSbartMale

871 post
View profile
Re: Quiz: the aspects of evaluation
172. post - 2011.10.17. 12:31:57 (Reply to post #171 of stekkos.)
Quote: stekkos - 2011.07.07. 21:42:57
Is not an easy thing to discuss with all evaluators and the final of from the admin. Please be patient


I am patient, but that was three months ago. How is it, should odd one out questions be allowed or not?
I see more and more of the every day (and some of them are quite ridiculous).
Score: 5
much likedVyasaMale
Global Moderator
109 post
View profile
Re: Quiz: the aspects of evaluation
173. post - 2011.10.17. 14:03:11 (Reply to post #172 of Sbart.)
Hello Sbart!

I asked it, Miklos' first post is authoritative.

"- Odd-one-out questions and questions about capitals are filtered. We have enough questions about capitals, thank you very much. Odd-one-out questions are always ambiguous, because any of the answers can be accepted from a certain perspective."

If you find odd-one-out questions (more or less ridiculous), you can post here or write to me in PM!
Score: 5
much likedVyasaMale
Global Moderator
109 post
View profile
Re: Quiz: the aspects of evaluation
174. post - 2011.11.28. 14:12:26
Quote: Johannes Buckbeak - 2011.11.27. 00:34:43
Quote: Viridel - 2010.05.01. 00:02:33
But it underlines a point that I and MANY others are getting REALLY PISSED OFF at the double standards (and the complete mystery standards) in the Quiz system.

At this stage, I would rather see it out of the game completely than have it remain as it currently is managed.


Well I would not like to see the quiz system removed - this was what brought me into the game in the first place, and it is the unique feature of Doomlord among all the fantasy games out there. But I do agree, the question evaluation system is badly broken. Questions are rejected with effectively no explanation - "Bad - incorrect question" is entirely useless, and "Bad or ambiguous question" is no better - especially when the question is not ambiguous. It would make a huge difference if there was even one line of feedback from the evaluator to give some clue as to why the question has been rejected. I am also unhappy at the inconsistency (double standards as you put it) when you see many of the questions which do make it into the database. I'm sure we could all name several.

One final point which puzzles me. If there are 25,000 questions in the database, how come I keep getting the same 50? They clearly go in waves - you see a new question, get it several times a day for a week or two, and then it vanishes. What happens to all the old questions? Do they ever get used again? If not, what does it mean to say that there are 25,000 questions?



Feedback works with this automatism (it's faster than messaging in PM, it is the reason) but among the plans there is a Comment option where quiz admins can write the concrete problem with the question.
We have topics for bad questions and duplications, where you can post them (I see you've already use it).
Score: 5
very popularJohannes Buckbeak

105 post
View profile
Re: Quiz: the aspects of evaluation
175. post - 2011.11.28. 14:36:47
Yes, I have discovered this forum now and it is very useful, and I intend to be more active, for example in reporting bad questions and bad spelling in the appropriate places.

However that doesn't change my basic point, that the evaluation system could be much improved fairly simply. As a great example, I have now discovered that there is a blanket ban on the "none of the above" type of answer. The main reason it took me so long was that the evaluator was constantly telling me there was a spelling error, which there wasn't. I am sure he was only doing what he had been instructed, but the problem is with the system. Even if you don't want to give them some "free form" text, there ought to be a greater variety of more meaningful responses. If there is a blanket ban of the type I have mentioned, then you simply need a standard response which says "None of the above" is not permitted as an answer. That would have saved me a lot of time and stress. probably adding half a dozen such standard responses would hugely improve evaluation feedback, and consequently the enjoyment of the customers for this game.
Score: 7.50
likedDancerMale

200 post
View profile
Re: Quiz: the aspects of evaluation
176. post - 2012.05.29. 07:29:40
Not sure about this one................

What is the most dangerous animal in Africa?

Lion
Hippopotamus
Mosquito
Forgotten this one,

The answer is flagged as Mosquito. Fair enough the mosquito is responsible for the most deaths, but it is not an animal.
And I think the question could be worded a lot more specifically.
**Modified to admit that a Mosquito is indeed an animal!!! too much brandy that night methinks!!!**
This post was modified by Dancer at 2012.08.05. 13:21:59
Score: 5
averageMosquito665Male

116 post
View profile
Re: Quiz: the aspects of evaluation
177. post - 2012.05.29. 07:41:27 (Reply to post #176 of Dancer.)
Quote: Dancer - 2012.05.29. 07:29:40
but it is not an animal.


what is it then? a plant?
Score: 5
averageborecdanMale

78 post
View profile
Re: Quiz: the aspects of evaluation
178. post - 2012.05.29. 21:31:56
Even though I disagree mosquito isnt an animal, I have to agree with the fact that the question is bad.

Being responsible for the most deaths and being the most dangerous are two different things and I think this question is vague.
Wicked - W2 - diamond
Score: 5
much likedChaoticGeminiFemale

583 post
View profile
Re: Quiz: the aspects of evaluation
179. post - 2012.06.01. 00:17:05
I agree too. The mosquito/animal question only had 56% correct answers while "What animal is responsible for the most human deaths worldwide?" has 90% correct. Clearly people understand the trivia, but miss that question. I just removed the question.
Score: 5
very populardigitalisMale

175 post
View profile
Re: Quiz: the aspects of evaluation
180. post - 2013.01.26. 10:18:20
IMHO if there are slight grammar faults in a question, be it spelling or punctuation, as long as the question is understandable then it should stand. As has already been stated in posts above alot of players have English as their second language and as such I take my hat off to them!!
English is my first and only language so I guess I pick up on all grammatical errors pretty quick, but to return a question to a DL for a small error when the player is trying damned hard in a second or third language does seem unfair. These players have my admiration.
Score: 5
Page: 123456789