Felhasználónév: Jelszó:
Oldal: 123456789
Szerző
Topic neve: Quiz: the aspects of evaluation
much likedImLittleJon
Végzetúr poronty
395 hozzászólás
Profil megtekintése
Re: Quiz: the aspects of evaluation
161. hozzászólás - 2010.08.21. 16:29:41 (Válasz stekkos #160 hozzászólására.)
Idézet: stekkos - 2010.08.21. 05:25:04
More mods would mean a lot of them will be sitting idle hoping to get a question evaluated by chance. We are currently 4, and most of the time there are only 3-4 questions waiting that did not get eveluated by someone else.


But in another part of her list, she said each question would have to be checked by 5 evaluators, so that would mean a lot more evaluations to do.
Pontszám: 5
averageViridel
Végzetúr poronty
796 hozzászólás
Profil megtekintése
Re: Quiz: the aspects of evaluation
162. hozzászólás - 2010.08.21. 18:08:38 (Válasz stekkos #160 hozzászólására.)
Idézet: stekkos - 2010.08.21. 05:25:04
More mods would mean a lot of them will be sitting idle hoping to get a question evaluated by chance. We are currently 4, and most of the time there are only 3-4 questions waiting that did not get eveluated by someone else.


But if a question got evaluated up to 5 times before being processed, then things will likely queue up again. Of course, if we don't need 100 Mods where 10 would do, then so be it.

Also, I guess I wasn't specific about this... You would "log in" to the Moderation Area (another box in the Misc area, for example) and evaluate from that queue... That way, there is no "chance" factor - you Mod when you want to Mod. If there's nothing there, then so be it.

Also i would love to see a space for comments, it would make my job a lot easier since i would not have to post every rejection in the forum. But about spelling i feel the player should know what the mistake was and correct it himself. That way will have to pay attention to grammar. If they know we will accept the question and fix the spell errors then why would they bother double checking before they post?


To say that people need to pay attention to grammar & spelling somewhat dismisses the fact that a lot of people are English second language. Beyond that, mistakes happen. In this post alone, you misspelled "eveluated" and "Separetely". Rejecting a question over something like this certainly seems unnecessary.

Separetely from hunt i also a nice way of avoiding rush decisions, i would like to hear the opinion of other evaluators on that. I will ask them personally if they do not post here
A hozzászólást Viridel módosította 2010.08.21. 18:13:17-kor
Pontszám: 5
averagefiranx
Végzetúr poronty
2 hozzászólás
Profil megtekintése
Re: Quiz: the aspects of evaluation
163. hozzászólás - 2010.11.04. 14:43:33
How long does question evaluation usually take?
Pontszám: 5
averageViridel
Végzetúr poronty
796 hozzászólás
Profil megtekintése
Re: Quiz: the aspects of evaluation
164. hozzászólás - 2010.11.04. 20:32:20 (Válasz firanx #163 hozzászólására.)
Idézet: firanx - 2010.11.04. 14:43:33
How long does question evaluation usually take?


Usually between 30 minutes and a week.
Pontszám: 5
much likedstekkos
Végzetúr poronty
1356 hozzászólás
Profil megtekintése
Re: Quiz: the aspects of evaluation
165. hozzászólás - 2010.11.05. 08:55:00 (Válasz Viridel #164 hozzászólására.)
come on Viridel, i think between 5min and a week is much more accurate
Pontszám: 5
much likedstekkos
Végzetúr poronty
1356 hozzászólás
Profil megtekintése
Re: Quiz: the aspects of evaluation
166. hozzászólás - 2010.11.05. 08:57:19
Anw, seriously now, if an evaluator is hunting when you post a question it might be evaluated right away. If not, then USUALLY someone will hunt and evaluate within 2-3 days. Ofcourse it can happen that all of us did not hunt for the same period of time which makes the questions pile up, leading to the week Viridel mentioned. I do think that is rare, am i wrong?
Pontszám: 5
averageViridel
Végzetúr poronty
796 hozzászólás
Profil megtekintése
Re: Quiz: the aspects of evaluation
167. hozzászólás - 2010.11.05. 18:17:59 (Válasz stekkos #166 hozzászólására.)
Idézet: stekkos - 2010.11.05. 08:57:19
Anw, seriously now, if an evaluator is hunting when you post a question it might be evaluated right away. If not, then USUALLY someone will hunt and evaluate within 2-3 days. Ofcourse it can happen that all of us did not hunt for the same period of time which makes the questions pile up, leading to the week Viridel mentioned. I do think that is rare, am i wrong?


1-3 days is certainly a realistic range, 80%+ of the time. I haven't written a question in a while (and my mental health thanks me), so I can't comment on the current turnaround
Pontszám: 5
averageDuke
Végzetúr poronty
3 hozzászólás
Profil megtekintése
Re: Quiz: the aspects of evaluation
168. hozzászólás - 2011.06.25. 23:34:20
i just wrote a question with correct answer and spelling and it was said, According to evaluators only a special group of people might know the answer.
I am confused to be honest, not complaining just confused and would like to make my point on the question. if there is another place to do this please let me know.
Thank you
Malick
Pontszám: 5
much likedstekkos
Végzetúr poronty
1356 hozzászólás
Profil megtekintése
Re: Quiz: the aspects of evaluation
169. hozzászólás - 2011.06.26. 08:41:30 (Válasz Duke #168 hozzászólására.)
Please post which was your question
Pontszám: 5
much likedSbartFérfi
Végzetúr poronty
879 hozzászólás
Profil megtekintése
Re: Quiz: the aspects of evaluation
170. hozzászólás - 2011.07.07. 12:04:22
According to the guidelines given in this post odd-one out questions should be filtered. Yet there is an ever increasing amount of these (odd-one out) questions in the database. Could we get a final answer to how odd-one out questions are suppose to be handled by the evaluators? Are they ok or not?
Pontszám: 5
much likedstekkos
Végzetúr poronty
1356 hozzászólás
Profil megtekintése
Re: Quiz: the aspects of evaluation
171. hozzászólás - 2011.07.07. 19:42:57 (Válasz Sbart #170 hozzászólására.)
Is not an easy thing to discuss with all evaluators and the final of from the admin. Please be patient
Pontszám: 5
much likedSbartFérfi
Végzetúr poronty
879 hozzászólás
Profil megtekintése
Re: Quiz: the aspects of evaluation
172. hozzászólás - 2011.10.17. 10:31:57 (Válasz stekkos #171 hozzászólására.)
Idézet: stekkos - 2011.07.07. 19:42:57
Is not an easy thing to discuss with all evaluators and the final of from the admin. Please be patient


I am patient, but that was three months ago. How is it, should odd one out questions be allowed or not?
I see more and more of the every day (and some of them are quite ridiculous).
Pontszám: 5
much likedVyasaFérfi
Global Moderátor
109 hozzászólás
Profil megtekintése
Re: Quiz: the aspects of evaluation
173. hozzászólás - 2011.10.17. 12:03:11 (Válasz Sbart #172 hozzászólására.)
Hello Sbart!

I asked it, Miklos' first post is authoritative.

"- Odd-one-out questions and questions about capitals are filtered. We have enough questions about capitals, thank you very much. Odd-one-out questions are always ambiguous, because any of the answers can be accepted from a certain perspective."

If you find odd-one-out questions (more or less ridiculous), you can post here or write to me in PM!
Pontszám: 5
much likedVyasaFérfi
Global Moderátor
109 hozzászólás
Profil megtekintése
Re: Quiz: the aspects of evaluation
174. hozzászólás - 2011.11.28. 13:12:26
Idézet: Johannes Buckbeak - 2011.11.26. 23:34:43
Idézet: Viridel - 2010.04.30. 22:02:33
But it underlines a point that I and MANY others are getting REALLY PISSED OFF at the double standards (and the complete mystery standards) in the Quiz system.

At this stage, I would rather see it out of the game completely than have it remain as it currently is managed.


Well I would not like to see the quiz system removed - this was what brought me into the game in the first place, and it is the unique feature of Doomlord among all the fantasy games out there. But I do agree, the question evaluation system is badly broken. Questions are rejected with effectively no explanation - "Bad - incorrect question" is entirely useless, and "Bad or ambiguous question" is no better - especially when the question is not ambiguous. It would make a huge difference if there was even one line of feedback from the evaluator to give some clue as to why the question has been rejected. I am also unhappy at the inconsistency (double standards as you put it) when you see many of the questions which do make it into the database. I'm sure we could all name several.

One final point which puzzles me. If there are 25,000 questions in the database, how come I keep getting the same 50? They clearly go in waves - you see a new question, get it several times a day for a week or two, and then it vanishes. What happens to all the old questions? Do they ever get used again? If not, what does it mean to say that there are 25,000 questions?



Feedback works with this automatism (it's faster than messaging in PM, it is the reason) but among the plans there is a Comment option where quiz admins can write the concrete problem with the question.
We have topics for bad questions and duplications, where you can post them (I see you've already use it).
Pontszám: 5
very popularJohannes Buckbeak
Végzetúr poronty
105 hozzászólás
Profil megtekintése
Re: Quiz: the aspects of evaluation
175. hozzászólás - 2011.11.28. 13:36:47
Yes, I have discovered this forum now and it is very useful, and I intend to be more active, for example in reporting bad questions and bad spelling in the appropriate places.

However that doesn't change my basic point, that the evaluation system could be much improved fairly simply. As a great example, I have now discovered that there is a blanket ban on the "none of the above" type of answer. The main reason it took me so long was that the evaluator was constantly telling me there was a spelling error, which there wasn't. I am sure he was only doing what he had been instructed, but the problem is with the system. Even if you don't want to give them some "free form" text, there ought to be a greater variety of more meaningful responses. If there is a blanket ban of the type I have mentioned, then you simply need a standard response which says "None of the above" is not permitted as an answer. That would have saved me a lot of time and stress. probably adding half a dozen such standard responses would hugely improve evaluation feedback, and consequently the enjoyment of the customers for this game.
Pontszám: 7.50
much likedDancerFérfi
Végzetúr poronty
200 hozzászólás
Profil megtekintése
Re: Quiz: the aspects of evaluation
176. hozzászólás - 2012.05.29. 05:29:40
Not sure about this one................

What is the most dangerous animal in Africa?

Lion
Hippopotamus
Mosquito
Forgotten this one,

The answer is flagged as Mosquito. Fair enough the mosquito is responsible for the most deaths, but it is not an animal.
And I think the question could be worded a lot more specifically.
**Modified to admit that a Mosquito is indeed an animal!!! too much brandy that night methinks!!!**
A hozzászólást Dancer módosította 2012.08.05. 11:21:59-kor
Pontszám: 5
averageMosquito665Férfi
Végzetúr poronty
116 hozzászólás
Profil megtekintése
Re: Quiz: the aspects of evaluation
177. hozzászólás - 2012.05.29. 05:41:27 (Válasz Dancer #176 hozzászólására.)
Idézet: Dancer - 2012.05.29. 05:29:40
but it is not an animal.


what is it then? a plant?
Pontszám: 5
averageborecdanFérfi
Végzetúr poronty
78 hozzászólás
Profil megtekintése
Re: Quiz: the aspects of evaluation
178. hozzászólás - 2012.05.29. 19:31:56
Even though I disagree mosquito isnt an animal, I have to agree with the fact that the question is bad.

Being responsible for the most deaths and being the most dangerous are two different things and I think this question is vague.
Wicked - W2 - diamond
Pontszám: 5
much likedChaoticGeminiNő
Végzetúr poronty
583 hozzászólás
Profil megtekintése
Re: Quiz: the aspects of evaluation
179. hozzászólás - 2012.05.31. 22:17:05
I agree too. The mosquito/animal question only had 56% correct answers while "What animal is responsible for the most human deaths worldwide?" has 90% correct. Clearly people understand the trivia, but miss that question. I just removed the question.
Pontszám: 5
very populardigitalisFérfi
Végzetúr tehetség
182 hozzászólás
Profil megtekintése
Re: Quiz: the aspects of evaluation
180. hozzászólás - 2013.01.26. 09:18:20
IMHO if there are slight grammar faults in a question, be it spelling or punctuation, as long as the question is understandable then it should stand. As has already been stated in posts above alot of players have English as their second language and as such I take my hat off to them!!
English is my first and only language so I guess I pick up on all grammatical errors pretty quick, but to return a question to a DL for a small error when the player is trying damned hard in a second or third language does seem unfair. These players have my admiration.
Pontszám: 5
Oldal: 123456789
HKK - Zén minikiegészítő: Zén Legendái: A sötétség gyermekeiAz Idő Kereke tovább forog! Megjelent a befejező trilógia utolsó része!