Quiz: the aspects of evaluation
1. hozzászólás - 2009.07.15. 21:43:18
1. hozzászólás - 2009.07.15. 21:43:18
Quiz: the aspects of evaluation
To our greatest pleasure the quiz has grown into a game-in-a-game and has at least as many questions as the framework game. Here you can find information about how the questions are evaluated.
The primary goal of the quiz is to ask questions that can be answered by an educated, intelligent person. Almost everyone. You don’t need specialized knowledge. Of course, one can say you can google most answers in a minute but we don’t really want questions that can only be answered by using a search engine. Some of the questions should make you think, make you rule out answers you that you will have an idea which answer to choose.
The most “convenient” general knowledge topics:
- Elementary school general knowledge questions. It might sound silly, but everybody learned about these, so no one can say they are specialized
- General knowledge, outside school questions, such as, films, famous people, rock bands, buildings etc.
- Current issues. These are a bit more complicated as questions become out-of-date in a few years, but there are still a lot of current topics worth asking a question about.
- Mind twisters. Even though not general knowledge questions, these still qualify as they do not require knowledge but simple brainwork. Please don’t write questions that require a calculator or equations.
- Linguistic questions. Spelling.
Bad questions:
First of all, it is important to highlight that even though a question might be accepted by the players it can still be disqualified by the admins later. There may be several reasons for this:
- Questions with a lot of spelling mistakes are rejected at once. It is simply impossible to correct each and every answer and we cannot run a spellchecker on the server for the time being. If you are unsure, please use a word processor to check the spelling. And please use the question mark at the end of the question.
-Questions that require a very special expertise for which an average person cannot know the answer. Many make the mistake of asking such questions. Don’t ask questions about games, hobbies, comic books, professions that can be only answered by someone who has years of experience in them.
-Lexical knowledge. Some questions about dates are acceptable, but only those that belong to the first category. Everyone should know the dates of the World Wars but not when a celebrity bought her first bicycle. What’s more, if you ask for a date, it is much more civilized to leave a bigger gap between the dates, than having 1 day between the four dates. In my personal opinion, the most acceptable method is to list four events and ask which one happened first or last.
- Extremely difficult questions are not good either. If you ask a question for which an average person might know the answer and you give four almost identical choices he will most certainly choose the wrong one. Please note that if 75% or more of the answers are incorrect the system will automatically disqualify the question. The purpose of this method is also to filter the correct questions where four good alternatives are listed but the person who asked the question selected the incorrect answer. If you ask a question that is already in our database, the question will be evaluated as wrong, but you will have the option to replace it with another without any penalty. But anyway, try to avoid asking trivial questions, e.g. When did WWI start?
- Odd-one-out questions and questions about capitals are filtered. We have enough questions about capitals, thank you very much. Odd-one-out questions are always ambiguous, because any of the answers can be accepted from a certain perspective. Getting back to filtering, if your question is rejected, please don’t submit it again using spelling mistakes, because it will be rejected as well. Please accept that the system will not accept it. And please also note that any questions containing vulgar/offensive words are also rejected.
- Maths. How much is (1+5)*3? Which is more, (4+3)*2 or (17/2-1)? Questions that require logic are OK, but trivial questions and equations that require a calculator are not.
- Foreign languages. You can ask what a certain famous thing was named after but you should not ask for plain translation of different words.
Questions to be deleted:
Simply put, when the player asks a question with the intention of submitting a correct one but he does not manage to, the question is evaluated as “wrong”. The “to be deleted” tag is given to those question which are incorrect on purpose. Examples are trivial questions, violating the netiquette or those that cannot be answered: What colour is the sky? Which Albanian folk band has not released an album? What colour was Amy Whitehouse’s hair in 2006? and so on.
Of course you can ask questions about Doomlord as well, but only ask questions that do not require the character to be high level and any player can answer, as a level 2 might also get the question: What skill does learning level 14 give? or What is the protection of the Nerubian robe? and won’t be able to answer.
Some examples for good and bad questions (please don’t submit them!)
‘How many members did the Beatles have?’
Although it is a bit trivial, still a good question, because everyone should know the band.
‘Which was the third album of the Australian Pigmies?
No one has ever heard of them, much less of their albums. Bad question.
Too special questions:
‘What is the width of the standard rails?‘
‘Who is the brother of the cartoon hero Glacius?’
‘What is the average wingspan of the Eastern mating mantis?’
‘Who was the coach of Manchester United in 2004?’
‘What is the name of the female protagonist in the Harry Potter books?’
Almost everyone knows the books, so you can ask such a question.
‘What is the title of the fourth book of the Harry Potter series?’
‘The goblet of fire, the qublet of fire, The calice of fire’
The difficulty is OK, but it is based on visual deception (q an g being similar) and contains a spelling mistake, calice instead of chalice. Bad question.
‘How old is the oldest woman on earth?’
There might be up-to-date information that becomes invalid in a month, should she die or become a year older. Try to avoid questions that might become out-of-date.
Of course we don’t expect anyone to submit correct questions all the time - I have also written questions that I would reject today. If 50% of your questions are rejected you get a warning and if this ration persists, you will lose the chance to ask questions. Also you can get the evaluation ‘to be deleted’ maximum three times. After the third occasion you will lose the opportunity to submit further questions permanently.
This is not a penalty, the SE reward is not significant. There are players who simply are unable to ask correct questions. That is all OK, they can still enjoy the other parts of the game.
Evaluators are also evaluated both automatically and manually. The aim is to let those players evaluate questions who give a realistic assessment. There is no penalty for “bad evaluation” you simply won’t be asked to evaluate further questions, however some system for a reward is on the planning table good assessors.
Note that now you have to option to choose the reason why the question was rejected. This way the player who asked the question will have some idea about what he had done wrong.
To our greatest pleasure the quiz has grown into a game-in-a-game and has at least as many questions as the framework game. Here you can find information about how the questions are evaluated.
The primary goal of the quiz is to ask questions that can be answered by an educated, intelligent person. Almost everyone. You don’t need specialized knowledge. Of course, one can say you can google most answers in a minute but we don’t really want questions that can only be answered by using a search engine. Some of the questions should make you think, make you rule out answers you that you will have an idea which answer to choose.
The most “convenient” general knowledge topics:
- Elementary school general knowledge questions. It might sound silly, but everybody learned about these, so no one can say they are specialized
- General knowledge, outside school questions, such as, films, famous people, rock bands, buildings etc.
- Current issues. These are a bit more complicated as questions become out-of-date in a few years, but there are still a lot of current topics worth asking a question about.
- Mind twisters. Even though not general knowledge questions, these still qualify as they do not require knowledge but simple brainwork. Please don’t write questions that require a calculator or equations.
- Linguistic questions. Spelling.
Bad questions:
First of all, it is important to highlight that even though a question might be accepted by the players it can still be disqualified by the admins later. There may be several reasons for this:
- Questions with a lot of spelling mistakes are rejected at once. It is simply impossible to correct each and every answer and we cannot run a spellchecker on the server for the time being. If you are unsure, please use a word processor to check the spelling. And please use the question mark at the end of the question.
-Questions that require a very special expertise for which an average person cannot know the answer. Many make the mistake of asking such questions. Don’t ask questions about games, hobbies, comic books, professions that can be only answered by someone who has years of experience in them.
-Lexical knowledge. Some questions about dates are acceptable, but only those that belong to the first category. Everyone should know the dates of the World Wars but not when a celebrity bought her first bicycle. What’s more, if you ask for a date, it is much more civilized to leave a bigger gap between the dates, than having 1 day between the four dates. In my personal opinion, the most acceptable method is to list four events and ask which one happened first or last.
- Extremely difficult questions are not good either. If you ask a question for which an average person might know the answer and you give four almost identical choices he will most certainly choose the wrong one. Please note that if 75% or more of the answers are incorrect the system will automatically disqualify the question. The purpose of this method is also to filter the correct questions where four good alternatives are listed but the person who asked the question selected the incorrect answer. If you ask a question that is already in our database, the question will be evaluated as wrong, but you will have the option to replace it with another without any penalty. But anyway, try to avoid asking trivial questions, e.g. When did WWI start?
- Odd-one-out questions and questions about capitals are filtered. We have enough questions about capitals, thank you very much. Odd-one-out questions are always ambiguous, because any of the answers can be accepted from a certain perspective. Getting back to filtering, if your question is rejected, please don’t submit it again using spelling mistakes, because it will be rejected as well. Please accept that the system will not accept it. And please also note that any questions containing vulgar/offensive words are also rejected.
- Maths. How much is (1+5)*3? Which is more, (4+3)*2 or (17/2-1)? Questions that require logic are OK, but trivial questions and equations that require a calculator are not.
- Foreign languages. You can ask what a certain famous thing was named after but you should not ask for plain translation of different words.
Questions to be deleted:
Simply put, when the player asks a question with the intention of submitting a correct one but he does not manage to, the question is evaluated as “wrong”. The “to be deleted” tag is given to those question which are incorrect on purpose. Examples are trivial questions, violating the netiquette or those that cannot be answered: What colour is the sky? Which Albanian folk band has not released an album? What colour was Amy Whitehouse’s hair in 2006? and so on.
Of course you can ask questions about Doomlord as well, but only ask questions that do not require the character to be high level and any player can answer, as a level 2 might also get the question: What skill does learning level 14 give? or What is the protection of the Nerubian robe? and won’t be able to answer.
Some examples for good and bad questions (please don’t submit them!)
‘How many members did the Beatles have?’
Although it is a bit trivial, still a good question, because everyone should know the band.
‘Which was the third album of the Australian Pigmies?
No one has ever heard of them, much less of their albums. Bad question.
Too special questions:
‘What is the width of the standard rails?‘
‘Who is the brother of the cartoon hero Glacius?’
‘What is the average wingspan of the Eastern mating mantis?’
‘Who was the coach of Manchester United in 2004?’
‘What is the name of the female protagonist in the Harry Potter books?’
Almost everyone knows the books, so you can ask such a question.
‘What is the title of the fourth book of the Harry Potter series?’
‘The goblet of fire, the qublet of fire, The calice of fire’
The difficulty is OK, but it is based on visual deception (q an g being similar) and contains a spelling mistake, calice instead of chalice. Bad question.
‘How old is the oldest woman on earth?’
There might be up-to-date information that becomes invalid in a month, should she die or become a year older. Try to avoid questions that might become out-of-date.
Of course we don’t expect anyone to submit correct questions all the time - I have also written questions that I would reject today. If 50% of your questions are rejected you get a warning and if this ration persists, you will lose the chance to ask questions. Also you can get the evaluation ‘to be deleted’ maximum three times. After the third occasion you will lose the opportunity to submit further questions permanently.
This is not a penalty, the SE reward is not significant. There are players who simply are unable to ask correct questions. That is all OK, they can still enjoy the other parts of the game.
Evaluators are also evaluated both automatically and manually. The aim is to let those players evaluate questions who give a realistic assessment. There is no penalty for “bad evaluation” you simply won’t be asked to evaluate further questions, however some system for a reward is on the planning table good assessors.
Note that now you have to option to choose the reason why the question was rejected. This way the player who asked the question will have some idea about what he had done wrong.
A hozzászólást Miklos módosította 2009.07.16. 09:47:03-kor
Pontszám: 5
