Do not think that this is the system we implemented right at the start (on the original servers) and it didn't change since. It changed a lot.
At the beginning, we had a very complicated system (something like Viridiel suggested...) Every question was evaluated by multiple players. If a huge majortiy accepted the question, it went through. If a huge majority rejected it, it got rejected. The system was checking the evaluators performance. If someone accepted too many questions which were rejected at the end, or vica versa, he lost the evaluator status.
Looks like the perfect system, right? Well, it didn't worked. At all. Tragical questions were accepted, and sometimes perfect questions were rejected. Why? Because the more people do a job, the less responsibilty they have, and the more of them has "suboptimal" skills and intelligence.
At the end, we realized that the result will be much more efficient, if we choose only a very few, but very intelligent, skilled and devoted people who do this right.

Back to the issue of accepting "sub-optimal" questions and correcting them. Believe or not, at the beginning, we did this. We did not want to make anyone unhappy by rejecting the question he/she worked on. So we accepted everything we could turn into a good question by correcting it. It was a horrible - and I mean, a HORRIBLE - amount of work. Like 20x or 30x more work like now. It was impossible to continue it on the long term. Yeah, yeah, we could select 20 more admins who do this, but there is no capacity for that, either in terms of rewarding them, or checking if they are doing their job right.
Same stands for writing an explanation for rejecting. And not only it would be a lot more work - then we would have to provide a kind of "customer service" line where people could argue about the explanation? It wouldn't make anything easier. 98% of the time whenever a question gets rejected, the reason is plain obvious from the automatic response and quiz rules, and even when it's not, people would just argue with the explanation as well.

And check it out - there are a lot of players, who have hundreds of accepted questions, and only a few rejected. So the system works. Yes, there are people too who have a lot of questions rejected - and eventually they give up on writing questions. But in most cases, this is not because the system is wrong, but because there are some people who are not that good at writing questions. Fortunately, the game is 100% enjoyable even without the quiz-writing aspect, it is left for those who really enjoy it, and are really good at it.

Idézet: Miklos - 2011.02.26. 14:23:16

It is unfortunate that you don't read the forums, as that all has been discussed, and some of the suggestions were proposed with that history in mind. The one that I think would be most useful would be to add some kind of appeals process. That shouldn't add much workload for the evaluators, as it would only come up for questions that got rejected, and the author felt strongly that it was a good question. The process could even be to come post in a thread here on the forums, but for this idea to work, there needs to be a way for stekkos or one of the other evaluators to manually credit someone with extra questions as compensation for having a question wrongly rejected.

Idézet: Miklos - 2011.02.26. 14:23:16

Stekkos and CG are already coming to the forums to post their explanation every time they reject a question. Adding that as a feature in-game would actually save work on their part. Likewise, there is already a "customer service" line...these forums. The only thing missing, as mentioned above, is some mechanism for handling the rare case when a good question got rejected.

Idézet: Miklos - 2011.02.26. 14:23:16

As you say in another thread, you never said the system is perfect. But your attitude is clearly that the system is good enough that there is no point in spending any time improving it.

Well, there IS a way to credit back a question to a player if it is proved that his/her perfect question was rejected! The exact same way if you have any other complaint / need for correction - write an email to the GM! She have the permission for this like for many other things. Also, our head quiz admin, Hatsepsut, has this privilege as well, you can contact her directly, or any evaluator can forward her such a request (and I know she does it when necessary).


Surely you feel this is a contradiction as we are talking here right now But as I said, collecting and forwarding ideas from the forum is the primary job of Kethios, I can't read all the posts. So if you think he is slacking, start chewing on him

Yup. My PM box is open for y'all.

I can' be on 24h/day so if you feel something very important - and I'm not on, just PM me and I'll come and check it.

Idézet: Miklos - 2011.03.01. 07:31:36

Great! That's good to know.

Idézet: Miklos - 2011.03.01. 07:31:36

True, I could have been more precise: It is fortunate that you are reading the forums at the moment, but unfortunate that you don't have time to read all the past posts.

Idézet: Miklos - 2011.03.01. 07:31:36

Oh, we chew on him plenty, believe me. My job involves moderating social media, so I can say that this setup is absolutely typical. The people who can make decisions about the product don't have time to read all the posts. But there need to be procedures in place so that the loop gets closed, which too often doesn't happen here. There are a number of possible points of failure in the loop:
- Kethios could be busy and not read the post
- Kethios could fail to forward it on to you
- You could fail to respond to Kethios
- Kethios could fail to forward your response on to us

Sometimes, all that happens, and then we're happy. Even if the response is "that's just too hard to implement". But most of the time, we make a suggestion, and either there's no response, or the only response is Kethios saying he will forward it on, and then we never hear anything more back. Ideally, every post in the suggestion thread should have a response. That thread has been up just over a year, and it has a total of 450 posts. There's a lot of discussion there, so there's probably only about 100 actual suggestions or so, which is less than 10 per month.

I assure you that anyone that has completed it, had to research half of those questions at some point prior to answering them...
Or...was in a clan that logged each Quiz Question and Answer.

Wow! I just discovered this thread and have read through from the beginning. It is nice that admins and mods have, at last, joined in but a shame that most of what they have to say is defensive. It would be better if they listened to and tried to act on what was said in the understanding that only those who really care about this game (and the quiz system with it) are posting. Some of the posts are heated as people are frustrated at what appears, at times, to be a wall of silence but that is not enough to ignore the points being made. (even now I can already see the quote of that last comment and defensive replies!). Please realise I am speaking generally, not specifically.

The recent, much appreciated, transparent wotk from stekkos (in particular) and CG has made the quiz errors thread a far less frustrated area. However, the fact that I and a few others are posting there on a daily basis with multiple quiz errors does show the system is far from perfect. I make as many mistakes as anyone but not as many as keep appearing in the questions. Myself, MartinBV, manukind and others are taking the trouble to point out these mistakes. Surely there can be room for people like us to help correct them before they appear on the database? I couldn't care less about rewards but do care about accuracy. That is the biggest issue. Not specialisation or USA or Hungary. As has been pointed out often, the internet will find the answer to pretty much any question in very little time. In fact, I quite enjoy the few questions where I have to go further tham wikipedia (the most untrustworthy site for facts there is!). [I do find it very dull when the first 3 'hits' on google show the question and answers all laid out because people are so lazy they copy and paste from quiz sites but hey!]

Please, admins and mods and devs (if there are any left), don't get upset and defensive about our comments. We want the game to remain and improve and are expressing our desire to help. If that is not possible, please do not be offended when we point out your inadequacies. It isn't an attack. It is showing you what you missed. Let's work together to keep the game fun, interesting and educational. There aren't many game left that can combine those three so let's make Doomlord the one we can enjoy.

One more thing. Is the question database really small? It seems I get the same questions over and over again and usually those that are quite new. It does speed up hunting but does not increase my knowledge!

Starting from last: The database is not small, there are approx 20 000 questions right now in it. I cannot explain why we get the same one, i was puzzled by this also especially after getting 5 times the same one in 2 mins

About the quiz: We started correcting mistakes on regular basis only a few months back. Most questions have already been submitted by then, so now you are slowly detecting and reporting them. The difference is now they are actually being fixed so in time you will see less and less. And you must have noticed that questions you and the other players post are not always mistakes. But as long as you find something you think it needs correction, post it and we are more than happy to look it up

As for the posts here, we did listen. The quiz now it is far from what it was 2 years ago and that is because of players recommendations. Even the ability to access the database and correct questions was given to us due to the need many players pointed out. As you must have seen though, many players also just attacked, made no valid point or simply let some steam out in this topic. Defensive was the only stand we could take.

Overall the quiz is constantly being fixed and refined. If you want to give more suggestions please do. And you are helping already a lot since only 4-5 players bother to report mistakes.

I know it hasn't been a high priority, but grammar and spelling errors should be down too since I have been fixing them as I encounter them.

I have also thinned out some of the duplicate questions. (i.e. The sky is? What color is the sky? What colour is the sky?) When removing I have pulled those with the fewest correct answers. In the event that they were about the same, I deleted the newest submission. There may be similar type questions still in, but I figured they were okay. Anyway, this may help with the repeats, but not solve the computers love to pick certain questions repeatedly.

ty stekkos and CG. Keep up the excellent work!

2 points really, but both with the same root and solution

1. I currently am answering 100 quiz questions every 2 and a bit days. If you dont have the resources to evaluate a question in this timeframe it makes a mockery of the 100 question = 1 new question to submit rule. Being swamped is no excuse it just shows a lack of foresight.
My current question under eval for the better part of a week now (w1) I now know it will be evalled as 'bad' because Ive just seen another very similar question but I can't cancel it and resubmit a new one

2. The painfully slow time it takes for forum queries etc. to be answered by administators.

If these problems are not addressed (more bodies needed) the whole system is in danger of becoming unuseable and collapsing

To your customers this is a game, but the administration side is part of the customer support business and should be treated as such. A 'suck it and see' approach just isn't good enough.

Yo!
There is such amount of wrong aswers for those questions, AND in the same time there are wrong questions, so I suggest that you(=game supervisors) do something about this It should not be such in a big role since the questions are even wrong but in the same time "right answer" is foundable in the net by google. Your "Quiz competition" is a google exercise, and you get a medal if you succeed. Please stop it. If you really want to develop the game you can create better system for example "Quiz competition". If you wanna live and get fat with those incomes of the game it suits me perfectly, no problem. Only problem is that game will die if you stop doing anything and think that small changes in the game will keep player playin'. But how you bring more players in the game and keep them playing. You've gotta take these thoughts and bring something to the table. I hope it's for the new players. And something new for the old players, who have been uplivin your REAL LIVE living. So WAKE UP! This game will go on like 3-6 months on its own, but after that plyers are dropping out. Do Somethin!
I am paying for belonging in this organization so I think, I have right to speak about those misdoings with the game.

btw.this Doomlord forum is hard to follow, but is is good to be.

Your speaking right!