• Quick note - the problem with Youtube videos not embedding on the forum appears to have been fixed, thanks to ZiprHead. If you do still see problems let me know.

Weird grade distribution. Statistics help?

Actually a really good test of this, I think, would be to give us here at the JERF the assignment and the test and see what our results are. I'm willing to guess that there would be 15 people from here willing to try the test and play the game.

I don't know much about statistics but I'd imagine a little more data can't hurt. :)
 
JFrankA said:
Actually a really good test of this, I think, would be to give us here at the JERF the assignment and the test and see what our results are. I'm willing to guess that there would be 15 people from here willing to try the test and play the game.
Click to expand...

Even better. Not sure if I'd do the whole test several times - but twice would be interesting. Once before ever playing the game or after 5 levels, and one after how ever far I could get.

I don't know much about statistics but I'd imagine a little more data can't hurt. :)
Click to expand...
Hardly.

Also, it would provide a much better insight into other possible reasons: Badly worded questions; maybe we'll just find that there wasn't enough time to answer everything, etc.

ETA: This is making the naive assumption that playing the game alone will be sufficient to answer any of the questions. That assumption might very well turn out to be wrong.
 
Last edited:
The test was not to see if they played Zuma well. Merely playing a game well has nothing to do with game design (though good designers can make better choices since they understand the game design). But as I said, they were given a sample test showing what kinds of things would be asked. What was required was playing a game and analyzing its design. That is a very important skill. I was very clear in my explanation when I gave the assignment what they were supposed to look for.

Question 15 was about a boardgame. They were given the 2-page rules, and were asked to find an exploit, since it had a huge hole in it.

Attached are the per-question scores. Question 11 has the best scores because it was the easiest question.

If anyone wants to try the test, I'll send it via PM. I will post here the example Bejeweled Blitz questions and answers.

sample questions said:
1. What parts of the interface changed from Bejeweled 2 to give the user
a feeling of speed?
2. How does the game reward speed?
3. How does Pop Cap earn money from Blitz?
4. How is this different from Bejeweled 2?
5. What can you buy with coins?
6. What motivates people to use coins?
7. How does the "Last Hurrah" work? How is it exciting?
8. Why is there a "Daily Spin?"
Click to expand...

sample answers said:
1. While it says "One second. Go!" you can move gems already. You don't have
to wait for the last move to finish before making another. Each match made
within a short time of the last speeds up the rate of switches and falling
gems.

2. Each match within about 3 seconds of the last gains a bonus of 100 x the
number of consecutive fast matches. Each match within about 1 second of
the last increases the speed of the game. Since the game is fixed at one
minute in length, this increases how many points you can score by
effectively giving you more time.

Each < 1 second match increases a speed bar. When this is full, the player
hits "Blazing Speed" mode. This lasts for about 5 seconds. Each match made
in this mode explodes, destroying all gems in a 3x3 square.

3. Players can buy coins in exchange for Facebook credits or other methods.

4. Bejeweled 2 can be played free on the web (with ads), which advertises the
"Deluxe Version" which costs $20.

5. Up to three abilities that increase your ability to score points.

6. Without coins, your scores are lower. On the right-hand side of the screen
the game shows the scores of people on your friends list. Peoples' natural
desire to compete drives them to spend coins to get higher on that list.

7. All power gems are triggered. Any new matches are scored, and further
formed power gems trigger again. It creates suspense because at this point
the user has no more control over the game, and the random draw of new
gems falling in controls whether they will score more points.

8. The daily spin starts the session with the slot-machine suspense mechanic.
In addition to the random reward, the user also gets 100 coins times the
number of consecutive days they have logged in (for a maximum of 700 coins
per day). This encourages the user to log in every day. Users who log in
every day are more likely to generate Facebook posts on their friends'
feeds which is good advertising.

The user also gets 100 coins times the number of people on their friends
list who have played (maximum 5000 coins). This encourages the player to
spread the word about the game among their friends.

If the user gets a big payoff, they are encouraged to post a news item
(which is further advertising) that gives the first 10 people to click
it free coins (which brings active players back sooner).
Click to expand...
 

Attachments

  • PerQuestionScores.jpg
    PerQuestionScores.jpg
    26.7 KB · Views: 13
Andrew Wiggin said:
It's possible that your grade distribution could be because you have two populations, one that has fast enough reflexes to get to the level you specified for a pass, and one that does not, with a few players falling in the middle.
Click to expand...

You gain levels in Zuma Blitz just by playing. So whether you play well or not, you will advance through the levels. It might take an extra... I don't know... maybe 50% more playing time if you are really bad at it, but you'll get there. And since there was no other assigned homework, I don't think it was asking too much.

EDIT: I should also say that there were no prerequisites to take the class. Absolutely anybody could walk in off the street and take the class. I'd say about half the class really wasn't very interested in it.
 
Last edited:
I wouldn't mind having a go at the questions if you could PM them to me - I play zuma blitz - I'm not that good, but I'm level 62. Lisa Simpson beats me by a huge margin every week. I don't imagine I'd do well in the test if there are similar questions about how blitz differs from other versions of the game though, as I haven't played any other version.
 
Professor Yaffle said:
I wouldn't mind having a go at the questions if you could PM them to me - I play zuma blitz - I'm not that good, but I'm level 62. Lisa Simpson beats me by a huge margin every week. I don't imagine I'd do well in the test if there are similar questions about how blitz differs from other versions of the game though, as I haven't played any other version.
Click to expand...

I did not assume people had played other versions of Zuma. The Bejeweled example questions were given after I presented Bejeweled 2 and Bejeweled Blitz to them.
 
Almo said:
<snip>

Attached are the per-question scores. Question 11 has the best scores because it was the easiest question.
<snip>
Click to expand...

Just looking at the attachment. You did not give the total scores for each person, so it is a little bit harder for me. However it appears that only the better people got marks for the more difficult questions. I think DevilsAdvocate has the answers.
 
rjh01 said:
Just looking at the attachment. You did not give the total scores for each person, so it is a little bit harder for me. However it appears that only the better people got marks for the more difficult questions. I think DevilsAdvocate has the answers.
Click to expand...

DevilsAdvocate said:
My guess would be that:

1) If you played the game as required, a score of 45% to 55% is very easy and expected.
2) If you didn’t played the game as required, a score of 5% to 15% is very easy and expected.

So the test tests whether or not somebody played the game as required (and not much else). Everybody fits into those categories, except the 35% which could be someone who either didn’t play the game fully as required or who did but just didn’t do as well as expected on the test.

If you were expecting the test to test game design skills, my guess is that it actually tested who played the game and who didn’t without much relevance to game design skills.
Click to expand...

In this case, my intention was to test knowledge of a particular game design, which is knowledge you get best by playing it. Part of the purpose of this excercise was to show them how complex even an apparently simple game design can be. It seems it was successful, to an extent.

Other assignments had them modify existing game designs, or come up with their own. So overall the class had a range of types of tasks. The final project focuses more on production, as they are making a game using Game Maker. The title of the class is Introduction to Video Game Design and Production.

Thanks to everyone for taking a look at my data and giving me some good advice about it. :)
 
DevilsAdvocate said:
My guess would be that:

1) If you played the game as required, a score of 45% to 55% is very easy and expected.
2) If you didn’t played the game as required, a score of 5% to 15% is very easy and expected.
Click to expand...
I'd agree with the first, but not the second. Model the data as a Poisson distribution with an upper limit at 60, and the data tapers away from that limit naturally. The extra 13% can be taken as an anomaly.
 
From adding the frequency of each range, it looks like you have 16 students?

The 50-60% range is only 4 students more than the 40-50% range. How big of an anomaly is an extra 4 people? If there were only 4 students total and they all got similar grades, how weird would that really be?

Also, the dip in the middle two bins is only about 1.5 students below the average number in all bins.

It is possible that this is reflecting a tendency toward grouping, but the low number of students seems likely to result in apparent patterns if you analyze too much even if there isn't any underlying bias.

Before getting into the exact pattern too deeply, I would wonder why none of your students scored higher than 56%. That's what has me wondering.
 
ConspicuousCarl said:
Before getting into the exact pattern too deeply, I would wonder why none of your students scored higher than 56%. That's what has me wondering.
Click to expand...

Agreed. My impression from talking with them is that most of them were busy either with jobs or other classes with stiffer requirements, and didn't take the assignment seriously. One said I should have required them to reach a certain level each week. I told him that I thought I could trust people who are at least 20 years old to pace their own progress in their work, especially with such a light assignment. I monitored their progress more closely on other tasks which I felt required more guidance.
 
You must log in or register to reply here.

Back
Top Bottom