A while back we ran a test of the alpha build. We asked 20 people unfamiliar with the game to give it a try, I personally instructed each person on how to play, and then watched and recorded as they played a 3 minute match. I want to share some of the more interesting results with you!
Let's get right to it. The players got to experience a build of the game similar to this one. http://www.youtube.com/watch?v=cmE2Eeq_mEE
1. Only 25% of players used the evil laughter mechanic (now called Tyranny) more than once.
What does this tell us? It could mean a few things. That the mechanic was not useful, too expensive, or they just forgot it was there! In this I think it was a combination of these factors. In addition to that, the button was inconveniently placed in the corner, away from the buttons players were usually interacting with. It should have been grouped closer together.
Result: Pretty much everything about this mechanic has been improved. It now does direct damage, it runs on a timer instead of a resource that is shared with something else, and it alerts the player when its ready. Its also in an easier location in the UI. I'm confident when we run this test again the results will be very different.
2. 70% of players spent their resources the moment they had them.
Most players were pretty much spamming different cards until they won. Part of this was because of the simple way resources were presented. Each card was costing a point, players held up to 6 points and generated new points over time. It makes it easy because if you have money at all you can play one card. I knew this mechanic was a bit odd, I've never seen it before but I wanted to try it out and see if it worked well. Its simplicity was nice in some ways, but in other ways it complicated the game's design.
Result: The old way of playing a card is long gone. You now need to spend spirit points, which are still accumulated over time. Cards now cost numbers like 70 or 350 spirit points instead of 1. The new design will allow for greater diversity in cards (a Skelesword will still be useful late in the game because its cheap!) It also allows for players to save large amounts of points in case things go unexpectedly south.
3. 35% of players made traps a part of their strategy.
This number has a lot to do with resource situation. By having a trap cost the same as a tower, a wall, or a soldier we were putting them on the same level. The trap was severely under powered. Not only that, but the way the AI found it's way around the very open level they would rarely step on a trap.
Result: Corridors were added into the levels for choke points, the power of the trap in question was doubled, and it's cost in spirit points is now much lower than deploying a soldier or a wall.
4. 80% of players constructed a wall.
Walls were really important in this early build of the game as they changed the flow of traffic in the level. I was actually expecting every player to use the wall as it was incredibly useful. So why did 20% of players still avoid the walls? The answer got was that they did not want to block the movement of their own soldiers. An aggressive strategy!
Result: This information doesn't' mean much, it was pretty much inline with my expectations. Because of how maps have changed since this testing, I now suspect that walls would be viewed less favorably.
5. 70% of players deployed archers out in the open.
I thought this stat was kind of funny. Most players put their archers on the front lines, pretty much unprotected and saw them dispatched immediately. This is learning that I will expect many players to need to go through, even in the final build. How weak are archers? There is only one real way for the player to find that out. That being said, this level design that we were testing did not have any of the gaps and various levels of height that I envisioned when I initially planned the game. If it did, there would have been plenty of logical locations to deploy the archers.
Result: Nothing much! The level design now has gaps and various levels of elevation that a smart player can take advantage of.
6. 60% of players achieved victory!
That many people survived for a whole 3 minutes. I feel that this was a pretty good number considering the audience tested and the type of game it is. All players understood how to play pretty much immediately, I don't think I even needed to show them. The players that lost tended to lose because they deployed only one or two types of things instead of the seven they had available.
Result: This is pretty much right what I wanted to see and confirms the difficulty was about right.
7. 20% of players protected their door from any damage.
This means that 20% of our players played well enough that their door took no damage at all! (The door can take a good amount of damage before it can lose.) Generally this was all done with the same strategy, becoming heavily entrenched around the door and concentrating all efforts in that general area.
Result: This is actually the same strategy I was using when testing the game, it seemed to work too well. What I decided to do is change how the doors work. Previously if you lose your door, you lost the level. Now levels have multiple doors! Players will often have to protect 2 or 3 at once.
8. Skeleswords were the most popular unit with an average of 20.3 deployed per game.
The Skelesword was clearly very popular with players, not only were they very useful, but they were also the most fun to watch. This unit is good all around, and it was the only melee unit present in the test. These guys have a fun behavior where they chase down the enemies.
Result: The Skelesword was meant to be a big part of the game, and while players relied on them more than slightly more than I anticipated, everything was perfect here. We eventually did knock the Skelesword's health down by about 20%.
9. Average number of things that caught fire? 11.7
The way fire works on ToT is really cool. Any unit on the field (enemy or ally) can catch fire. Fire slowly does small amounts of damage over time. Fire will be spread if a unit contacts with others. One strategy is to set one of your own more aggressive units on fire, and watch as they set fire to the enemy ranks. During this test, fire spread around 3-7 times per use.
Result: While fire was constantly at the players disposal in this build, that will not be the case in the actual game. The rate that fire spreads was increased.
10. 95% of players survived longer than 2 minutes!
This was another good result! Players were suppose to play for a total of 3 minutes, the majority of enemy pressure comes in at the last minute. This metric shows we were applying the right amount of pressure for the desired difficulty.
Result: This metric doesn't have much of an effect on the actual game, it just helped me gauge the average ability of new players.
Ok that's it! Stay tuned for more!