Yesterday afternoon I participated in a Weekend Testing Session. The topic for the session was on Designing Data Sets for Testing. More details on the complete testing session can be found here.
For the session we had the choice of three different web applications to test. Our task was to identify a set of test data we could use to test our chosen application.
I decided to test the on-line game Zork. Zork is a text based adventure game where you are able to issue text commands and actions to the game which then allows you to move and interact within a fictional world.
I had played adventure games like this before, in fact the first ever game we got for our home computer back in 1982 was a text based adventure game on our Atari home computer called Escape from Tramm. I particularly remember this game and one of the rooms in where we were stuck for 3 whole months until we realised we had to push a pin on one of the doors in the room!
I decided I would go about this task by exploring and effectively playing the game. In this particular scenario my test data was going to be the actions or movements I typed into the game. To keep track of the test data I started capturing my test ideas in a mind map.
So I started with simple stuff using the Open action. As I was standing by a house I tested out Open Window, Open Door, Open Mailbox. When I opened the mailbox there was a leaflet inside. My testing was going well! Well it was until I tried to open the leaflet and I was told the game it did not know how to open a leaflet. I noted this down as an observation and I decided to move on.
Moving on with my testing I tried different compass directions such as Go North, Go South, Go East. I also chose actions such as run, jump and walk which didn’t seem to get me very far but reported interesting responses such as “which way shall I run?” and the rather odd response to jump which was “Are you trying to impress me?”.
As I went on through my testing I came to realise I didn’t actually know whether I am seeing desirable behaviour or not in the game. It struck me that if if someone had given me a list of commands and a map of the world that the way I approached the testing and selected the commands to test may of been completely different.
It makes me wonder whether it helps or hinders us in our testing if we know too much or too little about an application. If you are over familiar with an application then there is a risk you may get used to using the application in a certain way or may be influenced into using the application in a certain way. Sometimes there is a chance that if you have never used an application before you may point out issues or inconsistencies that people may of not noticed or thought were problems.
As testers I think it is important we are aware of potential pitfalls and traps like this and ensure that are always thinking of new ways to test an application, using both our creativity and experimentation skills in the process.
I really enjoyed this session, for me it brought into focus the importance of the test data we use but more interestingly the importance of variation and experimentation when we choose our test data.
If you have never done a weekend testing session before I would highly recommend it as it gives you a chance to learn more about the way you test and an opportunity to interact with other testers in the wider the testing community.