Testing

This week I’ve:

  • Oversteer Racing: updated code to remove deprecated API references for level loading
  • Oversteer Racing: updated code for the particle system as some of the current code is deprecated
  • Oversteer Racing: written¬†the latest dev blog
  • Glossary: removed AddThis as it cannot be configured not to set cookies any more
  • RF1: set up Bitbucket Pipelines to run automated tests for RF1
  • Misc: switched over my backups to the new EC2 instance
  • Misc: drafted a blog post on testing Django applications using Bitbucket Pipelines

In the last sprint I completed:

  1. Stop new bubbles overlapping
  2. Made re-instantiated bubbles faster
  3. Stopped the ‘slow down’ power-up being shown when only one bubble remains
  4. Cleared power-ups when the last bubble has burst (i.e. end of the game)
  5. Tweaked the text colour used in the ‘end game’ screen
  6. Read the Unity documentation on testing
  7. Play testing

A few other things I did:

  1. Did some analysis on link profiles and thin content (SEO) for one of my sites

I’m expecting to have to spend some time on my fantasy F1 project in the next couple of weeks so I can launch it well ahead of the coming season. This means I’ll only be able to put in a limited amount of time on Bubblision and just have a few stories to focus on (albeit important ones).

Sprint stories

  1. Create the GUI for all remaining screens
  2. Sort out the correct scaling for various screen dimensions
  3. Start creating some tests

Expected duration

Two weeks (ending 23/02/2014)

Yay for testing!

I’m also excited about the release of official testing tools by Unity itself. Given that the impression I got from various gaming events I’ve been to is that few game devs do any TDD (test driven development) or TSD (test supported development) I’m really pleased that Unity themselves are making it easier for devs to test their own code.

That said, I still fully expect to hear comments such as “We’re too busy coding to do any testing” and “I can’t see how I’d test my code” whenever I bring up the subject for some time to come. ūüôĀ

In order to help migrate a CakePHP 1.3 app to CakePHP 2 I wanted to get the SimpleTests I created for 1.3 working in v2 so that I can be sure the migration has worked correctly. The Cake docs suggest that SimpleTest will work with CakePHP 2:

Of course you can continue to use SimpleTest in your application by replacing the related files.
http://book.cakephp.org/2.0/en/appendices/2-0-migration-guide.html#phpunit-instead-of-simpletest

Well… maybe you can but I tried a few different things and couldn’t get it to work at all. I put SimpleTest in my vendors directory in 2.3 and tried replacing webroot/test.php with the same file from 1.3 (+ fixing paths), moving the files from cake/tests/lib/ (1.3) to lib/Cake/TestSuite (2.3) and renaming to the new filename conventions. Nothing I tried worked….

Eventually I ran out of things to try… so I asked on the official CakePHP IRC channel. Unfortunately no-one there could help either. The main suggestion was that I shouldn’t use SimpleTest and should just migrate to PHPUnit which unfortunately misses the point of using tests to help make sure the migration was a success. If you migrate your test suite at the same time then it defeats the object of the exercise.

Unfortunately the result was that I couldn’t get CakePHP 2 to work with SimpleTest and neither the documentation or people on the official CakePHP IRC channel could help… so I did have to migrate my test suite (to PHPUnit) at the same time as migrating my app from CakePHP 1.3 to 2. This was pretty disappointing but, if there’s one tiny positive from this, I could at least refer to the tests I’d written previously when creating the new ones rather than having to come up with a list of things to test again.

Background

The background to this project is that I’m having to migrate an existing CakePHP 1.3 application to CakePHP 2. Most importantly I want to be able to test the application once I’ve migrated it (using the same test suite used in 1.3) to make sure it still works as expected. Once that’s done I can deploy the upgraded app and then set about re-writing the tests to use PHPUnit.

The CakePHP 2 migration notes state that you can run SimpleTest tests with CakePHP 2 despite it using PHPUnit as its testing framework of choice now:-

Of course you can continue to use SimpleTest in your application by replacing the related files.
http://book.cakephp.org/2.0/en/appendices/2-0-migration-guide.html#phpunit-instead-of-simpletest

Update: If you’re migrating between 1.3 and 2 then please read ‘Using SimpleTest with CakePHP 2‘ before carrying on (it might save you a lot of time).

The elephant in the room…

Recently I’ve been spending most of my time working with Django and have been spoilt by the quality of the testing framework that Django has. In comparison, using SimpleTest to test this PHP app has been pretty frustrating and disappointing…. that said, the clue is in the name as it’s obviously not designed for anything other than simple tests. Obviously CakePHP now uses PHPUnit instead of SimpleTest but you have to migrate to version 2 before you get the goodies. Using tests on 1.3 to help with that migration was pretty tricky.

SimpleTest is OK for unit testing but has a few issues when it comes to web/integration testing:

  1. There’s no support for fixtures in the Web Tester
  2. Testing against a database means testing against your default database and not even the test database

So, any changes you make during the course of testing your application are persisted in the default database used by your application. Obviously you’ll be testing in a development environment but even so that’s still frustrating.

A work-around… of sorts

This is a fairly brute force way of getting the Web Tester to use a test database and to make sure that database is cleaned up between tests. I’m only interested in creating a semi-temporary test suite to help with the migration (I’ll switch to PHPUnit as soon as possible after the migration) so this approach is (very) rough and ready. The process is as follows:

  1.  Use a shell command (called from within the tests) to rebuild our test database before each set of tests are run
  2. Force requests from SimpleTest’s Web Tester to use our test database

We create a database for use with our web tests and then use a shell command (triggered in the tests) to rebuild that database before each set of tests are run. We get MySQL to rebuild the database by reloading a dump file we generated by running mysqldump against a database (probably our default one originally). This also enables you to use a subset of data (if you want) and effectively gives you the fixture loading functionality you’re missing.

Creating the ‘fixture’ for the test database

I chose to create a separate test database for web testing rather than reuse Cake’s test database but it’s up to you. Once the database is created I imported a dump file from the live database to give us data to test against. Once this is in the test database we can then reduce and/or anonymise any of the data as required. Once I was happy with the data in the test database I dumped the data to a file. We’ll use this file to restore the database after each test has run.

The utility function to reset the database

I created a new directory: app/test/cases/views and in there created a new file called util.php. This file contains a function for resetting our database. All it does is run a shell command to load the contents of the database dump file back into our test database (effectively removing any changes made by the tests). One thing worth noting is that the database that gets reset is the default¬†one but ciritcally we’re going to change our config file so that when the SimpleTest Web Test tool runs then the default database is actually the new test database we’ve set up… with me?

Making sure the Web Tester uses our test database

This is a bit harder than you might expect (although others may have suggestions). As I’m only planning on testing on my dev environment I made changes to the local database.php file as I don’t check this into my code repo (for obvious reasons) and so these changes never appear in the live environment. I also don’t want the code that overrides the current database accidentally triggered when in production even though this is unlikely and we wouldn’t have a web test database in that situation anyway. I created a __construct() function in database.php to perform the switch (see Easy dynamic database connection in CakePHP).

Originally I thought I could trigger the switch based solely on requests coming from a known IP and by setting a ‘referer’ on SimpleTest’s Web Tester.

Although this appeared to work the fact that the tests are triggered by a GET request when you follow the link in your normal browser you end up with a few problems. This initial request to run the test case selects the normal default database and¬†prevents the overriding of the database connection when the web tests are then run. To fix this I changed the __construct() function to switch to using the web test database based on a specific query string (in addition to the IP address and referer). This solves the issue and also prevents the database switch happening when running other test cases you may have (there’s also another downside covered below).

The __construct() function to the end of the app/config/database.php file:

¬†Setting the referer for SimpleTest’s Web Tester

When writing tests you often find yourself needing to refer to things like hostnames, URLs and so on. I tend to create these in a separate function and can then include them in my tests to avoid repeating things. As a result this is a good location to also set the ‘referer’ for the SimpleTest Web Tester client.

The tests/cases/views/url.php file:

An example test

Example code for a test that utilises this setup:

Limitations

  • It’s a pretty hacky way of getting to a useful test environment but you need to bear in mind that it’s intended to be a stop gap to enable us to migrate and use Cakephp 2 and PHPunit. Also, you don’t have much choice if you want to do integration testing on CakePHP 1.3.
  • If your database is big, and you can’t limit the data in the fixture, then the tests might be slow.
  • The use of environment variables in the __construct() function means the web tests can’t be run from the Cake console.
  • Edit: As it turned out this didn’t help with my migration to CakePHP 2 anyway