Testing Django applications with Bitbucket Pipelines and MySQL

Posted on by Kieren

If you’re a Bitbucket user then you probably already know about Bitbucket Pipelines. If you’ve not heard of Bitbucket Pipelines then they are Bitbucket’s continuous delivery mechanism. They allow you to run commands, on a new commit for example, to run tests and then deploy your code to production (if you want).

I’ve been using Jenkins for a while but wanted to try out Pipelines on one of my Django projects. Specifically I wanted to automatically run my test suite when code was committed – I’ll think about using Bitbucket Pipelines for deployment later. Bitbucket Pipelines uses Docker containers to run the tests but has the constraint that you can only use one Docker image and doesn’t currently support things like Docker compose. This makes things hard for testing applications that have a number of dependencies and, in most cases, your applications are going to have dependencies (such as needing a database).

Currently there are two solutions to this:

  1. Use a single Docker image that contains all your dependencies. You can search through the Docker hub for something suitable or create your own.
  2. Set up your pipeline/application to use one or more Bitbucket Pipelines services.

In this blog post I’m going to focus on using the MySQL service with Django in Bitbucket Pipelines. I’m writing this post because I didn’t find the existing documentation to be sufficient to get things going and, in some cases, I think there may be errors or ambiguities that cost me quite a lot of time so hopefully this will save other people time. That said, this is just how I got things working, it may not be the only way and probably isn’t the best way. Some things I don’t intend to cover are configuring Bitbucket Pipelines for your project in Bitbucket, Django installation, running tests (bar including the command to do this).

Basic requirements

For this project I have a number of requirements:-

  • Python 2.7
  • Django (for this example I’m using 1.8 but it doesn’t matter)
  • MySQL
  • pip (to install a number of other dependencies from requirements.txt)

There is some documentation showing how to set up your bitbucket-pipelines.yml and how to add a MySQL service to the build. So, I started off with something like:

image: python:2.7.13
pipelines:
  default:
    - step: 
      services: 
        - mysql
      script:
        - pip install -r requirements.txt
        - python manage.py test myapp.tests.unit_tests
        - python manage.py test myapp.tests.integration_tests
definitions: 
  services:
    mysql:
      image: mysql 
      environment: 
        MYSQL_DATABASE: test_pipelines 
        MYSQL_RANDOM_ROOT_PASSWORD: yes 
        MYSQL_USERNAME: test_user 
        MYSQL_PASSWORD: test_user_password

Error 1 – bitbucket-pipelines.yml file must be a map

However, pipelines wouldn’t run and I kept getting the following error:

The 'environment' section in your bitbucket-pipelines.yml file must be a map.

This was caused by the MYSQL_RANDOM_ROOT_PASSWORD line. It turns out there was an error (well two) in the Bitbucket documentation and the relevant section needs to have values quoted and MYSQL_USERNAME should be MYSQL_USER. Bitbucket have now fixed the docs so hopefully no-one else will have these issues. For completeness the section should be:

...
definitions: 
  services:
    mysql:
      image: mysql 
      environment: 
        MYSQL_DATABASE: 'test_pipelines' 
        MYSQL_RANDOM_ROOT_PASSWORD: 'yes' 
        MYSQL_USER: 'test_user' 
        MYSQL_PASSWORD: 'test_user_password'

Error 2 – django.db.utils.OperationalError: (2002, “Can’t connect to local MySQL server through socket ‘/var/run/mysqld/mysqld.sock’ (2)”)

So, this took a while to sort out and appears to be an issue related to ‘localhost’. The Bitbucket docs say the MySQL database will be available on ‘localhost’ and the Django settings for databases say leave the database setting for HOST as ” for localhost. However, I found that this wouldn’t work and neither would: HOST: ‘localhost’ in my Django settings file. I had to set it as follows:

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql',
        'NAME': 'test_pipelines',
        'USER': 'test_user',
        'PASSWORD': 'test_user_password',
        'HOST': '127.0.0.1',
        'PORT': '3306',
        'OPTIONS': {
               "init_command": "SET default_storage_engine=MyISAM",
        },
        'TEST': {
            'NAME': 'test_pipelines',
            'CHARSET': 'UTF8',
        },
    }
}

It is also important to note that the ‘NAME’ for the database and test database for Django must be the same. When running the pipeline you only get one database and Django will, by default, prefix the default database name with ‘test_’ and that database won’t exist… so you’ll get errors. To get around this, set the test database name explicitly. I set this up in a separate set of settings that I only use for my Bitbucket Pipeline.

Error 3 – Django can’t create or drop the test database

At the start of the test run Django will try and drop and recreate the database but it can’t and the pipeline stops waiting for the answer to:-

Type 'yes' if you would like to try deleting the test database 'test_pipelines', or 'no' to cancel:

To solve this issue we need to get the tests to use the existing database. To do this we need to use the –keepdb flag when running tests. So, we change our bitbucket-pipelines.yml to use:

- python manage.py test myapp.tests.unit_tests --failfast --keepdb

Error 4 – Django test leakage

I’ve found that Django isn’t always great at keeping tests isolated (this is unrelated to Bitbucket Pipelines). This means that tests that run earlier can make database changes that affect the outcomes of later tests. This seems to be particularly evident when running my integration tests immediately after my unit tests. To avoid this I found I need to run the sets of tests separately and flush the database between the runs. So I added the following to my bitbucket-pipelines.yml:

- python manage.py test myapp.tests.unit_tests --failfast --keepdb
- python manage.py flush --noinput
- python manage.py test myapp.tests.integration_tests --failfast --keepdb

It’s important to note that ‘flush’ doesn’t drop and create the database it only flushes the data tables (it doesn’t flush the Django migrations table). It’s also important to use the ‘–noinput’ flag to prevent the pipeline hanging whilst waiting for you to confirm the flush (which you can’t).

Error 5 – OperationalError: (1366, Incorrect string value…

OK, so we’re getting close now. At this point most of my tests are running but the suite itself is ultimately failing because of an “OperationalError” and reporting an “Incorrect string value”.

The test that is failing is a test to make sure my application works with UTF-8 characters. The tests pass locally so I know there’s nothing wrong with my code and it must be the collation used on the MySQL database. The MySQL database that pipelines creates uses the default collation (latin1_swedish_ci) and so the tests fail when inserting and retrieving UTF-8.

At this point I tried a number of different things to try and configure the collation (through MySQL settings such as “–character-set-server=utf8” and “–collation-server=utf8_unicode_ci”) but none of them seemed to work. As a result I resorted to using the code from this StackOverflow answer to change the collation for the existing database tables before I run my tests. It is worth noting that you need to change the “host” setting to be “127.0.0.1” rather than “localhost”. Once that’s done I stored this code in a separate directory (with a few other pipeline specific things such as test settings etc.) and can then run the code to fix the database prior to running my tests using:

- python myapp/pipelines/fix_db.py

Conclusion

After all this I now have a pipeline that’s able to run my Django test suite against a MySQL database using the correct collation. My final ‘bitbucket-pipelines.yml’ file looks something like:

pipelines:
  default:
    - step:
        image: python:2.7.13
        services:
          - mysql
        script:
          - mv pipelines/pipelines_extra_settings.txt private_settings.py
          - pip install -r requirements.txt
          - python myapp/pipelines/fix_db.py
          - python manage.py test myapp.tests.unit_tests --failfast --keepdb
          - python manage.py flush --noinput
          - python manage.py test myapp.tests.integration_tests --failfast --keepdb
definitions:
  services:
    mysql:
      image: mysql
      command: mysqld --character-set-server=utf8 --collation-server=utf8_unicode_ci --default-storage-engine=MyISAM
      environment:
        MYSQL_DATABASE: 'test_pipelines' 
        MYSQL_RANDOM_ROOT_PASSWORD: 'yes' 
        MYSQL_USER: 'test_user' 
        MYSQL_PASSWORD: 'test_user_password'

As I said, I hope this helps get you started with Bitbucket pipelines but I’m sure it’s not the only way to resolve the issues I came across and, over time, I’m sure Bitbucket/Atlassian will address some of these issues and I can remove the workarounds.