Incorrect Test Fixtures lead to Weird Testing Results

After recently praising test fixtures as an important (but time-consuming) part of building your tests, there is something to watch out for.

If you get one of your foreign key references wrong in your fixture data you can get odd results. In particular I had a test that passsed when run using ruby test/unit/my_test.rb but failed when run using rake test:units.

The cause? Simple. I had two fixtures files, :orders and :work_items - where work_items have a detail relationship to orders (with a foreign key order_id). My orders fixtures looked something like this:

daves_first_order: 
  id: 1
  some_field: whatever
daves_second_order: 
  id: 2
  some_field: thingy
georges_important_order:
  id: 3
  some_field: wotsit

My work_items fixtures looked something like this:

first_line_of_daves_first_order:
  id: 1
  order_id: 1 # daves first order
  some_field: fish
second_line_of_daves_first_order:
  id: 2
  order_id: 1 # daves first order
  some_field: gerbil
first_line_of_georges_important_order:
  id: 3
  order_id: 4 # georges important order
  some_field: giraffe

Spot the error? It took me a while - and as I say, the only symptom was this odd behaviour where the test passed in one case but not in the other.

It turns out that the fault was this - one part of the test was creating a new order from scratch and assigning work_items to it. When run via rake the test database was torn down and rebuilt, matching the structure of the development database. Also meaning that every time the test was run, MySql new to give the next record in the orders table an id of 4. So the typo in the work_items fixture file (where first_line_of_georges_important_order has an incorrect order_id of 4) meant that the work_item pointed at the wrong order - leading to a cascade of further failures. For some reason, when run without rake the test passed as the order_id of 4 actually resolved to :georges_important_order (although I'm still not sure why).

The moral of the story? If you get odd behaviour, where your tests behave inconsistently, then examine your fixtures files in great, great detail.

Dog by kufrik