In looking into #563, I realized there are gaps in test coverage. For example, `bin/annotate` doesn't have any tests. 

Tidying the spec directory to make it easier to add tests in future work.