Go advanced testing tips & tricks

This post is based on talk I gave at Vilnius Golang meetup. 
I have read many blogs, watched talks and gathered all these tips & tricks into a single place. Firstly I would like thank the people who came up with these ideas and shared them with community. I have used information and some examples from the following work:

Before reading this, I suggest that you should already know how to do table driven tests and use interfaces for your mock/stub injection. So here goes the tips:

Tip 1. Don’t use frameworks

Ben Johnson’s tip. Go has a really cool testing framework, it allows you to write test code using the same language, without needing to learn any library or test engine, use it! Also checkout Ben Johnson’s

Tip 2. Use the “underscore test” package

Ben Johnson’s tip. Using *_test package doesn’t allow you to enter unexported identifiers. This puts you into position of a package’s user, allowing you to check whether package’s public API is useful.

Tip 3. Avoid global constants

Mitchell Hashimoto’s tip. Tests cannot configure or change behavior if you use global const identifiers. The exception to this tip is that global constants can be useful for default values. Take a look at the example below:

// Bad, tests cannot change value!
const port = 8080

// Better, tests can change the value.
var port = 8080

// Even better, tests can configure Port via struct.
const defaultPort = 8080

type AppConfig {
  Port int // set it to defaultPort using constructor.
}

Here goes some tricks, that hopefully will make your testing code better:

Trick 1. Test fixtures

This trick is used in the standard library. I learned it from Mitchell Hashimoto’s and Dave Cheney’s work. go test has good support for loading test data from files. Firstly, go build ignores directory named testdata. Secondly, when go test runs, it sets current directory as package directory. This allows you to use relative path testdata directory as a place to load and store your data. Here is an example:

func helperLoadBytes(t *testing.T, name string) []byte {
  path := filepath.Join("testdata", name) // relative path
  bytes, err := ioutil.ReadFile(path)
  if err != nil {
    t.Fatal(err)
  }
  return bytes
}

Trick 2. Golden files

This trick is also used in the standard library, but I learned it from Mitchell Hashimoto’s talk. The idea here is to save expected output as a file named .golden and provide a flag for updating it. Here is an example:

var update = flag.Bool("update", false, "update .golden files")

func TestSomething(t *testing.T) {
  actual := doSomething()
  golden := filepath.Join(“testdata”, tc.Name+”.golden”)
  if *update {
    ioutil.WriteFile(golden, actual, 0644)
  }
  expected, _ := ioutil.ReadFile(golden)
 
  if !bytes.Equal(actual, expected) {
    // FAIL!
  }
}

This trick allows you to test complex output without hardcoding it.

Trick 3. Test Helpers

Mitchell Hashimoto’s trick. Sometimes testing code gets a bit complex. When you need to do proper setup for your test case it often contains many unrelated err checks, such as checking whether test file loaded, checking whether the data can be parsed as json, etc.. This gets ugly pretty fast!
In order to solve this problem, you should separate unrelated code into helper functions. These functions should never return an error, but rather take *testing.T and fail if some of the operations fail. 
Also, if your helper needs to cleanup after itself, you should return a function that does the cleanup. Take a look at the example below:

func testChdir(t *testing.T, dir string) func() {
  old, err := os.Getwd()
  if err != nil {
    t.Fatalf("err: %s", err)
  }

  if err := os.Chdir(dir); err != nil {
    t.Fatalf("err: %s", err)
  }

  return func() {
    if err := os.Chdir(old); err != nil {
       t.Fatalf("err: %s", err)
    }
  }
}

func TestThing(t *testing.T) {
  defer testChdir(t, "/other")()
  // ...
}

(Note: This example is taken from the Mitchell Hashimoto — Advanced Testing with Go talk). Another cool trick in this example is the usage of defer. defer testChdir(t, “/other")() in this code launches testChdir function and defers the cleanup function returned by testChdir.

Trick 4. Subprocessing: Real

Sometimes you need to test code that depends on executable. For example, your program uses git. One way to test that code would be to mock out git’s behavior, but that would be really hard! The other way to actually use git executable. But what if user that runs the tests doesn’t have git installed? 
This trick solves this issue by checking whether system has git and skipping the test otherwise. Here is an example:

var testHasGit bool

func init() {
  if _, err := exec.LookPath("git"); err == nil {
    testHasGit = true
  }
}

func TestGitGetter(t *testing.T) {
  if !testHasGit {
    t.Log("git not found, skipping")
    t.Skip()
  }
  // ...
}

Trick 5. Subprocessing: Mock

Andrew Gerrand’s / Mitchell Hashimoto’s trick. Following trick let’s you mock a subprocess, without leaving testing code. Also, this idea is seen in the standard library tests. Let’s suppose we want to test scenario, when git is failing. Let’s take a look at the example:

func CrashingGit() {
  os.Exit(1)
}

func TestFailingGit(t *testing.T) {
  if os.Getenv("BE_CRASHING_GIT") == "1" {
    CrashingGit()
    return
  }
  cmd := exec.Command(os.Args[0], "-test.run=TestFailingGit")
  cmd.Env = append(os.Environ(), "BE_CRASHING_GIT=1")

  err := cmd.Run()
  if e, ok := err.(*exec.ExitError); ok && !e.Success() {
    return
  }
  t.Fatalf("Process ran with err %v, want os.Exit(1)", err)
}

The idea here is to run go testing framework as a subprocess with slight modification (os.Args[0]- is the generated go test binary). The slight modification is to run only the same test (-test.run=TestFailingGit part) with environment variable BE_CRASHING_GIT=1, this way in a test you can differentiate when the test is run as a subprocess vs the normal execution.

Trick 6. Put mocks, helpers into testing.go files

An interesting suggestion by Hashimoto is to make helpers, fixtures, stubs exported and put into testing.go files. (Note that testing.go files are treated as normal code, not as test code.) This enables you to use your mocks and helpers in different packages and allows users of your package to use them in their test code.

Trick 7. Take care of slow running tests

Peter Bourgon trick. When you have some slowly running tests, it gets annoying to wait for all the tests to complete, especially when you want to know right away whether the build runs. The solution to this problem is to move slowly running tests to *_integration_test.go files and add build tag in the beginning of the file. For example:

// +build integration

This way go test won’t include tests, which have build tags.
In order to run all the tests you have to specify build tag in go test:

go test -tags=integration

Personally, I use alias, which runs all tests in current and all sub-packages except vendor directory:

alias gtest="go test \$(go list ./… | grep -v /vendor/) 
-tags=integration"

This alias works with verbose flag:

 $ gtest
 …
 $ gtest -v
 …

Thanks for reading! If you have any questions or want to provide feedback, you can find me on my blog https://povilasv.me or contact me via twitter @PofkeVe.