Testing in Go
Hybrid Meetup #45 took place 2024-10-29 19:00 at CHECK24 Leipzig Office at Altes Landratsamt and we were thrilled to see people joining from Leipzig and beyond, both on site and online. We had two great input presentations about open source projects in the testing domain.
Coverage metrics
Fabian analyzed the Go coverage tool cmd/cover, listed some of its shortcomings and demonstrated a tool to address them and to improve some metrics: gocoverageplus.
The coverage tool shipped with Go operates on the source code and uses instrumentation to estimate coverage:
When generated instrumented code, the cover tool computes approximate basic block information by studying the source. It is thus more portable than binary-rewriting coverage tools, but also a little less capable. For instance, it does not probe inside && and || expressions, and can be mildly confused by single statements with multiple function literals. – cmd/cover
In the past, go cover had a couple of issues, among them: #23883, #51430, #58770, #65570, #65653, …
Also, there is a currently open proposal:
Adding branch coverage would be beneficial, as it would allow developers to better understand which branches (conditional paths) in their code are being executed during tests, rather than just which lines.
The gocoverageplus addresses some of these shortcomings and also supports cobertura output format, as well as complexity metrics.
A short usage demo:
$ go install github.com/Fabianexe/gocoverageplus@latest
$ go test -cover -coverprofile=c.out
Put a config file into your projects folder, then:
$ gocoverageplus -i c.out -o p.out
Then render the report, e.g. as HTML or to stdout. A nice little TUI for coverage reports is: gocovsh.
Go scheduler overlay
We took another look on testing, through the lens of timestone,
a library to create deterministic and easy-to-understand unit tests for time-dependent, concurrent go code.
It requires modification of existing code (replace go routine invocations with timestone), but after that, tests can be run in either with the system scheduler, i.e. using goroutines or a simulation scheduler, that gives the caller control over the process execution.
By default, Go programs can exhibit non-determinism in a few places, among others in:
- randomized work stealing in the scheduler
- randomized map iteration
- randomized select clause
With timestone, it is possible to get around the scheduling randomness, when this interferes with test results. Check out some of the examples included in the library, here: timestone/examples.
Misc
- for integration testing, testcontainers can come in handy; you can simulate the actual services your code interfaces with, like key-value stores, object stores, search engines, databases, etc. – a handy, albeit slightly slower, alternative to mock objects
- for a more unusual test setup, see this lightning talk about running go tests in a VM w/ qemu
