Concurrent woes
We had two input presentations, both involving concurrent constructs.
First, we heard from the author of package
waitgroup about problems with concurrent
code, waitgroups and shutdowns (especially in cloud native and serverless
scenarios). The basic problem: At shutdown time, you want to be graceful, but
at the same time enforce a hard timeout on a number of processes running (and
potentially joining in a wg.Wait).
Enter package waitgroup which allows to preempt running goroutines. The general problem of goroutine termination is hard, and even harder, if you have a large code base and a tree of concurrent routines in flight, that need some kind of cancellation (modern packages use context, or maybe manual timeouts).
The solution of waitgroup is to embed the standard library waitgroup and combine it with a timeout. The package comes with examples too:
Testing concurrent code is its own challange, but this package has examples for that as well.
The second lightning talk summarized chapter 2 on “Filters” from Software Tools by Kernighan and Plauger (1976) and highlighted some timeless ideas on software style (today, you might say clean code). Go has great facilities for streaming data processing in the io package. Furthermore, you can spice up filters with parallel processing, without adding much complexity to the program.
That’s what parallel is about - it allow to write parallel filters fast. The package is best used as a little utility for one off filters.
Misc
The best about meetups are things beyond slides and code: discussions, screen shared setups, neat tools people use, and much more.
Here’s a list of things to check out:
- The ko project helps to get Go applications to run on kubernetes fast, with minimal effort. If you run go on k8s, you will need this.
- The zap logger is a fast structured logging library.
- Just as you can combine waitgroups with timeouts, you can blend context and readers, for details see: https://pace.dev/blog/2020/02/03/context-aware-ioreader-for-golang-by-mat-ryer.html
- Behaviour driven development (BDD) uses a semi-formal language to describe test cases, in Go we have ginkgo - the terminology is language-agnostic.
Can you tame complexity with BDD style development? A VMware Office of the CTO tech deep dive addresses this question in a blog post:
- Taming Complexity in Software Development with Behavior Driven Development (BDD) (vanished as of 05/2024)
As for stream processing, there is a classic blog post, as well as a few interesting projects:
- Go concurrency patterns: Pipelines and cancellation
- Automi (prototype)
- RxGo, an implementation of reactive style (in short: observer pattern + iterator pattern + functional programming)
Fast things:
Basic (database) things:
- BW trees, What is a BW-tree?, Paper
- log-structured merge trees
- and many more (see: databass.dev/)
Join our meetup to get notified of upcoming events!
