Fast Things First
Hybrid Meetup #42 took place 2024-04-30 at CHECK24 Leipzig Office at Altes Landratsamt. We had two talks that revolved around Go and performance: Patrick Vahl gave a great overview about SIMD in Go and what is possible (and not possible), today. Examples from the talk can be found here: Crash129/go-simd-example. A few points from the talk:
- performance gains can be significant
- try to isolate SIMD as much as possible
The difficulty with a general purpose approach to SIMD […] is that the performance can be dramatically different on different processors. – go/issues/53171
- Go uses its own assembly syntax as an intermediate representation, more on that in The Design of the Go Assembler (2016)
- there are various approaches to SIMD and Go including avo, gocc, and more - see: Examples of methods to use SIMD in Go
- explore the assembly generated for Go programs via compiler explorer
While SIMD support is still a topic of discussion, e.g. in #53171, there exist usable approaches today.
Martin Czygan took a Go implementation of the 1BRC from 4 minutes to 4s, using fewer allocations, parallel processing and mmap.
Notes: 1brc-in-go. Please submit a PR with a variant you found!
Thanks a lot to CHECK24 for hosting the event in their nice office (with a view). Looking forward to our next event together which is planned for October 2024.
Misc
- A good use case for mmap is a parallel processing of read only data (that can be bigger than the physical RAM); there is an experimental package in the x/exp/mmap and a library edsrzf/mmap-go. The concept of virtual memory goes back to one of the first super computer Atlas.
- The simdjson project explores the use of SIMD for record breaking, high performance JSON processing
