Platform-Independent SIMD in Go

Motivation variation among SIMD architectures

Go 1.26 and 1.27 include experimental APIs for Single Instruction Multiple Data operations. SIMD is a native feature of many modern CPUs that allows software to perform uniform operations across vectors of data very quickly, such as adding 8 pairs of float64 values in a single instruction. It can significantly speed up many computationally-intensive tasks, ranging from cryptography to data processing to AI.

Prior to these new experimental APIs, the only way to access this functionality from Go was by writing Go assembly. This was only worth it for truly performance-critical compute kernels, which meant plenty of software that could benefit from SIMD simply left a lot of the CPU unused. Go 1.26 introduced a SIMD API for amd64, and Go 1.27 added APIs for arm64 and wasm.

However, a basic challenge for a SIMD API is the enormous variation between platforms, not simply in what operations they support, but even in how vectors are represented. Some platforms provide fixed-size vectors, typically between 128 bits and 512 bits, while on others the vector size isn’t known at build time. To provide full access to the breadth of these platforms, these APIs live in an architecture-dependent archsimd package.

But Go 1.27 goes beyond these architecture-dependent APIs and introduces an experimental, fully portable, platform- and size-agnostic SIMD interface. The goal is to support write-once near-asm-performance simd code on platforms with SIMD support, and to provide a competent emulation on those platforms that do not have SIMD support.

API intersection and method emulation

SIMD architectures vary in several dimensions including fixed vector sizes, variable vector sizes, and feature checks. Different SIMD architectures also vary in how they handle vector masking. A third source of variation is in the operations themselves, as each architecture provides its own primitives for rearranging vector elements and supporting crypto-related operations.

The new simd package hides these differences by removing fixed-size vectors from the type system, and by only supporting those operations that are in the intersection of all the different platforms. On platforms that lack SIMD instructions or that lack support in archsimd, all of the operations are emulated, so code written using the simd package will always run. To use this experimental package, set GOEXPERIMENT=simd.

Whatever operations the simd package offers need to run acceptably well on most architectures. A naive intersection across SIMD methods from different architectures still leaves plenty of holes, which are filled by adding emulations to the various architecture-specific archsimd APIs. The constraint of supporting all platforms forces a somewhat conservative approach to which methods we add to simd.

GODEBUG settings and implementation details

On platforms where there is some hardware support, behavior can be modified with GODEBUG, to make it easier to test simd-using code with various hardware configurations. In Go 1.27, levels of SIMD support are roughly described by vector length. For instance, GODEBUG=simd=0 means use emulation for SIMD operations even if the hardware support is available, while GODEBUG=simd=128 means use 128-bit vectors and their features.

If you are debugging code that uses simd, or even just look at a stack trace, you will notice some weird extra types and methods. The reason is that simd is both a package, an internal implementation package, and some AST rewriting in the front end of the compiler. The AST rewrite creates multiple specialized copies of functions, variables, and types that mention simd types.

What is coming next

We plan to publish a blog post describing archsimd in greater detail soon. For Go 1.28, we intend to add SVE support to archsimd, and also hope to add that to simd. More importantly, we hope to add additional SIMD operations to those that the simd package already supports.

Go 1.28 will also include a small number of feature variants to avoid downgrading all the way to full emulation for platforms that have a hardware vector implementation but just lack one or a few operations. These improvements will continue to expand the power and reach of Go vector programming.