CppCon 2024

Functions are the fundamental unit of interoperability in software design: users write applications by composing functions provided by various libraries. However, naive function composition can result in poor performance due to memory overheads that exist at function boundaries. Operator fusion --- combining the execution of several operators into one operator --- is a well-known technique to combat this overhead. Previous solutions to add fusion rely on complex, monolithic compilers and languages that force performance engineers to rediscover old solutions in new ecosystems, leading to decades of performance engineering work going underutilized. In this work, I attempt to identify the minimal ingredients required to enable fusion on top of existing library interfaces. In doing so, I propose a lightweight enrichment of function interfaces that exposes data production and consumption patterns of functions. I show how to implement these ideas in C++ and demonstrate the benefits of my system by showing that it is competitive with state-of-the-art high-performance libraries, and that it can fuse across library boundaries for unforeseen workloads.