CppCon 2024

Programming normal C++ leads to the use of dynamic memory, pointer chasing, and other issues that zap performance. Because there haven’t been practical solutions to those problems we seldom discuss them, creating the impression that they are unavoidable. Fortunately, the same C++ that binds us frees us to devise techniques centered around the concept of “relocatability” to solve those issues. We will explain what relocatability is and how to achieve it.

Consider the example of runtime polymorphism. You may want to have interfaces and implementations of interfaces. If your object refers to a “polymorphic” member, it normally cannot be implemented as a concrete sub-object, only as a base class pointer that points to a dynamically allocated instance of a derived class. This leads to millions of small objects dispersed throughout memory, all referencing one another. This is already very expensive and other issues around object layout makes things worse. One presenter witnessed 7% of Google Search Qrewrite CPU spent on dealloc, which must chase pointers to finish!

When we want to save the state of such programs, we must traverse those complex object graphs and encode them in complex file formats: serialization is required. Restore is further complicated by needing millions of small allocations which are often ephemeral. In general, working with data architected in this way is error prone and has abysmal performance. Our alternative to these complex object graphs is to lay out program data according to program design objectives, in our case allowing object moves. This is the essence of relocation. The set of techniques we will share with you is complete. Now that we can represent program state as simple, relocatable data, save and restore is simply memory mapping buffers.

For runtime polymorphism, we can use “Type Erasure”, the Design Pattern behind `std::function`, tailored for relocatability. Our Type Erasure performs better than normal polymorphism. We get better performance, we have no allocations, no deallocations, and no pointer chasing; resulting in happier programming. For other issues we get benefits of the same magnitude.

We will explain both well known and novel relocation techniques. The novel techniques have required the formulation of Generic Programming concepts including what we call “Value Manager”. We are presenting these for the first time.