Ali Rostami Blogs

C++ Memory Models - How Modern C++ Manages Memory Safety

contact@alirostami.net (Ali Rostami) — Sun, 24 Nov 2024 19:45:00 +0000

C++ is a one of my favorite languages. It gives full control over memory and performance. But with this great power comes great responsibility. Issues like memory leaks, dangling pointers, and race conditions is what caused the concerns over these vulnerabilities to rise up to the point that recently led the US government to discourage the use of C++ in favor of memory-safe alternatives.

But I don’t think these claims are justified. Modern C++ has come a long way. With tools like smart pointers, RAII, and thread-safe primitives introduced in C++11 and later standards, managing memory is much safer than it used to be. Also the static analyzers would yell at you relentlessly if you try to do manual memory management.

# The C++ Memory Model

The C++ memory model defines the rules for how threads interact with memory, ensuring that operations like reading and writing data happen in a predictable way.

Before C++11, the language had no formal memory model. We had to often rely on compiler and platform-specific implementations, which made multi-threaded programming dangerous and hard to debug. C++11 changed this by introducing a well-defined memory model, making it easier to write concurrent code without surprises. The C++ memory model provides:

Sequential Consistency: A guarantee that operations appear to execute in the order they’re written. At least from the perspective of a single thread.
Atomic Operations: Safe, lock-free access to shared variables, which eliminates common data races.
Memory Orderings: Fine-grained control over how operations on memory are synchronized between threads.

Without a memory model, concurrent programs could behave unpredictably. Different CPUs might reorder instructions.

# Memory Management Techniques in C++

When it comes to managing memory, C++ has all the tools we need. Along with plenty of ways to shoot yourself in the foot if we’re not careful. Going from manual control with raw pointers to smart pointers and STL containers, memory management in C++ has evolved to reduce the risk of common pitfalls.

# Manual Memory Management

In the pre-C++11 days, you’d allocate memory with new and free it with delete. While this gave developers complete control, it also introduced plenty of opportunities for mistakes:

Memory Leaks: Forget to call delete.
Double Delete: Call delete twice.
Dangling Pointers: Use a pointer after it’s been deleted.

Manual memory management is still useful in low-level code where every byte of memory matters, but for most cases, modern C++ offers safer options.

# RAII (Resource Acquisition Is Initialization)

RAII is a confusing way of saying, “Tie the lifetime of your resources (e.g. memory, file, network) to the lifetime of an object.” With RAII, you don’t need to worry about explicitly freeing resources as they’re automatically cleaned up when the object goes out of scope.

#include <iostream>

class Resource {
private:
 int* data;

public:
 // Constructor: Allocate memory
 Resource(int value) {
 data = new int(value);
 std::cout << "Resource acquired with value: " << *data << std::endl;
 }

 // Destructor: Free the allocated memory
 ~Resource() {
 delete data;
 std::cout << "Resource released." << std::endl;
 }

 // Member function to access the resource
 int getValue() const {
 return *data;
 }

 void setValue(int value) {
 *data = value;
 }
};

void useResource() {
 Resource res(10); // Resource is automatically acquired here
 std::cout << "Using resource with value: " << res.getValue() << std::endl;

 res.setValue(20);
 std::cout << "Resource value updated to: " << res.getValue() << std::endl;
 // Resource is automatically released here when it goes out of scope
}

int main() {
 useResource();
 return 0;
}

The output of the above program:

Resource acquired with value: 10
Using resource with value: 10
Resource value updated to: 20
Resource released.

# Smart Pointers

C++ introduced smart pointers in C++11 to make memory management safer and convinient. There are three types:

# `std::unique_ptr`

Ensures that only one object owns a given piece of memory.
Automatically deletes the memory when the unique_ptr goes out of scope.

# `std::shared_ptr`

Allows multiple objects to share ownership of a single piece of memory.
Memory is only freed when the last shared_ptr owning it is destroyed.

# `std::weak_ptr`

Works with shared_ptr to prevent circular references.
Doesn’t contribute to the ownership count, so it won’t block memory cleanup.

# STL Containers

Containers like std::vector, std::array, std::map, and std::string manage their memory internally, so you don’t need to worry about allocating or freeing memory. For most use cases, they’re the safest and most efficient choice.

# Concurrency and Memory Safety

As soon as you introduce multiple threads into a program, memory safety becomes a whole new problem. Threads can overwrite each other’s changes, read stale data, or even crash the program. This is where the modern C++ memory model comes into play by providing tools to manage memory safely in multi-threaded environments.

# Data Races

A data race happens when:

Two or more threads access the same memory location.
At least one of the accesses is a write.
There’s no synchronization to coordinate the access.

Here’s an example of a potential data race:

#include <iostream>
#include <thread>

int counter = 0;

void increment() {
 for (int i = 0; i < 1000; ++i) {
 ++counter; // Not thread-safe!
 }
}

int main() {
 std::thread t1(increment);
 std::thread t2(increment);
 std::thread t3(increment);
 std::thread t4(increment);

 t1.join();
 t2.join();
 t3.join();
 t4.join();

 std::cout << "Counter value: " << counter << std::endl;
 return 0;
}

Expected Output: 4000 Actual Output: Well, mostly the expected output but running the program multiple times would give different and occasionally wrong outputs. This the output of 10 consecutive runs on my machine:

Counter value: 4000
Counter value: 4000
Counter value: 4000
Counter value: 4000
Counter value: 4000
Counter value: 3019 (**)
Counter value: 4000
Counter value: 3079 (**)
Counter value: 4000
Counter value: 4000

# One Possible Solution

The std::atomic type ensures that operations on shared memory are performed atomically (as indivisible units). This eliminates data races without needing a lock. Here’s how you fix the example above:

...
std::atomic<int> counter = 0;

void increment() {
 for (int i = 0; i < 1000; ++i) {
 ++counter; // Thread-safe increment
 }
}

int main() {
...

Output: 2000

The std::atomic<int> type ensures that all operations on counter are atomic. This means threads cannot interrupt each other during an update.

# Another Approach Memory Orderings

The C++ memory model goes beyond atomicity and lets you fine-tune how operations are synchronized between threads using memory orderings.

std::memory_order_relaxed: Allows maximum performance but no guarantees about visibility between threads.
std::memory_order_acquire and std::memory_order_release: Ensure that reads and writes happen in a specific order.
std::memory_order_seq_cst (default): Provides the strongest guarantees of sequential consistency.

#include <iostream>
#include <thread>
#include <atomic>

std::atomic<int> flag = 0;
int data = 0;

void writer() {
 data = 42;
 flag.store(1, std::memory_order_release);
}

void reader() {
 while (flag.load(std::memory_order_acquire) != 1) {
 // Wait until flag is set
 }
 std::cout << "Read data: " << data << std::endl;
}

int main() {
 std::thread t1(writer);
 std::thread t2(reader);

 t1.join();
 t2.join();

 return 0;
}

std::memory_order_release ensures data = 42 is visible before flag = 1. std::memory_order_acquire ensures the reader waits until the write is complete.

# Conclusion

C++ gives its users control and performance, but it requires great level of attention from users to avoid various memory related issues. Modern features like smart pointers and RAII make memory management easier but it is up to the developers to use them accordingly.

Languages like Rust and Go offer their own takes on memory management. Rust focuses on safety and Go on simplicity. Both offer ways to do unsafe code as well if you want. It is a matter of which is the default and how easy it is to step into unsafe territory. Ultimately it is the developers responsibility to choose the right tool for their project, which is not all about memory safety either.

Pointers are a Double-Edged Sword

contact@alirostami.net (Ali Rostami) — Fri, 17 May 2024 19:46:00 +0000

Pointers are handy. They allow us to pass heavy objects around with little computational overhead. However, this assumption sometimes leads developers to design their program’s data flow based on pointer types. But this is where things could go wrong.

Let’s look at an example of when overusing pointer types to pass data around the program could damage the system’s performance. Consider the copy function in Go. It takes two arguments: the destination slice and the slice that is being copied into the destination slice.

func copy(dst, src []Type) int

You might wonder why the copy function doesn’t simply take the source slice and return a copied slice. The reason is that, in Go (as any other garbage collected language), a function returning references to its internal variables would actually slow the program down!

# But why is Returning References Slow?

In Go, Slices are passed by reference, which means the underlying array is not copied; instead, just a reference to it gets passed around. This, in theory, should make things faster! After all, it’s not copying the data, but that’s not always the case.

We wrote a copy function that just uses Go’s built-in copy function to copy a slice, but we did it in two different ways. In one of them, the destination slice is passed down as an argument, but in the other one, it is created inside the function and returned at the end.

func CopyPointerAsParam(dest []int, src []int) {
	copy(dest, src)
}

func CopyPointerAsReturn(src []int) []int {
	dst := make([]int, len(src))
	copy(dst, src)
	return dst
}

We wrote some benchmarks for them as well:

func BenchmarkCopyPointerAsParam(b *testing.B) {
	for i := 0; i < b.N; i++ {
		src := []int{1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15}
		var dest []int
		CopyPointerAsParam(dest, src)
	}
}

func BenchmarkCopyPointerAsReturn(b *testing.B) {
	for i := 0; i < b.N; i++ {
		src := []int{1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15}
		_ = CopyPointerAsReturn(src)
	}
}

Let’s run the benchmarks and see which one is faster:

$ go test -bench ./...

goos: linux
goarch: amd64
pkg: github.com/alirostami1/escape-analysis-exp/pointer_as_param
cpu: 12th Gen Intel(R) Core(TM) i7-12650H
BenchmarkCopyPointerAsParam-16 423221650 2.817 ns/op
BenchmarkCopyPointerAsReturn-16 43333108 27.54 ns/op
PASS
ok github.com/alirostami1/escape-analysis-exp/pointer_as_param 2.706s

By running the benchmarks, we see that the version with the destination slice being passed down as an argument is ten times faster than the one that returns the destination slice back to the caller function. Odd! Isn’t it?

This is happening because when returning a slice (which is a reference to an underlying array defined inside the function), the compiler is forced to store the array in the heap memory; otherwise, after the function returns and the function stack gets cleaned, the slice would be pointing to a position in the stack that isn’t part of the stack anymore. It points to garbage data that could be overwritten by another function at any time!

# Enters Heap

Instead of storing the underlying array in stack memory, the Go compiler detects that it might be referenced outside the function, so it escapes the array from the stack memory to the heap memory. This means that any slice referencing the array is still valid even after the function returns.

We can see this escape from stack memory to heap memory by enabling the -m compiler flag which prints out the go compiler’s optimization decisions including Escape Analysis:

$ go test -gcflags="-m" ./... -bench .
...
pointer_as_param/pointer_as_param.go:4:25: dest does not escape
pointer_as_param/pointer_as_param.go:4:37: src does not escape
pointer_as_param/pointer_as_param.go:9:26: src does not escape
pointer_as_param/pointer_as_param.go:10:13: make([]int, len(src)) escapes to heap
...

As you can see, while no variable inside the func CopyPointerAsParam(dest []int, src []int) escapes, the make([]int, len(src)) in the func CopyPointerAsReturn(src []int) []int escapes to heap.

But why is this slowing the program down? Because heap memory is slow compared to stack memory. With stack, the runtime can just move the stack pointer down to where the function started when it returns, and everything after the pointer will be discarded. However, the heap memory is different; the runtime has to keep track of each object and its references to identify and delete unreachable objects. The garbage collector makes a graph of objects and their references and periodically traverses the graph to identify unreachable nodes, which adds significant complexity and computational overhead.

Passing the slice down to the copy function allows the compiler to keep the underlying array inside the main function stack frame, as it will no longer be discarded after the copy function returns. As a general rule, pointers and data types that are passed by reference, like maps and slices, should be passed down the call stack, not up.

I Made A Website As You've Probably Guessed

contact@alirostami.net (Ali Rostami) — Wed, 08 May 2024 19:45:00 +0000

So, I made a little website for myself and I plan to write blog posts every week and share anything interesting I learn with the community. The website is written with Astro and it is hosted on Cloudflare Pages. You can checkout out the code (although it’s fairly simple) on Github.

There is Also an RSS feed available.

Let’s Go!