C++ Condition Variables - The Basics

, in Computing

Trains need careful marshalling, your threads are the same. I apologize for this illustration, it is bad even by my standards. In my defense, I drew it on an airplane.
Trains need careful marshalling, your threads are the same. I apologize for this illustration, it is bad even by my standards. In my defense, I drew it on an airplane.

Condition variables (CVs) are supposed to be the easy-to-use, hard-to-mess-up synchronization primitive in C++. There are still plenty of way to mess them up, some subtle enough that I often encounter issues in production code. This article is a gentle walk through some of the common errors.

Key Takeaways

Edge-Triggered vs Level-Triggered

These are terms from electrical engineering that I am stealing. A CV wait() is edge triggered - notifying a condition variable wakes up any wait()s that occurred before the notify() but crucially any wait() that occurs after the notify() will block. A notify()ed CV does not stay notified.

This means the following (nonsensical) code will always block.

void deadLocksInstantly()
{
    std::condition_variable cv;
    std::mutex mutex;

    std::unique_lock<std::mutex> lock(mutex, std::defer_lock);
    cv.notify_one(); // notify ourselves

    cv.wait(lock);  // never returns
    lock.unlock(); 
}

Here we notify the CV before we wait. The cv.wait() call doesn't care that the CV was notified, it will block and wait for the next one.

That was a contrived example but how about this:

std::thread t([&]()
              { 
                // do some work, notify the main thread
                cv.notify_one(); 
              });

// wait for our background thread to notify us
std::unique_lock<std::mutex> lock(mutex);
cv.wait(lock);

This code can also deadlock. We do not control the relative order of the statements between the threads and it is perfectly possible for the thread to start executing and call notify() before the main thread gets a chance to call wait().

On my MacBook M1 this happens about 1 in 500 times - a hard bug to catch in testing.

Using condition_variable::lock() With a Predicate

You almost certainly want to use the second form of cv::wait(), which takes a predicate function. This turns the edge-triggered blocking wait() into a level-triggered function that will not block if the predicate is met.

I always forget which way around the predicate function goes. The predicate function return false if the wait() needs to block.

Using std::atomic<> In A Predicate is a Code Smell

Knowing this, a developer might rewrite the previous example with a flag to signal that the background thread has finished its work. Not being a total noob, they use std::atomic<bool> for the flag because it will be shared between the threads.

std::atomic_bool workDone{false};

std::thread t([&]()
    {
      // do some work
      workDone = true;
      cv.notify_one(); 
    });

// wait for our background thread to notify us
std::unique_lock<std::mutex> lock(mutex);
cv.wait(lock, [&]() -> bool
        { return workDone; });

Unfortunately this code can also race despite the careful use of atomic<bool>. On my laptop this code deadlocks about 1 in 800 runs.

Remember that calling wait() with a predicate is equivalent to this:

while (!predicate()) {
  wait();
}

Consider the following sequence of events:

Main ThreadBackground Thread
Calls wait(predicate)
Runs predicate, workDone is false
Sets workDone = true
Calls notify_one()
Predicate returns false
blocks forever

I see more complicated versions of this code in PRs quite often, so much so that I consider any use of std::atomic<> in a condition variable predicate to be a code smell.

The one exception is that if you really just need to wait on a single value then c++20 has you covered with std::atomic<>::wait(), which allows you to wait() on the variable directly without mucking about with mutexes and locks.

std::atomic<bool> workFinished{false};

std::thread t([&]()
    {
        // do some work
        workFinished = true;
        workFinished.notify_one(); 
    });

workFinished.wait(false);

Protect All Variables Used in the Predicate With The Same Mutex

For more complex situations or if you can't use C++20, the solution is to protect any modification of any variable used in the predicate - no exceptions. You predicate will always be called with a lock holding the mutex but any other code must always explicitly lock even if the variables themselves are thread-safe.

bool shouldStop{false};  // no need for atomic

std::thread t([&]()
    {
        // do some work
        {
            std::lock_guard<std::mutex> lock(mutex);
            shouldStop = true;
        }
        cv.notify_one(); 
    });

std::unique_lock<std::mutex> lock(mutex);
cv.wait(lock, [&]() -> bool
        { return shouldStop; });

Other Primitives Exist

There are other ways to safely synchronize threads that might be more suitable.