In 2017, I said that “asynchronous Rust programming is a disaster and a mess”. In 2021 a lot more of the Rust ecosystem has become asynchronous – such that it might be appropriate to just say that Rust programming is now a disaster and a mess. As someone who used to really love Rust, this makes me quite sad.

I’ve had a think about this, and I’m going to attempt to explain how we got here. Many people have explained the problems with asynchronous programming – the famous what colour is your function essay, for example.1 However, I think there are a number of things specific to the design of Rust that make asynchronous Rust particularly messy, on top of the problems inherent to doing any sort of asynchronous programming.

In particular, I actually think the design of Rust is almost fundamentally incompatible with a lot of asynchronous paradigms. It’s not that the people designing async were incompetent or bad at their jobs – they actually did a surprisingly good job given the circumstances2 I just don’t think it was ever going to work out cleanly – and to see why, you’re going to have to read a somewhat long blog post!

A study in async

I’d like to make a simple function that does some work in the background, and lets us know when it’s done by running another function with the results of said background work.

`use std::thread;

/// Does some strenuous work "asynchronously", and calls func with the /// result of the work when done. fn do_work_and_then(func: fn(i32)) { thread::spawn(move || { // Figuring out the meaning of life... thread::sleep_ms(1000); // gee, this takes time to do... // ah, that's it! let result: i32 = 42; // let's call the func and tell it the good news... func(result) }); }`

There’s this idea called “first-class functions” which says you can pass around functions as if they were objects. This would be great to have in Rust, right?

See that func: fn(i32)? fn(i32) is the type of a function that takes in one singular i32 and returns nothing. Thanks to first-class functions, I can pass a function to do_work_and_then specifying what should happen next after I’m done with my work – like this:

fn main() { do_work_and_then(|meaning_of_life| { println!("oh man, I found it: {}", meaning_of_life); }); // do other stuff thread::sleep_ms(2000); }

Because do_work_and_then is asynchronous, it returns immediately and does its thing in the background, so the control flow of main isn’t disrupted. I could do some other form of work, which would be nice (but here I just wait for 2 seconds, because there’s nothing better to do). Meanwhile, when we do figure out the meaning of life, it gets printed out. Indeed, if you run this program, you get:

oh man, I found it: 42

This is really exciting; we could build whole web servers and network stuff and whatever out of this! Let’s try a more advanced example: I have a database I want to store the meaning of life in when I find it, and then I can run a web server in the foreground that enables people to get it once I’m done (and returns some error if I’m not done yet).

`struct Database { data: Vec<i32> } impl Database { fn store(&mut self, data: i32) {; } }

fn main() { let mut db = Database { data: vec![] }; do_work_and_then(|meaning_of_life| { println!("oh man, I found it: {}", meaning_of_life);; }); // I'd read from db here if I really were making a web server. // But that's beside the point, so I'm not going to. // (also db would have to be wrapped in an Arc<Mutex<T>>) thread::sleep_ms(2000); }`

Let’s run this…oh.

error[E0308]: mismatched types
  --> src/
27 |       do_work_and_then(|meaning_of_life| {
   |  ______________________^
28 | |         println!("oh man, I found it: {}", meaning_of_life);
29 | |;
30 | |     });
   | |_____^ expected fn pointer, found closure
   = note: expected fn pointer `fn(i32)`
                 found closure `[closure@src/ 30:6]`

I see.

Hang on a minute…

So, this is actually quite complicated. Before, the function we passed to do_work_and_then was pure: it didn’t have any associated data, so you could just pass it around as a function pointer (fn(i32)) and all was grand. However, this new function in that last example is a closure: a function object with a bit of data (a &mut Database) tacked onto it.