Nibbles of Rust

The $crate Token

Rust has two systems for defining macros. There is the macro_rules! builtin macro, which consumes a special language designed for pattern matching on input syntax, and procedural macros.

Procedural macros are defined as functions exported from a special kind of Rust crate, which consume a TokenStream (sometimes two TokenStreams, see custom attributes), and return a new TokenStream. How that new TokenStream is used depends on whether the macro in question is a “derive macro” or not, but we’re not super concerned with that particular difference here.

macro_rules! my_macro_rules {
    ($x:literal) => {
        {$x + $x}
    };
}

// In a `proc-macro = true` crate:
use proc_macro::TokenStream;

#[proc_macro]
pub fn my_macro_is_procedural(x: TokenStream) -> TokenStream {
    x
}

With that background, let me pose you a question: How can a macro generate code which calls into functions defined somewhere other than the crate from which that macro is called?

You might guess that you can just name the required crate in the output, like some_crate::some_item(10). This wouldn’t be wrong, but it risks running afoul of hygiene. If your macro spits out some_crate::some_item(10), but there’s a mod some_crate declaration in the context, then your macro’s output will end up calling the wrong thing, or very possibly failing to compile entirely.

// In `muh_crate`, we have this macro definition:
#[macro_export]
macro_rules! owo {
    ($x:literal) => {
        some_crate::some_item($x)
    }
}

// In `some_crate`, we have this function definition:
pub fn some_item(_: i32) -> i32 {
    42 // the answer
}

// Now in some other crate, which depends on both of the above crates, we have this:
use muh_crate::owo;
mod some_crate;

fn find_answer() -> i32 {
    // Oh no! This macro expands to `some_crate::some_item(64)`,
    // which ends up trying to use the `some_crate` module we defined locally!
    owo!(64)
}

A decent first guess for how to fix this is to make your macro output ::some_crate::some_item(10) instead, note the :: in front. This makes the compiler perform resolution for some_crate in the top namespace, thereby avoiding any local modules which happen to share the name.

#[macro_export]
macro_rules! owo {
    ($x:literal) => {
        // Now `find_answer()` will compile!
        ::some_crate::some_item($x)
    }
}

That’s a pretty okay solution, and what the majority of procedural macros do. But we can do better, using a facility provided to macros defined using macro_rules!: The $crate special identifier.

The $crate token is a special thing which can be emitted by macro_rules! macros, which is guaranteed to refer to the crate in which the emitting macro was defined. We can use it to guarantee owo! uses the correct some_crate by making a public re-export for owo! to use.

#[macro_rules]
macro_rules! owo {
    ($x:literal) => {
        $crate::__private::some_item($x)
    }
}

/// Secret module of re-exports for use by our macros.
#[doc(hidden)]
pub mod __private {
    pub use some_crate::some_item;
}

But how is this better?

This works, but why not just use the ::some_crate::some_item version? It’s shorter and clearer about what it’s referring to when you read the macro’s definition!

Well.

Declaring Dependencies

Using ::some_crate::some_item requires that the user of the macro write some_crate in their Cargo.toml, while the $crate::__private::some_item version handles that transparently.

This doesn’t fully justify $crate’s existence, as the pattern could just as well work like ::muh_crate::__private::some_item. But, $crate is no less convenient in this case.

Crate Renaming

What truly makes $crate necessary is that Rust doesn’t actually guarantee that crates are presented to the compiler with the name written in their Cargo.toml. Instead, it is trivial to rename a crate on which you depend in your Cargo.toml, like so:

[dependencies]
lol_its_some_crate = { package = "some_crate", version = "0.0.1" }

When you do this, the name which is presented to the Rust compiler for this crate, when compiling your crate, becomes lol_its_some_crate, despite its naming by its author.

To handle its users doing this, a robust macro crate must re-export every item its macros’ output depends on, and refer to them through $crate.

What about procedural macros?

This story doesn’t have a happy ending. Function-like procedural macros can use this trick by having a wrapping macro_rules! macro which is defined in a crate which does the re-exports and which forwards the $crate token into the actual procedural macro, but attribute and derive macros have no such recourse. This is why most proc macros stop at using ::some_crate::some_item or ::muh_crate::__private::some_item.

Sad.

Writing Group

I was motivated to write this post by a writing group we formed in the RPLCS Discord. Other posts by our group are listed here: https://www.catmonad.xyz/writing_group/.

Edit (May 2nd)

Removed scare quotes around “macro” when introducing macro_rules!, as they obscured the meaning of the text.