Advanced Techniques

Tips and tricks for writing complex macro_rules.

Counting with null

Let’s write a macro that counts the number of token trees in its input. We’ll do this by replacing each token tree with 1 + and then ending it of with a 0.

We can write a recursive macro to recursively replace the first token tree, one-by-one:

Source

# coding: macro-polo

macro_rules! count_tts_recursive:
    [$t:tt $($rest:tt)*]:
        1 + count_tts_recursive!($($rest)*)

    []: 0


print(count_tts_recursive![a b c d e])

Expanded

print(1 + 1 + 1 + 1 + 1 + 0)

Output

$ python3 count_tts_recursive.py 
5

Alternatively, we can use the null capture type to “count” the number of tts, and then emit the same number of 1 +s, all in one go:

Source

# coding: macro-polo

macro_rules! count_tts_with_null:
    [$($_:tt $counter:null)*]:
        $($counter 1 +)* 0

print(count_tts_with_null![a b c d e])

Expanded

print(1 + 1 + 1 + 1 + 1 + 0)

Output

$ python3 count_tts_with_null.py 
5

Matching terminators with negative lookahead

Let’s write a macro that replaces ;s with newlines.

Source

# coding: macro-polo

macro_rules! replace_semicolons_with_newlines_naive:
    [$($($line:tt)*);*]:
        $($($line)*)$^*

replace_semicolons_with_newlines_naive! { if 1: print(1); if 2: print(2) }

Expanded

if 1 :print (1 );if 2 :print (2 )

Output

$ python3 replace_semicolons_with_newlines_naive.py 
  File "/home/docs/checkouts/readthedocs.org/user_builds/macro-polo/checkouts/latest/docs/intro/macro_rules/_scripts/replace_semicolons_with_newlines_naive.py", line 2
    
    ^
SyntaxError: invalid syntax

When we try to run this, however, we get a SyntaxError.

If we look at the expanded source code, we notice something strange: the input is left completely unchanged!

The reason for this is actually quite simple: the $line:tt capture variable matches the semicolon, so the the entire input is captured in a single repition (of the outer repeater). What we really want is for $line:tt to match anything except ;, which we can do with a negative lookahead:

Source

# coding: macro-polo

macro_rules! replace_semicolons_with_newlines_naive:
    [$($($[!;] $line:tt)*);*]:
        $($($line)*)$^*

replace_semicolons_with_newlines_naive! { if 1: print(1); if 2: print(2) }

Expanded

if 1:
    print(1)
if 2:
    print(2)

Output

$ python3 replace_semicolons_with_newlines.py 
1
2

Notice the addition of $[!;] before $line:tt. Now when we run this code, we get the output we expected.