Watch Ruby as it parses your code

I thought it would be fun to take a break from practical, useful content to show you a neat Ruby party trick.

Before Ruby runs your program it has to parse it. The parser is a kind of a state machine. And there is a little-known command line flag that you can use to make Ruby log everything the state machine does.

Take the following example:

a = 1 + 2

If I run this using the -y flag, I get the following output:

$ ruby -y sample.rb
Starting parse
Entering state 0
Reducing stack by rule 1 (line 903):
-> $$ = nterm $@1 ()
Stack now 0
Entering state 2
Reading a token: Next token is token tIDENTIFIER ()
Shifting token tIDENTIFIER ()
Entering state 35
Reading a token: Next token is token '=' ()
Reducing stack by rule 509 (line 4417):
   $1 = token tIDENTIFIER ()
-> $$ = nterm user_variable ()
Stack now 0 2
Entering state 113
Next token is token '=' ()
Reducing stack by rule 100 (line 1764):
   $1 = nterm user_variable ()
-> $$ = nterm lhs ()
Stack now 0 2
...
140 more lines

What we're seeing here is the Ruby parser cycling through each token in the file and performing the following operations:

• Add the token to a stack
• Compare the stack to a list of rules
  • If the token matches a rule, do a state transition
  • If no match, add another token to the stack and try again.

All of the states and rules are defined in parse.y a file that is processed by the bison parser generator to generate the actual parser which is in C.