Skip to content

★ Lab 24.3 · A Pebble front end in Rust, held to the conformance suite

Chapter: 24 · The Complete Pebble Compiler — and Beyond · Lessons: 1–11 once more, in another language; the protocol is docs/architecture.md §4–§6 · Time: 20–40 hours · Tests: ch24.rust-e2e-ch11, ch24.rust-e2e-ch24 (labels ch24, star; configured only when cargo is on PATH)

1. Goal

The first ten chapters wrote a front end in C++ against LLVM's ADT. This lab writes a second one, in Rust, with no LLVM and no C++: an ordinary cargo binary that reads a .pbl file and prints a PIR module. pebblec --frontend=<path> runs it out of process, verifies the PIR it prints, and continues with the same code generator and pipeline as for the C++ front end. The definition of done is the conformance suite: every program of tests/ch11/e2e/ and tests/ch24/e2e/, compiled through the Rust front end, behaves exactly as through the C++ one (stdout, stderr including trap messages with their columns, exit status), at every optimization level.

2. The contract

The out-of-process protocol of docs/architecture.md §4:

pebble-frontend-rust --emit=pir --pir-version=1.0 [<frontend-arg>...] <input-path>
  • stdout: a complete PIR module in canonical form (docs/pir/pir-spec.md), nothing else;
  • stderr: diagnostics, one JSON object per line (§5: severity, message, and code, file, line, column when known);
  • exit status 0 (success: stdout is parsed and verified by pebblec; invalid PIR is E0003 and never a silent miscompile), 1 (compile errors), 2 (--emit=tokens/ast unsupported).

The crate here (Cargo.toml, src/) is the reference implementation: lexer.rs, parser.rs (recursive descent with precedence climbing), ast.rs, sema.rs (scopes, bidirectional type checking, definite assignment, reference parameters and exclusivity (spec §9.3), the E0xxx codes of docs/language/pebble-spec.md §13), pir.rs (a builder that mirrors pebble/include/pebble/PIR/Builder.h: numbered locals, blocks, the current @line:col), lower.rs (the reference lowering scheme of spec §15), diag.rs, main.rs. Write your own in a copy of the crate, or replace the reference's modules one at a time, lexer first: the module boundaries are the chapter boundaries.

3. Requirements

  • R1 (protocol). The command line, streams and exit statuses above; --pir-version and unknown -- arguments are accepted and ignored; a non-UTF-8 file is E0109; an unreadable file is E0001.
  • R2 (conformance). All 66 programs of tests/ch11/e2e/ and the 8 of tests/ch24/e2e/ pass pebble-e2e.py with --param frontend=<binary>: the PIR verifies, pir-run and the native executables agree with the EXPECT-* lines, at -O0, -O1 and -O2.
  • R3 (positions). Every PIR statement carries the @line:col of the source construct that the spec's lowering scheme assigns (spec §15), so that trap messages name the same column as the C++ front end: e2e-trap-*.pbl compare pebble: trap: ... at file:line:col exactly, and columns count bytes.
  • R4 (diagnostics). The error programs of Chapters 5–7 (tests/ch05/lit/err-*.test, tests/ch06/lit/err-*.test, tests/ch07/lit/err-infer.test and their Inputs/) are rejected with the same primary error code and position as the C++ front end reports: pebblec --frontend=<binary> --diagnostics=json shows the code your front end printed. Spot-check at least E0104/E0105 (lexing), E0201/E0203 (parsing), E0301–E0315 (names and scopes), E0401–E0424 (types and references), the codes the reference crate emits.
  • R5 (self-contained). cargo build --release with no dependencies beyond the standard library (the reference uses none); the binary is found as pebble-frontend-rust on PATH (--frontend=rust) or by path.

4. What the tests check

When cargo is found at configure time, CMake builds the crate as the target pebble-frontend-rust, copies the binary next to pebblec, and adds two ctest entries that run pebble-lit with --param frontend=<binary> over tests/ch11/e2e and tests/ch24/e2e (R2). Without cargo nothing is configured, and the ★ tests are simply absent. R1, R3 and R4 are checked by hand:

cargo build --release --manifest-path labs/ch24-rust-frontend/Cargo.toml
FE=$PWD/labs/ch24-rust-frontend/target/release/pebble-frontend-rust
build/linux/bin/pebblec --frontend=$FE --emit=pir tests/ch11/e2e/prog-sieve.pbl -o /tmp/s.pir && build/linux/bin/pir-run /tmp/s.pir
build/linux/bin/pebblec --frontend=$FE --diagnostics=json tests/ch06/lit/Inputs/<an error program>.pbl
build/linux/bin/pebble-lit --param frontend=$FE tests/ch11/e2e tests/ch24/e2e

5. Milestones

  1. Protocol. A main that prints the fixed module of docs/architecture.md §6 for any input and exits 0; pebblec --frontend=$FE x.pbl links and runs it.
  2. Lexer. Spec §3: tokens, positions in bytes, string escapes and interpolation pieces; E01xx.
  3. Parser and AST. Spec §4; precedence climbing for §8's table; E02xx.
  4. Sema. Scopes and mutability (§6), definite assignment and missing returns (§12), bidirectional checking with the literal rule (§8.1, §5), reference parameters and exclusivity (§9.3); E03xx/E04xx. Run Chapters 5–7's error programs through both front ends and diff the codes.
  5. Lowering. The scheme of §15: evaluation order, checked operators as saddo/ssubo/smulo plus assert, bounds checks, short-circuit as control flow, the copy rule for aggregates, @line:col on every statement. e2e-arith-*, then e2e-trap-*, then everything.
  6. Conformance. Both suites green through pebble-lit --param frontend=.

6. Hints

  1. Mirror the C++ front end's data structures where the spec does not force a choice; where it does (§13's codes, §14's dump formats if you implement --emit=tokens), follow the spec, not the C++ code.
  2. Print PIR through one builder type; never format instructions by hand in the lowering. The verifier (pir-opt --verify-only) is your unit test for the builder.
  3. pir-run is the oracle for the lowering; the C++ front end's --emit=pir is the oracle for the text: diff <(pebblec --emit=pir x.pbl -o -) <(pebblec --frontend=$FE --emit=pir x.pbl -o -) after canonicalization with pir-opt.
  4. Columns: count bytes, not chars; a tab is one byte. Which position a checked operator's assert carries is fixed by the lowering scheme (spec §15) and visible in the C++ front end's --emit=pir output: diff it before running the trap tests.
  5. The reference crate's sema.rs is the largest module for a reason: the exclusivity rule needs the set of live borrows per scope, and definite assignment needs a small dataflow over the AST's control flow (if joins, loops).

7. Stretch goals

  • --emit=tokens and --emit=ast in the spec's §14 formats, checked against the C++ dumps with diff.
  • A --diagnostics=text mode that renders diagnostics with the source line and a ^~~~ marker, so the crate is usable standalone.
  • Emit LLVM IR directly with inkwell and compare with pebblec --from-llvm (docs/runtime-abi.md §8): what you lose (the verifier, the interpreter oracle) is the lesson.