Skip to content

Foundation Spec (for course authors / build agents)

This is the contract that every piece of course infrastructure and every chapter follows. The learner-facing overview is docs/PROPOSAL.md. Read that first, especially §5, §5.8 and §6.0.

0. Hard requirements

  • C++23 (CMAKE_CXX_STANDARD 23, no extensions). Use modern library features where they help: std::expected, std::print/std::format, ranges, std::span, std::string_view. No C++20 modules.
  • LLVM 23 only. find_package(LLVM 23.1 REQUIRED CONFIG) (LLVM matches major.minor exactly; the major version is hard-checked in cmake/PebbleLLVM.cmake). Use only modern APIs:
  • the new pass manager
  • the plugin header llvm/Plugins/PassPlugin.h with LLVM_PLUGIN_API_VERSION
  • opaque pointers
  • debug records
  • ORC LLJIT
  • No exceptions for control flow. LLVM is built with -fno-exceptions. Report errors with std::expected<T, pebble::Error> or llvm::Expected. Match LLVM's RTTI setting: if LLVM_ENABLE_RTTI is OFF, compile with -fno-rtti.
  • Platforms: macOS (primary: Apple Silicon, Homebrew llvm 23, Homebrew clang) and Linux. Never depend on Apple clang.
  • Python ≥ 3.11 for tooling, managed by uv: dependencies go in pyproject.toml (lit, PyYAML; dependency groups site = MkDocs Material + pymdown-extensions for the website, dev), exact versions in uv.lock (uv lock after every edit; CI runs uv sync --locked). Nothing else without a strong reason. ./course re-executes through uv run --project <repo> unless it already runs in .venv or COURSE_NO_UV=1.
  • Website: mkdocs.yml + tools/site/hooks.py + tools/course/site.py build the site in place from the repository's Markdown (./course serve, ./course site build → build/site, strict in CI).

1. Directory ownership

CMakeLists.txt, CMakePresets.json, cmake/          build system
.github/workflows/, .devcontainer/, docs/setup/   CI + environment
pebble/include/pebble/{Support,AST,Lex,PIR,Frontend,Driver}/  provided interfaces
pebble/lib/<Component>/                           learner code (skeleton with PEBBLE_TODO) + provided code
pebble/runtime/                                   C runtime (provided)
pebble/tools/{pebblec,pir-run,pir-opt}/           tools
labs/chNN-<name>/                                 standalone comparison labs (skeletons)
solutions/                                        mirrors pebble/ and labs/ paths for every learner-implemented file,
                                                  plus solutions/quizzes/chNN.yaml (plaintext quiz sources)
tests/chNN/{unit,lit}/                            per-chapter tests
tests/conformance/                                front-end conformance suite (PIR goldens + e2e)
chapters/NN-slug/                                 README.md (technique map), lessons/*.md, quiz.yaml,
                                                  flashcards.tsv, exercises.md
tools/course/  + ./course                         learner CLI (Python)
pyproject.toml, uv.lock, mkdocs.yml, tools/site/  Python environment (uv) + course website
docs/language/pebble-spec.md, docs/pir/pir-spec.md, docs/runtime-abi.md, docs/authoring/*

Chapter slugs (fixed): 00-architectures 01-lexing 02-top-down-parsing 03-bottom-up-parsing 04-parsing-in-practice 05-names-and-scopes 06-type-checking 07-type-inference 08-ir-design 09-llvm-ir 10-llvm-cpp-api 11-lowering-and-codegen 12-passes-and-testing 13-local-optimization 14-dataflow 15-dominance-and-loops 16-ssa 17-scalar-optimizations 18-loop-optimizations 19-alias-analysis 20-interprocedural 21-instruction-selection 22-register-allocation 23-scheduling 24-capstone

2. CMake API (implemented in cmake/Pebble*.cmake)

  • PEBBLE_USE_SOLUTION is a cache string: a ;- or ,-separated list of switch names, or all. Switch names are chapter-scoped components such as lexer, parser, ll1-toolkit, dominance.
  • pebble_add_component(<name> [SWITCH <switch>] [PROVIDED_SOURCES ...] [EXERCISE_SOURCES ...] [DEPS ...] [LLVM_COMPONENTS ...])
  • Creates the static library pebble_<name>.
  • PROVIDED_SOURCES always come from the current source dir.
  • EXERCISE_SOURCES come from the current source dir, or from solutions/<same relative path>/ when <switch> is selected.
  • Public include dir: pebble/include. If solutions/.../include exists for the component it is added first when switched, because solutions may add private headers.
  • pebble_add_lab(<name> CHAPTER chNN [SWITCH <switch>] PROVIDED_SOURCES ... EXERCISE_SOURCES ... [DEPS ...] [LLVM_COMPONENTS ...]) — the same, for labs/.
  • pebble_add_unittest(<name> CHAPTER chNN SOURCES ... [DEPS ...]) — a GoogleTest executable, registered with gtest_discover_tests, labels chNN;unit.
  • pebble_add_lit_suite(CHAPTER chNN [DEPENDS targets...]) — runs lit on tests/chNN/lit as one ctest test, labels chNN;lit. It uses the shared tests/lit.cfg.py with these substitutions:
  • LLVM tools: %opt %llc %lli %clang %FileCheck %not %llvm-as %llvm-dis
  • course tools: %pebblec %pir-run %pir-opt
  • %plugin, the path of the PebblePasses plugin
  • %runtime, the path of the runtime library
  • %S %s %t as usual
  • pebble_add_pass_plugin(<name> SOURCES ...) — a MODULE library. On Darwin it links with -undefined dynamic_lookup.
  • GoogleTest: find_package(GTest) first (Homebrew googletest, apt libgtest-dev), falling back to FetchContent.
  • LLVM linking: link LLVM (the dylib) when LLVM_LINK_LLVM_DYLIB is set or the shared library exists; otherwise use llvm_map_components_to_libnames.
  • Presets: macos (Homebrew clang/LLVM, via $(brew --prefix llvm)), linux (LLVM_DIR from the environment or /usr/lib/llvm-23), ci-solutions (PEBBLE_USE_SOLUTION=all). Each has a debug variant.

3. Skeleton convention

  • Learner-implemented function bodies call PEBBLE_TODO("chNN", "what to implement"). It is declared in pebble/Support/Todo.h, prints TODO(chNN): ... to stderr and exits with code 99. Skeleton code must compile, and its tests must fail with that message, not crash obscurely.
  • Every learner-implemented file has a byte-for-byte path mirror under solutions/.
  • Solutions keep the learner file's public API exactly. They are a drop-in replacement.

4. Container dev environment (this cloud sandbox only; not learner docs)

  • LLVM 23.1.2 lives in /opt/llvm-23 (from conda-forge). LLVM_DIR=/opt/llvm-23/lib/cmake/llvm. FileCheck is at /opt/llvm-23/libexec/llvm/FileCheck.
  • Compiler: CXX=/opt/llvm-23/bin/clang++-23, CC=/opt/llvm-23/bin/clang-23, with CXXFLAGS=--gcc-install-dir=/usr/lib/gcc/x86_64-linux-gnu/14 (libstdc++ 14 headers).
  • Runtime: binaries need /opt/llvm-23/lib on the rpath or on LD_LIBRARY_PATH (for its newer libstdc++ and libLLVM). The build should set CMAKE_BUILD_RPATH to include LLVM_LIBRARY_DIR.
  • lit comes from the uv-managed .venv (uv sync), and GoogleTest comes from apt. The canonical local build is: 
    cmake --preset linux -DLLVM_DIR=/opt/llvm-23/lib/cmake/llvm -DPEBBLE_FILECHECK=/opt/llvm-23/libexec/llvm/FileCheck

5. Chapter deliverables (every chapter agent)

  • The width-and-depth contract from docs/PROPOSAL.md §6.0, and docs/authoring/STYLE.md.
  • chapters/NN-slug/:
  • README.md: the technique map and comparison table
  • lessons/NN-*.md: one per technique family, each covering all nine depth-contract items
  • exercises.md
  • flashcards.tsv
  • quiz.yaml: built from solutions/quizzes/chNN.yaml with ./course quiz build chNN
  • Drills: tools/course/drills/<name>.py, registered with the chapter.
  • Code: skeleton + solution + tests/chNN/. You must show both:
  • PEBBLE_USE_SOLUTION=all: every chNN test passes.
  • The default skeleton build compiles, and the chNN tests fail with TODO(chNN).
  • Register the chapter in tools/course/chapters.yaml.

6. Integration notes (foundation hand-off)

  • Lexer (ch01) done: pebble_add_component(lex SWITCH lexer CHAPTER ch01 … EXERCISE_DIR src). The learner writes pebble/lib/Lex/src/; Token.cpp and LexFile.cpp are provided. The public API is unchanged.
  • The remaining front-end stages (Parser ch04, Sema ch05–07, LowerToPIR ch11) are currently PEBBLE_TODO stubs listed as PROVIDED_SOURCES, because no solutions exist yet. Each chapter agent must move its stage to EXERCISE_SOURCES with a SWITCH and add the solution file. Until then, .pbl compilation stops at the first TODO, even in solution mode.
  • Frontend::compile returns std::expected<FrontendOutput, DiagnosticList>, where FrontendOutput is {pir::Module Module; DiagnosticList Warnings;}.
  • Runtime ABI: the entry point is i64 @pebble_main(). Traps call pebble_trap(...), which exits with 101. pir-run exits with 70 on undefined behavior. Diagnostic codes live in pebble/Support/DiagnosticKinds.def (append only).
  • The -O1 course pipeline hook is driver::getCoursePipeline() in pebble/lib/Driver/CoursePipeline.cpp.
  • Pilot findings for Phase 3. Pass plugins are currently per chapter (PebbleCh15Passes), and analysis code lives in labs/ch15-dominance because pebble/lib/Analysis isn't wired in yet. Before Phase 3, decide whether to add a shared PebblePasses plugin and pebble/lib/Analysis (needed by Ch 16 and later chapters, which reuse dominance and liveness).
  • Portability rule (macOS): don't use floating-point std::from_chars; libc++ marks it unavailable before macOS 26. Use llvm::StringRef::getAsDouble. Integer from_chars, and std::format/std::print of doubles (macOS 13.3+), are fine.

7. Passes & analyses registry (chapters 12–24)

Supersedes the §6 pilot note: there is one pass plugin, PebblePasses, and one home for reusable analyses, pebble/lib/Analysis. Chapters written in parallel never edit a shared file: every hook below is picked up by a directory glob or a static self-registration.

Layout

pebble/include/pebble/Analysis/*.h       contract headers of shared analyses (Graph, DFS, Dominators, ... from Ch 15)
pebble/lib/Analysis/<Topic>/             one library per topic: pebble_analysis_<topic>  (Dominance = Ch 15)
pebble/lib/Analysis/CMakeLists.txt       globs */CMakeLists.txt; pebble_analysis = INTERFACE aggregate of all topics
pebble/include/pebble/Passes/Registry.h  registration macros + Registrar
pebble/lib/Passes/Registry.cpp           registry core (pebble_pass_registry)
pebble/lib/Passes/Plugin.cpp             llvmGetPassPluginInfo -> registerAll(PB)
pebble/lib/Passes/<Topic>/               one pass library per topic: pebble_passes_<topic>  (Dominance = Ch 15 printers)
pebble/lib/Passes/CMakeLists.txt         globs */CMakeLists.txt; builds pebble_passes (aggregate) + PebblePasses (plugin)
solutions/pebble/lib/{Analysis,Passes}/<Topic>/   solution mirrors
Use topic names (Dataflow, SSA, ScalarOpts, Loops, IPO, ...) that are unique to your chapter. A topic library depends on another through DEPS (for example, DEPS pebble_analysis_dominance); CMake resolves target names after configuration, so glob order does not matter.

CMake API (additions to §2)

  • pebble_add_component(... EXERCISE_DIR <dir>): every *.cpp below <dir> (recursive, CONFIGURE_DEPENDS) that is not a PROVIDED_SOURCE is learner code. This lets learners design their own files (minimal scaffolding). When the SWITCH is selected, the *.cpp files below solutions/<same path>/<dir> are compiled instead: the solution replaces the whole directory, and it may have a different file structure. A missing solution directory is a configure error. It can be combined with EXERCISE_SOURCES. The same options work for pebble_add_lab, pebble_add_analysis and pebble_add_passes.
  • pebble_add_analysis(<topic> CHAPTER chNN [SWITCH s] [PROVIDED_SOURCES ..] [EXERCISE_SOURCES ..] [EXERCISE_DIR d] [DEPS ..]) creates pebble_analysis_<topic> and adds it to pebble_analysis. It does not link LLVM, because it ends up in the plugin. Executables and tests add LLVM with pebble_link_llvm.
  • pebble_add_passes(<topic> CHAPTER chNN [GROUP g] ...same options...) creates pebble_passes_<topic> and links it into its group's aggregate with $<LINK_LIBRARY:WHOLE_ARCHIVE,...>, so self-registering objects are not dropped by the static linker. Group course (the default) is pebble_passes. Other groups are pebble_pass_group_<g> (used for tests). Always link the aggregate, never pebble_passes_<topic> directly. Mixing the two on one link line is a CMake error.
  • pebble_add_registry_plugin(<target> [GROUP g]) builds a plugin exposing a group. PebblePasses is pebble_add_registry_plugin(PebblePasses).
  • pebble_add_lit_suite(... SUITE <dir>) adds a second lit directory, tests/chNN/<dir>, run as ctest chNN.<dir>. With no PLUGIN, %plugin is PebblePasses, and it is now a build dependency of check-chNN.

Linking: pebble_driver links pebble_passes, so pebblec has every course pass built in. --passes=pebble-x and -O1 need no --pass-plugin, and optimizeModule calls pebble::passes::registerAll(PB) before loading any plugin. PebblePasses.so contains the same code. It does not link LLVM: it takes LLVM from opt, and on macOS it links with -undefined dynamic_lookup. Registry symbols have hidden visibility, so every binary or plugin keeps its own registry, even when several are loaded into one process.

Registering a pass: steps for a chapter agent

  1. Create pebble/lib/Passes/<Topic>/CMakeLists.txt: 
    pebble_add_passes(<topic> CHAPTER ch13 SWITCH <switch>
  PROVIDED_SOURCES ...            # optional (printers, wrappers)
  EXERCISE_DIR .                  # learner writes any *.cpp here
  DEPS pebble_analysis_dataflow)  # optional
    If you have an analysis library, add pebble/lib/Analysis/<Topic>/CMakeLists.txt with pebble_add_analysis(...) in the same way, and put its contract header in pebble/include/pebble/Analysis/.
  2. In the pass's .cpp (the learner's own file, or the solution in solutions/pebble/lib/Passes/<Topic>/), register the pass at namespace scope next to its definition: 
    #include "pebble/Passes/Registry.h"
PEBBLE_FUNCTION_PASS("pebble-dce", DCEPass);              // also MODULE/CGSCC/LOOP _PASS
PEBBLE_FUNCTION_ANALYSIS("pebble-liveness", LivenessAnalysis); // also MODULE/CGSCC/LOOP _ANALYSIS
PEBBLE_FUNCTION_PASS("print<pebble-liveness>", LivenessPrinter);
PEBBLE_REGISTER_PASSES(my_id) {                           // params / anything else; body sees R
  R.functionPass<LICMPass>("pebble-licm", [](llvm::StringRef P) -> std::optional<LICMPass> {...});
  R.passBuilder().registerPeepholeEPCallback(...);         // escape hatch
}
PEBBLE_COURSE_PIPELINE_STEP(1310, "function(pebble-dce)"); // optional: join -O1
    Names must be pebble-<name> or print<pebble-<name>>. pebble-x also accepts pebble-x<params>, and print<pebble-x> also accepts print<pebble-x;params>. When a parser returns std::nullopt, opt reports "unknown pass name". Analyses get require<pebble-x>/invalidate<pebble-x> automatically. A duplicate or badly prefixed name is a fatal error that names both source files. Passes should derive from llvm::RequiredPassInfoMixin/OptionalPassInfoMixin (LLVM 23).
  3. Test through %plugin: RUN: %opt -load-pass-plugin=%plugin -passes=pebble-dce -S %s | %FileCheck %s. In tests/chNN/CMakeLists.txt, use pebble_add_lit_suite(CHAPTER chNN). Unit tests link pebble_analysis_<topic> (plus pebble_link_llvm), or pebble_passes when they need the registrations.
  4. Check opt -load-pass-plugin=<build>/lib/PebblePasses.so -passes='print<pebble-passes>' -disable-output x.ll, which lists every registration (chapter, kind, name) and the effective -O1 pipeline.

Test contract under minimal scaffolding

A pass's contract is its pipeline name plus observable IR behavior, checked by lit + FileCheck. An analysis's contract is either its printer's documented output format (print<pebble-x>) or a small contract header in pebble/include/pebble/Analysis/ declaring the few functions unit tests call. Learners choose everything else: file names, classes and helpers. Skeleton directories may contain only a README or nothing at all. An empty EXERCISE_DIR yields an INTERFACE library and registers nothing, so the pass is simply "unknown" until the learner writes it. Tests of a pass not yet written fail with "unknown pass name 'pebble-x'". This is the expected skeleton failure for scaffold-free passes, alongside TODO(chNN) for provided stubs. Chapter docs should say so.

The -O1 course pipeline

driver::getCoursePipeline() (pebble/lib/Driver/CoursePipeline.cpp) joins all PEBBLE_COURSE_PIPELINE_STEPs linked into pebblec, sorted by order key and then by chapter. With no steps it is default<O1>. Order keys are chapter * 100 + slot (for example, Ch 13: 1300–1399), so a chapter's steps land in chapter order without anyone editing a list. Chapter 24 adjusts keys or replaces the function. Register steps only from code that works: the learner's finished pass or the solution file, never a provided skeleton stub. Otherwise -O1 would stop at a TODO in skeleton builds.

Chapter 15 migration (done)

  • The analyses moved from labs/ch15-dominance/{lib,include} to pebble/lib/Analysis/Dominance/ + pebble/include/pebble/Analysis/, as pebble_analysis_dominance (switch dominance). After the post-pilot upgrade, the learner surface is the contract header pebble/include/pebble/Analysis/Dominance.h plus src/Stub.cpp (EXERCISE_DIR src). Provided infrastructure lives in provided/ (Graph.h, DomTree.h). The solutions are in solutions/pebble/lib/Analysis/Dominance/src/. This is the reference example of the minimal-scaffolding layout for analysis chapters.
  • The printers and analyses moved to pebble/lib/Passes/Dominance/Ch15Passes.cpp, registered in PebblePasses. PebbleCh15Passes is gone, and tests/ch15 uses %plugin.
  • labs/ch15-dominance/ keeps only the comparison lab (ch15-dombench). The paths in the chapter's exercises, lessons and README were updated.

Self-test

tests/infra/registry/{ChapterA,ChapterB} are two independent stand-in chapters (ch98, ch99) in group infra. They cover module, CGSCC, function and loop passes and analyses, a printer, parameters, and -O1 steps whose order key beats chapter order. They are loaded through PebbleInfraRegistryPlugin (infra.registry-lit) and linked statically in infra.PassRegistry.*, which also checks the duplicate-name and prefix errors.