Parasol 0.9.0  ·  compiler internals

From .psol to .sol

A module declares its own grammar in its header, and that header is the only thing that changes how the rest of the file parses. Here is the whole path, traced through one real file.

trace.psol → parasol → trace.sol + .sol.map → solas → trace.sob → solvm

Parasol is a translator. A .psol goes in, Solveig source comes out, and it never runs a program — not yours, and not its own. Everything below happens in one pass over one file.

DIALECT FILE lib/control.psol @use TRACE.PRO HEADER · DIRECTIVES @language solveig. @use "control.psol". BODY · STATEMENTS n := #3. while n > #0 do (…). if total > #5 then … builds the tables DIALECT precedence table 10 infix · 1 prefix form table 6 forms · 2 op templates fills infix · unary · form tokens STEP 3 READER reader.c · 1442 lines tree — a MACRO node at each form use STEP 4 EXPANDER substitute · hygiene · spans expanded tree — every node carries a span STEP 5 EMITTER emit.c — the only file trace.sol Solveig source trace.sol.map back to the .psol then solas, then solvm. Parasol is done here.
The edge that matters is the violet one. The header is read first and builds the dialect; the dialect then configures the reader, before a single statement of the body is parsed. Everything else on the page is a consequence of that ordering.
declared syntax — header, tables, templates written code — your statements and arguments

That pairing runs through every figure below.

00 A file is two halves

The core grammar is module = { directive } { statement }. Every @-directive comes first, then the code. Nothing after the first statement can change the syntax.

trace.psol

@language solveig.
@use "control.psol".

n := #3.
total := #0.
while n > #0 do (total := total + n. n := n - #1).
if total > #5 then total:print else "small":print.

The one exception proves the rule: @include is Solveig's own directive, so Parasol treats it as a statement and passes it straight through.

01 Gather the unit

@use "control.psol" pulls in the dialect file, which itself uses arith.psol. A file is looked for beside the one using it, then each -I directory, then PARASOL_PATH — and it is read once, so a diamond costs nothing and a dialect never collides with itself.

All the files share one offset space. That detail looks like bookkeeping until step 5, where it is the reason a single generated line can name two origins.

02 The header builds two tables

Reading top to bottom, the directives fill a ParasolDialect. For this file the chain is control.psol → arith.psol, and it comes to eleven operators and six forms.

directivelands inas
@infix + 60 add.precedence table+ binds at 60, becomes :add
@infix > 40 greaterThan.precedence table> binds at 40, becomes :greaterThan
@infix /\ 30 => …bothan operator with a template is a form, and is registered as one
@syntax while <t> do <b> => …form tableword while, two holes, plus the template
@syntax if <c> then <a> => …form tableword if, two parts
@syntax if <c> then <a> else <b> => …form tableword if, three parts

Two if forms under one word, and nothing anywhere declares which is meant. The next step is why that is fine.

These tables are all that varies. The token set and the rest of the grammar are the same in every .psol there will ever be: #3, 'sym, x:msg(y), { }, . between statements. A dialect gets the characters that mean nothing until it says so — and |, :, . and , are never among them.

03 Parse the body, consulting the tables

The parser has exactly three holes in it — infix, unary and form — and the dialect fills them. It hits while, finds it in the form table, and a declared form beats anything else that name could be. Otherwise while would just be an identifier and f(x) would mean x:f, which is exactly what happens to every name the header did not declare.

Then it matches the pattern, and this is the part worth being precise about.

TOKENS → if total > #5 then total:print else "small":print if <c> then <a> complete here — but the next token is `else` ✕ if <c> then <a> else <b> wants `else` here, and gets it ✓ one token settles it — no backtracking ALIVE ALIVE
Every form under one word advances together, candidates dropping out as tokens arrive. Both readings of if stay alive through <c> then <a>; the following token decides. Had it been anything but else, the marks swap. Two forms can therefore only part company on a word — which is why the roadmap's optional-parts item insists an optional part must begin with one.

What comes out is a tree with an unexpanded MACRO node at each use, tagged with the declaration that matched. Operators resolve in the same pass, so n > #0 inside the form is already n:greaterThan(#0) before expansion starts.

A hole can also say what it will take — expression, name, literal, block or place. All five are decided by looking at what was parsed, so none of them needs an evaluator. That is why swap #1 and u is refused at the use rather than after expansion with a trail leading into the library.

04 Expand

parasol_expand replaces each MACRO with an instance of its template. Three things happen at once, and they happen together because doing any of them later would mean doing the others again.

TEMPLATE · lib/control.psol:24 { t }:whileTrue({ b }) the braces are the template's doing THE USE · trace.psol:6 while n > #0 do (…) t ← n > #0    b ← (…)   unevaluated expand THE TREE THAT COMES OUT SPAN POINTS AT send whileTrue control.psol:24 block control.psol:24 send greaterThan trace.psol:6 name n trace.psol:6 integer 0 trace.psol:6 block control.psol:24 sequence (…) trace.psol:6 A violet skeleton with teal grafts. The graft keeps the span it arrived with, so an error inside an argument points at the argument.
Substitution, hygiene and provenance, in one pass. Run parasol --tree on the file and this is literally what prints: the whileTrue node carries offset @1178, in control.psol; the greaterThan beneath it carries @70, in trace.psol. Each node also records scope (which expansion) and from (which form), so a diagnostic can print the whole trail.

Hygiene, and why it shipped in the same commit

The swap form binds a t. So does the caller. Every name a template binds is renamed to one nothing in the module uses:

written

t := #1.
u := #2.
swap t and u

emitted

t := #1.
u := #2.
{ | t__1 |
    t__1 := t.
    t := u.
    u := t__1 }:value.

Without the rename that swap destroys a value instead of exchanging one. Hygiene went in with forms rather than after them, because a system that expands without it grows programs that depend on the capture — and those programs are what make hygiene impossible to add later.

It terminates, and not because a counter says so

A template is read under the header as it stood at its own line, so it can only mention forms declared above it. Expanding form N yields uses of forms below N and never of N itself, and the highest index strictly falls. That is a property of reading the header top to bottom, not a limit bolted on afterwards.

05 Emit, and the map

Only emit.c has ever heard of Solveig — the lexer, the dialect table, the tree and the spans are all substrate-agnostic. It walks the expanded tree and prints:

trace.sol

; Generated by Parasol 0.9.0. Edit the .psol, not this.

n := #3.
total := #0.
{ n:greaterThan(#0) }:whileTrue({ (total := total:add(n). n := n:sub(#1)) }).
total:greaterThan(#5):ifElse({ total:print }, { "small":print }).

And alongside it, the reason those spans were carried all the way through.

TRACE.SOL · LINE 5 { n:greaterThan(#0) }:whileTrue({ … }) TRACE.PRO:6:7 n > #0 the condition you wrote LIB/CONTROL.PRO:24:40 { t }:whileTrue({ b }) the template it was put inside TRACE.SOL.MAP generated  ·  source  ·  offset  ·  file 5:1    24:40   1172   lib/control.psol 5:3    6:7     68
Generated line 5 has two origins, and the map knows which is which. Column 1 came from the template; column 3 came from your line 6. That is the payback for carrying a span on every node through expansion — without it, generated code is where the trail goes cold.

From here Parasol is finished. solas turns the source into a .sob and solvm runs it; the two projects meet at a file format and a command line, and that is the whole of the coupling.

Not a preprocessor

Parasol sits where a precompiler sits — before the real compiler, emitting source, never running your program. The mechanism is a different thing entirely.

a preprocessor (cpp, m4)Parasol
operates ontokens or text, with no grammara tree, matched inside the parser
bad outputcan emit something that will not parsecannot — expansion is over a parse that succeeded
capturenone; SWAP eats your tevery template-bound name renamed per expansion
positions#line, by handa span on every node, and a map
extension pointa substitutionthe grammar itself — precedence and pattern shape

The nearer relatives are syntax-rules, macro_rules! and Dylan — hygienic syntax macros. The line Parasol keeps that Lisp does not is that nothing runs at expand time. A template is a pattern and a tree, never a computation. That single restriction is what the two hardest open questions are both about: whether a guard may evaluate anything (it may not, or parsing would depend on running the file), and whether the expander may fold #32:sub(#17) when integer:sub is a slot a program may reassign.

Three invariants

Every decision on the roadmap is downstream of one of these.

  1. The header is the only thing that varies, and it is read, never run. Any tool can tokenise and parse any .psol top to bottom without executing it. Forth and TeX moved that line and became languages no tool can read; if it ever moves here, it moves at the module boundary and nowhere else.
  2. Matching happens while parsing, not afterwards. There is no scan-and-rewrite pass over finished code. That buys single-token decisions and no backtracking — and it is also why choosing the wrong shape is silent, a pattern where a call was meant being a perfectly legal parse.
  3. Expansion is one pass, and it carries provenance out with it. Substitution, hygiene and spans arrive together, because each one done later would mean redoing the other two.

A dialect gets the characters that mean nothing until it says so. The answer to a want the fixed vocabulary cannot spell is to grow the vocabulary — not to make the vocabulary declarable.