Module
lexie
src/lexie.pookaTeach a computer to read your language, one word at a time.
Lexie turns a token specification, written in TOML, into plain Pooka
modules: a Token family, a lexer, source positions and, if you ask, a
syntax highlighter. The generated code needs nothing but the standard
library, so lexie is only ever a development dependency.
Most of the time you will not call this module yourself. Add lexie to a
project and run it with pooka run -m lexie; the guide shows how. The
functions here are for tools that want the files without the disk:
generate checks a specification and returns the files to write, or
every problem at once.
import lexie
import lexie/config
import pooka/list
let assert Ok(files) = lexie.generate(config.default_config())
assert list.map(files, fn(f: lexie.GeneratedFile) { f.path }) == [
"src/location.pooka",
"src/token.pooka",
"src/lexer.pooka"
]
API reference
class GeneratedFile(path: String, content: String)
A file to write: where it goes, relative to the project, and what it says.
path: String
Stringcontent: String
String
fn(String, String) -> lexie.GeneratedFilegenerate(c: Config) -> Result(List(GeneratedFile), List(ConfigError))
fn(lexie/config.Config) -> Result(List(lexie.GeneratedFile), List(lexie/config.ConfigError))Check a specification and generate its files: location.pooka,
token.pooka and lexer.pooka, plus highlighter.pooka when the
highlighter is enabled. Nothing is written; that is up to the caller.
A specification with problems produces no files at all, and every problem comes back at once, so they can be fixed in one sitting.
import lexie
import lexie/config
let result = lexie.generate(config.default_config())
// Ok(files), ready to write; no files have been touched yet.
read(text: String) -> Result(#(Config, List(ConfigError)), String)
fn(String) -> Result(#(lexie/config.Config, List(lexie/config.ConfigError)), String)Read standalone TOML text, retaining every configuration problem.
import lexie
let text = "[keywords]\nitems = [\"let\"]\n"
let result = lexie.read(text)
// Ok(#(specification, problems)); generate validates before emitting code.
is_generated(content: String) -> Bool
fn(String) -> BoolIs this file one lexie generated? Only those are ever overwritten or removed: lexie looks for its marker on the very first line.
import lexie
assert lexie.is_generated("//// Code generated by lexie. DO NOT EDIT.\n")
assert !lexie.is_generated("module lexer\n")