The yodl
program is a single-pass interpreter of its input files. The
program does not build an internal data representation of the text and
commands that it encounters; therefore, all actions must be completed in one
pass. The basic way by which yodl
does this, is by reading input, taking
actions and possibly `pushing back' information on the input.
This is best illustrated with an example. Given the code,
DEFINESYMBOL(sym) IFDEF(sym) (Symbol sym is defined! DEFINEMACRO(testmac)(0)(Testmac now stands for one thing.) ) (Symbol sym is not defined! DEFINEMACRO(testmac)(0)(Testmac now stands for another thing.) ) testmac()
yodl
will take the following actions:
DEFINESYMBOL
command updates an internal table of symbols, no
other action is necessary here.
IFDEF
command is parsed, yodl
eats up all of the
necessary parts from the input: the keyword itself, and three parameter
lists. The IFDEF
test obviously yields `true', and therefore yodl
`pushes back' the true-list
Symbol sym is defined! DEFINEMACRO(testmac)(0)(Testmac now stands for one thing.)
yodl
sees normal text (Symbol
sym
is
etc.). This
is sent to the output file.
DEFINEMACRO
statement is seen, which defines
testmac()
as a placeholder for one
thing
.
testmac()
is seen. The parser recognizes a user-defined
macro having no arguments, which conforms to the definition. The text
Testmac now stands for one thing.
is pushed back.
yodl
reads this last string from its input and sends it
to the output.
The most complex part of the program is therefore the tokenizer (lexical analyzer): that part is responsible for matching tokens such as identifiers in the input, and for pushing back text. The pushing back is implemented as a relocatable buffer, which is expanded as text is pushed into it and which shrinks as the lexical analyzer processes input. I already achieved about 100% speed gain by optimizing the tokenizer, but this implementation can still be somewhat of a CPU hog. (But hey, premature optimzation is the root of all evil, says D.E. Knuth. So why optimize, as long as you don't know whether you're being premature about it?)
The lexical analyzer is furthermore responsible for the line continuation
feature (see section ??) and for the SUBST
mechanism (see
section ??). These mechanisms are also implemented with the push-back
method.
Please send Yodl questions and comments to yodl@icce.rug.nl.
Please send comments on these web pages to (address unknown)
Copyright (c) 1997, 1998, 1999 Karel Kubat and Jan Nieuwenhuizen.
Verbatim copying and distribution of this entire article is permitted in any medium, provided this notice is preserved.