Chapter 2: Yodl User Guide

This section describes the yodl program from the point of a meta-user, one who is interested in how macro files work, or one who wants to write a new converter. If you're just interested in using Yodl with the pre-existing converters and macro files, skip this chapter and continue with the macro package description (chapter 3).

The yodl program the main converter of the Yodl package. The basic usage of the yodl program, yodl's built-in macros, and the syntax of the Yodl language is described here.

2.1: Using the yodl program

The yodl program is reads one or more input files, interprets the commands therein, and writes one output file. The program is started as:

yodl flags inputfile [inputfile...]

In this specification, the flags are optional and may be:

  • -D name: Defines name as a symbol. This flag is similar to the DEFINESYMBOL statement which may appear in the text. The purpose of defining symbols is described later.

  • -I directory: This defines the system-wide include directory where yodl searches for its input files when they cannot be located by their bare name. E.g. a statement to include a given file:

    INCLUDEFILE(latex)
    

    will search for the file latex in the current directory, and when that fails, in the system-wide include directory. The system-wide include directory is typically the place where the maintainer of a system stores macro-files for Yodl. This searching process applies to files that are included inside a document but also applies to filenames on the command line when invoking the yodl program.

    The name of the included file (latex in the above example) is the bare name, the yodl program will supply a default extension if necessary.

    The -I flag overrules Yodl's built-in name for the system-wide include directory. The built-in name is defined when compiling Yodl, and is usually /usr/local/lib/yodl.

  • -l number: This flag controls the live data policy; number is default 0. This means that macros that can access your operating system (these are the macros SYSTEM and PIPETHROUGH) are disabled. The number valuse can be:

  • 0: (the default): No potentially harmful macros are allowed.

  • 1: The macros can be executed, but only after user confirmation is obtained. The macros in question are shown while the Yodl document is processed.

  • 2: The macros are executed, but also shown during the Yodl run.

  • 3: The macros are executed, and not shown during the run. Use -l3 only when a document has proven `unharmful'.

  • These macros are further discussed in the sections 2.3.32 (the SYSTEM macro) and 2.3.25 (the PIPETHROUGH macro).

  • -o file: This option causes Yodl to write its output to file. By default, the output goes to the standard output stream. E.g., you can use the yodl program to read a file input and to write to output with the following two commands:

    yodl input > output
    yodl -ooutput input
    

  • -P cmd: This flag `pre-loads' the string cmd to the yodl processor. It is as if the cmd were before any actual input.

    More than one -P cmd flags may be present on the command line. Each of the commands is sent to the processor before any input is read.

  • -p maxpass: This flag causes Yodl to abort when the number of parsing cycles exceeds maxpass, which is usually 20. Exceeding this number usually means a circular definition somewhere in the document. This is the case when, e.g., a macro a expands to b, and b expands again to a. Alternatively, the maxpass cycle can be exceeded when a file a.yo includes b.yo, while b.yo includes a.yo etc..

  • -t: This flag enables tracing: while parsing, Yodl writes its output to the standard error stream. As is the case with the -k flag, this option is for debugging purposes.

  • -v: This flag increases Yodl's `verbosity level' and may occur more than once. The presence of one -v causes Yodl to show which files are parsed; four -v's show the inner workings of Yodl's lexical analyzer.

  • -w: The presence of this flag caused Yodl to warn when, e.g., symbols are redefined.

  • The inputfile elements on the command line specify which files Yodl should process. All names are supplied with an extension (this extension is defined in the installation of Yodl and is usually .yo). The files are then searched for `as-is', or in the system-wide include directory.

    Note that all filenames on the command line are input files. To define an output file, either use the -o flag or redirect the output.

    2.2: The Yodl grammar

    The grammar which is used by the Yodl system mixes `real' text that should appear on the output with commands for Yodl. The commands must follow a certain grammar, which is described in this section.

    2.2.1: Language elements

    The basic elements of the Yodl grammar are identifiers, names, numbers and parameter lists.

    Identifiers
    are names that can have a special meaning in the Yodl language. E.g., the sequence INCLUDEFILE is such an identifier: when followed by a filename in parentheses, Yodl will take some special action (in this case, read the file).

    Identifiers may consist of uppercase or lowercase characters. No other characters may appear in them.

    A name
    is a `one-word' sequence, i.e., a series of characters where no spaces, tabs or newlines appear. E.g., the Yodl language requires a name to follow the INCLUDEFILE identifier, enclosed in parentheses.

    Numbers
    consist of digits. Yodl seldom uses numbers, but they are required.

    Parameter lists
    are the arguments that names or numbers that are arguments to a Yodl command. A parameter list must be enclosed by parentheses.

    Basically, Yodl only does `something special' when it encounters a recognized identifier, followed by a parameter list. Sometimes an identifier may be followed by more than one parameter list, in that case, the action requires more than one argument. The grammar of Yodl requires that the identifier and its first parameter list follow right after each other, without intervening spaces. That means that

    somemacro (argument)
    

    will not start any Yodl action (supposing that somemacro is the name of a defined macro), since the parameter list and the identifier are separated by a space. The Yodl grammar is very strict in this sence; this rule however simplifies the parsing process of the input.

    To make life easier for those who write documentation using Yodl, there may be spaces or newlines between the parameter lists (i.e., between the first and second parameter list, between the second and third, and so on). Therefore, assuming that somemacro now is defined as a macro having three arguments, the following will work:

    somemacro(argument one)
        (and another argument, number two)
        (and the last one)
    

    Summarized, an identified macro is only expanded by Yodl when it is followed by a parameter list without spaces in between. Further parameter lists may follow separated by spaces, tabs or newline characters. All required parameter lists must however be in the same input file.

    2.2.1.1: Unbalanced parameter lists

    Yodl recognizes the arguments to a macro as parameter lists, i.e., delimited by ( and ). As long as the number of opening and closing parentheses matches, Yodl will correctly recognize the list. E.g., given a hypothetical macro somemacro, the following code sample shows the macro followed by one parameter list:

    somemacro(Here is a chunk of text.)
    somemacro(Here is a some (more) text.)
    

    A problem arises when the number of parentheses is unbalanced: i.e., when the parameter list consists of more opening than closing parentheses or v.v.. To handle such situations, Yodl offers a `literal-character' mechanism (see the CHAR macro in 2.3.3) and a `global substitution' mechanism (see the SUBST macro in 2.3.30). For example, to send the text

    here's a ")" closing parenthesis
    

    as an argument to our hypothetical macro somemacro, the following can be used:

    COMMENT(-- Alternative 1: using CHAR --)
    somemacro(here's a "CHAR(41)" closing parenthesis)
    
    COMMENT(-- Alternative 2: using SUBST --)
    SUBST(closepar)(CHAR(41))
    .
    .
    somemacro(here's a "closepar" closing parenthesis)
    

    Both methods have disadvantages: the CHAR method requires you to remember that an ASCII 41 is a closing parenthesis. The SUBST method defines a string closepar that is always expanded to a closing parenthesis, wherever in the text it occurs. Nevertheless, unbalanced parameter lists can be handled by Yodl. (If the here described method proves to be too much of a pain, I'll think of something. As for myself, I've never yet have felt the need to use an unbalanced parameter list -- except in this document..) Also, remember that unbalanced parenthesis pairs are only relevant in argument lists. Yodl handles parentheses in normal text as ordinary characters.

    2.2.2: Line continuation

    To make the typing of input easier, Yodl allows you to end a line with a backslash character \ and to continue it on the next line. That way you can split long lines to fit your screen. When processing its input, Yodl will treat these lines as one long line, and will of course ignore the \ character. This feature only works when the \ character is the last one on the line (no spaces may follow).

    When the line following the one with the \ character has leading spaces, then these are omitted. This allows you to `indent' a file as you wish, while the space characters of the indentation are ignored by the yodl program.

    A trivial example is the following:

    Grampa and\ 
        grandma are sitting on the sofa.
    

    Due to the occurrence of the \ character in the sequence and\, Yodl will combine the lines into

    Grampa andgrandma are sitting on the sofa.
    

    Note that the spaces before grandma are ignored, since this is the second line following a \ character.

    If you do want one or more spaces while joining lines with \, put the spaces before the \ character.

    Summarized:

  • A Line that ends in a backslash character is merged with the following line.

  • This only occurs if the \ character is the last character of the line, no spaces may occur after the character.

  • When merging lines, Yodl ignores leading spaces of the second line.

  • The question is of course, how do you accomplish that a line really ends with a \, when you do not want Yodl to merge it with the following line? In such a case, type a space character following your \: Yodl won't combine the lines. Or set the \ character as CHAR(\) or CHAR(92) (see section 2.3.3 for the CHAR macro). In my opinion, the ease of line continuation in Yodl files outweighs the extra necessary actions to typeset a real \ character at the end of a line.

    2.2.3: The +identifier sequence

    One more feature of the Yodl language remains to be described. There may be situations in which you must type a macro name right after a sequence of characters, while Yodl should recognize this. Imagine that someone wrote a great macro footnote for you (someone did, in fact, see the next chapter), to typeset footnotes. If you'd type in a document:

    The C Programming Languagefootnote(as defined by 
    Kernighan and Ritchie) ...
    

    then of course Yodl would fail to see the start of a macro in the sequence Languagefootnote. You could say

    The C Programming Language footnote(as defined by 
    Kernighan and Ritchie) ...
    

    but that would introduce a space between Language and the footnote. Probably you don't want that, since spaces between a word and a footnote number look awful and because of the fact that the footnote number might be typeset on the following line.

    For these special situations, Yodl recognizes the +identifier sequence as the start of a macro, while the + sign is effectively ignored. In the above example you could therefore use

    The C Programming Language+footnote(as defined by 
    Kernighan and Ritchie) ...
    

    The +identifier recognition only works when the identifier following the + sign is a macro. In all other situations, a + is just a plus-sign.

    (The +identifier sequence furthermore plays an important role in macro packages. If you're interested, see the file shared.yo which is by default installed to /usr/local/lib/yodl.)

    2.3: Yodl's builtin commands

    As mentioned previously. Yodl's input consists of text and of commands. Yodl supports a number of built-in commands which may either be used in a Yodl document, or which can be used to create a macro package.

    Don't despair if you find that the description of this section is too technical. Exactly for this reason, Yodl supports the macro packages to make the life of a documentation writer easier. E.g., see chapter 3 that describes a macro package for Yodl.

    2.3.1: ADDTOCOUNTER

    The ADDTOCOUNTER macro adds a given value to a counter. It expects two parameter lists: the counter name, and the value to add. The counter must be previously created with NEWCOUNTER.

    The value to add can be negative; in that case, a value is of course subtracted from the counter.

    See further section 2.6.

    2.3.2: ATEXIT

    The macro ATEXIT takes one parameter list as argument. The text of the parameter list is appended to the output file. Note that this text is subject to character table translations etc..

    A good example of the usage of this macro is the following. A document in the LaTeX typesetting language requires \end{document} to occur at the end of the document. To automatically append this string to the output file, the following specification can be used:

    ATEXIT(NOEXPAND(\end{document}))
    

    Several ATEXIT lists can be defined. They are appended to the output file in the reverse order of specification; i.e., the first ATEXIT list is appended to the output file last. That means that in general the ATEXIT text should be specified when a `matching' starting command is sent to the output file; as in:

    COMMENT(Start the LaTeX document.)
    NOEXPAND(\begin{document})
    
    COMMENT(Make sure that the ending is
            on time.)
    ATEXIT(NOEXPAND(\end{document}))
    

    2.3.3: CHAR

    The command CHAR takes one argument, a number or a character, and outputs its corresponding ASCII character to the final output file. This command is built for `emergency situations', where you need to typeset a character despite the fact that it may be redefined in the current character table (for a discussion of character tables, see 2.4). Also, the CHAR macro can be used to circumvent Yodl's way of matching parentheses in a parameter list.

    When you're sure that you want to send a printable character that is not a closing parenthesis to the output file, you can use the form CHAR(c), c being the character (as in, CHAR(")). To send a non-printable character or a closing parenthesis to the output file, look up the ASCII number of the character, and supply that number as argument to the CHAR command.

    Example: The following two statements send an A to the output file.

    CHAR(65)
    CHAR(A)
    

    The following statement sends a closing parenthesis:

    CHAR(41)
    

    Another way to send a string to the output file without expansion by character tables or by macro interpretation, is by using the macro NOTRANS (see section 2.3.22). If you want to send a string to the output without macro interpretation, but with character table translation, use NOEXPAND (see section 2.3.21).

    2.3.4: CHDIR

    The command CHDIR takes one argument, a directory to change to. This command is implemented to simplify the working with includefile (see includefile in yodlmacros(7)). As a demonstration by example, consider the following fragment:

    includefile(subdir/onefile)
    includefile(subdir/anotherfile)
    includefile(subdir/yetanotherfile)
    

    This fragment can be changed to:

    CHDIR(subdir)
    includefile(onefile)
    includefile(anotherfile)
    includefile(yetanotherfile)
    CHDIR(..)
    

    The current directory, as given to CHDIR, only affects how includefile will search for its files.

    2.3.5: COMMENT

    The COMMENT macro takes one parameter list. The text in the list is treated as comment; i.e., nothing is done with it. The text is not copied to the final output file; this command is meant as comment for Yodl files only.

    2.3.6: COUNTERVALUE

    The macro COUNTERVALUE expands to the value of a counter. Its only parameter list is a counter name. The counter must be previously created by NEWCOUNTER.

    See further section 2.6.

    2.3.7: DEFINECHARTABLE

    The macro DEFINECHARTABLE is used to define a character translation table. Its complement, USECHARTABLE, activates the table. The discussion of character tables is postponed to section 2.4.

    2.3.8: DEFINEMACRO

    The macro DEFINEMACRO lets you define new macros. This macro requires three parameter lists:

  • An identifier, being the name of the macro to define. This identifier must only consist of uppercase or lowercase characters.

  • A number, stating the number of arguments that the macro will require once used. The number must be in the range 0 to 61.

  • The text that the macro will expand to, once used. This text may contain the strings ARGx, x being 1, 2, etc.. At these places the arguments to the macro will be pasted in. The numbers that identify the arguments are 1 to 9, then A to Z and finally a to z. This gives a range of 61 expandable arguments, that should be enough for most applications.

  • For example, the following fragment defines a macro bookref, which can be used to typeset a reference to a book. It requires three arguments; say, an author, a title and the name of a publisher:

    DEFINEMACRO(bookref)(3)
    (Author(s):           ARG1
    Book title:          ARG2
    Published by:        ARG3)
    

    Such a macro could be used as:

    bookref(Sobotta/Becher)
        (Atlas der Anatomie des Menschen)
        (Urban und Schwarzenberg, Berlin, 1972)
    

    It would of course lead to:

    Author(s):           Sobotta/Becher
    Book title:          Atlas der Anatomie des Menschen
    Published by:        Urban und Schwarzenberg, Berlin, 1972
    

    While applying a macro, the three parameter lists are pasted to the places where ARG1, ARG2 etc. occur in the definition.

    A few caveats when defining new macros are:

  • The parameter list containing the name of the new macro, (bookref) in the above example, must occur right after DEFINEMACRO. No spaces are allowed in between. Space characters and newlines may however occur following this first parameter list.

    This behavior of the yodl program is similar to the usage of the defined macro: the author information must, enclosed in parentheses, follow right after the bookref identifier. I implemented this feature to improve the distinguishing between macros and real text. E.g., a macro me might be defined, but the text

    I like me (but so do you)
    

    still is simple text; the macro me only is activated when a parenthesis immediately follows it.

  • Be careful when placing newlines or spaces in the definition of a new macro. E.g., the definition

    DEFINEMACRO(bookref)(3)(
    Author(s):           ARG1
    Book title:          ARG2
    Published by:        ARG3)
    

    introduces an extra newline, that will be copied to the output each time that the macro is used. The extra newline occurs, of course, right before the sequence Author(s):. A simple backslash character at the end of the DEFINEMACRO line would prevent the insertion of an extra newline character.

  • Note that when a macro is used which requires no arguments at all, one empty parameter list still must be specified. E.g., my macro package (see chapter 3) defines a macro it that starts a bullet item in a list. The macro takes no arguments, but still must be typed as it().

    This behavior is consistent: it helps distinguish which identifiers are macros and which are simple text.

  • 2.3.9: DEFINESYMBOL

    The macro DEFINESYMBOL takes one argument, an identifier, which is treated as a symbol that is set to the logic value `true'. The true-ness or false-ness of this symbol may subsequently be tested with IFDEF (see section 2.3.13).

    The corresponding macro UNDEFINESYMBOL removes the definition of a symbol.

    Example:

    Definining "somesymbol"...
    DEFINESYMBOL(somesymbol)
    IFDEF(somesymbol)
        (Symbol "somesymbol" is true.)
        (Symbol "somesymbol" is false.)
    
    Removing definition of "somesymbol"...
    UNDEFINESYMBOL(somesymbol)
    IFDEF(somesymbol)
        (Symbol "somesymbol" is true.)
        (Symbol "somesymbol" is false.)
    

    This produces output similar to:

    Definining "somesymbol"...
    Symbol "somesymbol" is true.
    Removing definition of "somesymbol"...
    Symbol "somesymbol" is false.
    

    2.3.10: DUMMY

    This macro takes one argument, which is not interpreted. The macro is a real dummy, it does nothing.

    (I implemented the macro in order to provide a simple debugging hook. After inserting DUMMY() somewhere in the macro file, I can start the yodl program under a debugger and set a breakpoint in the dummy-routine; then watch what happens from there.)

    2.3.11: ENDDEF

    The macro ENDDEF takes one (empty) parameter list. It signals the ending of a Yodl definition section. See the description of STARTDEF, 2.3.31.

    2.3.12: ERROR

    The ERROR macro takes one argument: text to display to the standard error stream. The current input file and line number are also displayed. After displaying the text, the yodl program aborts with an exit status of 1.

    This command can be used, e.g., in a macro package when an incorrect macro is expanded. In my macro package (see chapter 3) the ERROR macro is used when the sectioning command chapter() is used in an article document (in the package, chapter's are only available in books or reports).

    An analogous builtin macro is WARNING, which also prints a message but does not exit (see section 2.3.39).

    2.3.13: IFDEF

    The IFDEF macro tests for the logical true value of the argument in its first parameter list. If `true', the second parameter list is evaluated, else, the third parameter list is evaluated. All three parameter lists (the variable, the true-list and the false-list) must be present; though the true-list and/or the false-list may be empty parameter lists.

    The variable in the first parameter list evaluates the macro to `true' when the first argument is:

  • a built-in macro,

  • a user-defined macro (see DEFINEMACRO, section 2.3.8),

  • a logical symbol (see DEFINESYMBOL, section 2.3.9),

  • a counter (see NEWCOUNTER, section 2.3.20).

  • For an example, see the description of DEFINESYMBOL, section 2.3.9.

    2.3.14: IFEMPTY

    The macro IFEMPTY expects three arguments: a symbol, a true-list and a false-list. The macro evaluates to the true-list if the symbol is an empty string; otherwise, it evaluates to the false-list.

    A trivial example is the following:

    IFEMPTY(something) \ 
        ("something" is supposedly nothing ?!?")
        ("something" is not an empty string)
    

    In the same way, IFEMPTY can be used to test whether a macro expands to a non-empty string. A more elaborate example follows below. Say you want to define a bookref macro to typeset information about an author, a book title and about the publisher. The publisher information may be absent, the macro then typesets unknown:

    DEFINEMACRO(bookref)(3)(\ 
    Author(s):      ARG1
    Title:          ARG2
    Published by:   IFEMPTY(ARG3)(Unknown)(ARG3))
    

    Using the macro, as in:

    bookref(Helmut Leonhardt)
           (Histologie, Zytologie und Microanatomie des Menschen)
           ()
    

    would of course supply Unknown in the Published by: line.

    2.3.15: IFSTREQUAL

    The macro IFSTREQUAL tests for the equality of two strings. It expects four arguments: two strings to match, a true-list and a false-list. The true-list is only evaluated when the two string arguments match exactly.

    2.3.16: IFSTRSUB

    The IFSTRSUB macro tests whether a string is a sub-string of another string. It is similar to IFSTREQUAL (see section 2.3.15), except that the test is whether the second string is part of the first one. Hence:

    IFSTRSUB(some piece of text)(ce o)
        (truelist)
        (falselist)
    

    evaluates to truelist.

    2.3.17: IFZERO

    The IFZERO macro expects three parameter lists, just as the other IF... macros: a symbol, a true-list and a false-list.

    The first argument defines whether the whole macro expands to the true-list or to the false-list. The first argument may be:

  • Nothing (empty): the macro expands to the true-list.

  • A number: when zero, the macro expands to the true-list.

  • The name of a counter: when the value of the counter is zero, the macro expands to the true-list.

  • Anything else: the macro expands to the false-list.

  • You can use COUNTERVALUE(somecounter) as the argument to IFZERO, it is treated in the same manner as the corresponding counter name.

    Example: The IFZERO macro offers a very simple way to pass a flag-argument (an on/off switch) to a macro. E.g., in LaTeX you start environments with \begin{environment} and end them with \end{environment}; environment being e.g., center, flushright, flushleft. A possible meta-macro for the environments might be:

    DEFINEMACRO(environment)(2)(\ 
        IFZERO(ARG2)\ 
            NOEXPAND(\end{ARG1})\ 
            NOEXPAND(\begin{ARG1}))
    

    Such a macro may be used as:

    environment(center)(1)
    Now comes centered text.
    environment(center)(0)
    

    which would of course lead to \begin and \end{center}. The numeric second argument is used here as a on/off switch.

    2.3.18: INCLUDEFILE

    The command INCLUDEFILE takes one argument, a filename. The file is included.

    The yodl program supplies, when necessary, an extension to the filename. The supplied extension is .yo, unless defined otherwise in the compilation of the program. Furthermore, yodl prefixes the file with the system-wide include directory if necessary. This directory is normally /usr/local/lib/yodl, unless overruled by the command line flag -I or defined otherwise in the compilation.

    E.g., if you install a macro package in a file latex.yo in the directory /usr/local/lib/yodl, then the command

    INCLUDEFILE(latex)
    

    will include it.

    2.3.19: INCLUDELITERAL

    The command INCLUDELITERAL takes one argument, a filename. The file is included.

    The filename is not changed in any way.

    Furthermore the contents of the file is included literally, not subject to macro expansion. Character translations are allowed.

    The purpose of INCLUDELITERAL is to include source code literally in the document, as in:

    INCLUDELITERAL(literal.c)
    

    This example shows how to put the contents of C file literal.c into the document.

    2.3.20: NEWCOUNTER

    The macro NEWCOUNTER creates a new counter, to be subsequently used by, e.g, the USECOUNTER macro. NEWCOUNTER expects one parameter list: the name of the counter to create. See further section 2.6.

    2.3.21: NOEXPAND

    The macro NOEXPAND is one of the ways in which text can be sent to the final output file without being expanded by Yodl (the other methods are the CHAR macro, see section 2.3.3, and the NOTRANS macro, see section 2.3.22). The macro NOEXPAND takes one parameter list, the text in question. Whatever occurs in the argument is not subject to parsing or expansion by Yodl, but is simply copied to the output file (except for CHAR macros in the argument, which are expanded). The contents of the parameter list are subject to character table translations, using the currently active table (see section 2.4).

    E.g., let's assume that you need to write in your document the following text:

    INCLUDEFILE(something or the other)
    IFDEF(onething) (..) (....)
    NOEXPAND(whatever)
    

    The way to accomplish this is by prefixing the text by NOEXPAND followed by an open parenthesis, and by postfixing it by a closing parenthesis. Otherwise, the text would be expanded by Yodl while processing it (and would lead to syntax errors, since the text isn't correct in the sence of the Yodl language).

    For this macro, keep the following caveats in mind:

  • There is only one thing that a NOEXPAND cannot protect from expansion: an ARGx in a macro definition. The argument specifier is always expanded. E.g., after

    DEFINEMACRO(thatsit)(1)(That is --> NOEXPAND(ARG1) <-- it!)
    
    thatsit(after all)
    

    the ARG1 inside the NOEXPAND statement is replaced with after all.

  • The NOEXPAND macro must, as all macros, be followed by a parameter list. The parentheses of the list must therefore be `balanced'. For unbalanced lists, use CHAR(40) to set an open parenthesis, or CHAR(41) to typeset a closing parenthesis.

  • 2.3.22: NOTRANS

    The macro NOTRANS copies its one argument literally to the output file, without expanding macros in it (except for CHAR, which is expanded) and without translating the characters with the current translation table. The NOTRANS macro is typically used to send commands for the output format to the output file.

    For example, consider the following code fragment:

    COMMENT(--- Define character translations for \{} in LaTeX. ---)
    DEFINECHARTABLE(standard)(
        '\\'    =    "$\\backslash$"
        '{'     =    "\\verb+{+"
        '}'     =    "\\verb+}+"
    )
    
    COMMENT(--- Activate the translation table. ---)
    USECHARTABLE(standard)
    
    COMMENT(--- Now two tests: ---)
    
    NOEXPAND(\input{epsf.tex})
    NOTRANS(\input{epsf.tex})
    

    The NOEXPAND macro in this example will send

    $\backslash$input\verb+{+epsf.tex\verb+}+
    

    since the characters in its argument are translated with the standard translation table. In contrast, the NOTRANS macro, will send literally \input{epsf.tex}.

    The parameter list of the NOTRANS macro must be balanced in respect to its parentheses. When using an unbalanced number of parentheses, use CHAR(40) to send a literal (, or CHAR(41) to send a ).

    2.3.23: NOUSERMACRO

    The macro NOUSERMACRO controls yodl's warnings in the following respect. When yodl is started with the -w flag on the command line, then warnings are generated when yodl encounters a possible macro name, followed by a parameter list, but fails to lookup the macro. The yodl program then prints something like cannot expand possible user macro.

    Examples of such sequences are, The necessary file(s) are in /usr/local/lib/yodl, or see the manual page for sed(1). The candidate macros are hee file and sed; these names could just as well be `valid' user macros followed by their parameter list.

    When a corresponding NOUSERMACRO statement appears before yodl encounters the candidate macros, no warning is generated. A fragment might therefore be:

    NOUSERMACRO(file sed)
    The necessary file(s) are in ...
    See the manual page for sed(1).
    
    

    The NOUSERMACRO accepts one or more names in its argument, separated by spaces, commas, colons, or semi-colons.

    2.3.24: PARAGRAPH

    The PARAGRAPH macro is defined by the Yodl program, but is had normally no actions. The macro works as follows.

    If you define the macro, then Yodl's parser will trigger it whenever a new paragraph is started. The macro must be defined as one with zero arguments, but leading to some (necessary) action. New paragraphs are -by definition- indicated in the source file as two or more consecutive newline characters, optionally separated by spaces. I.e., in the text

    Here is a line of text.
    
    And here's another one.
    

    one new paragraph occurs: the first line is terminated by a newline, then one newline follows right behind. Given such input, the PARAGRAPH macro is triggered right before the parsing process of the line starting with And.

    Some macro packages do not need paragraph starts; e.g., LaTeX does its own paragraph handling. Other macro packages do need it: typically, PARAGRAPH is then defined in a macro file to trigger some special action. E.g., a HTML converter might define a paragraph as:

    DEFINEMACRO(PARAGRAPH)(0)(<p>)
    

    Note that the definition of the action of the PARAGRAPH macro is again parsed and expanded by Yodl. Therefore, if you put two consecutive newlines in the definition itself, the definition will again trigger the PARAGRAPH macro, etcetera, ad infinitum. If you want the PARAGRAPH redefinition to insert a blank line, protect the line in a NOEXPAND, as in:

    DEFINEMACRO(PARAGRAPH)(0)(NOEXPAND(
    
    )<p>)
    

    2.3.25: PIPETHROUGH

    The builtin macro PIPETHROUGH is, besides SYSTEM, the second macro with which a Yodl document can affect its environment. Therefore, the danger of `live data' exists which is also described in the section about SYSTEM (see section 2.3.32). Nevertheless, PIPETHROUGH can be very useful. It is intended to use external programs to accomplish special features. The idea is that an external command is started, to which a block of text from within a Yodl document is `piped'. The output of that child program is piped back into the Yodl document; hence, a block of text is `piped through' an external program. Whatever is received again in the Yodl run, is further processed.

    The PIPETHROUGH macro takes two arguments:

  • the command to run, and

  • the text to send to that command.

  • Functionally, the occurrence of the PIPETHROUGH macro and of its two arguments is replaced by whatever the child program produces on its standard output.

    An example might be the inclusion of the current date, as in:

    The current date is:
    PIPETHROUGH(date)()
    

    In this example the command is date and the text to send to that program is empty.

    The main purpose of this macro is to provide a way by which external programs can be used to create, e.g., tables or figures for a given output format. Further releases of Yodl may contain such dedicated programs for the output formats.

    2.3.26: POPCHARTABLE

    Character tables which are pushed onto the table stack using PUSHCHARTABLE() are restored (popped) using POPCHARTABLE(). For a description of this mechanism please refer to section 2.4.3.

    2.3.27: PUSHCHARTABLE

    After a character table has been defined, it can be "pushed" onto a stack; to be "popped" later. This discussion is postponed to section 2.4.3.

    2.3.28: RENAMEMACRO

    The command RENAMEMACRO takes two arguments: the name of a built-in macro (such as INCLUDEFILE) and its new name.

    E.g., after

    RENAMEMACRO(INCLUDEFILE)(include)
    

    a file must be included by include(file), and no longer using INCLUDEFILE.

    Note that following the RENAMEMACRO action, the old name can no longer be used; it is then an unknown symbol.

    If you want to make an alias for a built-in command, do it with DEFINEMACRO. E.g., after:

    DEFINEMACRO(include)(1)(INCLUDEFILE(ARG1))
    

    both INCLUDEFILE and include can be used to read in a file.

    2.3.29: SETCOUNTER

    The SETCOUNTER macro expects two parameter lists: one counter name, and one numeric value. The corresponding counter (which must be previously created with NEWCOUNTER) is set to the numeric value.

    See also section 2.6.

    2.3.30: SUBST

    The SUBST macro is a general-purpose substitution mechanism for strings in the input. It takes two arguments: a search string and a substitution string. E.g., after

    SUBST(VERSION)(1.00)
    

    the yodl program will output 1.00 for each occurence of VERSION in its input.

    The SUBST macro is also useful in situations where multi-character sequeces should be converted to accent characters. E.g., a LaTeX converter might define:

    SUBST('e)(NOTRANS(\'{e}))
    

    Each 'e in the input would then be converted to é.

    The SUBST macro may also be useful in combination with the command line flag -P, as in a invocation

    yodl2html -P'SUBST(VERSION)(1.00)' myfile.yo
    

    A further useful substitution may be the following:

    SUBST(_OP_)(CHAR(40))
    SUBST(_CP_)(CHAR(41))
    

    which defines an opening parenthesis (_OP_) and a closing parenthesis (_CP_) as mapped to the CHAR macro. The strings _OP_ and _CP_ might then be used in unbalanced parameter lists.

    Note that:

  • The first argument of the SUBST command, the search string, is taken literally. Yodl does not expand it; the string must be literally matched in the input.

  • The second argument, the replacement, is further processed by Yodl. Protect this text by NOTRANS or NOEXPAND where appropriate.

  • 2.3.31: STARTDEF

    The STARTDEF macro typically occurs in files that define a macro package for Yodl. The macro signals the yodl program that the following input holds only the definitions of macros, symbols etc..

    When yodl is inside a definition, the generation of empty lines on the output is suppressed, and yodl warns when non-whitespace output is generated. Using STARTDEF is never obligatory, but it is useful in a macro file. Macro files tend to define a lot of symbols or commands, leading to unnecessary spaces and newlines.

    The definition section is ended with the ENDDEF macro.

    Example:

    STARTDEF()
    
    DEFINESYMBOL(....)
    DEFINEMACRO(...)(...)(...)
    
    ENDDEF()
    

    Without the STARTDEF and ENDDEF, the above definition would generate four empty lines in the output.

    2.3.32: SYSTEM

    The SYSTEM macro takes one argument: a command to execute. The command is run via the standard C function system. The presence of this macro in the Yodl language introduces the danger of live data; imagine someone sending you a document with

    SYSTEM(rm *)
    

    in it. To avoid such malevolent side effects, the yodl program has a flag -l to define the `live data policy'. By default, -l0 is implied which suppresses the SYSTEM macro and the related PIPETHROUGH macro. See also section 2.1.

    Despite the potential danger, SYSTEM can be useful in many ways. E.g., you might want to log when someone processes your document, as in:

    SYSTEM(echo Document processed! | mail myself@my.host)
    

    2.3.33: TYPEOUT

    The macro TYPEOUT requires one parameter list. The text of the list is sent to the standard error stream, followed by a newline. This feature can be handy to show, e.g., messages such as version numbers in macro package files.

    Example: The following macro includes a file and writes to the screen that this file is currently processed.

    
    DEFINEMACRO(includefile)(1)(\ 
        TYPEOUT(About to process document: ARG1)\ 
        INCLUDEFILE(ARG1))
    

    2.3.34: UNDEFINEMACRO

    The macro UNDEFINEMACRO removes a definition of a macro that was defined by DEFINEMACRO. This macro takes one argument: the macro name to remove.

    There is no error condition (except for syntax errors): when no macro with a matching name was previously defined, no action is taken.

    For example, the safe way to define a macro is by first undefining it. This ensures that possible previous definitions are removed first:

    UNDEFINEMACRO(mymacro)
    DEFINEMACRO(mymacro)(1)(This is my macro with argument ARG1.)
    

    2.3.35: UNDEFINESYMBOL

    The macro UNDEFINESYMBOL removes the definition of a logical symbol. It expects one parameter list, holding the variable to undefined.

    This macro has no error condition (except for syntax errors): the symbol in question may be previously defined, but that is not necessary.

    2.3.36: UPPERCASE

    The UPPERCASE macro converts a string or a part of it to upper case. It has two arguments:

  • The string to convert;

  • A length, indicating how many characters (starting from the beginning of the string) should be converted.

  • The length indicator can be smaller than one or larger than the length of the string; in that case, the whole string is convertered.

    Example:

    UPPERCASE(hello world)(1)
    UPPERCASE(hello world)(5)
    UPPERCASE(hello world)(0)
    

    This code sample expands to:

    Hello world
    HELLO world
    HELLO WORLD
    

    2.3.37: USECHARTABLE

    The macro USECHARTABLE takes one parameter list: the name of a translation table to activate. The table must be previously defined using DEFINECHARTABLE. See section 2.4 for a description of character translation tables.

    Alternatively, the name may be empty in which case the default character mapping is restored.

    2.3.38: USECOUNTER

    The macro USECOUNTER is a combination of ADDTOCOUNTER and COUNTERVALUE. It expects one parameter list: a counter name, that must be previously created with NEWCOUNTER.

    The counter is first increased by 1. Then the macro expands to the number of the counter.

    See also section 2.6.

    2.3.39: WARNING

    The WARNING macro takes one argument: text to display as a warning. The yodl program makes sure that before showing the text, the current file and line number are printed. Other than this, WARNING works just as TYPEOUT (see section 2.3.33).

    Note that an analogous macro ERROR exists, which prints a message and then terminates the program (see section 2.3.12).

    2.4: Character tables

    The Yodl language provides a way to define character translation tables, to activate them, and to deactivate them. A character translation table defines how a character in the input will appear in the output.

    There are two main reasons for the need of character translation tables. First, a document language becomes much easier to use when you can type an asterisk as * instead of $*$ or \verb/*/ (these are sequences from the LaTeX document language). Hence, a mechanism that expands a * in the input to to \verb/*/ on the output, saves the users a lot of typing.

    Second, forcing users to type weird sequences won't work if you're planning on converting the same Yodl document to a different output format. If the user types \verb/*/ in the input to typeset an asterisk in the output, how should he or she arrive at a single * in the output in another output format?

    The solution is of course to define the translation for an input character like * given the output format.

    2.4.1: Defining a translation table

    The built-in macro DEFINECHARTABLE defines a character translation table. It takes two parameter lists: the name of the table and the character translations. Hence, each table is defined by its own name.

    As an example of a table, consider the following fragment. It defines a table that translates the upper case characters A to E to their lower case equivalents:

    DEFINECHARTABLE(tolower)(
        'A' = "a"
        'B' = "b"
        'C' = "c"
        'D' = "d"
        'E' = "e"
    )
    

    Each DEFINECHARTABLE statement must have a non-empty second parameter. "Empty" character tables cannot be defined, though one non-translation table is built-in.

    The syntaxis of the second parameter list is as follows:

  • On separate lines, input characters are mapped to a sequence to appear on the output.

  • Per line, the input character is specified as 'c', c being any character. Escape-sequences from the C programming language can be used in this specification; Yodl supports the sequences \a (alert), \b (beep), \f (formfeed), \n (newline), \r (carriage return), \t (tab), and \v (vertical tab). Any other character following a \ defines itself: \\ defines one backslash character.

  • Following the character specification, a = must appear.

  • Following that, a sequence of one or more characters appears, enclosed in double quotes, defining the translation. Again, escape sequences can be used, as in:

    '\n' = "End of line\n"
    

    Such a mapping adds the text End of line to each line, since each newline character in the input is replaced by the text End of line, followed by the newline itself.

  • Translations which are not specified in the table are left to the default, which is to output the character as-is.

    Note that the character table translation is something that the yodl program does as one of its last actions, just before sending text to the output file. The expansion text is not further processed by yodl, except for the conversion of C-type escape sequences to ordinary characters. The expansion text should therefore not be protected by, e.g., NOTRANS (unless of course you want some character to generate the text NOTRANS on the output).

    2.4.2: Using a translation table

    A defined translation table is activated by the macro USECHARTABLE. This macro takes one parameter list, which may be:

  • empty, in which case the default mapping is restored,

  • a name of a previously defined character table.

  • The default mapping, selected when an empty parameter list is given, means that Yodl enters its `zero translation state', meaning no character translation at all.

    2.4.3: Pushing and popping character tables

    Besides the previously described macro USECHARTABLE(), Yodl has one other mechanism of activating and deactivating character translation tables. This mechanism uses a stack, and hence, the related macros are appropriately named PUSHCHARTABLE() and POPCHARTABLE().

  • PUSHCHARTABLE(name) pushes the currently active translation table onto a stack, and activates the table identified by name. The argument may be emtpy; in that case, the zero-translation table is activated (analogously to USECHARTABLE()).

  • POPCHARTABLE() activates the translation table that was last pushed. There is no argument to this macro.

  • Using the push/pop mechanism is handy when a table must be temporarily activated, but when it is not known which table exacty is active prior to the temporary activation. E.g., imagine that you need to use a character table called listing to typeset a listing, but that you do not know the current table. The pushing and popping mechanism is then used as follows:

    COMMENT(First, we save the current table on the stack and
            we activate our "listing" table.)
    PUSHCHARTABLE(listing)
    
    COMMENT(Now the text is question is typeset.)
    ...
    
    COMMENT(The previously active table is re-activated, whatever its name.)
    POPCHARTABLE()
    

    2.5: Sending literal text to the output

    The Yodl program has three built-in macros to send literal text to the output file. The macros are listed in the above section 2.3 and are furthermore described here.

  • The CHAR macro takes one argument: the ASCII number of a character or the character itself. The character is sent to the output file without being translated with the currently active character translation table.

  • The NOTRANS macro takes one argument: the text in question. The text is neither parsed (i.e., macros in it are not expanded), nor translated with the current character translation table.

    The NOTRANS macro is conceptually like a series of CHAR macros.

  • The NOEXPAND macro takes one argument: the text in question. The text is not parsed, but it is translated with the current character translation table.

  • To illustrate the need for the distinction between NOTRANS and NOEXPAND, consider the following. The HTML converter (described in chapter 3) must be able to send HTML commands to the output file, but must also be able to send literal text (e.g., a source file listing). The HTML commands of course must be neither translated with any character table, nor must they be expanded in regard to macros. In contrast, a source file listing must be subject to character translations: the &, < and > characters can cause difficulties. Two possible macros for a HTML converter are:

    COMMENT(--- htmlcommand(cmd) sends its argument as a HTML command 
                to the output ---)
    DEFINEMACRO(htmlcommand)(1)(NOTRANS(ARG1))
        
    COMMENT(--- verb(listing) sends the listing to the output ---)
    DEFINECHARTABLE(list)(
        '&'     =   "&amp;"
        '<'     =   "&lt;"
        '>'     =   "&gt;"
    )
    
    DEFINEMACRO(verb)(1)( \ 
        USECHARTABLE(list) \ 
        NOTRANS(<listing>) \ 
        NOEXPAND(ARG1) \ 
        NOTRANS(</listing>) \ 
        USECHARTABLE(standard))
    

    In this example it is assumed that a character translation table standard exists, defining the `normal' translations. This table is re-activated in the verb macro.

    2.6: Counters

    Some document languages (notably LaTeX) automatically prefix numbers when typesetting sections, subsections, tables, figures etc.. Other document languages (e.g. HTML) unfortunately don't.

    Therefore, a macro package that converts a Yodl document to LaTeX doesn't need to provide the numbering of sections etc.. However, if you do want the numbering and if you want to convert documents to, say, HTML, then you must take care of the numbering yourself.

    This section describes the counters in Yodl: how to create a new counter, how to use it, etc..

    2.6.1: Creating a counter

    Before a counter can be used, it must be created with the macro NEWCOUNTER. This macro expects one parameter list: the name of the counter. The initial value of the counter is set to zero.

    As for an example, let's say that our macro package should provide two sectioning commands: section and subsection. The sections should be numbered 1, 2, etc., and the subsections 1.1, 1.2, 1.3 etc.. Hence we'd need two counters:

    NEWCOUNTER(sectcounter)
    NEWCOUNTER(subsectcounter)
    

    2.6.2: Using counters

    Yodl has four macros to modify and/or to set the values of counters. The macros are:

  • COUNTERVALUE(somecounter): This macro expands to the value of somecounter. E.g., if the current value is 2, then the macro writes 2 on the output file.

  • SETCOUNTER(somecounter)(number): This macro sets the value of somecounter to the value of number. The second parameter list must be an integer number (i.e., consisting of the characters 0 to 9, optionally prefixed by a - sign). The macro does not expand to anything; i.e., it does not write to the output file.

  • ADDTOCOUNTER(somecounter)(number): This macro adds the value of number to somecounter. The number may be negative.

  • USECOUNTER(somecounter): This macro first increases the value of somecounter by 1, and then writes the value of the counter to the output file.

    This command is particularly useful in combination with NEWCOUNTER: since NEWCOUNTER initializes a counter to zero, USECOUNTER can be used to increase the value and to output it. The first time that USECOUNTER is used on a new counter, the number 1 appears on the output file. The next time, number 2 appears on the output file etc..

  • Given the numbering requirements of the hypothetical commands section and subsection (see the previous section), we can now complete the definitions:

    NEWCOUNTER(sectcounter)
    NEWCOUNTER(subsectcounter)
    
    DEFINEMACRO(section)(1)(\ 
    SETCOUNTER(subsectcounter)(0)\ 
    USECOUNTER(sectcounter) ARG1)
    
    DEFINEMACRO(subsection)(1)(\ 
    COUNTERVALUE(sectcounter).USECOUNTER(subsectcounter) ARG1)
    




    Go back to index of Yodl.

    Please send Yodl questions and comments to yodl@icce.rug.nl.

    Please send comments on these web pages to (address unknown)

    Copyright (c) 1997, 1998, 1999 Karel Kubat and Jan Nieuwenhuizen.

    Verbatim copying and distribution of this entire article is permitted in any medium, provided this notice is preserved.


    This page was built from Yodl-1.31.18 by

    <(address unknown)>, Tue Aug 16 22:51:36 2005 EDT.