Preliminaries

• What This Module Does - An overview of the words module's role and abilities.

• How To Include This Module - What to do to make use of the words module in a new command-line tool.

Chapter 1: Setting Up

Building on the foundation module.

• Words Module - Setting up the use of this module.

Chapter 2: Words in Isolation

Recognising different words, and storing phrases made of them.

• Vocabulary - To classify the words in the lexical stream, where two different words are considered equivalent if they are unquoted and have the same text, taken case insensitively.

• Word Assemblages - To manage arbitrary assemblies of vocabulary, if a little slowly.

• Numbered Words - Some utilities for handling single words referred to by number.

Chapter 3: Words in Sequence

Reading in arbitrary text and breaking it into a numbered sequence of words.

• Lexer - To break down a stream of characters into a numbered sequence of words, literal strings and literal I6 inclusions, removing comments and unnecessary whitespace.

• Wordings - To manage contiguous word ranges.

• Text From Files - This is where source text is read in, whether from extension files or from the main source text file, and fed into the lexer.

• Feeds - Feeds are conduits for arbitrary text to flow into the lexer, and to be converted into wordings.

• Identifiers - To summarise wordings into alphanumeric identifiers of the kind used by standard programming languages.

Chapter 4: Parsing

• About Preform - A brief guide to Preform and how to use it.

• Nonterminals - The angle-bracketed terms appearing in Preform grammar.

• Loading Preform - To read in structural definitions of natural language written in the meta-language Preform.

• The Optimiser - To precalculate data which enables rapid parsing of source text against a Preform grammar.

• Length Extremes - To precalculate data which enables rapid parsing of source text against a Preform grammar.

• Nonterminal Incidences - To work out bitmaps of nonterminal incidences in grammar.

• Preform - To parse the word stream against a general grammar defined by Preform.

• Basic Nonterminals - A handful of bare minimum Preform syntax.

• Instrumentation - To provide debugging and tuning data on the Preform parser's performance.

• Preform Utilities - Other uses for Preform grammar, including the generation of adaptive text, and word inflection.