The Tangler

Transforming code written in literate programming notation into regular code, fit for a compiler.

§1. The code in this section was substantially redeveloped in 2025. Before that time, a "simple tangler" provided minimal tangling support: just enough to handle kit source, but notably not providing for definitions, or holons. In 2025 more or less all of what had been Inweb was turned into foundation library material in order that the tangler here could become a full-strength tangler, and not a much-reduced stopgap.

It can nevertheless be used with some flexibility, and doesn't have to work on an entire web: it can also tangle text in memory.

During any tangle, a collection of settings is held in an instance of the following:

typedef struct tangle_docket {
    void (*command_callback)(struct text_stream *, struct text_stream *,
        struct text_stream *, struct tangle_docket *);
    void (*bplus_callback)(struct text_stream *, struct tangle_docket *);
    void (*source_marker_callback)(struct text_stream *, struct tangle_docket *);
    void (*error_callback)(struct text_file_position *, char *, struct text_stream *);
    void *state;
    struct text_file_position at;
    struct tangle_target *target;
    struct linked_list *conventions; /* of |ls_conventions| */
    struct pathname *external_path;
    struct linked_list *external_files; /* of |tangle_external_file| */
} tangle_docket;

typedef struct tangle_external_file {
    struct filename *fn;
    struct text_stream stream;
    CLASS_DEFINITION
} tangle_external_file;

The structure tangle_docket is accessed in 1/ws, 1/wcp, 1/cnv, 2/wi, 3/as, 3/cl, 4/cs, 5/apacs, 5/hf and here.
The structure tangle_external_file is accessed in 5/wt, 5/ptf, 5/tf, 5/hf, 5/df and here.

§2. The idea here is that a docket contains optional callback functions to deal with situations arising during the tangle. In each case they can be NULL, to take no special action.

tangle_docket Tangler::new_docket(
    void (*B)(struct text_stream *, struct text_stream *,
        struct text_stream *, struct tangle_docket *),
    void (*C)(struct text_stream *, struct tangle_docket *),
    void (*D)(struct text_stream *, struct tangle_docket *),
    void (*E)(struct text_file_position *, char *, struct text_stream *),
    void *initial_state) {
    tangle_docket docket;
    docket.command_callback = B;
    docket.bplus_callback = C;
    docket.source_marker_callback = D;
    docket.error_callback = E;
    docket.state = initial_state;
    docket.at = TextFiles::nowhere();
    docket.target = NULL;
    docket.conventions = NEW_LINKED_LIST(ls_conventions);
    ADD_TO_LINKED_LIST(Conventions::generic(), ls_conventions, docket.conventions);
    docket.external_path = NULL;
    docket.external_files = NEW_LINKED_LIST(tangle_external_file);
    return docket;
}

text_stream *Tangler::obtain_side_stream(tangle_docket *docket, filename *F) {
    text_stream *TO = NULL;
    tangle_external_file *tef;
    LOOP_OVER_LINKED_LIST(tef, tangle_external_file, docket->external_files)
        if (Filenames::eq(F, tef->fn)) {
            TO = &(tef->stream);
            break;
        }
    if (TO == NULL) {
        tef = CREATE(tangle_external_file);
        tef->fn = F;
        TO = &(tef->stream);
        if (STREAM_OPEN_TO_FILE(TO, F, UTF8_ENC) == FALSE)
            Errors::fatal_with_file("unable to write tangle file", F);
        ADD_TO_LINKED_LIST(tef, tangle_external_file, docket->external_files);
    }
    return TO;
}

void Tangler::finish_with_docket(tangle_docket *docket) {
    tangle_external_file *tef;
    LOOP_OVER_LINKED_LIST(tef, tangle_external_file, docket->external_files)
        STREAM_CLOSE(&(tef->stream));
}

§3. This tangles just a small excerpt of code held as text in memory, i.e., not to be read in from a file somewhere. Note that since we never create a surrounding web for this material to live in (instead we'll create a single literate source unit), we call down to the lower levels of the tangler, and skip the upper levels concerned with the web superstructure.

void Tangler::tangle_code_with_docket(OUTPUT_STREAM, tangle_docket *docket, text_stream *text,
    text_file_position origin, programming_language *language) {
    docket->target = TangleTargets::ad_hoc_target(language);
    docket->at = origin;
    ls_unit *lsu = LiterateSource::code_fragment_to_unit(WebNotation::default(FALSE), language,
        text, docket->at);
    Tangler::report_errors(docket, lsu);
    docket->external_files = NEW_LINKED_LIST(tangle_external_file);
    Tangler::tangle_holons_in_segment(OUT, lsu, docket, MAIN_TANGLE_SEGMENT);
    Tangler::finish_with_docket(docket);
}

§4. Otherwise, we'll be tangling an entire web.

As a convenience, this loads the web at a given path, and then tangles it. But we need to know in advance what programming language the program is.

void Tangler::tangle_web_directory_with_docket(OUTPUT_STREAM, tangle_docket *docket,
    pathname *P, programming_language *language) {
    wcl_declaration *D = WCL::read_web_or_halt(P, NULL, NULL);
    ls_web *W = WebStructure::from_declaration(D);
    WebStructure::set_language(W, language);
    WebStructure::read_web_source(W, FALSE, FALSE);
    docket->target = TangleTargets::primary_target(W);
    Conventions::establish(W, NULL);
    docket->conventions = W->conventions;
    Tangler::tangle_web_inner(OUT, docket, W, NULL);
}

§5. But the most general way to tangle a web already loaded in is to use this:

void Tangler::tangle_web(OUTPUT_STREAM, ls_web *W, pathname *extracts_path,
    tangle_target *target) {
    tangle_docket docket = Tangler::new_docket(NULL, NULL, NULL, NULL, NULL);
    docket.target = target;
    docket.conventions = W->conventions;
    docket.external_path = extracts_path;
    Tangler::tangle_web_inner(OUT, &docket, W, extracts_path);
}

§6. When a paragraph contains a holon of code, it might belong to any of three "segments": though in practice almost all of the program is in the main one. Only code specially marked as early or very early is an exception.

define VERY_EARLY_TANGLE_SEGMENT 1

define EARLY_TANGLE_SEGMENT 2

define MAIN_TANGLE_SEGMENT 3

define LATE_TANGLE_SEGMENT 4

define VERY_LATE_TANGLE_SEGMENT 5

int Tangler::segment_of_par(ls_paragraph *par) {
    if (LiterateSource::par_contains_very_early_code(par)) return VERY_EARLY_TANGLE_SEGMENT;
    if (LiterateSource::par_contains_early_code(par)) return EARLY_TANGLE_SEGMENT;
    if (LiterateSource::par_contains_late_code(par)) return LATE_TANGLE_SEGMENT;
    if (LiterateSource::par_contains_very_late_code(par)) return VERY_LATE_TANGLE_SEGMENT;
    return MAIN_TANGLE_SEGMENT;
}

§7. We will also need a way to report any parsing errors, since the likelihood is that they have been silently noted up to now.

void Tangler::report_errors(tangle_docket *docket, ls_unit *lsu) {
    if (docket->error_callback) {
        ls_error *error;
        LOOP_OVER_LINKED_LIST(error, ls_error, lsu->errors) {
            if (error->warning) {
                TEMPORARY_TEXT(msg)
                WRITE_TO(msg, "warning: %S", error->message);
                (*(docket->error_callback))(&(error->tfp), "tangle warning: '%S'", msg);
                DISCARD_TEXT(msg)
            } else {
                (*(docket->error_callback))(&(error->tfp), "tangle error: '%S'", error->message);
            }
        }
    }
}

§8. So, then, the following shows the basic structure of tangle output.

void Tangler::tangle_web_inner(OUTPUT_STREAM, tangle_docket *docket, ls_web *W,
    pathname *extracts_path) {
    tangle_target *target = docket->target;
    if (target == NULL) internal_error("tangling web with no language");
    programming_language *language = target->tangle_language;
    if (language == NULL) internal_error("tangling web with no language");

    ls_chapter *C; ls_section *S;
    LOOP_OVER_TARGET_SECTIONS(C, S, target)
        Tangler::report_errors(docket, S->literate_source);

    LanguageMethods::shebang(OUT, language, W, target);
    LanguageMethods::additional_early_matter(OUT, language, W, target, docket);

    LOOP_OVER_TARGET_SECTIONS(C, S, target)
        Tangler::tangle_holons_in_segment(OUT, S->literate_source, docket, VERY_EARLY_TANGLE_SEGMENT);

    if (W->definitions_chunk == NULL) Tangler::tangle_constants(OUT, docket, W);

    LOOP_OVER_TARGET_SECTIONS(C, S, target)
        Tangler::tangle_holons_in_segment(OUT, S->literate_source, docket, EARLY_TANGLE_SEGMENT);

    LOOP_OVER_TARGET_SECTIONS(C, S, target)
        Tangler::tangle_holons_in_segment(OUT, S->literate_source, docket, MAIN_TANGLE_SEGMENT);

    LOOP_OVER_TARGET_SECTIONS(C, S, target)
        Tangler::tangle_holons_in_segment(OUT, S->literate_source, docket, LATE_TANGLE_SEGMENT);

    LOOP_OVER_TARGET_SECTIONS(C, S, target)
        Tangler::tangle_holons_in_segment(OUT, S->literate_source, docket, VERY_LATE_TANGLE_SEGMENT);

    LanguageMethods::gnabehs(OUT, language, W);

    if (extracts_path) {
        Tangle any imported headers8.1;
        Tangle any extract files not part of the target itself8.2;
    }
    LanguageMethods::additional_tangling(language, W, target);
    Tangler::finish_with_docket(docket);
}

§9.

void Tangler::tangle_constants(OUTPUT_STREAM, tangle_docket *docket, ls_web *W) {
    tangle_target *target = docket->target;
    programming_language *language = target->tangle_language;
    ls_chapter *C;
    ls_section *S;
    LOOP_OVER_TARGET_CHUNKS(C, S, target)
        if (L_chunk->chunk_type == DEFINITION_LSCT)
            if ((L_chunk->metadata.minor == DEFINE_COMMAND_MINLC) ||
                (L_chunk->metadata.minor == ENUMERATE_COMMAND_MINLC))
                Define the constant9.1;
    LOOP_OVER_TARGET_CHUNKS(C, S, target)
        if (L_chunk->chunk_type == DEFINITION_LSCT)
            if (L_chunk->metadata.minor == DEFAULT_COMMAND_MINLC) {
                LanguageMethods::open_ifdef(OUT, language, L_chunk->symbol_defined, FALSE);
                Define the constant9.1;
                LanguageMethods::close_ifdef(OUT, language, L_chunk->symbol_defined, FALSE);
            }
    Enumerations::define_extents(OUT, target, language, docket);
    LanguageMethods::additional_predeclarations(OUT, language, docket, W);
}

§9.1. Define the constant9.1 =

    IfdefTags::open_ifdefs(OUT, L_par);
    LanguageMethods::start_definition(OUT, language,
        L_chunk->symbol_defined, L_chunk->code_excerpt, S, L_chunk->first_line, docket);
    IfdefTags::close_ifdefs(OUT, L_par);

This code is used in §9 (twice).

§8.1. Some C programs, in particular, may need additional header files added to any tangle in order for them to compile. (The Inform project uses this to get around the lack of some POSIX facilities on Windows.)

Tangle any imported headers8.1 =

    filename *F;
    LOOP_OVER_LINKED_LIST(F, filename, W->header_filenames)
        Shell::copy(F, WebStructure::tangled_folder(W), "");

This code is used in §8.

§8.2. The following simple implementation splices raw lines from text (probably code, or configuration gobbledegook) marked as "to ...", giving a leafname. We place files of those leafnames in the same directory as the tangle target.

define MAX_EXTRACT_FILES 10

Tangle any extract files not part of the target itself8.2 =

    text_stream *extract_names[MAX_EXTRACT_FILES];
    text_stream extract_files[MAX_EXTRACT_FILES];
    int no_extract_files = 0;
    ls_chapter *C; ls_section *S;
    LOOP_OVER_TARGET_CHUNKS(C, S, target)
        if (Str::len(L_chunk->extract_to) > 0)
            for (ls_line *lst = L_chunk->first_line; lst; lst = lst->next_line) {
                int j = no_extract_files;
                for (int i=0; i<no_extract_files; i++)
                    if (Str::eq(L_chunk->extract_to, extract_names[i])) j = i;
                if (j == no_extract_files) {
                    if (j == MAX_EXTRACT_FILES)
                        Errors::fatal("too many extract files in tangle");
                    extract_names[j] = Str::duplicate(L_chunk->extract_to);
                    filename *F = Filenames::in(extracts_path, L_chunk->extract_to);
                    if (STREAM_OPEN_TO_FILE(&(extract_files[j]), F, UTF8_ENC) == FALSE)
                        Errors::fatal_with_file("unable to write extract file", F);
                    no_extract_files++;
                }
                WRITE_TO(&(extract_files[j]), "%S\n", lst->text);
            }
    for (int i=0; i<no_extract_files; i++) STREAM_CLOSE(&(extract_files[i]));

This code is used in §8.

§10. A traditional LP tool might tangle only the main holon, i.e., the first in the web, which would then include all of the others. But we allow nameless holons to be concatenated into the program, which is much simpler.

void Tangler::tangle_holons_in_segment(OUTPUT_STREAM, ls_unit *lsu,
    tangle_docket *docket, int segment) {
    if (lsu == NULL) internal_error("no holon");
    for (ls_paragraph *par = lsu->first_par; par; par = par->next_par) {
        if ((lsu->context) && (lsu->context->definitions_chunk) &&
            (lsu->context->definitions_chunk->owner == par) && (segment == MAIN_TANGLE_SEGMENT))
            Tangler::tangle_constants(OUT, docket, lsu->context);
        if ((par->holon) && (par->holon->top_level) && (par->holon->addendum_to == NULL)) {
            text_stream *TO = OUT;
            text_stream *N = Holons::external_filename(par->holon);
            if (Str::len(N) > 0) {
                filename *F = Filenames::from_text_relative(docket->external_path, N);
                TO = Tangler::obtain_side_stream(docket, F);
            }
            if (segment == Tangler::segment_of_par(par)) {
                IfdefTags::open_ifdefs(TO, par);
                Tangler::tangle_holon(TO, par->holon, docket, NULL);
                IfdefTags::close_ifdefs(TO, par);
            }
        }
    }
}

§11. Note that this function is recursive: that's how holons incorporate each other, in a tangle. We enter it with a nameless holon, then it calls itself to include tangled contents of any named holons referred to, and so on.

void Tangler::tangle_holon(OUTPUT_STREAM, ls_holon *main_holon, tangle_docket *docket,
    text_stream *indentation) {
    if (main_holon == NULL) internal_error("no holon");
    tangle_target *target = docket->target;
    if (target == NULL) internal_error("tangling holon with no target");

    ls_holon *holon = main_holon;
    Tangle this holon alone11.1;
    LOOP_OVER_LINKED_LIST(holon, ls_holon, main_holon->addenda)
        Tangle this holon alone11.1;
    WRITE("\n%S", indentation);
}

§11.1. Tangle this holon alone11.1 =

    Tangler::tangle_code_excerpt(OUT, NULL, holon->corresponding_chunk->code_excerpt, NULL, NULL, docket, indentation);

This code is used in §11 (twice).

§12.

void Tangler::tangle_code_excerpt(OUTPUT_STREAM, text_stream *prefix, ls_code_excerpt *ex,
    text_stream *suffix, text_stream *line_suffix, tangle_docket *docket, text_stream *indentation) {
    holon_splice *hs;
    ls_line *last_line = NULL;
    tangle_target *target = docket->target;
    if (target == NULL) internal_error("tangling holon with no target");
    int next_line_does_not_follow = TRUE;
    TEMPORARY_TEXT(buffer)
    LOOP_OVER_CODE_EXCERPT(hs, ex) {
        ls_line *lst = hs->line;
        if (lst != last_line) {
            if (last_line != NULL) WRITE("%S\n%S", line_suffix, indentation);
            last_line = lst;
            int did_insert = FALSE;
            LanguageMethods::insert_in_tangle(OUT, &(did_insert), target->tangle_language, lst, docket);
            if (did_insert) next_line_does_not_follow = TRUE;
        }
        if (lst->suppress_tangling) {
            next_line_does_not_follow = TRUE;
            continue;
        }
        if (next_line_does_not_follow) {
            Tangler::tangle_line_marker(OUT, lst, docket);
            WRITE("%S", indentation);
            next_line_does_not_follow = FALSE;
        }
        if (prefix) { WRITE("%S", prefix); prefix = NULL; }
        Tangle this splice12.1;
    }
    DISCARD_TEXT(buffer)
    WRITE("%S", suffix);
    if (last_line != NULL) WRITE("\n%S", indentation);
}

§13. Much of the complexity here lies in tracking whether we need to insert another line marker or not. These are compiler features saying "the next line derived from line X of file Y, and any errors in it should be reported as if from there". In principle we could be safe by providing a line marker in front of every line we ever tangle, but that would be very cumbersome. So we simply mark lines where we can't prove that they immediately follow the previous line.

void Tangler::tangle_line_marker(OUTPUT_STREAM, ls_line *lst, tangle_docket *docket) {
    docket->at = lst->origin;
    if (docket->source_marker_callback) {
        (*(docket->source_marker_callback))(OUT, docket);
    } else {
        LanguageMethods::insert_line_marker(OUT, docket->target->tangle_language, lst);
    }
}

§14. We sometimes want to tangle a line in isolation, not an entire holon, and then we really can't be sure that a line marker is unnecessary, so we always issue one.

void Tangler::tangle_line(OUTPUT_STREAM, ls_line *lst, tangle_docket *docket) {
    if (lst->owning_chunk == NULL) internal_error("loose line");
    ls_holon *holon = lst->owning_chunk->holon;
    if (holon == NULL) internal_error("no holon");
    Tangler::tangle_line_marker(OUT, lst, docket);
    TEMPORARY_TEXT(buffer)
    holon_splice *hs;
    LOOP_OVER_HOLON_DEFINITION(hs, holon)
        if (lst == hs->line)
            Tangle this splice12.1;
    WRITE("\n");
    DISCARD_TEXT(buffer)
}

§12.1. Whether tangling a holon or just a line, then, we need this:

Tangle this splice12.1 =

    switch (hs->type) {
        case CODE_LSHST: Tangle in this splice of raw content12.1.4; break;
        case EXPANSION_LSHST: Recursively tangle this named holon in12.1.1; break;
        case COMMAND_LSHST: Act on this tangler command12.1.2; break;
        case VERBATIM_LSHST: Act on this verbatim12.1.3; break;
        case COMMENT_LSHST: break;
        default: internal_error("unimplemented holon splice type");
    }

This code is used in §12 and §14.

§12.1.1. Here's the recursion taking place:

Recursively tangle this named holon in12.1.1 =

    if (docket->target)
        LanguageMethods::before_holon_expansion(OUT,
            docket->target->tangle_language, docket, hs->expansion->corresponding_chunk->owner);
    text_stream *indentation = NULL;
    if ((docket->target->tangle_language->indent_holon_expansion) &&
        (Str::len(buffer) > 0) && (Str::is_whitespace(buffer))) indentation = buffer;
    Tangler::tangle_holon(OUT, hs->expansion, docket, indentation);
    if (docket->target)
        LanguageMethods::after_holon_expansion(OUT,
            docket->target->tangle_language, docket, hs->expansion->corresponding_chunk->owner);
    Tangler::tangle_line_marker(OUT, hs->line, docket);

This code is used in §12.1.

§12.1.2. This is a similar matter, except that it expands bibliographic data:

    printf("This is build [[Build Number]].\n");

takes the bibliographic data for "Build Number" (as set on the web's contents page) and substitutes that, so that we end up with (say)

    printf("This is build 5Q47.\n");

In some languages there are also special expansions (for example, in InC [[nonterminals]] has a special meaning).

If the text in double-squares isn't recognised, that's not an error: it simply passes straight through. So [[water]] becomes just [[water]].

Act on this tangler command12.1.2 =

    ls_chunk *chunk = hs->line->owning_chunk;
    int handled = LanguageMethods::special_tangle_command(OUT, docket->target->tangle_language, hs->texts[0]);
    ls_notation *ntn = chunk->owner->owning_unit->unit_notation;
    if (handled == FALSE) {
        ls_section *sect = LiterateSource::section_of_par(chunk->owner);
        if (sect) {
            ls_web *W = sect->owning_chapter->owning_web;
            if ((W) && (Bibliographic::look_up_datum(W, hs->texts[0]))) {
                WRITE("%S", Bibliographic::get_datum(W, hs->texts[0]));
                handled = TRUE;
            }
        }
    }
    if (handled == FALSE) {
        WRITE("%S%S%S", WebNotation::left(ntn, METADATA_IN_STRINGS_NTNMARKER),
            hs->texts[0], WebNotation::right(ntn, METADATA_IN_STRINGS_NTNMARKER));
    }

This code is used in §12.1.

§12.1.3. Act on this verbatim12.1.3 =

    Str::clear(buffer);
    Str::copy(buffer, hs->texts[0]);
    Tangler::tangle_illiterate_code_fragment(OUT, buffer, docket);

This code is used in §12.1.

§12.1.4. Tangle in this splice of raw content12.1.4 =

    Str::clear(buffer);
    Str::copy(buffer, hs->texts[0]);
    Tangler::tangle_illiterate_code_fragment(OUT, buffer, docket);

This code is used in §12.1.

§15. Before we get down to illiterate code (i.e., code with all literate programming syntax removed), here's a convenient way to tangle a small text which may, nevertheless, still contain some markup for named holons:

void Tangler::tangle_literate_code_fragment(OUTPUT_STREAM, text_stream *code,
    tangle_docket *docket, ls_line *lst) {
    ls_unit *lsu = LiterateSource::unit_of_line(lst);
    finite_state_machine *machine =
        HolonSyntax::get(lsu->unit_notation, docket->target->tangle_language);
    FSM::reset_machine(machine);
    TEMPORARY_TEXT(name)
    TEMPORARY_TEXT(output)
    for (int i=0; i<Str::len(code); i++) {
        inchar32_t c = Str::get_at(code, i);
        int event = FSM::cycle_machine(machine, c, NULL);
        PUT_TO(name, c);
        PUT_TO(output, c);
        switch (event) {
            case NAME_START_FSMEVENT: {
                Str::clear(name);
                break;
            }
            case NAME_END_FSMEVENT: {
                int ho_len = Str::len(WebNotation::left(lsu->unit_notation, NAMED_HOLONS_NTNMARKER));
                int hc_len = Str::len(WebNotation::right(lsu->unit_notation, NAMED_HOLONS_NTNMARKER));
                Str::truncate(name, Str::len(name) - hc_len);
                ls_holon *holon = Holons::find_holon(name, lsu->local_holon_namespace, FALSE, NULL);
                if (holon) {
                    int excess = ho_len + Str::len(name) + hc_len;
                    Str::truncate(output, Str::len(output) - excess);
                    LanguageMethods::before_holon_expansion(output,
                        docket->target->tangle_language, docket, holon->corresponding_chunk->owner);
                    Tangler::tangle_holon(output, holon, docket, NULL);
                    LanguageMethods::after_holon_expansion(output,
                        docket->target->tangle_language, docket, holon->corresponding_chunk->owner);
                }
                break;
            }
            case FILE_NAME_START_FSMEVENT: {
                Str::clear(name);
                break;
            }
            case FILE_NAME_END_FSMEVENT: {
                int ho_len = Str::len(WebNotation::left(lsu->unit_notation, FILE_NAMED_HOLONS_NTNMARKER));
                int hc_len = Str::len(WebNotation::right(lsu->unit_notation, FILE_NAMED_HOLONS_NTNMARKER));
                Str::truncate(name, Str::len(name) - hc_len);
                ls_holon *holon = Holons::find_holon(name, lsu->local_holon_namespace, FALSE, NULL);
                if (holon) {
                    int excess = ho_len + Str::len(name) + hc_len;
                    Str::truncate(output, Str::len(output) - excess);
                    LanguageMethods::before_holon_expansion(output,
                        docket->target->tangle_language, docket, holon->corresponding_chunk->owner);
                    Tangler::tangle_holon(output, holon, docket, NULL);
                    LanguageMethods::after_holon_expansion(output,
                        docket->target->tangle_language, docket, holon->corresponding_chunk->owner);
                }
                break;
            }
        }
    }
    DISCARD_TEXT(name)
    Tangler::tangle_illiterate_code_fragment(OUT, output, docket);
}

§16. At last, genuinely illiterate code. It might seem that the thing now is just to print it verbatim to the output, but we still have to deal with two markup syntaxes used internally in the Inform compiler, and which trigger callbacks.

If we're tangling a literate program in the ordinary way, those callback functions won't be in the docket, so we will indeed just print the text, and this function will exit very quickly.

void Tangler::tangle_illiterate_code_fragment(OUTPUT_STREAM, text_stream *text,
    tangle_docket *docket) {
    if ((docket->command_callback) || (docket->bplus_callback)) {
        TEMPORARY_TEXT(command)
        TEMPORARY_TEXT(argument)
        int sfp = 0;
        inchar32_t cr = 0;
        do {
            Str::clear(command);
            Str::clear(argument);
            Read next character16.1;
            NewCharacter: if (cr == 0) break;
            Deal with material which isn't commentary16.2;
        } while (cr != 0);
        DISCARD_TEXT(command)
        DISCARD_TEXT(argument)
    } else if (docket->target) {
        LanguageMethods::tangle_line(OUT, docket->target->tangle_language, text);
    } else {
        WRITE("%S", text);
    }
}

§16.1. Read next character16.1 =

    cr = Str::get_at(text, sfp++);

This code is used in §16, §16.2 (twice), §16.2.1 and §16.2.2.

§16.2. Deal with material which isn't commentary16.2 =

    if (cr == '{') {
        Read next character16.1;
        if ((cr == '-') && (docket->command_callback)) {
            Read up to the next close brace as a braced command and argument16.2.1;
            if (Str::get_first_char(command) == '!') continue;
            (*(docket->command_callback))(OUT, command, argument, docket);
            continue;
        } else { /* otherwise the open brace was a literal */
            PUT_TO(OUT, '{');
            goto NewCharacter;
        }
    }
    if ((cr == '(') && (docket->bplus_callback)) {
        Read next character16.1;
        if (cr == '+') {
            Read up to the next plus close-bracket as an I7 expression16.2.2;
            continue;
        } else { /* otherwise the open bracket was a literal */
            PUT_TO(OUT, '(');
            goto NewCharacter;
        }
    }
    PUT_TO(OUT, cr);

This code is used in §16.

§16.2.1. And here we read a normal command. The command name must not include } or :. If there is no : then the argument is left unset (so that it will be the empty string: see above). The argument must not include }.

Read up to the next close brace as a braced command and argument16.2.1 =

    Str::clear(command);
    Str::clear(argument);
    int com_mode = TRUE;
    while (TRUE) {
        Read next character16.1;
        if ((cr == '}') || (cr == 0)) break;
        if ((cr == ':') && (com_mode)) { com_mode = FALSE; continue; }
        if (com_mode) PUT_TO(command, cr);
        else PUT_TO(argument, cr);
    }

This code is used in §16.2.

§16.2.2. And similarly, for the (+ ... +) notation which was once used to mark I7 material within I6:

Read up to the next plus close-bracket as an I7 expression16.2.2 =

    TEMPORARY_TEXT(material)
    while (TRUE) {
        Read next character16.1;
        if (cr == 0) break;
        if ((cr == ')') && (Str::get_last_char(material) == '+')) {
            Str::delete_last_character(material); break; }
        PUT_TO(material, cr);
    }
    (*(docket->bplus_callback))(material, docket);
    DISCARD_TEXT(material)

This code is used in §16.2.