Phase II of the Markdown parser: fragmenting inline material into emphasis, links, backticked code and so forth.

• §1. Disclaimer
• §2. Inline code
• §4. Links and images
• §6. Emphasis

§1. Disclaimer. Do not call functions in this section directly: use the API in Markdown.

§2. Inline code. Phase II of the parser consists entirely of walking the tree built in Phase I, with the following function.

void MDInlineParser::inline_recursion?Usage of MDInlineParser::inline_recursion:
Markdown - §5.2(markdown_variation *variation,
    md_links_dictionary *link_refs, markdown_item *at) {
    if (at == NULL) return;
    if ((at->type == PARAGRAPH_MIT) ||
        (at->type == HEADING_MIT) ||
        ((at->type == TABLE_COLUMN_MIT) && (Str::len(at->stashed) > 0))) {
        markdown_item *matter = MDInlineParser::inline(variation, link_refs, at->stashed);
        Markdown::add_to(matter, at);
    }
    for (markdown_item *c = at->down; c; c = c->next)
        MDInlineParser::inline_recursion(variation, link_refs, c);
}

§3. What it does, then, is to look at the text of paragraphs and headings to process them as "inline matter", which may contain emphasis, code excerpts and raw HTML, or images, or links. Since all these constructions can be nested, the result is a further tree, using a whole lot of new items never generated in Phase I.

To recap the syntaxes we're looking for:

    This is a line of plain copy.
    This one has `a backtick string`.
    This one an autolink, <stanley.gibbons@heaven.org>, in email form.
    This one an autolink, <https://www.stanleygibbons.com>, in web form.
    This one [uses a link](https://www.stanleygibbons.com).
    This one depicts ![the Blue Mauritius](https://upload.wikimedia.org/wikipedia/commons/8/8e/1847_BothPost_Office_Jakubek.jpg).
    This one uses **strong** and *weak* emphasis.
    This one also uses __strong__ and _weak_ emphasis.

Every inline string becomes a subtree under a MATERIAL_MIT item. We then proceed in three stages: make the "chain", which involves finding backtick strings, raw HTML and autolinks, all constructions of equal precedence. Then we look for links and images, images being essentially a form of link for most purposes. Finally, but with some difficulty, we look for emphasis notation.

markdown_item *MDInlineParser::inline?Usage of MDInlineParser::inline:
§2(markdown_variation *variation,
    md_links_dictionary *link_refs, text_stream *text) {
    markdown_item *owner = Markdown::new_item(MATERIAL_MIT);
    MDInlineParser::make_inline_chain(variation, owner, text);
    MDInlineParser::links_and_images(variation, link_refs, owner, FALSE);
    int mask = 0;
    if (MarkdownVariations::supports(variation, ASTERISK_EMPHASIS_MARKDOWNFEATURE))
        mask += ASTERISK_EMPHASIS_BIT;
    if (MarkdownVariations::supports(variation, UNDERSCORE_EMPHASIS_MARKDOWNFEATURE))
        mask += UNDERSCORE_EMPHASIS_BIT;
    if (MarkdownVariations::supports(variation, STRIKETHROUGH_MARKDOWNFEATURE))
        mask += TILDE_STRIKETHROUGH_BIT;
    if (mask) MDInlineParser::emphasis(variation, owner, mask);
    return owner;
}

markdown_item *MDInlineParser::make_inline_chain(markdown_variation *variation,
    markdown_item *owner, text_stream *text) {
    int i = 0;
    while (Str::get_at(text, i) == ' ') i++;
    int from = i, escaped = FALSE;
    for (; i<Str::len(text); i++) {
        if ((escaped == FALSE) && (Str::get_at(text, i) == '\\') &&
            (Characters::is_ASCII_punctuation(Str::get_at(text, i+1)))) {
            escaped = TRUE;
        } else {
            if (escaped == FALSE) {
                Does a backtick begin here?3.2;
                Does an index mark begin here?3.3;
                Does an autolink begin here?3.4;
                Does a raw HTML tag begin here?3.9;
            }
            Does a hard or soft line break occur here?3.10;
            ContinueOuter: ;
            escaped = FALSE;
        }
    }
    if (from <= Str::len(text)-1) {
        int to = Str::len(text)-1;
        while (Str::get_at(text, to) == ' ') to--;
        if (to >= from) {
            markdown_item *md = Markdown::new_slice(PLAIN_MIT, text, from, to);
            Markdown::add_to(md, owner);
        }
    }
    return owner;
}

§3.1. See CommonMark 6.1: "A backtick string is a string of one or more backtick characters that is neither preceded nor followed by a backtick." This returns the length of a backtick string beginning at at, if one does, or 0 if it does not.

int MDInlineParser::backtick_string?Usage of MDInlineParser::backtick_string:
§3.2(text_stream *text, int at) {
    int count = 0;
    while (Str::get_at(text, at + count) == '`') count++;
    if (count == 0) return 0;
    if ((at > 0) && (Str::get_at(text, at - 1) == '`')) return 0;
    return count;
}

§3.2. Does a backtick begin here?3.2 =

    if (MarkdownVariations::supports(variation, BACKTICKED_CODE_MARKDOWNFEATURE)) {
        int count = MDInlineParser::backtick_string(text, i);
        if (count > 0) {
            for (int j=i+count+1; j<Str::len(text); j++) {
                if (MDInlineParser::backtick_string(text, j) == count) {
                    if (i-1 >= from) {
                        markdown_item *md = Markdown::new_slice(PLAIN_MIT, text, from, i-1);
                        Markdown::add_to(md, owner);
                    }
                    Insert an inline code item3.2.1;
                    i = j+count; from = j+count;
                    goto ContinueOuter;
                }
            }
        }
    }

• This code is used in §3.

§3.2.1. "The contents of the code span are the characters between these two backtick strings". Inside it, "line endings are converted to spaces", and "If the resulting string both begins and ends with a space character, but does not consist entirely of space characters, a single space character is removed from the front and back."

Insert an inline code item3.2.1 =

    int start = i+count, end = j-1;
    text_stream *codespan = Str::new();
    int all_spaces = TRUE;
    for (int k=start; k<=end; k++) {
        inchar32_t c = Str::get_at(text, k);
        if (c == '\n') c = ' ';
        if (c != ' ') all_spaces = FALSE;
        PUT_TO(codespan, c);
    }
    if ((all_spaces == FALSE) && (Str::get_first_char(codespan) == ' ')
         && (Str::get_last_char(codespan) == ' ')) {
        markdown_item *md = Markdown::new_slice(CODE_MIT, codespan, 1, Str::len(codespan)-2);
        Markdown::set_backtick_count(md, count);
        Markdown::add_to(md, owner);
    } else {
        markdown_item *md = Markdown::new_slice(CODE_MIT, codespan, 0, Str::len(codespan)-1);
        Markdown::set_backtick_count(md, count);
        Markdown::add_to(md, owner);
    }

• This code is used in §3.2.

§3.3. This provides an extension borrowed from traditional TeX manual-indexing notation, also used by indoc.

Does an index mark begin here?3.3 =

    if (MarkdownVariations::supports(variation, INDEXING_MARKS_MARKDOWNFEATURE)) {
        if (Str::get_at(text, i) == '^') {
            int j = i+1, count = 1;
            if (Str::get_at(text, j) == '^') j++, count++;
            if (Str::get_at(text, j) == '^') j++, count++;
            if (Str::get_at(text, j) == '{') {
                TEMPORARY_TEXT(lemma)
                j++;
                int escaped = FALSE, end_found = FALSE;
                while (Str::get_at(text, j)) {
                    if ((escaped == FALSE) && (Str::get_at(text, j) == '\\'))
                        escaped = TRUE;
                    else {
                        if ((escaped == FALSE) && (Str::get_at(text, j) == '}')) {
                            end_found = TRUE; break;
                        }
                        PUT_TO(lemma, Str::get_at(text, j));
                        escaped = FALSE;
                    }
                    j++;
                }
                if (end_found) {
                    if (i-1 >= from) {
                        markdown_item *md = Markdown::new_slice(PLAIN_MIT, text, from, i-1);
                        Markdown::add_to(md, owner);
                    }
                    Insert an index item3.3.1;
                    i = j; from = j+1;
                }
                DISCARD_TEXT(lemma)
                if (end_found) goto ContinueOuter;
            }
        }
    }

• This code is used in §3.

§3.3.1. Insert an index item3.3.1 =

    markdown_item *md = Markdown::new_item(INDEX_MARKER_MIT);
    md->stashed = Str::duplicate(lemma);
    md->details = count;
    Markdown::add_to(md, owner);
    if (count == 1) {
        markdown_item *md = Markdown::new_slice(PLAIN_MIT, Str::duplicate(lemma), 0, Str::len(lemma)-1);
        Markdown::add_to(md, owner);
    }

• This code is used in §3.3.

§3.4. Does an autolink begin here?3.4 =

    if (Str::get_at(text, i) == '<') {
        for (int j=i+1; j<Str::len(text); j++) {
            inchar32_t c = Str::get_at(text, j);
            if (c == '>') {
                int link_from = i+1, link_to = j-1, count = j-i+1;
                if (tracing_Markdown_parser) {
                    text_stream *OUT = STDOUT;
                    WRITE("Investigating potential autolink: ");
                    for (int k=i; k<=j; k++) PUT(Str::get_at(text, k));
                    WRITE("\n");
                }
                if (MarkdownVariations::supports(variation, WEB_AUTOLINKS_MARKDOWNFEATURE))
                    Test for URI autolink3.4.1;
                if (MarkdownVariations::supports(variation, EMAIL_AUTOLINKS_MARKDOWNFEATURE))
                    Test for email autolink3.4.2;
                break;
            }
            if ((c == '<') ||
                (Characters::is_Unicode_whitespace(c)) ||
                (Characters::is_control_character(c)))
                break;
        }
    }
    if (MarkdownVariations::supports(variation, EXTENDED_AUTOLINKS_MARKDOWNFEATURE)) {
        if ((MDInlineParser::extended_autolink_domain_char(Str::get_at(text, i))) &&
            ((i == 0) ||
                (Str::get_at(text, i-1) == '\n') ||
                (Str::get_at(text, i-1) == '*') ||
                (Str::get_at(text, i-1) == '_') ||
                (Str::get_at(text, i-1) == '~') ||
                ((Str::get_at(text, i-1) == '(') && (Str::get_at(text, i-2) != ']')) ||
                (Characters::is_Unicode_whitespace(Str::get_at(text, i-1)))))
                    Test for extended autolink3.4.3;
    }

• This code is used in §3.

§3.4.1. "A URI autolink consists of... a scheme followed by a colon followed by zero or more characters other than ASCII control characters, space, <, and >... a scheme is any sequence of 2–32 characters beginning with an ASCII letter and followed by any combination of ASCII letters, digits, or the symbols plus, period, or hyphen."

Test for URI autolink3.4.1 =

    int colon_at = -1;
    for (int k=link_from; k<=link_to; k++) if (Str::get_at(text, k) == ':') { colon_at = k; break; }
    if (colon_at >= 0) {
        int scheme_valid = TRUE;
        Vet the scheme3.4.1.1;
        int link_valid = TRUE;
        Vet the link3.4.1.2;
        if ((scheme_valid) && (link_valid)) {
            if (i-1 >= from) {
                markdown_item *md = Markdown::new_slice(PLAIN_MIT, text, from, i-1);
                Markdown::add_to(md, owner);
            }
            markdown_item *md = Markdown::new_slice(URI_AUTOLINK_MIT,
                text, link_from, link_to);
            Markdown::add_to(md, owner);
            i = link_to+1; from = link_to+2;
            if (tracing_Markdown_parser) PRINT("Found URI from = %c\n", Markdown::get_at(md, from));
            goto ContinueOuter;
        } else if (tracing_Markdown_parser) {
            if (scheme_valid == FALSE) PRINT("Colon suggested URI but scheme invalid\n");
            if (link_valid == FALSE) PRINT("Colon suggested URI but link invalid\n");
        }
    } else {
        if (tracing_Markdown_parser) PRINT("Not a URI: no colon\n");
    }

• This code is used in §3.4.

§3.4.1.1. Vet the scheme3.4.1.1 =

    int scheme_length = colon_at - link_from;
    if ((scheme_length < 2) || (scheme_length > 32)) scheme_valid = FALSE;
    for (int i=link_from; i<colon_at; i++) {
        inchar32_t c = Str::get_at(text, i);
        if (!((Characters::is_ASCII_letter(c)) ||
            ((i > link_from) &&
                ((Characters::is_ASCII_digit(c)) || (c == '+') || (c == '-') || (c == '.')))))
            scheme_valid = FALSE;
    }

• This code is used in §3.4.1.

§3.4.1.2. Vet the link3.4.1.2 =

    for (int i=colon_at+1; i<=link_to; i++) {
        inchar32_t c = Str::get_at(text, i);
        if ((c == '<') || (c == '>') || (c == ' ') ||
            (Characters::is_control_character(c)))
            link_valid = FALSE;
    }

• This code is used in §3.4.1.

§3.4.2. Test for email autolink3.4.2 =

    int atsign_at = -1;
    for (int k=link_from; k<=link_to; k++) if (Str::get_at(text, k) == '@') { atsign_at = k; break; }
    if (atsign_at >= 0) {
        int username_valid = TRUE;
        Vet the username3.4.2.1;
        int domain_valid = TRUE;
        Vet the domain name3.4.2.2;
        if ((username_valid) && (domain_valid)) {
            if (i-1 >= from) {
                markdown_item *md = Markdown::new_slice(PLAIN_MIT, text, from, i-1);
                Markdown::add_to(md, owner);
            }
            markdown_item *md = Markdown::new_slice(EMAIL_AUTOLINK_MIT,
                text, link_from, link_to);
            Markdown::set_add_protocol_state(md, TRUE);
            Markdown::add_to(md, owner);
            i = j+count; from = j+count;
            if (tracing_Markdown_parser) PRINT("Found email\n");
            goto ContinueOuter;
        } else if (tracing_Markdown_parser) {
            if (username_valid == FALSE) PRINT("At suggested email but username invalid\n");
            if (domain_valid == FALSE) PRINT("At suggested email but domain invalid\n");
        }
    } else {
        if (tracing_Markdown_parser) PRINT("Not an email: no at-sign\n");
    }

• This code is used in §3.4.

§3.4.2.1. What constitutes a legal email address follows the HTML 5 regular expression, according to CommonMark. Good luck using {{@1-x.2.z.w as your email address, but you absolutely can.

Vet the username3.4.2.1 =

    int username_length = atsign_at - link_from;
    if (username_length < 1) username_valid = FALSE;
    for (int i=link_from; i<atsign_at; i++) {
        inchar32_t c = Str::get_at(text, i);
        if (!((Characters::is_ASCII_letter(c)) ||
                (Characters::is_ASCII_digit(c)) ||
                (c == '.') ||
                (c == '!') ||
                (c == '#') ||
                (c == '$') ||
                (c == '%') ||
                (c == '&') ||
                (c == '\'') ||
                (c == '*') ||
                (c == '+') ||
                (c == '/') ||
                (c == '=') ||
                (c == '?') ||
                (c == '^') ||
                (c == '_') ||
                (c == '`') ||
                (c == '{') ||
                (c == '|') ||
                (c == '}') ||
                (c == '~') ||
                (c == '-')))
            username_valid = FALSE;
    }

• This code is used in §3.4.2.

§3.4.2.2. Vet the domain name3.4.2.2 =

    int segment_length = 0;
    for (int i=atsign_at+1; i<=link_to; i++) {
        inchar32_t c = Str::get_at(text, i);
        if (segment_length == 0) {
            if (!((Characters::is_ASCII_letter(c)) || (Characters::is_ASCII_digit(c))))
                domain_valid = FALSE;
        } else {
            if (c == '.') { segment_length = 0; continue; }
            if (c == '-') {
                if ((Str::get_at(text, i+1) == 0) || (Str::get_at(text, i+1) == '.'))
                    domain_valid = FALSE;
            } else if (!((Characters::is_ASCII_letter(c)) || (Characters::is_ASCII_digit(c))))
                domain_valid = FALSE;
        }
        segment_length++;
        if (segment_length >= 64) domain_valid = FALSE;
    }
    if (segment_length >= 64) domain_valid = FALSE;

• This code is used in §3.4.2.

§3.4.3. Extended autolinks are a GitHub-flavored Markdown extension, and allow (some) websites and email addresses to be turned into links without needing angle brackets around them. This part of GitHub's specification uses sloppily different criteria for what is a legal domain or email address, but I follow it as written.

Test for extended autolink3.4.3 =

    int domain_from = i, to = i;
    int add_protocol = FALSE, email_address = FALSE, email_required = FALSE, xmpp = FALSE;
    Look for a domain prefix3.4.3.1;
    if ((email_required) || (to == domain_from)) Look for an email address opening3.4.3.2;
    if (((email_required == FALSE) || (email_address)) && (to > domain_from)) {
        if (tracing_Markdown_parser) {
            PRINT("Found valid extended autolink prefix: ");
            for (int j=domain_from; j<=to; j++) PUT_TO(STDOUT, Str::get_at(text, j));
            PRINT("\n");
        }
        Look for the rest of the domain3.4.3.3;
        int domain_name_invalid = FALSE;
        Test the domain name for validity3.4.3.4;
        if (domain_name_invalid == FALSE) {
            Push the link forwards through URL or resource3.4.3.5;
            Insert as an extended autolink3.4.3.6;
        }
    }

• This code is used in §3.4.

§3.4.3.1. Look for a domain prefix3.4.3.1 =

    if ((Str::get_at(text, domain_from) == 'w') &&
        (Str::get_at(text, domain_from+1) == 'w') &&
        (Str::get_at(text, domain_from+2) == 'w') &&
        (Str::get_at(text, domain_from+3) == '.') &&
        (MDInlineParser::extended_autolink_domain_char(Str::get_at(text, domain_from+4)))) {
        add_protocol = TRUE;
        to = domain_from+4;
    } else if ((Str::get_at(text, domain_from) == 'm') &&
        (Str::get_at(text, domain_from+1) == 'a') &&
        (Str::get_at(text, domain_from+2) == 'i') &&
        (Str::get_at(text, domain_from+3) == 'l') &&
        (Str::get_at(text, domain_from+4) == 't') &&
        (Str::get_at(text, domain_from+5) == 'o') &&
        (Str::get_at(text, domain_from+6) == ':') &&
        (MDInlineParser::extended_autolink_domain_char(Str::get_at(text, domain_from+7)))) {
        to = domain_from+7; email_required = TRUE;
    } else if ((Str::get_at(text, domain_from) == 'x') &&
        (Str::get_at(text, domain_from+1) == 'm') &&
        (Str::get_at(text, domain_from+2) == 'p') &&
        (Str::get_at(text, domain_from+3) == 'p') &&
        (Str::get_at(text, domain_from+4) == ':') &&
        (MDInlineParser::extended_autolink_domain_char(Str::get_at(text, domain_from+5)))) {
        to = domain_from+5; email_required = TRUE; xmpp = TRUE;
    } else if ((Str::get_at(text, domain_from) == 'h') &&
        (Str::get_at(text, domain_from+1) == 't') &&
        (Str::get_at(text, domain_from+2) == 't') &&
        (Str::get_at(text, domain_from+3) == 'p') &&
        (Str::get_at(text, domain_from+4) == ':') &&
        (Str::get_at(text, domain_from+5) == '/') &&
        (Str::get_at(text, domain_from+6) == '/') &&
        (MDInlineParser::extended_autolink_domain_char(Str::get_at(text, domain_from+7)))) {
        to = domain_from+7;
    } else if ((Str::get_at(text, domain_from) == 'h') &&
        (Str::get_at(text, domain_from+1) == 't') &&
        (Str::get_at(text, domain_from+2) == 't') &&
        (Str::get_at(text, domain_from+3) == 'p') &&
        (Str::get_at(text, domain_from+4) == 's') &&
        (Str::get_at(text, domain_from+5) == ':') &&
        (Str::get_at(text, domain_from+6) == '/') &&
        (Str::get_at(text, domain_from+7) == '/') &&
        (MDInlineParser::extended_autolink_domain_char(Str::get_at(text, domain_from+8)))) {
        to = domain_from+8;
    }

• This code is used in §3.4.3.

§3.4.3.2. Look for an email address opening3.4.3.2 =

    int j = to;
    while ((MDInlineParser::extended_autolink_email_char(Str::get_at(text, j))) ||
        (Str::get_at(text, j) == '+') ||
        (Str::get_at(text, j) == '.')) j++;
    if ((j > to) && (Str::get_at(text, j) == '@')) {
        to = j+1; email_address = TRUE;
        if (email_required == FALSE) add_protocol = TRUE;
    }

• This code is used in §3.4.3.

§3.4.3.3. Look for the rest of the domain3.4.3.3 =

    if (email_address) {
        while (TRUE) {
            if (MDInlineParser::extended_autolink_email_char(Str::get_at(text, to+1))) to++;
            else if ((Str::get_at(text, to+1) == '.') && (Str::get_at(text, to) != '.')) to++;
            else break;
        }
    } else {
        while (TRUE) {
            if (MDInlineParser::extended_autolink_domain_char(Str::get_at(text, to+1))) to++;
            else if ((Str::get_at(text, to+1) == '.') && (Str::get_at(text, to) != '.')) to++;
            else break;
        }
    }
    if (Str::get_at(text, to) == '.') to--;

• This code is used in §3.4.3.

§3.4.3.4. Test the domain name for validity3.4.3.4 =

    int dot_count = 0;
    for (int j=domain_from; j<=to; j++) if (Str::get_at(text, j) == '.') dot_count++;
    if (dot_count == 0) domain_name_invalid = TRUE;
    if (email_address) {
        if ((Str::get_at(text, to) == '_') || (Str::get_at(text, to) == '-'))
            domain_name_invalid = TRUE;
    } else {
        for (int j=domain_from, dots_passed=0; j<=to; j++) {
            inchar32_t c = Str::get_at(text, j);
            if (c == '.') dots_passed++;
            if ((c == '_') && (dots_passed >= dot_count - 2)) domain_name_invalid = TRUE;
        }
    }

• This code is used in §3.4.3.

§3.4.3.5. Push the link forwards through URL or resource3.4.3.5 =

    int domain_to = to;
    if (email_address == FALSE) {
        while ((Str::get_at(text, to+1) != 0) && (Str::get_at(text, to+1) != '<') &&
            (Characters::is_Unicode_whitespace(Str::get_at(text, to+1)) == FALSE)) to++;
        while (MDInlineParser::extended_autolink_trailing_punctuation_char(Str::get_at(text, to))) to--;
    } else if (xmpp) {
        if (Str::get_at(text, to+1) == '/') {
            to++;
            while (MDInlineParser::extended_autolink_xmpp_resource_char(Str::get_at(text, to+1))) to++;
        }
    }
    while (TRUE) {
        int initial_to = to;
        if (Str::get_at(text, to) == ')') {
            int bl = 0;
            for (int j=domain_to+1; j<=to; j++) {
                if (Str::get_at(text, j) == '(') bl++;
                if (Str::get_at(text, j) == ')') bl--;
                if (bl < 0) { to = j-1; break; }
            }
        } else if (Str::get_at(text, to) == ';') {
            int b = to-1;
            while (Characters::isalnum(Str::get_at(text, b))) b--;
            if (Str::get_at(text, b) == '#') b--;
            if (Str::get_at(text, b) == '&') to = b-1;
        }
        if (to == initial_to) break;
    }

• This code is used in §3.4.3.

§3.4.3.6. Insert as an extended autolink3.4.3.6 =

    if (i-1 >= from) {
        markdown_item *md = Markdown::new_slice(PLAIN_MIT, text, from, i-1);
        Markdown::add_to(md, owner);
    }
    int link_from = domain_from, link_to = to;
    i = link_to; from = link_to+1;
    int type = URI_AUTOLINK_MIT;
    if (email_address) type = EMAIL_AUTOLINK_MIT;
    if (xmpp) type = XMPP_AUTOLINK_MIT;
    markdown_item *md = Markdown::new_slice(type,
        text, link_from, link_to);
    Markdown::set_add_protocol_state(md, add_protocol);
    Markdown::add_to(md, owner);
    if (tracing_Markdown_parser) {
        PRINT("Found extended autolink ");
        for (int j=domain_from; j<=to; j++) PUT_TO(STDOUT, Str::get_at(text, j));
        PRINT("\n");
    }
    goto ContinueOuter;

• This code is used in §3.4.3.

§3.5. "Alphanumeric characters, underscores and hyphens." It's not entirely clear what the GitHub-flavored Markdown specification means by "alphanumeric" here: are arbitrary Unicode letters and numerals allowed? I'm going to say not.

int MDInlineParser::extended_autolink_domain_char?Usage of MDInlineParser::extended_autolink_domain_char:
§3.4, §3.4.3.1, §3.4.3.3(inchar32_t c) {
    if ((Characters::isalnum(c)) || (c == '_') || (c == '-')) return TRUE;
    return FALSE;
}

§3.6. "One ore [sic] more characters which are alphanumeric, or ., -, _, or +." But we take care of + and . above.

int MDInlineParser::extended_autolink_email_char?Usage of MDInlineParser::extended_autolink_email_char:
§3.4.3.2, §3.4.3.3(inchar32_t c) {
    if ((Characters::isalnum(c)) || (c == '_') || (c == '-')) return TRUE;
    return FALSE;
}

§3.7. "The resource can contain all alphanumeric characters, as well as @ and .."

int MDInlineParser::extended_autolink_xmpp_resource_char?Usage of MDInlineParser::extended_autolink_xmpp_resource_char:
§3.4.3.5(inchar32_t c) {
    if ((Characters::isalnum(c)) || (c == '@') || (c == '.')) return TRUE;
    return FALSE;
}

§3.8. "Trailing punctuation (specifically, ?, !, ., ,, :, *, _, and ~) will not be considered part of the autolink."

int MDInlineParser::extended_autolink_trailing_punctuation_char?Usage of MDInlineParser::extended_autolink_trailing_punctuation_char:
§3.4.3.5(inchar32_t c) {
    if ((c == '?') || (c == '!') || (c == ',') || (c == '.') ||
        (c == ':') || (c == '*') || (c == '_') || (c == '~')) return TRUE;
    return FALSE;
}

§3.9. Does a raw HTML tag begin here?3.9 =

    if (MarkdownVariations::supports(variation, INLINE_HTML_MARKDOWNFEATURE)) {
        int filter = NOT_APPLICABLE;
        if (Str::get_at(text, i) == '<') {
            switch (Str::get_at(text, i+1)) {
                case '?': Does a processing instruction begin here?3.9.1; break;
                case '!':
                    if ((Str::get_at(text, i+2) == '-') && (Str::get_at(text, i+3) == '-'))
                        Does an HTML comment begin here?3.9.2;
                    if ((Str::get_at(text, i+2) == '[') && (Str::get_at(text, i+3) == 'C') &&
                        (Str::get_at(text, i+4) == 'D') && (Str::get_at(text, i+5) == 'A') &&
                        (Str::get_at(text, i+6) == 'T') && (Str::get_at(text, i+7) == 'A') &&
                        (Str::get_at(text, i+8) == '['))
                        Does a CDATA section begin here?3.9.4;
                    if (Characters::is_ASCII_letter(Str::get_at(text, i+2)))
                        Does an HTML declaration begin here?3.9.3;
                    break;
                case '/': Does a close tag begin here?3.9.6; break;
                default: Does an open tag begin here?3.9.5; break;
            }
            NotATag: ;
        }
    }

• This code is used in §3.

§3.9.1. The content of a PI must be non-empty.

Does a processing instruction begin here?3.9.1 =

    for (int j = i+3; j<Str::len(text); j++)
        if ((Str::get_at(text, j) == '?') && (Str::get_at(text, j+1) == '>')) {
            int tag_from = i, tag_to = j+1;
            Allow it as a raw HTML tag3.9.1.1;
        }

• This code is used in §3.9.

§3.9.2. A comment can be empty, but cannot end in a dash or contain a double-dash:

Does an HTML comment begin here?3.9.2 =

    int bad_start = FALSE;
    if (Str::get_at(text, i+4) == '>') bad_start = TRUE;
    if ((Str::get_at(text, i+4) == '-') && (Str::get_at(text, i+5) == '>')) bad_start = TRUE;
    if (bad_start == FALSE)
        for (int j = i+4; j<Str::len(text); j++)
            if ((Str::get_at(text, j) == '-') && (Str::get_at(text, j+1) == '-')) {
                if (Str::get_at(text, j+2) == '>') {
                    int tag_from = i, tag_to = j+2;
                    Allow it as a raw HTML tag3.9.1.1;
                }
                break;
            }

• This code is used in §3.9.

§3.9.3. The content of a declaration can be empty.

Does an HTML declaration begin here?3.9.3 =

    for (int j = i+2; j<Str::len(text); j++)
        if (Str::get_at(text, j) == '>') {
            int tag_from = i, tag_to = j;
            Allow it as a raw HTML tag3.9.1.1;
        }

• This code is used in §3.9.

§3.9.4. The content of a CDATA must be non-empty.

Does a CDATA section begin here?3.9.4 =

    for (int j = i+10; j<Str::len(text); j++)
        if ((Str::get_at(text, j) == ']') && (Str::get_at(text, j+1) == ']') &&
            (Str::get_at(text, j+2) == '>')) {
            int tag_from = i, tag_to = j+2;
            Allow it as a raw HTML tag3.9.1.1;
        }

• This code is used in §3.9.

§3.9.5. Does an open tag begin here?3.9.5 =

    int at = i+1;
    filter = FALSE;
    Advance past tag name3.9.5.1;
    Advance past attributes3.9.5.2;
    Advance past optional tag-whitespace3.9.5.4;
    if (Str::get_at(text, at) == '/') at++;
    if (Str::get_at(text, at) == '>') {
        int tag_from = i, tag_to = at;
        Allow it as a raw HTML tag3.9.1.1;
    }

• This code is used in §3.9.

§3.9.6. Does a close tag begin here?3.9.6 =

    int at = i+2;
    Advance past tag name3.9.5.1;
    Advance past optional tag-whitespace3.9.5.4;
    if (Str::get_at(text, at) == '>') {
        int tag_from = i, tag_to = at;
        Allow it as a raw HTML tag3.9.1.1;
    }

• This code is used in §3.9.

§3.9.5.1. Advance past tag name3.9.5.1 =

    inchar32_t c = Str::get_at(text, at);
    if (Characters::is_ASCII_letter(c) == FALSE) goto NotATag;
    TEMPORARY_TEXT(tag)
    while ((c == '-') || (Characters::is_ASCII_letter(c)) || (Characters::is_ASCII_digit(c))) {
        PUT_TO(tag, c);
        c = Str::get_at(text, ++at);
    }
    if ((filter == FALSE) &&
        (MarkdownVariations::supports(variation, DISALLOWED_RAW_HTML_MARKDOWNFEATURE)) &&
        (Markdown::tag_should_be_filtered(tag))) filter = TRUE;
    DISCARD_TEXT(tag)

• This code is used in §3.9.5, §3.9.6.

§3.9.5.2. Advance past attributes3.9.5.2 =

    while (TRUE) {
        int start_at = at;
        Advance past optional tag-whitespace3.9.5.4;
        if (at == start_at) break;
        inchar32_t c = Str::get_at(text, at);
        if ((c == '_') || (c == ':') || (Characters::is_ASCII_letter(c))) {
            while ((c == '_') || (c == ':') || (c == '.') || (c == '-') ||
                (Characters::is_ASCII_letter(c)) || (Characters::is_ASCII_digit(c)))
                c = Str::get_at(text, ++at);
            int start_value_at = at;
            Advance past optional tag-whitespace3.9.5.4;
            if (Str::get_at(text, at) != '=') {
                at = start_value_at; goto DoneValueSpecification;
            }
            at++;
            Advance past optional tag-whitespace3.9.5.4;
            Try for a single-quoted attribute value3.9.5.2.2;
            Try for a double-quoted attribute value3.9.5.2.3;
            Try for an unquoted attribute value3.9.5.2.1;
            DoneValueSpecification: ;
        } else { at = start_at; break; }
    }

• This code is used in §3.9.5.

§3.9.5.2.1. Try for an unquoted attribute value3.9.5.2.1 =

    int k = at;
    while (TRUE) {
        inchar32_t c = Str::get_at(text, k);
        if ((c == ' ') || (c == '\t') || (c == '\n') || (c == '"') || (c == '\'') ||
            (c == '=') || (c == '<') || (c == '>') || (c == '`') || (c == 0))
            break;
        k++;
    }
    if (k == at) { at = start_value_at; goto DoneValueSpecification; }
    at = k; goto DoneValueSpecification;

• This code is used in §3.9.5.2.

§3.9.5.2.2. Try for a single-quoted attribute value3.9.5.2.2 =

    if (Str::get_at(text, at) == '\'') {
        int k = at + 1;
        while ((Str::get_at(text, k) != '\'') && (Str::get_at(text, k) != 0))
            k++;
        if (Str::get_at(text, k) == '\'') { at = k+1; goto DoneValueSpecification; }
        at = start_value_at; goto DoneValueSpecification;
    }

• This code is used in §3.9.5.2.

§3.9.5.2.3. Try for a double-quoted attribute value3.9.5.2.3 =

    if (Str::get_at(text, at) == '"') {
        int k = at + 1;
        while ((Str::get_at(text, k) != '"') && (Str::get_at(text, k) != 0))
            k++;
        if (Str::get_at(text, k) == '"') { at = k+1; goto DoneValueSpecification; }
        at = start_value_at; goto DoneValueSpecification;
    }

• This code is used in §3.9.5.2.

§3.9.5.3. Advance past compulsory tag-whitespace3.9.5.3 =

    inchar32_t c = Str::get_at(text, at);
    if ((c != ' ') && (c != '\t') && (c != '\n')) goto NotATag;
    Advance past optional tag-whitespace3.9.5.4;

• This code is never used.

§3.9.5.4. Advance past optional tag-whitespace3.9.5.4 =

    int line_ending_count = 0;
    while (TRUE) {
        inchar32_t c = Str::get_at(text, at++);
        if (c == '\n') {
            line_ending_count++;
            if (line_ending_count == 2) break;
        }
        if ((c != ' ') && (c != '\t') && (c != '\n')) break;
    }
    at--;

• This code is used in §3.9.5, §3.9.6, §3.9.5.2 (three times), §3.9.5.3.

§3.9.1.1. Allow it as a raw HTML tag3.9.1.1 =

    if (i-1 >= from) {
        markdown_item *md = Markdown::new_slice(PLAIN_MIT, text, from, i-1);
        Markdown::add_to(md, owner);
    }
    markdown_item *md = Markdown::new_slice(INLINE_HTML_MIT, text, tag_from, tag_to);
    if (filter == TRUE) Markdown::set_filtered_state(md, TRUE);
    Markdown::add_to(md, owner);
    i = tag_to; from = tag_to + 1;
    if (tracing_Markdown_parser) PRINT("Found raw HTML\n");
    goto ContinueOuter;

• This code is used in §3.9.1, §3.9.2, §3.9.3, §3.9.4, §3.9.5, §3.9.6.

§3.10. Does a hard or soft line break occur here?3.10 =

    if (Str::get_at(text, i) == '\n') {
        int soak = 0;
        if (Str::get_at(text, i-1) == '\\') soak = 2;
        int preceding_spaces = 0;
        while (Str::get_at(text, i-1-preceding_spaces) == ' ') preceding_spaces++;
        if (preceding_spaces >= 2) soak = preceding_spaces+1;
        if (soak > 0) {
            if (i-soak >= from) {
                markdown_item *md = Markdown::new_slice(PLAIN_MIT, text, from, i-soak);
                Markdown::add_to(md, owner);
            }
            markdown_item *md = Markdown::new_slice(LINE_BREAK_MIT, I"\n\n", 0, 1);
            Markdown::add_to(md, owner);
        } else {
            if (i-preceding_spaces-1 >= from) {
                markdown_item *md = Markdown::new_slice(PLAIN_MIT, text, from, i-preceding_spaces-1);
                Markdown::add_to(md, owner);
            }
            markdown_item *md = Markdown::new_slice(SOFT_BREAK_MIT, I"\n", 0, 0);
            Markdown::add_to(md, owner);
        }
        i++;
        while (Str::get_at(text, i) == ' ') i++;
        from = i;
        i--;
        if (tracing_Markdown_parser) PRINT("Found line break\n");
        goto ContinueOuter;
    }

• This code is used in §3.

§4. Links and images. Well, so now we come to the middle-precedence items: links and images. These have equal precedence, so they ought to be read from left to right, but there's a twist: a link is not valid if its linked material contains another link.

    This [looks like [a link within a link](somewhere.html)](nowhere.html).

results in only the inner link being processed as such. That restriction does not hold for images, so this is all legal:

    This ![picture of [an unperforated Malaya strip](malaya.html)](strip.jpg).

Indeed, image notations can contain other image notations. Of course, that does no good, because rendering uses the textual part of an image as its "alt" text, which is not allowed to contain tags, only plain text. But syntactically it is legal Markdown.

void MDInlineParser::links_and_images?Usage of MDInlineParser::links_and_images:
§3, §4.2(markdown_variation *variation,
    md_links_dictionary *link_refs, markdown_item *owner, int images_only) {
    if (owner == NULL) return;
    if ((MarkdownVariations::supports(variation, LINKS_MARKDOWNFEATURE) == FALSE) &&
        (MarkdownVariations::supports(variation, IMAGES_MARKDOWNFEATURE) == FALSE))
        return;
    if (tracing_Markdown_parser) {
        PRINT("Beginning link/image pass:\n");
        Markdown::debug_subtree(STDOUT, owner);
    }
    md_charpos leftmost_pos = Markdown::left_edge_of(owner->down);
    while (TRUE) {
        if (tracing_Markdown_parser) {
            if (Markdown::somewhere(leftmost_pos)) {
                PRINT("Link/image notation scan from %c\n",
                    Markdown::get(leftmost_pos));
                Markdown::debug_subtree(STDOUT, leftmost_pos.md);
            } else {
                PRINT("Link/image notation scan from start\n");
            }
        }
        md_link_parse found = MDInlineParser::first_valid_link(variation,
            link_refs, leftmost_pos, Markdown::nowhere(), images_only, FALSE);
        if (found.is_link == NOT_APPLICABLE) break;
        md_link_dictionary_entry *ref = found.link_reference;
        if (tracing_Markdown_parser) {
            PRINT("Link matter: ");
            if (found.link_text_empty) PRINT("EMPTY\n");
            else Markdown::debug_interval(STDOUT, found.link_text_from, found.link_text_to);
            if (ref) {
                PRINT("Link destination (reference): %S\n", ref->destination);
                PRINT("Link title (reference): %S\n", ref->title);
            } else {
                PRINT("Link destination: ");
                if (found.link_destination_empty) PRINT("EMPTY\n");
                else Markdown::debug_interval(STDOUT, found.link_destination_from, found.link_destination_to);
                PRINT("Link title: ");
                if (found.link_title_empty) PRINT("EMPTY\n");
                else Markdown::debug_interval(STDOUT, found.link_title_from, found.link_title_to);
            }
        }
        Deal with a valid link or image4.2;
    }
}

§4.1. The following structure holds the findings of MDInlineParser::first_valid_link if it succeeds. Note that a link/image which makes a reference is only syntactically valid if the reference matches something in the dictionary.

It might seem redundant to have the *_empty fields below, but there are some awkward cases where, as a result of earlier splicing, it's difficult to represent nonexistent text with md_charpos structs.

typedef struct md_link_parse {
    int is_link;  TRUE for link, FALSE for image, NOT_APPLICABLE for fail
    struct md_charpos first;                   leftmost character of the whole construct
    struct md_charpos link_text_from;          leftmost character inside the squares
    struct md_charpos link_text_to;            ...and so on
    int link_text_empty;                       TRUE or FALSE
    struct md_charpos link_destination_from;
    struct md_charpos link_destination_to;
    int link_destination_empty;
    struct md_charpos link_title_from;
    struct md_charpos link_title_to;
    int link_title_empty;
    struct md_link_dictionary_entry *link_reference;  or NULL if it's not by reference
    struct md_charpos last;  rightmost character of the whole construct
} md_link_parse;

• The structure md_link_parse is private to this section.

§4.2. See Markdown for how all this splicing is done.

Deal with a valid link or image4.2 =

    markdown_item *chain = owner->down, *found_text = NULL, *remainder = NULL;
    Markdown::cut_interval(chain, found.first, found.last, &chain, &found_text, &remainder);

    markdown_item *link_text = NULL;
    markdown_item *link_destination = NULL;
    markdown_item *link_title = NULL;
    if (found.link_text_empty == FALSE)
        Markdown::cut_interval(found_text, found.link_text_from, found.link_text_to,
            NULL, &link_text, &found_text);
    if ((Markdown::somewhere(found.link_destination_from)) &&
        (found.link_destination_empty == FALSE))
        Markdown::cut_interval(found_text, found.link_destination_from, found.link_destination_to,
            NULL, &link_destination, &found_text);
    if ((Markdown::somewhere(found.link_title_from)) && (found.link_title_empty == FALSE))
        Markdown::cut_interval(found_text, found.link_title_from, found.link_title_to,
            NULL, &link_title, &found_text);
    markdown_item *link_item = Markdown::new_item((found.is_link == TRUE)?LINK_MIT:IMAGE_MIT);
    markdown_item *matter = Markdown::new_item(MATERIAL_MIT);
    if (found.link_text_empty == FALSE) matter->down = link_text;
    Markdown::add_to(matter, link_item);
    if (found.is_link == TRUE) MDInlineParser::links_and_images(variation, link_refs, matter, TRUE);
    else MDInlineParser::links_and_images(variation, link_refs, matter, FALSE);
    if (ref) {
        if (Str::len(ref->destination) > 0) {
            markdown_item *dest_item = Markdown::new_item(LINK_DEST_MIT);
            dest_item->down = Markdown::new_slice(PLAIN_MIT, ref->destination, 0, Str::len(ref->destination)-1);
            Markdown::add_to(dest_item, link_item);
        }
        if (Str::len(ref->title) > 0) {
            markdown_item *title_item = Markdown::new_item(LINK_TITLE_MIT);
            title_item->down = Markdown::new_slice(PLAIN_MIT, ref->title, 0, Str::len(ref->title)-1);
            Markdown::add_to(title_item, link_item);
        }
    } else {
        if (link_destination) {
            markdown_item *dest_item = Markdown::new_item(LINK_DEST_MIT);
            if (found.link_destination_empty == FALSE) dest_item->down = link_destination;
            Markdown::add_to(dest_item, link_item);
        }
        if (link_title) {
            markdown_item *title_item = Markdown::new_item(LINK_TITLE_MIT);
            if (found.link_title_empty == FALSE) title_item->down = link_title;
            Markdown::add_to(title_item, link_item);
        }
    }
    if (chain) {
        owner->down = chain;
        while (chain->next) chain = chain->next; chain->next = link_item;
    } else {
        owner->down = link_item;
    }
    link_item->next = remainder;
    if (tracing_Markdown_parser) {
        PRINT("After link surgery:\n");
        Markdown::debug_subtree(STDOUT, owner);
    }
    leftmost_pos = Markdown::left_edge_of(remainder);

• This code is used in §4.

§5. Okay, so it's time to go looking for the leftmost valid link or image in the given part of a chain of nodes. Links begin with an unescaped [ and images with an unescaped ![.

md_link_parse MDInlineParser::first_valid_link?Usage of MDInlineParser::first_valid_link:
§4, §5.2.1(markdown_variation *variation,
    md_links_dictionary *link_refs, md_charpos from, md_charpos to,
    int images_only, int links_only) {
    md_link_parse result;
    Initialise the parse result to a no5.1;
    inchar32_t prev_c = 0;
    md_charpos prev_pos = Markdown::nowhere();
    int escaped = FALSE;
    while ((Markdown::somewhere(from)) && (Markdown::plainish(from.md) == FALSE))
        from = Markdown::advance_up_to(from, to);
    for (md_charpos pos = from; Markdown::somewhere(pos); pos = Markdown::advance_up_to(pos, to)) {
        inchar32_t c = Markdown::get(pos);
        if ((c == '\\') && (escaped == FALSE)) escaped = TRUE;
        else {
            if ((c == '[') && (escaped == FALSE)) {
                md_charpos pass_pos = pos;
                for (int pass=1; pass<=2; pass++) {
                    if (tracing_Markdown_parser) {
                        PRINT("Pass %d: at ", pass);
                        Markdown::debug_pos(STDOUT, pos);
                        PRINT("\n");
                    }
                    if (pass == 2) pos = pass_pos;
                    See if a link begins here5.2;
                    AbandonHope: ;
                }
            }
            if (escaped == FALSE) {
                prev_c = c;
                prev_pos = pos;
            }
            escaped = FALSE;
        }
    }
    return result;
}

§5.1. Initialise the parse result to a no5.1 =

    result.is_link = NOT_APPLICABLE;
    result.first = Markdown::nowhere();
    result.link_text_from = Markdown::nowhere();
    result.link_text_to = Markdown::nowhere();
    result.link_text_empty = NOT_APPLICABLE;
    result.link_destination_from = Markdown::nowhere();
    result.link_destination_to = Markdown::nowhere();
    result.link_destination_empty = NOT_APPLICABLE;
    result.link_title_from = Markdown::nowhere();
    result.link_title_to = Markdown::nowhere();
    result.link_title_empty = NOT_APPLICABLE;
    result.link_reference = FALSE;
    result.last = Markdown::nowhere();

• This code is used in §5.

§5.2.

define ABANDON_LINK(reason)
    { if (tracing_Markdown_parser) { PRINT("Link abandoned: %s\n", reason); }
    pos = abandon_at; goto AbandonHope; }

See if a link begins here5.2 =

    if (((links_only == FALSE) || (prev_c != '!')) &&
        ((images_only == FALSE) || (prev_c == '!'))) {
        int link_rather_than_image = TRUE, uses = LINKS_MARKDOWNFEATURE;
        result.first = pos;
        if ((prev_c == '!') && (links_only == FALSE)) {
            link_rather_than_image = FALSE;
            uses = IMAGES_MARKDOWNFEATURE;
            result.first = prev_pos;
        }
        if (MarkdownVariations::supports(variation, uses)) {
            if (link_rather_than_image) {
                if (tracing_Markdown_parser) PRINT("Potential link found\n");
            } else {
                if (tracing_Markdown_parser) PRINT("Potential image found\n");
            }
            md_charpos abandon_at = pos;
            Work out the link text5.2.1;
            if (Markdown::get(pos) == '[') {
                Work out the reference5.2.2;
            } else {
                if ((Markdown::get(pos) != '(') || (pass == 2)) {
                    TEMPORARY_TEXT(label)
                    for (md_charpos pos = result.link_text_from; Markdown::somewhere(pos);
                        pos = Markdown::advance(pos)) {
                        PUT_TO(label, Markdown::get(pos));
                        if (Markdown::pos_eq(pos, result.link_text_to)) break;
                    }
                    Deal with escape characters in the label5.2.3;
                    md_link_dictionary_entry *ref = Markdown::look_up(link_refs, label);
                    if (ref == NULL) ABANDON_LINK("no '(' and not a valid reference");
                    #ifdef SUPERVISOR_MODULE
                    DocumentationCompiler::notify_image_use(ref->destination);
                    #endif
                    result.link_reference = ref;
                    pos = result.link_text_to;
                    pos = Markdown::advance_up_to(pos, to);
                    DISCARD_TEXT(label)
                } else {
                    pos = Markdown::advance_up_to_quasi_plainish_only(pos, to);
                    Advance pos by optional small amount of white space5.2.6;
                    if (Markdown::get(pos) != ')') Work out the link destination5.2.4;
                    Advance pos by optional small amount of white space5.2.6;
                    if (Markdown::get(pos) != ')') Work out the link title5.2.5;
                    Advance pos by optional small amount of white space5.2.6;
                    if (Markdown::get(pos) != ')') ABANDON_LINK("no ')'");
                }
            }
            result.last = pos;
            result.is_link = link_rather_than_image;
            if (tracing_Markdown_parser) PRINT("Confirmed\n");
            return result;
        }
    }

• This code is used in §5.

§5.2.1. Work out the link text5.2.1 =

    inchar32_t c = Markdown::get(pos);
    md_charpos prev_pos = pos;
    result.link_text_from = Markdown::advance_up_to(pos, to);
    inchar32_t prev_c = 0;
    int bl = 0, count = 0, escaped = FALSE;
    while (c != 0) {
        if ((c == '\\') && (escaped == FALSE)) {
            escaped = TRUE;
        } else {
            count++;
            if ((c == '[') && (escaped == FALSE)) bl++;
            if ((c == ']') && (escaped == FALSE)) { bl--; if (bl == 0) break; }
            escaped = FALSE;
        }
            prev_pos = pos;
            prev_c = c;
            pos = Markdown::advance_up_to(pos, to);
            c = Markdown::get(pos);
    }
    if (c == 0) { pos = abandon_at; ABANDON_LINK("no end to linked matter"); }
    result.link_text_empty = (count<=2)?TRUE:FALSE;
    result.link_text_to = prev_pos;
    if (link_rather_than_image) {
        md_link_parse nested =
            MDInlineParser::first_valid_link(variation, link_refs,
                result.link_text_from, result.link_text_to, FALSE, TRUE);
        if (nested.is_link != NOT_APPLICABLE) return nested;
    }
    pos = Markdown::advance_up_to_plainish_only(pos, to);

• This code is used in §5.2.

§5.2.2. Work out the reference5.2.2 =

    md_charpos prev_pos = pos;
    pos = Markdown::advance_up_to_plainish_only(pos, to);
    result.link_destination_from = pos;
    inchar32_t prev_c = 0;
    int bl = 1, escaping = FALSE;
    TEMPORARY_TEXT(label)
    inchar32_t c = Markdown::get(pos);
    while (c != 0) {
        if ((c == '\\') && (escaping == FALSE)) {
            escaping = TRUE;
        } else {
            if (escaping) {
                if ((c != '[') && (c != ']') && (c != '\\')) PUT_TO(label, '\\');
            } else {
                if (c == '[') bl++;
                if (c == ']') { bl--; if (bl == 0) break; }
            }
            PUT_TO(label, c);
            escaping = FALSE;
        }
        prev_pos = pos;
        prev_c = c;
        pos = Markdown::advance_up_to_plainish_only(pos, to);
        c = Markdown::get(pos);
    }
    if (c == 0) { pos = abandon_at; ABANDON_LINK("no end to reference"); }
    if (Str::len(label) == 0) {
        for (md_charpos pos = result.link_text_from; Markdown::somewhere(pos);
            pos = Markdown::advance(pos)) {
            PUT_TO(label, Markdown::get(pos));
            if (Markdown::pos_eq(pos, result.link_text_to)) break;
        }
    }
    md_link_dictionary_entry *ref = Markdown::look_up(link_refs, label);
    if (ref == NULL) ABANDON_LINK("unknown reference");
    result.link_reference = ref;
    #ifdef SUPERVISOR_MODULE
    DocumentationCompiler::notify_image_use(ref->destination);
    #endif