Characters that killed a render, what may sit next to a klammer, and ref/

A character written in the ^...^ mnemonic form -- ^oe^, ^zz^, ^hello^ --
used to kill ktext outright: no message, no output, and an exit status
that looked like a crash because it was one.  Only the forms that happen
to spell hexadecimal digits escaped, and they were right by luck rather
than by reading.  Any caret form that names no character is now a located
error that quotes what was written.

A klammer may now sit inside square brackets and braces as it already
could inside parentheses, and may be followed by the punctuation that
ends a sentence: [@i word@], {@b word@}, @i word@'s.  The rule is a fixed
list of characters, because none of them can occur inside a klammer name
and so none of them is ambiguous next to a delimiter.  A letter still
cannot: "x@" is the named closing delimiter, so a letter before "@" has
two readings and no rule can separate them.

The abbreviations an author writes -- the hyphen form @i-word, a klammer
glued to its neighbouring punctuation -- are now recognised as author
conveniences only.  They are not applied to @@ definitions or @@@ system
commands, which are written by someone who needs no such shorthand and
whose forms are already whole.

Also fixed, each of them silent: an exported shell function in the
environment broke every @eval in the document; a line break written with
@nl was destroyed when it passed through a klammer that generates a
table, printing as \textbackslash{}newline in the finished PDF; and
"kdesc --coverage" shortened its file column by comparing the spelling of
a path rather than the path, so a klammer set reached by a symbolic link
showed a full path where its neighbours showed a short one.

New in the distribution: ref/, a small klammerset that describes the
Klammermachine's own tables in Klammertext.  It currently provides the
character tables -- the diacritics and the extended Latin letters -- as
klammers that render to any target, so the documentation of what can be
written cannot drift from what the machine accepts.  It is early, and
will grow.

The Klammertext taxonomy now has a section on targets, describes the
@@@klammerset system command alongside the other three, and states the
precedence of "^" -- what a caret means before a letter, a digit, a
punctuation character, and a quote, in one ordered list.

(from dev 6594c7ff8c1f)

mac/character.cpp

@@ -22,6 +22,7 @@ static std::string compose(const std::string& base, const std::string& mark_cp)
     return base + utf8char(std::stoi(mark_cp, nullptr, 16));
 }
 
+/*
 inline
 std::string klammertext_special_characters { "@|*^#" };
 
@@ -33,6 +34,7 @@ std::string diacritic_symbols = "-'`h~\"cbrdwa";
 
 inline
 std::string diacritic_symbols_order = "'`h~\"c-brdwa";
+*/
 
 std::string utf8char(int cp)
 {
@@ -122,6 +124,12 @@ std::string process_diacritics(std::string s)
     return result;
 }
 
+// A delimited caret form left unresolved by every pass in encode().  The
+// character class matches the nonascii Ktype's second alternative exactly:
+// what the lexer accepts as a shape, encode() must either resolve or
+// reject, or the two disagree about what is writable.
+const std::regex unresolved_nonascii_re(R"(\^[A-Za-z0-9]{1,6}\^)");
+
 std::string extended_latin_symbol_pattern()
 {
     std::string result {};
@@ -234,6 +242,26 @@ std::string encode(const std::string& s, const Locator& loc)
     result = process_pinyin(result);
     if (dbg) std::cout << "pinyin:   " << result << "\n";
 
+    // A "^...^" form that survives every pass above resolves to no
+    // character: it is not in the extended-Latin table, not a code point,
+    // not a diacritic, not a pinyin tone.  That is an ERROR here rather
+    // than text in the document -- the same rule as an invalid code point
+    // a few lines up, and for the same reason: a typo must not reach a
+    // finished-looking document as literal "^zz^".  Only the DELIMITED
+    // form is checked.  The two-character form (^a, ^-) is deliberately
+    // exempt: quoted punctuation resolves later, in hide_quoted_punctuation().
+    if (std::regex_search(result, unresolved_nonascii_re)) {
+        std::smatch hit {};
+        std::regex_search(result, hit, unresolved_nonascii_re);
+        throw Parsing_error("\"" + hit[0].str() + "\" is not a character code. "
+                            "It names no extended-Latin mnemonic and is not a "
+                            "Unicode code point. The forms are:\n\n"
+                            "    ^oe^      a mnemonic (kdesc -c lists them)\n"
+                            "    ^263A^    a code point, 4 to 6 hex digits\n"
+                            "    ^o\"       a diacritic on a base letter\n",
+                            loc);
+    }
+
     result = string_replace(result, lit_start, "^'");
     result = string_replace(result, lit_end, "'^");
 
@@ -259,6 +287,7 @@ void write_kt_example_file(std::stringstream& kt, std::string kt_filename)
 }
 
 
+/*
 void diacritics_examples(const std::string& kt_filename)
 {
     if (diacritics.size() != diacritic_symbols_order.size()) {
@@ -297,8 +326,9 @@ void diacritics_examples(const std::string& kt_filename)
     if (write_kt_file)
         write_kt_example_file(kt, kt_filename);
 }
+*/
 
-
+/*
 void extended_latin_examples(const std::string& kt_filename)
 {
     std::stringstream kt {};
@@ -328,7 +358,7 @@ void extended_latin_examples(const std::string& kt_filename)
     if (write_kt_file)
         write_kt_example_file(kt, kt_filename);
 }
-
+*/
 
 void pinyin_examples(const std::string& kt_filename)
 {
@@ -357,6 +387,11 @@ void pinyin_examples(const std::string& kt_filename)
         write_kt_example_file(kt, kt_filename);
 }
 
+/*
+  Outside my window
+  crows build a nest in a tree...
+  slowly... carefully.
+*/
 
 void unicode_examples(const std::string& kt_filename)
 {
@@ -399,7 +434,7 @@ void unicode_examples(const std::string& kt_filename)
 void show_special_characters()
 {
     std::cout << std::setfill(' ');
-    diacritics_examples();
-    extended_latin_examples();
+    // diacritics_examples();
+    // extended_latin_examples();
     pinyin_examples();
 }