ack/klammertext
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Characters that killed a render, what may sit next to a klammer, and ref/

Browse Source 
A character written in the ^...^ mnemonic form -- ^oe^, ^zz^, ^hello^ --
used to kill ktext outright: no message, no output, and an exit status
that looked like a crash because it was one.  Only the forms that happen
to spell hexadecimal digits escaped, and they were right by luck rather
than by reading.  Any caret form that names no character is now a located
error that quotes what was written.

A klammer may now sit inside square brackets and braces as it already
could inside parentheses, and may be followed by the punctuation that
ends a sentence: [@i word@], {@b word@}, @i word@'s.  The rule is a fixed
list of characters, because none of them can occur inside a klammer name
and so none of them is ambiguous next to a delimiter.  A letter still
cannot: "x@" is the named closing delimiter, so a letter before "@" has
two readings and no rule can separate them.

The abbreviations an author writes -- the hyphen form @i-word, a klammer
glued to its neighbouring punctuation -- are now recognised as author
conveniences only.  They are not applied to @@ definitions or @@@ system
commands, which are written by someone who needs no such shorthand and
whose forms are already whole.

Also fixed, each of them silent: an exported shell function in the
environment broke every @eval in the document; a line break written with
@nl was destroyed when it passed through a klammer that generates a
table, printing as \textbackslash{}newline in the finished PDF; and
"kdesc --coverage" shortened its file column by comparing the spelling of
a path rather than the path, so a klammer set reached by a symbolic link
showed a full path where its neighbours showed a short one.

New in the distribution: ref/, a small klammerset that describes the
Klammermachine's own tables in Klammertext.  It currently provides the
character tables -- the diacritics and the extended Latin letters -- as
klammers that render to any target, so the documentation of what can be
written cannot drift from what the machine accepts.  It is early, and
will grow.

The Klammertext taxonomy now has a section on targets, describes the
@@@klammerset system command alongside the other three, and states the
precedence of "^" -- what a caret means before a letter, a digit, a
punctuation character, and a quote, in one ordered list.

(from dev 6594c7ff8c1f)
This commit is contained in:
ack committed 2026-09-07 22:37:05 +02:00
1 parent b80e823ab3
commit e4f1307aa9
53 files changed
+2014 -350
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

mac/katom.cpp
+13 -179
View File
@@ -5,6 +5,7 @@
#include "katom.h"
#include "ktype.h"
#include "katomize.h"
#include "error.h"
#include "log.h"
#include "show.h"
@@ -13,8 +14,20 @@
bool dbg = false;
// KATOMIZATION IS NOT HERE. Turning text into katoms moved to
// katomize.{h,cpp} (2026-09-02), leaving this file the KATOM: what it is,
// what its type means, and the operations on a list of them. The test that
// decided the split is in katomize.h -- does a character stream enter?
// Everything below answers no.
size_t Katom::index = 0;
// A katom holds its source text and its text, and decides nothing about
// either. It used to strip the "^" from a quoted special here -- knowledge
// of what a construct looks like IN SOURCE, inside the datum, which is the
// one thing the katom/katomization split says does not belong here. That
// moved to source_katom() in katomize.{h,cpp} on 2026-09-02, the one place
// a katom is made from matched source text.
Katom::Katom(const std::string& src, katom_t type, Locator loc)
: m_index(Katom::index++)
, m_text(src)
@@ -23,9 +36,6 @@ Katom::Katom(const std::string& src, katom_t type, Locator loc)
, m_type(type)
, m_initial_type(type)
{
if (m_type == katom_t::special && m_text[0] == '^') {
m_text.erase(0, 1);
}
}
int active_count(const katom_list& katoms)
@@ -45,146 +55,10 @@ bool active(const katom_list& katoms)
return active_count(katoms) > 0;
}
std::string expand_compound_katom(const std::string& s, std::regex rgx, const std::string& expanded)
{
if (s.find('-') != std::string::npos) { // Hyphen shortcut for arguments that allow it
const std::regex shortcut_rgx(R"((@\w+)(-\w+)+)");
std::smatch match {};
if (std::regex_match(s, match, shortcut_rgx)) {
std::string modified = string_replace(s, "-", " | ") + " @";
return std::regex_replace(modified, std::regex(R"((@\w+) \|)"), "$1");
}
}
return std::regex_replace(s, rgx, expanded);
}
katom_list make_katoms_from_word(std::string s, const std::string& source_desc, int line, int chr)
{
if (dbg) msg() << " make word: " << broken_bar << s << broken_bar << " [" << chr << "]\n";
auto ktyp = std::find_if(katom_types.begin(), katom_types.end(), [&](const auto& ktype) { return ktype.match(s); });
if (ktyp != katom_types.end()) {
return { Katom(s, ktyp->m_type, Locator(source_desc, line, chr)) };
} else {
if (dbg) {
std::cout << "\nBREAK: |" << s << "|\n";
}
for (const auto& [desc, rgx, expanded] : katom_rewrite_rules) {
std::string modified = expand_compound_katom(s, rgx.m_regex, expanded);
if (dbg) {
msg() << "Rewrite: " << desc << " " << rgx.m_pattern << " " << expanded << "\n"; // output-policy-ok (dbg)
}
if (modified != s) {
auto parts = word_split(modified);
if (dbg) std::cout << " Parts: " << parts << "\n";
if (show_rewrite_rules) {
Locator loc(source_desc, line, chr);
std::cout << loc << " Rewrite (" << desc << "): " << rgx.m_pattern
<< " " << right_arrow << " " << expanded << "\n";
}
katom_list klist {};
for (const std::string& p : parts) {
if (!p.empty()) {
auto ks = make_katoms_from_word(p, source_desc, line, chr);
std::copy(ks.begin(), ks.end(), std::back_inserter(klist));
}
}
return klist;
}
}
Katom k(s, katom_t::word, Locator(source_desc, line, chr));
k.m_unparsed = true; // Warning deferred to warn_unparsed_katoms()
return std::vector{ k };
}
}
// Warn about words that matched no katom type -- but only those that
// survive processing: text removed by #, ##, or #[...]#, replaced spans,
// and literal content (definition interiors, @code bodies) never warn.
// Called at the end of Machine::process_katoms(), after those passes have
// marked the katoms. Clears the flag after warning so repeated processing
// of the same katom list does not warn twice. With warn=false (the @eval
// read-back sub-Machine, whose katoms hold machine-generated result text)
// no warning is printed and every flag is cleared, so the katoms stay
// silent after they are spliced into the calling Machine's list.
void warn_unparsed_katoms(katom_list& katoms, bool warn)
{
for (Katom& k : katoms) {
if (!k.m_unparsed) continue;
if (!warn) {
k.m_unparsed = false;
continue;
}
if (k.m_type != katom_t::ignored
&& k.m_type != katom_t::replaced
&& k.m_type != katom_t::literal) {
// One of the two surviving WARNINGS (the commands otherwise treat
// an anomaly as an error and stop -- see the output policy in
// CLAUDE.md). It stays a warning because the judgment is a
// HEURISTIC, not exact: the @eval read-back legitimately fails to
// parse raw target markup, which is why m_warn_unparsed exists. A
// heuristic that halts turns every false positive into a blocker on
// a document that would otherwise render. Promote it to an error
// when the judgment becomes exact.
//
// Through warning(), not a raw std::cerr, so every warning the
// commands issue has one format: it said "[warning]" where
// everything else says "(warning)".
warning("Word not parsed:\n " + k.m_text
+ "\nTo include a punctuation character literally, put "
"\"^\" before it.",
k.m_loc);
k.m_unparsed = false;
}
}
}
katom_list split_into_katoms(std::string s, const std::string& source, int source_line)
{
if (dbg) {
msg() << "Make katoms: " << s << "<\n"; // output-policy-ok (dbg)
}
// const std::string middle_dot { "\u00B7" };
s = string_replace(s, "\r", "");
s = string_replace(s, "\t", " ");
//std::regex words_regex("^\||[ ]|[\\n]|[^\\s]+|.+");
//std::regex words_regex(R"((?:[^][|])|[ ]|[\n]|[^\s]+|.+)");
std::regex words_regex(R"([ ]|[\n]|[^\s]+|.+)");
auto words_begin = std::sregex_iterator(s.begin(), s.end(), words_regex);
auto words_end = std::sregex_iterator();
if (dbg) {
std::cout << "Found " << std::distance(words_begin, words_end) << " words:\n";
}
strings_t atoms {};
for (std::sregex_iterator iter = words_begin; iter != words_end; ++iter) {
if (dbg) {
std::cout << middle_dot << iter->str();
}
atoms.push_back(iter->str());
}
if (dbg) std::cout << middle_dot << "\n";
//std::cout << kall << ktype;
katom_list result {};
int cpos = 0;
for (const auto& a : atoms) {
auto k = make_katoms_from_word(a, source, source_line, cpos);
for (auto& kk : k) {
Locator loc(source, source_line, cpos);
//kk->m_loc = loc;
kk.m_loc = loc;
//m_katoms.push_back(kk);
result.push_back(kk);
if (dbg) {
//std::cout << "LOOP: " << kk.m_text << " - " << cpos << "\n";
}
cpos += kk.m_src.size();
}
}
return result;
}
void restore_initial_type(katom_iter begin, katom_iter end)
{
@@ -292,51 +166,11 @@ katom_list trim(const katom_list& katoms, bool trim_inactive)
return trim(result, trim_types);
}
strings_t line_split(const std::string& s)
{
std::istringstream is(s);
strings_t result {};
std::string line;
while (std::getline(is, line)) {
result.push_back(line);
}
result.push_back("\n");
return result;
}
std::pair<std::string, strings_t> line_split(const fs::path& pathname)
{
std::string source {};
strings_t lines {};
std::ifstream infile(pathname);
std::string line;
while (std::getline(infile, line)) {
source += line + "\n";
lines.push_back(line);
}
if (!source.empty()) {
source.pop_back();
}
return {source, lines};
}
// I'm, like, this is
// some weird shit;
// what the fuck?
katom_list katomize(const strings_t& lines, const std::string& source_desc)
{
(void)K::log(3);
katom_list katoms {};
int i = 0;
for (std::string line : lines) {
(void)K::log(4, line);
katom_list ks = split_into_katoms(line + "\n", source_desc, ++i);
katoms.insert(katoms.end(), ks.begin(), ks.end());
}
katoms.pop_back();
return katoms;
}
// Whitespace