ack/klammertext
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

An output policy for the three commands, and @cond as a true special form

Browse Source 
A snapshot of the development tree.  The substantial changes since the last one:

COMMAND OUTPUT POLICY.  The three commands display text in exactly three cases,
and each owns a stream: LOGGING under "-v" greater than 0 and an ERROR before
termination go to STDERR; OUTPUT THE USER ASKED FOR goes to STDOUT.  For ktext
that output is a document, so "ktext doc.kt -d | ..." is now safe -- logging
used to share the stream and land inside the document.  A bare command prints
its usage and succeeds rather than failing.  Colour is emitted only to a
terminal, per stream, and NO_COLOR is honoured.

"-v 1" reports every decision whose outcome you could not have read off your own
input: the klammerset that was loaded and from which file, a font's directory, a
":files" name's file, how "-o" was expanded.  Higher levels are the trace.

The commands no longer warn and continue: an anomaly is an error, described with
its location.  Two exceptions remain, each for a stated reason -- a condition
that is expected and temporary by design, and a judgment that is a heuristic
rather than exact.

@cond IS NOW A TRUE SPECIAL FORM, resolved at APPLICATION time rather than when
the file is read.  Two consequences for a writer:

  * a state variable reaches the predicate.  "@@@state Flag :value true @@@
    @cond *Flag* | T | F @" renders "T"; it used to see the literal "*Flag*" and
    silently take the false branch.  The document now behaves like a klammer
    body, whose arguments are bound before its conditionals are decided.
  * nothing in a discarded branch happens -- it is not read, not evaluated, not
    expanded.  An @eval in the branch not taken used to run anyway.

Its predicate relation is total and strict: true, True, 1; false, False, 0, and
empty; anything else is an error at the @cond rather than silently false.

@eval REACHING OUTSIDE.  ":shell" and ":haskell" now keep the command's standard
error out of the document (it appears under "-v 1") and treat a nonzero exit as
an error naming what the command reported.  A command that exits nonzero on
purpose -- "grep" finding no match -- says so with "|| true".

KLAMMER SETS.  Several combine: "--klammersets a b c" loads all three in the
order given, sharing one namespace, with the definition modes deciding
collisions.  "none" means none and may not be combined with other symbols.  A
klammerset with symbol X is declared in a file X/X.k, which is what lets two
sets require the same third set without loading it twice.

TESTS.  Four new suites: the kdiag command's interface, the @eval primitive's
contract with the outside world, and verbosity at both tiers.  Three suites
that could not run on macOS at all now do.

Assembled from dev commit 6c8ee6c22fca.
This commit is contained in:
Andy Kopra
2026-08-16 01:37:59 +02:00
parent 59c1599bc9
commit 240cff4278
84 changed files with 2523 additions and 494 deletions
Download Patch File Download Diff File Expand all files Collapse all files

225
mac/machine.cpp
View File

@@ -81,6 +81,9 @@ void Machine::process_eval_katoms(katom_list& katoms)
if (std::find_if(katoms.begin(), katoms.end(), begin_eval) != katoms.end()) {
for (const auto& [op, cl] : find_spans(katoms, level_increase, level_decrease, true, "eval")) {
auto [begin, end] = find_span_katoms(katoms, op, cl);
// Inert: inside a @cond branch that has not been selected. An
// @eval in a discarded branch must not run.
if (begin->m_deferred) continue;
if (begin_eval(*begin)) {
Eval E(*this, begin->m_loc);
katom_list eval_katoms = E.eval(begin, end);
@@ -138,18 +141,22 @@ bool is_true(const std::string& s)
return s == "True" || s == "true" || s == "1";
}
// @cond's predicate relation is currently partial in effect: is_true()
// recognizes three strings as true and treats EVERYTHING else as false, so a
// misspelled state variable, a "TRUE", a "yes", or a Python traceback all
// silently select the false branch.
// @cond's predicate relation is TOTAL AND STRICT (Andy, 2026-08-15, deciding
// notes/Klammertext_improvements.md §4.1): there is a defined true set, a
// defined false set, and anything else is an error at the @cond.
//
// What the truth values should be is an open language-policy question (see
// notes/Klammertext_improvements.md, "The @cond predicate relation"), so the
// semantics here is deliberately unchanged. What is added is visibility: a
// predicate outside the provisionally recognized sets below is reported, with
// its value and location, so the cases can be found in real documents while
// the policy is decided. The recognized false set carries no semantics -- it
// exists only to keep the diagnostic quiet for values that plainly mean false.
// It was partial in effect until then -- is_true() recognized three strings and
// treated EVERYTHING else as false, so a misspelled state variable, a "TRUE", a
// "yes", or a Python traceback all silently selected the false branch. The
// 2026-08-01 work made that visible with a warning while the policy was
// undecided; the warning found nothing in the SKS, which is the evidence that
// the corpus uses well-formed predicates and that the blast radius is small.
//
// Empty stays in the FALSE set, and deliberately: an optional argument that
// was not written substitutes as empty, so "absent means false" is what
// carries the "@cond *opt* | ... @" idiom. The entangled sub-question in
// §4.1 -- whether empty means false or means "not supplied" -- is answered
// "false" by that use, not left open.
bool is_recognized_predicate(const std::string& s)
{
return s.empty()
@@ -157,25 +164,71 @@ bool is_recognized_predicate(const std::string& s)
|| s == "False" || s == "false" || s == "0";
}
void warn_unrecognized_predicate(const std::string& predicate, const Locator& loc)
void check_predicate(const std::string& predicate, const Locator& loc)
{
if (is_recognized_predicate(predicate)) return;
std::stringstream ss {};
ss << "The @cond predicate " << q_(predicate)
<< " is not a recognized truth value, so the false branch was taken.\n"
<< " Recognized: true, True, 1 (true); false, False, 0, empty (false).";
warning(ss.str(), loc);
ss << "The @cond predicate " << q_(predicate) << " is not a truth value. "
<< "Recognized: true, True, 1 (true); false, False, 0, and empty (false). "
<< "A value outside these is an error rather than false, so a misspelled "
<< "variable or a failed @eval cannot silently select a branch.";
throw Argument_error(ss.str(), loc);
}
void Machine::process_cond_katoms(katom_list& katoms)
// Mark the interior of every @cond span INERT. Runs during process_katoms,
// before the passes with observable effects, so that @eval and @read inside a
// branch do nothing until a branch is selected -- which is the non-strictness
// doc/cond_evaluation_order.md already specifies ("with side-effecting
// @read/@eval, wrong ... must not read the missing file") and which @eval did
// not honour: an @eval in a discarded branch used to run, because eval swept
// the list before cond did.
//
// The delimiters themselves stay unmarked, so the span is still found later.
// Nested @cond spans are marked by the enclosing one and become live only when
// the branch holding them is selected and processed.
void Machine::mark_cond_content(katom_list& katoms)
{
(void)K::log(4);
if (std::find_if(katoms.begin(), katoms.end(), begin_cond) == katoms.end()) {
return;
}
for (const auto& [op, cl] : find_spans(katoms, level_increase, level_decrease, true, "cond")) {
auto [begin, end] = find_span_katoms(katoms, op, cl);
if (!begin_cond(*begin)) continue;
// Only the BRANCHES are inert. The predicate is always evaluated --
// that is what a conditional is -- so marking from "begin + 1" would
// stop "@cond @eval 1==1 @ | yes | no @" from ever computing its own
// predicate. Mark from the first depth-0 bar onward.
std::vector<katom_iter> bars = cond_separator_bars(begin, end);
if (bars.empty()) continue; // malformed; reported when it resolves
for (auto k = bars[0] + 1; k < end - 1; ++k) {
k->m_deferred = true;
}
}
}
// Resolve the @cond spans in `katoms`, at APPLICATION time. Returns the number
// resolved, so the caller's fixed point accounts for them.
//
// The selected branch is spliced and then processed exactly as a klammer body
// is (process_katoms + apply, the two lines apply_klammer already uses): that
// is what makes the document behave like a function body whose state variables
// are its arguments -- they are bound by the substitution at the top of
// Machine::apply, BEFORE any conditional in the document is decided. Resolving
// @cond at read time meant a top-level "@cond *Flag*" saw the literal "*Flag*".
int Machine::resolve_cond_katoms(katom_list& katoms, const std::string& target)
{
int resolved = 0;
if (std::find_if(katoms.begin(), katoms.end(), begin_cond) != katoms.end()) {
(void)K::log(3);
//for (auto [op, cl] : find_spans(katoms, begin_cond, end_apply, true, "cond")) {
for (const auto& [op, cl] : find_spans(katoms, level_increase, level_decrease, true, "cond")) {
auto [begin, end] = find_span_katoms(katoms, op, cl);
// msg() << "find_spans: " << std::pair(begin, end) << "\n";
if (begin_cond(*begin)) {
// A @cond nested inside an unresolved outer @cond is still
// inert; the outer one will process it when its branch is
// selected. Without this an inner branch would be decided
// before it is known whether it is reached at all.
if (begin->m_deferred) continue;
// Delimit @cond's arguments by the bars at depth 0 within the
// span, so that bars belonging to nested klammers are not
// mistaken for @cond's own separators (see cond_separator_bars).
@@ -183,7 +236,7 @@ void Machine::process_cond_katoms(katom_list& katoms)
check_bar_count(begin, bars.size());
auto bar_1 = bars[0];
std::string predicate = to_string(begin + 1, bar_1, true);
warn_unrecognized_predicate(predicate, begin->m_loc);
check_predicate(predicate, begin->m_loc);
katom_list true_clause {};
katom_list false_clause {};
if (bars.size() == 2) {
@@ -199,11 +252,22 @@ void Machine::process_cond_katoms(katom_list& katoms)
// (@cond is a non-strict special form).
katom_list result = is_true(predicate)
? trim_whitespace(true_clause) : trim_whitespace(false_clause);
// The selected branch becomes live: clear the inert flag and
// give it the same processing a klammer body gets. The
// unselected branch is discarded still inert, so nothing in it
// ever ran.
for (Katom& k : result) {
k.m_deferred = false;
}
process_katoms(result, command_name);
apply(m_klammers, result, target);
std::for_each(begin, end, mark_as_replaced);
katoms.insert(end, result.begin(), result.end());
++resolved;
}
}
}
return resolved;
}
@@ -339,8 +403,15 @@ void Machine::process_katoms(
if (ignore) mark_ignored_katoms(katoms);
if (whitespace) process_whitespace_modifiers(katoms);
if (klammers) process_klammer_katoms(katoms);
// @cond is no longer RESOLVED here -- it is resolved in the apply fold
// (see mark_cond_content / resolve_cond_katoms). What happens here is the
// marking that makes its branches inert, and it must run BEFORE the eval
// and read passes, which are the ones with observable effects. Leaving it
// where process_cond_katoms used to sit -- after eval -- kept the old
// "eval in a discarded branch runs anyway" behaviour, since the marking
// arrived too late to stop it.
if (cond) mark_cond_content(katoms);
if (eval) process_eval_katoms(katoms);
if (cond) process_cond_katoms(katoms);
if (read) expand_read_katoms(
katoms, source,
nonascii, literal, ignore, whitespace, klammers, eval, cond, read);
@@ -402,6 +473,8 @@ void Machine::expand_read_katoms(
(void)K::log(3);
for (const auto& [op, cl] : find_spans(katoms, begin_apply, end_apply, true, "read")) {
auto [begin, end] = find_span_katoms(katoms, op, cl);
// Inert: see the same guard in process_eval_katoms.
if (begin->m_deferred) continue;
if (begin_read(*begin)) {
std::string read_filename = to_string(begin + 1, end - 1, true);
@@ -441,7 +514,27 @@ void Machine::expand_read_katoms(
}
void Machine::extract_machine_definitions()
// One @@@ declaration. Factored out so the tolerant and strict paths share
// it; `rescan` is set when a klammerset has spliced files into the stream,
// which invalidates the caller's span list.
void Machine::add_machine_definition(
const std::string& name, katom_iter begin, katom_iter end, bool& rescan)
{
if (name == "@@@target") {
m_targets.add(begin, end, m_katoms);
} else if (name == "@@@argtype") {
m_argtypes.add(begin, end, m_katoms);
} else if (name == "@@@state") {
m_state.parse_state_katoms(begin, end, m_katoms);
} else if (name == "@@@klammerset") {
if (auto klammerset = m_klammersets.add(begin, end, m_katoms)) {
load_klammerset_files(*klammerset, end);
rescan = true;
}
}
}
void Machine::extract_machine_definitions(bool tolerant)
{
(void)K::log(3);
if (m_katoms.empty()) {
@@ -456,20 +549,23 @@ void Machine::extract_machine_definitions()
rescan = false;
for (const auto& [op, cl] : find_spans(m_katoms, begin_machine_def, end_machine_def, true, command_name)) {
auto [begin, end] = find_span_katoms(m_katoms, op, cl);
//std::string name = trim_char(begin->m_text, '@');
std::string name = begin->m_text;
if (name == "@@@target") {
m_targets.add(begin, end, m_katoms);
} else if (name == "@@@argtype") {
m_argtypes.add(begin, end, m_katoms);
} else if (name == "@@@state") {
m_state.parse_state_katoms(begin, end, m_katoms);
} else if (name == "@@@klammerset") {
if (auto klammerset = m_klammersets.add(begin, end, m_katoms)) {
load_klammerset_files(*klammerset, end);
rescan = true;
break;
if (tolerant) {
// kdiag: a declaration that cannot be carried out -- a
// @@@klammerset naming a file that is not there, say -- is
// skipped rather than fatal. Its span stays unreplaced, so
// the katoms remain visible and report the failure themselves.
try {
add_machine_definition(name, begin, end, rescan);
} catch (Error& e) {
K::log(1, "Machine definition not registered: " + e.m_desc);
continue;
}
} else {
add_machine_definition(name, begin, end, rescan);
}
if (rescan) {
break;
}
}
}
@@ -491,17 +587,39 @@ void Machine::load_klammerset_files(const Klammerset& klammerset, katom_iter ins
// entries are always filenames (this set's own definition files).
std::vector<std::string> filenames {};
for (const auto& required : klammerset.m_requires) {
if (is_klammerset_symbol(required)) {
filenames.push_back(
resolve_klammerset_symbol(required, base_dir, klammerset.m_loc).string());
} else {
filenames.push_back(required);
std::string path = is_klammerset_symbol(required)
? resolve_klammerset_symbol(required, base_dir, klammerset.m_loc).string()
: required;
// ALREADY LOADED? Decided BEFORE the file is opened. The guard used
// to sit in Klammerset_registry::add, which only runs once the file has
// been read and its declaration reached -- by which time the declaring
// file's OWN definitions have been re-executed. Two sets requiring a
// third therefore died on "Target ... is already defined", pointing at
// a line the author wrote once.
//
// Deciding it here is possible only because a klammerset symbol X is
// declared in X/X.k, so the symbol is a function of the path: it is
// known for a ":requires" written as a filename just as much as for one
// written as a symbol. That is what the X/X.k requirement bought.
std::string symbol = is_klammerset_symbol(required)
? required : fs::path(path).stem().string();
if (m_klammersets.has(symbol)) {
(void)K::log(1, "Klammerset \"" + symbol + "\": already loaded, skipped");
continue;
}
filenames.push_back(path);
}
filenames.insert(filenames.end(), klammerset.m_files.begin(), klammerset.m_files.end());
// Collect all files into one list and insert once: insert_at is
// invalidated by the first insertion into m_katoms.
// Recorded on the set: what it actually opened, canonical, so a display
// can tell its klammers from the input's. The declaring file counts --
// definitions may sit before or after the declaration in it.
strings_t loaded_files {};
if (fs::exists(declaring)) {
loaded_files.push_back(fs::canonical(declaring).string());
}
katom_list loaded {};
for (const auto& filename : filenames) {
fs::path pathname = resolve_relative_to(filename, base);
@@ -512,6 +630,7 @@ void Machine::load_klammerset_files(const Klammerset& klammerset, katom_iter ins
klammerset.m_loc);
}
pathname = fs::canonical(pathname);
loaded_files.push_back(pathname.string());
m_state.add_search_dir(pathname.parent_path().string());
std::string text = trim_right(string_from_file(pathname.string()));
katom_list ks = katomize(line_split(text), pathname);
@@ -519,6 +638,9 @@ void Machine::load_klammerset_files(const Klammerset& klammerset, katom_iter ins
loaded.insert(loaded.end(), ks.begin(), ks.end());
}
m_katoms.insert(insert_at, loaded.begin(), loaded.end());
// The registry holds the registered copy; the parameter is a const
// reference to it, so the record goes back through the registry.
m_klammersets.set_loaded_files(klammerset.m_symbol, loaded_files);
}
// Register one "@@...@@" definition. An ".o" target declares an option set
@@ -541,19 +663,29 @@ void Machine::add_definition(katom_list& katoms, const Katom& op, const Katom& c
void Machine::extract_klammer_definitions(katom_list katoms)
{
fmsg() << katoms << "\n";
(void)K::log(3);
// fmsg() << katoms << "\n";
(void)K::log(3, katoms);
for (const auto& [op, cl] : find_spans(katoms, begin_klammer_def, end_klammer_def, true, command_name)) {
add_definition(katoms, op, cl);
}
m_klammers.rationalize(m_targets);
}
void Machine::extract_klammer_definitions()
void Machine::extract_klammer_definitions(bool tolerant)
{
(void)K::log(3);
for (const auto& [op, cl] : find_spans(m_katoms, begin_klammer_def, end_klammer_def, true, command_name)) {
add_definition(m_katoms, op, cl);
if (tolerant) {
// add_definition marks the span replaced only on success, so a
// skipped definition keeps its katoms and stays visible.
try {
add_definition(m_katoms, op, cl);
} catch (Error& e) {
K::log(1, "Definition not registered: " + e.m_desc);
}
} else {
add_definition(m_katoms, op, cl);
}
}
m_klammers.rationalize(m_targets);
}
@@ -677,7 +809,12 @@ int Machine::apply(
Klammer_registry& klammer_registry, katom_list& katoms, const std::string& target)
{
(void)K::log(3, "Klammer_registry");
int applied = 0;
// @cond is a special form handled here rather than by the klammer loop
// below: its span head is a cond_begin, which begin_klammer_apply does not
// match. Resolving first means a klammer revealed by the selected branch
// is applied in this same pass. The count is returned with the klammer
// applications, so the caller's fixed point iterates while either happens.
int applied = resolve_cond_katoms(katoms, target);
for (const auto& [op, cl] : find_spans(
katoms, begin_klammer_apply, end_klammer_apply, true, command_name)) {
auto [begin, end] = find_span_katoms(katoms, op, cl);