Andy Kopra
240cff4278
A snapshot of the development tree. The substantial changes since the last one:
COMMAND OUTPUT POLICY. The three commands display text in exactly three cases,
and each owns a stream: LOGGING under "-v" greater than 0 and an ERROR before
termination go to STDERR; OUTPUT THE USER ASKED FOR goes to STDOUT. For ktext
that output is a document, so "ktext doc.kt -d | ..." is now safe -- logging
used to share the stream and land inside the document. A bare command prints
its usage and succeeds rather than failing. Colour is emitted only to a
terminal, per stream, and NO_COLOR is honoured.
"-v 1" reports every decision whose outcome you could not have read off your own
input: the klammerset that was loaded and from which file, a font's directory, a
":files" name's file, how "-o" was expanded. Higher levels are the trace.
The commands no longer warn and continue: an anomaly is an error, described with
its location. Two exceptions remain, each for a stated reason -- a condition
that is expected and temporary by design, and a judgment that is a heuristic
rather than exact.
@cond IS NOW A TRUE SPECIAL FORM, resolved at APPLICATION time rather than when
the file is read. Two consequences for a writer:
* a state variable reaches the predicate. "@@@state Flag :value true @@@
@cond *Flag* | T | F @" renders "T"; it used to see the literal "*Flag*" and
silently take the false branch. The document now behaves like a klammer
body, whose arguments are bound before its conditionals are decided.
* nothing in a discarded branch happens -- it is not read, not evaluated, not
expanded. An @eval in the branch not taken used to run anyway.
Its predicate relation is total and strict: true, True, 1; false, False, 0, and
empty; anything else is an error at the @cond rather than silently false.
@eval REACHING OUTSIDE. ":shell" and ":haskell" now keep the command's standard
error out of the document (it appears under "-v 1") and treat a nonzero exit as
an error naming what the command reported. A command that exits nonzero on
purpose -- "grep" finding no match -- says so with "|| true".
KLAMMER SETS. Several combine: "--klammersets a b c" loads all three in the
order given, sharing one namespace, with the definition modes deciding
collisions. "none" means none and may not be combined with other symbols. A
klammerset with symbol X is declared in a file X/X.k, which is what lets two
sets require the same third set without loading it twice.
TESTS. Four new suites: the kdiag command's interface, the @eval primitive's
contract with the outside world, and verbosity at both tiers. Three suites
that could not run on macOS at all now do.
Assembled from dev commit 6c8ee6c22fca.
490 lines
20 KiB
C++
490 lines
20 KiB
C++
#include <algorithm>
|
|
#include <functional>
|
|
#include <iomanip>
|
|
#include <ostream>
|
|
#include <set>
|
|
|
|
#include "coverage.h"
|
|
#include "machine.h"
|
|
#include "katom.h"
|
|
#include "util.h"
|
|
|
|
namespace {
|
|
|
|
using target_set = std::set<std::string>;
|
|
|
|
// The written definitions of one klammer, split by what they tell us. A
|
|
// definition filed under the general name is the only one whose coverage has
|
|
// to be worked out; a definition written for a target IS its own statement.
|
|
struct Written
|
|
{
|
|
const Klammer::components* m_general { nullptr };
|
|
target_set m_targets {}; // targets with a definition of their own
|
|
bool m_declared { false }; // has a ".k" declaration
|
|
};
|
|
|
|
Written written_definitions(const Klammer& klammer)
|
|
{
|
|
Written w {};
|
|
for (const auto& def : klammer.m_defs) {
|
|
if (def.target == Target_registry::general_name) {
|
|
w.m_general = &def;
|
|
} else if (def.target == Target_registry::declare_name) {
|
|
w.m_declared = true;
|
|
} else if (def.target != Target_registry::optionset_name) {
|
|
w.m_targets.insert(def.target);
|
|
}
|
|
}
|
|
return w;
|
|
}
|
|
|
|
// What a general body is made of. The three findings are ordered by how
|
|
// firmly they settle the question: anything the engine cannot interpret ends
|
|
// the analysis, and only a body of plain klammer calls is derivable.
|
|
struct Body_scan
|
|
{
|
|
bool m_undecidable { false };
|
|
std::string m_reason {};
|
|
std::vector<std::string> m_calls {};
|
|
};
|
|
|
|
Body_scan scan_body(const katom_list& body)
|
|
{
|
|
Body_scan scan {};
|
|
target_set seen {};
|
|
for (const auto& k : body) {
|
|
switch (k.m_type) {
|
|
case katom_t::eval_begin:
|
|
// Which targets a Python function answers for is undecidable, so
|
|
// the analysis stops here and the targets must be declared.
|
|
scan.m_undecidable = true;
|
|
scan.m_reason = "@eval body";
|
|
return scan;
|
|
case katom_t::read_begin:
|
|
scan.m_undecidable = true;
|
|
scan.m_reason = "@read body";
|
|
return scan;
|
|
case katom_t::literal_begin:
|
|
// A ^'...'^ span exists to carry raw target markup past the
|
|
// escaping pass. A general body holding one is target-specific
|
|
// with nothing for the intersection rule to see -- the blind spot
|
|
// a body of plain text would otherwise hide.
|
|
scan.m_undecidable = true;
|
|
scan.m_reason = "^'...'^ literal span";
|
|
return scan;
|
|
case katom_t::apply_begin: {
|
|
// The body read here is the STORED body, which is the body as
|
|
// written with one exception: a general klammer that takes no
|
|
// parameters is a constant, and a constant's body is spliced into
|
|
// later definitions at definition time. So a call to a constant
|
|
// does not appear here -- what appears is whatever the constant
|
|
// expanded to. That is the right thing for coverage (the calls
|
|
// that remain are the ones that will be applied), but it makes
|
|
// the "from" list a statement about the stored body, not about
|
|
// the source text.
|
|
std::string name = trim_char(k.m_text, '@');
|
|
if (seen.insert(name).second) {
|
|
scan.m_calls.push_back("@" + name);
|
|
}
|
|
break;
|
|
}
|
|
default:
|
|
break;
|
|
}
|
|
}
|
|
return scan;
|
|
}
|
|
|
|
target_set intersect(const target_set& a, const target_set& b)
|
|
{
|
|
target_set result {};
|
|
std::set_intersection(a.begin(), a.end(), b.begin(), b.end(),
|
|
std::inserter(result, result.begin()));
|
|
return result;
|
|
}
|
|
|
|
std::vector<std::string> as_vector(const target_set& s)
|
|
{
|
|
return { s.begin(), s.end() };
|
|
}
|
|
|
|
// Shorten a definition's pathname for the report. What identifies a
|
|
// definition to a reader is its tail -- "sks/block/block.k" -- not the
|
|
// absolute path the file happened to be read from, which is the same long
|
|
// prefix on every row. A klammer set outside $KLAMMERTEXT_HOME keeps its
|
|
// path in full rather than being shortened to something ambiguous.
|
|
std::string short_path(const std::string& path)
|
|
{
|
|
auto pos = path.rfind("/sks/");
|
|
if (pos != std::string::npos) {
|
|
return path.substr(pos + 1);
|
|
}
|
|
const char* home = std::getenv("KLAMMERTEXT_HOME");
|
|
if (home != nullptr) {
|
|
std::string prefix = std::string(home) + "/";
|
|
if (path.size() > prefix.size() && path.compare(0, prefix.size(), prefix) == 0) {
|
|
return path.substr(prefix.size());
|
|
}
|
|
}
|
|
return path;
|
|
}
|
|
|
|
} // namespace
|
|
|
|
std::vector<Klammer_coverage> klammer_coverage(const Machine& machine,
|
|
const strings_t& defined_outside)
|
|
{
|
|
target_set all_targets {};
|
|
for (const auto& t : machine.m_targets.user_defined()) {
|
|
all_targets.insert(t);
|
|
}
|
|
|
|
// Pass 1: what each klammer's definitions say, without resolving anything.
|
|
std::map<std::string, Written> written {};
|
|
std::map<std::string, Body_scan> scans {};
|
|
for (const auto& [name, klammer] : machine.m_klammers.m_klammers) {
|
|
written[name] = written_definitions(klammer);
|
|
if (written[name].m_general != nullptr) {
|
|
scans[name] = scan_body(written[name].m_general->body);
|
|
}
|
|
}
|
|
|
|
// Pass 2: the greatest fixpoint. Every general klammer starts optimistic
|
|
// -- all targets -- and the intersection rule is applied until nothing
|
|
// shrinks. Starting optimistic is what makes a cycle terminate: two
|
|
// klammers calling each other simply keep each other's sets, and a set
|
|
// that only loses members cannot iterate forever.
|
|
std::map<std::string, target_set> general {};
|
|
for (const auto& [name, scan] : scans) {
|
|
general[name] = all_targets;
|
|
}
|
|
// A klammer's coverage, for use as an operand of the intersection: what
|
|
// its own definitions cover, plus whatever its general body covers.
|
|
auto coverage_of = [&](const std::string& name) -> target_set {
|
|
auto w = written.find(name);
|
|
if (w == written.end()) return {}; // not defined; contributes nothing
|
|
target_set result = w->second.m_targets;
|
|
auto g = general.find(name);
|
|
if (g != general.end()) {
|
|
result.insert(g->second.begin(), g->second.end());
|
|
}
|
|
return result;
|
|
};
|
|
bool changed = true;
|
|
while (changed) {
|
|
changed = false;
|
|
for (auto& [name, targets] : general) {
|
|
const Body_scan& scan = scans[name];
|
|
if (scan.m_undecidable || scan.m_calls.empty()) continue;
|
|
target_set next = all_targets;
|
|
for (const auto& call : scan.m_calls) {
|
|
std::string called = trim_char(call, '@');
|
|
if (called == name) continue; // self-reference constrains nothing
|
|
next = intersect(next, coverage_of(called));
|
|
}
|
|
if (next != targets) {
|
|
targets = next;
|
|
changed = true;
|
|
}
|
|
}
|
|
}
|
|
|
|
// Pass 3: classify and collect. The filter applies HERE and not earlier:
|
|
// passes 1 and 2 must see every klammer, because a derived coverage is the
|
|
// intersection of what the called klammers cover and those are mostly the
|
|
// klammerset's.
|
|
std::vector<Klammer_coverage> result {};
|
|
for (const auto& [name, klammer] : machine.m_klammers.m_klammers) {
|
|
if (!defined_outside.empty() && !klammer.defined_outside(defined_outside)) {
|
|
continue;
|
|
}
|
|
const Written& w = written.at(name);
|
|
Klammer_coverage kc {};
|
|
kc.m_name = name;
|
|
kc.m_declared = w.m_declared;
|
|
kc.m_written = as_vector(w.m_targets);
|
|
// Written definitions only (m_defs), so this is where the klammer is
|
|
// WRITTEN. m_defloc would also carry the targets a general body was
|
|
// copied to, which name the same file again.
|
|
for (const auto& def : klammer.m_defs) {
|
|
std::string file = short_path(def.loc.m_filename);
|
|
if (!is_in(file, kc.m_files)) {
|
|
kc.m_files.push_back(file);
|
|
}
|
|
}
|
|
for (const auto& [target, loc] : klammer.m_defloc) {
|
|
if (target != Target_registry::declare_name &&
|
|
target != Target_registry::general_name &&
|
|
target != Target_registry::optionset_name) {
|
|
kc.m_effective.push_back(target);
|
|
}
|
|
}
|
|
if (klammer.m_general_declared) {
|
|
// The author has said "every target". That is a statement, not
|
|
// something to be re-derived from the body: the whole point of
|
|
// writing ".*" is to assert what an @eval body cannot be read to
|
|
// mean.
|
|
kc.m_kind = coverage_t::all_declared;
|
|
target_set covered = w.m_targets;
|
|
covered.insert(all_targets.begin(), all_targets.end());
|
|
kc.m_targets = as_vector(covered);
|
|
} else if (w.m_general == nullptr) {
|
|
kc.m_kind = w.m_targets.empty() ? coverage_t::none : coverage_t::declared;
|
|
kc.m_targets = kc.m_written;
|
|
} else {
|
|
const Body_scan& scan = scans.at(name);
|
|
target_set covered = w.m_targets;
|
|
if (scan.m_undecidable) {
|
|
kc.m_kind = coverage_t::undecidable;
|
|
kc.m_reason = scan.m_reason;
|
|
// What it covers is not knowable here; report what is written.
|
|
kc.m_targets = kc.m_written;
|
|
} else if (scan.m_calls.empty()) {
|
|
kc.m_kind = coverage_t::all;
|
|
covered.insert(all_targets.begin(), all_targets.end());
|
|
kc.m_targets = as_vector(covered);
|
|
} else {
|
|
kc.m_kind = coverage_t::derived;
|
|
kc.m_from = scan.m_calls;
|
|
const target_set& g = general.at(name);
|
|
covered.insert(g.begin(), g.end());
|
|
kc.m_targets = as_vector(covered);
|
|
}
|
|
}
|
|
result.push_back(kc);
|
|
}
|
|
return result;
|
|
}
|
|
|
|
namespace {
|
|
|
|
std::string list_of(const std::vector<std::string>& v)
|
|
{
|
|
return v.empty() ? "--" : join(v, " ");
|
|
}
|
|
|
|
void section(std::ostream& os, const std::string& title, size_t count,
|
|
const std::string& explanation)
|
|
{
|
|
os << "\n" << title << " (" << count << ")\n";
|
|
if (!explanation.empty()) {
|
|
os << explanation;
|
|
}
|
|
}
|
|
|
|
} // namespace
|
|
|
|
void report_coverage(const Machine& machine,
|
|
const std::vector<Klammer_coverage>& coverage,
|
|
bool full, std::ostream& os)
|
|
{
|
|
strings_t targets = machine.m_targets.user_defined();
|
|
os << "Klammer coverage\n"
|
|
<< "================\n\n"
|
|
<< coverage.size() << " " << plural("klammer", static_cast<int>(coverage.size()))
|
|
<< ", " << targets.size() << " " << plural("target", static_cast<int>(targets.size()))
|
|
<< ": " << join(targets, " ") << "\n";
|
|
|
|
auto of_kind = [&coverage](coverage_t kind) {
|
|
std::vector<const Klammer_coverage*> result {};
|
|
for (const auto& kc : coverage) {
|
|
if (kc.m_kind == kind) result.push_back(&kc);
|
|
}
|
|
return result;
|
|
};
|
|
size_t width = 0;
|
|
for (const auto& kc : coverage) {
|
|
width = std::max(width, kc.m_name.size());
|
|
}
|
|
// The indent of a continuation line, under the name column.
|
|
std::string continuation(2 + 1 + width, ' ');
|
|
|
|
auto name_of = [&](const Klammer_coverage& kc) {
|
|
std::stringstream ss {};
|
|
ss << " @" << std::left << std::setw(width) << kc.m_name;
|
|
return ss.str();
|
|
};
|
|
|
|
// A section's rows, held until the whole section is built so the
|
|
// source-file column can be aligned. The file goes LAST because it is
|
|
// reference information: what the row says comes first, and the reader
|
|
// looks right only when they want to go and edit it. Only a row naming
|
|
// ONE klammer carries a file -- the "All targets" section lists many
|
|
// names on a line and has nothing to attach one to.
|
|
using Row = std::pair<std::string, std::string>; // text, file
|
|
auto emit = [&os, full](const std::vector<Row>& rows) {
|
|
size_t text_width = 0;
|
|
if (full) {
|
|
for (const auto& [text, file] : rows) {
|
|
if (!file.empty()) text_width = std::max(text_width, text.size());
|
|
}
|
|
}
|
|
for (const auto& [text, file] : rows) {
|
|
if (full && !file.empty()) {
|
|
os << std::left << std::setw(text_width) << text << " " << file << "\n";
|
|
} else {
|
|
os << trim_right(text) << "\n";
|
|
}
|
|
}
|
|
};
|
|
|
|
// ---- what works, first --------------------------------------------
|
|
//
|
|
// The reporting categories come before the problems because a terminal is
|
|
// read from the BOTTOM: an eighty-klammer listing scrolls a three-line
|
|
// warning off the screen entirely, so the actionable part has to be last,
|
|
// where "| tail" finds it.
|
|
//
|
|
// The two halves of the report hide an empty category for different
|
|
// reasons, so they are two functions rather than one with a flag.
|
|
using Row_of = std::function<std::string(const Klammer_coverage&)>;
|
|
// The explanation under a heading is for a reader learning the categories,
|
|
// so it appears only under "all" -- the same argument that shows the empty
|
|
// ones. A default report is headings and rows.
|
|
auto write = [&](const std::string& title,
|
|
const std::vector<const Klammer_coverage*>& group,
|
|
const std::string& explanation, const Row_of& row) {
|
|
section(os, title, group.size(), full ? explanation : "");
|
|
std::vector<Row> rows {};
|
|
for (const auto* kc : group) {
|
|
rows.push_back({row(*kc), join(kc->m_files, ", ")});
|
|
}
|
|
emit(rows);
|
|
};
|
|
|
|
// A REPORTING category describes the shape of the klammer set, so an
|
|
// empty one still says something ("nothing here uses .*") and "all" shows
|
|
// it as a designer's checklist.
|
|
auto reporting = [&](const std::string& title,
|
|
const std::vector<const Klammer_coverage*>& group,
|
|
const std::string& explanation, const Row_of& row) {
|
|
if (group.empty() && !full) return;
|
|
write(title, group, explanation, row);
|
|
};
|
|
|
|
// A PROBLEM category is different: it sits under a banner that says
|
|
// "Needs attention", and an empty one does not. Printing "Covers no
|
|
// target (0)" there states the opposite of the heading above it, so it is
|
|
// hidden whether or not "all" was given -- "all" is for information that
|
|
// is missing, and a category with nothing in it is not missing anything.
|
|
auto problem = [&](const std::string& title,
|
|
const std::vector<const Klammer_coverage*>& group,
|
|
const std::string& explanation, const Row_of& row) {
|
|
if (group.empty()) return;
|
|
write(title, group, explanation, row);
|
|
};
|
|
auto with_targets = [&](const Klammer_coverage& kc) {
|
|
return name_of(kc) + " " + list_of(kc.m_targets);
|
|
};
|
|
|
|
reporting("Defined per target", of_kind(coverage_t::declared), "", with_targets);
|
|
|
|
reporting("All targets, declared", of_kind(coverage_t::all_declared),
|
|
" Written \".*\": the author states that these work for every target,\n"
|
|
" including targets that do not exist yet.\n", with_targets);
|
|
|
|
// Many names on one line, so no file column: there is nothing for a file
|
|
// to attach to.
|
|
auto all = of_kind(coverage_t::all);
|
|
if (!all.empty() || full) {
|
|
section(os, "All targets, derived", all.size(),
|
|
full ? " No target suffix and a body of plain text, so nothing in them is\n"
|
|
" target-specific. This is the writer's macro form -- a repeated\n"
|
|
" phrase, not a klammer set -- and needs no declaration.\n" : "");
|
|
strings_t names {};
|
|
for (const auto* kc : all) {
|
|
names.push_back("@" + kc->m_name);
|
|
}
|
|
if (!names.empty()) {
|
|
os << " " << join(names, " ") << "\n";
|
|
}
|
|
}
|
|
|
|
// A derived klammer covering NOTHING is unusable, so it is reported with
|
|
// the problems rather than here.
|
|
std::vector<const Klammer_coverage*> derived {};
|
|
std::vector<const Klammer_coverage*> uncoverable {};
|
|
for (const auto* kc : of_kind(coverage_t::derived)) {
|
|
(kc->m_targets.empty() ? uncoverable : derived).push_back(kc);
|
|
}
|
|
reporting("Derived from the klammers the body calls", derived,
|
|
" No target suffix and a body of klammer calls, so the coverage is the\n"
|
|
" intersection of what those klammers cover.\n",
|
|
[&](const Klammer_coverage& kc) {
|
|
std::stringstream ss {};
|
|
ss << name_of(kc) << " " << std::left << std::setw(22) << list_of(kc.m_targets)
|
|
<< " from " << join(kc.m_from, " ");
|
|
return ss.str();
|
|
});
|
|
|
|
// ---- then what needs doing ----------------------------------------
|
|
//
|
|
// Four categories, ordered by severity: the first two mean the klammer
|
|
// cannot be used at all, the third that its coverage is a guess, the
|
|
// fourth that it is undocumented. Counted by DISTINCT klammer -- "no .k"
|
|
// is orthogonal to the others, so a klammer can be in two categories and
|
|
// summing the counts would overstate the work.
|
|
auto undecidable = of_kind(coverage_t::undecidable);
|
|
auto none = of_kind(coverage_t::none);
|
|
std::vector<const Klammer_coverage*> undescribed {};
|
|
for (const auto& kc : coverage) {
|
|
if (!kc.m_declared) undescribed.push_back(&kc);
|
|
}
|
|
target_set needing {};
|
|
for (const auto* group : { &none, &uncoverable, &undecidable, &undescribed }) {
|
|
for (const auto* kc : *group) needing.insert(kc->m_name);
|
|
}
|
|
// Nothing to attend to, nothing said -- the same rule as the categories
|
|
// below it. "Needs attention: 0" under "all" was the banner contradicting
|
|
// itself, exactly as an empty category under it would.
|
|
if (!needing.empty()) {
|
|
os << "\nNeeds attention: " << needing.size() << " "
|
|
<< plural("klammer", static_cast<int>(needing.size())) << "\n";
|
|
}
|
|
|
|
problem("Declared but never defined", none,
|
|
" A \".k\" declaration with no definition for any target, so the klammer\n"
|
|
" can never be applied.\n",
|
|
[&](const Klammer_coverage& kc) { return name_of(kc); });
|
|
problem("Covers no target", uncoverable,
|
|
" The klammers this one calls have no target in common, so the\n"
|
|
" intersection is empty and it can never be applied. The klammers\n"
|
|
" named are the ones to look at.\n",
|
|
[&](const Klammer_coverage& kc) {
|
|
return name_of(kc) + " from " + join(kc.m_from, " ");
|
|
});
|
|
problem("Must be declared", undecidable,
|
|
" A general definition whose body the engine cannot interpret, so it is\n"
|
|
" offered to EVERY target whether or not its code answers for that\n"
|
|
" target. Name the targets it does answer for -- @@name.html,tex :: --\n"
|
|
" or, if it works for any target at all, @@name.* ::\n",
|
|
[&](const Klammer_coverage& kc) { return name_of(kc) + " " + kc.m_reason; });
|
|
problem("No \".k\" declaration", undescribed,
|
|
" These render, but nothing describes them: a klammer without a \".k\"\n"
|
|
" has no description, so kdesc can say nothing about what it does and\n"
|
|
" \"kdesc -k <text>\" can only find it by name.\n", with_targets);
|
|
|
|
// The progress number. "Undecided" is the accurate label, and the one
|
|
// that asserts no more than was measured: a klammer that does not cover a
|
|
// target has not been declared unavailable there -- no notation for that
|
|
// exists yet -- it simply has no definition. "Unsupported" or "excluded"
|
|
// would each claim a decision nobody made.
|
|
os << "\nBy target\n";
|
|
for (const auto& target : targets) {
|
|
size_t covered = 0;
|
|
size_t offered = 0;
|
|
for (const auto& kc : coverage) {
|
|
if (is_in(target, kc.m_targets)) covered++;
|
|
if (is_in(target, kc.m_effective)) offered++;
|
|
}
|
|
os << " " << std::left << std::setw(10) << target
|
|
<< std::right << std::setw(4) << covered << " covered"
|
|
<< std::setw(6) << (coverage.size() - covered) << " undecided";
|
|
if (offered > covered) {
|
|
os << " (" << offered - covered
|
|
<< " more currently offered by an underivable general body)";
|
|
}
|
|
os << "\n";
|
|
}
|
|
}
|