What a Windows editor saves, and a standard user's Windows install (from dev 9d6f66456a3d)

- CRLF line endings are read as newlines on every system (the reader used
  to delete the pair, collapsing a Notepad++ file into one line on Linux
  and macOS); a leading UTF-8 byte-order mark is ignored; a UTF-16 or
  Windows-1252 source is refused with the encoding, the place and the
  remedy named. .gitattributes pins LF in every clone.
- Windows source install: the repository path must hold no space (GNU
  Make), accented letters are fine (every linker input is relative now);
  python.org's Python is selected ahead of an MSYS2 one that poppler pulls
  in; TeX Live goes to its Windows default C:\texlive\2026.
- The Python binding needs an ASCII installation path on Windows, and its
  suite says so there.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
ackandClaude Fable 5.1 committed 2026-09-26 16:17:59 +02:00
1 parent fc2e848e91
commit 715d939727
15 files changed
+236 -41

No files matched your search

+9
View File
@@ -0,0 +1,9 @@
# Every text file is LF in the working tree on every system, whatever
# core.autocrlf says: .kt, .k and the shell suites must be LF (2026-09-26).
* text=auto eol=lf
# Shipped verbatim, as the foundries wrote them (CRLF): never normalized.
fnt/**/*.txt -text
sks/document/fonts/**/*.txt -text
# Not shipped; kept byte-stable.
dbg/** -text
sks.alpha/** -text
+1 -1
View File
@@ -53,7 +53,7 @@ are regenerated on each release — patches cannot be merged directly.
Report problems (or send patches) to the author; accepted changes are Report problems (or send patches) to the author; accepted changes are
applied to the development tree and appear in a following snapshot. applied to the development tree and appear in a following snapshot.
This snapshot was assembled from development commit `d5f9c20b94b3`. This snapshot was assembled from development commit `9d6f66456a3d`.
## License ## License
+3
View File
@@ -26,6 +26,9 @@ current file, and this package uses exactly those two things.
version qualifies). version qualifies).
- For the Run commands: `python` on the `Path`, which the Klammertext - For the Run commands: `python` on the `Path`, which the Klammertext
installation already needs. installation already needs.
- Files saved as UTF-8 (Notepad++'s default for new files; **Encoding →
Convert to UTF-8** for an older one). Line endings do not matter, and
a file in another encoding is refused by `ktext` with the remedy named.
## Install the highlighting ## Install the highlighting
+39 -24
View File
@@ -97,11 +97,17 @@ git clone https://git.andykopra.com/ack/klammertext.git
cd klammertext cd klammertext
``` ```
Git for Windows must not translate line endings: Klammertext's `.kt` The repository's path must contain no space: GNU Make cannot build under
files, `.k` files and shell scripts are LF files. The clone above respects one. If your user name has a space in it (`C:\Users\Anna Maria`), clone
that as long as `core.autocrlf` is not `true`; check with `git config somewhere else, for example into `C:\klammertext`, and write that path
--get core.autocrlf` and, if it prints `true`, set it to `false` and clone wherever `C:\Users\<you>\klammertext` appears below. Accented letters in
again. the path are fine for the commands; only the Python binding (`kpy\`) needs
a path of plain ASCII letters.
Klammertext's `.kt` files, `.k` files and shell scripts are LF files, and
the repository says so itself (`.gitattributes`), so the clone is correct
whatever your Git's `core.autocrlf` setting. A `.kt` you write with
Windows line endings is fine too: ktext reads them as newlines.
## Environment variables ## Environment variables
@@ -110,14 +116,14 @@ Klammertext's runtime environment is one self-configuring file, the same
one Linux and macOS use. Source it from the UCRT64 shell's profile: one Linux and macOS use. Source it from the UCRT64 shell's profile:
```bash ```bash
echo 'source /c/Users/<you>/klammertext/env/runtime.env' >> ~/.bashrc echo 'source "/c/Users/<you>/klammertext/env/runtime.env"' >> ~/.bashrc
source ~/.bashrc source ~/.bashrc
``` ```
On Windows it sets `KLAMMERTEXT_HOME` and `TMPDIR` in the form Windows On Windows it sets `KLAMMERTEXT_HOME` and `TMPDIR` in the form Windows
programs understand (`C:/Users/<you>/klammertext`), adds `bin/` to `PATH`, programs understand (`C:/Users/<you>/klammertext`), adds `bin/` to `PATH`,
finds a TeX Live installed under `C:\Users\<you>\external\texlive\<year>` finds a TeX Live installed under `C:\texlive\<year>` (see below), and
(see below), and sets `PYTHON=python`. Nothing is edited by hand. sets `PYTHON=python`. Nothing is edited by hand.
## Build ## Build
@@ -171,26 +177,28 @@ ktext hello.kt -t html # writes hello\index.html
## TeX Live (for PDF output) ## TeX Live (for PDF output)
The Standard Klammer Set renders PDF with **XeLaTeX**. Install TeX Live's The Standard Klammer Set renders PDF with **XeLaTeX**. Install TeX Live's
`scheme-small` plus the packages the SKS needs into `scheme-small` plus the packages the SKS needs into `C:\texlive\2026`,
`C:\Users\<you>\external\texlive\2026`, where `runtime.env` finds it. This TeX Live's own default location on Windows, where `runtime.env` finds it.
is a download of about 600 MB and takes twenty to thirty minutes. (It also looks under `C:\Users\<you>\external\texlive\<year>`, the place
the Linux and macOS pages use, but TeX Live does not work under a path
with a space or an accented letter, and a user name may have either.)
This is a download of about 600 MB and takes twenty to thirty minutes.
1. Download `install-tl.zip` from a CTAN mirror, for example 1. Download `install-tl.zip` from a CTAN mirror, for example
`https://ctan.math.illinois.edu/systems/texlive/tlnet/install-tl.zip`, `https://ctan.math.illinois.edu/systems/texlive/tlnet/install-tl.zip`,
and unzip it. It unpacks into a folder named `install-tl-<date>`. and unzip it. It unpacks into a folder named `install-tl-<date>`.
2. In that folder, create a file `texlive.profile` with this content, 2. In that folder, create a file `texlive.profile` with this content:
replacing `<you>` with your Windows user name:
``` ```
selected_scheme scheme-small selected_scheme scheme-small
TEXDIR C:/Users/<you>/external/texlive/2026 TEXDIR C:/texlive/2026
TEXMFLOCAL C:/Users/<you>/external/texlive/2026/texmf-local TEXMFLOCAL C:/texlive/2026/texmf-local
TEXMFSYSCONFIG C:/Users/<you>/external/texlive/2026/texmf-config TEXMFSYSCONFIG C:/texlive/2026/texmf-config
TEXMFSYSVAR C:/Users/<you>/external/texlive/2026/texmf-var TEXMFSYSVAR C:/texlive/2026/texmf-var
TEXMFHOME C:/Users/<you>/texmf TEXMFHOME C:/texlive/2026/texmf-home
TEXMFVAR C:/Users/<you>/.texlive2026/texmf-var TEXMFVAR C:/texlive/2026/texmf-uvar
TEXMFCONFIG C:/Users/<you>/.texlive2026/texmf-config TEXMFCONFIG C:/texlive/2026/texmf-uconfig
tlpdbopt_install_docfiles 0 tlpdbopt_install_docfiles 0
tlpdbopt_install_srcfiles 0 tlpdbopt_install_srcfiles 0
tlpdbopt_autobackup 0 tlpdbopt_autobackup 0
@@ -214,7 +222,7 @@ is a download of about 600 MB and takes twenty to thirty minutes.
installs on Linux and macOS — and rebuild the formats: installs on Linux and macOS — and rebuild the formats:
```powershell ```powershell
$tl = "$env:USERPROFILE\external\texlive\2026\bin\windows" $tl = "C:\texlive\2026\bin\windows"
$env:Path = "$tl;$env:Path" $env:Path = "$tl;$env:Path"
& "$tl\tlmgr.bat" option repository https://ctan.math.illinois.edu/systems/texlive/tlnet & "$tl\tlmgr.bat" option repository https://ctan.math.illinois.edu/systems/texlive/tlnet
& "$tl\tlmgr.bat" install adjustbox collectbox collection-fontsrecommended enumitem fontaxes footmisc inconsolata layouts mdframed multirow needspace opensans pict2e textpos titlesec upquote zref & "$tl\tlmgr.bat" install adjustbox collectbox collection-fontsrecommended enumitem fontaxes footmisc inconsolata layouts mdframed multirow needspace opensans pict2e textpos titlesec upquote zref
@@ -244,7 +252,7 @@ from PowerShell):
```powershell ```powershell
[Environment]::SetEnvironmentVariable("KLAMMERTEXT_HOME", "C:\Users\<you>\klammertext", "User") [Environment]::SetEnvironmentVariable("KLAMMERTEXT_HOME", "C:\Users\<you>\klammertext", "User")
$p = [Environment]::GetEnvironmentVariable("Path", "User") $p = [Environment]::GetEnvironmentVariable("Path", "User")
[Environment]::SetEnvironmentVariable("Path", "$p;C:\Users\<you>\klammertext\bin;C:\Users\<you>\external\texlive\2026\bin\windows", "User") [Environment]::SetEnvironmentVariable("Path", "$p;C:\Users\<you>\klammertext\bin;C:\texlive\2026\bin\windows", "User")
``` ```
(Not `setx Path "$env:Path;…"`: `$env:Path` is the machine and user (Not `setx Path "$env:Path;…"`: `$env:Path` is the machine and user
@@ -374,8 +382,15 @@ incomplete or `KLAMMERTEXT_HOME` points elsewhere.
**"The 'xelatex' command required to make PDF files is not installed".** **"The 'xelatex' command required to make PDF files is not installed".**
TeX Live's `bin\windows` directory is not on the `Path` of the shell you TeX Live's `bin\windows` directory is not on the `Path` of the shell you
are using. In the UCRT64 shell `runtime.env` adds it when the tree is under are using. In the UCRT64 shell `runtime.env` adds it when the tree is under
`C:\Users\<you>\external\texlive\<year>`; in PowerShell add it as shown `C:\texlive\<year>` or `C:\Users\<you>\external\texlive\<year>`; in
above. PowerShell add it as shown above.
**`make` stops at once with `"python" is C:/msys64/ucrt64/python, not
python.org's interpreter`.** MSYS2's own Python was installed as a
dependency (poppler brings it), and in the UCRT64 shell it precedes
python.org's on the `Path`. `runtime.env` puts python.org's first: source
it (a new UCRT64 shell does, once the line is in `~/.bashrc`) and run
`make` again.
**`git` or `python` are "command not found" in the UCRT64 shell.** **`git` or `python` are "command not found" in the UCRT64 shell.**
`MSYS2_PATH_TYPE` is not set to `inherit`, or the shell was opened before `MSYS2_PATH_TYPE` is not set to `inherit`, or the shell was opened before
+13
View File
@@ -34,6 +34,19 @@ endif
ifneq (,$(filter MINGW% MSYS%,$(UNAME_S))) ifneq (,$(filter MINGW% MSYS%,$(UNAME_S)))
PYTHON := python PYTHON := python
PYTHON_PREFIX := $(shell cygpath -m "$$($(PYTHON) -c 'import sys; print(sys.base_prefix)')") PYTHON_PREFIX := $(shell cygpath -m "$$($(PYTHON) -c 'import sys; print(sys.base_prefix)')")
# The interpreter "python" resolves to must be python.org's, which has an
# import library under libs/. MSYS2's own python (installed as a dependency
# of poppler, through glib2) sits in /ucrt64/bin ahead of it and has none:
# a build against it compiles nothing (no Include/Python.h) or embeds an
# interpreter without the user's pip packages. env/runtime.env puts
# python.org's directory first; refuse rather than build the wrong thing.
# (Asked of the interpreter, not $(wildcard): the prefix "C:/Program Files"
# has a space, on which wildcard splits.)
ifeq (,$(NOPYTHON))
ifneq (yes,$(shell $(PYTHON) -c 'import os, sys; print("yes" if os.path.isdir(os.path.join(sys.base_prefix, "libs")) else "no")'))
$(error "python" is $(PYTHON_PREFIX)/python, not python.org's interpreter (no libs/ there); source env/runtime.env, which puts python.org's Python first on PATH)
endif
endif
PYTHON_INC := -I"$(PYTHON_PREFIX)/Include" PYTHON_INC := -I"$(PYTHON_PREFIX)/Include"
PYTHON_LIB := -L"$(PYTHON_PREFIX)/libs" -l$(shell $(PYTHON) -c 'import sys; print("python%d%d" % sys.version_info[:2])') PYTHON_LIB := -L"$(PYTHON_PREFIX)/libs" -l$(shell $(PYTHON) -c 'import sys; print("python%d%d" % sys.version_info[:2])')
else else
+27 -6
View File
@@ -31,6 +31,21 @@ case "$_kt_uname" in
export KLAMMERTEXT_HOME="$(cygpath -m "$KLAMMERTEXT_HOME")" export KLAMMERTEXT_HOME="$(cygpath -m "$KLAMMERTEXT_HOME")"
export TMPDIR="$(cygpath -m "${TMPDIR:-/tmp}")" export TMPDIR="$(cygpath -m "${TMPDIR:-/tmp}")"
export PYTHON="${PYTHON:-python}" # python.org installs no python3 export PYTHON="${PYTHON:-python}" # python.org installs no python3
# "python" must be python.org's interpreter: the build embeds it and
# pip installs OpenImageIO into it. But the UCRT64 shell puts
# /ucrt64/bin first on PATH, and an MSYS2 package installed as a
# dependency (poppler -> glib2 -> python) puts ITS python there, which
# then shadows python.org's for make, the suites and the tooling.
# python.org's launcher knows only python.org installs: put the
# directory of the interpreter it selects first. (Its output ends
# in CR LF, hence the tr.)
if command -v py >/dev/null 2>&1; then
_kt_py="$(py -3 -c 'import sys; print(sys.executable)' 2>/dev/null | tr -d '\r')"
if [ -n "$_kt_py" ]; then
export PATH="$(dirname "$(cygpath -u "$_kt_py")"):$PATH"
fi
unset _kt_py
fi
;; ;;
esac esac
@@ -50,13 +65,19 @@ fi
# just returns nothing. The -path pattern keeps the 4-digit-year restriction. # just returns nothing. The -path pattern keeps the 4-digit-year restriction.
# Windows: a user installs under the Windows home, %USERPROFILE%\external # Windows: a user installs under the Windows home, %USERPROFILE%\external
# (MSYS2's $HOME is C:\msys64\home\<user>, which no Windows guide would name). # (MSYS2's $HOME is C:\msys64\home\<user>, which no Windows guide would name).
_kt_tlroot="$HOME/external/texlive" # Windows also: TeX Live's own default, C:\texlive\<year> -- the place for a
case "$_kt_uname" in MINGW*|MSYS*) _kt_tlroot="$(cygpath -u "$USERPROFILE")/external/texlive";; esac # user whose profile path holds a space or an accented letter, which TeX
_kt_tl="$(find "$_kt_tlroot" -maxdepth 3 -type d \ # Live's installer and kpathsea do not take (2026-09-26). The newest YEAR
-path "*/[0-9][0-9][0-9][0-9]/bin/$_kt_arch" 2>/dev/null \ # wins across both roots (the year is the sort key, not the path).
| sort -V | tail -1)" _kt_tlroot="$HOME/external/texlive"; _kt_tlroot2=""
case "$_kt_uname" in MINGW*|MSYS*) _kt_tlroot="$(cygpath -u "$USERPROFILE")/external/texlive"; _kt_tlroot2="/c/texlive";; esac
_kt_tl="$( { find "$_kt_tlroot" -maxdepth 3 -type d \
-path "*/[0-9][0-9][0-9][0-9]/bin/$_kt_arch" 2>/dev/null
[ -n "$_kt_tlroot2" ] && find "$_kt_tlroot2" -maxdepth 3 -type d \
-path "*/[0-9][0-9][0-9][0-9]/bin/$_kt_arch" 2>/dev/null; } \
| awk -F/ '{ print $(NF-2) " " $0 }' | sort -V | tail -1 | cut -d' ' -f2-)"
[ -n "$_kt_tl" ] && export KLAMMERTEXT_TEXLIVE_BIN="$_kt_tl" [ -n "$_kt_tl" ] && export KLAMMERTEXT_TEXLIVE_BIN="$_kt_tl"
unset _kt_tl _kt_arch _kt_tlroot unset _kt_tl _kt_arch _kt_tlroot _kt_tlroot2
# --- PATH (TeX Live appended only if one was found) --------------------------- # --- PATH (TeX Live appended only if one was found) ---------------------------
export PATH="$_kt_home_posix/bin:$_kt_home_posix/tst${KLAMMERTEXT_TEXLIVE_BIN:+:$KLAMMERTEXT_TEXLIVE_BIN}:$PATH" export PATH="$_kt_home_posix/bin:$_kt_home_posix/tst${KLAMMERTEXT_TEXLIVE_BIN:+:$KLAMMERTEXT_TEXLIVE_BIN}:$PATH"
+3 -1
View File
@@ -18,7 +18,9 @@ K := $(KLAMMERTEXT_HOME)
include $(K)/env/makefile.env include $(K)/env/makefile.env
LIBDIR := $(K)/lib # Relative, never $(K)/...: the mingw linker cannot open a path holding a
# non-ASCII letter (C:/Übungen/...), while the compiler can (2026-09-26).
LIBDIR := ../lib
LIBRARY := $(LIBDIR)/$(KLAMMERTEXT_LIBFILE) LIBRARY := $(LIBDIR)/$(KLAMMERTEXT_LIBFILE)
EXT_SUFFIX := $(shell $(PYTHON) -c 'import sysconfig; print(sysconfig.get_config_var("EXT_SUFFIX"))') EXT_SUFFIX := $(shell $(PYTHON) -c 'import sysconfig; print(sysconfig.get_config_var("EXT_SUFFIX"))')
+75 -1
View File
@@ -325,7 +325,20 @@ std::string string_from_file(const std::string& pathname, bool strip_surrounding
} else { } else {
result = buffer.str(); result = buffer.str();
result = string_replace(result, "\r\n", ""); // Urgh. // A leading UTF-8 byte-order mark is not text: Windows tools
// (PowerShell's Set-Content, Notepad's "UTF-8 with BOM") put
// one at the start of a file, and without this a .kt beginning
// "@document" was reported as "Word not parsed" at 1:1, the
// offending word invisible (decided 2026-09-26).
if (result.compare(0, 3, "\xEF\xBB\xBF") == 0)
result.erase(0, 3);
// Windows line endings become newlines. (This line DELETED the
// pair until 2026-09-26, so on Linux and macOS a file written by
// Notepad++ or a DOS-mode Emacs collapsed into one line and its
// spans no longer matched; on Windows the C runtime's text mode
// had already translated the pair, which hid the bug there.)
result = string_replace(result, "\r\n", "\n");
if (std::regex_match(pathname, klammertext_filename_re)) { if (std::regex_match(pathname, klammertext_filename_re)) {
@@ -575,6 +588,67 @@ void write_to_cache(const std::string& cache_dir, const std::string& basename, c
string_to_file(output_filename, text); string_to_file(output_filename, text);
} }
// The offset of the first byte that is not part of well-formed UTF-8, or
// npos. Overlong forms, surrogates and code points above 10FFFF are rejected
// like a decoder would (RFC 3629, table 3-7 of the Unicode standard).
static std::size_t first_invalid_utf8(const std::string& s)
{
const auto n = s.size();
for (std::size_t i = 0; i < n;) {
const unsigned char b = static_cast<unsigned char>(s[i]);
std::size_t len; unsigned char lo = 0x80, hi = 0xBF;
if (b < 0x80) { i += 1; continue; }
else if (b >= 0xC2 && b <= 0xDF) len = 2;
else if (b == 0xE0) { len = 3; lo = 0xA0; }
else if (b >= 0xE1 && b <= 0xEC) len = 3;
else if (b == 0xED) { len = 3; hi = 0x9F; }
else if (b >= 0xEE && b <= 0xEF) len = 3;
else if (b == 0xF0) { len = 4; lo = 0x90; }
else if (b >= 0xF1 && b <= 0xF3) len = 4;
else if (b == 0xF4) { len = 4; hi = 0x8F; }
else return i;
if (i + len > n) return i;
for (std::size_t k = 1; k < len; ++k) {
const unsigned char c = static_cast<unsigned char>(s[i + k]);
const unsigned char l = (k == 1) ? lo : 0x80, h = (k == 1) ? hi : 0xBF;
if (c < l || c > h) return i;
}
i += len;
}
return std::string::npos;
}
std::string klammertext_from_file(const std::string& pathname)
{
std::string text = string_from_file(pathname);
// A UTF-16 file (Notepad++ offers "UCS-2 LE BOM") begins with FF FE or
// FE FF; read as bytes it is NUL-interleaved and every word is "not
// parsed". Name the encoding and the remedy instead.
if (text.size() >= 2 && ((static_cast<unsigned char>(text[0]) == 0xFF && static_cast<unsigned char>(text[1]) == 0xFE)
|| (static_cast<unsigned char>(text[0]) == 0xFE && static_cast<unsigned char>(text[1]) == 0xFF))) {
throw File_error("File \"" + pathname + "\" is UTF-16 (it begins with a byte-order mark), and Klammertext reads UTF-8. "
"Save it as UTF-8 in your editor (Notepad++: Encoding, Convert to UTF-8) and run the command again.",
Locator(fs::path(pathname), 1, 0)); // the Locator's character counts from 0
}
// A file saved as ANSI (Windows-1252, the default of some editors) is
// not UTF-8 from its first accented letter on; the bytes otherwise
// reach the embedded Python and fail there with a message about
// something else entirely.
if (const std::size_t bad = first_invalid_utf8(text); bad != std::string::npos) {
int line = 1, chr = 1;
for (std::size_t i = 0; i < bad; ++i) {
if (text[i] == '\n') { ++line; chr = 1; }
else if ((static_cast<unsigned char>(text[i]) & 0xC0) != 0x80) ++chr; // count characters, not bytes
}
char hex[8]; std::snprintf(hex, sizeof hex, "%02X", static_cast<unsigned char>(text[bad]));
throw File_error("File \"" + pathname + "\" is not UTF-8 at line " + std::to_string(line) + ", character "
+ std::to_string(chr) + " (byte " + hex + "), and Klammertext reads UTF-8. "
"Save it as UTF-8 in your editor (Notepad++: Encoding, Convert to UTF-8) and run the command again.",
Locator(fs::path(pathname), line, chr - 1));
}
return text;
}
std::string read_from_cache(const std::string& cache_dir, const std::string& basename) std::string read_from_cache(const std::string& cache_dir, const std::string& basename)
{ {
std::string input_filename = cache_dir + "/" + basename; std::string input_filename = cache_dir + "/" + basename;
+6
View File
@@ -70,6 +70,12 @@ std::vector<std::string> group_filename_tokens(
std::vector<std::string> resolve_filename_list( std::vector<std::string> resolve_filename_list(
const std::string& text, const std::string& base_dir=""); const std::string& text, const std::string& base_dir="");
std::string string_from_file(const std::string& pathname, bool strip_surrounding_whitespace=false); std::string string_from_file(const std::string& pathname, bool strip_surrounding_whitespace=false);
// A Klammertext source (a .kt, a .k, a @read file): string_from_file() plus
// the check that it IS text Klammertext can read -- a File_error names the
// first byte that is not UTF-8 (line and character), or a UTF-16 file by its
// byte-order mark, with what to do in an editor. string_from_file() itself
// stays permissive: it also reads XeTeX logs and a shell command's stderr.
std::string klammertext_from_file(const std::string& pathname);
void string_to_file(const std::string& pathname, std::string contents); void string_to_file(const std::string& pathname, std::string contents);
std::vector<std::string> get_files_in_directory(const std::string& dir); std::vector<std::string> get_files_in_directory(const std::string& dir);
std::string find_file(const std::string& basename, std::vector<std::string> search_path, std::string find_file(const std::string& basename, std::vector<std::string> search_path,
+1 -1
View File
@@ -221,7 +221,7 @@ static std::string klammerset_description(const fs::path& declaration)
{ {
std::string text {}; std::string text {};
try { try {
text = string_from_file(declaration.string()); text = klammertext_from_file(declaration.string());
} catch (...) { } catch (...) {
return ""; return "";
} }
+3 -3
View File
@@ -491,7 +491,7 @@ void Machine::read(const fs::path& pathname)
// :cpp libraries live next to the file that uses them). // :cpp libraries live next to the file that uses them).
m_state.add_search_dir(fs::absolute(pathname).parent_path().string()); m_state.add_search_dir(fs::absolute(pathname).parent_path().string());
m_state.open_frame("Machine state: " + pathname.string()); m_state.open_frame("Machine state: " + pathname.string());
input(string_from_file(pathname.string()), pathname.generic_string()); input(klammertext_from_file(pathname.string()), pathname.generic_string());
m_sources.push_back(pathname); m_sources.push_back(pathname);
} }
@@ -552,7 +552,7 @@ void Machine::expand_read_katoms(
input_filename = fs::canonical(input_filename); input_filename = fs::canonical(input_filename);
m_state.add_search_dir(input_filename.parent_path().string()); m_state.add_search_dir(input_filename.parent_path().string());
std::string text = trim_right(string_from_file(input_filename.string())); std::string text = trim_right(klammertext_from_file(input_filename.string()));
katom_list ks = katomize_lines(line_split(text), input_filename.generic_string()); katom_list ks = katomize_lines(line_split(text), input_filename.generic_string());
// ks = // ks =
process_katoms( process_katoms(
@@ -684,7 +684,7 @@ void Machine::load_klammerset_files(const Klammerset& klammerset, katom_iter ins
pathname = fs::canonical(pathname); pathname = fs::canonical(pathname);
loaded_files.push_back(pathname.generic_string()); loaded_files.push_back(pathname.generic_string());
m_state.add_search_dir(pathname.parent_path().string()); m_state.add_search_dir(pathname.parent_path().string());
std::string text = trim_right(string_from_file(pathname.string())); std::string text = trim_right(klammertext_from_file(pathname.string()));
katom_list ks = katomize_lines(line_split(text), pathname.generic_string()); katom_list ks = katomize_lines(line_split(text), pathname.generic_string());
process_katoms(ks, pathname.generic_string()); process_katoms(ks, pathname.generic_string());
loaded.insert(loaded.end(), ks.begin(), ks.end()); loaded.insert(loaded.end(), ks.begin(), ks.end());
+3 -1
View File
@@ -19,7 +19,9 @@ include $(K)/env/makefile.env
# CPPFLAGS already carries -I$(KLAMMERTEXT_HOME)/mac, which is every header # CPPFLAGS already carries -I$(KLAMMERTEXT_HOME)/mac, which is every header
# reflect.cpp includes (machine, character, error, show). No local -I. # reflect.cpp includes (machine, character, error, show). No local -I.
LIBDIR := $(K)/lib # Relative, never $(K)/...: the mingw linker cannot open a path holding a
# non-ASCII letter (C:/Übungen/...), while the compiler can (2026-09-26).
LIBDIR := ../lib
LIBRARY := $(LIBDIR)/$(KLAMMERTEXT_LIBFILE) LIBRARY := $(LIBDIR)/$(KLAMMERTEXT_LIBFILE)
REFLECT := reflect$(DSO) REFLECT := reflect$(DSO)
+5 -3
View File
@@ -11,7 +11,9 @@ include $(K)/env/makefile.env
LOCAL_CPPFLAGS := -I$(KM) -I$(KS)/kutil -I$(KS)/target LOCAL_CPPFLAGS := -I$(KM) -I$(KS)/kutil -I$(KS)/target
# Shared library location # Shared library location
LIBDIR := $(K)/lib # Relative, never $(K)/...: the mingw linker cannot open a path holding a
# non-ASCII letter (C:/Übungen/...), while the compiler can (2026-09-26).
LIBDIR := ../../lib
LIBRARY := $(LIBDIR)/$(KLAMMERTEXT_LIBFILE) LIBRARY := $(LIBDIR)/$(KLAMMERTEXT_LIBFILE)
# The product: document.so, on Windows document.dll (makefile.env's DSO) # The product: document.so, on Windows document.dll (makefile.env's DSO)
@@ -25,8 +27,8 @@ LOCAL_OBJECTS := document_class.o document_html.o document_latex.o heading.o ref
LOCAL_DEPFILES := document_class.d document_html.d document_latex.d heading.d reference.d document.d LOCAL_DEPFILES := document_class.d document_html.d document_latex.d heading.d reference.d document.d
# External object files from sks/ (mac/*.o now in the shared library) # External object files from sks/ (mac/*.o now in the shared library)
SKS_OBJECTS := $(KS)/kutil/kutil.o $(KS)/kutil/klammer_base.o \ SKS_OBJECTS := ../kutil/kutil.o ../kutil/klammer_base.o \
$(KS)/target/html_util.o $(KS)/target/latex_util.o ../target/html_util.o ../target/latex_util.o
ALL_OBJECTS := $(LOCAL_OBJECTS) $(SKS_OBJECTS) ALL_OBJECTS := $(LOCAL_OBJECTS) $(SKS_OBJECTS)
+35
View File
@@ -36,6 +36,10 @@ mkdir "$DIR/my dir"
printf 'ALPHA\n' > "$DIR/my file.kt" printf 'ALPHA\n' > "$DIR/my file.kt"
printf 'BETA\n' > "$DIR/b.kt" printf 'BETA\n' > "$DIR/b.kt"
printf 'GAMMA\n' > "$DIR/my dir/c.kt" printf 'GAMMA\n' > "$DIR/my dir/c.kt"
printf '\357\273\277DELTA\n' > "$DIR/bom.kt" # a UTF-8 byte-order mark, then the text
printf 'EPSILON\r\nZETA\r\n' > "$DIR/crlf.kt" # Windows line endings
printf '\377\376h\000i\000\n\000' > "$DIR/utf16.kt" # UTF-16LE with its byte-order mark
printf 'ETA\nTHETA M\374ller\n' > "$DIR/cp1252.kt" # Windows-1252: an umlaut that is not UTF-8
cd "$DIR" || exit 1 cd "$DIR" || exit 1
# check TEST_NAME EXPECTED_STDOUT EXPECTED_STDERR_SUBSTRING KTEXT_ARGS... # check TEST_NAME EXPECTED_STDOUT EXPECTED_STDERR_SUBSTRING KTEXT_ARGS...
@@ -151,6 +155,37 @@ check_error "8. unrescuable name is reported as written" \
"no such.kt" \ "no such.kt" \
"no such.kt" -d --klammersets none "no such.kt" -d --klammersets none
# A leading UTF-8 byte-order mark is ignored (2026-09-26): Windows tools write
# one, and it was reported as "Word not parsed" at 1:1 with an invisible word.
check "9. a leading UTF-8 BOM in the input file is ignored" \
"DELTA" "" \
bom.kt -d --klammersets none
check "10. a leading UTF-8 BOM in a @read file is ignored" \
"DELTA" "" \
-s '@read bom.kt @' -d --klammersets none
# Windows line endings are newlines (2026-09-26): the reader DELETED the pair,
# so on Linux and macOS a CRLF file collapsed into one line.
check "11. CRLF line endings are read as newlines" \
$'EPSILON\nZETA' "" \
crlf.kt -d --klammersets none
# What a Windows editor can also save (2026-09-26): a UTF-16 file and an
# ANSI (Windows-1252) file are named as such, with the remedy, instead of
# "Word not parsed" or a failure inside the embedded Python.
check_error "12. a UTF-16 file is named as such" \
"UTF-16" \
utf16.kt -d --klammersets none
check_error "13. a Windows-1252 file is located at its first non-UTF-8 byte" \
"line 2, character 8" \
cp1252.kt -d --klammersets none
check_error "14. ... through @read as well" \
"not UTF-8" \
-s '@read cp1252.kt @' -d --klammersets none
echo echo
echo "=======================" echo "======================="
echo "Results: ${green}$PASS passed${reset}, ${red}$FAIL failed${reset}" echo "Results: ${green}$PASS passed${reset}, ${red}$FAIL failed${reset}"
+13
View File
@@ -49,6 +49,19 @@ trap 'rm -rf "$OUTF" "$ERRF" "$WORK"' EXIT
if command -v cygpath >/dev/null 2>&1; then sep=';'; else sep=':'; fi if command -v cygpath >/dev/null 2>&1; then sep=';'; else sep=':'; fi
export PYTHONPATH="$K/kpy${PYTHONPATH:+$sep$PYTHONPATH}" export PYTHONPATH="$K/kpy${PYTHONPATH:+$sep$PYTHONPATH}"
# On Windows the binding cannot load under a KLAMMERTEXT_HOME that holds a
# non-ASCII character: inside python.exe, a process without Klammertext's
# UTF-8 manifest, the narrow getenv() is the ANSI code page, and the engine
# takes every path as UTF-8. A documented limitation, not a defect to fix
# (TODO 70 (f), Andy 2026-09-26): the binding needs an ASCII installation
# path there. Say so and skip, rather than fail every case with a decode
# error that names no cause.
if command -v cygpath >/dev/null 2>&1 && LC_ALL=C grep -q '[^ -~]' <<<"$K"; then
echo "SKIP kpy_test: KLAMMERTEXT_HOME ($K) holds a non-ASCII character, and the Python binding needs an ASCII installation path on Windows (TODO 70 (f))"
echo "Passed: 0, Failed: 0"
exit 0
fi
# A DEBUG=1 build instruments the module with AddressSanitizer, and an # A DEBUG=1 build instruments the module with AddressSanitizer, and an
# instrumented extension imported into an uninstrumented python needs the # instrumented extension imported into an uninstrumented python needs the
# ASan runtime loaded FIRST -- the import otherwise fails with "ASan runtime # ASan runtime loaded FIRST -- the import otherwise fails with "ASan runtime