# Chatter — Stylus & Pen-Rendering Research

A record of what we learned trying to reproduce the reMarkable stock pen
behavior in Chatter, so we can resume this later. (Status 2026-06-26: the solid
ink pipeline is done and shipping; calligraphy is "close enough for now" per the
user and is **not essential** for the conversational-assistance use case.)

## 1. The render pipeline (solved)

The reMarkable epaper Qt backend is a **software scene-graph renderer**:
- Custom `QSGGeometryNode`s are silently dropped — they don't render.
- `QQuickPaintedItem` textures render but always via the e-ink **dashed
  two-pass** refresh, with latency. Screen mode (Pen/Mono) and antialiasing do
  not change this.
- **Solution:** bypass Qt for ink. Capture the framebuffer `QImage` by
  interposing `EPFramebuffer::setBuffers` (cross-DSO, interposable; the
  executable is linked `-Wl,--export-dynamic` so its symbol wins). Draw strokes
  straight into that buffer (`InkEngine`) and refresh each segment with an
  explicit `EPFramebuffer::swapBuffers(rect, …)` — which renders **solid,
  single-pass**. Qt Quick is used only for the static UI.
- Buffer A is `960×1696 RGB32` (logical screen 954×1696, padded to 960);
  `bytesPerLine 3840`. There is **no PNG image-format plugin** on the device
  (only gif/ico/jpeg/svg) — save snapshots as **BMP**.

## 2. The Calligraphy pen model

reMarkable's Calligraphy is **not a fixed geometric/flat nib**. Confirmed by
experiment + reMarkable docs: it's a **dynamic** model combining:
- **Stroke direction** — downstrokes thick, upstrokes thin. (User's stock-pen
  circles: thick on the descending side — right for clockwise, left for CCW. A
  4-direction asterisk showed *no* variation, because quick uniform straight
  strokes don't trigger it.)
- **Pressure** — heavy pressure ≈ doubles the width.
- **Speed** — faster = thinner.
- **Tilt/orientation** — simulates an angled nib.

The exact angle/algorithm is compiled into xochitl's proprietary brush engine
(assets `LS_Calligraphy_*` / `P_Calligraphy_*`, `rm-brushgfx`) — **not** in a
readable config, so exact duplication would need deep reverse-engineering.

## 3. Width calibration

- `LastPenSize` (from `xochitl.conf` `LastWritingTool`) is only a **category**:
  1/2/3 = thin/thicker/thickest. Not a pixel width.
- Measured on the physical tablet: **thicker(2) ≈ 1 mm, thickest(3) ≈ 2 mm,
  thinnest ≈ 2 px**. Pressure can **almost double** the width.
- Display is **264 PPI → 264/25.4 ≈ 10.4 px/mm.**
- Chatter mapping (`main.cpp`): `maxMm = max(0.2, (size-1)·1.0)` →
  size3 = 2 mm, size2 = 1 mm; `setWidthRange(2 px, maxMm·pxPerMm)`.

## 4. Chatter's current approximation (`InkEngine`, pen type 21)

```
f      = 0.25·dirF + 0.50·pressure + 0.12·speedF + 0.13·tiltMag   // clamp 0..1
width  = minWidth + (maxWidth − minWidth)·f
  dirF    = 0.5 + 0.5·(Δy/len)     // downstroke → 1, upstroke → 0
  speedF  = 1 − clamp(speed/3, 0, 1)   // speed = len/dt (px/ms), slow → 1
  tiltMag = clamp(hypot(tiltX, tiltY), 0, 1)   // from event2 ABS_TILT_X/Y (±9000)
```
Uniform (non-calligraphy) pens: `f = 0.5 + 0.5·pressure`.
`PenDevice` emits `strokeMove(pos, pressure, tiltX, tiltY, eraser)`.

## 5. Tooling (in `tools/`)

- `setbufshim.cpp` — LD_PRELOAD shim; proved `setBuffers` capture (feasibility).
- `fbdump.cpp` — LD_PRELOAD into xochitl; snapshots the framebuffer to
  `/home/root/fbdump.{png,bmp}` when `/tmp/fbdump` is touched (use BMP — no PNG
  plugin). Note: `setBuffers` interposition into *xochitl* via LD_PRELOAD was
  flaky/unconfirmed in one attempt; the in-process `--export-dynamic` capture in
  Chatter itself is reliable.

## 6. Open questions / ways to go deeper later

- Recover exact reMarkable widths-per-size and the nib/pressure curves by
  snapshotting stock strokes (fix the xochitl fbdump path) and measuring, or by
  parsing the v6 `.rm` per-point width/direction data.
- Add a proper speed estimate (smoothed) and tilt-direction (not just magnitude).
- Confirm `LastPenSize` values for the thin/medium categories (only size 3 = 3.0
  is confirmed).