Initial commit: Chatter — assistive-writing app for reMarkable Paper Pro Move
Direct-framebuffer ink pipeline (stock-quality strokes), finger-wipe erase, growable scrolling canvas with color-ghost cleanup, bidirectional toggle with a persistent 4-finger return launcher, instant button feedback. Includes prebuilt aarch64 binaries (dist/), build/deploy/install scripts, a user guide, and a complete technical reference. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
140
doc/Chatter_phase1_findings.md
Normal file
140
doc/Chatter_phase1_findings.md
Normal file
@@ -0,0 +1,140 @@
|
||||
# Chatter — Phase 1 On-Device Findings
|
||||
|
||||
**Date:** 2026-06-25
|
||||
**Device:** reMarkable Paper Pro Move (RM03A, codename "Chiappa")
|
||||
**Method:** Read-only SSH probing from Jatke (`ssh chatter`). No changes made to
|
||||
the device except enabling developer mode / SSH (Phase 0) and adding our SSH key.
|
||||
|
||||
This document records what we learned and the implementation decisions those
|
||||
findings support. It feeds the phase plan in
|
||||
[`Chatter_implementation.md`](Chatter_implementation.md).
|
||||
|
||||
---
|
||||
|
||||
## 1. Hardware & OS
|
||||
|
||||
- **SoC / arch:** NXP **i.MX93**, **aarch64** (Arm Cortex-A55). Kernel
|
||||
`6.12.49+git-imx93-chiappa`.
|
||||
- **OS:** Codex Linux 5.7.121 (Yocto **scarthgap**), image `3.27.1.0`. glibc
|
||||
(`/lib/ld-linux-aarch64.so.1`, `libc.so.6`).
|
||||
- **RAM:** 2 GB (~1.4 GB free) — ample for our app.
|
||||
- **Storage (matters for deployment):**
|
||||
- `/` (rootfs): ~435 MB, **only ~88 MB free (78% used)** — also an
|
||||
OS-managed overlay, likely reset by firmware updates. **Do not install here.**
|
||||
- `/home`: **46 GB encrypted volume, ~45.8 GB free.** Install Chatter under
|
||||
`/home` (e.g. `/home/root/chatter`).
|
||||
|
||||
## 2. Input devices
|
||||
|
||||
Pen and finger are **separate evdev devices** — finger-vs-stylus discrimination
|
||||
is free, exactly what Chatter's gesture design needs.
|
||||
|
||||
| Node | Name | Role | Notes |
|
||||
|---|---|---|---|
|
||||
| `event0` | `bbnsm:pwrkey` | Power button | |
|
||||
| `event1` | Hall effect sensors | Folio open/close | switch events |
|
||||
| **`event2`** | **Elan marker input** | **Stylus / Marker** | on SPI |
|
||||
| **`event3`** | **Elan touch input** | **Capacitive multitouch (finger)** | `INPUT_PROP_DIRECT` |
|
||||
|
||||
**Pen (`event2`)** — digitizer space **6760 × 11960**:
|
||||
- Buttons: `BTN_TOOL_PEN`, **`BTN_TOOL_RUBBER`** (Marker Plus eraser end —
|
||||
flip-to-erase is detectable), `BTN_TOUCH`, `BTN_STYLUS`, `BTN_STYLUS2`.
|
||||
- `ABS_PRESSURE` 0–4096, `ABS_DISTANCE` (hover) 0–65535, `ABS_TILT_X/Y` ±9000.
|
||||
|
||||
**Touch (`event3`)** — grid **1248 × 2208**:
|
||||
- Up to **10 contacts** (`ABS_MT_SLOT` 0–9) with `ABS_MT_POSITION_X/Y`,
|
||||
`ABS_MT_PRESSURE` (0–255), `ABS_MT_TRACKING_ID`, `ABS_MT_TOOL_TYPE`.
|
||||
|
||||
All three coordinate spaces share the screen's portrait aspect (~0.5625):
|
||||
panel 954×1696, pen 6760×11960, touch 1248×2208.
|
||||
|
||||
## 3. Display pipeline
|
||||
|
||||
- **DRM/KMS only — there is NO `/dev/fb*`** and `/sys/class/graphics` is empty.
|
||||
Driver is **`imx-drm`**; device `/dev/dri/card0`; connector
|
||||
**`card0-LVDS-1`** ("connected", "disabled" at rest — normal for e-ink, which
|
||||
holds its image without continuous scanout).
|
||||
- The connector advertises a **packed `365×1700` mode** (hardware buffer for the
|
||||
E Ink Gallery 3 color subpixel layout); the logical screen is 954×1696. The
|
||||
mapping is handled by the epaper plugin (below) — we don't touch it directly.
|
||||
- **Implication:** the `mmap /dev/fb0` approach used by rmkit / libreMarkable on
|
||||
older grayscale models **does not apply** to this device.
|
||||
|
||||
## 4. Graphics / UI stack — the key finding
|
||||
|
||||
- The stock app **`xochitl`** is a **Qt 6.8.2 / Qt Quick (QML)** application.
|
||||
- It renders through a **custom Qt platform (QPA) plugin: `epaper`**
|
||||
(`/usr/lib/plugins/platforms/libepaper.so`, links `libdrm`). Available QPA
|
||||
plugins on-device: **`epaper`**, `minimal`, `offscreen`, `vnc`.
|
||||
- Qt 6.8.2 runtime libraries and QML modules (`QtQuick`, `QtQuickControls2`,
|
||||
etc.) are all present under `/usr/lib` and `/usr/lib/qml`. No dev headers
|
||||
on-device (expected — we cross-build).
|
||||
|
||||
**This means the native path is wide open:** build Chatter as a **Qt 6 app** and
|
||||
run it with **`-platform epaper`** to inherit working e-ink display + refresh,
|
||||
instead of reverse-engineering DRM/KMS and waveform handling. This resolves the
|
||||
project's highest-risk unknown in our favor.
|
||||
|
||||
## 5. Stock app & services
|
||||
|
||||
- `xochitl.service` — the main UI app (`/usr/bin/xochitl`, pid varies).
|
||||
- `marker-manager.service` — "Remarkable CSL Marker Manager" (pen support).
|
||||
- `rm-sync.service` — document sync to the reMarkable cloud.
|
||||
- Plus metrics / MDM / crash-uploader services.
|
||||
|
||||
**Coexistence / toggle implication:** on a DRM/KMS device only one process owns
|
||||
the display (DRM master) at a time, and xochitl holds it (and likely grabs the
|
||||
input devices) while running. So the Chatter ↔ standard-GUI **toggle is a
|
||||
hand-off** (stop or pause `xochitl` ↔ run Chatter, and a return path), **not an
|
||||
overlay**. The exact mechanism (systemctl stop/start vs. SIGSTOP/SIGCONT) is to
|
||||
be validated in the Phase 2 spike.
|
||||
|
||||
## 6. Document store (for the Save phase)
|
||||
|
||||
Location: **`/home/root/.local/share/remarkable/xochitl/`**. Per document:
|
||||
|
||||
- `UUID.metadata` — JSON: `visibleName`, `parent` (empty = top level; a folder
|
||||
is a `CollectionType` document whose UUID is used as children's `parent`),
|
||||
`type` (`DocumentType`), timestamps, `pinned`, etc.
|
||||
- `UUID.content` — JSON: page list under `cPages.pages[]` (each page has an `id`,
|
||||
ordering `idx`, `template`, scroll position, CRDT-style `timestamp` fields).
|
||||
- `UUID/` — directory of per-page **`<pageUUID>.rm`** files.
|
||||
- `UUID.thumbnails/` — page thumbnails.
|
||||
- Store-root extras (Codex additions): `.tree` (binary "rM sync tree" index) and
|
||||
`rm-search-index.db` (SQLite **search index**, not the canonical store).
|
||||
|
||||
**`.rm` page format:** header confirmed as **`reMarkable .lines file,
|
||||
version=6`** — the v6 binary scene-tree format (documented by the community,
|
||||
e.g. `rmscene`). Authoring valid v6 is possible but non-trivial.
|
||||
|
||||
**Save options (decide in Phase 5):**
|
||||
1. **PDF/PNG-backed document** — render Chatter's canvas to PDF/PNG and create a
|
||||
document that references it (xochitl already supports PDF documents). Far
|
||||
simpler and robust; still a real, renamable/syncable library item.
|
||||
2. **Native v6 `.rm`** — author the binary page format so the transcript is an
|
||||
editable notebook. More work; revisit only if option 1 proves insufficient.
|
||||
|
||||
Either way, transcripts go under a **`Chatter`** `CollectionType` folder, with
|
||||
filenames derived from date/time.
|
||||
|
||||
---
|
||||
|
||||
## 7. Decisions supported by Phase 1
|
||||
|
||||
1. **Stack: C++ / Qt 6.8.2**, rendered via the on-device **`epaper` QPA plugin**.
|
||||
Cross-build with a Yocto **scarthgap** aarch64 SDK matching Qt 6.8.2 / the
|
||||
device glibc.
|
||||
2. **Input:** pen from `event2` (including `BTN_TOOL_RUBBER` for flip-to-erase),
|
||||
finger from `event3` (10-pt). Confirm in Phase 2 whether Qt+epaper already
|
||||
delivers these or we read evdev directly for custom gestures.
|
||||
3. **Toggle = display/input hand-off** with xochitl, not an overlay.
|
||||
4. **Save = PDF-backed document** (leaning), in a `Chatter` folder.
|
||||
5. **Deploy under `/home`,** never the rootfs.
|
||||
|
||||
## 8. Open items for the Phase 2 spike
|
||||
|
||||
- Stand up the cross-SDK and build a trivial Qt app; run it with
|
||||
`-platform epaper` **after stopping xochitl**; confirm it draws and refreshes.
|
||||
- Verify how input arrives in a Qt app under epaper (Qt event stream vs. raw
|
||||
evdev) and how reliably finger and pen separate at the Qt layer.
|
||||
- Validate the toggle hand-off (stop/start vs. stop/cont) and a return path.
|
||||
Reference in New Issue
Block a user