ack/chatter
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial commit: Chatter — assistive-writing app for reMarkable Paper Pro Move

Browse Source 
Direct-framebuffer ink pipeline (stock-quality strokes), finger-wipe erase,
growable scrolling canvas with color-ghost cleanup, bidirectional toggle with a
persistent 4-finger return launcher, instant button feedback. Includes prebuilt
aarch64 binaries (dist/), build/deploy/install scripts, a user guide, and a
complete technical reference.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Andy Kopra
2026-06-27 16:34:28 +02:00
commit 5f21d9099c
43 changed files with 2878 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

19
.gitignore vendored Normal file
View File

@@ -0,0 +1,19 @@
# Build output
/build/
build*/
# Secrets — device credentials & cloud tokens must never be committed
doc/tablet_access.md
*.token
# Editor/OS cruft
*~
.DS_Store
# Local Claude Code data
.claude/
# Compiled tool binaries (built by build.sh / manually)
/tools/chatter-launcher
/tools/grabtest
/tools/*.so

42
CMakeLists.txt Normal file
View File

@@ -0,0 +1,42 @@
cmake_minimum_required(VERSION 3.16)
project(chatter VERSION 0.1 LANGUAGES CXX)
set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_AUTOMOC ON)
# Per reMarkable's qt_epaper guide: pure Qt Quick (no Qt Widgets).
find_package(Qt6 REQUIRED COMPONENTS Quick Gui)
qt_standard_project_setup(REQUIRES 6.5)
qt_add_executable(chatter
src/main.cpp
src/PenDevice.h
src/PenDevice.cpp
src/AppControl.h
src/AppControl.cpp
src/InkEngine.h
src/InkEngine.cpp
src/FbCapture.h
src/FbCapture.cpp
src/epfb.h
)
qt_add_qml_module(chatter
URI Chatter
VERSION 1.0
QML_FILES
qml/Main.qml
)
target_link_libraries(chatter PRIVATE
Qt6::Quick
Qt6::Gui
# EPFramebuffer::instance/swapBuffers live in the epaper scenegraph plugin.
${CMAKE_SYSROOT}/usr/lib/plugins/scenegraph/libqsgepaper.so
)
# Export the executable's dynamic symbols so our setBuffers interposer (in
# FbCapture.cpp) wins the cross-DSO call from the epaper platform plugin.
target_link_options(chatter PRIVATE -Wl,--export-dynamic)

134
README.md Normal file
View File

@@ -0,0 +1,134 @@
# Chatter
A minimal, purpose-built writing interface for the **reMarkable Paper Pro Move**
(model RM03A), used as an **assistive-speech device**: the user writes, her
conversation partner reads. Chatter *augments* the stock reMarkable software; it
does not replace it.
- **End-user instructions:** [`doc/Chatter_user_guide.md`](doc/Chatter_user_guide.md)
- **Full technical reference (developers start here):**
[`doc/Chatter_technical_reference.md`](doc/Chatter_technical_reference.md)
- **Plan & history:** [`doc/Chatter_implementation.md`](doc/Chatter_implementation.md)
## Repository layout
```
dist/ Prebuilt tablet binaries (ready to install — see Quick install)
src/ C++ sources (ink engine, framebuffer capture, pen reader, …)
qml/ Qt Quick UI (the static Back/Clear bar + touch handling)
tools/ Standalone helpers (the return-to-Chatter launcher, probes)
scripts/ Build, deploy, and install scripts
doc/ Documentation (the technical reference has a full document map)
```
---
## Quick install (prebuilt binary — no build needed)
For installing/testing on a tablet, you do **not** need to build anything. The
repo ships ready-to-run aarch64 binaries in [`dist/`](dist/) (`dist/chatter`,
`dist/chatter-launcher`). You just need SSH access to the tablet.
### 1. One-time device setup
On the tablet (this **factory-resets** it — back up / sync first):
1. **Enable developer mode:** Hamburger menu → Settings → Software → Advanced →
Developer mode → Accept. (Details: [`doc/developer_mode_screen.md`](doc/developer_mode_screen.md).)
2. Reconnect wifi (the reset wipes it).
3. Get the root credentials: Settings → Help → **Copyright and licenses** →
under **GPLv3 Compliance** (root username, password, and the device IPs).
4. **Enable SSH over wifi** (off by default): connect over USB (`10.11.99.1`),
SSH in, run `rm-ssh-over-wlan on`.
5. Add your workstation's SSH public key to the tablet's
`/home/root/.ssh/authorized_keys`.
### 2. Point the `chatter` SSH alias at the tablet
In your `~/.ssh/config`:
```
Host chatter
HostName 192.168.x.x # the tablet's wifi IP (or 10.11.99.1 over USB)
User root
IdentityFile ~/.ssh/id_ed25519
```
> The tablet's wifi IP can change (DHCP), and it drops wifi when asleep. If
> `ssh chatter` stops connecting, wake the tablet and re-check the IP on the same
> **Copyright and licenses → GPLv3 Compliance** screen (a DHCP reservation on your
> router avoids this).
### 3. Install
```
scripts/deploy-and-run.sh # copies dist/chatter and launches it on the panel
scripts/install-launcher.sh # installs the persistent 4-finger return launcher
```
`scripts/restore-xochitl.sh` returns to the standard reMarkable GUI at any time.
> The tablet's actual root password / IPs are **not** in this repo (they live in
> the gitignored `doc/tablet_access.md`) — get them from Andreas.
---
## Building from source
Only needed if you're **changing the code**. Chatter is a Qt 6 app
**cross-compiled** to the tablet's aarch64 CPU; you build on a host and copy the
binary over.
### Prerequisites
1. **A Linux host** (the reMarkable SDK is a Yocto toolchain — Linux only).
2. **The reMarkable "Chiappa" SDK** (cross-compiler + Qt 6.8.2 sysroot, ~486 MB),
from developer.remarkable.com. Install it, e.g.:
```
./remarkable-…-chiappa-…-toolchain.sh -d ~/external/remarkable-sdk/chiappa-3.27.0.97
```
The confirmed **x86_64-host** build is:
`https://storage.googleapis.com/remarkable-codex-toolchain/3.27.0.97/chiappa/remarkable-production-image-5.7.119-chiappa-public-x86_64-toolchain.sh`
An **aarch64-host** variant exists at the same path with `x86_64`→`aarch64`
(confirm the exact link on developer.remarkable.com).
### Build & deploy
```
CHATTER_SDK=/path/to/chiappa-sdk scripts/build.sh # -> build/chatter + tools/chatter-launcher
scripts/deploy-and-run.sh # prefers build/chatter when present
scripts/install-launcher.sh
```
### Building from macOS (Linux container)
There is **no macOS build of the reMarkable SDK**, so build inside a **Linux
container**. The cross-compiled output targets the tablet either way; only the
*host* running the SDK must be Linux.
1. Install a container runtime — **OrbStack** or **Docker Desktop**.
2. Start a Linux container matching your Mac's CPU and mount the repo:
- **Apple Silicon Mac:** an **arm64** image (runs natively, no emulation):
```
docker run -it --platform linux/arm64 -v "$PWD":/work ubuntu:24.04
```
Inside it, install the **aarch64-host** Chiappa SDK.
- **Intel Mac:** an **x86_64** image + the **x86_64** SDK.
3. In the container, install what the SDK/build need
(`apt-get update && apt-get install -y build-essential cmake file which`),
run the SDK self-extractor, then `cd /work && CHATTER_SDK=… scripts/build.sh`.
4. Deploy from the container (or from the Mac, if SSH to the tablet is set up
there): `scripts/deploy-and-run.sh build/chatter`.
> This container route is the standard reMarkable cross-build approach but has
> not been verified end-to-end here — if you hit a missing-package error in the
> container, install that package and continue.
---
## Status
Working on hardware: stock-quality ink matching the selected pen, finger-wipe
erase, whole-page Clear, stylus eraser, bidirectional toggle with a persistent
return launcher, and vertical scrolling on a growable canvas. Open items (Save,
horizontal scroll/zoom, field tuning) are in the technical reference §10.

BIN
dist/chatter vendored Executable file
View File

Binary file not shown.

BIN
dist/chatter-launcher vendored Executable file
View File

Binary file not shown.

2
doc/.gitignore vendored Normal file
View File

@@ -0,0 +1,2 @@
tablet_access.md

96
doc/Chatter_application_proposal.md Normal file
View File

@@ -0,0 +1,96 @@
# Chatter — A custom application for the reMarkable tablet
## Background
I am helping someone who has a hard time speaking and who is using a
[reMarkable tablet](https://image.email.remarkable.com/lib/fe3511737364047c771479/m/1/e39f7595-fc99-4f8c-b7e3-dde042451668.pdf)
in conversations to write what she cannot say. I believe that the
reMarkable table is the best choice for a digital handwriting interface.
However, the tablet is designed for taking notes that can be preserved or even
transformed into digital text. It is not intentionally designed to be used as an
*assistive-speech device*. There are also a wide variety of customizable options
that are typically unnecessary if using the table in that way.
## Project goal
I would like to create a replacement for the reMarkable graphical user interface
(GUI) that would be tailored for using the table as an assistive-speech device.
This interface would not replace the standard GUI; choosing tablet settings for
the writing style or other interface parameters would still be done in the
standard GUI. This assumes, however, that the user can change settings as needed
in the normal way (or that technical help is available when necessary).
The goal here is to make the interaction with the tablet during conversation as
simple as possible.
The custom application's name is "Chatter", with the implication that
conversational interaction can be accelerated. The word also can refer to the
speaker as well as to the result of speaking.
## User-interface elements
In watching the user with the tablet's standard GUI, I have noticed a confusion
about whether writing or erasing is the active mode. A simple action like
clearing the page requires several user-interface gestures. The user's
inability to easily use the table is often not wholly due to a lack of technical
understanding but a failure to remember technical details when they are not
intuitive.
Chatter implements four actions, ordered here by their potential frequency of
use:
- *Erase part of the text* — This should be simple and intuitive gesture. For
example, I have seen the user wiping her fingers back and forth across some
text to erase it.
- *Erase the page* — Erasing the entire page should be a single gesture. The
blank page that results continues to use the current pen styling and other
parameters set in the standard GUI.
- *Save current text to a file named by time* — The current page is saved
with an automatically generated name that contains a human-readable text of the year,
month, date, hour, minutes and seconds. The seconds may not be necessary, but
I would rather have an elaborate name than risk losing a previously saved
file. These Chatter transcripts should be saved to a separate directory to
avoid cluttering the main screen. Deleting and renaming these files are
possible through the standard GUI; a tutorial about that should be provided if
the user expresses interesting in managing the transcripts in that way.
- *Toggle between the standard GUI and Chatter* — The standard GUI should still
be available through a single gesture; Chatter augments the tablet's interface
and does not replace it. However, returning to Chatter will require an
addition to the standard GUI that will be memorable for the user.
### Implementation considerations
If the table can differentiate between a fingertip and the stylus, then erasing
part of the text as well as the entire page could be implemented by a finger
gesture. Rubbing the finger back and forth of an area of text could remove any
drawing close to that area (where "close" might be determined experimentally
working with the user). A diagonal stroke with a finger from upper-right to
lower-left would be interpreted as a request to erase the entire page.
If differentiating between fingertip and stylus is not possible, then Chatter
should define some shortcut for the stylus that easily creates the standard
region-based erase gesture. For example, if the user circles an area twice,
that could signify erasure, not drawing. A stylus stroke from upper-right to
lower-left could specify whole-page erasure, even if the finger stroke
method is also possible.
The toggling the standard GUI and saving a file are not actions that the user
would frequently perform. I think that small icons at the bottom of the page,
activated by stylus or finger, would provide an ongoing reminder of how to do
those actions. The standard GUI will need to provide some way of switching to
Chatter; the user should suggest a method that she would find memorable.
## Deployment and refinement
A developer working on this project will install the custom GUI on the user's
tablet. After verifying that the installation is successful, the developer will
watch the user with the tablet to evaluate the current interface efficiency of
Chatter, talking with the user about what is missing and what could be improved.
Paying attention to the user's questions that begin, "Why does this..." and "Why
doesn't this..." will be very important in refining the design. I believe that
using a custom interface for the tablet might also inspire the user to suggest
other features that would not be obvious to the developers.

381
doc/Chatter_implementation.md Normal file
View File

@@ -0,0 +1,381 @@
# Chatter — Implementation Plan & Method
**Status:** Living document — revised as we learn. Read the Changelog at the bottom for what changed.
**Version:** 0.7 (2026-06-26)
**Audience:** The Chatter development team (Andreas, Matt, and Claude as development assistant).
> This document is the shared source of truth for *how* we build Chatter. It is
> deliberately incremental: early phases will answer questions that later phases
> depend on, and this document will be updated as those answers come in. If
> something here is marked **(unknown / to confirm on-device)**, treat it as an
> open question, not a decision.
---
## 1. What Chatter is
Chatter is a minimal, purpose-built interface for a reMarkable tablet used as an
**assistive-speech device** by a user who has difficulty speaking — she writes,
and her conversation partner reads. Chatter **augments** the tablet; it does not
replace the standard reMarkable GUI.
The design priority is **intuitiveness under intermittent recall**: the user
should not have to remember technical steps. See the full motivation and the
user-experience reasoning in [`Chatter_application_proposal.md`](Chatter_application_proposal.md).
Chatter's core actions, by expected frequency:
1. **Erase part of the text** — a simple, intuitive gesture (e.g. wiping a
fingertip back and forth over an area).
2. **Erase the whole page** — a single gesture (e.g. a diagonal finger stroke,
upper-right → lower-left).
3. **Toggle to/from the standard GUI** — *necessary*; Chatter must be able to
hand off to the stock interface and be returned to.
4. **Save the current page** — *importance to the user is not yet confirmed*; a
single button writes a timestamped transcript into a "Chatter" folder.
---
## 2. Target device and key constraints
**Device: reMarkable Paper Pro Move — model RM03A.** (Identified from the
official user-guide PDF linked in the proposal.)
| Property | Value | Why it matters |
|---|---|---|
| Display | E Ink Gallery 3, **color**, 7.3", 954×1696 (264 PPI), backlit | Color refresh pipeline differs from older grayscale models |
| Touch | Multi-point capacitive | Finger gestures (erase) are viable |
| Pen | Active Marker / Marker Plus (flip-to-erase on Plus) | Pen and touch are **separate input streams** → we can tell finger from stylus natively |
| SoC / RAM | ARM 1.7 GHz dual A55, 2 GB RAM, 64 GB | Standard ARM-Linux cross-compile target |
| OS | "Codex" (custom Linux) | Stock note app is Xochitl (Qt/C++, proprietary) |
| Dev access | Enable **developer mode**, which requires a **factory reset** | **Back up the device first** |
**Key risk — this is the newest model.** The mature community frameworks
(rmkit, libreMarkable, Plato, KOReader ports) were built against the older
RM1/RM2 grayscale framebuffer and refresh ioctls. They likely **do not support
the Paper Pro Move yet**, or only partially. We should expect to do
**first-principles work** — discovering this model's framebuffer format, color
refresh path, and input device nodes by probing the hardware over SSH — rather
than relying on an existing framework. This raises the value of hands-on device
access early and argues for a small, well-proven first milestone.
---
## 3. Roles
- **Andreas** — development, with Claude's assistance, on his own Paper Pro Move
over SSH.
- **Matt** — installation and QA, including on-site testing with the end user on
her tablet. Matt has the same model (Paper Pro Move). Matt may also take on
development; the split will be worked out as the project progresses.
- **Claude** — development assistant: writes/reviews code, drives the
build-deploy loop over SSH where possible, and maintains this document.
This document is how Matt stays in sync. When the design changes, the change
lands here first.
---
## 4. Development phases
The phases are ordered so each de-risks the next. **Toggle (necessary) precedes
Save (provisional).**
### Phase 0 — Safety & access
- **Back up first** (critical on the user's device; skippable on a new tablet
with nothing saved). Enabling developer mode **performs a factory reset** — on
the Paper Pro family, SSH is unavailable until developer mode is on, and
turning it on wipes the device. Sync to the reMarkable cloud account before
enabling. Document this backup procedure so Matt can repeat it.
- **Enable developer mode:** Hamburger menu (upper left) → Settings → Software →
Advanced → Developer mode → Accept. This factory-resets the device (deletes
all local files; user data cleared on reboot). Re-pair with the reMarkable
account afterward to access cloud content. (Disabling later requires
reMarkable's recovery application.)
- **Reconnect wifi** after the reset (the wifi config is wiped).
- **Get SSH credentials:** Menu → Settings → Help → Copyright and licenses;
the root username (`root`), password, and device IP addresses are under the
"GPLv3 Compliance" header. USB exposes the tablet at `10.11.99.1`.
- **Enable SSH over wifi (it is OFF by default):** connect over USB, SSH in,
and run `rm-ssh-over-wlan on`. Only then is the listed `192.168.x.x` address
reachable; after that the USB cable is optional.
- Install your workstation's SSH public key into the tablet's
`~/.ssh/authorized_keys` (user `root`) for passwordless access.
- Confirm SSH login, `scp` of a binary, and remote execution.
- Stand up the x86_64-hosted cross-toolchain for Codex OS; verify a trivial
ARM binary runs on-device.
> Dev environment note: development host is **Jatke** (on the wired LAN); we
> connect to the tablet **over wifi** (USB not in use).
### Phase 1 — On-device investigation (no app code; produces a findings writeup)
Answer the questions that can only be answered with the tablet over SSH. See the
checklist in §5. Output: a short findings document added to `doc/`.
**Status: substantially complete (2026-06-25)** — see
[`Chatter_phase1_findings.md`](Chatter_phase1_findings.md). Key results: pen
(`event2`) and finger (`event3`) are separate evdev devices; the display is
DRM/KMS only (no fbdev) driven by `imx-drm`; and the stock app is a Qt 6.8.2 app
rendering through a reusable **`epaper` QPA plugin** — so Chatter should be a
Qt 6 app run with `-platform epaper`. Remaining sub-items (verifying `-platform
epaper` from a third-party binary, input delivery, toggle hand-off) roll into
the Phase 2 spike.
### Phase 2 — Minimal canvas spike
Read the pen, render strokes to the (color) display, and clear the page —
nothing more. This proves the two hardest pieces (input + color refresh) end to
end and de-risks everything after it.
**Status (2026-06-25): display/toolchain half PROVEN on hardware.** A
third-party Qt Quick app (`build/chatter`) cross-built with the Chiappa SDK
renders crisp black-on-white on the e-paper via `-platform epaper`
(`QT_QUICK_BACKEND=epaper`).
**Status (2026-06-26): inking WORKS, with one known quality issue.** Custom
evdev pen reader (`src/PenDevice.cpp`, reads `event2`) drives a
`QQuickPaintedItem` canvas (`src/InkCanvas.cpp`): ink tracks the pen, orientation
correct, eraser (BTN_TOOL_RUBBER) erases, finger-tap "Clear" works, latency good.
- **Known issue:** strokes render as a "dashed line that fills in" (two-pass),
unlike the solid stock pen. **Root cause:** we use the panel's default
*grayscale* mode. **Fix path:** xochitl wraps its writing area in a QML
`ScreenModeItem` set to **Mono/FAST** with a dedicated **pen waveform**
(`/usr/share/remarkable/ct33_pen.bin`). The control is the private
`EPScreenModeItem` class in `libqsgepaper.so` (`setMode()`, modes
Mono/FAST/FastGrayscale) — not a public QML type, no SDK header — so Chatter
must integrate it directly (instantiate, link the plugin, drive `mode` via the
meta-object, wrap the canvas). Antialiasing on/off does not affect it.
- **Pen-style decision:** Chatter uses the pen style set in the standard GUI
(xochitl.conf `LastPen`/`LastPenSize`/`LastPenColor`, readable via QSettings),
re-read so it tracks changes the user makes after toggling — no style menu in
Chatter.
**Lessons learned (encoded in `scripts/`):**
- The e-paper framebuffer is a **singleton guarded by an flock**
(`/tmp/epframebuffer.lock`, from `libqsgepaper`'s `EPFramebufferAcep2`). Only
ONE process may hold it; a stranded instance shows `Failed to lock
epframebuffer` / `Failed to initialize SWTCON` and the panel stays blank.
Always stop the previous instance first. Journal success marker:
`SWTCON initialized \o/`.
- **Run as a transient systemd service** (`systemd-run --unit=chatter …`), not an
ssh background job (which holds the SSH channel open and can strand the
process). Manage with `systemctl stop chatter` / `journalctl -u chatter`.
- A full-screen `Window` needs an **explicit size** (`width: Screen.width;
height: Screen.height`); `visibility: FullScreen` alone left it unsized/blank.
- xochitl runs with **no special Qt env** — `-platform epaper` +
`QT_QUICK_BACKEND=epaper` suffices.
**Run recipe:** `scripts/build.sh && scripts/deploy-and-run.sh`;
`scripts/restore-xochitl.sh` to return to the standard GUI.
### Phase 3 — Erase gestures
- Region erase: finger wipe removes ink near the wiped area ("near" tuned
experimentally with the user).
- Whole-page erase: single diagonal finger stroke (upper-right → lower-left).
- Provide stylus fallbacks if finger/stylus separation proves unreliable.
### Phase 4 — Toggle to/from the standard GUI *(necessary)*
**Status (2026-06-26): BIDIRECTIONAL toggle works on-device (verified).**
- **Chatter → standard:** top-left **"Back"** button → `AppControl::returnToStandard()`
→ transient unit stops chatter, starts xochitl.
- **standard → Chatter:** a persistent **launcher daemon** (`tools/chatter_launcher.c`,
installed as a rootfs systemd service via `scripts/install-launcher.sh`) watches
the touch device and, on a **4-finger hold (~700 ms)** while xochitl is in front,
runs `to-chatter.sh` to switch to Chatter. Gesture is tunable; finalize the
memorable one with the user.
- **Deployment note:** `/etc` and `/run` are volatile overlays (wiped on reboot);
persistent installs go to `/home` (app/binaries/scripts) or the rootfs
(`mount -o remount,rw /`, lost on OS update). Disable OS auto-updates on a
delivered device.
- A gesture/button in Chatter hands off to the standard reMarkable GUI.
- A **memorable** mechanism returns from the standard GUI to Chatter. The
proposal asks that the user help choose this; we will prototype options and
let her pick.
- **Phase 1 insight:** because the display is DRM/KMS (single master) and
xochitl owns it while running, the toggle is a **hand-off**, not an overlay —
stop/pause `xochitl` to give Chatter the display, and reverse to return.
Mechanism (`systemctl stop/start` vs. `SIGSTOP/SIGCONT`) to be validated.
### Phase 5 — Save the page *(provisional — pending confirmation it matters to the user)*
- A single visible button saves the current page.
- Filename fully derived from date/time (year, month, day, hour, minute,
second).
- Written into a folder named **"Chatter"**, using the same on-disk document
format the stock app uses, so transcripts appear as normal notebooks and can
be renamed/deleted/synced via the standard GUI (see §6).
### Phase 6 — Field test & refine
Matt installs and observes the user. Capture especially the "Why does this…" and
"Why doesn't this…" questions; feed them back into this document and the phase
plan.
---
## 5. On-device investigation checklist (Phase 1)
These are the open questions the hands-on phase exists to answer.
- **Input devices:** which `/dev/input/event*` is the pen vs. the capacitive
touch; event format, coordinate range and orientation, pressure; the
flip-to-erase signal on Marker Plus.
- **Display:** how the color Gallery 3 panel is driven — framebuffer device and
pixel format, and the refresh mechanism (ioctls or this model's equivalent).
*(Highest-risk unknown.)*
- **Document store:** where Xochitl stores notebooks; the
`.metadata` / `.content` / per-page `.rm` layout for this model; the `.rm`
format version (newer, color-capable); how a *collection* (folder) is
represented; and whether a hand-built document appears in the library after a
restart/sync.
- **Coexistence & toggle:** how to launch our binary alongside / over Xochitl,
and what a reliable toggle and return-to-Chatter mechanism looks like.
---
## 6. The "save" approach (clarification)
reMarkable exposes **no public callable on-device API** for creating notebooks.
The stock app simply **writes files to an on-disk document store** in a specific
format (roughly: a UUID per document, a `.metadata` JSON, a `.content` JSON, and
one binary `.rm` line-file per page, plus the color/folder data and a sync
layer). A "folder" such as the **Chatter** folder is itself a document of type
*collection*.
So "save using the same method the environment uses" means: **write our canvas
into the exact on-disk format Xochitl reads, for this model's format version.**
The open question for Phase 5 is confirming/replicating the Paper Pro Move's
current `.rm` format. This is investigated in Phase 1.
---
## 6a. Confirmed device facts (from SSH, 2026-06-25)
- **Internal codename:** "Chiappa" (`/proc/device-tree/model` = "reMarkable
Chiappa"; hostname `imx93-chiappa`). This is the Paper Pro Move (RM03A).
- **SoC:** NXP **i.MX93** (Arm Cortex-A55). Relevant to the toolchain target and
to how the display is driven.
- **OS:** Codex Linux **5.7.121** (Yocto **scarthgap**), image version
`3.27.1.0`.
- **SSH server:** **dropbear** (not OpenSSH). Root login; `~/.ssh/authorized_keys`
honored. Wifi SSH enabled via `rm-ssh-over-wlan on`.
- **Access from Jatke:** passwordless via ed25519 key as `root` over wifi (the
tablet's IP) or `root@10.11.99.1` (USB, fixed on every reMarkable). The tablet's
actual IP/MAC and credentials are in `tablet_access.md` (SENSITIVE, gitignored).
**Phase 0 status:** access established (dev mode, SSH over USB + wifi, key-based
login, scp verified). Remaining for Phase 0: stand up the cross-toolchain and
run a trivial ARM binary on-device.
## 6b. Toolchain / build & deploy (researched 2026-06-25)
**Official SDK (cross-toolchain + Qt 6.8.2 sysroot), self-extracting `.sh`,
x86_64 host only.** For this device (codename **chiappa**):
```
https://storage.googleapis.com/remarkable-codex-toolchain/3.27.0.97/chiappa/remarkable-production-image-5.7.119-chiappa-public-x86_64-toolchain.sh
```
- ~486 MB. Device image is `3.27.1.0` (Codex 5.7.121); newest published Chiappa
SDK is `3.27.0.97` (5.7.119) — same `3.27.x` family, so binaries built against
it run on the device. (An aarch64-host variant also exists; we use x86_64 for
Jatke.) Download links live on developer.remarkable.com/links; the GCS bucket
itself is not directly listable (403).
- Build host **Jatke** (x86_64, ample disk). SDKs expect a Yocto-supported Linux
host.
**Install & activate:**
```
chmod u+x remarkable-production-image-5.7.119-chiappa-public-x86_64-toolchain.sh
./remarkable-production-image-5.7.119-chiappa-public-x86_64-toolchain.sh -d <install-dir>
source <install-dir>/environment-setup-cortexa55-remarkable-linux # exact filename confirmed post-install (i.MX93 = Cortex-A55)
```
**Build a Qt Quick app** (per developer.remarkable.com/documentation/qt_epaper):
CMake with `find_package(Qt6 REQUIRED COMPONENTS Quick)` +
`qt_add_executable` / `qt_add_qml_module`; `cmake . -B build && cmake --build
build` after sourcing the SDK env.
**Deploy & run on device:**
```
scp build/<app> chatter:/home/root/chatter/ # deploy under /home, not rootfs
ssh chatter 'systemctl stop xochitl' # release the display (pre-authorized)
ssh chatter 'cd /home/root/chatter && QT_QUICK_BACKEND=epaper ./<app> -platform epaper'
```
Constraints (official + Phase 1): **Qt Quick/QML only, no Qt Widgets**; **xochitl
must be stopped** (DRM single-master); **touch is automatic, pen is custom**
(read `event2`). Some SDK/OS combos require copying `libqsgepaper.so` to the
device — verify during the spike.
## 7. Decisions & open questions
**Decided (from Phase 1):**
- **Implementation stack: C++ / Qt 6.8.2**, rendered via the device's **`epaper`
QPA plugin** (`-platform epaper`). Chosen because the stock app proves this
path works and gives us e-ink display + refresh for free; the older
fbdev-mmap approach is not available (DRM/KMS only). Cross-build with a Yocto
scarthgap aarch64 SDK matching Qt 6.8.2. *(Matt — flag any objection.)*
- **Deploy location: under `/home`** (45 GB free), never the rootfs (~88 MB
free, overlay, reset by updates).
- **Toggle is a display hand-off** with xochitl (see Phase 4), not an overlay.
**Still open:**
- **Save mechanism** — leaning **PDF-backed document** over authoring native v6
`.rm`; decide in Phase 5 (and gated by whether save matters to the user).
- **Whether "save" matters to the user** — learned from observation.
- **Toggle gesture + return-to-Chatter mechanism** — prototype and choose with
the user (Phase 4).
- **Erase "closeness" thresholds** — tuned experimentally with the user
(Phase 3).
---
## Changelog
- **0.6 (2026-06-26)** — Inking engine working on hardware: custom evdev pen
reader (`event2`) + `QQuickPaintedItem` canvas — pen draws, eraser erases,
finger-tap **Clear**, good latency. Added top control bar (**Back** left,
**Clear** right); Back hands off to the standard GUI (verified). Bound the
private `EPScreenModeItem` (key-function trick) to control panel refresh mode —
but the **dashed two-pass stroke refresh is the render loop's behavior,
independent of mode**, so it's deferred to a framebuffer-direct effort
(needs disassembly of `swapBuffers` for `EPContentType`/`UpdateFlag`).
Implemented **pen-style matching**: Chatter reads the standard GUI's selected
pen (type/size/color) from `xochitl.conf` `LastWritingTool` and applies width +
color — verified (Calligraphy, size 3, black).
- **0.7 (2026-06-26)** — ⭐ **Solved stroke quality with a direct-framebuffer ink
pipeline.** After proving Qt Quick can't render stock-quality e-ink strokes
(geometry nodes unsupported by the software backend; painted-textures always
get the dashed two-pass; screen-mode and antialiasing irrelevant), switched
architecture: `FbCapture` interposes `EPFramebuffer::setBuffers` in-process
(`-Wl,--export-dynamic`) to capture the real framebuffer image; `InkEngine`
draws strokes straight into it and refreshes each segment with an explicit
`swapBuffers` — **solid, crisp, low-latency, correctly-aligned strokes on
hardware (verified), plus working Clear.** Qt Quick now only renders the static
Back/Clear UI. Key RE findings: `swapBuffers` renders solid (not dashed) on a
single explicit call; intra-DSO calls aren't LD_PRELOAD-interposable but the
cross-DSO `setBuffers` is. Remaining: calligraphic nib (uniform round width vs
the angled direction-dependent Calligraphy nib), return-to-Chatter trigger,
Save, field testing.
- **0.5 (2026-06-25)** — Phase 2 spike: cross-built a Qt Quick hello-app with the
Chiappa SDK and rendered it on the e-paper via `-platform epaper` (proven on
hardware). Recorded lessons (panel flock singleton, run-as-systemd-service,
explicit Window size) and added `scripts/build.sh`, `deploy-and-run.sh`,
`restore-xochitl.sh`. Seeded the source tree (`CMakeLists.txt`, `src/main.cpp`,
`qml/Main.qml`).
- **0.4 (2026-06-25)** — Researched the toolchain (§6b): identified the official
Chiappa SDK (`3.27.0.97` / Codex 5.7.119, x86_64 host, ~486 MB), install/
activate steps, the Qt Quick build + `-platform epaper` run commands, and the
deploy-and-stop-xochitl flow. Confirmed Qt-Quick-only + custom-pen constraints.
- **0.3 (2026-06-25)** — Phase 1 on-device investigation substantially complete
(see `Chatter_phase1_findings.md`). Resolved the stack decision to C++/Qt 6 +
`epaper` QPA plugin; recorded display = DRM/KMS only, pen/finger as separate
evdev devices, toggle = display hand-off, deploy under `/home`, and the
document-store format for the Save phase.
- **0.2 (2026-06-25)** — Phase 0 access established. Added confirmed device
facts (§6a): codename "Chiappa", NXP i.MX93 SoC, Codex Linux 5.7.121
(scarthgap), dropbear SSH, passwordless key access from Jatke. Corrected the
dev-mode menu path and documented that wifi SSH is off by default
(`rm-ssh-over-wlan on`).
- **0.1 (2026-06-25)** — Initial version: scope, target device (RM03A) and its
constraints, roles, phase plan (with Toggle before Save), Phase 1 investigation
checklist, save-format clarification, and open decisions.

140
doc/Chatter_phase1_findings.md Normal file
View File

@@ -0,0 +1,140 @@
# Chatter — Phase 1 On-Device Findings
**Date:** 2026-06-25
**Device:** reMarkable Paper Pro Move (RM03A, codename "Chiappa")
**Method:** Read-only SSH probing from Jatke (`ssh chatter`). No changes made to
the device except enabling developer mode / SSH (Phase 0) and adding our SSH key.
This document records what we learned and the implementation decisions those
findings support. It feeds the phase plan in
[`Chatter_implementation.md`](Chatter_implementation.md).
---
## 1. Hardware & OS
- **SoC / arch:** NXP **i.MX93**, **aarch64** (Arm Cortex-A55). Kernel
`6.12.49+git-imx93-chiappa`.
- **OS:** Codex Linux 5.7.121 (Yocto **scarthgap**), image `3.27.1.0`. glibc
(`/lib/ld-linux-aarch64.so.1`, `libc.so.6`).
- **RAM:** 2 GB (~1.4 GB free) — ample for our app.
- **Storage (matters for deployment):**
- `/` (rootfs): ~435 MB, **only ~88 MB free (78% used)** — also an
OS-managed overlay, likely reset by firmware updates. **Do not install here.**
- `/home`: **46 GB encrypted volume, ~45.8 GB free.** Install Chatter under
`/home` (e.g. `/home/root/chatter`).
## 2. Input devices
Pen and finger are **separate evdev devices** — finger-vs-stylus discrimination
is free, exactly what Chatter's gesture design needs.
| Node | Name | Role | Notes |
|---|---|---|---|
| `event0` | `bbnsm:pwrkey` | Power button | |
| `event1` | Hall effect sensors | Folio open/close | switch events |
| **`event2`** | **Elan marker input** | **Stylus / Marker** | on SPI |
| **`event3`** | **Elan touch input** | **Capacitive multitouch (finger)** | `INPUT_PROP_DIRECT` |
**Pen (`event2`)** — digitizer space **6760 × 11960**:
- Buttons: `BTN_TOOL_PEN`, **`BTN_TOOL_RUBBER`** (Marker Plus eraser end —
flip-to-erase is detectable), `BTN_TOUCH`, `BTN_STYLUS`, `BTN_STYLUS2`.
- `ABS_PRESSURE` 0–4096, `ABS_DISTANCE` (hover) 0–65535, `ABS_TILT_X/Y` ±9000.
**Touch (`event3`)** — grid **1248 × 2208**:
- Up to **10 contacts** (`ABS_MT_SLOT` 0–9) with `ABS_MT_POSITION_X/Y`,
`ABS_MT_PRESSURE` (0–255), `ABS_MT_TRACKING_ID`, `ABS_MT_TOOL_TYPE`.
All three coordinate spaces share the screen's portrait aspect (~0.5625):
panel 954×1696, pen 6760×11960, touch 1248×2208.
## 3. Display pipeline
- **DRM/KMS only — there is NO `/dev/fb*`** and `/sys/class/graphics` is empty.
Driver is **`imx-drm`**; device `/dev/dri/card0`; connector
**`card0-LVDS-1`** ("connected", "disabled" at rest — normal for e-ink, which
holds its image without continuous scanout).
- The connector advertises a **packed `365×1700` mode** (hardware buffer for the
E Ink Gallery 3 color subpixel layout); the logical screen is 954×1696. The
mapping is handled by the epaper plugin (below) — we don't touch it directly.
- **Implication:** the `mmap /dev/fb0` approach used by rmkit / libreMarkable on
older grayscale models **does not apply** to this device.
## 4. Graphics / UI stack — the key finding
- The stock app **`xochitl`** is a **Qt 6.8.2 / Qt Quick (QML)** application.
- It renders through a **custom Qt platform (QPA) plugin: `epaper`**
(`/usr/lib/plugins/platforms/libepaper.so`, links `libdrm`). Available QPA
plugins on-device: **`epaper`**, `minimal`, `offscreen`, `vnc`.
- Qt 6.8.2 runtime libraries and QML modules (`QtQuick`, `QtQuickControls2`,
etc.) are all present under `/usr/lib` and `/usr/lib/qml`. No dev headers
on-device (expected — we cross-build).
**This means the native path is wide open:** build Chatter as a **Qt 6 app** and
run it with **`-platform epaper`** to inherit working e-ink display + refresh,
instead of reverse-engineering DRM/KMS and waveform handling. This resolves the
project's highest-risk unknown in our favor.
## 5. Stock app & services
- `xochitl.service` — the main UI app (`/usr/bin/xochitl`, pid varies).
- `marker-manager.service` — "Remarkable CSL Marker Manager" (pen support).
- `rm-sync.service` — document sync to the reMarkable cloud.
- Plus metrics / MDM / crash-uploader services.
**Coexistence / toggle implication:** on a DRM/KMS device only one process owns
the display (DRM master) at a time, and xochitl holds it (and likely grabs the
input devices) while running. So the Chatter ↔ standard-GUI **toggle is a
hand-off** (stop or pause `xochitl` ↔ run Chatter, and a return path), **not an
overlay**. The exact mechanism (systemctl stop/start vs. SIGSTOP/SIGCONT) is to
be validated in the Phase 2 spike.
## 6. Document store (for the Save phase)
Location: **`/home/root/.local/share/remarkable/xochitl/`**. Per document:
- `UUID.metadata` — JSON: `visibleName`, `parent` (empty = top level; a folder
is a `CollectionType` document whose UUID is used as children's `parent`),
`type` (`DocumentType`), timestamps, `pinned`, etc.
- `UUID.content` — JSON: page list under `cPages.pages[]` (each page has an `id`,
ordering `idx`, `template`, scroll position, CRDT-style `timestamp` fields).
- `UUID/` — directory of per-page **`<pageUUID>.rm`** files.
- `UUID.thumbnails/` — page thumbnails.
- Store-root extras (Codex additions): `.tree` (binary "rM sync tree" index) and
`rm-search-index.db` (SQLite **search index**, not the canonical store).
**`.rm` page format:** header confirmed as **`reMarkable .lines file,
version=6`** — the v6 binary scene-tree format (documented by the community,
e.g. `rmscene`). Authoring valid v6 is possible but non-trivial.
**Save options (decide in Phase 5):**
1. **PDF/PNG-backed document** — render Chatter's canvas to PDF/PNG and create a
document that references it (xochitl already supports PDF documents). Far
simpler and robust; still a real, renamable/syncable library item.
2. **Native v6 `.rm`** — author the binary page format so the transcript is an
editable notebook. More work; revisit only if option 1 proves insufficient.
Either way, transcripts go under a **`Chatter`** `CollectionType` folder, with
filenames derived from date/time.
---
## 7. Decisions supported by Phase 1
1. **Stack: C++ / Qt 6.8.2**, rendered via the on-device **`epaper` QPA plugin**.
Cross-build with a Yocto **scarthgap** aarch64 SDK matching Qt 6.8.2 / the
device glibc.
2. **Input:** pen from `event2` (including `BTN_TOOL_RUBBER` for flip-to-erase),
finger from `event3` (10-pt). Confirm in Phase 2 whether Qt+epaper already
delivers these or we read evdev directly for custom gestures.
3. **Toggle = display/input hand-off** with xochitl, not an overlay.
4. **Save = PDF-backed document** (leaning), in a `Chatter` folder.
5. **Deploy under `/home`,** never the rootfs.
## 8. Open items for the Phase 2 spike
- Stand up the cross-SDK and build a trivial Qt app; run it with
`-platform epaper` **after stopping xochitl**; confirm it draws and refreshes.
- Verify how input arrives in a Qt app under epaper (Qt event stream vs. raw
evdev) and how reliably finger and pen separate at the Qt layer.
- Validate the toggle hand-off (stop/start vs. stop/cont) and a return path.

79
doc/Chatter_stylus_research.md Normal file
View File

@@ -0,0 +1,79 @@
# Chatter — Stylus & Pen-Rendering Research
A record of what we learned trying to reproduce the reMarkable stock pen
behavior in Chatter, so we can resume this later. (Status 2026-06-26: the solid
ink pipeline is done and shipping; calligraphy is "close enough for now" per the
user and is **not essential** for the conversational-assistance use case.)
## 1. The render pipeline (solved)
The reMarkable epaper Qt backend is a **software scene-graph renderer**:
- Custom `QSGGeometryNode`s are silently dropped — they don't render.
- `QQuickPaintedItem` textures render but always via the e-ink **dashed
two-pass** refresh, with latency. Screen mode (Pen/Mono) and antialiasing do
not change this.
- **Solution:** bypass Qt for ink. Capture the framebuffer `QImage` by
interposing `EPFramebuffer::setBuffers` (cross-DSO, interposable; the
executable is linked `-Wl,--export-dynamic` so its symbol wins). Draw strokes
straight into that buffer (`InkEngine`) and refresh each segment with an
explicit `EPFramebuffer::swapBuffers(rect, …)` — which renders **solid,
single-pass**. Qt Quick is used only for the static UI.
- Buffer A is `960×1696 RGB32` (logical screen 954×1696, padded to 960);
`bytesPerLine 3840`. There is **no PNG image-format plugin** on the device
(only gif/ico/jpeg/svg) — save snapshots as **BMP**.
## 2. The Calligraphy pen model
reMarkable's Calligraphy is **not a fixed geometric/flat nib**. Confirmed by
experiment + reMarkable docs: it's a **dynamic** model combining:
- **Stroke direction** — downstrokes thick, upstrokes thin. (User's stock-pen
circles: thick on the descending side — right for clockwise, left for CCW. A
4-direction asterisk showed *no* variation, because quick uniform straight
strokes don't trigger it.)
- **Pressure** — heavy pressure ≈ doubles the width.
- **Speed** — faster = thinner.
- **Tilt/orientation** — simulates an angled nib.
The exact angle/algorithm is compiled into xochitl's proprietary brush engine
(assets `LS_Calligraphy_*` / `P_Calligraphy_*`, `rm-brushgfx`) — **not** in a
readable config, so exact duplication would need deep reverse-engineering.
## 3. Width calibration
- `LastPenSize` (from `xochitl.conf` `LastWritingTool`) is only a **category**:
1/2/3 = thin/thicker/thickest. Not a pixel width.
- Measured on the physical tablet: **thicker(2) ≈ 1 mm, thickest(3) ≈ 2 mm,
thinnest ≈ 2 px**. Pressure can **almost double** the width.
- Display is **264 PPI → 264/25.4 ≈ 10.4 px/mm.**
- Chatter mapping (`main.cpp`): `maxMm = max(0.2, (size-1)·1.0)` →
size3 = 2 mm, size2 = 1 mm; `setWidthRange(2 px, maxMm·pxPerMm)`.
## 4. Chatter's current approximation (`InkEngine`, pen type 21)
```
f = 0.25·dirF + 0.50·pressure + 0.12·speedF + 0.13·tiltMag // clamp 0..1
width = minWidth + (maxWidth − minWidth)·f
dirF = 0.5 + 0.5·(Δy/len) // downstroke → 1, upstroke → 0
speedF = 1 − clamp(speed/3, 0, 1) // speed = len/dt (px/ms), slow → 1
tiltMag = clamp(hypot(tiltX, tiltY), 0, 1) // from event2 ABS_TILT_X/Y (±9000)
```
Uniform (non-calligraphy) pens: `f = 0.5 + 0.5·pressure`.
`PenDevice` emits `strokeMove(pos, pressure, tiltX, tiltY, eraser)`.
## 5. Tooling (in `tools/`)
- `setbufshim.cpp` — LD_PRELOAD shim; proved `setBuffers` capture (feasibility).
- `fbdump.cpp` — LD_PRELOAD into xochitl; snapshots the framebuffer to
`/home/root/fbdump.{png,bmp}` when `/tmp/fbdump` is touched (use BMP — no PNG
plugin). Note: `setBuffers` interposition into *xochitl* via LD_PRELOAD was
flaky/unconfirmed in one attempt; the in-process `--export-dynamic` capture in
Chatter itself is reliable.
## 6. Open questions / ways to go deeper later
- Recover exact reMarkable widths-per-size and the nib/pressure curves by
snapshotting stock strokes (fix the xochitl fbdump path) and measuring, or by
parsing the v6 `.rm` per-point width/direction data.
- Add a proper speed estimate (smoothed) and tilt-direction (not just magnitude).
- Confirm `LastPenSize` values for the thin/medium categories (only size 3 = 3.0
is confirmed).

334
doc/Chatter_technical_reference.md Normal file
View File

@@ -0,0 +1,334 @@
# Chatter — Technical Reference (master document)
**Audience:** a programmer who needs to understand, build, modify, or maintain
Chatter — including how the reMarkable standard software works and how Chatter
fits alongside it.
This is the **index + complete architecture**. Several topics have their own
deep-dive documents; this file summarizes each and links to it, then fills in
everything not covered elsewhere. Read this first; follow the links for detail.
## Document map
| Document | What it covers |
|---|---|
| [`Chatter_application_proposal.md`](Chatter_application_proposal.md) | The why: the assistive-speech use case, user-experience goals. |
| [`Chatter_implementation.md`](Chatter_implementation.md) | The phased plan and its running changelog; device facts, toolchain, decisions. |
| [`Chatter_phase1_findings.md`](Chatter_phase1_findings.md) | On-device investigation results (input devices, display, doc store). |
| [`Chatter_stylus_research.md`](Chatter_stylus_research.md) | Pen rendering, the Calligraphy model, width calibration, tooling. |
| [`Chatter_user_guide.md`](Chatter_user_guide.md) | The end-user-facing instructions. |
| [`developer_mode_screen.md`](developer_mode_screen.md) | Enabling developer mode on the device. |
| `tablet_access.md` | **SENSITIVE** (root password + IPs) — gitignored, not in the repo. |
---
## 1. The device
**reMarkable Paper Pro Move — model RM03A, codename "Chiappa".**
- **SoC:** NXP i.MX93 (Arm Cortex-A55, aarch64), 2 GB RAM, 64 GB storage.
- **OS:** "Codex" Linux 5.7.121 (Yocto *scarthgap*), image `3.27.1.0`. SSH server
is **dropbear** (not OpenSSH).
- **Display:** E Ink **Gallery 3 — color** (ACeP, "acep2"), 7.3", **954×1696
logical / 960×1696 framebuffer, 264 PPI** (≈10.4 px/mm). Driven by **DRM/KMS
only** (`imx-drm`); there is **no `/dev/fb*`**.
- **Input:** pen = `/dev/input/event2` (Elan marker: `ABS_X` 0–6760, `ABS_Y`
0–11960, `ABS_PRESSURE` 0–4096, `ABS_TILT_X/Y` ±9000, `BTN_TOOL_PEN`/
`BTN_TOOL_RUBBER` for flip-to-erase). Finger = `/dev/input/event3` (10-point
capacitive, `ABS_MT_*`, grid 1248×2208). Power = `event0`, hall/folio = `event1`.
Pen and finger are **separate evdev devices** — Chatter tells them apart natively.
Full investigation: [`Chatter_phase1_findings.md`](Chatter_phase1_findings.md).
### 1.1 Filesystem / persistence model (critical)
- **`/` (rootfs, `/dev/mmcblk0p3`)** — ext4, mounted **read-only**, but
**persistent**. Remount rw (`mount -o remount,rw /`) to write; survives reboots,
**lost on an OS update** (A/B partition swap).
- **`/etc`, `/run`, `/var/volatile`** — **VOLATILE overlays** (upperdir on tmpfs).
Anything written here is **lost on reboot**. (This is why a systemd unit dropped
in `/etc/systemd/system` vanished after a reboot.)
- **`/home`** — encrypted, **fully persistent**, ~45 GB free. Everything Chatter
installs lives here (`/home/root/chatter/`). SSH keys persist because they are
under `/home/root/.ssh`.
**Consequence:** the launcher's systemd unit is installed onto the **rootfs**
(`/usr/lib/systemd/system`) so it survives reboots; the binary and scripts live
under `/home`. **An OS update wipes rootfs changes — disable OS auto-updates on a
delivered device, and keep `scripts/install-launcher.sh` to reinstall.**
---
## 2. The standard reMarkable software stack
Understanding the stock stack is necessary because Chatter reuses its display
plumbing and hands off to it.
- **`xochitl`** — the stock note app. Qt **6.8.2** / Qt Quick, proprietary. Owns
the display while running. Started/stopped as the `xochitl` systemd service.
- **The `epaper` QPA platform plugin** (`libqsgepaper.so`, in
`/usr/lib/plugins/`). A Qt platform + **software** scene-graph renderer for the
e-paper. A Qt app runs on it with `-platform epaper` (+ `QT_QUICK_BACKEND=epaper`).
Key facts learned by reverse engineering:
- It is a **software** renderer: custom `QSGGeometryNode`s are silently dropped;
only textures/rects/glyphs render. (This is why Chatter cannot draw ink via
the scene graph — see §3.)
- **`EPFramebuffer`** is the panel singleton (actually `EPFramebufferAcep2` on
this color device), guarded by an **flock** at `/tmp/epframebuffer.lock` —
only one process may hold the panel. Success marker in the journal:
`SWTCON initialized \o/`. A stranded holder → `Failed to lock epframebuffer` /
`Failed to initialize SWTCON` and a blank panel. **Always stop the previous
holder first.**
- Relevant `EPFramebuffer` methods (mangled symbols bound in `src/epfb.h`):
- `instance()` → the singleton.
- `setBuffers(std::tuple<QImage,QImage>, QImage*)` — sets the front/back
buffers; **cross-DSO and interposable** (see §3).
- `swapBuffers(QRect, EPContentType, EPScreenMode, QFlags<UpdateFlag>)` —
pushes a region to the panel. A single explicit call renders **solid**
(no dashed two-pass). Screen modes (from `EPScreenModeItem::Mode`):
`Pen=0, Mono=1, Animation=2, UI=3, Content=4, Sleep=5`. Chatter uses
`Pen` (0) for fast ink and `Content` (4, "full update, STD") for full
refreshes.
- `ghostControl(GhostControlMode)` — the panel's anti-ghosting API. Modes 0/3
do an immediate full-screen de-ghost via the *region* `swapBuffers`; mode 1
schedules one. **Chatter does NOT call it** — its region-swap path uses
internal `EPContentMap`/`EPScreenModeMap` members the stock app maintains and
we do not, so calling it corrupts state and hangs after a few calls. Chatter
de-ghosts manually instead (§6.3).
- The panel uses **ACeP color waveforms** (`acep2_lut`, `get_waveform_data`,
software TCON "SWTCON"). Color **ghosting** is real and only cleared by a
full-update waveform driven to the **dark** extreme (§6.3).
- **Pen styles** live in `~/.config/remarkable/xochitl.conf`, key
**`LastWritingTool`** (a `@Variant` `QVariantMap`, readable via `QSettings`):
`LastPen` (tool id, e.g. 21 = Calligraphy, 16 = a fineliner), `LastPenSize`
(category 1/2/3 = thin/thicker/thickest), `LastPenColorCode` (`0xAARRGGBB`).
**xochitl writes this to disk only when you leave a document** (return to the
document list) — not on toolbar taps, and not reliably on an abrupt stop.
---
## 3. Chatter architecture (the core idea)
**Problem:** the make-or-break requirement is that ink look like the stock pen —
solid, crisp, low-latency. The `epaper` software scene graph **cannot** do this:
geometry nodes don't render, and `QQuickPaintedItem` textures always come out as
the e-ink **dashed two-pass** refresh, regardless of screen mode or antialiasing.
**Solution — a direct-framebuffer ink pipeline that bypasses the Qt scene:**
1. **`FbCapture`** (`src/FbCapture.{h,cpp}`) interposes
`EPFramebuffer::setBuffers` **in-process**. The executable is linked
`-Wl,--export-dynamic`, so its definition of that symbol wins the cross-DSO
call from the plugin; we capture the real framebuffer's pixel memory and wrap
it as a `QImage` (`FbCapture::framebuffer()`) with **no copy** (`constBits`).
(Intra-DSO calls like `swapBuffers` are *not* LD_PRELOAD-interposable due to
direct binding — but in-process `--export-dynamic` on `setBuffers` works.)
2. **`InkEngine`** (`src/InkEngine.{h,cpp}`) draws strokes with `QPainter`
**straight into that framebuffer image**, then calls
`EPFramebuffer::swapBuffers` (via `src/epfb.h`) on just the dirty rectangle.
One explicit swap → **solid, single-pass** ink.
3. **Qt Quick renders only the static UI** (the two buttons). The scene never
recomposites over the ink because nothing in it animates.
This is what gives stock-quality strokes. The reverse-engineering trail and the
dead-ends (geometry nodes, screen modes, antialiasing) are in
[`Chatter_stylus_research.md`](Chatter_stylus_research.md) and the
`Chatter_implementation.md` changelog (v0.7).
### 3.1 Process / run model
- Chatter is a single Qt 6 executable run as a **transient systemd service**
(`systemd-run --unit=chatter …`), with `QT_QUICK_BACKEND=epaper`,
`LD_LIBRARY_PATH=/usr/lib/plugins/scenegraph`, `-platform epaper`.
- It **requires xochitl to be stopped** (single DRM master + the panel flock).
- Managed via `systemctl {stop,status} chatter` and `journalctl -u chatter`.
---
## 4. Source components
```
src/
main.cpp Entry point: reads pen style, wires PenDevice → InkEngine,
exposes `ink`/`appControl` to QML, loads the QML UI.
FbCapture.{h,cpp} Interposes EPFramebuffer::setBuffers; exposes the live
framebuffer QImage + FbCapture::ready().
InkEngine.{h,cpp} THE core. Virtual canvas, ink drawing, finger-erase, buttons
(paint/hit-test/actions), two-finger scroll, de-ghosting.
PenDevice.{h,cpp} evdev reader for the pen (event2); maps to screen coords;
emits strokeStart / strokeMove(pos,pressure,tiltX,tiltY,eraser)
/ strokeEnd.
AppControl.{h,cpp} returnToStandard(): transient unit stops chatter, starts xochitl.
epfb.h asm-label bindings to the private EPFramebuffer symbols
(instance, swapBuffers, ghostControl).
EPScreenModeItem.{h,cpp} Legacy/unused: binding to the private screen-mode item
from the scene-graph era. Kept for reference.
InkCanvas.*, Experiment.* Legacy from the Qt-Quick-canvas spike; NOT built.
qml/Main.qml White Window; the Back/Clear buttons (TopButton); a
MultiPointTouchArea for finger erase (1) and scroll (2).
tools/
chatter_launcher.c The return-to-Chatter daemon (4-finger watcher on event3).
grabtest.c Probe whether an input device is EVIOCGRAB-exclusive.
fbdump.cpp LD_PRELOAD framebuffer snapshot (BMP; device has no PNG plugin).
swapshim.cpp, setbufshim.cpp Feasibility shims used during RE.
scripts/
build.sh Source the SDK env, cmake build.
deploy-and-run.sh Stop xochitl+chatter, scp, relaunch as a transient unit.
to-chatter.sh Switch standard → Chatter (run by the launcher).
install-launcher.sh Persistently install the launcher unit on the rootfs.
chatter-launcher.service The systemd unit (installed to rootfs).
restore-xochitl.sh Return to the standard GUI.
```
---
## 5. Input & gesture model
Two independent input streams, never confused:
- **Pen (`event2`)** — read by `PenDevice` and delivered to `InkEngine`. The
`epaper` QPA does **not** deliver the pen to Qt, so the pen never triggers QML.
- **Finger (`event3`)** — delivered by the `epaper` QPA to Qt as touch, handled in
QML (`MultiPointTouchArea`, and the buttons' `MouseArea`s).
Gesture map:
| Input | Action | Where handled |
|---|---|---|
| Stylus draw | Ink (flip = erase via `BTN_TOOL_RUBBER`) | PenDevice → InkEngine |
| Stylus tap on a button | Button action | InkEngine hit-tests (`m_buttons`) |
| **1 finger** drag | **Erase wipe** (~12 mm; a tap erases nothing) | QML MultiPointTouchArea → `InkEngine::erase*` |
| **2 fingers** drag | **Vertical scroll** | QML → `InkEngine::panBy/panEnd` |
| Finger tap on a button | Button action (gray feedback) | QML MouseArea → `InkEngine::flashButton/activateButton` |
| **4 fingers** hold (~700 ms) | **Return to Chatter** (only while xochitl is front) | `tools/chatter_launcher.c` |
The MultiPointTouchArea **latches** the gesture type until all fingers lift, so a
two-finger scroll never degrades into an erase when one finger is raised. Erase
requires movement past a ~1.5 mm threshold (tap-safe).
Buttons (`Back`, `Clear`) are a **single source of truth** in `InkEngine`: QML
registers their geometry (`registerButton`) so the engine can exclude ink, redraw
them after a blit, and hit-test stylus taps. Press feedback (`flashButton`) and
actions (`activateButton`) are drawn directly to the framebuffer (instant, no
scene flashing); both finger and stylus route through them.
---
## 6. The virtual canvas, scrolling, and de-ghosting
### 6.1 Growable raster canvas
`InkEngine` holds a `QImage` **canvas** larger than the screen (starts 2× tall,
**grows downward** as you write near the bottom). The screen is a **viewport**
into it at vertical offset `m_panY`. Drawing maps screen→canvas (`+m_panY`); a
dirty canvas rect is blitted back to the framebuffer (`blitRegion`) and the
buttons are repainted on top. (Storage model chosen: raster, to preserve the
exact ink quality; trade-off is no crisp zoom-in. Horizontal/zoom are future work.)
### 6.2 Scrolling
Two-finger drag → `panBy(dy)` (natural: content follows fingers), clamped to the
canvas, fast-blitted per step. `panEnd()` debounces a de-ghost (§6.3).
### 6.3 De-ghosting (color ACeP ghosting)
Fast `Pen`-waveform swaps leave **color residue** ("faint red duplicate") that
accumulates while scrolling and is **not** cleared by a white redraw — it is
panel retention, cleared only by a full-update waveform driven to **black**
(white/gray do not clear it; this was tested). Chatter's `fullRefresh()`:
fills the screen **black** + full-update swap (`screenMode=Content`), waits
~220 ms, then full-updates the real content. `ghostControl()` would be the
"proper" API but corrupts state (§2), so this manual flash is used.
To keep the flash from being intrusive:
- It runs **only after scrolling**, **debounced** ~1 s after the last scroll
(not on every finger-lift).
- **Clear has two methods:** if there was **no scrolling** since the last clear
(the common fill-one-screen case) it uses the **gentle fast clear**; if there
**was** scrolling, it does the **black de-ghost** clear. (`m_scrolled` flag.)
- Tunables via env: `CHATTER_FULL_SM` (full-update screen mode, default 4),
`CHATTER_FLASH_GRAY` (flash level 0=black..255; black is what actually clears).
A black flash is intrinsic to clearing color ghosting (the stock UI flashes on
its full refreshes too); we minimized *when* it happens rather than eliminating it.
---
## 7. Pen-style matching, width calibration, calligraphy
- Chatter reads `xochitl.conf` `LastWritingTool` at startup (`readPenStyle` in
`main.cpp`) and applies tool type, size, and color — **no style menu in
Chatter**. Because each switch-to-Chatter restarts the process, it re-reads the
current pen. **The flush sequence matters** (§2): set the pen, *use it*, **leave
the document**, then switch to Chatter.
- **Width** is calibrated to measured widths on the 264-PPI panel:
size 3 ≈ 2 mm, size 2 ≈ 1 mm, thinnest ≈ 2 px; pressure ≈ doubles width.
- **Calligraphy** (tool 21) is approximated with a dynamic
direction/pressure/speed/tilt width model. Full detail, measurements, and the
formula: [`Chatter_stylus_research.md`](Chatter_stylus_research.md).
---
## 8. Toggle & the return launcher
- **Chatter → standard:** `Back` → `InkEngine::backRequested` →
`AppControl::returnToStandard()` → a transient unit stops chatter and starts
xochitl (sequenced so the panel lock is released first).
- **standard → Chatter:** `tools/chatter_launcher.c` runs always as a rootfs
systemd service. It reads `event3` **without grabbing it** (verified possible
via `tools/grabtest.c` — xochitl does not hold an exclusive grab), counts
multitouch slots, and on a **4-finger hold ≥700 ms** while xochitl is the front
app runs `to-chatter.sh`. It reopens the device on any read interruption (sleep/
wake) so it never dies.
- **Install persistently:** `scripts/install-launcher.sh` (remounts the rootfs rw,
places the unit + its `multi-user.target.wants` symlink under
`/usr/lib/systemd/system`). See the persistence model in §1.1.
---
## 9. Build & deploy
**Toolchain:** official reMarkable Chiappa SDK `3.27.0.97` (Qt 6.8.2 sysroot,
x86_64 host). Install, then `source environment-setup-cortexa55-remarkable-linux`.
```
scripts/build.sh # source SDK env + cmake build -> build/chatter
scripts/deploy-and-run.sh # stop xochitl+chatter, scp, relaunch as transient unit
scripts/restore-xochitl.sh # back to the standard GUI
scripts/install-launcher.sh # one-time (and after each OS update): persist the launcher
```
`CMakeLists.txt`: Qt6 Quick app (`qt_add_executable` + `qt_add_qml_module`), links
`libqsgepaper.so`, and crucially `target_link_options(... -Wl,--export-dynamic)`
so the `setBuffers` interposition wins.
**Device housekeeping:** only one process may hold the panel — always stop the
previous holder. Deploy under `/home/root/chatter`, never the rootfs (except the
launcher unit). The device sleeps and DHCP may reassign its IP on wake; if you
script a reconnect, rescan for the tablet's MAC address (shown on the device's
GPLv3-compliance screen, alongside its IPs).
---
## 10. Status & open items
**Working on hardware:** solid stock-quality ink matching the selected pen;
finger-wipe erase; whole-page Clear (two methods); stylus eraser; bidirectional
toggle (Back + 4-finger launcher, persistent); vertical scroll on a growable
canvas with debounced de-ghosting; instant button feedback (finger + stylus).
**Open / future:**
- **Save** (Phase 5) — write a timestamped transcript into a "Chatter" folder in
xochitl's on-disk document format; importance to the user still unconfirmed.
See `Chatter_implementation.md` §6 / §6a.
- **Horizontal scroll & zoom-out overview** — deferred extensions of the canvas.
- **Calligraphy fidelity** — "close enough" approximation; exact nib unknown.
- **Erase / gesture thresholds** — to be tuned with the end user (Matt's field test).
- **OS auto-update** would wipe the rootfs launcher unit (and could break paths) —
disable it on a delivered device.
---
*This document is the technical entry point. When the design changes, update this
file and the relevant component doc; keep `Chatter_implementation.md`'s changelog
as the chronological record.*

58
doc/Chatter_user_guide.md Normal file
View File

@@ -0,0 +1,58 @@
# Chatter — How to Use It
Chatter is a simple writing screen. You write with the pen; the person you are
talking with reads what you wrote. That's it.
## Writing
- **Write with the pen**, just like on paper.
- The pen's color and thickness are whatever is set in the normal reMarkable
screen. (To change them, see "Changing the pen" below.)
## The two buttons (top of the screen)
```
┌──────┐ ┌───────┐
│ Back │ │ Clear │
└──────┘ └───────┘
```
- **Clear** (top right) — erases everything and gives you a fresh page.
- **Back** (top left) — leaves Chatter and goes to the normal reMarkable screen.
A button turns **gray** the moment you touch it, so you know it heard you. You
can press a button with **either the pen or your finger**.
## Erasing
- **Erase a little:** rub a **finger** back and forth over what you want to
remove, like erasing pencil with your fingertip.
- **Erase everything:** tap **Clear**.
- You can also flip the pen over and use the **eraser end**, like a pencil.
## More room to write
- **Two fingers, drag up or down** to scroll, when you want more space than one
screen. The page follows your fingers.
- (After scrolling, the screen may blink once to clean itself up. That's normal.)
## Going back and forth
- **Leave Chatter:** tap **Back**.
- **Return to Chatter** from the normal reMarkable screen: place **four fingers
on the screen and hold** for about a second.
## Changing the pen (color / thickness)
Chatter copies the pen you last used in the normal reMarkable screen. To change
it:
1. Tap **Back** to go to the normal screen.
2. Open a page, pick the pen, color, and thickness you want, and **write a little
with it**.
3. **Go back to the document list** (this is what saves your choice).
4. Return to Chatter (four-finger hold). Your new pen is now used.
---
*Questions or problems during setup go to Matt.*

17
doc/developer_mode_screen.md Normal file
View File

@@ -0,0 +1,17 @@
Developer mode
This mode is designed for experienced developers to make local software changes via SH access to the system.
Please note that any changes to software puts the device security at increased risk.
- Access to root shell is offered in compliance with GPL V3 license.
- Running custom or modified software may cause the device to stop working or adversely affect performance.
- A factory reset is required to enter.
- This will delete all files stored on your paper tablet.
- User data will be cleared once the device is rebooted.
- The device will no longer verify the authenticity of the software, putting your system stability and data security at risk.
- Access any content stored in the cloud by re-pairing with your reMarkable account.
For instructions on how to exit developer mode, visit support.remarkable.com
(Accept)

BIN
doc/initial_Claude_discussion.pdf Normal file
View File

Binary file not shown.

106
qml/Main.qml Normal file
View File

@@ -0,0 +1,106 @@
// Chatter — UI only. The white "page" plus the top control bar. Ink is drawn
// directly into the framebuffer by the C++ InkEngine, not by the scene graph,
// so there is no canvas item here. Keep the scene STATIC (only the buttons ever
// repaint, on tap) so it never recomposites over the direct-drawn ink.
import QtQuick
import QtQuick.Window
Window {
id: root
width: Screen.width
height: Screen.height
visible: true
color: "white"
// Finger gestures (the stylus is handled in C++ and is NOT delivered here):
// 1 finger = erase wipe (a tap erases nothing — only movement does)
// 2 fingers = vertical scroll (content follows the fingers)
// The gesture type is latched until all fingers lift, so a 2-finger scroll
// never turns into an erase when one finger is raised. Buttons (z:10) are above.
MultiPointTouchArea {
id: touch
anchors.fill: parent
z: 0
minimumTouchPoints: 1
maximumTouchPoints: 2
touchPoints: [
TouchPoint { id: tpA },
TouchPoint { id: tpB }
]
property int mode: 0 // 0 idle, 1 erase, 2 scroll
property real lastPanY: 0
function activeCount() { return (tpA.pressed ? 1 : 0) + (tpB.pressed ? 1 : 0); }
function soloPoint() { return tpA.pressed ? tpA : tpB; }
function handle() {
var n = activeCount();
if (n >= 2) {
var cy = (tpA.y + tpB.y) / 2;
if (mode !== 2) {
if (mode === 1) ink.eraseEnd();
mode = 2;
lastPanY = cy;
} else {
ink.panBy(cy - lastPanY);
lastPanY = cy;
}
} else if (n === 1) {
if (mode === 0) { mode = 1; ink.eraseStart(soloPoint().x, soloPoint().y); }
else if (mode === 1) { ink.eraseMove(soloPoint().x, soloPoint().y); }
// mode === 2 (one finger left after a scroll): stay idle until all up
} else {
if (mode === 1) ink.eraseEnd();
else if (mode === 2) ink.panEnd(); // full refresh to clear scroll ghosts
mode = 0;
}
}
onPressed: handle()
onUpdated: handle()
onReleased: handle()
}
// Buttons keep a FIXED appearance in the Qt scene (rendered once); their
// pressed/normal feedback and actions are handled by the ink engine
// (ink.flashButton / ink.activateButton) so both finger AND stylus work and
// updates are instant single-pass (no scene flashing).
component TopButton: Rectangle {
id: btn
height: 90
// Size to the label so horizontal padding ≈ vertical padding.
width: label.implicitWidth + (height - label.implicitHeight)
color: "white"
border.color: "black"
border.width: 3
radius: 8
z: 10
property alias text: label.text
Text { id: label; anchors.centerIn: parent; anchors.verticalCenterOffset: -2; font.pixelSize: 38 }
// Register geometry once laid out (ink-exclusion, clear-redraw, stylus hit-test).
Component.onCompleted: Qt.callLater(function() {
ink.registerButton(btn.x, btn.y, btn.width, btn.height, btn.text);
})
MouseArea {
anchors.fill: parent
onPressed: ink.flashButton(btn.x, btn.y, btn.width, btn.height, btn.text, true)
onClicked: ink.activateButton(btn.text)
}
}
TopButton {
id: backBtn
text: "Back"
anchors.left: parent.left
anchors.top: parent.top
anchors.margins: 24
}
TopButton {
id: clearBtn
text: "Clear"
anchors.right: parent.right
anchors.top: parent.top
anchors.margins: 24
}
}

28
scripts/build.sh Executable file
View File

@@ -0,0 +1,28 @@
#!/usr/bin/env bash
# Cross-build Chatter with the reMarkable Paper Pro Move (Chiappa) SDK.
#
# Usage: scripts/build.sh [extra cmake args]
# Override SDK location with CHATTER_SDK=...
set -euo pipefail
SDK="${CHATTER_SDK:-/home/ack/external/remarkable-sdk/chiappa-3.27.0.97}"
ENV="$SDK/environment-setup-cortexa55-remarkable-linux"
[ -f "$ENV" ] || { echo "SDK env-setup not found: $ENV"; exit 1; }
cd "$(dirname "$0")/.."
# Yocto SDKs refuse to operate with LD_LIBRARY_PATH set (Jatke sets it globally).
unset LD_LIBRARY_PATH
# shellcheck disable=SC1090
. "$ENV"
cmake -S . -B build "$@"
cmake --build build -j"$(nproc)"
echo "built: build/chatter"
# The return-to-Chatter launcher is a tiny standalone C daemon (not part of the
# CMake target). Build it here too so install-launcher.sh has its binary.
# $CC carries flags (mcpu, sysroot, …) so it must stay UNquoted.
# shellcheck disable=SC2086
$CC -O2 -o tools/chatter-launcher tools/chatter_launcher.c
echo "built: tools/chatter-launcher"

13
scripts/chatter-launcher.service Normal file
View File

@@ -0,0 +1,13 @@
[Unit]
Description=Chatter launcher (multi-finger gesture watcher)
After=home.mount data.mount multi-user.target
# Binary + scripts live on the persistent /home volume.
RequiresMountsFor=/home/root/chatter
[Service]
ExecStart=/home/root/chatter/chatter-launcher
Restart=always
RestartSec=2
[Install]
WantedBy=multi-user.target

51
scripts/deploy-and-run.sh Executable file
View File

@@ -0,0 +1,51 @@
#!/usr/bin/env bash
# Deploy the chatter binary to the tablet and run it on the e-paper display.
#
# Usage: scripts/deploy-and-run.sh [path-to-binary]
# default binary: build/chatter
#
# Requires: `ssh chatter` working (see doc/tablet_access.md).
#
# IMPORTANT lessons baked in here:
# * The e-paper framebuffer is a singleton guarded by an flock
# (/tmp/epframebuffer.lock). Only ONE process may hold it. A stranded
# instance blanks the panel for everyone, so we always stop the previous
# chatter unit first.
# * Launch as a transient systemd service (systemd-run), NOT an ssh background
# job: that detaches cleanly (no held SSH channel) and is stoppable via
# `systemctl stop chatter`.
# * xochitl owns the display while running, so stop it first (pre-authorized
# during development). Run scripts/restore-xochitl.sh to bring it back.
set -euo pipefail
cd "$(dirname "$0")/.."
HOST="chatter"
DEST="/home/root/chatter" # under /home, NOT the nearly-full rootfs
# Use an explicit binary if given; else a fresh local build; else the prebuilt.
BIN="${1:-}"
if [ -z "$BIN" ]; then
if [ -f build/chatter ]; then BIN=build/chatter; else BIN=dist/chatter; fi
fi
[ -f "$BIN" ] || { echo "binary not found: $BIN (build it, or use the prebuilt dist/chatter)"; exit 1; }
echo ">> stopping any previous chatter + xochitl (release the panel lock)"
ssh "$HOST" 'systemctl stop chatter 2>/dev/null; systemctl reset-failed chatter 2>/dev/null; systemctl stop xochitl 2>/dev/null; sleep 1'
echo ">> deploying $(basename "$BIN") to $HOST:$DEST"
ssh "$HOST" "mkdir -p $DEST"
scp "$BIN" "$HOST:$DEST/"
echo ">> launching chatter as a transient systemd service"
# LD_LIBRARY_PATH lets the loader resolve libqsgepaper.so (linked for the private
# EPScreenModeItem type), which lives in the scenegraph plugin dir.
ssh "$HOST" "systemd-run --unit=chatter --collect \
--setenv=QT_QUICK_BACKEND=epaper \
--setenv=LD_LIBRARY_PATH=/usr/lib/plugins/scenegraph \
--working-directory=$DEST \
$DEST/$(basename "$BIN") -platform epaper"
sleep 4
ssh "$HOST" 'echo "chatter: $(systemctl is-active chatter)"'
echo ">> logs: ssh chatter 'journalctl -u chatter -f'"
echo ">> stop: ssh chatter 'systemctl stop chatter' (then scripts/restore-xochitl.sh)"

34
scripts/install-launcher.sh Normal file
View File

@@ -0,0 +1,34 @@
#!/usr/bin/env bash
# Persistently install the chatter-launcher systemd service ON THE DEVICE.
#
# Why this is needed: on the reMarkable Paper Pro Move, /etc and /run are
# VOLATILE overlays (upperdir on tmpfs) — units placed there are lost on reboot.
# The root partition is read-only but persistent, so we remount it rw and place
# the unit (and its enable symlink) under /usr/lib/systemd/system, which survives
# reboots. (An OS update replaces the rootfs and would require re-running this.)
#
# Uses the prebuilt dist/chatter-launcher if present, else a locally built one.
set -euo pipefail
cd "$(dirname "$0")/.."
HOST="${1:-chatter}"
LAUNCHER=dist/chatter-launcher
[ -f "$LAUNCHER" ] || LAUNCHER=tools/chatter-launcher
[ -f "$LAUNCHER" ] || { echo "launcher binary not found (dist/ or tools/) — run scripts/build.sh"; exit 1; }
scp -q "$LAUNCHER" "$HOST:/home/root/chatter/chatter-launcher"
scp -q scripts/to-chatter.sh scripts/chatter-launcher.service "$HOST:/home/root/chatter/"
ssh "$HOST" '
set -e
chmod +x /home/root/chatter/chatter-launcher /home/root/chatter/to-chatter.sh
mount -o remount,rw /
cp /home/root/chatter/chatter-launcher.service /usr/lib/systemd/system/chatter-launcher.service
mkdir -p /usr/lib/systemd/system/multi-user.target.wants
ln -sf ../chatter-launcher.service /usr/lib/systemd/system/multi-user.target.wants/chatter-launcher.service
sync
mount -o remount,ro /
systemctl daemon-reload
systemctl restart chatter-launcher.service
echo "launcher: $(systemctl is-active chatter-launcher)"
'

5
scripts/restore-xochitl.sh Executable file
View File

@@ -0,0 +1,5 @@
#!/usr/bin/env bash
# Bring the standard reMarkable GUI back after running Chatter.
set -euo pipefail
ssh chatter 'systemctl stop chatter 2>/dev/null; systemctl reset-failed chatter 2>/dev/null; systemctl start xochitl'
echo "chatter stopped; xochitl restarted — standard GUI restored."

12
scripts/to-chatter.sh Normal file
View File

@@ -0,0 +1,12 @@
#!/bin/sh
# Switch from the standard interface to Chatter (invoked by the launcher daemon
# on the multi-finger gesture). Lives on the device at /home/root/chatter/.
systemctl stop xochitl 2>/dev/null
systemctl stop chatter 2>/dev/null
systemctl reset-failed chatter 2>/dev/null
sleep 1
systemd-run --unit=chatter --collect \
--setenv=QT_QUICK_BACKEND=epaper \
--setenv=LD_LIBRARY_PATH=/usr/lib/plugins/scenegraph \
--working-directory=/home/root/chatter \
/home/root/chatter/chatter -platform epaper

14
src/AppControl.cpp Normal file
View File

@@ -0,0 +1,14 @@
#include "AppControl.h"
#include <QProcess>
void AppControl::returnToStandard()
{
// Launch a transient unit so the work isn't killed when chatter stops.
QProcess::startDetached(
QStringLiteral("systemd-run"),
{QStringLiteral("--collect"),
QStringLiteral("--unit=chatter-to-standard"),
QStringLiteral("/bin/sh"), QStringLiteral("-c"),
QStringLiteral("systemctl stop chatter; systemctl start xochitl")});
}

15
src/AppControl.h Normal file
View File

@@ -0,0 +1,15 @@
#pragma once
#include <QObject>
// Small controller exposed to QML for system actions (e.g. the "Back" button
// returning to the standard reMarkable GUI).
class AppControl : public QObject {
Q_OBJECT
public:
explicit AppControl(QObject *parent = nullptr) : QObject(parent) {}
// Hand the display back to xochitl: detached so it survives this process
// being stopped, and sequenced (stop chatter -> release panel -> start
// xochitl) to avoid framebuffer-lock contention.
Q_INVOKABLE void returnToStandard();
};

34
src/EPScreenModeItem.h Normal file
View File

@@ -0,0 +1,34 @@
#pragma once
#include <QQuickItem>
// EPScreenModeItem is a PRIVATE reMarkable type that lives in libqsgepaper.so
// (the epaper scenegraph backend). It is a QQuickItem that tags the screen region
// it covers with a panel refresh "mode" (Mono / FAST / FastGrayscale ...), which
// selects the e-ink waveform. xochitl wraps its writing area in one of these to
// get crisp, solid pen strokes instead of the default grayscale two-pass refresh.
//
// The SDK ships no header for it, so we declare a minimal interface matching the
// exported symbols (the constructor and the Q_OBJECT staticMetaObject) and link
// against the plugin. We drive its `mode` property via the runtime meta-object,
// so we never hardcode the Mode enum's integer values.
class EPScreenModeItem : public QQuickItem {
public:
// Values confirmed at runtime from the real meta-object.
enum Mode { Pen = 0, Mono = 1, Animation = 2, UI = 3, Content = 4, Sleep = 5 };
explicit EPScreenModeItem(QQuickItem *parent = nullptr);
// Exported by libqsgepaper (_ZN16EPScreenModeItem7setModeENS_4ModeE). The
// `mode` property is read-only, so this is how you actually change it.
void setMode(Mode m);
Mode mode() const; // _ZNK16EPScreenModeItem4modeEv
// Declared (defined in libqsgepaper.so, symbol exported) so this override is
// the class "key function". That stops our TU from emitting a competing weak
// vtable, so the loader uses libqsgepaper's real vtable — giving the real
// metaObject (with the Mode enum) and real setMode().
const QMetaObject *metaObject() const override;
// Resolved at link time from libqsgepaper.so (_ZN16EPScreenModeItem16staticMetaObjectE).
static const QMetaObject staticMetaObject;
};

45
src/Experiment.cpp Normal file
View File

@@ -0,0 +1,45 @@
#include "Experiment.h"
#include "InkCanvas.h"
#include "epfb.h"
#include <QTimer>
#include <QRect>
Experiment::Experiment(QObject *parent) : QObject(parent)
{
// Sweep screenMode Pen(0) and Mono(1) across contentType 0..5, flags 0.
for (int sm : {0, 1})
for (int ct = 0; ct < 6; ++ct)
m_combos.append({sm, ct, 0});
}
void Experiment::start()
{
auto *t = new QTimer(this);
connect(t, &QTimer::timeout, this, &Experiment::tick);
t->start(3000);
QTimer::singleShot(600, this, &Experiment::tick);
}
void Experiment::tick()
{
if (!m_canvas || m_combos.isEmpty())
return;
m_idx = (m_idx + 1) % m_combos.size();
const Combo c = m_combos[m_idx];
m_canvas->drawTestStroke();
m_status = QStringLiteral("[%1/%2] screen=%3 content=%4 flags=%5")
.arg(m_idx + 1).arg(m_combos.size()).arg(c.sm).arg(c.ct).arg(c.flags);
emit statusChanged();
qInfo("EXPERIMENT %s", qPrintable(m_status));
// After the scene composites the test pattern, force a refresh of just the
// test region with these parameters.
QTimer::singleShot(350, this, [this, c]() {
const QRect r = m_canvas->testRectQ();
void *fb = epfb_instance();
if (fb)
epfb_swapBuffers(fb, {r.left(), r.top(), r.right(), r.bottom()}, c.ct, c.sm, c.flags);
});
}

34
src/Experiment.h Normal file
View File

@@ -0,0 +1,34 @@
#pragma once
#include <QObject>
#include <QString>
#include <QVector>
class InkCanvas;
// Screen-mode experiment harness. On a timer it cycles through (screenMode,
// contentType, flags) combinations: for each, it redraws a fixed test pattern
// and then calls EPFramebuffer::swapBuffers on the test region with those
// parameters. The current combination is shown on screen (status), so we can
// see which combination renders the strokes SOLID rather than dashed.
class Experiment : public QObject {
Q_OBJECT
Q_PROPERTY(QString status READ status NOTIFY statusChanged)
public:
explicit Experiment(QObject *parent = nullptr);
QString status() const { return m_status; }
void setCanvas(InkCanvas *c) { m_canvas = c; }
void start();
signals:
void statusChanged();
private:
void tick();
InkCanvas *m_canvas = nullptr;
QString m_status;
int m_idx = -1;
struct Combo { int sm; int ct; int flags; };
QVector<Combo> m_combos;
};

49
src/FbCapture.cpp Normal file
View File

@@ -0,0 +1,49 @@
#define _GNU_SOURCE 1
#include "FbCapture.h"
#include <tuple>
#include <dlfcn.h>
#include <QtGlobal>
namespace {
uchar *g_bits = nullptr;
int g_w = 0, g_h = 0, g_bpl = 0, g_fmt = 0;
}
// Interposes EPFramebuffer::setBuffers(std::tuple<QImage,QImage>, QImage*).
// The tuple is non-trivially-copyable, so passed by reference (a pointer); the
// QImage* is a pointer. We capture buffer A's pixel pointer/geometry, then chain
// to the real implementation.
extern "C" void _ZN13EPFramebuffer10setBuffersESt5tupleIJ6QImageS1_EEPS1_(
void *self, void *tuplePtr, void *imgPtr)
{
static void (*real)(void *, void *, void *) = nullptr;
if (!real)
real = (void (*)(void *, void *, void *))dlsym(
RTLD_NEXT, "_ZN13EPFramebuffer10setBuffersESt5tupleIJ6QImageS1_EEPS1_");
auto *t = reinterpret_cast<std::tuple<QImage, QImage> *>(tuplePtr);
const QImage &a = std::get<0>(*t);
g_bits = const_cast<uchar *>(a.constBits());
g_w = a.width();
g_h = a.height();
g_bpl = a.bytesPerLine();
g_fmt = int(a.format());
qInfo("FbCapture: framebuffer %dx%d fmt=%d bpl=%d bits=%p",
g_w, g_h, g_fmt, g_bpl, (void *)g_bits);
if (real) real(self, tuplePtr, imgPtr);
}
namespace FbCapture {
bool ready() { return g_bits != nullptr; }
QImage framebuffer()
{
if (!g_bits)
return QImage();
return QImage(g_bits, g_w, g_h, g_bpl, QImage::Format(g_fmt));
}
} // namespace FbCapture

12
src/FbCapture.h Normal file
View File

@@ -0,0 +1,12 @@
#pragma once
#include <QImage>
// Captures the e-paper framebuffer's backing image by interposing
// EPFramebuffer::setBuffers (defined in FbCapture.cpp). The executable is linked
// with -Wl,--export-dynamic so its symbol wins the cross-DSO call from the
// epaper platform plugin into libqsgepaper. The ink engine then draws strokes
// straight into the framebuffer memory and refreshes with an explicit swap.
namespace FbCapture {
bool ready();
QImage framebuffer(); // a QImage wrapping the real buffer (no copy) — draw into it
}

157
src/InkCanvas.cpp Normal file
View File

@@ -0,0 +1,157 @@
#include "InkCanvas.h"
#include "epfb.h"
#include <QPainter>
#include <QPolygon>
#include <QTimer>
InkCanvas::InkCanvas(QQuickItem *parent) : QQuickPaintedItem(parent)
{
setRenderTarget(QQuickPaintedItem::Image);
// Crisp 1-bit black so the Mono waveform renders single-pass (no gray dither).
setAntialiasing(false);
}
void InkCanvas::ensureBuffer(const QSize &size)
{
if (size.isEmpty() || m_buffer.size() == size)
return;
QImage img(size, QImage::Format_RGB32);
img.fill(Qt::white);
if (!m_buffer.isNull()) {
QPainter p(&img);
p.drawImage(0, 0, m_buffer);
}
m_buffer = img;
}
void InkCanvas::geometryChange(const QRectF &newGeometry, const QRectF &oldGeometry)
{
QQuickPaintedItem::geometryChange(newGeometry, oldGeometry);
ensureBuffer(newGeometry.size().toSize());
update();
}
void InkCanvas::paint(QPainter *painter)
{
if (!m_buffer.isNull())
painter->drawImage(0, 0, m_buffer);
}
void InkCanvas::strokeStart(QPointF p)
{
ensureBuffer(boundingRect().size().toSize());
m_last = p;
m_drawing = true;
}
void InkCanvas::strokeMove(QPointF p, qreal pressure, bool eraser)
{
if (m_buffer.isNull())
ensureBuffer(boundingRect().size().toSize());
if (m_buffer.isNull())
return;
if (!m_drawing) {
m_last = p;
m_drawing = true;
}
qreal w;
QPainter painter(&m_buffer);
painter.setRenderHint(QPainter::Antialiasing, false);
if (eraser) {
w = 40.0;
painter.setPen(QPen(Qt::white, w, Qt::SolidLine, Qt::RoundCap, Qt::RoundJoin));
} else {
w = m_baseWidth * (0.6 + 0.4 * pressure);
painter.setPen(QPen(m_inkColor, w, Qt::SolidLine, Qt::RoundCap, Qt::RoundJoin));
}
painter.drawLine(m_last, p);
painter.end();
QRectF dirty = QRectF(m_last, p).normalized().adjusted(-w - 1, -w - 1, w + 1, w + 1);
m_last = p;
update(dirty.toRect());
queueRefresh(dirty.toRect());
scheduleFlush();
}
void InkCanvas::strokeEnd()
{
m_drawing = false;
scheduleFlush();
}
void InkCanvas::queueRefresh(const QRect &r)
{
m_pending = m_pending.isValid() ? m_pending.united(r) : r;
}
// Coalesce: schedule one delayed flush (GUI thread). The delay lets the scene
// composite the new ink into the framebuffer before we re-push it solid; the
// throttle keeps us from flooding the SWTCON.
void InkCanvas::scheduleFlush()
{
if (m_flushScheduled)
return;
m_flushScheduled = true;
QTimer::singleShot(40, this, [this]() {
m_flushScheduled = false;
flushRefresh();
});
}
// GUI thread: re-push the accumulated region with a single explicit swapBuffers,
// which renders SOLID (replacing the scene's dashed two-pass). Params are not
// critical — screen=Pen(0), content=0, flags=0.
void InkCanvas::flushRefresh()
{
const QRect r = m_pending;
m_pending = QRect();
if (!r.isValid())
return;
if (void *fb = epfb_instance())
epfb_swapBuffers(fb, {r.left(), r.top(), r.right(), r.bottom()}, 0, 0, 0);
// If more ink arrived while we were flushing, keep cleaning up.
if (m_drawing)
scheduleFlush();
}
void InkCanvas::clearPage()
{
if (m_buffer.isNull())
return;
m_buffer.fill(Qt::white);
update();
queueRefresh(boundingRect().toRect());
}
QRect InkCanvas::testRectQ() const
{
return QRect(80, 300, 780, 280);
}
void InkCanvas::drawTestStroke()
{
ensureBuffer(boundingRect().size().toSize());
if (m_buffer.isNull())
return;
m_buffer.fill(Qt::white);
QPainter p(&m_buffer);
p.setRenderHint(QPainter::Antialiasing, false);
p.setPen(QPen(Qt::black, m_baseWidth, Qt::SolidLine, Qt::RoundCap, Qt::RoundJoin));
const QRect r = testRectQ();
// A zigzag plus the two diagonals — a recognizable, stroke-like pattern.
QPolygon zig;
int i = 0;
for (int x = r.left(); x <= r.right(); x += 70, ++i)
zig << QPoint(x, (i % 2 == 0) ? r.top() + 40 : r.bottom() - 40);
p.drawPolyline(zig);
p.drawLine(r.topLeft(), r.bottomRight());
p.drawLine(r.bottomLeft(), r.topRight());
p.end();
update();
}

52
src/InkCanvas.h Normal file
View File

@@ -0,0 +1,52 @@
#pragma once
#include <QQuickPaintedItem>
#include <QImage>
#include <QPointF>
#include <QRect>
#include <QColor>
#include <QMutex>
// Drawing surface. Renders strokes via QPainter into an offscreen QImage that is
// blitted as a texture (QSGPainterNode) — the only stroke-rendering path the
// device's *software* epaper scene-graph backend actually draws (it silently
// drops custom QSGGeometryNodes). The trade-off is the e-ink's grayscale
// two-pass ("dashed→fill") refresh for texture updates; eliminating that needs a
// framebuffer-direct pen path (see doc).
class InkCanvas : public QQuickPaintedItem {
Q_OBJECT
public:
explicit InkCanvas(QQuickItem *parent = nullptr);
void paint(QPainter *painter) override;
public slots:
void strokeStart(QPointF p);
void strokeMove(QPointF p, qreal pressure, bool eraser);
void strokeEnd();
void clearPage();
// Experiment harness: clear + draw a fixed recognizable test pattern.
void drawTestStroke();
QRect testRectQ() const;
void setInkColor(const QColor &c) { m_inkColor = c; }
void setBaseWidth(qreal w) { m_baseWidth = w; }
protected:
void geometryChange(const QRectF &newGeometry, const QRectF &oldGeometry) override;
private:
void ensureBuffer(const QSize &size);
void queueRefresh(const QRect &r); // accumulate a region needing a solid swap
void scheduleFlush(); // coalesce; fire flushRefresh once, soon
void flushRefresh(); // GUI thread: explicit solid swapBuffers
QImage m_buffer;
QPointF m_last;
bool m_drawing = false;
QColor m_inkColor = Qt::black;
qreal m_baseWidth = 8.0;
QRect m_pending; // region awaiting an explicit solid refresh
bool m_flushScheduled = false;
};

330
src/InkEngine.cpp Normal file
View File

@@ -0,0 +1,330 @@
#include "InkEngine.h"
#include "FbCapture.h"
#include "epfb.h"
#include <QImage>
#include <QPainter>
#include <QLineF>
#include <QFont>
#include <QThread>
#include <QtGlobal>
#include <cmath>
InkEngine::InkEngine(QObject *parent) : QObject(parent)
{
m_timer.start();
bool ok = false;
const int v = qEnvironmentVariableIntValue("CHATTER_FULL_SM", &ok);
if (ok) m_fullSm = v;
const int g = qEnvironmentVariableIntValue("CHATTER_FLASH_GRAY", &ok);
if (ok) m_flashGray = qBound(0, g, 255);
// De-ghost once the user settles, not on every scroll-lift.
m_deghostTimer.setSingleShot(true);
connect(&m_deghostTimer, &QTimer::timeout, this, [this] { renderAll(true); });
}
void InkEngine::pushRect(const QRect &r)
{
if (!r.isValid())
return;
if (void *fb = epfb_instance())
epfb_swapBuffers(fb, {r.left(), r.top(), r.right(), r.bottom()}, 0, 0, 0);
}
// Manual de-ghost: drive the whole panel to black with a full update, then a
// full update to the real content. The black full-update cycles every particle
// (incl. the color ones that leave the red residue). Uses only our reliable
// QRect swapBuffers — NOT ghostControl(), whose region-swap corrupts internal
// state we don't maintain. Only called on scroll-end / clear (rare), so the
// ~0.4 s flash is acceptable.
void InkEngine::fullRefresh()
{
if (!FbCapture::ready() || m_canvas.isNull())
return;
QImage fb = FbCapture::framebuffer();
const QRect full(0, 0, m_screenW, m_screenH);
const EpRect er{full.left(), full.top(), full.right(), full.bottom()};
void *inst = epfb_instance();
{ QPainter p(&fb); p.fillRect(full, QColor(m_flashGray, m_flashGray, m_flashGray)); }
if (inst) epfb_swapBuffers(inst, er, 0, m_fullSm, 0);
QThread::msleep(220);
{ QPainter p(&fb);
p.drawImage(full.topLeft(), m_canvas, full.translated(0, m_panY));
for (const ButtonDef &b : m_buttons) paintButton(p, b.rect, b.text, false); }
if (inst) epfb_swapBuffers(inst, er, 0, m_fullSm, 0);
}
// ---- virtual canvas ------------------------------------------------------
void InkEngine::ensureCanvas()
{
if (!m_canvas.isNull() || !FbCapture::ready())
return;
const QImage fb = FbCapture::framebuffer();
m_screenW = fb.width();
m_screenH = fb.height();
m_canvas = QImage(m_screenW, m_screenH * 2, QImage::Format_RGB32);
m_canvas.fill(Qt::white);
}
void InkEngine::growIfNeeded(int canvasBottomY)
{
if (m_canvas.isNull() || canvasBottomY < m_canvas.height() - m_screenH / 2)
return;
QImage bigger(m_screenW, m_canvas.height() + m_screenH, QImage::Format_RGB32);
bigger.fill(Qt::white);
QPainter p(&bigger);
p.drawImage(0, 0, m_canvas);
p.end();
m_canvas = bigger;
}
// Copy a screen-space rect from the canvas window into the framebuffer, keep the
// buttons on top, and refresh just that rect.
void InkEngine::blitRegion(const QRect &screenRect, bool full)
{
if (!FbCapture::ready() || m_canvas.isNull())
return;
if (full) { // full-screen de-ghosting refresh (draws content itself)
fullRefresh();
return;
}
const QRect sr = screenRect.intersected(QRect(0, 0, m_screenW, m_screenH));
if (sr.isEmpty())
return;
QImage fb = FbCapture::framebuffer();
QPainter p(&fb);
p.drawImage(sr.topLeft(), m_canvas, sr.translated(0, m_panY)); // canvas -> screen
for (const ButtonDef &b : m_buttons)
if (b.rect.intersects(sr))
paintButton(p, b.rect, b.text, false);
p.end();
pushRect(sr);
}
void InkEngine::renderAll(bool full) { blitRegion(QRect(0, 0, m_screenW, m_screenH), full); }
void InkEngine::panBy(qreal dyScreen)
{
ensureCanvas();
if (m_canvas.isNull())
return;
const int maxPan = qMax(0, m_canvas.height() - m_screenH);
const int np = qBound(0, m_panY - qRound(dyScreen), maxPan); // natural scroll
if (np == m_panY)
return;
m_deghostTimer.stop(); // still moving — postpone the de-ghost
m_scrolled = true;
m_panY = np;
renderAll(false); // fast during the drag
}
// Scrolling stopped: de-ghost once after a short settle (debounced), so rapid
// scrolling doesn't flash on every lift.
void InkEngine::panEnd() { m_deghostTimer.start(1000); }
// ---- buttons -------------------------------------------------------------
void InkEngine::registerButton(int x, int y, int w, int h, const QString &text)
{
const QRect r(x, y, w, h);
m_buttons.append({r, text});
m_uiMask = m_uiMask.united(QRegion(r));
}
int InkEngine::buttonAt(const QPointF &p) const
{
for (int i = 0; i < m_buttons.size(); ++i)
if (m_buttons[i].rect.contains(p.toPoint()))
return i;
return -1;
}
void InkEngine::paintButton(QPainter &p, const QRect &rect, const QString &text, bool pressed) const
{
p.setRenderHint(QPainter::Antialiasing, true);
const QRectF box(rect.x() + 1.5, rect.y() + 1.5, rect.width() - 3.0, rect.height() - 3.0);
p.setPen(QPen(Qt::black, 3));
p.setBrush(pressed ? QColor(0x9e, 0x9e, 0x9e) : QColor(Qt::white));
p.drawRoundedRect(box, 8, 8);
p.setPen(Qt::black);
QFont f = p.font();
f.setPixelSize(38);
p.setFont(f);
p.drawText(rect.translated(0, -2), Qt::AlignCenter, text); // keep in sync with Main.qml offset
}
void InkEngine::flashButton(int x, int y, int w, int h, const QString &text, bool pressed)
{
if (!FbCapture::ready())
return;
QImage fb = FbCapture::framebuffer();
QPainter p(&fb);
paintButton(p, QRect(x, y, w, h), text, pressed);
p.end();
pushRect(QRect(x - 2, y - 2, w + 4, h + 4));
}
void InkEngine::activateButton(const QString &text)
{
if (text == QLatin1String("Clear"))
clearPage(); // also repaints the buttons (reverting any pressed state)
else if (text == QLatin1String("Back"))
emit backRequested(); // wired to AppControl::returnToStandard
}
// ---- finger-wipe erase ---------------------------------------------------
void InkEngine::eraseStart(qreal x, qreal y)
{
ensureCanvas();
m_erasePress = QPointF(x, y + m_panY); // canvas coords
m_eraseLast = m_erasePress;
m_erasing = false; // wait for real movement (tap-safe)
}
void InkEngine::eraseMove(qreal x, qreal y)
{
if (!FbCapture::ready())
return;
ensureCanvas();
if (m_canvas.isNull())
return;
const QPointF cp(x, y + m_panY); // canvas coords
if (!m_erasing) {
// Ignore tap jitter; only a deliberate wipe (moved > ~1.5 mm) erases.
if (QLineF(m_erasePress, cp).length() < 16.0)
return;
m_erasing = true;
m_eraseLast = m_erasePress; // erase the whole wipe, from first contact
}
constexpr qreal w = 124.0; // finger-wipe width (~12 mm @ 264 PPI)
QPainter painter(&m_canvas);
painter.setPen(QPen(Qt::white, w, Qt::SolidLine, Qt::RoundCap, Qt::RoundJoin));
painter.drawLine(m_eraseLast, cp);
painter.end();
const QRect dirty = QRectF(m_eraseLast, cp).normalized()
.adjusted(-w - 2, -w - 2, w + 2, w + 2)
.toRect();
m_eraseLast = cp;
blitRegion(dirty.translated(0, -m_panY));
}
void InkEngine::eraseEnd() { m_erasing = false; }
// ---- ink -----------------------------------------------------------------
void InkEngine::strokeStart(QPointF p)
{
m_btnDown = buttonAt(p); // screen coords
if (m_btnDown >= 0) {
const ButtonDef &b = m_buttons[m_btnDown];
flashButton(b.rect.x(), b.rect.y(), b.rect.width(), b.rect.height(), b.text, true);
m_drawing = false;
return;
}
ensureCanvas();
m_last = p + QPointF(0, m_panY); // canvas coords
m_drawing = true;
m_lastT = m_timer.elapsed();
}
void InkEngine::strokeMove(QPointF p, qreal pressure, qreal tiltX, qreal tiltY, bool eraser)
{
if (!FbCapture::ready())
return;
// In a stylus button-tap: if the pen slides off the button, cancel it.
if (m_btnDown >= 0) {
if (buttonAt(p) != m_btnDown) {
const ButtonDef &b = m_buttons[m_btnDown];
flashButton(b.rect.x(), b.rect.y(), b.rect.width(), b.rect.height(), b.text, false);
m_btnDown = -1;
}
return; // never ink during a button tap
}
ensureCanvas();
if (m_canvas.isNull())
return;
const QPointF cp = p + QPointF(0, m_panY); // canvas coords
if (!m_drawing) {
m_last = cp;
m_drawing = true;
m_lastT = m_timer.elapsed();
}
QPainter painter(&m_canvas);
painter.setRenderHint(QPainter::Antialiasing, false);
const qreal span = m_maxWidth - m_minWidth;
qreal extent; // half-width used to size the dirty rect
if (eraser) {
extent = 40.0;
painter.setPen(QPen(Qt::white, 40.0, Qt::SolidLine, Qt::RoundCap, Qt::RoundJoin));
painter.drawLine(m_last, cp);
} else if (m_penType == 21) {
// reMarkable Calligraphy is a DYNAMIC model (per reMarkable docs +
// observed behavior): thicker on downstrokes, with more pressure, when
// slow, and when the stylus is tilted — not a fixed geometric nib.
const qreal len = QLineF(m_last, cp).length();
const qreal dyN = len > 1e-3 ? (cp.y() - m_last.y()) / len : 0.0; // +1 down, -1 up
const qreal dirF = 0.5 + 0.5 * dyN; // 0 (up) .. 1 (down)
const qint64 now = m_timer.elapsed();
const qreal dt = qMax<qint64>(1, now - m_lastT);
m_lastT = now;
const qreal speed = len / dt; // px / ms
const qreal speedF = 1.0 - qBound(0.0, speed / 3.0, 1.0); // slow -> 1
const qreal tiltMag = qBound(0.0, std::hypot(tiltX, tiltY), 1.0);
const qreal f = 0.25 * dirF + 0.50 * pressure + 0.12 * speedF + 0.13 * tiltMag;
const qreal w = m_minWidth + span * qBound(0.0, f, 1.0);
painter.setPen(QPen(m_color, w, Qt::SolidLine, Qt::RoundCap, Qt::RoundJoin));
painter.drawLine(m_last, cp);
extent = w;
} else {
const qreal w = m_minWidth + span * qBound(0.0, 0.5 + 0.5 * pressure, 1.0);
painter.setPen(QPen(m_color, w, Qt::SolidLine, Qt::RoundCap, Qt::RoundJoin));
painter.drawLine(m_last, cp);
extent = w;
}
painter.end();
const QRect dirty = QRectF(m_last, cp).normalized()
.adjusted(-extent - 2, -extent - 2, extent + 2, extent + 2)
.toRect();
growIfNeeded(qMax(m_last.y(), cp.y()) + extent + 2); // may reallocate the canvas
m_last = cp;
blitRegion(dirty.translated(0, -m_panY));
}
void InkEngine::strokeEnd()
{
if (m_btnDown >= 0) {
const QString text = m_buttons[m_btnDown].text;
m_btnDown = -1;
activateButton(text); // Clear repaints (reverts); Back leaves it pressed
return;
}
m_drawing = false;
}
void InkEngine::clearPage()
{
if (!FbCapture::ready())
return;
ensureCanvas();
m_deghostTimer.stop();
if (!m_canvas.isNull())
m_canvas.fill(Qt::white);
m_panY = 0;
// Two Clear methods: if the user scrolled, de-ghost (black flash); otherwise
// (the common fill-one-screen-then-clear case) use the gentle fast clear.
renderAll(m_scrolled);
m_scrolled = false;
}

96
src/InkEngine.h Normal file
View File

@@ -0,0 +1,96 @@
#pragma once
#include <QObject>
#include <QPointF>
#include <QRect>
#include <QRegion>
#include <QColor>
#include <QString>
#include <QVector>
#include <QImage>
#include <QTimer>
#include <QElapsedTimer>
class QPainter;
// Draws ink strokes DIRECTLY into the e-paper framebuffer (captured via
// FbCapture) and refreshes each segment with an explicit EPFramebuffer swap,
// which renders solid/single-pass — the reMarkable fast-pen path. No Qt scene
// graph in the ink loop. Driven by PenDevice signals. Also owns the top-bar
// button visuals/hit-testing so both finger and stylus can drive them.
class InkEngine : public QObject {
Q_OBJECT
public:
explicit InkEngine(QObject *parent = nullptr);
void setInkColor(const QColor &c) { m_color = c; }
void setWidthRange(qreal minPx, qreal maxPx) { m_minWidth = minPx; m_maxWidth = maxPx; }
void setPenType(int t) { m_penType = t; }
// QML registers each top-bar button's geometry + label (single source of
// truth): used to exclude ink, redraw on clear, and detect stylus taps.
Q_INVOKABLE void registerButton(int x, int y, int w, int h, const QString &text);
// Show a button's pressed/normal state directly in the framebuffer (single
// clean swap, no scene flashing). Called from QML on finger press.
Q_INVOKABLE void flashButton(int x, int y, int w, int h, const QString &text, bool pressed);
// Run a button's action: Clear here, Back via the backRequested() signal.
Q_INVOKABLE void activateButton(const QString &text);
// Finger-wipe erase (finger touches arrive via QML; the stylus does not).
// Erases ink along the finger path; a tap (no movement) erases nothing.
Q_INVOKABLE void eraseStart(qreal x, qreal y);
Q_INVOKABLE void eraseMove(qreal x, qreal y);
Q_INVOKABLE void eraseEnd();
// Two-finger vertical scroll. dyScreen = centroid movement since last call
// (natural: content follows the fingers). Grows the canvas downward as needed.
Q_INVOKABLE void panBy(qreal dyScreen);
// Called when the scroll gesture ends: one full-refresh pass to clear the
// ghosting that the fast per-step swaps leave on the color panel.
Q_INVOKABLE void panEnd();
signals:
void backRequested();
public slots:
void strokeStart(QPointF p);
void strokeMove(QPointF p, qreal pressure, qreal tiltX, qreal tiltY, bool eraser);
void strokeEnd();
void clearPage();
private:
struct ButtonDef { QRect rect; QString text; };
void pushRect(const QRect &r); // fast (Pen waveform) — ink strokes
void fullRefresh(); // ghostControl() de-ghost (scroll/clear)
void paintButton(QPainter &p, const QRect &rect, const QString &text, bool pressed) const;
int buttonAt(const QPointF &p) const; // index into m_buttons, or -1
void ensureCanvas(); // lazily create the virtual canvas
void growIfNeeded(int canvasBottomY); // extend the canvas downward
void blitRegion(const QRect &screenRect, bool full = false); // canvas -> framebuffer
void renderAll(bool full = false); // re-blit the whole viewport (on pan)
QPointF m_last;
bool m_drawing = false;
QColor m_color = Qt::black;
qreal m_minWidth = 2.0; // hairline (px)
qreal m_maxWidth = 14.0; // size's max (px), from DPI + size category
int m_penType = 2; // reMarkable tool id (21 = Calligraphy)
QRegion m_uiMask; // button regions excluded from ink/clear
QVector<ButtonDef> m_buttons;
int m_btnDown = -1; // button under an in-progress stylus tap, or -1
QPointF m_erasePress; // finger-wipe erase state
QPointF m_eraseLast;
bool m_erasing = false; // true once movement passed the tap threshold
QImage m_canvas; // virtual document (>= screen; grows downward)
int m_panY = 0; // top of the viewport within the canvas
int m_screenW = 0, m_screenH = 0;
int m_fullSm = 4; // screen mode for full updates (env CHATTER_FULL_SM)
int m_flashGray = 0; // de-ghost flash level 0=black..255 (env CHATTER_FLASH_GRAY)
QTimer m_deghostTimer; // debounce: de-ghost once ~1s after scrolling stops
bool m_scrolled = false; // any scrolling since last clear? -> Clear de-ghosts
QElapsedTimer m_timer;
qint64 m_lastT = 0;
};

80
src/PenDevice.cpp Normal file
View File

@@ -0,0 +1,80 @@
#include "PenDevice.h"
#include <QSocketNotifier>
#include <fcntl.h>
#include <unistd.h>
#include <sys/ioctl.h>
#include <linux/input.h>
PenDevice::PenDevice(int screenWidth, int screenHeight,
const QString &devicePath, QObject *parent)
: QObject(parent), m_screenW(screenWidth), m_screenH(screenHeight)
{
m_fd = ::open(devicePath.toLocal8Bit().constData(), O_RDONLY | O_NONBLOCK);
if (m_fd < 0) {
qWarning("PenDevice: cannot open %s", qPrintable(devicePath));
return;
}
input_absinfo ai;
if (ioctl(m_fd, EVIOCGABS(ABS_X), &ai) == 0) { m_xMin = ai.minimum; m_xMax = ai.maximum; }
if (ioctl(m_fd, EVIOCGABS(ABS_Y), &ai) == 0) { m_yMin = ai.minimum; m_yMax = ai.maximum; }
if (ioctl(m_fd, EVIOCGABS(ABS_PRESSURE), &ai) == 0) { m_pMin = ai.minimum; m_pMax = ai.maximum; }
m_notifier = new QSocketNotifier(m_fd, QSocketNotifier::Read, this);
connect(m_notifier, &QSocketNotifier::activated, this, &PenDevice::readEvents);
}
PenDevice::~PenDevice()
{
if (m_fd >= 0)
::close(m_fd);
}
QPointF PenDevice::mapToScreen() const
{
double nx = (m_xMax > m_xMin) ? double(m_rawX - m_xMin) / double(m_xMax - m_xMin) : 0.0;
double ny = (m_yMax > m_yMin) ? double(m_rawY - m_yMin) / double(m_yMax - m_yMin) : 0.0;
if (m_invertX) nx = 1.0 - nx;
if (m_invertY) ny = 1.0 - ny;
if (m_swapXY) { double t = nx; nx = ny; ny = t; }
return QPointF(nx * m_screenW, ny * m_screenH);
}
void PenDevice::readEvents()
{
struct input_event ev;
ssize_t n;
while ((n = ::read(m_fd, &ev, sizeof(ev))) == sizeof(ev)) {
switch (ev.type) {
case EV_ABS:
if (ev.code == ABS_X) m_rawX = ev.value;
else if (ev.code == ABS_Y) m_rawY = ev.value;
else if (ev.code == ABS_PRESSURE) m_rawP = ev.value;
else if (ev.code == ABS_TILT_X) m_rawTiltX = ev.value;
else if (ev.code == ABS_TILT_Y) m_rawTiltY = ev.value;
break;
case EV_KEY:
if (ev.code == BTN_TOUCH) {
m_touching = (ev.value != 0);
if (m_touching) emit strokeStart(mapToScreen());
else emit strokeEnd();
} else if (ev.code == BTN_TOOL_RUBBER) {
m_eraser = (ev.value != 0);
}
break;
case EV_SYN:
if (ev.code == SYN_REPORT && m_touching) {
double pressure = (m_pMax > m_pMin)
? double(m_rawP - m_pMin) / double(m_pMax - m_pMin) : 1.0;
const double tiltX = m_rawTiltX / 9000.0; // -1 .. 1
const double tiltY = m_rawTiltY / 9000.0;
emit strokeMove(mapToScreen(), pressure, tiltX, tiltY, m_eraser);
}
break;
default:
break;
}
}
}

52
src/PenDevice.h Normal file
View File

@@ -0,0 +1,52 @@
#pragma once
#include <QObject>
#include <QPointF>
#include <QString>
class QSocketNotifier;
// Reads the reMarkable stylus (Elan marker, /dev/input/event2) directly from
// evdev. The epaper Qt platform delivers finger touch (event3) but NOT the pen,
// so Chatter handles the stylus itself. Maps raw digitizer coordinates to screen
// pixels and emits stroke signals for the ink canvas. The eraser end of the
// Marker Plus arrives as BTN_TOOL_RUBBER.
class PenDevice : public QObject {
Q_OBJECT
public:
explicit PenDevice(int screenWidth, int screenHeight,
const QString &devicePath = QStringLiteral("/dev/input/event2"),
QObject *parent = nullptr);
~PenDevice() override;
bool isOpen() const { return m_fd >= 0; }
signals:
void strokeStart(QPointF pos);
void strokeMove(QPointF pos, qreal pressure, qreal tiltX, qreal tiltY, bool eraser);
void strokeEnd();
private:
void readEvents();
QPointF mapToScreen() const;
int m_fd = -1;
QSocketNotifier *m_notifier = nullptr;
int m_screenW;
int m_screenH;
// Device axis ranges (filled from EVIOCGABS; defaults are the measured ones).
int m_xMin = 0, m_xMax = 6760;
int m_yMin = 0, m_yMax = 11960;
int m_pMin = 0, m_pMax = 4096;
// Orientation — flip these if a test stroke comes out mirrored/rotated.
bool m_invertX = false;
bool m_invertY = false;
bool m_swapXY = false;
// Current sample state.
int m_rawX = 0, m_rawY = 0, m_rawP = 0;
int m_rawTiltX = 0, m_rawTiltY = 0; // ABS_TILT_X/Y, range +/-9000
bool m_touching = false;
bool m_eraser = false;
};

19
src/epfb.h Normal file
View File

@@ -0,0 +1,19 @@
#pragma once
// Direct access to the private EPFramebuffer singleton in libqsgepaper.so, for
// the screen-mode experiment. We bind to the exact mangled symbols via asm
// labels (no class declaration, so no vtable issues). QRect is passed by value
// as its raw {left,top,right,bottom} layout; the enums/QFlags are int-sized.
struct EpRect { int x1, y1, x2, y2; }; // == QRect internal layout
void *epfb_instance() asm("_ZN13EPFramebuffer8instanceEv");
void epfb_swapBuffers(void *self, EpRect r, int contentType, int screenMode, int flags)
asm("_ZN13EPFramebuffer11swapBuffersE5QRect13EPContentType12EPScreenMode6QFlagsINS_10UpdateFlagEE");
// Anti-ghost full refresh. The color panel (EPFramebufferAcep2) overrides this;
// modes 0/3 do an immediate full-screen de-ghosting refresh of the current
// framebuffer, mode 1 schedules one. Call with the instance() pointer.
void epfb_ghostControl(void *self, int mode)
asm("_ZN18EPFramebufferAcep212ghostControlEN13EPFramebuffer16GhostControlModeE");

89
src/main.cpp Normal file
View File

@@ -0,0 +1,89 @@
// Chatter — Qt Quick app with a direct-framebuffer ink pipeline.
// Pen strokes are drawn straight into the e-paper framebuffer (captured by
// FbCapture) and refreshed with an explicit solid swap (InkEngine). Qt Quick is
// used only for the static UI (Back / Clear). This is what gives stock-quality
// solid, low-latency ink — the Qt scene graph can't.
#include <QGuiApplication>
#include <QQmlApplicationEngine>
#include <QQmlContext>
#include <QScreen>
#include <QSettings>
#include <QVariantMap>
#include <QColor>
#include <QRegion>
#include <QRect>
#include "PenDevice.h"
#include "AppControl.h"
#include "InkEngine.h"
#include "FbCapture.h" // links the setBuffers interposer into the executable
struct PenStyle {
int type = 2;
qreal size = 2.0;
QColor color = Qt::black;
};
// Read the pen the user last selected in the standard GUI (xochitl.conf,
// [General]/LastWritingTool — flushed when xochitl stops, i.e. on Back).
static PenStyle readPenStyle()
{
PenStyle ps;
QSettings xs(QStringLiteral("/home/root/.config/remarkable/xochitl.conf"),
QSettings::IniFormat);
const QVariantMap t = xs.value(QStringLiteral("LastWritingTool")).toMap();
if (t.isEmpty()) {
qWarning("readPenStyle: LastWritingTool unreadable; using defaults");
return ps;
}
ps.type = t.value("LastPen").toInt();
ps.size = t.value("LastPenSize").toDouble();
const uint code = t.value("LastPenColorCode").toUInt();
if (code != 0)
ps.color = QColor::fromRgba(QRgb(code));
qInfo("readPenStyle: pen=%d size=%.2f color=%s", ps.type, ps.size,
qPrintable(ps.color.name(QColor::HexArgb)));
return ps;
}
int main(int argc, char *argv[])
{
QGuiApplication app(argc, argv);
const PenStyle penStyle = readPenStyle();
const QSize screen = app.primaryScreen()->geometry().size();
PenDevice pen(screen.width(), screen.height());
AppControl appControl;
InkEngine ink;
ink.setInkColor(penStyle.color);
// reMarkable exposes only the size CATEGORY (LastPenSize 1/2/3 =
// thin/thicker/thickest). Calibrate to measured widths on this 264-PPI panel:
// thicker ~1mm, thickest ~2mm, thinnest ~2px. px/mm = 264/25.4 ~= 10.4.
constexpr double pxPerMm = 264.0 / 25.4;
const double maxMm = qMax(0.2, (penStyle.size - 1.0) * 1.0); // sz3->2mm, sz2->1mm
ink.setWidthRange(2.0, maxMm * pxPerMm);
ink.setPenType(penStyle.type);
// Back button (finger or stylus) → return to the standard GUI.
QObject::connect(&ink, &InkEngine::backRequested, &appControl, &AppControl::returnToStandard);
// Pen drives the ink engine directly (no QML in the ink loop).
QObject::connect(&pen, &PenDevice::strokeStart, &ink, &InkEngine::strokeStart);
QObject::connect(&pen, &PenDevice::strokeMove, &ink, &InkEngine::strokeMove);
QObject::connect(&pen, &PenDevice::strokeEnd, &ink, &InkEngine::strokeEnd);
QQmlApplicationEngine engine;
engine.rootContext()->setContextProperty("appControl", &appControl);
engine.rootContext()->setContextProperty("ink", &ink);
QObject::connect(
&engine, &QQmlApplicationEngine::objectCreationFailed,
&app, []() { QCoreApplication::exit(-1); },
Qt::QueuedConnection);
engine.loadFromModule("Chatter", "Main");
return app.exec();
}

86
tools/chatter_launcher.c Normal file
View File

@@ -0,0 +1,86 @@
// Chatter launcher daemon. Runs always (systemd service), reads the touch device
// (event3) WITHOUT grabbing it, and when it sees a multi-finger hold gesture
// (>= FINGERS contacts held >= HOLD_MS) while xochitl is the active app, it
// switches to Chatter. This is the "return to Chatter from the standard
// interface" trigger (xochitl has no plugin API to add a real button).
#include <errno.h>
#include <fcntl.h>
#include <linux/input.h>
#include <stdio.h>
#include <stdlib.h>
#include <time.h>
#include <unistd.h>
#define DEV "/dev/input/event3"
#define MAX_SLOTS 16
#define FINGERS 4 // contacts required
#define HOLD_MS 700 // how long they must be held
#define COOLDOWN_MS 4000 // ignore re-triggers for this long
static long now_ms(void)
{
struct timespec ts;
clock_gettime(CLOCK_MONOTONIC, &ts);
return ts.tv_sec * 1000L + ts.tv_nsec / 1000000L;
}
int main(void)
{
long lastTrigger = 0;
fprintf(stderr, "chatter-launcher: started (fingers=%d hold=%dms)\n", FINGERS, HOLD_MS);
// Outer loop: (re)open the device forever, so a read interruption (e.g. the
// device sleeping/waking) never kills the daemon.
for (;;) {
int fd = open(DEV, O_RDONLY);
if (fd < 0) { sleep(2); continue; }
int slot = 0;
int tid[MAX_SLOTS];
for (int i = 0; i < MAX_SLOTS; i++)
tid[i] = -1;
long holdStart = 0;
int triggered = 0;
struct input_event ev;
while (read(fd, &ev, sizeof(ev)) == (ssize_t)sizeof(ev)) {
if (ev.type == EV_ABS) {
if (ev.code == ABS_MT_SLOT) {
slot = ev.value;
if (slot < 0) slot = 0;
if (slot >= MAX_SLOTS) slot = MAX_SLOTS - 1;
} else if (ev.code == ABS_MT_TRACKING_ID) {
tid[slot] = ev.value; // >=0 active, -1 released
}
} else if (ev.type == EV_SYN && ev.code == SYN_REPORT) {
int active = 0;
for (int i = 0; i < MAX_SLOTS; i++)
if (tid[i] >= 0) active++;
const long t = now_ms();
if (active >= FINGERS) {
if (holdStart == 0)
holdStart = t;
else if (!triggered && (t - holdStart) >= HOLD_MS &&
(t - lastTrigger) >= COOLDOWN_MS) {
triggered = 1;
lastTrigger = t;
if (system("systemctl is-active --quiet xochitl") == 0) {
fprintf(stderr, "chatter-launcher: gesture -> switching to Chatter\n");
system("/home/root/chatter/to-chatter.sh >/dev/null 2>&1 &");
}
}
} else {
holdStart = 0;
triggered = 0;
}
}
}
close(fd);
fprintf(stderr, "chatter-launcher: input read ended, reopening\n");
sleep(1);
}
return 0;
}

56
tools/fbdump.cpp Normal file
View File

@@ -0,0 +1,56 @@
// Framebuffer snapshot shim: preloaded into xochitl, it captures the e-paper
// framebuffer (via setBuffers) and, when the trigger file /tmp/fbdump appears,
// saves the current framebuffer to /home/root/fbdump.png. Used to recover the
// real Calligraphy nib angle by measuring xochitl's actual rendered strokes.
#define _GNU_SOURCE 1
#include <QImage>
#include <tuple>
#include <dlfcn.h>
#include <pthread.h>
#include <unistd.h>
#include <sys/stat.h>
#include <cstdio>
namespace {
uchar *g_bits = nullptr;
int g_w = 0, g_h = 0, g_bpl = 0, g_fmt = 0;
bool g_started = false;
void *dumper(void *)
{
fprintf(stderr, "FBSHIM dumper thread started\n");
for (;;) {
usleep(400000);
struct stat st;
if (g_bits && stat("/tmp/fbdump", &st) == 0) {
QImage snap = QImage(g_bits, g_w, g_h, g_bpl, QImage::Format(g_fmt)).copy();
bool ok = snap.save("/home/root/fbdump.png");
if (!ok) ok = snap.save("/home/root/fbdump.bmp", "BMP");
::unlink("/tmp/fbdump");
fprintf(stderr, "FBDUMP %dx%d saved ok=%d\n", g_w, g_h, int(ok));
}
}
return nullptr;
}
}
extern "C" void _ZN13EPFramebuffer10setBuffersESt5tupleIJ6QImageS1_EEPS1_(
void *self, void *tuplePtr, void *imgPtr)
{
static void (*real)(void *, void *, void *) = nullptr;
if (!real)
real = (void (*)(void *, void *, void *))dlsym(
RTLD_NEXT, "_ZN13EPFramebuffer10setBuffersESt5tupleIJ6QImageS1_EEPS1_");
auto *t = reinterpret_cast<std::tuple<QImage, QImage> *>(tuplePtr);
const QImage &a = std::get<0>(*t);
g_bits = const_cast<uchar *>(a.constBits());
g_w = a.width(); g_h = a.height(); g_bpl = a.bytesPerLine(); g_fmt = int(a.format());
fprintf(stderr, "FBSHIM setBuffers %dx%d fmt=%d\n", g_w, g_h, g_fmt);
if (!g_started) {
g_started = true;
pthread_t th;
pthread_create(&th, nullptr, dumper, nullptr);
}
if (real) real(self, tuplePtr, imgPtr);
}

26
tools/grabtest.c Normal file
View File

@@ -0,0 +1,26 @@
// Probe whether an input device is exclusively grabbed (EVIOCGRAB) by another
// process (e.g. xochitl). If we can grab it, it was free -> a background watcher
// can read it alongside xochitl. If EBUSY, xochitl holds it exclusively.
#include <errno.h>
#include <fcntl.h>
#include <linux/input.h>
#include <stdio.h>
#include <string.h>
#include <unistd.h>
#include <sys/ioctl.h>
int main(int argc, char **argv)
{
const char *dev = argc > 1 ? argv[1] : "/dev/input/event3";
int fd = open(dev, O_RDONLY);
if (fd < 0) { printf("%s: open failed: %s\n", dev, strerror(errno)); return 1; }
int r = ioctl(fd, EVIOCGRAB, (void *)1);
if (r == 0) {
ioctl(fd, EVIOCGRAB, (void *)0); // release immediately
printf("%s: NOT exclusively grabbed -> a watcher can read it\n", dev);
} else {
printf("%s: GRABBED by another process (%s)\n", dev, strerror(errno));
}
close(fd);
return 0;
}

35
tools/setbufshim.cpp Normal file
View File

@@ -0,0 +1,35 @@
// Feasibility check: can we intercept EPFramebuffer::setBuffers to capture the
// framebuffer's backing QImages? setBuffers is called from the epaper platform
// plugin (libepaper) into libqsgepaper — a cross-DSO call, so unlike swapBuffers
// it should be interposable via LD_PRELOAD. If this fires and reports real image
// dimensions + a pixel pointer, the direct-framebuffer ink pipeline is viable.
//
// ABI: std::tuple<QImage,QImage> is non-trivially-copyable, so it's passed by
// reference (a pointer); QImage* is a pointer. So three pointer args after this.
#include <QImage>
#include <tuple>
#include <dlfcn.h>
#include <cstdio>
extern "C" void _ZN13EPFramebuffer10setBuffersESt5tupleIJ6QImageS1_EEPS1_(
void *self, void *tuplePtr, void *imgPtr)
{
static void (*real)(void *, void *, void *) = nullptr;
if (!real)
real = (void (*)(void *, void *, void *))dlsym(
RTLD_NEXT, "_ZN13EPFramebuffer10setBuffersESt5tupleIJ6QImageS1_EEPS1_");
auto *t = reinterpret_cast<std::tuple<QImage, QImage> *>(tuplePtr);
QImage *c = reinterpret_cast<QImage *>(imgPtr);
const QImage &a = std::get<0>(*t);
const QImage &b = std::get<1>(*t);
fprintf(stderr,
"SETBUFFERS self=%p | A=%dx%d fmt=%d bytesPerLine=%d cbits=%p | "
"B=%dx%d fmt=%d | C=%p %dx%d\n",
self, a.width(), a.height(), int(a.format()), a.bytesPerLine(),
(const void *)a.constBits(), b.width(), b.height(), int(b.format()),
(void *)c, c ? c->width() : -1, c ? c->height() : -1);
if (real) real(self, tuplePtr, imgPtr);
}

BIN
tools/setbufshim.so Executable file
View File

Binary file not shown.

46
tools/swapshim.cpp Normal file
View File

@@ -0,0 +1,46 @@
// LD_PRELOAD trace shim: intercept EPFramebuffer::swapBuffers in libqsgepaper to
// learn the exact (contentType, screenMode, flags) the stock app uses for pen
// strokes. We define functions with the real mangled names so the dynamic loader
// interposes them, log the args, then chain to the real implementation.
//
// QRect is passed by value; its memory layout is {int x1,y1,x2,y2} (l,t,r,b), so
// we model it as a 16-byte POD to match the ABI without linking Qt. The enums and
// QFlags are all 4-byte int-sized.
#include <dlfcn.h>
#include <cstdio>
struct RawRect { int x1, y1, x2, y2; };
extern "C" {
// swapBuffers(QRect, EPContentType, EPScreenMode, QFlags<UpdateFlag>)
void _ZN13EPFramebuffer11swapBuffersE5QRect13EPContentType12EPScreenMode6QFlagsINS_10UpdateFlagEE(
void *self, RawRect r, int contentType, int screenMode, int flags)
{
static void (*real)(void *, RawRect, int, int, int) = nullptr;
if (!real)
real = (void (*)(void *, RawRect, int, int, int))dlsym(
RTLD_NEXT,
"_ZN13EPFramebuffer11swapBuffersE5QRect13EPContentType12EPScreenMode6QFlagsINS_10UpdateFlagEE");
fprintf(stderr, "SWAPTRACE1 rect=(%d,%d)-(%d,%d) %dx%d content=%d screen=%d flags=%d\n",
r.x1, r.y1, r.x2, r.y2, r.x2 - r.x1 + 1, r.y2 - r.y1 + 1,
contentType, screenMode, flags);
if (real) real(self, r, contentType, screenMode, flags);
}
// swapBuffers(const QRegion&, const EPContentMap&, const EPScreenModeMap&, QFlags<UpdateFlag>)
void _ZN13EPFramebuffer11swapBuffersERK7QRegionRK12EPContentMapRK15EPScreenModeMap6QFlagsINS_10UpdateFlagEE(
void *self, const void *region, const void *contentMap, const void *modeMap, int flags)
{
static void (*real)(void *, const void *, const void *, const void *, int) = nullptr;
if (!real)
real = (void (*)(void *, const void *, const void *, const void *, int))dlsym(
RTLD_NEXT,
"_ZN13EPFramebuffer11swapBuffersERK7QRegionRK12EPContentMapRK15EPScreenModeMap6QFlagsINS_10UpdateFlagEE");
fprintf(stderr, "SWAPTRACE2 region=%p contentMap=%p modeMap=%p flags=%d\n",
region, contentMap, modeMap, flags);
if (real) real(self, region, contentMap, modeMap, flags);
}
} // extern "C"

BIN
tools/swapshim.so Executable file
View File

Binary file not shown.