From ac7d5c24ec544eb4a935fe1ade479610a7ae0694 Mon Sep 17 00:00:00 2001 From: Khoa Vo Date: Fri, 26 Jun 2026 15:22:07 +0700 Subject: [PATCH] docs: update README and CHANGELOG for v0.1.0 overhaul - README: update architecture diagrams, feature table, VNI key table - CHANGELOG: document engine rewrite, injection overhaul, bug fixes --- CHANGELOG.md | 60 +++++++++++++++----------- README.md | 119 +++++++++++++++++++++------------------------------ 2 files changed, 83 insertions(+), 96 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index e79b9a6..49b4e14 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,35 +2,45 @@ ## v0.1.0 (2026-06-26) -Initial release. +Initial release and major overhaul. -### Engine -- Direct Input engine — no pre-edit buffer, no underline, no text duplication -- Telex and VNI input methods -- Flexible diacritic placement (tone/modifier at end of syllable) -- Auto-restore English words on space/ESC -- ESC undo (strip all tones from current word) -- Macro expansion with custom shortcuts -- Casing preservation (titlecase, uppercase) +### Engine (major rewrite) -### Injection -- X11 keyboard capture via XGrabKeyboard (no root, no /dev/input) -- Direct X11 clipboard injection (XSetSelectionOwner + XTest Ctrl+V) -- Bundled xclip + wl-copy for Wayland fallback -- Unified injection channel to prevent ordering race conditions +- **Bamboo engine port** — Replaced custom Telex/VNI state machines with a Rust port of bamboo-core's transformation model. Marks and tones are applied to characters in a composition buffer, with proper tone placement for all Vietnamese diphthongs. +- **Flexible backtracking** — Mark/tone keys scan up to 5 characters backward to find the target vowel. Type the full syllable, then add marks at the end: `nguye6n4` → `nguyễn`. +- **Smart uo→ươ cluster** — Single `w`/`7` key after a `uo` pair converts both to `ươ`, even through consonants: `chuong7` → `chương`. +- **Correct tone placement** — Fixed tone positioning for `io` (gió), `uâ` (xuất), `yê` (nguyễn), `oa`/`oe`, `uy`, `iê`, `uô`, `ươ` clusters. +- **Consume stale marks** — VNI/Telex control keys (digits, `f`/`s`/`r`/`x`/`j`/`w`) are consumed silently when they produce no change (e.g., pressing `5` on an already-toned `ạ`). +- **63 focused unit tests** covering Telex, VNI, tone placement, marks, macros, and uppercase. -### Daemon -- **Backspace-Replay pattern** — replays entire keystroke history through a fresh engine on every keypress, eliminating state desync -- FocusIn/FocusOut detection for automatic engine reset -- CPU pinning to P-cores (0-3) + nice(-10) priority boost -- Hot-reload config without restart -- Smart app memory (per-application Vietnamese/English) -- Persistent logging with 10MB rotation +### Injection (major overhaul) + +- **Uinput injection** — ASCII and backspace via Linux evdev keycodes (`/dev/uinput`). Correct keycodes per keyboard hardware, no X11 keycode mismatches. +- **Vietnamese Unicode** — Clipboard paste via persistent X11 connection + XTest Ctrl+V. Text is split only at trailing whitespace/punctuation boundary (no mid-word splitting). Persistent X11 display opened once and reused. +- **Uinput daemon** (`vietc-uinputd`) — Privileged Unix socket server for `/dev/uinput` injection. VMK-style architecture with capability separation. The main daemon communicates via socket, falling back to in-process uinput. +- **X11Injector** uses `XKeysymToKeycode` for Ctrl+V keycodes, adapting to the actual keyboard layout. + +### Capture + +- **Evdev preferred** — Keyboard capture via `/dev/input/event*` with device grab is now the primary path. More reliable than X11 XRecord. +- **X11 XRecord fallback** — X11 passive monitoring via C helper (`vietc-xrecord`) as fallback when evdev is unavailable. + +### Bug Fixes + +- **Fix `Xutf8LookupString` signature** — Missing `XIC` parameter caused all keycodes to map to `\0`. Fixed by adding `*mut c_void` as first argument and passing `NULL`. +- **Fix `execute_commands` backspace count** — The X11 path incorrectly passed `grabbed=true`, subtracting 1 from every backspace. Changed to `false` so full backspace count is used. +- **Fix flush backspace overcount** — `prev_len + 1` erased one character beyond the word. Fixed to `prev_len`. +- **Fix `apply_mark` char removal** — Removed `pattern.len()` chars from composition, but the current key hadn't been appended yet. Fixed to `pattern.len() - 1`. +- **Fix mark backtrack position** — Marks were applied at the end of composition instead of at the found position. Added position-aware `apply_mark_at`. ### Packaging -- AppImage with bundled xclip (no manual setup needed) -- Debian package with proper conffiles, maintainer scripts, and lintian overrides -- Systemd user service + +- AppImage bundles `vietc-uinputd`, `vietc-xrecord`, `xclip`. +- AppRun preserves `LD_LIBRARY_PATH` with system library paths for `dlopen`. +- AppRun auto-starts `vietc-uinputd` via `pkexec`/`sudo` when available. +- Cleaned up `vietc-xrecord` compilation flags (only `-lX11 -lXtst` needed). ### Testing -- 255+ unit tests across engine, protocol, daemon config, and replay + +- 63 focused engine tests covering Telex, VNI, marks, tones, macros, casing. +- Removed old auto-generated bulk tests (850+ tests for deprecated engine). diff --git a/README.md b/README.md index bd860dd..f6bc20b 100644 --- a/README.md +++ b/README.md @@ -44,12 +44,12 @@ Physical Keyboard ┌──────────────────────────────────────────────────────────────┐ │ Stage 1: KEY CAPTURE │ │ │ -│ X11: XGrabKeyboard intercepts all key events │ -│ evdev: /dev/input/event* reads kernel events │ +│ evdev: /dev/input/event* grabs keyboard (primary, reliable) │ +│ X11: XRecord passive monitoring (fallback) │ │ │ │ ┌─────────────┐ ┌──────────────┐ ┌──────────────────┐ │ -│ │ X11Capture │ │ evdev grab │ │ FocusIn/FocusOut │ │ -│ │ (libX11.so) │ │ (libevdev) │ │ detection │ │ +│ │ evdev grab │ │ X11Capture │ │ FocusIn/FocusOut │ │ +│ │ (libevdev) │ │ (XRecord) │ │ detection │ │ │ └─────────────┘ └──────────────┘ └──────────────────┘ │ └──────────────────────────────────────────────────────────────┘ │ @@ -61,58 +61,31 @@ Physical Keyboard │ Ctrl+Space → toggle Vietnamese ON/OFF │ │ Backspace → replay_backspace() │ │ Characters → replay_and_inject(ch) │ +│ VNI/Telex control keys → consume when no match │ └──────────────────────────────────────────────────────────────┘ │ ▼ ┌──────────────────────────────────────────────────────────────┐ -│ Stage 3: BACKSPACE-REPLAY │ +│ Stage 3: BAMBOO ENGINE │ │ │ -│ keystroke_history = ['c', 'h', 'a', 'o', 's'] │ -│ │ │ -│ ▼ │ -│ ┌──────────────────────────────────────────────┐ │ -│ │ Create FRESH engine │ │ -│ │ Replay ALL keystrokes through it │ │ -│ │ engine.buffer() = "cháo" ← correct output │ │ -│ └──────────────────────────────────────────────┘ │ -│ │ │ -│ ▼ │ -│ screen_output = "cháo" │ -│ diff = backspaces(0) + type("cháo") │ -│ (or no change if screen already shows "cháo") │ +│ Transformation model: keystrokes produce composition │ +│ changes. Marks and tones modify existing characters. │ +│ Flexible backtracking scans up to 5 chars for vowels. │ +│ Smart uo→ươ cluster with backtrack. │ +│ Only emits Replace events when output actually changes. │ └──────────────────────────────────────────────────────────────┘ │ ▼ ┌──────────────────────────────────────────────────────────────┐ -│ Stage 4: OUTPUT COMMANDS │ +│ Stage 4: KEY INJECTION │ │ │ -│ EngineEvent::Replace { backspaces: 4, insert: "cháo" } │ -│ │ │ -│ ▼ │ -│ OutputCommand::Backspace(4) │ -│ OutputCommand::Type("cháo") │ -└──────────────────────────────────────────────────────────────┘ - │ - ▼ -┌──────────────────────────────────────────────────────────────┐ -│ Stage 5: KEY INJECTION │ +│ ASCII: direct Linux keycodes via /dev/uinput │ +│ Backspace: Linux keycode 14 via uinput │ +│ Vietnamese Unicode: clipboard paste + trailing ASCII via │ +│ uinput (split only at whitespace/punctuation boundary) │ +│ Persistent X11 connection for Ctrl+V (no per-call overhead) │ │ │ -│ X11 path: │ -│ ┌─────────────────────────────────────────────────────┐ │ -│ │ 1. Ungrab keyboard (XUngrabKeyboard) │ │ -│ │ 2. Send backspaces via XTestFakeKeyEvent │ │ -│ │ 3. Set clipboard via XChangeProperty │ │ -│ │ 4. Handle SelectionRequest events │ │ -│ │ 5. Send Ctrl+V via XTestFakeKeyEvent │ │ -│ │ 6. Regrab keyboard (XGrabKeyboard) │ │ -│ └─────────────────────────────────────────────────────┘ │ -│ │ -│ uinput path (Wayland): │ -│ ┌─────────────────────────────────────────────────────┐ │ -│ │ 1. Send backspaces via /dev/uinput (EV_KEY 14) │ │ -│ │ 2. For ASCII: send keycodes via uinput │ │ -│ │ 3. For Unicode: wl-copy + Ctrl+V via uinput │ │ -│ └─────────────────────────────────────────────────────┘ │ +│ Fallback: vietc-uinputd Unix socket daemon (privileged) │ └──────────────────────────────────────────────────────────────┘ │ ▼ @@ -153,17 +126,18 @@ This means: ``` vietc/ -├── engine/ # Core Vietnamese composition engine +├── engine/ # Vietnamese composition engine (bamboo-core Rust port) │ ├── engine.rs # Orchestrator + replay_keystrokes() -│ ├── telex.rs # Telex state machine (688 lines) -│ ├── vni.rs # VNI state machine (593 lines) -│ ├── english.rs # English auto-restore dictionary +│ ├── bamboo.rs # Bamboo engine: transformation model, composition, tone placement +│ ├── input_method.rs # Telex/VNI rule definitions │ └── spelling.rs # Vietnamese syllable validation │ ├── protocol/ # Keyboard capture & injection │ ├── inject.rs # KeyInjector trait -│ ├── x11_capture.rs # XGrabKeyboard + XNextEvent loop +│ ├── x11_capture.rs # XRecord keyboard capture via C helper │ ├── x11_inject.rs # XTest injection + direct clipboard +│ ├── uinput_monitor.rs # /dev/uinput injection for ASCII + Unicode +│ ├── uinput_client.rs # Unix socket client for vietc-uinputd │ └── wayland_im.rs # Wayland IM protocol │ ├── daemon/ # Main daemon process @@ -172,6 +146,9 @@ vietc/ │ ├── app_state.rs # Per-app Vietnamese/English memory │ └── display.rs # X11/Wayland/compositor detection │ +├── uinputd/ # Privileged uinput backspace daemon (VMK-style) +│ └── main.rs # Unix socket server for /dev/uinput injection +│ ├── ui/ # System tray icon │ └── main.rs # Tray + daemon launcher │ @@ -250,19 +227,19 @@ vietc/ ### VNI -| Key | Result | -|-----|--------| -| `a1` | á | -| `a2` | à | -| `a3` | ả | -| `a4` | ã | -| `a5` | ạ | -| `a6` | â | -| `a8` | ă | -| `e6` | ê | -| `o6` | ô | -| `o7` | ơ | -| `u7` | ư | +| Key | Result | Example | +|-----|--------|---------| +| `1` | á (sắc) | `a1` → `á` | +| `2` | à (huyền) | `a2` → `à` | +| `3` | ả (hỏi) | `a3` → `ả` | +| `4` | ã (ngã) | `a4` → `ã` | +| `5` | ạ (nặng) | `a5` → `ạ` | +| `6` | â/ê/ô | `a6` → `â`, `e6` → `ê`, `o6` → `ô` | +| `7` | ơ/ư | `o7` → `ơ`, `u7` → `ư` | +| `8` | ă | `a8` → `ă` | +| `9` | đ | `d9` → `đ` | + +Flexible typing: type the full syllable, then add marks/tone keys at the end. Example: `nguye6n4` → `nguyễn`. The engine scans backward up to 5 characters to find the target vowel. --- @@ -270,18 +247,18 @@ vietc/ | Feature | How It Works | |---------|-------------| -| **Direct Input** | No pre-edit buffer. Keystrokes instantly become Unicode via XTest/uinput injection | -| **Backspace-Replay** | Replays entire keystroke history in a fresh engine on every keypress — zero state desync | -| **Flexible Placement** | Type tone/modifier at end of syllable (`tranaf` → `trần`) — engine scans backward to find the vowel | -| **Smart Clusters** | `uo` → `ươ`, `ươ` + `o` → `uô`, shape modifier overriding (â↔ă, ô↔ơ) | -| **Auto-Restore** | ~250 English words recognized — typing `hello` won't become Vietnamese. Triggered on space/ESC | -| **ESC Undo** | Strip all tones from current word instantly | +| **Direct Input** | No pre-edit buffer. Keystrokes instantly become text via uinput/XTest injection | +| **Bamboo Engine** | Transformation model ported from bamboo-core — composition, marks, tones, flexible backtracking | +| **Flexible Backtrack** | Type tone/modifier at end of syllable (`tranaf` → `trần`). Scans up to 5 chars backward | +| **Smart Clusters** | `uo` → `ươ` with backtrack (`chuong7` → `chương`) | +| **Tone Placement** | Correct tone positioning for all Vietnamese diphthongs (io→gió, uâ→xuất, yê→nguyễn) | | **Macro Expansion** | `ko` → `không`, `dc` → `được`, custom shortcuts | -| **Casing Preservation** | `SATS` → `SÁT`, `Saa` → `Sả` — matches your typing pattern | +| **Casing Preservation** | `Tieengs` → `Tiếng`, `TIEENGS` → `TIẾNG` | | **App Memory** | Per-app Vietnamese/English state, saved to `overrides.toml` | | **Hot Reload** | Config changes apply without restart (polls mtime every 1.5s) | -| **Focus Reset** | FocusIn/FocusOut clears engine state — no stale injection on window switch | +| **Focus Reset** | Focus change clears engine state — no stale injection on window switch | | **CPU Priority** | Pins daemon to P-cores (0-3) + nice(-10) for low-latency input | +| **Uinput Daemon** | Privileged `vietc-uinputd` for clean backspace injection (Unix socket, VMK-style) | ---