chore: remove internal chunk planning files from repository
This commit is contained in:
Schwitzz
-78
@@ -1,78 +0,0 @@
|
|||||||
# CHUNK #1 — Scaffold & Foundations
|
|
||||||
|
|
||||||
**Status:** ✅ **COMPLETE** — verified green (typecheck, test, lint, build, and `docker build` + runtime health check all pass). **Duration:** ~1 day · **Prereqs:** none (this chunk creates the repo) · **Follows:** — · **Unblocks:** CHUNK #2
|
|
||||||
|
|
||||||
**Purpose:** Stand up the entire monorepo with strict TypeScript, the app shell, the theming system, the `packages/core` contracts, a Fastify server skeleton, and a green CI pipeline. When this chunk is done, `pnpm dev` runs a themed, navigable app shell and `pnpm test` is green — everything later chunks build on.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Steps
|
|
||||||
|
|
||||||
### 1. Workspace & tooling
|
|
||||||
1. `cd` to the repo root and run:
|
|
||||||
- `pnpm init` → set `"name": "pubscript"`, `"packageManager": "pnpm@latest"`.
|
|
||||||
- Create `pnpm-workspace.yaml`:
|
|
||||||
```yaml
|
|
||||||
packages:
|
|
||||||
- "apps/*"
|
|
||||||
- "packages/*"
|
|
||||||
- "server"
|
|
||||||
```
|
|
||||||
2. Root `package.json` scripts: `dev`, `build`, `test`, `lint`, `typecheck`, `contract` (each fanning out to workspaces via `pnpm -r run <name>`).
|
|
||||||
3. `tsconfig.base.json` with `"strict": true`, `"noUncheckedIndexedAccess": true`, `"exactOptionalPropertyTypes": true`, `"moduleResolution": "bundler"`, `"target": "ES2022"`. Every package extends it.
|
|
||||||
4. ESLint (flat config, `typescript-eslint` recommended + `react-hooks`) and Prettier at root; `.editorconfig`; `.gitignore` (node_modules, dist, .env, *.sqlite, desktop/target).
|
|
||||||
5. `vitest.config.ts` at root (workspace mode) so `pnpm test` runs every package.
|
|
||||||
|
|
||||||
### 2. `apps/web` — the app
|
|
||||||
1. Scaffold: Vite 6 + React 19. Add `react-router` v7, `zustand` v5, Tailwind v4 (CSS-first config via `@tailwindcss/vite`).
|
|
||||||
2. Create the **app shell** in `apps/web/src/`:
|
|
||||||
- `AppShell.tsx` — left sidebar (Projects / Characters / World / Plot / Whiteboard / AI / Settings), top bar (project title, save status, search), main content outlet.
|
|
||||||
- `CommandPalette.tsx` — `Ctrl/Cmd+K` palette (jump to chapter, entity, setting, action).
|
|
||||||
- Routes: `/projects`, `/projects/:id/write`, `/projects/:id/characters`, `/world`, `/plot`, `/whiteboard`, `/ai`, `/settings`.
|
|
||||||
3. **Settings store** (`apps/web/src/state/settings.ts`, zustand + persist): theme id, accent, font family/size/leading/measure, density, autosave interval, per-role AI defaults. Persisted to localStorage (web) / Tauri store (desktop) behind one `SettingsBackend` interface.
|
|
||||||
|
|
||||||
### 3. `packages/ui` — design system
|
|
||||||
1. **Theme engine** (`packages/ui/src/theme.ts`): CSS-variable token map. Implement the four themes — **Ink** (dark, default), **Parchment** (warm light), **Nocturne** (deep indigo), **Terminal** (monochrome mono) — as token objects; `applyTheme(id)` sets `document.documentElement.dataset.theme`; accent color and density are runtime overrides.
|
|
||||||
2. Component kit (typed, Tailwind-styled, no external UI framework): `Button`, `Input`, `Select`, `Modal`, `Drawer`, `Tooltip`, `Tabs`, `Badge`, `EmptyState`, `Spinner`, `Toast`.
|
|
||||||
3. `packages/ui/src/AppShell.css` — base typography, focus rings, scrollbars, reduced-motion support.
|
|
||||||
|
|
||||||
### 4. `packages/core` — shared contracts
|
|
||||||
1. `packages/core/src/types.ts` — **every** domain type from PLAN.md §3.3 (User, Project, Book, Chapter, Character, WorldItem, PlotBeat, TimelineEvent, Board, Note, AISession, Suggestion) as strict TS interfaces + a `Role`/`Permission` union.
|
|
||||||
2. `packages/core/src/storage.ts` — the `StorageAdapter` interface exactly as in PLAN.md §3.4, plus `StoreEvent` union (`doc-changed`, `entity-changed`, `board-changed`, `ai-session-changed`, `presence`).
|
|
||||||
3. `packages/core/src/ids.ts` — ULID generator + validator. `packages/core/src/time.ts` — UTC ISO helpers.
|
|
||||||
4. `packages/core/src/tokens.ts` — `countTokens(text)` (wrap `gpt-tokenizer`, lazy-loaded) + `allocateBudget({system, context, history, output, total})` returning per-slot token caps. Unit-test both.
|
|
||||||
|
|
||||||
### 5. `server/` — Fastify skeleton
|
|
||||||
1. `server/package.json` (ESM): `fastify`, `@fastify/static`, `@fastify/websocket`, `@fastify/cookie`, `better-sqlite3`, `argon2`, `zod`.
|
|
||||||
2. `server/src/index.ts` — boots Fastify, serves `apps/web/dist` statically (so one container serves app + API), exposes:
|
|
||||||
- `GET /api/health` → `{ ok, version, db: 'up' }`
|
|
||||||
- `GET /api/version`
|
|
||||||
3. `server/src/config.ts` — env-driven: `PUBSCRIPT_DB` (path, default `./data/pubscript.sqlite`), `PORT`, `PUBSCRIPT_ORIGIN` (CORS), `PUBSCRIPT_ADMIN_EMAIL` (bootstrap admin), `PUBSCRIPT_TOKEN_TTL`.
|
|
||||||
|
|
||||||
### 6. CI — `.github/workflows/ci.yml`
|
|
||||||
On push/PR: `pnpm install --frozen-lockfile` → `pnpm lint` → `pnpm typecheck` → `pnpm test` → `pnpm build` (web + server) → `docker build -f docker/Dockerfile .` (smoke). Post the test summary as an artifact.
|
|
||||||
|
|
||||||
### 7. `docker/`
|
|
||||||
- `docker/Dockerfile` — multi-stage: `node:22-alpine` build stage (pnpm, build web + server) → slim runtime stage serving the built app + running the API. Non-root user. `HEALTHCHECK` hitting `/api/health`.
|
|
||||||
- `docker/compose.yaml` — service `pubscript`, port 8080, volume `./data:/app/data`, `restart: unless-stopped`.
|
|
||||||
|
|
||||||
### 8. Docs & conventions
|
|
||||||
- `docs/CONVENTIONS.md` — commit style, file layout rules, "code against `packages/core`, never against a backend" rule.
|
|
||||||
- `README.md` at root — 10-line quickstart: `pnpm i && pnpm dev`, `docker compose up`, where things live.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Acceptance criteria (all must pass)
|
|
||||||
- [ ] `pnpm i && pnpm build` succeeds with zero TS errors under `strict`.
|
|
||||||
- [ ] `pnpm dev` → app shell loads at `/`, sidebar + top bar render, all 8 routes navigate, `Ctrl+K` palette opens and jumps to a route.
|
|
||||||
- [ ] Switching theme (Ink → Parchment → Nocturne → Terminal) restyles the whole shell instantly; choice survives a reload.
|
|
||||||
- [ ] `pnpm test` green: token budget math, ULID, theme token map, settings persistence.
|
|
||||||
- [ ] `docker compose up` → `curl localhost:8080/api/health` returns `{"ok":true,...}` and the app shell loads in the browser.
|
|
||||||
- [ ] CI run is green end-to-end (lint, typecheck, test, build, docker).
|
|
||||||
|
|
||||||
## Definition of done
|
|
||||||
Repo is a working, themed, CI-green monorepo. Any later chunk can assume: strict types exist in `packages/core`, the shell renders, settings persist, and `docker compose up` serves the app.
|
|
||||||
|
|
||||||
## Risks & fallbacks
|
|
||||||
- **Tailwind v4 + React 19 friction** → if a component misbehaves, pin exact versions from the Vite 6 template and record them in `docs/CONVENTIONS.md`.
|
|
||||||
- **better-sqlite3 native build in Docker** → use `node:22-alpine` with `python3 make g++` in the build stage; if it still fails, fall back to `node:22-slim` (Debian) — record the choice in an ADR.
|
|
||||||
-74
@@ -1,74 +0,0 @@
|
|||||||
# CHUNK #10 — Multi-User Admin & Sharing
|
|
||||||
|
|
||||||
**Duration:** ~2 days · **Prereqs:** CHUNK #2 (auth skeleton, ServerAdapter) · **Follows:** CHUNK #9 · **Unblocks:** CHUNK #11 (parity: admin works in web + desktop identically)
|
|
||||||
|
|
||||||
**Purpose:** Complete the multi-user product: full admin user management (create, disable, change role), project sharing with read/edit permissions, an audit log, and hard multi-tenant isolation — all reachable from **both** the web app and the desktop app identically.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Steps
|
|
||||||
|
|
||||||
### 1. Admin API (`server/src/routes/admin.ts`)
|
|
||||||
1. `GET /api/admin/users` — list all users (name, email, role, status, lastActive).
|
|
||||||
2. `POST /api/admin/users` — create a user (email + role; server generates a one-time password or sends an invite link).
|
|
||||||
3. `PATCH /api/admin/users/:id` — change role (`admin`/`author`) or disable/enable.
|
|
||||||
4. `GET /api/admin/activity` — audit log tail (filtered by user/action/date).
|
|
||||||
5. `GET /api/admin/stats` — instance-wide: user count, project count, storage used.
|
|
||||||
6. All routes behind `requireAdmin` middleware.
|
|
||||||
|
|
||||||
### 2. Sharing API (`server/src/routes/projects.ts` — extend Chunk 2)
|
|
||||||
1. `POST /api/projects/:id/shares` — invite by email, grant `read`/`edit`.
|
|
||||||
2. `PATCH /api/projects/:id/shares/:userId` — change permission.
|
|
||||||
3. `DELETE /api/projects/:id/shares/:userId` — revoke.
|
|
||||||
4. `POST /api/projects/:id/transfer` — transfer ownership (owner only).
|
|
||||||
5. `GET /api/projects` — return owned ∪ shared (with `perm` on each).
|
|
||||||
|
|
||||||
### 3. Access control (`server/src/auth.ts` — extend Chunk 2)
|
|
||||||
1. `canAccessProject(projectId, perm)` — enforces owner OR shared-with (correct `perm`). Middleware on every content route.
|
|
||||||
2. **Multi-tenant isolation**: every query filters by `project_id` the user can access. A user can never read another's project — enforced at the SQL layer, not just the app layer.
|
|
||||||
3. Session revocation: disabling a user → their session token is invalidated within one event round-trip (Chunk 2 acceptance criterion).
|
|
||||||
|
|
||||||
### 4. Audit log (`server/src/db.ts` — extend Chunk 2)
|
|
||||||
1. `audit(id, at, user_id, action, detail JSON)` — log: user create/disable, role change, share grant/revoke, ownership transfer, project delete.
|
|
||||||
2. `GET /api/admin/activity` returns the tail; retention: 1 year (v1, no purge).
|
|
||||||
|
|
||||||
### 5. Admin UI — web (`apps/web/src/features/admin/AdminView.tsx`)
|
|
||||||
1. **Users tab**: table (name, email, role, status, lastActive); "Create user" modal; inline role toggle; disable/enable switch.
|
|
||||||
2. **Sharing tab** (per project): list shared users with permission dropdown; transfer ownership; revoke.
|
|
||||||
3. **Activity tab**: filterable audit log table.
|
|
||||||
4. **Stats tab**: instance-wide counters.
|
|
||||||
|
|
||||||
### 6. Admin UI — desktop parity
|
|
||||||
1. The desktop app renders the **same** `AdminView` (it's part of the shared React app). No desktop-specific admin code — parity is structural.
|
|
||||||
2. Desktop uses bearer token for auth (Chunk 2); web uses cookie. Both hit the same API.
|
|
||||||
|
|
||||||
### 7. Invite flow
|
|
||||||
1. `POST /api/admin/users` with `sendInvite: true` → generates an invite token, stores a pending invite, (v1) logs the invite link to the audit log; the invitee clicks → `/invite/:token` → sets password → becomes an author.
|
|
||||||
|
|
||||||
### 8. Tests
|
|
||||||
1. Unit: `canAccessProject` — owner can edit; shared-read can read not edit; unrelated user gets 403.
|
|
||||||
2. Integration: admin creates user → user logs in → sees only shared projects; admin disables user → user's next API call gets 401.
|
|
||||||
3. Integration: project isolation — user A's adapter never returns user B's project (contract suite extension).
|
|
||||||
4. E2E: admin creates a user, shares a project (edit) → user sees it; revoke → it disappears.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Acceptance criteria (all must pass)
|
|
||||||
- [x] Admin creates/disables/changes-role of users from the web app.
|
|
||||||
- [x] Admin manages sharing (grant/revoke/transfer) from the web app.
|
|
||||||
- [x] Admin sees activity log + instance stats.
|
|
||||||
- [ ] The **same** admin UI works identically in the desktop app.
|
|
||||||
- [x] A shared-with-read user cannot edit; a shared-with-edit user can.
|
|
||||||
- [x] Multi-tenant isolation: user A can never read user B's project (enforced at SQL layer).
|
|
||||||
- [x] Disabling a user revokes their session within one event round-trip.
|
|
||||||
- [x] Audit log records all admin actions.
|
|
||||||
- [x] Contract suite still green on both adapters.
|
|
||||||
- [x] **Admin UI** (`apps/web/src/features/admin/AdminView.tsx`): the Users, Activity/Audit, and Stats views (with user creation, role updates, disable/enable, and deletion) are wired to the backend and rendered by `apps/web/src/routes/Admin.tsx`.
|
|
||||||
|
|
||||||
## Definition of done
|
|
||||||
PubScript is a real multi-user product: admins manage users and sharing from both web and desktop, projects are shared with granular permissions, and isolation is guaranteed at the database layer.
|
|
||||||
|
|
||||||
## Risks & fallbacks
|
|
||||||
- **Permission bugs → data leaks** → enforce isolation at the SQL layer (WHERE project_id in accessible); contract suite tests cross-user access.
|
|
||||||
- **Session revocation latency** → check token validity on every request (small cost) rather than caching; cap the auth lookup.
|
|
||||||
- **Invite email delivery** → v1 logs the invite link (no SMTP); document SMTP as a later addition.
|
|
||||||
-74
@@ -1,74 +0,0 @@
|
|||||||
# CHUNK #11 — Packaging, Parity & Polish
|
|
||||||
|
|
||||||
**Duration:** ~3 days · **Prereqs:** CHUNK #10 (multi-user complete) · **Follows:** CHUNK #10 · **Unblocks:** — (final chunk)
|
|
||||||
|
|
||||||
**Purpose:** Package PubScript for every platform — Docker Compose for self-hosting, Tauri 2 native bundles for Linux/Windows/macOS — and prove web/desktop parity. Add onboarding, import/backup, performance/a11y polish, and a parity matrix that documents every feature's cross-platform status.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Steps
|
|
||||||
|
|
||||||
### 1. Docker self-hosting (`docker/`)
|
|
||||||
1. **`docker/Dockerfile`** (extend Chunk 1): multi-stage — `node:22-alpine` build (pnpm, build web + server) → slim runtime serving the built app + running the API + SQLite on a volume. Non-root user. `HEALTHCHECK` → `/api/health`.
|
|
||||||
2. **`docker/compose.yaml`** (extend Chunk 1): `pubscript` service, port 8080, `./data:/app/data` volume, `restart: unless-stopped`, env from `.env`.
|
|
||||||
3. **`docker/languagetool/compose.yaml`**: optional LanguageTool sidecar (Chunk 7) — documented, not required.
|
|
||||||
4. **Backup scripts** (`docker/backup.sh`): `pg_dump`-style SQLite snapshot (`cp` the `.sqlite` file with WAL checkpoint) → timestamped `.sqlite.bak`; restore script.
|
|
||||||
5. **`docker/README.md`**: "one `docker compose up`" quickstart; env var reference; data location; backup/restore.
|
|
||||||
|
|
||||||
### 2. Tauri desktop bundles (`desktop/`)
|
|
||||||
1. **Tauri 2 config** (`desktop/tauri.conf.json`, extend Chunk 1): app metadata, icons, bundle targets per OS, CSP, and the capability to access the local AI endpoints + filesystem.
|
|
||||||
2. **Native capabilities**:
|
|
||||||
- File-system access (open/save anywhere) via `@tauri-apps/plugin-fs`.
|
|
||||||
- System print-to-PDF (Chunk 6 PDF backend default).
|
|
||||||
- Local AI plumbing (detect Ollama/llama.cpp on LAN).
|
|
||||||
3. **Build matrix** (`.github/workflows/desktop.yml`):
|
|
||||||
- **Linux**: AppImage + `.deb` + `.rpm`.
|
|
||||||
- **Windows**: NSIS installer + portable `.zip`.
|
|
||||||
- **macOS**: `.dmg` (arm64 + x86_64); notarization as explicit later polish.
|
|
||||||
4. **Icons**: generate per-platform icon sets (scripts in `desktop/icons/`).
|
|
||||||
|
|
||||||
### 3. Parity proof
|
|
||||||
1. **Parity matrix** (`docs/PARITY.md`): every feature × {web, desktop-local, desktop-server} with a ✓/—/⚠. Document where desktop adds native capability (file access, print, local AI) but never forks features.
|
|
||||||
2. **Contract suite** (Chunk 2) already runs Local vs. Server adapters identically — extend it to assert the *same* feature behavior across both. CI gate.
|
|
||||||
3. **Playwright parity e2e**: run the same smoke suite against the web build AND the Tauri-built app (via `@tauri-apps/cli` test) — write → autosave → reopen → AI suggestion accept.
|
|
||||||
|
|
||||||
### 4. Onboarding (`apps/web/src/features/onboarding/`)
|
|
||||||
1. First-run flow: create account (or local mode toggle) → create first project → seed the demo ("The Glass Harbor") OR start blank → guided tour of the 5 surfaces (write, characters, plot, whiteboard, AI).
|
|
||||||
2. **Local mode banner**: "Local mode — data stays on this device" when no server is configured.
|
|
||||||
|
|
||||||
### 5. Import & backup (UI)
|
|
||||||
1. **Import** (`apps/web/src/features/import/`): import a `.md` file into a project (Chunk 3); import an EPUB (parse → chapters) as a bonus.
|
|
||||||
2. **Backup** (settings): "Export all data" → zip of the SQLite dump + covers; "Restore" → import the zip.
|
|
||||||
|
|
||||||
### 6. Performance & a11y polish
|
|
||||||
1. Re-run all performance budgets (Chunk 3: 100k-doc < 2s open, < 16ms typing; Chunk 5: 200-node board 60fps) — fix regressions.
|
|
||||||
2. **a11y**: axe-core audit in CI; ensure contrast ≥ 4.5:1 in all themes, full keyboard nav, screen-reader labels on all surfaces.
|
|
||||||
|
|
||||||
### 7. Final docs
|
|
||||||
1. `README.md` (extend Chunk 1): full quickstart for web + desktop + self-host.
|
|
||||||
2. `docs/ROLE_PROMPTS.md`, `docs/DATA_MODEL.md`, `docs/EDITOR.md`, `docs/BOARDS.md`, `docs/PARITY.md` — all complete.
|
|
||||||
|
|
||||||
### 8. Tests
|
|
||||||
1. CI: full matrix — lint, typecheck, unit, contract, exporters golden-file, perf, a11y, web e2e, desktop e2e, docker build.
|
|
||||||
2. All performance + a11y budgets pass.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Acceptance criteria (all must pass)
|
|
||||||
- [x] `docker compose up` → app + API + SQLite running; `/api/health` green; app loads.
|
|
||||||
- [ ] Linux build produces AppImage + `.deb` + `.rpm`; Windows: NSIS + portable; macOS: `.dmg`.
|
|
||||||
- [ ] Desktop app: native file save/open, system print-to-PDF, local AI detection.
|
|
||||||
- [ ] Parity matrix documents every feature; contract suite + Playwright prove web/desktop parity.
|
|
||||||
- [ ] Onboarding flow guides a first-time user through all 5 surfaces.
|
|
||||||
- [ ] Import `.md`/`.epub`; backup/restore round-trips all data.
|
|
||||||
- [ ] All performance budgets pass (open, typing, board fps).
|
|
||||||
- [ ] a11y: axe-core clean, contrast ≥ 4.5:1, full keyboard nav.
|
|
||||||
- [ ] CI green end-to-end across the full matrix.
|
|
||||||
|
|
||||||
## Definition of done
|
|
||||||
PubScript ships as a self-hostable multi-user web app AND native desktop apps on three OSes, with proven parity, polished performance and accessibility, and complete docs. The full vision from PLAN.md is realized.
|
|
||||||
|
|
||||||
## Risks & fallbacks
|
|
||||||
- **Tauri cross-OS packaging friction** (esp. macOS notarization) → Linux-first; Windows as a CI matrix; macOS notarization is explicit later polish, not a v1 blocker.
|
|
||||||
- **Parity gaps slip through** → parity matrix + automated contract/playwright gates in CI make gaps visible.
|
|
||||||
- **Docker native-build issues** (better-sqlite3) → `node:22-alpine` with build tools; fall back to `node:22-slim` (documented in an ADR, Chunk 1).
|
|
||||||
-93
@@ -1,93 +0,0 @@
|
|||||||
# CHUNK #2 — Data Model, Storage & Auth
|
|
||||||
|
|
||||||
**Duration:** ~3 days · **Prereqs:** CHUNK #1 (green CI, app shell) · **Follows:** CHUNK #1 · **Unblocks:** everything (this is the spine)
|
|
||||||
|
|
||||||
**Purpose:** Implement the full domain model, both `StorageAdapter` implementations (local + server), the multi-user Fastify API with auth, roles, and project sharing, the autosave engine, and — critically — the **storage contract test suite** that proves Local and Server adapters behave identically. After this chunk, the app can store and sync real content, and users can sign in, create projects, and share them.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Steps
|
|
||||||
|
|
||||||
### 1. Domain model finalization (`packages/core`)
|
|
||||||
1. In `packages/core/src/types.ts`, ensure every type from PLAN.md §3.3 is complete and exported. Add:
|
|
||||||
- `ProjectMeta` (list view), `NewProject`, `DocumentRef { id, projectId, bookId?, title, doc: ProseMirrorDoc, rev: number }`.
|
|
||||||
- `ProseMirrorDoc = { type: 'doc', content: JsonNode[] }` (JSON form of the TipTap schema).
|
|
||||||
- `StoreEvent` payloads (typed per event kind), `Entity` = `Character | WorldItem | Note` (discriminated union on `kind`).
|
|
||||||
- `AISession`, `Suggestion`, `Role`, `Permission` (already in Chunk 1 — verify against PLAN.md).
|
|
||||||
2. `packages/core/src/schema.ts` — a single `ENTITY_FIELDS` registry: for each entity kind, the ordered field list (`{ key, label, type: 'text'|'textarea'|'select'|'list'|'link', options? }`). This registry drives the generic entity editor (Chunk 4) **and** the server's validation. Single source of truth.
|
|
||||||
|
|
||||||
### 2. SQLite schema (`server/src/db.ts`)
|
|
||||||
1. Create the DB (WAL mode) with migrations in `server/src/migrations/` (plain SQL, applied in order, tracked in a `schema_migrations` table):
|
|
||||||
- `users(id, email UNIQUE, pass_hash, display_name, role, created_at)`
|
|
||||||
- `sessions(token_hash PK, user_id, expires_at, created_at)`
|
|
||||||
- `projects(id, owner_id, title, settings JSON, cover, created_at, updated_at)`
|
|
||||||
- `project_shares(project_id, user_id, perm, PRIMARY KEY(project_id,user_id))`
|
|
||||||
- `books(id, project_id, title, ord)` · `chapters(id, book_id, project_id, title, doc JSON, rev, ord, updated_at)`
|
|
||||||
- `entities(id, project_id, kind, name, fields JSON, created_at, updated_at, UNIQUE(project_id,kind,id))`
|
|
||||||
- `boards(id, project_id, nodes JSON, edges JSON, updated_at)`
|
|
||||||
- `ai_sessions(id, project_id, role, model, settings JSON, messages JSON, suggestions JSON, updated_at)`
|
|
||||||
- `audit(id, at, user_id, action, detail JSON)`
|
|
||||||
2. Indexes: `chapters(project_id)`, `entities(project_id,kind)`, `projects(owner_id)`.
|
|
||||||
3. Seed bootstrap: on first boot with `PUBSCRIPT_ADMIN_EMAIL`, create that admin user (password from `PUBSCRIPT_ADMIN_PASSWORD` or interactive prompt) — idempotent.
|
|
||||||
|
|
||||||
### 3. Auth (`server/src/auth.ts`)
|
|
||||||
1. `POST /api/auth/register` — email+password (argon2id), creates `author`; first-ever user becomes `admin` if no users exist.
|
|
||||||
2. `POST /api/auth/login` → sets httpOnly `ps_session` cookie (web) **and** returns a bearer token (desktop uses bearer). `POST /api/auth/logout` revokes.
|
|
||||||
3. `GET /api/me` → current user + role.
|
|
||||||
4. Middleware: `requireAuth`, `requireAdmin`, `canAccessProject(projectId, perm)` — enforces owner-or-shared. Every content route uses these.
|
|
||||||
5. Rate limit `@fastify/rate-limit` on auth endpoints (10/min/IP).
|
|
||||||
|
|
||||||
### 4. Content API (`server/src/routes/`)
|
|
||||||
Implement against the core types (zod-validated):
|
|
||||||
- `projects.ts`: `GET /api/projects` (owned ∪ shared), `POST /api/projects`, `GET/PATCH/DELETE /api/projects/:id`, `POST /api/projects/:id/shares` (invite by email, `read|edit`), `DELETE` share, `POST /api/projects/:id/transfer`.
|
|
||||||
- `books.ts` + `chapters.ts`: CRUD + `PATCH /api/chapters/:id` (body: `{ doc, baseRev }` — **optimistic concurrency**: 409 if `baseRev` stale).
|
|
||||||
- `entities.ts`: CRUD per kind; `GET /api/projects/:id/entities?kind=&q=` (fuzzy).
|
|
||||||
- `boards.ts`, `ai.ts` (session CRUD + `POST /api/ai/sessions/:id/suggestions`), `notes.ts`.
|
|
||||||
- `admin.ts` (`requireAdmin`): `GET /api/admin/users`, `POST /api/admin/users` (create), `PATCH /api/admin/users/:id` (role/disable), `GET /api/admin/activity` (audit tail).
|
|
||||||
- `ws.ts`: `GET /ws` (WebSocket, auth via token query or cookie) → fan-out `StoreEvent`s per project to subscribed clients.
|
|
||||||
|
|
||||||
### 5. `ServerAdapter` (`packages/api-client`)
|
|
||||||
1. New package `packages/api-client` implementing `StorageAdapter`:
|
|
||||||
- REST calls with bearer (desktop) or cookies (web) — chosen by `navigator`/Tauri detection.
|
|
||||||
- `onEvent` → WebSocket subscription; reconnect with backoff; event → `StoreEvent`.
|
|
||||||
- 409 handling → emit `conflict` event (UI shows "reloaded newer version").
|
|
||||||
2. **`LocalAdapter`** (`packages/storage-local`): Dexie (IndexedDB) tables mirroring the SQLite schema; same method signatures; `onEvent` via a local event bus. In Tauri, persist to disk via `@tauri-apps/plugin-fs` (same Dexie code, different driver).
|
|
||||||
|
|
||||||
### 6. Autosave engine (`packages/core/src/autosave.ts`)
|
|
||||||
1. `createAutosave(adapter, { intervalMs = 800 })` — debounced `saveDocument`; tracks `dirty`, `saving`, `lastSavedAt`, exposes `onStatus` for the top-bar indicator (● unsaved / ◌ saving / ✓ saved 12:04).
|
|
||||||
2. On `conflict`: keep local, fetch remote, show a merge dialog (v1: last-write-wins with explicit user choice).
|
|
||||||
3. `beforeunload` flush.
|
|
||||||
|
|
||||||
### 7. App wiring (`apps/web`)
|
|
||||||
1. **Auth screens:** `/login`, `/register` (use `packages/ui` components). On boot: `GET /api/me` (web) or stored token (desktop) → set current user; no server configured (local mode) → skip auth, banner "Local mode — data stays on this device".
|
|
||||||
2. **Projects page:** list (owned + shared, with share badges), new-project wizard (title, cover, first book), delete/duplicate.
|
|
||||||
3. **Project workspace:** left rail = document tree (Book → Part → Chapter, dnd reorder, per-chapter word count); center = placeholder "editor slot" (real editor lands in Chunk 3); right rail = placeholder for AI/notes.
|
|
||||||
4. **Settings → Accounts** (admin sees user management stub — full UI in Chunk 10).
|
|
||||||
5. **Seed script** `server/src/seed.ts` (`pnpm --filter server seed`): creates a demo project "The Glass Harbor" — 3 chapters of real prose, 4 characters with relationships, 1 world location, 1 board with 6 nodes — so every later chunk has something to show.
|
|
||||||
|
|
||||||
### 8. Contract test suite (`packages/core/src/storage.contract.test.ts`)
|
|
||||||
1. One suite, parameterized over **both** adapters (Local via Dexie in jsdom/happy-dom; Server against a real Fastify instance + SQLite in `:memory:` or a temp file):
|
|
||||||
- project CRUD round-trips; document save/load byte-identical; entity CRUD per kind; board save/load; AI session + suggestions round-trip.
|
|
||||||
- multi-tenant isolation: user A can never read user B's project (server adapter only — Local adapter trivially passes).
|
|
||||||
- `onEvent`: a save in one "client" is observed by a second subscriber (WS on server; bus on local).
|
|
||||||
- optimistic concurrency: stale `baseRev` → conflict.
|
|
||||||
2. Wire as `pnpm contract` and add to CI.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Acceptance criteria (all must pass)
|
|
||||||
- [ ] `pnpm contract` green against **both** adapters.
|
|
||||||
- [ ] Register → login → create project → save a document → reload: content intact.
|
|
||||||
- [ ] Admin creates a second user; that user logs in, sees only their projects; admin shares a project (edit) → it appears; disable → their session is dead within one event round-trip.
|
|
||||||
- [ ] Autosave indicator cycles correctly; kill the tab mid-typing, reopen → ≤1 s of work lost.
|
|
||||||
- [ ] Two browser windows on the same project: save in one appears in the other within ~1 s (WS).
|
|
||||||
- [x] Seed project opens with 3 chapters, 4 linked characters, 1 board.
|
|
||||||
- [ ] CI green including the contract suite.
|
|
||||||
|
|
||||||
## Definition of done
|
|
||||||
The app is a real multi-user product skeleton: accounts, roles, sharing, and a storage layer that is *provably* identical whether data lives on disk or on a server. Chunks 3–10 only add features on top of this spine.
|
|
||||||
|
|
||||||
## Risks & fallbacks
|
|
||||||
- **Dexie in Tauri vs. browser** → abstract the driver behind one `KVBackend` interface; test both.
|
|
||||||
- **WS reconnect storms** → exponential backoff + jitter; cap at 30 s.
|
|
||||||
- **JSON doc storage size** → store ProseMirror JSON (not HTML); chapters are individually loadable, so a 100k-word novel never loads as one blob.
|
|
||||||
-62
@@ -1,62 +0,0 @@
|
|||||||
# CHUNK #3 — Writing Surface
|
|
||||||
|
|
||||||
**Duration:** ~3 days · **Prereqs:** CHUNK #2 (storage spine, autosave) · **Follows:** CHUNK #2 · **Unblocks:** CHUNK #6 (export), #7 (spellcheck), #9 (AI suggestions render over this editor)
|
|
||||||
|
|
||||||
**Purpose:** Build the actual writing experience: the TipTap 3 editor with the full node schema, the document tree, the font system, focus/typewriter modes, live stats, and Markdown import/export — performing well enough that a 100k-word project opens in under 2 seconds and typing never lags.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Steps
|
|
||||||
|
|
||||||
### 1. Editor package (`packages/editor`)
|
|
||||||
1. `packages/editor/src/schema.ts` — TipTap extensions, all registered once:
|
|
||||||
- Base: `Document`, `Paragraph`, `Text`, `Heading` (1–3), `Blockquote`, `HorizontalRule`, `HardBreak`, `Bold`, `Italic`, `Strike`, `Code`, `CodeBlock`.
|
|
||||||
- **`SceneBreak`** (node) — a centered `* * *` divider, draggable, with a NodeView; splits scenes for stats and AI targeting.
|
|
||||||
- **`Footnote`** (inline + a `FootnoteArea` block at doc end) — v1 simple: number, reference, definition.
|
|
||||||
- **`CharacterMention`** (inline atom) — rendered as a styled chip linking to a character entity (inserted by autocomplete in Chunk 4; the node type lands here).
|
|
||||||
- Markdown I/O: `prosemirror-markdown` with rules for scene breaks (`***`), mentions (`@Name`), footnotes.
|
|
||||||
2. `packages/editor/src/Editor.tsx` — the headless wrapper:
|
|
||||||
- `createEditor({ doc, onUpdate, decorations })` returning `{ view, state, dispatch, applyTransaction }`.
|
|
||||||
- **Decoration pipeline:** `useDecorations(view, sources[])` — merges decorations from spellcheck (Chunk 7), AI suggestions (Chunk 9), and mentions into one `DecorationSet`, memoized, recomputed only on relevant transactions. This is the seam later chunks plug into — build it now, even with zero sources.
|
|
||||||
- `applyEdit(range, before, after)` — the transaction used by "Accept suggestion".
|
|
||||||
3. `packages/editor/src/perf.ts` — `measureTypingLatency()` (rAF-based) + a dev overlay showing ms/keystroke.
|
|
||||||
|
|
||||||
### 2. Writing view (`apps/web/src/features/write/`)
|
|
||||||
1. `WritingView.tsx` — layout: document tree (left, from Chunk 2), editor (center), context rail (right: current chapter's linked characters/notes, collapsible).
|
|
||||||
2. **Toolbar** — floating (appears on selection) + docked (top): headings, bold/italic/strike, blockquote, scene break, footnote, mention, undo/redo. Slash menu (`/`) for block insertion.
|
|
||||||
3. **Keyboard map** (`packages/editor/src/keys.ts`): `Ctrl/Cmd+B/I/K`, `Ctrl+/` scene break, `Alt+↑/↓` chapter nav, `F2` focus mode.
|
|
||||||
4. **Focus mode** — dims everything but the current paragraph band; **Typewriter mode** — auto-scrolls the active line to a fixed vertical position. Both toggle from toolbar + settings.
|
|
||||||
5. **Stats bar** (footer): word count (chapter/book/project), session words, reading time; per-chapter counts in the tree.
|
|
||||||
6. **Font system** (`packages/ui/src/fonts.ts`): bundle via `@fontsource` — **EB Garamond**, **Source Serif 4**, **Inter**, **Space Grotesk**, **JetBrains Mono** (+ weights). Settings controls: family, size (14–22 px), leading (1.3–2.0), measure (50–90 ch), margins. Apply via CSS variables on the editor root; persist in settings store.
|
|
||||||
7. **Markdown I/O:** `exportMarkdown(doc)` / `importMarkdown(text)` in `packages/editor/src/markdown.ts`. Import dialog in the tree ("Import .md…") with a preview + conflict check; per-chapter and whole-project export (feeds Chunk 6).
|
|
||||||
|
|
||||||
### 3. Performance work
|
|
||||||
1. **Lazy chapters:** only the open chapter's doc is in the editor; switching chapters swaps state without unmounting the view.
|
|
||||||
2. **Virtualize the tree** for books with 500+ chapters (`@tanstack/react-virtual`).
|
|
||||||
3. Add a Playwright perf test (`apps/web/e2e/perf.spec.ts`): seed a 100k-word doc (script generates 40k paragraphs), assert: open < 2 s; 20 keystrokes median latency < 16 ms (using the dev overlay's API); tree scroll at 60 fps.
|
|
||||||
|
|
||||||
### 4. Polish
|
|
||||||
1. Placeholder states (empty book, empty chapter), save indicator already wired (Chunk 2).
|
|
||||||
2. Accessibility: visible focus rings, `aria` on toolbar/tree, full keyboard operation of the tree.
|
|
||||||
3. `docs/EDITOR.md` — node reference (for humans) + "how to add a node type" (for agents).
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Acceptance criteria (all must Pass)
|
|
||||||
- [ ] Type a full chapter: no dropped keystrokes; latency overlay < 16 ms median on a 100k-doc.
|
|
||||||
- [ ] 100k-word seeded project opens < 2 s (Playwright perf test in CI).
|
|
||||||
- [ ] Scene breaks insert via `***` and slash menu; stats split by scene.
|
|
||||||
- [ ] Footnotes render and export to Markdown correctly.
|
|
||||||
- [ ] Font/size/leading/measure changes apply live and persist across reload.
|
|
||||||
- [ ] Focus + typewriter modes work and are keyboard-toggleable.
|
|
||||||
- [ ] Import a 50-chapter `.md` file → tree + docs correct; re-export round-trips (diff of re-imported Markdown is empty modulo whitespace).
|
|
||||||
- [ ] Decoration pipeline: register a fake decoration source → renders; (Chunk 9 will replace it with real AI suggestions.)
|
|
||||||
- [ ] A11y pass: toolbar fully keyboard-operable; contrast ≥ 4.5:1 in all four themes.
|
|
||||||
|
|
||||||
## Definition of done
|
|
||||||
An author can sit down and *write* — fast, beautiful, with the story structure around them — and the editor exposes a stable decoration/edit API that spellcheck and AI plug into without touching core code.
|
|
||||||
|
|
||||||
## Risks & fallbacks
|
|
||||||
- **TipTap v3 API churn** → pin the exact v3 release; wrap all v3-specific calls in `packages/editor` so the app never imports TipTap directly.
|
|
||||||
- **Markdown round-trip loss** (footnotes/mentions) → keep a canonical ProseMirror-JSON store; Markdown is an *exchange* format, not the source of truth (document this in `docs/EDITOR.md`).
|
|
||||||
- **Perf test flakiness in CI** → run perf specs on a dedicated runner with 3 runs, take the median; fail only on >2× regression.
|
|
||||||
@@ -1,34 +0,0 @@
|
|||||||
# CHUNK #3 — Editor & Writing View
|
|
||||||
|
|
||||||
**Status:** In progress. Steps 1–3 committed. Step 4 (polish) in progress.
|
|
||||||
|
|
||||||
## Bug fixes
|
|
||||||
- [x] **UI disappears when navigating to Write** — WritingView passed `chapter.doc.content` (a bare ProseMirror node array) as editor `content`, so `EditorState.create` built a doc node with an undefined type and threw `Cannot read properties of undefined (reading 'schema')`, unmounting the write view. Fixed by passing the full doc object (`{ type: 'doc', content }`) on init and on `setContent` sync. Verified in a headless browser: editor renders the seeded chapter ("The Glass Harbor") with no console errors. Regression tests in `packages/editor/src/use-editor-doc.test.ts`.
|
|
||||||
|
|
||||||
## Progress
|
|
||||||
- [x] Step 1: Editor package (TipTap 3 schema, wrapper, markdown, key bindings) — committed
|
|
||||||
- [x] Step 2: Writing view in apps/web wired to storage adapter
|
|
||||||
- [x] Add @pubscript/editor to web deps
|
|
||||||
- [x] project-store (zustand) over storageLocal
|
|
||||||
- [x] client-side seed for local adapter
|
|
||||||
- [x] WritingView component (editor + autosave + word/scene count + chapter nav)
|
|
||||||
- [x] wire Write route
|
|
||||||
- [x] Step 3: Performance (autosave via createAutosave, idle detection via createIdleDetector)
|
|
||||||
- [ ] Step 4: Polish (editor.css import ✅, fonts 🟡, SVG monogram logo ✅)
|
|
||||||
- [x] editor.css import in main.tsx
|
|
||||||
- [x] SVG monogram logo in Sidebar
|
|
||||||
- [x] Sidebar reflects the real seeded project ("The Glass Harbor") + live word count
|
|
||||||
- [ ] font family/size/leading/measure controls wired to CSS variables (size only so far)
|
|
||||||
|
|
||||||
## Key facts
|
|
||||||
- Editor package exports: usePubScriptEditor, createEditorInstance, emptyDecorationSource, throttle, createIdleDetector, serializeToMarkdown, parseMarkdown, node types + constants.
|
|
||||||
- PubScriptEditor React component re-exports EditorContent + Editor type from @pubscript/editor (web never imports TipTap directly).
|
|
||||||
- storageLocal (offline-first, IndexedDB) chosen for web wiring (server routes require auth).
|
|
||||||
- Demo prose lives in server/src/seed.ts ("The Glass Harbor", 3 chapters, 4 chars, 1 world, 1 board).
|
|
||||||
- Web app uses @pubscript/core/local (storageLocal) — no server needed to run offline.
|
|
||||||
- Server runs on PORT (default 8080; set PORT=8081 when 8080 is taken by Docker WordPress).
|
|
||||||
|
|
||||||
## Remaining (finish Step 4)
|
|
||||||
- Wire font family/leading/measure settings to CSS variables (fontSize already wired via applyFontSettings).
|
|
||||||
- Add a settings UI control for font size (or confirm existing one applies).
|
|
||||||
- Final polish pass: focus mode visual, empty-state copy, a11y labels.
|
|
||||||
-68
@@ -1,68 +0,0 @@
|
|||||||
# CHUNK #4 — Characters & World-Building
|
|
||||||
|
|
||||||
**Duration:** ~2 days · **Prereqs:** CHUNK #2 (entity storage), CHUNK #3 (editor for mentions) · **Follows:** CHUNK #3 · **Unblocks:** CHUNK #5 (plot links entities), #9 (AI context pulls from entities)
|
|
||||||
|
|
||||||
**Purpose:** Make the story's moving parts first-class: character sheets with typed relationships, flexible world items, fast fuzzy search, backlinks, and `@mention` links from prose to entities. After this chunk, the "tools to facilitate and integrate characters, world building" requirement is fully delivered.
|
|
||||||
|
|
||||||
**Status (updated 2026-10-25):** 🟡 In progress. Data layer and entity UI layer complete and CI-green (`pnpm typecheck`, `pnpm test`, `pnpm lint`, `pnpm build` all pass; 65 tests green across the workspace). See "Remaining" below.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Steps
|
|
||||||
|
|
||||||
### 1. Entity engine (`apps/web/src/features/entities/`)
|
|
||||||
1. **Generic field renderer** (`EntityFields.tsx`) — driven entirely by `ENTITY_FIELDS` from `packages/core/src/schema.ts` (Chunk 2): renders `text`, `textarea`, `select`, `list` (add/remove/reorder chips), `link` (entity reference) fields. Adding a field to the registry = it appears in the UI. No per-entity UI code.
|
|
||||||
2. **Entity panel** (`EntitiesView.tsx`) — tabbed: **Characters · World · Notes**. Each tab: search box (fuzzy, <50 ms at 500 entities), grouped list, create button, card grid.
|
|
||||||
3. **Entity editor** (`EntityEditor.tsx`) — right-rail drawer: fields from the registry, plus:
|
|
||||||
- **Relationships editor** (characters): add `{ target: character, kind: 'ally'|'rival'|'family'|'mentor'|'romantic'|'custom', note }`; render as a list with the linked character's chip.
|
|
||||||
- **Backlinks** tab: "Appears in" — chapters containing a mention of this entity, plus entities that reference it.
|
|
||||||
- Duplicate, delete (with confirm + backlink warning), "pin to context rail".
|
|
||||||
4. **World kinds** — `location`, `faction`, `item`, `glossary`, `system` — each with its field set from the registry (e.g. location: region, climate, key-people list; faction: goals, rivals list; item: origin, powers, holder).
|
|
||||||
5. **Relationship map** (`RelationshipMap.tsx`) — a small force/graph view (reuse React Flow with a static layout) of the selected character's web; click a node → open that character.
|
|
||||||
|
|
||||||
### 2. Mentions & linking
|
|
||||||
1. **Autocomplete** (`packages/editor/src/mention.ts`): typing `@` opens an entity picker (fuzzy over characters + world items); Enter inserts a `CharacterMention` atom (node from Chunk 3) carrying `entityId`.
|
|
||||||
2. **Mention chip** — styled, hover → mini popover (name + one-line bio + "Open sheet"); click → opens the entity editor.
|
|
||||||
3. **Backlink computation** (`packages/core/src/backlinks.ts`): walk docs' JSON for mention nodes + entity-name matches; cache per project, invalidate on doc save. Power the "Appears in" tab and the context rail.
|
|
||||||
4. **Consistency check** (small win, big value): `findInconsistencies(project)` — e.g. a character marked `status: 'dead'` who is mentioned *after* their death chapter (by chapter order). Surface as a warnings list in the Characters tab.
|
|
||||||
|
|
||||||
### 3. Search & navigation
|
|
||||||
1. Global search (`Ctrl/Cmd+K` palette, from Chunk 1) now indexes: chapters, entities, notes, boards. Grouped results, jump-to with context snippet.
|
|
||||||
2. Context rail (right side of WritingView) shows: this chapter's mentioned characters (auto), pinned entities, and related notes.
|
|
||||||
|
|
||||||
### 4. Data & seed
|
|
||||||
1. Extend `server/src/seed.ts`: the demo project's 4 characters get full sheets (bio, voice, 2 relationships each); 5 world items (a city, a faction, an artifact, 2 glossary terms).
|
|
||||||
2. `docs/DATA_MODEL.md` — the entity registry table (kind → fields) as the human reference.
|
|
||||||
|
|
||||||
### 5. Tests
|
|
||||||
1. Unit: field renderer per type; fuzzy search (500 entities < 50 ms); backlink computation (property test: insert mention → backlink appears; remove → disappears).
|
|
||||||
2. Contract: entity CRUD + `?q=` search through both adapters (add to Chunk 2's suite).
|
|
||||||
3. E2E: create character → `@mention` it in a chapter → backlinks tab lists the chapter → open sheet from the chip.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Acceptance criteria (all must pass)
|
|
||||||
- [~] Create a character with all field types (incl. list + link) via the generic renderer — no custom code added. Generic renderer done for `text`/`textarea`/`checkbox`/`tags` (list); `link` field type pending.
|
|
||||||
- [x] Fuzzy search over 500 seeded entities returns in < 50 ms (measured in test).
|
|
||||||
- [~] `@` autocomplete → chip → popover → open sheet, all working; works while typing fast (no stale menus). Autocomplete + chip + open sheet done; hover popover pending.
|
|
||||||
- [x] Backlinks correct and live-updating (save doc → "Appears in" updates within one event round-trip).
|
|
||||||
- [ ] Consistency check flags the seeded dead-character-after-death case. Logic in `packages/core/src/consistency.ts` done; UI wiring + seeded case pending.
|
|
||||||
- [x] Relationship map renders and navigates.
|
|
||||||
- [ ] Global palette finds a chapter, a character, and a note from one box.
|
|
||||||
- [x] Contract suite still green on both adapters.
|
|
||||||
|
|
||||||
## Remaining work
|
|
||||||
- Wire `findInconsistencies()` into the Characters tab as a warnings list (Chunk 3.2).
|
|
||||||
- Index entities in the global `Ctrl/Cmd+K` palette (Chunk 3.1).
|
|
||||||
- Context rail (right side of WritingView): this chapter's mentioned characters, pinned entities, related notes (Chunk 3.2).
|
|
||||||
- Extend `server/src/seed.ts`: full character sheets (bio, voice, 2 relationships each) + 5 world items (Chunk 4.1).
|
|
||||||
- `docs/DATA_MODEL.md`: the entity registry table (kind → fields) as the human reference (Chunk 4.2).
|
|
||||||
- E2E: create character → `@mention` it in a chapter → backlinks tab lists the chapter → open sheet from the chip (Chunk 5.3).
|
|
||||||
|
|
||||||
## Definition of done
|
|
||||||
The author's canon — who, what, where — is structured, searchable, linked to the prose, and checkable. This is the data the AI roles will reason from in Chunk 9.
|
|
||||||
|
|
||||||
## Risks & fallbacks
|
|
||||||
- **Fuzzy search quality** → use a small, well-tested scorer (e.g. `fuse.js` or a hand-rolled subsequence scorer in `packages/core`); keep it dependency-light and unit-tested.
|
|
||||||
- **Backlink cost on huge docs** → compute incrementally on save (diff the doc), not full re-scan; cache in the DB (`mentions(entity_id, chapter_id)` table).
|
|
||||||
- **Mention vs. plain text ambiguity** → only explicit `@`-inserted atoms are links; name-matching is *suggested*, never auto-applied.
|
|
||||||
-60
@@ -1,60 +0,0 @@
|
|||||||
# CHUNK #5 — Plot & Whiteboard
|
|
||||||
|
|
||||||
**Duration:** ~3 days · **Prereqs:** CHUNK #2 (entity storage) · **Follows:** CHUNK #4 · **Unblocks:** CHUNK #9 (AI reasons over plot structure)
|
|
||||||
|
|
||||||
**Purpose:** Give authors three ways to see and shape their story's structure — a beat board with genre templates, a configurable timeline, and a free-form whiteboard — all backed by React Flow 12 and all linked to chapters and entities. After this chunk, plot structure is first-class and the whiteboard handles 200+ nodes at 60 fps.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Steps
|
|
||||||
|
|
||||||
### 1. Beat board (`apps/web/src/features/plot/`)
|
|
||||||
1. **Beat editor** (`BeatEditor.tsx`): a beat is a `PlotBeat` with `template`, `title`, `description`, `status` (`draft` → `revised` → `locked`), optional `chapterId` link, `order`, `notes`. Edit inline in a list or in a right-rail drawer.
|
|
||||||
2. **Templates** (`apps/web/src/features/plot/templates.ts`): ship four — **Three-Act** (Setup / Confrontation / Resolution), **Save-the-Cat** (all 15 beats), **Seven-Point** (Hook, Plot Turn 1, Midpoint, Plot Turn 2, Climax), and **Custom**. Selecting a template seeds N empty beats with the template's titles; the author edits from there.
|
|
||||||
3. **Board view** (`BeatBoard.tsx`): horizontal swimlanes per act/template-section, or a single ordered list. Drag to reorder; each beat shows status color, word count of linked chapter, and a "link to chapter" picker. Progress bar = % of beats that are `revised` or `locked`.
|
|
||||||
4. **Locking**: a `locked` beat is read-only until unlocked (with reason). Prevents accidental edits during a draft pass.
|
|
||||||
|
|
||||||
### 2. Timeline (`apps/web/src/features/timeline/`)
|
|
||||||
1. **TimelineView.tsx**: a horizontal time axis. Events (`TimelineEvent`) carry `when: {era, day} | date` (configurable: real dates OR a fictional calendar with named eras + day numbers), `title`, `description`, optional `chapterId`, and `links[]` (causal links to other events).
|
|
||||||
2. **Filters**: show/hide by character (events mentioning a character), by location, by era. Multi-select.
|
|
||||||
3. **Causal links**: draw an arrow from cause → effect; rendered as a separate layer behind the events. Click an event → open a read-only panel (full edit in the beat/entity system).
|
|
||||||
|
|
||||||
### 3. Whiteboard flow editor (`packages/boards`)
|
|
||||||
1. **Board model** (`packages/boards/src/types.ts`): `Board { id, projectId, nodes: FlowNode[], edges: FlowEdge[] }` where a `FlowNode` is a typed card — `character`, `world`, `chapter`, `note`, `image`, or `group`. Edges are typed (`leads-to`, `contrasts`, `causes`, `custom`).
|
|
||||||
2. **Node views** (`packages/boards/src/nodes/`): each node type renders a styled card with the entity's key fields (name, role, word count, etc.) and a drag handle. Groups are translucent containers that children snap to.
|
|
||||||
3. **Canvas** (`packages/boards/src/BoardCanvas.tsx`): React Flow 12 with:
|
|
||||||
- **Minimap** (top-right), **zoom/pan** (mouse + keyboard), **auto-layout** (grid, force, or radial) via `@xyflow/layout`.
|
|
||||||
- **Multi-select** (shift-click, marquee), **group/ungroup**, **align** (distribute edges).
|
|
||||||
- **Keyboard**: `Del` delete, `Ctrl/Cmd+D` duplicate, `Ctrl/Cmd+G` group, arrow keys nudge.
|
|
||||||
- **Persistence**: every node/edge change → autosave (Chunk 2 engine) with optimistic `rev`.
|
|
||||||
4. **Performance** (`packages/boards/src/perf.ts`): virtualize off-screen nodes, memoize node views, cap visible edges. **200 nodes at 60 fps** is the hard budget — a Playwright perf test seeds 200 nodes and measures frame time.
|
|
||||||
|
|
||||||
### 4. Cross-links
|
|
||||||
1. **Board ↔ chapters**: a `chapter` node links to a `Chapter` doc; click → opens it in the writer.
|
|
||||||
2. **Board ↔ entities**: `character`/`world` nodes link to entities; click → opens the entity editor.
|
|
||||||
3. **Import from beat board**: "Add beats to board" bulk-imports locked/revised beats as nodes.
|
|
||||||
|
|
||||||
### 5. Tests
|
|
||||||
1. Unit: template seeding (correct beat count + titles); board save/load round-trip (both adapters).
|
|
||||||
2. E2E: create a 6-node board, drag nodes, draw edges, save → reload → identical layout.
|
|
||||||
3. Perf: 200-node board at 60 fps (Playwright).
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Acceptance criteria (all must pass)
|
|
||||||
- [ ] Select "Save-the-Cat" → 15 beats seeded with correct titles; edit one → saves.
|
|
||||||
- [ ] Reorder beats by drag; progress bar updates (% revised/locked).
|
|
||||||
- [ ] Lock a beat → editing is blocked with a reason prompt; unlock → editable.
|
|
||||||
- [ ] Timeline renders events on a configurable calendar (real dates OR era/day); filters by character/location work.
|
|
||||||
- [ ] Whiteboard: create nodes of every type, drag, group, align, draw typed edges, save → reload identical.
|
|
||||||
- [ ] 200-node board renders at 60 fps (Playwright perf test in CI).
|
|
||||||
- [ ] A chapter node opens the writer; a character node opens the entity editor.
|
|
||||||
- [ ] Contract suite still green on both adapters.
|
|
||||||
|
|
||||||
## Definition of done
|
|
||||||
Authors can structure a story three ways — beats, timeline, whiteboard — all linked to the prose and entities, and the whiteboard stays smooth at scale.
|
|
||||||
|
|
||||||
## Risks & fallbacks
|
|
||||||
- **React Flow 12 API churn** → pin the exact version; wrap all calls in `packages/boards` so the app never imports `@xyflow/react` directly.
|
|
||||||
- **200-node perf** → if force-layout is too slow, default to grid layout and make force-layout opt-in; document the node cap in `docs/BOARDS.md`.
|
|
||||||
- **Drag-and-drop jank on low-end machines** → virtualize off-screen nodes; throttle autosave to 800ms.
|
|
||||||
-77
@@ -1,77 +0,0 @@
|
|||||||
# CHUNK #6 — Export Pipeline
|
|
||||||
|
|
||||||
**Status:** ✅ Complete — committed `63f34b3` (feat(export): Chunk #6 Export Pipeline). All 7 exporters, the ExportDialog, and the test suite are green.
|
|
||||||
|
|
||||||
**Duration:** ~3 days · **Prereqs:** CHUNK #3 (editor, ProseMirror JSON) · **Follows:** CHUNK #5 · **Unblocks:** CHUNK #11 (packaging ships exports)
|
|
||||||
|
|
||||||
**Purpose:** Ship a single export engine that turns a project's ProseMirror JSON into Markdown, single-file HTML, EPUB 3, DOCX, ODT, and PDF — all with title/author/cover/TOC/styling options. Everything shares one flat content model so formats stay consistent and tests are golden-file based.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Steps
|
|
||||||
|
|
||||||
### 1. Shared flat model (`packages/exporters/src/model.ts`)
|
|
||||||
1. `flattenProject(project)` → a linear list of `Block`s: `{ type: 'heading'|'paragraph'|'sceneBreak'|'blockquote'|'codeBlock'|'list'|'footnote', text, marks[], level?, children? }`, plus frontmatter `{ title, author, language, cover, chapters: [{title, order}] }`.
|
|
||||||
2. This is the single source every exporter consumes — no exporter touches raw ProseMirror JSON directly.
|
|
||||||
|
|
||||||
### 2. Markdown & HTML (`packages/exporters/src/md.ts`, `html.ts`)
|
|
||||||
1. **Markdown export** (`exportMarkdown(blocks)`): headings (`#`–`###`), scene breaks as `***`, blockquotes, code fences, lists, footnotes as `[1]` refs with a notes section. Round-trips with Chunk 3's importer.
|
|
||||||
2. **Single-file HTML** (`exportHtml(blocks, opts)`): one self-contained `.html` with inline CSS (theme-aware: Ink/Parchment/Nocturne/Terminal), a generated TOC (anchored headings), print styles (`@page` margins), and a cover page. No external assets.
|
|
||||||
|
|
||||||
### 3. EPUB 3 (`packages/exporters/src/epub.ts`)
|
|
||||||
1. Build a valid EPUB 3 container **by hand** (no runtime deps):
|
|
||||||
- `mimetype` (uncompressed, `application/epub+zip`) first in the zip.
|
|
||||||
- `META-INF/container.xml` → `content.opf`.
|
|
||||||
- `content.opf`: metadata (title, author, language, UUID, cover), manifest (`chapterN.xhtml`, `toc.ncx`, `cover.svg`, `style.css`), spine (chapter order).
|
|
||||||
- `chapterN.xhtml`: one file per chapter, semantic HTML5 (`<h1>`, `<p>`, `<hr>` for scene breaks), a `nav` TOC.
|
|
||||||
- `cover.svg`: renders the project's cover color + title/author.
|
|
||||||
2. Zip with `jszip` (already a dep). Output `.epub`.
|
|
||||||
3. **Validation**: golden-file test + `epubcheck` (run in CI via a small Node wrapper or Docker image).
|
|
||||||
|
|
||||||
### 4. DOCX (`packages/exporters/src/docx.ts`)
|
|
||||||
1. Use the maintained **`docx`** npm package: `Document` → `Section` → `Paragraph`/`Heading`/`Quote`; add a `TableOfContents` (field `TOC`), a cover paragraph, and page breaks between chapters.
|
|
||||||
2. Styles: map themes to Word styles (font, size, heading colors). Export `.docx`.
|
|
||||||
|
|
||||||
### 5. ODT (`packages/exporters/src/odt.ts`)
|
|
||||||
1. Build a valid OpenDocument container (parallel to EPUB):
|
|
||||||
- `mimetype` (`application/vnd.oasis.opendocument.text`) first, uncompressed.
|
|
||||||
- `content.xml`: `text:p` paragraphs, `text:h` headings, `text:section`, with `style:name` references.
|
|
||||||
- `manifest.xml`, `meta.xml` (title/author), `styles.xml` (basic paragraph styles), `thumbs/thumbnail.png` (optional).
|
|
||||||
2. Zip with `jszip`. Output `.odt`. (LibreOffice-openable; golden-file unzip-validate in CI.)
|
|
||||||
|
|
||||||
### 6. PDF (`packages/exporters/src/pdf.ts`)
|
|
||||||
1. **Two backends** (chosen by env):
|
|
||||||
- **Server/desktop:** headless Chromium `page.pdf()` over the single-file HTML → pixel-perfect, themes intact, real TOC. This is the default.
|
|
||||||
- **Browser-only fallback:** `jspdf` + `html2canvas` (lower fidelity, no TOC automation) — used when no server is available.
|
|
||||||
2. `exportPdf(blocks, opts)` returns a `Uint8Array`. Add title page, page numbers (`- {page}` footer), and `@page` margins from the HTML path.
|
|
||||||
|
|
||||||
### 7. Export dialog (`apps/web/src/features/export/ExportDialog.tsx`)
|
|
||||||
1. Select: format (multi-select), scope (current chapter / whole book / whole project), options (cover, TOC, theme, page size).
|
|
||||||
2. Live preview thumbnail (HTML/EPUB), progress bar, download button(s). Desktop uses Tauri FS save dialog; web uses native download.
|
|
||||||
3. **Batch export**: "Export all formats" → zips results.
|
|
||||||
|
|
||||||
### 8. Tests
|
|
||||||
1. **Golden-file tests**: for each format, snapshot the output bytes of the seed project; assert stable across runs (whitespace-normalized for HTML/PDF).
|
|
||||||
2. **Round-trip**: MD export → import → MD export is identical.
|
|
||||||
3. **Validation**: `epubcheck` on every EPUB; unzip + XML well-formedness on ODT/EPUB.
|
|
||||||
4. **CI**: run epubcheck; unzip-validate all archives.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Acceptance criteria (all must pass)
|
|
||||||
- [x] Export a project to MD, HTML, EPUB, DOCX, ODT, PDF — all succeed from the dialog.
|
|
||||||
- [x] EPUB passes `epubcheck` with zero errors; opens in Calibre/Readers.
|
|
||||||
- [x] ODT opens cleanly in LibreOffice; styles preserved.
|
|
||||||
- [x] DOCX opens in Word; TOC field present (update-on-open).
|
|
||||||
- [x] PDF has title page, TOC (HTML backend), page numbers, correct margins.
|
|
||||||
- [x] Theme applied in HTML/EPUB/PDF (Ink dark / Parchment light, etc.).
|
|
||||||
- [x] Golden-file tests stable; round-trip MD identical.
|
|
||||||
- [x] CI validates EPUB + unzip-validates all archives.
|
|
||||||
|
|
||||||
## Definition of done
|
|
||||||
An author can export a finished manuscript to every major format from one dialog, with consistent styling and validated archives.
|
|
||||||
|
|
||||||
## Risks & fallbacks
|
|
||||||
- **EPUB/ODT spec edge cases** → golden-file tests + `epubcheck` in CI; keep the container builders tiny and unit-tested.
|
|
||||||
- **PDF fidelity without a server** → prefer headless Chromium; fall back to `jspdf` only when needed and note the limitation.
|
|
||||||
- **Cover image sizing** → normalize covers to EPUB-recommended dimensions (e.g. 1600×2560) before embedding.
|
|
||||||
-75
@@ -1,75 +0,0 @@
|
|||||||
# CHUNK #7 — Spell & Grammar
|
|
||||||
|
|
||||||
**Status:** ✅ Complete — all steps implemented, tests passing, build green.
|
|
||||||
|
|
||||||
**Duration:** ~2 days · **Prereqs:** CHUNK #3 (editor, decoration pipeline) · **Follows:** CHUNK #6 · **Unblocks:** CHUNK #9 (AI suggestions share the decoration pipeline)
|
|
||||||
|
|
||||||
**Purpose:** Add offline spellcheck (hunspell compiled to WASM) and optional grammar/style checking (self-hostable LanguageTool server), rendered as inline squiggles with a side panel to jump/apply/ignore. Everything runs in-browser with zero network for spellcheck; grammar is opt-in.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Steps
|
|
||||||
|
|
||||||
### 1. Spellcheck engine (`packages/editor/src/spell/`)
|
|
||||||
1. **hunspell-WASM** (`packages/editor/src/spell/hunspell.ts`): lazy-load `hunspell-wasm` (codecs + dictionaries from the `hunspell-wasm/dicts` package). Initialize with a default dictionary (`en_US`) + a per-user custom dictionary.
|
|
||||||
2. **Dictionary management** (`packages/editor/src/spell/dicts.ts`):
|
|
||||||
- Load dictionaries on demand (lazy, per active language).
|
|
||||||
- **Custom dictionary**: author-approved words persisted to `localStorage` (web) / Tauri store / server profile. Words added here are never re-underlined.
|
|
||||||
- Language detection: per-document `lang` attribute (default from settings); switch mid-doc.
|
|
||||||
3. **Checker** (`packages/editor/src/spell/checker.ts`): given a ProseMirror doc, produce a list of `SpellHit { from, to, word, suggestions: string[] }` — skipping code, already-accepted mentions, and custom-dictionary words.
|
|
||||||
|
|
||||||
### 2. Grammar engine (`packages/editor/src/grammar/`)
|
|
||||||
1. **LanguageTool adapter** (`packages/editor/src/grammar/languagetool.ts`):
|
|
||||||
- Configured via provider settings: `{ kind: 'languagetool', baseUrl }` (default `http://localhost:8080`, self-hosted Docker).
|
|
||||||
- `check(text, lang)` → POST to `/v2/check`, parse `matches[]` → `GrammarHit { from, to, message, replacements[], ruleId }`.
|
|
||||||
- Debounced (800ms) + cached per text hash to avoid spamming the server.
|
|
||||||
2. **Self-hostable server** (`docker/languagetool/`): a `docker-compose` snippet that runs the official LanguageTool image; documented in `docker/README.md`. API is OpenAI-compatible-ish (documented endpoints).
|
|
||||||
|
|
||||||
### 3. Decoration integration (`packages/editor/src/decorations/`)
|
|
||||||
1. Plug both engines into the **decoration pipeline** built in Chunk 3 (`useDecorations`):
|
|
||||||
- Spellcheck → red squiggle (`Decoration.inline` with `class="ps-spell"`).
|
|
||||||
- Grammar → blue squiggle (`class="ps-grammar"`).
|
|
||||||
2. Memoize: decorations recomputed only on relevant transactions (doc change, accept/reject).
|
|
||||||
|
|
||||||
### 4. Side panel (`apps/web/src/features/check/CheckPanel.tsx`)
|
|
||||||
1. Lists all hits grouped by chapter: `word/message`, suggested replacements (click to apply), "Add to dictionary", "Ignore this time", "Ignore all".
|
|
||||||
2. **Jump**: clicking a hit scrolls the editor to that range and highlights it.
|
|
||||||
3. Counts: X spelling / Y grammar issues remaining; per-chapter breakdown.
|
|
||||||
|
|
||||||
### 5. Settings
|
|
||||||
1. Toggle spellcheck/grammar on/off, per-language dictionaries, grammar server URL, check-on-type vs. check-on-save.
|
|
||||||
|
|
||||||
### 6. Tests
|
|
||||||
1. Unit: `checker` on a known doc → correct `SpellHit`s; custom dictionary word ignored.
|
|
||||||
2. Unit: grammar adapter parses a mock LanguageTool response correctly.
|
|
||||||
3. E2E: type a misspelled word → squiggle appears; add to dictionary → squiggle gone; reload → persists.
|
|
||||||
4. E2E (optional): with LanguageTool running, grammar hit appears + apply suggestion edits the text.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Acceptance criteria (all must pass)
|
|
||||||
- [x] Misspelled word → red squiggle; suggestions shown in panel.
|
|
||||||
- [x] "Add to dictionary" → word never underlined again; persists across reload.
|
|
||||||
- [x] Grammar (if LanguageTool configured) → blue squiggle; apply suggestion edits text.
|
|
||||||
- [x] Panel jumps to a hit and highlights it in the editor.
|
|
||||||
- [x] Code blocks and mentions are never spellchecked.
|
|
||||||
- [x] Toggle spellcheck off → no squiggles; setting persists.
|
|
||||||
- [x] Contract suite still green on both adapters.
|
|
||||||
|
|
||||||
## Definition of done
|
|
||||||
Authors get inline spellcheck (fully offline) and optional grammar checking, with a clean panel to review and act — no wall of unactionable errors.
|
|
||||||
|
|
||||||
## Completed (implementation notes)
|
|
||||||
- **Spell engine** (`packages/editor/src/spell/`): `hunspell-engine.ts` wraps `hunspell-wasm` with a lazy `load()` + `check()`/`suggest()` API; `spell.ts` provides `findSpellErrors()` (tokenizes with `WORD_RE`, dedupes, honors `minWordLength`) and `applySuggestion()`. Dictionaries are base64-decoded with `atob()` + `TextDecoder` (browser-safe, per project convention #74).
|
|
||||||
- **Grammar engine** (`packages/editor/src/grammar/`): `grammar.ts` has a regex tokenizer (`SENTENCE_RE`, `WORD_RE`) plus `parseGrammarMatches()` that maps LanguageTool `/v2/check` `matches[]` → `GrammarIssue[]` (offset/length normalization, `ruleId`/`category`/`message`/`replacements`).
|
|
||||||
- **CheckService** (`packages/editor/src/check/check-service.ts`): orchestrates both engines, exposes `checkText()`/`checkBlock()`/`addWordToDictionary()`/`ignoreWord()`/`ignoreAll()`, and emits `check:progress` events.
|
|
||||||
- **Decorations** (`packages/editor/src/decorations/`): `toCheckedIssues()` merges spell + grammar hits into `CheckedIssue[]`; `collectBlock()` assembles block text (single source of truth, per #73); `useDecorations()` renders red squiggles (`ps-spell`) and blue squiggles (`ps-grammar`) and applies accepted suggestions.
|
|
||||||
- **Web integration** (`apps/web/src/`): `useCheckService.ts` hook wires the service into `WritingView.tsx`; `CheckPanel.tsx` shows counts, per-issue message/replacements, "Add to dictionary", "Ignore"/"Ignore all", and jump-to-issue navigation; `check.css` styles squiggles + panel; `main.tsx` imports `check.css`.
|
|
||||||
- **Settings**: spellcheck toggle + grammar toggle in `Settings.tsx` (persisted via `useSettings`).
|
|
||||||
- **Docker**: `docker/languagetool/docker-compose.yml` + `README.md` for self-hosting LanguageTool (opt-in, graceful degradation if unreachable).
|
|
||||||
- **Tests**: `spell.test.ts` (9 tests) and `grammar.test.ts` (12 tests) — all 30 editor tests green.
|
|
||||||
|
|
||||||
## Risks & fallbacks
|
|
||||||
- **hunspell-WASM bundle size** → lazy-load dictionaries; only load the active language(s).
|
|
||||||
- **LanguageTool self-host complexity** → make it opt-in and clearly documented; app degrades gracefully (grammar disabled) if the server is unreachable.
|
|
||||||
- **Squiggle perf on huge docs** → check only the visible viewport + a buffer, not the whole doc, on each scroll.
|
|
||||||
-78
@@ -1,78 +0,0 @@
|
|||||||
# CHUNK #8 — AI Provider Layer
|
|
||||||
|
|
||||||
**Duration:** ~2 days · **Prereqs:** CHUNK #2 (storage, `packages/core` token budget) · **Follows:** CHUNK #7 · **Unblocks:** CHUNK #9 (roles + suggestions build on this)
|
|
||||||
|
|
||||||
**Purpose:** Build the provider abstraction that talks to any OpenAI-compatible endpoint (local Ollama/llama.cpp/LM Studio/vLLM) plus Anthropic, Gemini, and OpenRouter via the Vercel AI SDK v5 — with streaming, thinking/reasoning params, and the token-budget engine that keeps every prompt inside the context window.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Steps
|
|
||||||
|
|
||||||
### 1. Provider config (`packages/ai/src/providers/`)
|
|
||||||
1. **Provider type** (`packages/ai/src/providers/types.ts`):
|
|
||||||
```ts
|
|
||||||
interface ProviderConfig {
|
|
||||||
id: string;
|
|
||||||
kind: 'openai-compatible' | 'anthropic' | 'google' | 'openrouter';
|
|
||||||
baseUrl?: string; // e.g. http://localhost:11434/v1, https://api.openai.com/v1
|
|
||||||
apiKey?: string; // optional for local
|
|
||||||
model: string;
|
|
||||||
temperature: number;
|
|
||||||
maxTokens: number;
|
|
||||||
thinking?: { enabled: boolean; budget?: number; effort?: 'low'|'medium'|'high' };
|
|
||||||
contextWindow: number;
|
|
||||||
}
|
|
||||||
```
|
|
||||||
2. **Discovery** (`packages/ai/src/providers/discover.ts`): auto-detect local providers by probing common endpoints (`http://localhost:11434/v1/models`, `http://localhost:1234/v1/models`). Show "detected local models" in settings.
|
|
||||||
3. **Storage**: providers persisted per-user (server) or per-machine (local) in the settings store.
|
|
||||||
|
|
||||||
### 2. Adapter layer (`packages/ai/src/adapters/`)
|
|
||||||
1. **OpenAI-compatible adapter** (`openaiCompatible.ts`): uses `@ai-sdk/openai` with a custom `apiBase` + `headers`. Handles streaming (`generateText`/`streamText`) and non-streaming.
|
|
||||||
2. **Anthropic adapter** (`anthropic.ts`): `@ai-sdk/anthropic`; maps `thinking.budget` → the SDK's `thinking` param.
|
|
||||||
3. **Google adapter** (`google.ts`): `@ai-sdk/google` (Gemini).
|
|
||||||
4. **OpenRouter adapter** (`openrouter.ts`): `@ai-sdk/openai-compatible` with `https://openrouter.ai/api/v1` base + special headers (`HTTP-Referer`, `X-Title`).
|
|
||||||
5. **Factory** (`getProvider(config)`): returns the right adapter; unknown `kind` throws a clear error.
|
|
||||||
|
|
||||||
### 3. Thinking / reasoning (`packages/ai/src/thinking.ts`)
|
|
||||||
1. Normalize `thinking` across providers:
|
|
||||||
- o-series (OpenAI) → `reasoningEffort`.
|
|
||||||
- Anthropic → `thinking` budget object.
|
|
||||||
- Local (e.g. Qwen3 via Ollama) → extra body params / `chat_template_kwargs`.
|
|
||||||
2. Parse streamed reasoning tokens when present (strip from final content, expose separately for debugging).
|
|
||||||
|
|
||||||
### 4. Token budget (`packages/ai/src/budget.ts`)
|
|
||||||
1. **Count** (`countTokens(text)`): wrap `gpt-tokenizer` (o200k_base), lazy-loaded. Already stubbed in Chunk 2 — flesh out with real tokenizer.
|
|
||||||
2. **Allocate** (`allocateBudget({ system, context, history, output, total })`): returns per-slot token caps (defaults: system+role 10%, context pack 40%, history 30%, output 20%). Adjustable per provider config.
|
|
||||||
3. **Guard** (`enforceBudget(prompt, budget)`): throws or truncates if the assembled prompt exceeds the context window; logs which slot overflowed.
|
|
||||||
|
|
||||||
### 5. Streaming client (`packages/ai/src/client.ts`)
|
|
||||||
1. `streamCompletion(provider, messages, opts)` → async iterator of `{ type: 'text'|'reasoning'|'tool-call'|'done', text? }`.
|
|
||||||
2. `complete(provider, messages, opts)` → full text (for non-streaming tool use).
|
|
||||||
3. Error handling: 429 → retry with backoff; auth errors → clear message; timeout → abort.
|
|
||||||
4. **Usage tracking**: capture `promptTokens`/`completionTokens` per call → stored in `AISession` (Chunk 9) + shown in settings.
|
|
||||||
|
|
||||||
### 6. Tests
|
|
||||||
1. Unit: `countTokens` on known strings (compare to gpt-tokenizer reference).
|
|
||||||
2. Unit: `allocateBudget` + `enforceBudget` edge cases (empty context, overflow).
|
|
||||||
3. Unit: thinking-param normalization for each provider.
|
|
||||||
4. Integration (mock): a fake OpenAI-compatible server returns a streamed response; adapter yields correct chunks. Use `msw` or a local Express stub.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Acceptance criteria (all must pass)
|
|
||||||
- [x] Configure an OpenAI-compatible provider (local Ollama URL) → streaming works.
|
|
||||||
- [x] Configure OpenAI/Anthropic/Gemini/OpenRouter with API keys → streaming works (mock keys in tests).
|
|
||||||
- [x] Auto-detect a local model running on `:11434`.
|
|
||||||
- [x] `thinking` enabled → correct provider param sent; reasoning tokens parsed when present.
|
|
||||||
- [x] `countTokens` matches gpt-tokenizer within tolerance.
|
|
||||||
- [x] Prompt exceeding context window → clear error, no crash.
|
|
||||||
- [x] Streaming chunks arrive in order; usage tracked per call.
|
|
||||||
- [x] CI green (unit + mocked integration).
|
|
||||||
|
|
||||||
## Definition of done
|
|
||||||
Every provider type is reachable through one streaming client, the token budget keeps prompts in-window, and local models are auto-discovered. Chunk 9 wires roles and suggestions on top.
|
|
||||||
|
|
||||||
## Risks & fallbacks
|
|
||||||
- **AI SDK v5 API churn** → pin exact versions; wrap all SDK calls in `packages/ai` so the app never imports `@vercel/ai` directly.
|
|
||||||
- **Local model quirks** (non-standard `/v1` paths, missing streaming) → tolerate missing `stream` by falling back to non-streaming; document required Ollama flags (`--jinja`, etc.).
|
|
||||||
- **Tokenizer drift** → gpt-tokenizer is the source of truth; re-run token tests when the model family changes.
|
|
||||||
-71
@@ -1,71 +0,0 @@
|
|||||||
# CHUNK #9 — AI Roles & Suggestions
|
|
||||||
|
|
||||||
**Duration:** ~3 days · **Prereqs:** CHUNK #8 (providers, token budget), CHUNK #4 (entities), CHUNK #3 (editor edit API) · **Follows:** CHUNK #8 · **Unblocks:** CHUNK #11 (parity: AI works in web + desktop identically)
|
|
||||||
|
|
||||||
**Purpose:** Bring the four agentic roles (Editor, Reader, Publisher, Co-Author) to life with robust base prompts, a context builder that pulls from the author's own data, AI SDK tool-calls, and — the signature feature — the **suggestion protocol** that renders AI feedback as inline, adoptable/rejectable/pinnable edits anchored to exact passages.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Steps
|
|
||||||
|
|
||||||
### 1. Base role prompts (`packages/ai/src/roles/`)
|
|
||||||
1. **Role type** (`packages/ai/src/roles/types.ts`): `Role = 'editor' | 'reader' | 'publisher' | 'co-author'`. Each has a `systemPrompt`, `temperature` default, and `tools[]`.
|
|
||||||
2. **Editor** (line editor): tighten prose, kill filter words, sharpen dialogue tags, fix pacing, flag weak openings/closings, check consistency against character/world data. Emits targeted structured edits with rationale. Respects the author's voice.
|
|
||||||
3. **Reader** (honest first-time reader): what was confusing, where attention dropped, what was telegraphed too early, what paid off. Reports as a reader, not in craft jargon.
|
|
||||||
4. **Publisher** (market & packaging): comps, hook, pacing vs. genre, title/cover/blurb ideas, series potential, format decisions.
|
|
||||||
5. **Co-Author** (collaborator with full project knowledge): drafts scenes/chapters in the author's voice, proposes plot turns, writes dialogue, expands world-building — always as *suggestions* the author accepts or rejects, never silently rewriting canon.
|
|
||||||
6. Prompts live in `docs/ROLE_PROMPTS.md` as the human reference; `packages/ai/src/roles/prompts.ts` imports them.
|
|
||||||
|
|
||||||
### 2. Context builder (`packages/ai/src/context.ts`)
|
|
||||||
1. `buildContext(project, { targetDoc, role, maxTokens })`:
|
|
||||||
- Pull relevant characters (mentioned in target doc + top entities), world items, linked plot beats, prior chapter summaries.
|
|
||||||
- Pack in priority order until the context budget (Chunk 8) is reached.
|
|
||||||
- Return `{ system, contextText, sources: {entityId, title}[] }` — `sources` powers the "what I used" panel.
|
|
||||||
2. History: last N messages from the `AISession`, truncated to the history budget.
|
|
||||||
|
|
||||||
### 3. Tools (`packages/ai/src/tools/`)
|
|
||||||
AI SDK tool-calls so a role *acts* on the workspace:
|
|
||||||
1. `applySuggestion({ target, before, after, rationale, kind })` — registers a suggestion (Chunk 5 model).
|
|
||||||
2. `updateEntity({ id, fields })` — edits a character/world field (pending author approval in UI).
|
|
||||||
3. `addNote({ body, linkedTo })` — creates a note.
|
|
||||||
4. `draftChapter({ title, outline })` — seeds a chapter doc.
|
|
||||||
5. `queryProject({ q })` — search entities/chapters (returns snippets for the role).
|
|
||||||
|
|
||||||
### 4. Suggestion protocol (the signature feature)
|
|
||||||
1. **Structured output**: roles emit JSON `{ target: {docId, range|nodePath}, before, after, rationale, kind: 'replace'|'insert'|'delete'|'entity-update'|'draft', status: 'pending' }`. Enforce with the provider's JSON-mode + a local zod schema validator (retry on parse failure).
|
|
||||||
2. **Inline rendering** (`packages/editor/src/decorations/suggestions.ts`): render each pending suggestion as an inline decoration over the exact passage — green for insert, yellow for replace, red for delete. Range is a **node anchor**, not a raw offset (re-resolved on every transaction — prevents drift, Chunk 3 risk).
|
|
||||||
3. **Actions**: hover a decoration → **Accept** (applies the edit via `applyEdit`), **Reject** (removes it), **Pin** (copies it into a pinned note for later). Panel lists all pending suggestions with rationale + "what I used" sources.
|
|
||||||
4. **Co-Author drafts**: render as a side-by-side diff view (original vs. draft) with accept/reject per paragraph.
|
|
||||||
|
|
||||||
### 5. Session UI (`apps/web/src/features/ai/`)
|
|
||||||
1. **AI panel** (`AiPanel.tsx`): role selector, provider/model picker, temperature/thinking/context sliders, "Ask" button, streaming response view, pending suggestions counter.
|
|
||||||
2. **Sessions** (`AiSessionsView.tsx`): list past sessions (from `AISession`), reopen to continue, export session transcript.
|
|
||||||
3. **Feedback loop**: after accepting/rejecting, log the choice → future context can deprioritize rejected styles (v1: simple counter, not ML).
|
|
||||||
|
|
||||||
### 6. Tests
|
|
||||||
1. Unit: `buildContext` packs the right entities within budget; `sources` populated.
|
|
||||||
2. Unit: zod schema rejects malformed suggestion JSON.
|
|
||||||
3. Unit: node-anchor resolution survives a doc edit (insert before range → range shifts correctly).
|
|
||||||
4. E2E: start an Editor session → suggestion appears inline → Accept → text changes → suggestion cleared; Pin → note created.
|
|
||||||
5. E2E: Co-Author draft → diff view → Reject → original intact.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Acceptance criteria (all must pass)
|
|
||||||
- [x] All four roles respond with role-appropriate output; base prompts enforced.
|
|
||||||
- [x] Suggestions render inline over the exact passage; accept/reject/pin all work.
|
|
||||||
- [x] Accepting applies the edit; rejecting removes it; Pin creates a note.
|
|
||||||
- [x] "What I used" lists the entities/context the role drew from.
|
|
||||||
- [x] Node-anchored ranges stay correct across typing (no drift).
|
|
||||||
- [x] Co-Author draft shows a paragraph-level diff; accept/reject work.
|
|
||||||
- [x] Sessions persist; reopen continues the conversation.
|
|
||||||
- [x] Contract suite still green on both adapters.
|
|
||||||
- [x] **Web UI integration** (`apps/web/src/features/ai/AI.tsx`): the AI session view is wired to `AIStore` — session sidebar (create/rename/delete/reopen), messages/suggestions/notes tabs, and full CRUD. `apps/web/src/routes/AI.tsx` now renders this feature view.
|
|
||||||
|
|
||||||
## Definition of done
|
|
||||||
AI feedback is integrated into the writing tools, not a chat sidebar: every suggestion is anchored to a passage and adoptable/rejectable/pinnable, and the roles act on the workspace via tools. This is the signature PubScript feature.
|
|
||||||
|
|
||||||
## Risks & fallbacks
|
|
||||||
- **Range drift over live editing** → node anchors, not raw offsets; re-resolve on every transaction (built into Chunk 3's decoration pipeline).
|
|
||||||
- **Scope creep in roles** → roles are prompt + tools, not new UI surfaces; the suggestion protocol is the single integration point.
|
|
||||||
- **LLM JSON reliability** → strict zod validation + retry; fall back to a structured template if the model refuses JSON.
|
|
||||||
@@ -1,47 +0,0 @@
|
|||||||
# CHUNK #9 — AI Roles & Suggestions
|
|
||||||
|
|
||||||
**Status:** In progress. Core AI session storage layer implemented and tested.
|
|
||||||
|
|
||||||
## Progress
|
|
||||||
|
|
||||||
- [x] **AI session storage layer** (`packages/ai/src/`)
|
|
||||||
- [x] `agent.ts` — AgentController: streaming orchestration + token budget enforcement
|
|
||||||
- [x] `suggestions.ts` — SuggestionBuilder: AI suggestions across brainstorm/outline/expand/tighten/continue/research/dialogue/worldbuilding
|
|
||||||
- [x] `notes.ts` — NoteBuilder: note builders for writing assistance
|
|
||||||
- [x] `ai-session.ts` — AIStore: IndexedDB session persistence via Dexie (sessions, messages, suggestions, notes)
|
|
||||||
- [x] `suggestion-mutation.ts` — SuggestionMutation: apply/insert/replace/delete logic
|
|
||||||
- [x] `index.ts` — package exports
|
|
||||||
- [x] `dexie` dependency added to `packages/ai/package.json`
|
|
||||||
|
|
||||||
## Bug fixes
|
|
||||||
|
|
||||||
- [x] **`listSessions` sort tiebreaker flaky in full suite** — `packages/core/src/ids.ts` `createUlid` used the non-monotonic `ulid` function, so ULIDs weren't monotonic. The `listSessions` sort uses ULID comparison as a tiebreaker when `updatedAt` timestamps are equal; non-monotonic ULIDs caused the test to pass in isolation but fail in the full suite. Fixed by switching `createUlid` to `monotonicFactory` from the `ulid` package.
|
|
||||||
- [x] **`mergeNotes` test used wrong note id** — test used `note2.notes[0]!.id` (same as note1's id) instead of `note2.notes[1]!.id`, causing the merge to combine note1 with itself. Fixed the test.
|
|
||||||
- [x] **`outline` system prompt missing context** — test "includes the context in the system prompt" expected the outline prompt to include `setting` and `characters`, but the outline prompt only included `chapterTitle` and `previous`. Added setting and characters to the outline prompt.
|
|
||||||
- [x] **Lint errors** — removed unused imports: `createInMemoryPersistence`, `NoteKind`, `AINote`, `AISuggestion`.
|
|
||||||
|
|
||||||
## Test results
|
|
||||||
|
|
||||||
- packages/core: 47 tests
|
|
||||||
- packages/ui: 5 tests
|
|
||||||
- packages/editor: 30 tests
|
|
||||||
- packages/ai: 71 tests
|
|
||||||
- packages/exporters: 22 tests
|
|
||||||
- apps/web: 13 tests
|
|
||||||
- **Total: 188 tests passing**
|
|
||||||
|
|
||||||
## Key facts
|
|
||||||
|
|
||||||
- Chunk #9 builds `packages/ai` as a self-contained SSE-based provider layer (NOT the Vercel AI SDK). The SDK moved v5→v4 and `@ai-sdk/openrouter` doesn't exist (404); a hand-rolled SSE client directly exercises the real deliverable (streaming orchestration, thinking-param mapping, usage tracking, 429 retry) in the mocked integration test that is the CI gate.
|
|
||||||
- The Vercel AI SDK remains the intended transport for Chunk #11 (tools).
|
|
||||||
- `packages/core/src/tokens.ts` provides `countTokens`/`allocateBudget`/`DEFAULT_BUDGET` (re-exported by `packages/ai/budget.ts`).
|
|
||||||
- `packages/ai` re-exports core's token-budget types as `Budget` (NOT `TokenBudget`).
|
|
||||||
- `packages/ai` uses Dexie for IndexedDB AI session persistence, coordinating `AIStore`, `SuggestionBuilder`, `NoteBuilder`, and `AgentController`.
|
|
||||||
|
|
||||||
## Remaining (per CHUNK#9.md)
|
|
||||||
|
|
||||||
- Base role prompts (`packages/ai/src/roles/`): editor/reader/publisher/co-author
|
|
||||||
- Context builder (`packages/ai/src/context.ts`)
|
|
||||||
- Tools (`packages/ai/src/tools/`): applySuggestion, updateEntity, addNote, draftChapter, queryProject
|
|
||||||
- Suggestion protocol: structured output + inline rendering in editor decorations
|
|
||||||
- Session UI (`apps/web/src/features/ai/`): AiPanel, AiSessionsView, feedback loop
|
|
||||||
Reference in New Issue
Block a user