Add PubScript master plan and 11 chunk work orders

2026-10-23 14:26:06 -05:00
commit 53f4c5b4c7
13 changed files with 1086 additions and 0 deletions
@@ -0,0 +1,78 @@
+# CHUNK #8 — AI Provider Layer
+
+**Duration:** ~2 days · **Prereqs:** CHUNK #2 (storage, `packages/core` token budget) · **Follows:** CHUNK #7 · **Unblocks:** CHUNK #9 (roles + suggestions build on this)
+
+**Purpose:** Build the provider abstraction that talks to any OpenAI-compatible endpoint (local Ollama/llama.cpp/LM Studio/vLLM) plus Anthropic, Gemini, and OpenRouter via the Vercel AI SDK v5 — with streaming, thinking/reasoning params, and the token-budget engine that keeps every prompt inside the context window.
+
+---
+
+## Steps
+
+### 1. Provider config (`packages/ai/src/providers/`)
+1. **Provider type** (`packages/ai/src/providers/types.ts`):
+   ```ts
+   interface ProviderConfig {
+     id: string;
+     kind: 'openai-compatible' | 'anthropic' | 'google' | 'openrouter';
+     baseUrl?: string;      // e.g. http://localhost:11434/v1, https://api.openai.com/v1
+     apiKey?: string;       // optional for local
+     model: string;
+     temperature: number;
+     maxTokens: number;
+     thinking?: { enabled: boolean; budget?: number; effort?: 'low'|'medium'|'high' };
+     contextWindow: number;
+   }
+   ```
+2. **Discovery** (`packages/ai/src/providers/discover.ts`): auto-detect local providers by probing common endpoints (`http://localhost:11434/v1/models`, `http://localhost:1234/v1/models`). Show "detected local models" in settings.
+3. **Storage**: providers persisted per-user (server) or per-machine (local) in the settings store.
+
+### 2. Adapter layer (`packages/ai/src/adapters/`)
+1. **OpenAI-compatible adapter** (`openaiCompatible.ts`): uses `@ai-sdk/openai` with a custom `apiBase` + `headers`. Handles streaming (`generateText`/`streamText`) and non-streaming.
+2. **Anthropic adapter** (`anthropic.ts`): `@ai-sdk/anthropic`; maps `thinking.budget` → the SDK's `thinking` param.
+3. **Google adapter** (`google.ts`): `@ai-sdk/google` (Gemini).
+4. **OpenRouter adapter** (`openrouter.ts`): `@ai-sdk/openai-compatible` with `https://openrouter.ai/api/v1` base + special headers (`HTTP-Referer`, `X-Title`).
+5. **Factory** (`getProvider(config)`): returns the right adapter; unknown `kind` throws a clear error.
+
+### 3. Thinking / reasoning (`packages/ai/src/thinking.ts`)
+1. Normalize `thinking` across providers:
+   - o-series (OpenAI) → `reasoningEffort`.
+   - Anthropic → `thinking` budget object.
+   - Local (e.g. Qwen3 via Ollama) → extra body params / `chat_template_kwargs`.
+2. Parse streamed reasoning tokens when present (strip from final content, expose separately for debugging).
+
+### 4. Token budget (`packages/ai/src/budget.ts`)
+1. **Count** (`countTokens(text)`): wrap `gpt-tokenizer` (o200k_base), lazy-loaded. Already stubbed in Chunk 2 — flesh out with real tokenizer.
+2. **Allocate** (`allocateBudget({ system, context, history, output, total })`): returns per-slot token caps (defaults: system+role 10%, context pack 40%, history 30%, output 20%). Adjustable per provider config.
+3. **Guard** (`enforceBudget(prompt, budget)`): throws or truncates if the assembled prompt exceeds the context window; logs which slot overflowed.
+
+### 5. Streaming client (`packages/ai/src/client.ts`)
+1. `streamCompletion(provider, messages, opts)` → async iterator of `{ type: 'text'|'reasoning'|'tool-call'|'done', text? }`.
+2. `complete(provider, messages, opts)` → full text (for non-streaming tool use).
+3. Error handling: 429 → retry with backoff; auth errors → clear message; timeout → abort.
+4. **Usage tracking**: capture `promptTokens`/`completionTokens` per call → stored in `AISession` (Chunk 9) + shown in settings.
+
+### 6. Tests
+1. Unit: `countTokens` on known strings (compare to gpt-tokenizer reference).
+2. Unit: `allocateBudget` + `enforceBudget` edge cases (empty context, overflow).
+3. Unit: thinking-param normalization for each provider.
+4. Integration (mock): a fake OpenAI-compatible server returns a streamed response; adapter yields correct chunks. Use `msw` or a local Express stub.
+
+---
+
+## Acceptance criteria (all must pass)
+- [ ] Configure an OpenAI-compatible provider (local Ollama URL) → streaming works.
+- [ ] Configure OpenAI/Anthropic/Gemini/OpenRouter with API keys → streaming works (mock keys in tests).
+- [ ] Auto-detect a local model running on `:11434`.
+- [ ] `thinking` enabled → correct provider param sent; reasoning tokens parsed when present.
+- [ ] `countTokens` matches gpt-tokenizer within tolerance.
+- [ ] Prompt exceeding context window → clear error, no crash.
+- [ ] Streaming chunks arrive in order; usage tracked per call.
+- [ ] CI green (unit + mocked integration).
+
+## Definition of done
+Every provider type is reachable through one streaming client, the token budget keeps prompts in-window, and local models are auto-discovered. Chunk 9 wires roles and suggestions on top.
+
+## Risks & fallbacks
+- **AI SDK v5 API churn** → pin exact versions; wrap all SDK calls in `packages/ai` so the app never imports `@vercel/ai` directly.
+- **Local model quirks** (non-standard `/v1` paths, missing streaming) → tolerate missing `stream` by falling back to non-streaming; document required Ollama flags (`--jinja`, etc.).
+- **Tokenizer drift** → gpt-tokenizer is the source of truth; re-run token tests when the model family changes.