Files
xop/HANDOFF.md
T
2026-10-02 18:25:29 +08:00

24 KiB
Raw Blame History

xop · Session Handoff

Pick up here in a new session at /Users/xs/Documents/xop. This doc tells you exactly where the project is, what's done, what's not, and what to do next.


TL;DR for the next session

cd /Users/xs/Documents/xop

# 必须先编译
cargo check --all-targets

# 单元测试(62 个,全跑;无 env 前置)
cargo test --lib

# 集成测试(15 个,需要 XOP_AUTH_BYPASS=1 + XOP_VAULT_PASSWORD=testpass)
# 注意:XOP_VAULT_PASSWORD 旁路只在 `--features test-bypass` 构建中有效;
# release binary 永远不读这个 env。cargo test --test integration 不带 feature 会失败。
XOP_AUTH_BYPASS=1 XOP_VAULT_PASSWORD=testpass \
    cargo test --features test-bypass --test integration -- --test-threads=1

# Smoke(默认 release build 不带 test-bypass feature)
cargo build --release
./target/release/xop            # banner
./target/release/xop help       # 完整帮助
./target/release/xop --dry-run --secret DB_PASS=db_pass --allow-net=db.local:5432 --deny-net=* -- echo hi
./target/release/xop run --secret SSHPASS=s11_pass --allow-net=192.168.1.19:22 --deny-net='*' -- \
    sshpass -e ssh -o StrictHostKeyChecking=accept-new [email protected] 'ls /Users'

v0.2.1 status (2026-10-02): working CLI + local vault + real Touch ID gate (LAContext biometrics via JXA — cancel = refuse exit 77, no system password ever; auth::Gate decision: local-biometric/headless → keyring silent, SSH & no-biometric & Linux-interactive → xop passphrase typed, keyring unused) + xop run + xop://KEY argv placeholders + file-keyring passphrase-free unlock (~/.xop/keyring, 0600 — NOT macOS Keychain, rejected after live test popped system dialogs) + real stdout/stderr leak masking (12****【xop禁止将密钥泄漏】) + partial-mask set echo + xop key [--save|--remove]. No external backend dependencies. The single most impactful next step: NetworkGuard / FileGuard OS-level enforcement (pf/iptables/sandbox-exec/landlock) + Askpass injection strategy (P1 below).


1. What's done

v0.2.1 additions (2026-10-02 — user live-test drove 4 fixes)

Layer Status File
set value no longer echoes plaintext (partial mask ab****) ✅ complete src/commands/set.rs (rpassword password_feedback_partial_mask)
File keyring unlock — ~/.xop/keyring 0600 hex key; OS gate runs first and its Gate verdict decides eligibility (biometric pass or headless → silent keyring; SSH/no-biometric → typed passphrase, keyring unused); agent-friendly xop run with zero passphrase ✅ complete + unit + integration tested src/vault/keyring.rs + src/vault/mod.rs::open
`xop key [--save --remove]` subcommand (status / enable on old vaults / revoke) ✅ complete
Real stdout leak masking (old guard_for_reader was a passthrough stub — that's why raw 123456 reached the user's terminal). Line-buffered filter, mask 12****【xop禁止将密钥泄漏】, kill/refuse → SIGKILL, audit leak_event ✅ complete + 8 unit + 2 e2e tests src/guard/stdout.rs
xop reset --yes cascades keyring deletion ✅ complete src/commands/reset.rs
OS gate rewritten to REAL fingerprint: LAContext .deviceOwnerAuthenticationWithBiometrics via JXA (biometrics-only — dialog has no password field). LAError classification: cancel/refuse (-128/-5/-12/unknown) → AuthCancelled exit 77 (old code degraded to OK and xop set stored the secret after "用户已取消") ✅ complete + 8 unit tests src/auth/touchid.rs
auth::Gate decision table (live-test #2 fix): authenticate now returns AllowKeyring / RequirePassphrase. Headless(non-TTY, incl. ssh host cmd) → AllowKeyring; SSH pty session → never attempts biometrics (dialog would pop on the local console), requires typed xop passphrase, keyring NOT consulted; local no-biometrics / broken osascript / Linux interactive → same RequirePassphrase. Rationale: silent keyring unlock needs an identity proof behind it. Linux pam.rs deleted (pkexec system password was the old bug) ✅ complete + 5 unit decide tests + 1 integration src/auth/mod.rs::decide + src/vault/mod.rs::open
macOS Keychain approach: implemented, then REJECTED & deleted — security(1) popped system authorization dialogs during background automation (2026-10-02 user veto: 不要放在钥匙串). xop must NEVER call security(1) against the user's keychain ❌ removed (git history only)

v0.2 (2026-10-01)

Layer Status File
Cargo.toml + deps (no backend crates) ✅ complete Cargo.toml
CLI arg surface (clap derive, Subcommand enum) ✅ complete src/args.rs
SecretRef parsing (bare KEY=NAME + argv_placeholder flag) ✅ complete + unit tested src/secret.rs
Typed XopError + exit code mapping (BSD sysexits) ✅ complete src/error.rs
Policy (NetPolicy / ReadPolicy / LeakAction) ✅ complete + unit tested src/policy.rs
Default deny-read list (pre-seeded) ✅ complete src/policy.rs::DEFAULT_DENY_READ
Local encrypted vault (AES-256-GCM, argon2id, atomic write) ✅ complete + unit tested src/vault/{mod,format}.rs
OS auth gate (v0.2 shipped osascript-admin + pkexec; rewritten in v0.2.1 to LAContext biometrics-only via JXA + Gate eligibility for keyring; SSH & Linux-interactive & no-biometric → typed passphrase — see table above) ✅ complete src/auth/{mod,touchid}.rs
6 subcommands (init/set/list/get/rm/reset) ✅ complete + integration tested src/commands/{init,set,list,get,rm,reset}.rs
xop run subcommand (envp inject + xop://KEY argv substitution; single vault open) ✅ complete + live SSH-verified src/lib.rs::run_join_and_exec + src/injector.rs::scrub_argv
TTY-aware passphrase prompt (/dev/tty for real terminal, stdin fallback for agent/pipe/expect) ✅ complete src/vault/mod.rs::rpassword_prompt
test-bypass Cargo feature (gates XOP_VAULT_PASSWORD env to test builds only) ✅ complete Cargo.toml::[features] + src/vault/mod.rs cfg-gate
Audit log (ref-only, JSONL, invariant test scans 4 keywords) ✅ complete src/audit.rs
secrecy::SecretBox<String> wrapper for values ✅ complete src/secret.rs::ResolvedValue
StdoutGuard (filter + pipe loop, async tokio) ✅ complete + unit tested src/guard/stdout.rs
NetworkGuard / FileGuard (interface) ✅ interface complete, OS impl stubbed src/guard/{network,file}.rs
Injector plan + build_envp_delta (real impl, skips argv_placeholder refs) ✅ complete src/injector.rs
Child env hardening (HISTFILE=/dev/null etc.) ✅ complete src/injector.rs::harden_child_envp
Spawn pipeline (tokio + pre_exec + RLIMIT_CORE=0) ✅ complete src/lib.rs::spawn_child
Dry-run mode ✅ complete src/lib.rs::run_dry
Design / threat-model / comparison / architecture docs ✅ complete docs/*.md
Examples (init, list, get, ssh, mysql, dry-run) ✅ complete examples/*.sh
Integration tests (assert_cmd) ✅ 11 tests, all green (with --features test-bypass) tests/integration.rs
Makefile (build / install / uninstall / test / clippy) ✅ complete Makefile
secret-agent cloned as reference ✅ complete refs/secret-agent/

2. What's NOT done

🟡 P1 — Important but not blocking v0.2

Item Where Effort Notes
NetworkGuard OS impl (pf / iptables) src/guard/network.rs::setup 2-3 hours macOS: Command::new("pfctl") + stdin pipe, anchor = xop/<pid>. Linux: iptables -m owner --pid-owner <pid> in OUTPUT chain.
FileGuard OS impl (sandbox-exec / landlock) src/guard/file.rs::setup 2-3 hours macOS: build_sandbox_profile exists; need to invoke sandbox-exec -p <profile>. Linux: prctl(PR_SET_NO_NEW_PRIVS) + landlock ruleset before execve.
Askpass injection strategy src/injector.rs 1-2 hours Write /tmp/.xop-askpass-$$ chmod 700, body echo "$XOP_SSH_PASSWORD". Set SSH_ASKPASS + SSH_ASKPASS_REQUIRE=force + DISPLAY=:0. srm -m + rm on child exit.
Fd injection strategy src/injector.rs 30 min Lower priority — for tools that read password from fd 3.
Argv scrubber src/injector.rs::scrub_argv 1 hour Detect --password=<value>, -p<password>, PASSWORD=... patterns in user's argv.
Signal forwarding (SIGINT/SIGTERM) src/lib.rs::spawn_child 30 min Command::kill_on_drop(true) already; need separate signal handler task to forward SIGINT/SIGTERM.
Entropy-based stdout detection src/guard/stdout.rs 4-6 hours High-entropy string match (gitleaks-style) for unknown leaks that evade name-based regex.

🟢 P2 — Nice to have

Item Where Notes
Library mode src/lib.rs pub use xop::exec for embedding in xstudio later.
Passphrase strength meter (zxcvbn) src/commands/init.rs Warn or refuse weak passphrases during init.
Ref-to-hash translation in history src/lib.rs So shell history doesn't reveal which vault items are in use.
macOS Keychain as passphrase-source — REJECTED 2026-10-02 (user veto after live test: security(1) popped system authorization dialogs mid-automation). Replaced by file keyring ~/.xop/keyring. Do NOT re-litigate.

3. Key design decisions (v0.2 pivot)

Decision Why Source
Local vault, no external backends Avoid per-resolve subprocess (op read, vault kv get) which leaks vault metadata into ps. Single binary, works on fresh machine. this doc, docs/design.md §2 P5
OS auth gate (real Touch ID, biometrics-only) + passphrase = 2FA; keyring silent-unlock only where identity is proven Biometric alone never decrypts; passphrase alone never reaches the vault. Dialog shows NO password option; cancel = exit 77 refusal; never the system login password (user hard rule 2026-10-02). SSH sessions & no-biometric devices must type the xop passphrase — keyring unused there; headless automation may use keyring silently. docs/design.md §7, docs/threat-model.md T11, src/auth/{mod,touchid}.rs
Whole-file single blob (TOML in AES-GCM) Simple. Atomic write. Re-randomized nonce per save. docs/architecture.md
op:///vault:///keychain:// schemes removed v0.1 leaked which backend was in use (metadata). v0.2 = bare key names only. src/secret.rs
Master key in vault file (decision 9A) Simpler than separate keystore. Argon2(passphrase) provides the key; OS auth is just a pre-gate. docs/design.md §7
Deno-style permission DSL --allow-net=, --deny-net=. Battle-tested in Deno runtime. docs/design.md §3
Audit log: NO value/plaintext/ciphertext/passphrase field Type-system + scan test enforce it. Invariant I2. src/audit.rs
SecretBox<String> for all values Debug impl prints <secret>. Prevents accidental println!("{:?}", v) leaks. src/secret.rs
pre_exec for RLIMIT_CORE=0 Only way to set rlimit on a tokio::process::Command child (runs after fork, before execve). src/lib.rs::spawn_child, docs/architecture.md
XOP_AUTH_BYPASS=1 for tests only Skip OS auth in CI/headless. Passphrase still required (invariant I8). src/auth/mod.rs
Atomic vault writes (tmp + rename) Survive partial writes / power loss. src/vault/format.rs::write
XOP_VAULT_PASSWORD is feature-gated Reading this env from a release binary is a backdoor — any process with the env var decrypts the whole vault. Compiled out of release via #[cfg(feature = "test-bypass")]. Verified by strings target/release/xop | grep XOP_VAULT_PASSWORD → empty. Cargo.toml::[features] + src/vault/mod.rs
xop run opens vault exactly once --secret env-inject refs and xop://KEY argv placeholders are both SecretRefs (differentiated by argv_placeholder flag), collected by collect_refs and passed to a single resolve_all call. User types passphrase once. src/lib.rs::run_join_and_exec + src/secret.rs::SecretRef::argv_placeholder
TTY-aware passphrase prompt is_terminal(stdin) decides: real terminal → /dev/tty (so xop set still reads the secret value from stdin); non-TTY (agent/pipe/expect) → stdin fallback. No env-var backdoor needed. src/vault/mod.rs::rpassword_prompt

4. How to test

cd /Users/xs/Documents/xop

# 1. Unit tests (fast, run often, no env needed)
cargo test --lib
# Expected: 62 tests pass (args, secret, policy, audit, vault::format, vault::mod, vault::keyring, auth(touchid LAError 分类 ×8 + decide 决策表 ×5), stdout-guard, error, injector).

# 2. Integration tests (need built binary + bypass env + test-bypass feature)
#    XOP_VAULT_PASSWORD 旁路只在 --features test-bypass 构建中有效
XOP_AUTH_BYPASS=1 XOP_VAULT_PASSWORD=testpass \
    cargo test --features test-bypass --test integration -- --test-threads=1
# Expected: 15 tests pass (help, banner, dry-run, bad-ref, secrets-file, env, init-vault, get-unknown, list-empty, run_masks_secret_in_child_stdout, run_masks_argv_placeholder_value, keyring_enables_passphrase_free_unlock, ssh_env_headless_still_unlocks_via_keyring).

# 3. Combined
cargo test --lib && \
    XOP_AUTH_BYPASS=1 XOP_VAULT_PASSWORD=testpass \
    cargo test --features test-bypass --test integration -- --test-threads=1

# 4. Live smoke — 必须真人跑,两种终端各测一遍
cargo build --release
#    (a) 本机 Terminal.app:
./target/release/xop init    # 不弹指纹(bootstrap);passphrase ×2 后自动写 keyring
./target/release/xop set DB_PASS                # 弹真指纹 → 通过 → 交互输入 secret value(不再问口令)
./target/release/xop list    # 弹指纹 → keyring 静默解锁
#    (b) SSH 会话(ssh <本机用户>@<host>):
./target/release/xop list    # 绝不弹指纹 → 提示输入 Vault passphrase(管理员密码)→ 成功
./target/release/xop set aa  # 同上:先输口令再输入值;点取消口令 = 失败
./target/release/xop --dry-run --secret DB_PASS=db_pass --allow-net=db.local:5432 --deny-net=* -- echo hi
./target/release/xop run --secret SSHPASS=ssh_pass \
    --allow-net=192.168.1.0/24 --deny-net='*' -- \
    sshpass -e ssh user@host 'ls /Users'

# 5. 验证 release binary 没有 XOP_VAULT_PASSWORD 旁路(应空)
strings target/release/xop | grep XOP_VAULT_PASSWORD

5. Common pitfalls for the next session

  1. bash expands $VAR — xop ssh user:$SSH_PASSWORD@host gets expanded by bash before xop sees it. Always use --secret KEY=NAME flag.

  2. rpassword::prompt_password requires a real TTY — can't be called from CI or piped stdin. Current rpassword_prompt is TTY-aware: stdin is TTY → /dev/tty (hidden), stdin is NOT TTY → stdin fallback (no env-var backdoor). Integration tests run cargo test --features test-bypass to enable the XOP_VAULT_PASSWORD env bypass.

  3. macOS Touch ID = osascript -l JavaScript (JXA) 调 LAContext,不是 admin 弹窗 — 旧写法 do shell script "..." with administrator privileges 是系统登录密码框(红线禁止),且 osascript 命中 admin 授权缓存造成"看起来免密"的假象。正确姿势:ctx.evaluatePolicyLocalizedReasonReply(1, reason, fn)(policy 1 = biometrics-only,弹窗无密码选项)+ 主线程 NSRunLoop.runUntilDate 自旋等 block 回调;JXA 会把私有队列的回调 marshal 回 JS 线程(实测 NSOperationQueue 探针 107ms)。LAContext 无缓存,每次真弹指纹。

  4. Linux 交互 = 直接输 xop passphrase(pam.rs 已删) — 旧实现 pkexec -- true 弹的是 polkit 系统登录密码框,违反"任何时候不要让用户输入系统密码"红线。Linux 没有可移植的"纯指纹"CLI 闸门(pam_fprint 罕见且发行版绑定),身份无法被生物特征证明 → decide(tty=true, ssh=false, bio=None) = RequirePassphrase,keyring 也不参与(否则等于零验证)。headless 自动化仍走 keyring 静默。

  5. pre_exec for RLIMIT_CORE=0 — tokio::process::Command doesn't expose setrlimit. Must use cmd.pre_exec(|| setrlimit(...)) closure (runs after fork, before execve). Pattern in src/lib.rs::spawn_child.

  6. Secret<String> in secrecy 0.10 = SecretBox<String> — don't follow 0.8 docs.

  7. stdout mask uses Vec<u8> not String — child stdout may be UTF-8 invalid (binary data); bytes avoid from_utf8 panic.

  8. audit log flush before child exits — AuditLog::write_event calls .flush(). If child crashes mid-exec, audit isn't lost.

  9. osascript -l JavaScript 调用姿势 — 用 Command::new("/usr/bin/osascript").arg("-l").arg("JavaScript").arg("-e").arg(JXA_SCRIPT).arg(reason):reason 走 argv 而非字符串拼接,完全没有 shell 引号转义问题。JXA 里 enum 名(LAPolicyDeviceOwnerAuthenticationWithBiometrics)不一定桥接成功,用数字 1 稳定。详见 src/auth/touchid.rs::JXA_SCRIPT。

  10. Argon2 parameters — m=64 MiB, t=3, p=4 is standard. Don't tune down (defeats offline brute force). Each xop invocation pays ~250ms per unlock.

  11. Vault file is per-user — ~/.xop/key.db resolves via $HOME. CI tests must set HOME to a temp dir before running, which is why --test-threads=1 matters.

  12. XOP_VAULT_PASSWORD 是 release 后门,必须 feature-gated — v0.2 早期版本把它当 CI/agent 旁路直接读 env,等于"任何有 env 的进程都能解密 vault"。当前用 #[cfg(feature = "test-bypass")] 包住;release 永远不编译这段。校验手段:strings target/release/xop | grep XOP_VAULT_PASSWORD 必须空。

  13. TTY-aware passphrase prompt — is_terminal(stdin) 决定走 /dev/tty 还是 stdin fallback。如果用 /dev/tty 文件存在性检测会误判(SSH 下文件存在但不可用)。用 std::io::IsTerminal(POSIX isatty(fd))才是真信号。

  14. vault 不要开两次 — 早期 xop run 内部先 expand_xop_placeholders 开一次 vault,再 run_exec 开一次,导致用户输两次 passphrase。修复:让 expand_xop_placeholders 只收集名字(不开 vault),把 placeholder 加进 SecretRef::argv_placeholder=true 的列表,让 run_exec::collect_refs 一次性带去 resolve_all。原则:vault 整个进程最多 open 一次。

  15. SecretRef::argv_placeholder 区分两种注入 — --secret KEY=NAME (envp) vs xop://KEY (argv)。两者走同一个 vault resolve,但 build_envp_delta 跳过 placeholder,scrub_argv 处理 placeholder。ResolvedValue 镜像标志位。

  16. 绝不调 macOS Keychain / security(1) — 实测:agent 后台会话里 security add/find/delete-generic-password 会弹系统授权对话框("xop wants to modify/delete keys"),用户体验等同攻击,整个 keychain 方案 2026-10-02 被否决删除。免密解锁 = ~/.xop/keyring 0600 文件(存 argon2 派生密钥 hex,非 passphrase 本身;mode 不合规直接拒用)。OS 预闸门顺序写死在 keyring 读取之前,且闸门结论决定 keyring 资格——本地指纹通过或 headless 才免输;SSH/无指纹交互必须输口令,keyring 根本不读。

  17. lib 单元测试禁止 set_var 全局 env — cargo test --lib 默认并行;keyring 测试曾用 set_var("HOME", tmpdir) + set_var("XOP_NO_KEYRING"),4 个测试随机红(别的测试同时读到被改的 HOME)。解法:文件逻辑抽成路径显式的 save_at/load_at/remove_at(&Path) 纯函数,wrapper 才解析 path()+env 门控;测试只打 tempdir 路径。决策逻辑同理抽成 auth::decide(tty, ssh, bio) 纯函数,测 SSH/headless 分支不碰 env。(集成测试单进程串行 + 子进程 env 隔离,set_var HOME 仍安全,保持 --test-threads=1。)

  18. SSH 会话 isatty(0) 也是 true——TTY ≠ 本地控制台 — sshd 分配的 pty 让 isatty(0)/is_terminal() 返回 true,旧闸门据此在 SSH 会话里调 LAContext:Touch ID 弹窗只会出现在本机物理控制台,远程用户看不到也点不了,等于卡死/错弹(2026-10-02 实测抱怨"ssh 状态也让输入指纹")。检测用 SSH_TTY/SSH_CONNECTION/SSH_CLIENT env(sshd 注入,本地 Terminal.app 没有)。语义:SSH 交互会话 → Gate::RequirePassphrase(输 xop 管理员密码,keyring 不参与——没有指纹现证就没有任何身份验证垫底);ssh host cmd 无 pty 的自动化 → 仍 headless → keyring 静默(xop run 无人值守不受影响)。判断顺序 !tty → ssh → bio,headless 检查必须在 ssh 之前。


6. Where to look first (file map)

Question File
How is the CLI parsed? src/args.rs
What does a ref look like (incl. argv_placeholder flag)? src/secret.rs
What does the policy look like? src/policy.rs
How is the vault encrypted? src/vault/mod.rs + src/vault/format.rs
How does the fingerprint gate work (LAContext via JXA; Outcome/classify)? 闸门结论怎么决定 keyring 能否静默(Gate/Bio/decide(tty,ssh,bio) 纯决策表:headless→静默 · SSH/无指纹→强制输 passphrase · 取消→exit 77)? src/auth/mod.rs + src/auth/touchid.rs
What's in each subcommand (incl. reset, key)? src/commands/{init,set,list,get,rm,reset,key}.rs
How does passphrase-free unlock work (file keyring, NOT Keychain)? src/vault/keyring.rs + src/vault/mod.rs::open(顺序:OS gate → Gate::AllowKeyring 才读 keyring;RequirePassphrase 直接手输,keyring 不参与)
How does xop run coordinate env-inject + xop:// placeholders in one vault open? src/lib.rs::run_join_and_exec + expand_xop_placeholders
How does argv placeholder substitution work? src/injector.rs::scrub_argv
What guards run around the child? src/guard/mod.rs + src/guard/{stdout,network,file}.rs
Where's the audit invariant enforced? src/audit.rs (test scans 4 keywords)
How is the child spawned? src/lib.rs::spawn_child
How is stdout filtered? src/guard/stdout.rs::pipe_filter_loop
How does passphrase prompt choose /dev/tty vs stdin? src/vault/mod.rs::rpassword_prompt
Where is XOP_VAULT_PASSWORD test bypass gated? Cargo.toml::[features] + src/vault/mod.rs #[cfg(feature = "test-bypass")] block
What's the overall flow? docs/architecture.md
What's the threat model? docs/threat-model.md
How does this compare to existing tools? docs/comparison.md

7. Next-step path (P1 work, 1-2 sessions)

If you have 2-3 hours and want the NetworkGuard wired up:

  1. Implement NetworkGuard::setup for macOS (pfctl + anchor) and Linux (iptables + owner match).
  2. Test: launch xop --allow-net=db.local:5432 --deny-net=* -- curl -s db.local:5432 and verify it succeeds; xop ... -- curl -s evil.com should fail.
  3. Implement FileGuard::setup for macOS (sandbox-exec) and Linux (landlock).
  4. Askpass strategy for SSH: write askpass script, set SSH_ASKPASS_REQUIRE=force + DISPLAY=:0, srm -m on exit.

That gets v0.2 to a fully-enforced posture (XIP, FS, network all live). Phase 2 adds entropy detection, signal forwarding, library mode.


8. Future: connecting to xstudio

xop is standalone. When (if) xstudio wants xop as its secrets layer:

  • xstudio already has a xstudio.ai.runAgentLoop MCP tool.
  • Add a new MCP tool xstudio.secrets.exec that calls xop::run(args) internally.
  • Or expose xop as a CLI that xstudio's agent runtime shells out to.
  • The library mode item in P2 is exactly for this — pub use xop::exec for embedding.

For now xop doesn't know xstudio exists. Keep it that way until there's a concrete integration ask.


9. Quick reference

What Where
Project root /Users/xs/Documents/xop
Reference impl refs/secret-agent/
Design docs docs/*.md
CLI entry src/main.rs
Orchestrator src/lib.rs::run
Vault module src/vault/mod.rs
Audit invariant test src/audit.rs::tests::no_value_field_in_any_event
Tests require integration: --features test-bypass build + XOP_AUTH_BYPASS=1 XOP_VAULT_PASSWORD=testpass