交付两件此前欠着的事,并在真机(192.168.1.102 海光 CSV)端到端跑通: 1. OCI 镜像流水线 build → verify → pack 三步在真机跑通,manifest digest 作 SessionBinding.code_measurement 钉进会话绑定值。 2. PSK / seed / root-key 不再内置 initrd,改由 guest 内 mpc tee kbs-get 按会话从宿主 KBS 取回(key_source=kbs)。 ★ 修掉两个框架级缺陷(都是「校验通过 ≠ 校验的是对的东西」): - guest 侧活文件链:旧 layers.sha256 只覆盖 blobs/ 下的惰性证据副本, 真正执行的 /init、/mpc 不在范围内 —— 真机改 /mpc 一个字节照样打印 oci_layers: verified。现改为从 cmdline 的 PIN 逐跳推导到活文件逐个校验, 失败时点名具体文件。 - 流水线侧自证:pack 打完 initrd 必须回读核对内嵌 digest / 清单 / manifest.json,不符即失败。缺前者不能保证「送进 guest 的是这个镜像」, 缺后者不能保证「guest 跑的是这个镜像里的活内容」,两者缺一不可。 新增真机可复现资产(deploy/tee/): - oci/oci-image.py:rootfs_state_from_layers() 直接从各层 tar 字节算活 rootfs 状态,不解包(免 mknod 权限,build 与 pack 共用同一函数防分叉); build 扫 base rootfs 里的密钥物料,命中即退出码 3。 - oci/selftest-chain.py:完整性链本地回归,含「旧口径对 /mpc 篡改完全 无感」的反向断言。 - real-machine/kbc-e2e.sh / kbc-negative.sh / kbs-net.sh:正向全链路与 5 条 fail-closed 负向矩阵、tap + bridge 出网链路。 - real-machine/tamper-initrd.py:精确解析 newc cpio 改目标文件数据区。 - real-machine/guest-syntax-check.sh:解包 initrd 后用解出来的 musl loader 跑解出来的 busybox,完全模拟 guest 内 ash 解析环境。 - real-machine/lint-shell-vars.py:shell 引号与变量名三类陷阱闸门。 框架侧:PlatformProfile.kbc 拆成 allowed_kbc(支持哪些)与 kbc(缺省); kbs_endpoint 改由节点描述给出,现场部署事实与代码分离。 真机实测:正向 KBC_E2E_OK(key_source=kbs),负向 5/5 全 fail-closed。 闸门扩到六条:ci.sh 新增 shell 引号/变量名卫生与 OCI 完整性链自测。
239 lines
9.8 KiB
Bash
Executable File
239 lines
9.8 KiB
Bash
Executable File
#!/usr/bin/env bash
|
||
# 海光 CSV 真机:OCI 镜像 + 在线 KBC(密钥由 KBS 按会话下发)端到端。
|
||
#
|
||
# 这条链路交付的是两件此前欠着的事:
|
||
# 1. OCI 镜像流水线真机复跑(build → verify → pack),manifest digest 作
|
||
# measurement 钉进会话绑定值;
|
||
# 2. PSK / seed / root-key **不再内置在镜像里**,改由 guest 内
|
||
# `mpc tee kbs-get` 按会话从宿主 KBS 取回。
|
||
#
|
||
# 网络前提(真机实测,2026-11-06):
|
||
# 海光的静态 qemu 是 kata-static 裁剪版,**没有 slirp**(`-netdev user` 直接
|
||
# 报 "Parameter 'type' expects a netdev backend type"),系统 qemu 有 slirp
|
||
# 但跑不了 CSV 机密 guest。所以 guest 访问宿主 KBS 走 **tap + bridge**:
|
||
# guest 拿 10.99.0.2/24,宿主 KBS 绑 10.99.0.1:18080,全程同网段直连。
|
||
#
|
||
# 实现要点:所有远端多步操作都写成脚本文件再 scp 过去执行 —— 经 SSH 传多行
|
||
# 命令时 `$(...)` 与换行会在多层引号里被吃掉(早先踩过:资源文件写出来只有
|
||
# 一个 "11")。
|
||
#
|
||
# 用法(在本机执行,凭据走 xop 键名):
|
||
# MPC_TEE_HYGON_USER_KEY=... MPC_TEE_HYGON_PASS_KEY=... \
|
||
# deploy/tee/real-machine/kbc-e2e.sh
|
||
set -u
|
||
|
||
HERE=$(cd "$(dirname "$0")" && pwd)
|
||
HYGON_HOST=${MPC_TEE_HYGON_HOST:-192.168.1.102}
|
||
HYGON_USER_KEY=${MPC_TEE_HYGON_USER_KEY:-tee_hg_102_user}
|
||
HYGON_PASS_KEY=${MPC_TEE_HYGON_PASS_KEY:-tee_hg_102_pass}
|
||
REMOTE_DIR=${MPC_TEE_REMOTE_DIR:-/tmp/mpc-tee-run-e2e}
|
||
LOG_DIR=${MPC_TEE_LOG_DIR:-/tmp/mpc-tee-logs}
|
||
SID=${MPC_TEE_SESSION_ID:-$(printf 'ab%.0s' $(seq 1 32))}
|
||
PK=${MPC_TEE_PUBKEY:-$(printf 'cd%.0s' $(seq 1 32))}
|
||
PORT=${MPC_KBS_PORT:-18080}
|
||
BR_IP=${MPC_KBS_BRIDGE_IP:-10.99.0.1/24}
|
||
BR_ADDR=${BR_IP%%/*}
|
||
GUEST_IP=${MPC_TEE_GUEST_IP:-10.99.0.2/24}
|
||
SSHO="-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o PreferredAuthentications=password -o PubkeyAuthentication=no -o ConnectTimeout=20"
|
||
|
||
MPC_BIN=${MPC_TEE_MPC_BIN:-}
|
||
if [ -z "$MPC_BIN" ]; then
|
||
MPC_BIN=$(cd "$HERE/../../.." && pwd)/target/x86_64-unknown-linux-musl/release/mpc
|
||
fi
|
||
[ -f "$MPC_BIN" ] || { echo "找不到 mpc 二进制: $MPC_BIN" >&2; exit 2; }
|
||
|
||
REMOTE_TOOLS=${MPC_TEE_REMOTE_TOOLS:-/tmp/mpc-tee-tools}
|
||
REMOTE_ALPINE=${MPC_TEE_REMOTE_ALPINE:-/tmp/alpine-root}
|
||
WORK=$(mktemp -d /tmp/mpc-kbc-e2e.XXXXXX)
|
||
trap 'rm -rf "$WORK"' EXIT
|
||
|
||
# xop 的 NetworkGuard/FileGuard 警告是本机沙箱噪音,会被插在 stdout 开头。
|
||
# 所以:**关键参数一律走「远端写文件 → scp 回本地」**,绝不从 stdout 里解析;
|
||
# stdout 只用来给人看。
|
||
NOISE='WARN NetworkGuard|WARN FileGuard|Permanently added'
|
||
|
||
remote() {
|
||
local raw rc
|
||
raw=$(mktemp)
|
||
xop run --secret "SSHPASS=$HYGON_PASS_KEY" --secret "TEE_USER=$HYGON_USER_KEY" -- \
|
||
sh -c "sshpass -e ssh $SSHO \"\$TEE_USER@$HYGON_HOST\" \"$1\"" < /dev/null > "$raw" 2>&1
|
||
rc=$?
|
||
grep -Ev "$NOISE" "$raw" || true
|
||
rm -f "$raw"
|
||
return $rc
|
||
}
|
||
|
||
scpto() {
|
||
local raw rc
|
||
raw=$(mktemp)
|
||
xop run --secret "SSHPASS=$HYGON_PASS_KEY" --secret "TEE_USER=$HYGON_USER_KEY" -- \
|
||
sh -c "sshpass -e scp $SSHO $1 \"\$TEE_USER@$HYGON_HOST:$2\"" < /dev/null > "$raw" 2>&1
|
||
rc=$?
|
||
grep -Ev "$NOISE" "$raw" || true
|
||
rm -f "$raw"
|
||
return $rc
|
||
}
|
||
|
||
# 从远端取一个文件到本地(远端 → 本地方向,与 scpto 相反)。
|
||
fetch_file() {
|
||
local remote_path=$1 local_path=$2 raw rc
|
||
raw=$(mktemp)
|
||
xop run --secret "SSHPASS=$HYGON_PASS_KEY" --secret "TEE_USER=$HYGON_USER_KEY" -- \
|
||
sh -c "sshpass -e scp $SSHO \"\$TEE_USER@$HYGON_HOST:$remote_path\" '$local_path'" < /dev/null > "$raw" 2>&1
|
||
rc=$?
|
||
grep -Ev "$NOISE" "$raw" || true
|
||
rm -f "$raw"
|
||
return $rc
|
||
}
|
||
|
||
# 本地写脚本 → scp → 远端 bash 执行。避免多层引号吃掉 $() 与换行。
|
||
remote_script() {
|
||
local name=$1 body=$2
|
||
printf '%s\n' "$body" > "$WORK/$name"
|
||
scpto "$WORK/$name" "$REMOTE_DIR/$name" || return 2
|
||
remote "bash $REMOTE_DIR/$name" || return 2
|
||
}
|
||
|
||
echo "===== 1. 分发框架与部署层脚本到真机 ====="
|
||
remote "mkdir -p $REMOTE_DIR $REMOTE_TOOLS" || exit 2
|
||
for f in "$HERE/../oci/oci-image.py" "$HERE/../kbs/serve.py" "$HERE/../guest/init.sh" \
|
||
"$HERE/run-guest.py" "$HERE/kbs-net.sh" "$HERE/guest-role.sh"; do
|
||
scpto "$f" "$REMOTE_TOOLS/" || exit 2
|
||
done
|
||
scpto "$MPC_BIN" "$REMOTE_TOOLS/mpc" || exit 2
|
||
scpto "$HERE/run-guest.py" "$REMOTE_DIR/run-guest.py" || exit 2
|
||
scpto "$HERE/kbs-net.sh" "$REMOTE_DIR/kbs-net.sh" || exit 2
|
||
scpto "$HERE/guest-role.sh" "$REMOTE_DIR/guest-role.sh" || exit 2
|
||
# kbs-net.sh 在 $REMOTE_DIR 里找同目录的 serve.py(两种布局都支持)。
|
||
scpto "$HERE/../kbs/serve.py" "$REMOTE_DIR/serve.py" || exit 2
|
||
|
||
echo "===== 2. 真机上准备 KBS 资源(按会话 id 落盘)====="
|
||
# 三类物料都是 64 位十六进制文本(与 kbs.rs 的 parse_hex_bytes 期望一致)。
|
||
remote_script prep-kbs.sh "
|
||
set -eu
|
||
RES='$REMOTE_DIR/kbs-res'
|
||
SID='$SID'
|
||
rm -rf \"\$RES\"
|
||
mkdir -p \"\$RES/mpc/psk\" \"\$RES/mpc/seed\" \"\$RES/mpc/root-key\"
|
||
hex() { printf '%s' \"\$(printf '\${1}%.0s' \$(seq 1 32))\"; }
|
||
hex 11 > \"\$RES/mpc/psk/\$SID\"
|
||
hex 22 > \"\$RES/mpc/seed/\$SID\"
|
||
hex 33 > \"\$RES/mpc/root-key/\$SID\"
|
||
wc -c \"\$RES/mpc/psk/\$SID\" \"\$RES/mpc/seed/\$SID\" \"\$RES/mpc/root-key/\$SID\"
|
||
" || exit 2
|
||
|
||
echo "===== 3. 真机上 OCI 镜像流水线(build → verify → pack)====="
|
||
remote_script build-image.sh "
|
||
set -eu
|
||
cd '$REMOTE_TOOLS'
|
||
rm -f '$REMOTE_ALPINE/psk' '$REMOTE_ALPINE/seed' '$REMOTE_ALPINE/init' '$REMOTE_ALPINE/root-key'
|
||
# 先删旧 initrd:绝不能拿上一轮的产物继续跑(pack 失败会被 '|| true' 吞掉,
|
||
# 结果 manifest.digest 是新的、initrd 是旧的 → guest 报「manifest 不一致」,
|
||
# 测的根本不是本条负向想测的东西)。
|
||
rm -f '$REMOTE_DIR/initrd.img'
|
||
python3 oci-image.py build --base '$REMOTE_ALPINE' --mpc mpc --init init.sh --out '$REMOTE_DIR/image'
|
||
python3 oci-image.py verify --image '$REMOTE_DIR/image'
|
||
# 注意:**不要**在 pack 后面接管道 | grep -v RuntimeWarning ——
|
||
# 管道的退出码取的是最后一条命令(grep)的,pack 失败会被静默吞掉,
|
||
# 旧 initrd 就会被当成新产物继续用(guest 侧表现为「manifest 不一致」,
|
||
# 而镜像本身完全正常,排查方向会被带偏)。
|
||
#
|
||
# ★ 这段注释在双引号字符串内部:**既不能用反引号**(bash 会当命令替换执行,
|
||
# 实测报 pack: command not found),**也不能用双引号**(会提前闭合外层
|
||
# 字符串,后面的 | grep 就跑到本地 shell 去了 → 实测报
|
||
# grep: bad regex)。要用引号就用「」。
|
||
python3 oci-image.py pack --image '$REMOTE_DIR/image' --out '$REMOTE_DIR/initrd.img' \
|
||
> '$REMOTE_DIR/pack.log' 2> '$REMOTE_DIR/pack.err' || {
|
||
echo 'PACK_FAILED'; cat '$REMOTE_DIR/pack.err'; exit 1; }
|
||
grep -v RuntimeWarning '$REMOTE_DIR/pack.err' || true
|
||
cat '$REMOTE_DIR/pack.log'
|
||
test -s '$REMOTE_DIR/initrd.img'
|
||
# pack 会自证(回读 initrd 核对内嵌 digest / 清单 / manifest)。
|
||
grep -q 'selfcheck=ok' '$REMOTE_DIR/pack.log' || { echo 'PACK_SELFCHECK_MISSING'; exit 1; }
|
||
cat '$REMOTE_DIR/image/manifest.digest'
|
||
" || exit 2
|
||
|
||
MANIFEST=$(fetch_file "$REMOTE_DIR/image/manifest.digest" "$WORK/manifest.digest" >/dev/null 2>&1 && \
|
||
tr -d ' \t\r\n' < "$WORK/manifest.digest")
|
||
echo "MANIFEST=$MANIFEST"
|
||
case "$MANIFEST" in
|
||
[0-9a-fA-F][0-9a-fA-F]*) [ "${#MANIFEST}" -eq 64 ] || { echo "manifest digest 长度不对" >&2; exit 2; } ;;
|
||
*) echo "manifest digest 非法: $MANIFEST" >&2; exit 2 ;;
|
||
esac
|
||
|
||
echo "===== 4. 写节点描述(kbc=online + kbs_endpoint)====="
|
||
remote_script write-node.sh "
|
||
set -eu
|
||
cat > '$REMOTE_DIR/node' <<'EOF'
|
||
site=bob
|
||
host=127.0.0.1
|
||
kind=hygon-csv
|
||
tcb=3
|
||
device_present=true
|
||
qemu_present=true
|
||
firmware_present=true
|
||
image_present=true
|
||
kbc=online
|
||
kbs_reachable=true
|
||
kbs_endpoint=http://$BR_ADDR:$PORT
|
||
measurement=$MANIFEST
|
||
qemu=/opt/confidential-containers/bin/qemu-system-x86_64
|
||
firmware=/opt/confidential-containers/share/ovmf/HYGONCSV.fd
|
||
kernel=/opt/confidential-containers/share/kata-containers/vmlinuz-5.10.134-102cc-csv-hygon-gpu
|
||
initrd=$REMOTE_DIR/initrd.img
|
||
EOF
|
||
cat '$REMOTE_DIR/node'
|
||
" || exit 2
|
||
|
||
echo "===== 5. 宿主侧出网链路 + KBS stub(tap + bridge)====="
|
||
remote_script kbs-up.sh "
|
||
set -eu
|
||
export MPC_KBS_RESOURCES='$REMOTE_DIR/kbs-res'
|
||
export MPC_KBS_SESSION='$SID'
|
||
export MPC_KBS_PORT='$PORT'
|
||
export MPC_KBS_BRIDGE_IP='$BR_IP'
|
||
bash '$REMOTE_DIR/kbs-net.sh' up
|
||
bash '$REMOTE_DIR/kbs-net.sh' status
|
||
" || exit 2
|
||
|
||
echo "===== 6. 框架 tee plan(在线 KBC 准入)+ direct boot ====="
|
||
mkdir -p "$LOG_DIR"
|
||
LOG="$LOG_DIR/hygon-kbc-e2e.log"
|
||
remote_script run-guest.sh "
|
||
set -eu
|
||
export MPC_TEE_RUN_DIR='$REMOTE_DIR'
|
||
export MPC_TEE_NODE_FILE='$REMOTE_DIR/node'
|
||
export MPC_TEE_MPC_BIN='$REMOTE_TOOLS/mpc'
|
||
export MPC_TEE_MANIFEST='$MANIFEST'
|
||
export MPC_TEE_KBC=cc_kbc::http://$BR_ADDR:$PORT
|
||
export MPC_TEE_GUEST_IP='$GUEST_IP'
|
||
export MPC_TEE_SESSION_ID='$SID'
|
||
export MPC_TEE_PUBKEY='$PK'
|
||
export MPC_TEE_KBS_REPO=mpc
|
||
export MPC_TEE_MIN_TCB=3
|
||
# 网络后端由 guest-role.sh 按 KBC 模式选(tap + script=no),这里不重复指定,
|
||
# 免得两处定义漂移。
|
||
timeout 180 bash '$REMOTE_DIR/guest-role.sh' none
|
||
" > "$LOG" 2>&1
|
||
RC=$?
|
||
GUEST_RC=$RC
|
||
echo "guest rc=$GUEST_RC"
|
||
|
||
echo "--- guest 日志关键行 ---"
|
||
# 注意把 rootfs_* / MPC_FAIL 也纳入:运行期完整性链的每一步都会打一行,
|
||
# 缺了就看不出新链到底跑到哪一步(早先就是因为过滤模式里没有它,
|
||
# 误以为新链没生效)。
|
||
grep -aE 'MPC TEE GUEST|kbs_endpoint|key_source|MPC_FAIL|oci_layers|rootfs_|manifest_digest|user_data|selfcheck|WARNING|GUEST DONE|tap' "$LOG" | head -40
|
||
|
||
echo
|
||
echo "===== 7. 判定 ====="
|
||
if grep -aq 'key_source=kbs' "$LOG" && grep -aq 'GUEST DONE' "$LOG"; then
|
||
echo "KBC_E2E_OK: guest 内密钥由 KBS 按会话下发;OCI 层校验与度量 pin 均通过"
|
||
echo "完整日志: $LOG"
|
||
exit 0
|
||
fi
|
||
echo "KBC_E2E_FAILED: 未看到 key_source=kbs / GUEST DONE(rc=${GUEST_RC})"
|
||
tail -50 "$LOG"
|
||
echo "完整日志: $LOG"
|
||
exit 1
|