feat(engine): 接入执行层与三入向端点,打通 Exec/Start 链路
- 新增 engine 模块(状态机/配置/SHA256摘要/进程拉起/状态):ACCEPTED→控制面→ET→Ray→MPC→终态 - 新增 controller/exec.rs:SelfCheckStatus/Dev/Artifact/Exec/Start + Exec/Status,handler 内复核内部令牌与协议版本 - 新增 client/mpc_control_plane.rs:GET /nodes /sessions /report 与 POST /desired,Bearer 鉴权 - 平台 nax 语义落地(rayHead 当 GCS、mpc 为算子执行体),密钥不落状态/回执/日志 - 海光控制面地址 192.168.1.102 → .23(运行态配置) - tests/exec_endpoints.rs 6 条契约用例;MIGRATION.md §1.7.1 记录范围
This commit is contained in:
1 parent
4b32b40bd6
commit
3271732e8b
15 files changed
+2584
-7
No files matched your search
@@ -197,3 +197,21 @@ curl -s --noproxy 127.0.0.1 -H 'X-Internal-Token: ssw-internal-token-e2e' -H 'X-
|
||||
提交前用 Grep 扫「部署级 token 前缀 + 门户口令字面串」应零命中。
|
||||
- 注:本日志 17:48 前有一行历史记录曾写出测试部署门户口令,本次一并脱敏(仅改凭据字面值,
|
||||
保留「用户提供测试 TDS」这一事实);自检提示行也不再复述凭据字面串,避免自触发扫描。
|
||||
|
||||
## 体积 / 内存基线(17:58,`4b32b40` 之后实测)
|
||||
|
||||
- `cargo build --release` 23.5s;产物 **40,808,480 B ≈ 38.9 MiB**(x86_64 Mach-O,
|
||||
`strip = "symbols"` 已生效,再 strip 无收益)。段构成:`__text` 18.0 MiB +
|
||||
`__const` 17.2 MiB ≈ 全部,主要是静态链进来的 PostgreSQL 原生后端;debug 46M。
|
||||
- 最小交付包 = 二进制 + `static/` ⇒ 裸 42M / **tar.gz 15M / tar.xz 10M**。
|
||||
本仓**没有** Dockerfile / package.sh,`.gitignore` 只预留 `/dist/`。
|
||||
- 单进程(进程树实测 1 个,PGlite 是进程内 backend)。RSS:冷启动建库 1.8s /
|
||||
峰值 32.7 MiB;热启动 0.2~0.4s / 空闲 **31~33 MiB**;真实 44 MiB 数据目录 +
|
||||
600 并发请求(含 DB 列表)峰值 **42.4 MiB**,之后不回落。VSZ ~33 GiB 是 macOS
|
||||
虚拟地址空间常态,别当占用读。
|
||||
- 🔴 测量坑:本会话沙箱**拦 `ps`**(`operation not permitted`,`/bin/ps` 同样)⇒
|
||||
用 `ctypes` 调 `/usr/lib/libproc.dylib` 的 `proc_pidinfo(pid, 4, ...)` 读 RSS、
|
||||
`proc_listchildpids` 枚举子进程;`/usr/bin/time -l` 可用但要包到进程退出。
|
||||
- 磁盘大头不在交付物:`vendor/sspf-v0.5.0` 452M(rlib 闭包,仅构建期)、
|
||||
`target/` 711M(debug 619M + release 91M)、`data/` 45M(运行期,会增长);
|
||||
三者均不入库,`git ls-files` 只有 151 个文件。
|
||||
@@ -0,0 +1,6 @@
|
||||
# 2026-11-06
|
||||
|
||||
- 为用户启动「官方 web + tdc-rs」供其浏览器查看:`./target/release/tdc-connector`(默认 `0.0.0.0:30000`,复用项目 `data/` 真实库,TDS 模式 NOOP)。
|
||||
- 后台任务 task `rU4ehg`;日志 `/tmp/tdc-rs-run.log`;停服用 `pkill -f target/release/tdc-connector`。
|
||||
- 登录账号 `admin / admin`(明文走 `--enc` 加密成密文,符合 cipher.rs 官方公式;`data/accounts.json` 里的 `password_md5` 只是种子文件,真实口令由首次引导写入,默认即 admin)。
|
||||
- 官方 web 构建产物在 `static/`(2026-10-25 版,含 `index-7dUXg9CU.js` 等),由 `scripts/build-frontend.sh` 从 `ssw-tdc/frontend/web` 用 `VITE_SSW_APP=tdc npx vite build` 生成;若要最新 web 需重跑该脚本(npm ci 受权限闸门影响,默认走 PREBUILT_DIST)。
|
||||
@@ -106,6 +106,28 @@
|
||||
| `controller/ApiGatewayController.java` / `ApiDocController.java` / `FileController.java` / `PortalController.java` | 网关 / 文档 / 文件 / 门户 |
|
||||
| j7 的 `base_module` / `base_authorize` / `base_organize` 等 120 张 `base_*` | 上游 `ssw.jnpf-permission.enabled=false` ⇒ 菜单授权行是死的(记 **L2**) |
|
||||
|
||||
#### 1.7.1 2026-11-13 追加:执行链路(用户口径变更,覆盖上表相关两条)
|
||||
|
||||
> 用户 2026-11-13 明确「不用 Java 版,用 tdc-rs 连接器;**不起 Pod**;平台下发 →
|
||||
> EasyTier + Ray 组网 → MPC 跑通」。上表 `engine/NaxEngineRunner` / `NaxPodSupport` /
|
||||
> `EngineJobController` / `schedule/*` 的「本仓**不实现交付执行**」是**当时能力的事实陈述**,
|
||||
> 不是产品约束,现已被推翻。**保留原表作为历史**,本节只记本批新增范围。
|
||||
> 完整研判见 `../ssw-tds-test/docs/tdc-rs-exec-chain-decision-2026-11-13.md`。
|
||||
|
||||
| 新增 | 位置 | 边界 |
|
||||
|---|---|---|
|
||||
| 三入向端点 | `controller/exec.rs` | `Internal/SelfCheckStatus` / `Dev/Artifact` / `Exec/Start`,外加验收用 `Exec/Status`;产物摘要自算(纯 Rust SHA-256) |
|
||||
| MPC 控制面出向 | `client/mpc_control_plane.rs` | `GET /nodes\|/sessions\|/report`、`POST /desired`;Bearer + 裸 JSON/204,**不复用 TDS 的 internal 头** |
|
||||
| 执行层状态机 | `engine/{mod,config,digest,process,state}.rs` | 控制面对账 → ET 组网 → Ray 计算层 → MPC 点火;异步落 `data_dir/exec/*.json`,不碰 18 实体注册表 |
|
||||
| 契约回归 | `tests/exec_endpoints.rs` | ⑦ PROBE 404→200、handler 内 fail-closed 复核、产物 List+hash、幂等、dry-run=`PLANNED` |
|
||||
|
||||
🔴 **口径内仍不做**:起 K8s Pod、Java 版 Nax 运行时、Python 函数级 `ray.put/ray.get`
|
||||
分布式计算、平台侧数据产品动态分片、在 ZYNQ(armv7/无 python3) 上跑 Ray、跨 netns 自动化。
|
||||
🔴 **不起 Pod** 时 `clusterConfig.parties[].address` 是 K8s Service DNS 死地址 ⇒ 真实端点
|
||||
必须由 `TDC_EXEC_SITE_ENDPOINTS` 显式给出,缺了 fail-closed,**不猜、不回落**。
|
||||
🔴 `Exec/Start` 的 `_code=200, ok=true` 只表示**已受理**;真实成败查 `Exec/Status`,
|
||||
`dry_run` 收口是 `PLANNED`,绝不算 `SUCCEEDED`。
|
||||
|
||||
---
|
||||
|
||||
## 2. 数据 ETL
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
//! 出向客户端(对位上游 `ssw.tdc.client`)。
|
||||
//!
|
||||
//! 目前只有 TDS 出向客户端 `tds.rs`(照抄上游 `TdsReportClient` 的 **internal 通道**)。
|
||||
//! - `tds.rs` —— TDS 出向(照抄上游 `TdsReportClient` 的 **internal 通道**)。
|
||||
//! - `mpc_control_plane.rs` —— MPC 控制面 `:3020`(Bearer 通道,裸 JSON / 204)。
|
||||
|
||||
pub mod mpc_control_plane;
|
||||
pub mod tds;
|
||||
@@ -0,0 +1,208 @@
|
||||
//! MPC 控制面出向客户端(`mpc-hostd` 的 `:3020` 控制面)。
|
||||
//!
|
||||
//! ## 为什么不能复用 `TdsClient`
|
||||
//!
|
||||
//! | | TDS 出向(`tds.rs`) | MPC 控制面(本模块) |
|
||||
//! |---|---|---|
|
||||
//! | 鉴权 | `X-Internal-Token` + `X-Protocol-Version: 3.2` | `Authorization: Bearer <明文原文>`(服务端只存 sha256) |
|
||||
//! | 响应 | 信封 `{code,msg,data}`,按 `code==200` 解包 | **裸 JSON** / `204 No Content` |
|
||||
//! | 路径 | 固定模板 + 基地址 | 同一套:`/nodes` `/sessions` `/desired` `/report` |
|
||||
//!
|
||||
//! ⇒ 只复用 **`platform::http::HttpClient`**(同一个 tokio 实例,闭包内无 `reqwest`),
|
||||
//! 头与解包规则另写一份。
|
||||
//!
|
||||
//! ## 能力边界(源码事实,别越界设计)
|
||||
//!
|
||||
//! `ssw-mpc/crates/mpc-hostd/src/control_plane_api.rs:60-95` 的路由全集是
|
||||
//! `POST /registry`、`GET /nodes`、`GET /sessions`、`POST /desired`、`GET /report`。
|
||||
//! **`POST /desired` 只做「选址 → hostd create/attest → 公布已证明端点」,不下发算子**
|
||||
//! (`control_plane_api.rs:133-186`)。所以「接入控制面调度」= 选址 + 起实例 + 拿端点 +
|
||||
//! 回报状态;算子点火由 `tdc::engine` 承担,参数映射(`parties p1/p2` → `sites alice/carol`)
|
||||
//! 也在执行层,不在本模块。
|
||||
|
||||
use std::time::Duration;
|
||||
|
||||
use axum::http::header::{AUTHORIZATION, CONTENT_TYPE};
|
||||
use axum::http::{HeaderMap, HeaderValue};
|
||||
use serde_json::{json, Value};
|
||||
|
||||
use crate::platform::http::{HttpClient, HttpResponse};
|
||||
|
||||
/// 连接超时(默认 3s,与 TDS 出向同口径)。
|
||||
pub const CONNECT_TIMEOUT: Duration = Duration::from_secs(3);
|
||||
/// 读取超时(默认 5s;控制面选址/证明可能稍慢,可由部署侧调大)。
|
||||
pub const READ_TIMEOUT: Duration = Duration::from_secs(5);
|
||||
|
||||
/// 未配置控制面地址时的降级文案(**NOOP 不伪装成调用成功**)。
|
||||
pub const NOOP_MESSAGE: &str = "未配置 MPC 控制面地址(TDC_EXEC_CONTROL_PLANE_URL)";
|
||||
|
||||
#[derive(Clone)]
|
||||
pub struct MpcControlPlaneClient {
|
||||
base_url: Option<String>,
|
||||
token: String,
|
||||
http: HttpClient,
|
||||
}
|
||||
|
||||
impl MpcControlPlaneClient {
|
||||
pub fn new(base_url: Option<String>, token: String) -> Self {
|
||||
Self::with_timeouts(base_url, token, CONNECT_TIMEOUT, READ_TIMEOUT)
|
||||
}
|
||||
|
||||
pub fn with_timeouts(
|
||||
base_url: Option<String>,
|
||||
token: String,
|
||||
connect_timeout: Duration,
|
||||
read_timeout: Duration,
|
||||
) -> Self {
|
||||
Self {
|
||||
base_url: base_url.map(|b| b.trim().trim_end_matches('/').to_string()),
|
||||
token,
|
||||
http: HttpClient::new(connect_timeout, read_timeout),
|
||||
}
|
||||
}
|
||||
|
||||
pub fn is_noop(&self) -> bool {
|
||||
self.base_url.is_none()
|
||||
}
|
||||
|
||||
/// 只回基地址,**不回 token**(日志/状态里绝不出现凭据)。
|
||||
pub fn endpoint(&self) -> Option<&str> {
|
||||
self.base_url.as_deref()
|
||||
}
|
||||
|
||||
/// 头:`Authorization: Bearer <原文>` + `Content-Type: application/json`。
|
||||
fn headers(&self) -> Result<HeaderMap, String> {
|
||||
if self.token.is_empty() {
|
||||
return Err("MPC 控制面 Bearer 未配置(TDC_EXEC_CONTROL_PLANE_TOKEN)".to_string());
|
||||
}
|
||||
let mut h = HeaderMap::new();
|
||||
h.insert(CONTENT_TYPE, HeaderValue::from_static("application/json"));
|
||||
let bearer = HeaderValue::from_str(&format!("Bearer {}", self.token))
|
||||
.map_err(|_| "MPC 控制面 Bearer 含非法字节".to_string())?;
|
||||
h.insert(AUTHORIZATION, bearer);
|
||||
Ok(h)
|
||||
}
|
||||
|
||||
/// `GET /nodes`:查注册表(选址输入)。
|
||||
pub async fn nodes(&self) -> Result<Value, String> {
|
||||
self.send("GET", "/nodes", None).await
|
||||
}
|
||||
|
||||
/// `GET /sessions`:查**已证明通过**的会话端点(计算面消费)。
|
||||
pub async fn sessions(&self) -> Result<Value, String> {
|
||||
self.send("GET", "/sessions", None).await
|
||||
}
|
||||
|
||||
/// `GET /report`:查最近一轮对账结论(回报状态的读侧)。
|
||||
pub async fn report(&self) -> Result<Value, String> {
|
||||
self.send("GET", "/report", None).await
|
||||
}
|
||||
|
||||
/// `POST /desired`:设期望态(起 TEE guest + 公布端点)。
|
||||
///
|
||||
/// 🔴 请求体必须是 `{"sessions":[…]}`,字段缺失控制面一律 400(不取默认值)。
|
||||
/// 参数由执行层按 `party→site` 映射产出,本模块只负责原样送达。
|
||||
pub async fn desired(&self, sessions: &Value) -> Result<Value, String> {
|
||||
let body = json!({ "sessions": sessions });
|
||||
self.send("POST", "/desired", Some(&body)).await
|
||||
}
|
||||
|
||||
/// `POST /registry`:自报本节点(可选,节点身份登记)。
|
||||
pub async fn registry(&self, descriptor: &Value) -> Result<Value, String> {
|
||||
self.send("POST", "/registry", Some(descriptor)).await
|
||||
}
|
||||
|
||||
async fn send(
|
||||
&self,
|
||||
method: &str,
|
||||
path: &str,
|
||||
body: Option<&Value>,
|
||||
) -> Result<Value, String> {
|
||||
let base = self.base_url.as_ref().ok_or_else(|| NOOP_MESSAGE.to_string())?;
|
||||
let url = format!("{}{}", base, path);
|
||||
let headers = self.headers()?;
|
||||
let payload = match body {
|
||||
Some(b) => Some(serde_json::to_vec(b).map_err(|e| format!("期望态序列化失败: {}", e))?),
|
||||
None => None,
|
||||
};
|
||||
let resp: HttpResponse = self
|
||||
.http
|
||||
.send(method, &url, &headers, payload.as_deref())
|
||||
.await
|
||||
.map_err(|e| format!("MPC 控制面不可达: {}", e))?;
|
||||
parse_response(method, path, &resp)
|
||||
}
|
||||
}
|
||||
|
||||
/// 裸 JSON / 204 解包(**纯函数**,便于单测,不起网络)。
|
||||
fn parse_response(method: &str, path: &str, resp: &HttpResponse) -> Result<Value, String> {
|
||||
if !(200..300).contains(&resp.status) {
|
||||
let excerpt: String = resp.body.chars().take(200).collect();
|
||||
return Err(format!(
|
||||
"MPC 控制面 {} {} 返回 {}: {}",
|
||||
method, path, resp.status, excerpt
|
||||
));
|
||||
}
|
||||
if resp.body.trim().is_empty() {
|
||||
// `POST /desired` 成功是 204 No Content ⇒ 用显式 ok 表达,不伪造业务数据。
|
||||
return Ok(json!({ "ok": true, "status": resp.status, "empty": true }));
|
||||
}
|
||||
serde_json::from_str::<Value>(&resp.body)
|
||||
.map_err(|e| format!("MPC 控制面 {} {} 响应不是 JSON: {}", method, path, e))
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn r(status: u16, body: &str) -> HttpResponse {
|
||||
HttpResponse {
|
||||
status,
|
||||
body: body.to_string(),
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn no_content_is_an_explicit_ok_not_fabricated_data() {
|
||||
let v = parse_response("POST", "/desired", &r(204, "")).unwrap();
|
||||
assert_eq!(v["ok"], true);
|
||||
assert_eq!(v["status"], 204);
|
||||
assert_eq!(v["empty"], true);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn bare_json_is_returned_as_is() {
|
||||
let v = parse_response("GET", "/nodes", &r(200, "{\"nodes\":[{\"id\":\"n1\"}]}")).unwrap();
|
||||
assert_eq!(v["nodes"][0]["id"], "n1");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn non_2xx_carries_the_real_reason() {
|
||||
let e = parse_response("POST", "/desired", &r(400, "期望态缺 sessions 数组")).unwrap_err();
|
||||
assert!(e.contains("400"));
|
||||
assert!(e.contains("期望态缺 sessions 数组"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn malformed_json_is_rejected() {
|
||||
assert!(parse_response("GET", "/sessions", &r(200, "not-json")).is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn missing_token_is_fail_closed_and_never_logged() {
|
||||
let c = MpcControlPlaneClient::new(Some("http://127.0.0.1:3020".into()), String::new());
|
||||
assert!(!c.is_noop());
|
||||
let e = c.headers().unwrap_err();
|
||||
assert!(e.contains("Bearer 未配置"));
|
||||
assert_eq!(c.endpoint(), Some("http://127.0.0.1:3020"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn url_is_normalized() {
|
||||
let c = MpcControlPlaneClient::new(
|
||||
Some(" http://127.0.0.1:3020/ ".into()),
|
||||
"t".into(),
|
||||
);
|
||||
assert_eq!(c.endpoint(), Some("http://127.0.0.1:3020"));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,234 @@
|
||||
//! 平台下发的三个执行端点(+ 一个验收用状态查询)。
|
||||
//!
|
||||
//! ## 契约(逐字对齐 `TdcDeliveryClient` / `fake-tdc-receiver.py`)
|
||||
//!
|
||||
//! | 端点 | 平台调用方 | 响应要点 |
|
||||
//! |---|---|---|
|
||||
//! | `POST /api/ssw/tdc/DeliveryTask/{id}/Internal/SelfCheckStatus` | `AlgorithmExecDispatchServiceImpl:1053`(`postInternalAt`) | `data.readyStatus = "READY"`,`ok=true`;平台判 `_code==200 \|\| ok` |
|
||||
//! | `GET /api/ssw/tdc/DeliveryTask/{id}/Dev/Artifact` | `TdcDeliveryClient.pullArtifactAt:735` | `data.files` 必须是 **List** `[{name,contentBase64}]`,`algorithmHash` 由接收端**自算** |
|
||||
//! | `POST /api/ssw/tdc/DeliveryTask/{id}/Exec/Start` | `TdcDeliveryClient.execStartAt:775` | `data.ok=true`;平台判 `_code==200 && ok`(`:779`) |
|
||||
//! | `GET /api/ssw/tdc/DeliveryTask/{id}/Exec/Status` | 验收/运维(平台不用) | 后台状态机轨迹 |
|
||||
//!
|
||||
//! `ActionResult::ok(data)` 天然产出 `{code:200,msg:null,data:{…}}`,平台把 `data` 平铺到顶层
|
||||
//! 并补 `_code = code` ⇒ **不需要另写 `_code`**。
|
||||
//!
|
||||
//! ## 为什么这三条路径进 `public_paths`,但 handler 内仍做 fail-closed 校验
|
||||
//!
|
||||
//! `build/verify-contract-driven-tunnel.sh:410-413` 的 ⑦ 是**裸 curl 存活探测**
|
||||
//! (`POST .../DeliveryTask/PROBE/Exec/Start`,不带任何凭证)。全局鉴权中间件在路由前,
|
||||
//! 若不放行,端点从 404 只会变成 401,无法证明「端点已存在」。
|
||||
//! ⇒ 三条路径进白名单,但**每个 handler 自己复核内部令牌 + 协议版本**(与中间件同口径),
|
||||
//! 仅 `id == "PROBE"` 的纯探测放行且**零副作用**(不落盘、不 spawn、不发控制面)。
|
||||
//!
|
||||
//! 🔴 Spring `RestTemplate` 可能用 `Transfer-Encoding: chunked`(无 `Content-Length`)
|
||||
//! ⇒ 用 `Bytes` 抽取器(hyper 层已解 chunked),不自己读 `Content-Length`。
|
||||
|
||||
use axum::body::Bytes;
|
||||
use axum::extract::{Path, RawQuery, State};
|
||||
use axum::http::{HeaderMap, StatusCode};
|
||||
use axum::response::{IntoResponse, Response};
|
||||
use axum::routing::{get, post};
|
||||
use axum::Router;
|
||||
use serde_json::{json, Value};
|
||||
|
||||
use crate::platform::auth::reject;
|
||||
use crate::platform::body::BODY_FORMAT_ERROR;
|
||||
use crate::platform::envelope::ActionResult;
|
||||
use crate::platform::protocol::{
|
||||
check_protocol, INTERNAL_TOKEN_HEADER, PROTOCOL_VERSION_HEADER,
|
||||
};
|
||||
use crate::tdc::state::TdcState;
|
||||
|
||||
/// ⑦ 存活探测使用的假 taskId(**只回静态 JSON,不产生任何副作用**)。
|
||||
pub const PROBE_TASK_ID: &str = "PROBE";
|
||||
|
||||
pub fn router() -> Router<TdcState> {
|
||||
Router::new()
|
||||
.route(
|
||||
"/api/ssw/tdc/DeliveryTask/{id}/Internal/SelfCheckStatus",
|
||||
post(self_check_status),
|
||||
)
|
||||
.route(
|
||||
"/api/ssw/tdc/DeliveryTask/{id}/Dev/Artifact",
|
||||
get(dev_artifact),
|
||||
)
|
||||
.route(
|
||||
"/api/ssw/tdc/DeliveryTask/{id}/Exec/Start",
|
||||
post(exec_start),
|
||||
)
|
||||
.route(
|
||||
"/api/ssw/tdc/DeliveryTask/{id}/Exec/Status",
|
||||
get(exec_status),
|
||||
)
|
||||
}
|
||||
|
||||
/// 与全局中间件**同口径**的入向复核(白名单放行不代表免鉴权)。
|
||||
fn guard(state: &TdcState, headers: &HeaderMap, id: &str) -> Result<(), Response> {
|
||||
if id == PROBE_TASK_ID {
|
||||
return Ok(());
|
||||
}
|
||||
let token = headers
|
||||
.get(INTERNAL_TOKEN_HEADER)
|
||||
.and_then(|v| v.to_str().ok())
|
||||
.unwrap_or("");
|
||||
if token.is_empty() || token != state.config.internal_token {
|
||||
return Err(reject(StatusCode::UNAUTHORIZED, "内部接口凭证无效"));
|
||||
}
|
||||
let declared = headers
|
||||
.get(PROTOCOL_VERSION_HEADER)
|
||||
.and_then(|v| v.to_str().ok());
|
||||
if !check_protocol(declared) {
|
||||
return Err(reject(
|
||||
StatusCode::UNAUTHORIZED,
|
||||
"跨端协议版本不兼容,请升级对端",
|
||||
));
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// `POST .../Internal/SelfCheckStatus` —— 环境自检。
|
||||
///
|
||||
/// 🔴 `readyStatus=READY` 只表示**接单能力就绪**(端点 + 状态机 + 落盘目录可用),
|
||||
/// 不等于 ET/Ray/mpc 二进制齐全 ⇒ 能力缺口在 `capabilities` 里如实列出,不伪装。
|
||||
async fn self_check_status(
|
||||
Path(id): Path<String>,
|
||||
State(state): State<TdcState>,
|
||||
headers: HeaderMap,
|
||||
_body: Bytes,
|
||||
) -> Response {
|
||||
if let Err(r) = guard(&state, &headers, &id) {
|
||||
return r;
|
||||
}
|
||||
ActionResult::ok(json!({
|
||||
"readyStatus": "READY",
|
||||
"ok": true,
|
||||
"engineReady": true,
|
||||
"taskId": id,
|
||||
"capabilities": state.exec.capabilities(),
|
||||
"note": "tdc-rs 执行端点就绪;READY 仅指能接单,真实能力见 capabilities",
|
||||
}))
|
||||
.into_response()
|
||||
}
|
||||
|
||||
/// `GET .../Dev/Artifact` —— 平台从使用方拉算法产物并按 `dirDigest` 重算摘要。
|
||||
async fn dev_artifact(
|
||||
Path(id): Path<String>,
|
||||
State(state): State<TdcState>,
|
||||
headers: HeaderMap,
|
||||
) -> Response {
|
||||
if let Err(r) = guard(&state, &headers, &id) {
|
||||
return r;
|
||||
}
|
||||
match state.exec.artifact() {
|
||||
Ok(data) => ActionResult::ok(data).into_response(),
|
||||
Err(reason) => ActionResult::fail(reason).into_response(),
|
||||
}
|
||||
}
|
||||
|
||||
/// `POST .../Exec/Start` —— 立即回执,后台跑状态机。
|
||||
async fn exec_start(
|
||||
Path(id): Path<String>,
|
||||
State(state): State<TdcState>,
|
||||
headers: HeaderMap,
|
||||
body: Bytes,
|
||||
) -> Response {
|
||||
if let Err(r) = guard(&state, &headers, &id) {
|
||||
return r;
|
||||
}
|
||||
// ⑦ 裸探测:只证明端点存在,零副作用。
|
||||
if id == PROBE_TASK_ID {
|
||||
return ActionResult::ok(json!({
|
||||
"ok": true,
|
||||
"probe": true,
|
||||
"endpoint": true,
|
||||
"engine": "tdc-rs",
|
||||
"note": "执行端点存在;纯存活探测,未落盘、未拉起进程、未调用控制面",
|
||||
}))
|
||||
.into_response();
|
||||
}
|
||||
let payload: Value = if body.is_empty() {
|
||||
json!({})
|
||||
} else {
|
||||
match serde_json::from_slice(&body) {
|
||||
Ok(v) => v,
|
||||
Err(_) => return ActionResult::fail(BODY_FORMAT_ERROR).into_response(),
|
||||
}
|
||||
};
|
||||
let (rec, duplicate) = state.exec.accept(&id, &payload);
|
||||
if !duplicate {
|
||||
let runtime = state.exec.clone();
|
||||
let exec_id = rec.exec_id.clone();
|
||||
tokio::spawn(async move {
|
||||
runtime.run(exec_id, payload).await;
|
||||
});
|
||||
}
|
||||
ActionResult::ok(json!({
|
||||
"ok": true,
|
||||
"accepted": true,
|
||||
"duplicate": duplicate,
|
||||
"execId": rec.exec_id,
|
||||
"phase": rec.phase.as_str(),
|
||||
"statusPath": format!("/api/ssw/tdc/DeliveryTask/{}/Exec/Status?execId={}", id, rec.exec_id),
|
||||
"executor": "tdc-rs (no-pod, host process)",
|
||||
"note": "仅表示已受理;后台执行状态查 statusPath,受理 ≠ 执行成功",
|
||||
}))
|
||||
.into_response()
|
||||
}
|
||||
|
||||
/// `GET .../Exec/Status` —— 查后台状态机轨迹(验收 / 运维用)。
|
||||
async fn exec_status(
|
||||
Path(id): Path<String>,
|
||||
State(state): State<TdcState>,
|
||||
headers: HeaderMap,
|
||||
query: RawQuery,
|
||||
) -> Response {
|
||||
if let Err(r) = guard(&state, &headers, &id) {
|
||||
return r;
|
||||
}
|
||||
let exec_id = query
|
||||
.0
|
||||
.as_deref()
|
||||
.unwrap_or("")
|
||||
.split('&')
|
||||
.find_map(|kv| kv.strip_prefix("execId="))
|
||||
.map(str::to_string);
|
||||
let records: Vec<Value> = state
|
||||
.exec
|
||||
.store
|
||||
.list()
|
||||
.into_iter()
|
||||
.filter(|r| r.task_id == id)
|
||||
.filter(|r| exec_id.as_ref().map(|e| &r.exec_id == e).unwrap_or(true))
|
||||
.map(|r| serde_json::to_value(r).unwrap_or(Value::Null))
|
||||
.collect();
|
||||
ActionResult::ok(json!({
|
||||
"ok": true,
|
||||
"taskId": id,
|
||||
"records": records,
|
||||
"note": "phase=FAILED 看 message;PLANNED 是 dry-run,不是成功",
|
||||
}))
|
||||
.into_response()
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn probe_id_is_a_side_effect_free_constant() {
|
||||
assert_eq!(PROBE_TASK_ID, "PROBE");
|
||||
}
|
||||
|
||||
/// 三端点路径必须与平台 / 假接收端逐字一致(改一个字符平台就 404)。
|
||||
#[test]
|
||||
fn paths_match_the_platform_client() {
|
||||
let paths = [
|
||||
"/api/ssw/tdc/DeliveryTask/{id}/Internal/SelfCheckStatus",
|
||||
"/api/ssw/tdc/DeliveryTask/{id}/Dev/Artifact",
|
||||
"/api/ssw/tdc/DeliveryTask/{id}/Exec/Start",
|
||||
];
|
||||
assert!(paths[0].ends_with("/Internal/SelfCheckStatus"));
|
||||
assert!(paths[1].ends_with("/Dev/Artifact"));
|
||||
assert!(paths[2].ends_with("/Exec/Start"));
|
||||
}
|
||||
}
|
||||
@@ -6,6 +6,7 @@
|
||||
pub mod account;
|
||||
pub mod auth;
|
||||
pub mod crud;
|
||||
pub mod exec;
|
||||
pub mod health;
|
||||
pub mod tds_biz;
|
||||
|
||||
@@ -31,5 +32,9 @@ pub fn build_router(state: TdcState) -> Router<TdcState> {
|
||||
.merge(account::router())
|
||||
.merge(crud::router())
|
||||
.merge(tds_biz::router())
|
||||
// 平台下发的三个执行端点(+1 个验收用状态查询)。
|
||||
// 🔴 通用 CRUD 只产生 `<path>/{id}`,不产生 `<path>/{id}/Exec/Start`
|
||||
// 这类两段子路径 ⇒ 不显式注册就是 404(② 的 ⑦ 探测正是验这个)。
|
||||
.merge(exec::router())
|
||||
.with_state(state)
|
||||
}
|
||||
@@ -0,0 +1,255 @@
|
||||
//! 执行层配置(`TDC_EXEC_*`,全部走环境变量 / 部署侧 ConfigMap)。
|
||||
//!
|
||||
//! 🔴 **凭据不入库、不落状态文件、不进日志**:控制面 Bearer 与 EasyTier `vnetSecret`
|
||||
//! 都只在进程内存里出现;后者由平台每次 `Exec/Start` 下发,落 0600 env 文件复用。
|
||||
//! env 名清单是部署契约,值一律不写进本仓。
|
||||
//!
|
||||
//! ## 口径边界(用户 2026-11-13 裁决)
|
||||
//!
|
||||
//! - EasyTier 负责**跨机网络**;
|
||||
//! - Ray 只负责**分布式计算层**(head/worker 放置 + actor 调度),不替代 `mpc` 算子执行;
|
||||
//! - tdc-rs 是统一接入点:接 TDS 下发、向 MPC 控制面做「选址 / 起实例 / 拿端点 / 回报状态」,
|
||||
//! 算子点火与 `party→site` 映射由本执行层承担。
|
||||
//!
|
||||
//! ⇒ 平台下发的 `clusterConfig.parties[p].address` 是 K8s Service DNS(不起 Pod 即死地址),
|
||||
//! 真实端点必须由 `TDC_EXEC_SITE_ENDPOINTS` 显式给出 —— **不猜、不兜底、缺了就 fail-closed**。
|
||||
|
||||
use std::collections::BTreeMap;
|
||||
use std::path::{Path, PathBuf};
|
||||
|
||||
use serde_json::Value;
|
||||
|
||||
pub const ENV_ARTIFACT_DIR: &str = "TDC_EXEC_ARTIFACT_DIR";
|
||||
pub const ENV_CONTROL_PLANE_URL: &str = "TDC_EXEC_CONTROL_PLANE_URL";
|
||||
pub const ENV_CONTROL_PLANE_TOKEN: &str = "TDC_EXEC_CONTROL_PLANE_TOKEN";
|
||||
pub const ENV_EASYTIER_BIN: &str = "TDC_EXEC_EASYTIER_BIN";
|
||||
pub const ENV_MPC_BIN: &str = "TDC_EXEC_MPC_BIN";
|
||||
pub const ENV_MPC_WORKDIR: &str = "TDC_EXEC_MPC_WORKDIR";
|
||||
pub const ENV_RAY_BIN: &str = "TDC_EXEC_RAY_BIN";
|
||||
pub const ENV_RAY_ENABLED: &str = "TDC_EXEC_RAY_ENABLED";
|
||||
pub const ENV_RAY_IS_HEAD: &str = "TDC_EXEC_RAY_IS_HEAD";
|
||||
pub const ENV_RAY_CLIENT_PORT: &str = "TDC_EXEC_RAY_CLIENT_PORT";
|
||||
pub const ENV_RAY_RESOURCES: &str = "TDC_EXEC_RAY_RESOURCES";
|
||||
pub const ENV_PARTY_SITE_MAP: &str = "TDC_EXEC_PARTY_SITE_MAP";
|
||||
pub const ENV_SITE_ENDPOINTS: &str = "TDC_EXEC_SITE_ENDPOINTS";
|
||||
pub const ENV_LOCAL_VIP: &str = "TDC_EXEC_LOCAL_VIP";
|
||||
pub const ENV_ET_NO_TUN: &str = "TDC_EXEC_ET_NO_TUN";
|
||||
pub const ENV_ET_PROXY_NETWORKS: &str = "TDC_EXEC_ET_PROXY_NETWORKS";
|
||||
pub const ENV_ET_RELAY: &str = "TDC_EXEC_ET_RELAY";
|
||||
pub const ENV_ET_LISTENERS: &str = "TDC_EXEC_ET_LISTENERS";
|
||||
pub const ENV_ET_RPC_PORTAL: &str = "TDC_EXEC_ET_RPC_PORTAL";
|
||||
pub const ENV_ET_SOCKS5: &str = "TDC_EXEC_ET_SOCKS5";
|
||||
pub const ENV_MPC_PORT: &str = "TDC_EXEC_MPC_PORT";
|
||||
pub const ENV_MPC_PSK: &str = "TDC_EXEC_MPC_PSK";
|
||||
pub const ENV_MPC_OPERATOR: &str = "TDC_EXEC_MPC_OPERATOR";
|
||||
pub const ENV_DRY_RUN: &str = "TDC_EXEC_DRY_RUN";
|
||||
/// 控制面 `POST /desired` 的期望态(JSON 数组,或 `@/path/to/file.json`)。
|
||||
pub const ENV_DESIRED_SESSIONS: &str = "TDC_EXEC_DESIRED_SESSIONS";
|
||||
|
||||
/// 执行层配置(构造一次,放进 `TdcState`)。
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct ExecConfig {
|
||||
/// 算法产物目录(`Dev/Artifact` 从这里读,摘要自算)。
|
||||
pub artifact_dir: PathBuf,
|
||||
/// 执行状态与 pid / env / 日志目录。
|
||||
pub state_dir: PathBuf,
|
||||
/// MPC 控制面基址;`None` ⇒ NOOP(**如实报「未配置」,不伪装调用成功**)。
|
||||
pub control_plane_url: Option<String>,
|
||||
/// `Authorization: Bearer <token>` 的明文原文(服务端只存 sha256)。
|
||||
pub control_plane_token: String,
|
||||
pub easytier_bin: PathBuf,
|
||||
pub ray_bin: PathBuf,
|
||||
pub mpc_bin: PathBuf,
|
||||
pub mpc_workdir: PathBuf,
|
||||
/// 是否启用 Ray head/worker 生命周期管理。
|
||||
///
|
||||
/// 默认 `false`:ZYNQ(armv7 / 无 python3)跑不了 Ray(见 `decision` §2.8),
|
||||
/// 由 `mpc cluster/agent` 单向可达形态等价接入;需要真起 head 的 x86_64 宿主机显式置 1。
|
||||
pub ray_enabled: bool,
|
||||
/// 本机是否承担 Ray head(head 起在使用方 vIP 侧)。
|
||||
pub ray_is_head: bool,
|
||||
/// Ray Client Server 端口(GCS 端口取平台 `rayHead` 的端口,两者需同时可达)。
|
||||
pub ray_client_port: u16,
|
||||
/// `ray start --resources='{...}'` 的资源标签(把参与方钉在指定物理机)。
|
||||
pub ray_resources: Option<String>,
|
||||
/// `party → site`:`p1=alice,p2=carol`。
|
||||
pub site_of_party: BTreeMap<String, String>,
|
||||
/// `site → host:port`:`alice=10.12.0.1:9449,carol=10.12.0.2:9449`。
|
||||
/// 空 ⇒ 执行层 fail-closed(**不**回落 K8s Service DNS)。
|
||||
pub endpoint_of_site: BTreeMap<String, String>,
|
||||
pub local_vip: Option<String>,
|
||||
/// ZYNQ 无 TUN:`--no-tun --proxy-networks … --socks5 …` 网关旁路形态。
|
||||
pub et_no_tun: bool,
|
||||
pub et_proxy_networks: Option<String>,
|
||||
pub et_relay: Option<String>,
|
||||
pub et_listeners: String,
|
||||
pub et_rpc_portal: String,
|
||||
pub et_socks5: Option<String>,
|
||||
pub mpc_port: u16,
|
||||
/// `MPC_PSK` 会话密钥(只进子进程 env)。
|
||||
pub mpc_psk: Option<String>,
|
||||
/// 算子名(`psi` / `lr-vertical`),默认 `psi`。
|
||||
pub mpc_operator: String,
|
||||
/// 控制面期望态原文(合法 JSON 数组文本,或 `@/path/to/file.json`)。
|
||||
///
|
||||
/// 🔴 `/desired` 的字段(`platform/min_tcb/csv_variant/pinned_measurement/session_id/...`)
|
||||
/// 平台 `Exec/Start` 一个都不给,控制面缺字段一律 400 ⇒ **不许凭空构造**,
|
||||
/// 必须由部署侧按站点准入(`site_admit`)产出并显式配置。
|
||||
/// `None` ⇒ 只做 `GET /nodes|/sessions|/report` 对账,不 POST /desired(如实标注)。
|
||||
pub desired_sessions: Option<String>,
|
||||
/// `true` ⇒ 只落**执行计划**(含完整命令 / 控制面调用),不 spawn、不 POST /desired。
|
||||
/// 状态标 `Planned`,绝不写成 `Succeeded`。
|
||||
pub dry_run: bool,
|
||||
}
|
||||
|
||||
impl ExecConfig {
|
||||
/// 从进程 env 装配(只在 `TdcState::new` 调一次)。
|
||||
pub fn from_env(data_dir: &Path) -> Self {
|
||||
let env: BTreeMap<String, String> = std::env::vars().collect();
|
||||
Self::from_raw(&env, data_dir)
|
||||
}
|
||||
|
||||
/// 🔴 **纯函数**(不读进程全局 env)⇒ 可单测、无并行竞态(对位 `TdcConfig::from_raw`)。
|
||||
pub fn from_raw(env: &BTreeMap<String, String>, data_dir: &Path) -> Self {
|
||||
let get = |k: &str| -> Option<String> {
|
||||
env.get(k)
|
||||
.map(|v| v.trim().to_string())
|
||||
.filter(|v| !v.is_empty())
|
||||
};
|
||||
let path = |k: &str, default: PathBuf| -> PathBuf {
|
||||
get(k).map(PathBuf::from).unwrap_or(default)
|
||||
};
|
||||
let flag = |k: &str| -> bool {
|
||||
matches!(get(k).as_deref(), Some("1") | Some("true") | Some("TRUE") | Some("yes"))
|
||||
};
|
||||
let num = |k: &str, default: u16| -> u16 {
|
||||
get(k).and_then(|v| v.parse::<u16>().ok()).unwrap_or(default)
|
||||
};
|
||||
let map = |k: &str, fallback: &[(&str, &str)]| -> BTreeMap<String, String> {
|
||||
parse_map(get(k).as_deref().unwrap_or("")).map_or_else(
|
||||
|| {
|
||||
fallback
|
||||
.iter()
|
||||
.map(|(a, b)| (a.to_string(), b.to_string()))
|
||||
.collect()
|
||||
},
|
||||
|m| m,
|
||||
)
|
||||
};
|
||||
|
||||
let state_dir = path(ENV_STATE_DIR, data_dir.join("exec"));
|
||||
Self {
|
||||
artifact_dir: path(ENV_ARTIFACT_DIR, data_dir.join("artifacts")),
|
||||
control_plane_url: get(ENV_CONTROL_PLANE_URL).map(|v| v.trim_end_matches('/').to_string()),
|
||||
control_plane_token: get(ENV_CONTROL_PLANE_TOKEN).unwrap_or_default(),
|
||||
easytier_bin: path(ENV_EASYTIER_BIN, PathBuf::from("/usr/local/bin/easytier-core")),
|
||||
ray_bin: path(ENV_RAY_BIN, PathBuf::from("ray")),
|
||||
mpc_bin: path(ENV_MPC_BIN, PathBuf::from("/usr/local/bin/mpc")),
|
||||
mpc_workdir: path(ENV_MPC_WORKDIR, PathBuf::from("/tmp/mpc-tunnel")),
|
||||
ray_enabled: flag(ENV_RAY_ENABLED),
|
||||
ray_is_head: flag(ENV_RAY_IS_HEAD),
|
||||
ray_client_port: num(ENV_RAY_CLIENT_PORT, 10001),
|
||||
ray_resources: get(ENV_RAY_RESOURCES),
|
||||
site_of_party: map(ENV_PARTY_SITE_MAP, &[("p1", "alice"), ("p2", "carol")]),
|
||||
endpoint_of_site: map(ENV_SITE_ENDPOINTS, &[]),
|
||||
local_vip: get(ENV_LOCAL_VIP),
|
||||
et_no_tun: flag(ENV_ET_NO_TUN),
|
||||
et_proxy_networks: get(ENV_ET_PROXY_NETWORKS),
|
||||
et_relay: get(ENV_ET_RELAY),
|
||||
et_listeners: get(ENV_ET_LISTENERS)
|
||||
.unwrap_or_else(|| "tcp://0.0.0.0:11012".to_string()),
|
||||
et_rpc_portal: get(ENV_ET_RPC_PORTAL)
|
||||
.unwrap_or_else(|| "127.0.0.1:15891".to_string()),
|
||||
et_socks5: get(ENV_ET_SOCKS5),
|
||||
mpc_port: num(ENV_MPC_PORT, 9449),
|
||||
mpc_psk: get(ENV_MPC_PSK),
|
||||
mpc_operator: get(ENV_MPC_OPERATOR).unwrap_or_else(|| "psi".to_string()),
|
||||
desired_sessions: get(ENV_DESIRED_SESSIONS).and_then(|raw| {
|
||||
// `@/path` ⇒ 读文件;否则当 JSON 文本。非法 JSON 一律丢弃(fail-closed),
|
||||
// run() 会如实记「期望态未配置/非法 ⇒ 未起 TEE 实例」。
|
||||
let text = raw
|
||||
.strip_prefix('@')
|
||||
.and_then(|p| std::fs::read_to_string(p).ok())
|
||||
.unwrap_or(raw);
|
||||
serde_json::from_str::<Value>(&text)
|
||||
.ok()
|
||||
.filter(|v| v.is_array())
|
||||
.map(|_| text)
|
||||
}),
|
||||
dry_run: flag(ENV_DRY_RUN),
|
||||
// state_dir 由 path() 提前算好(不能在 struct 字面量里引用自身)
|
||||
state_dir,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// 状态目录 env(单独常量,便于文档引用)。
|
||||
pub const ENV_STATE_DIR: &str = "TDC_EXEC_STATE_DIR";
|
||||
|
||||
/// `k=v,k=v` 解析;空串 ⇒ `None`(调用方决定回落默认),含空键/值 ⇒ `None`(fail-closed)。
|
||||
fn parse_map(raw: &str) -> Option<BTreeMap<String, String>> {
|
||||
if raw.trim().is_empty() {
|
||||
return None;
|
||||
}
|
||||
let mut out = BTreeMap::new();
|
||||
for pair in raw.split(',') {
|
||||
let (k, v) = pair.trim().split_once('=')?;
|
||||
let (k, v) = (k.trim(), v.trim());
|
||||
if k.is_empty() || v.is_empty() {
|
||||
return None;
|
||||
}
|
||||
out.insert(k.to_string(), v.to_string());
|
||||
}
|
||||
Some(out)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn env(pairs: &[(&str, &str)]) -> BTreeMap<String, String> {
|
||||
pairs.iter().map(|(k, v)| (k.to_string(), v.to_string())).collect()
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn defaults_are_fail_closed() {
|
||||
let c = ExecConfig::from_raw(&env(&[]), Path::new("/data"));
|
||||
assert_eq!(c.artifact_dir, PathBuf::from("/data/artifacts"));
|
||||
assert_eq!(c.state_dir, PathBuf::from("/data/exec"));
|
||||
assert!(c.control_plane_url.is_none(), "未配控制面必须 NOOP");
|
||||
assert!(!c.ray_enabled, "ZYNQ 侧默认不启用 Ray");
|
||||
assert!(c.endpoint_of_site.is_empty(), "端点映射不许猜");
|
||||
assert_eq!(c.site_of_party.get("p1").map(String::as_str), Some("alice"));
|
||||
assert!(!c.dry_run);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn maps_and_flags_are_parsed() {
|
||||
let c = ExecConfig::from_raw(
|
||||
&env(&[
|
||||
(ENV_CONTROL_PLANE_URL, "http://192.168.1.23:3020/"),
|
||||
(ENV_RAY_ENABLED, "1"),
|
||||
(ENV_RAY_CLIENT_PORT, "10002"),
|
||||
(ENV_PARTY_SITE_MAP, "p1=alice,p2=bob"),
|
||||
(ENV_SITE_ENDPOINTS, "alice=10.12.0.1:9449,bob=10.12.0.2:9449"),
|
||||
(ENV_DRY_RUN, "1"),
|
||||
]),
|
||||
Path::new("/data"),
|
||||
);
|
||||
assert_eq!(
|
||||
c.control_plane_url.as_deref(),
|
||||
Some("http://192.168.1.23:3020")
|
||||
);
|
||||
assert!(c.ray_enabled);
|
||||
assert_eq!(c.ray_client_port, 10002);
|
||||
assert_eq!(c.site_of_party.get("p2").map(String::as_str), Some("bob"));
|
||||
assert_eq!(c.endpoint_of_site.len(), 2);
|
||||
assert!(c.dry_run);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn malformed_map_falls_back_to_defaults_safely() {
|
||||
let c = ExecConfig::from_raw(&env(&[(ENV_PARTY_SITE_MAP, "p1=,p2")]), Path::new("/d"));
|
||||
assert_eq!(c.site_of_party.get("p1").map(String::as_str), Some("alice"));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,210 @@
|
||||
//! 算法产物摘要 + base64(**零外部依赖的纯 Rust 实现**)。
|
||||
//!
|
||||
//! ## 为什么自己写 SHA-256
|
||||
//!
|
||||
//! `sha2` / `base64` 的 rlib 确实在平台闭包 `vendor/sspf-v0.5.0/lib/` 里,但**不在**
|
||||
//! `.cargo/config.toml` 的 `--extern` 白名单中,且 `digest` 有两份不同 hash。补 extern
|
||||
//! 会让本仓与「哪一份 digest」绑定,宿主 / armv7 两套闭包各要重生成一次配置,交叉编译时
|
||||
//! 极易踩 `E0308`。产物摘要只有 ~100 行标准算法 ⇒ **不引入闭包污染**,两套 target 同一份代码。
|
||||
//!
|
||||
//! ## 与平台的逐字对齐
|
||||
//!
|
||||
//! `AlgorithmExecDispatchServiceImpl.dirDigest(files)`(Java):
|
||||
//! ```text
|
||||
//! sha256( join( [ name + "\t" + sha256(content.getBytes(UTF_8)) for name in sorted(files) ], "\n" ) )
|
||||
//! ```
|
||||
//! 参考实现见 `ssw-tds-test/build/fake-tdc-receiver.py:39-44`。
|
||||
//!
|
||||
//! 🔴 **摘要必须和回给平台的 `files` 出自同一次读盘**:先读字节、再对同一份字节算 hash
|
||||
//! 与 base64。两处独立读盘 / 独立计算 ⇒ 平台重算不相等就硬拒("审的 ≠ 要跑的")。
|
||||
|
||||
use std::collections::BTreeMap;
|
||||
|
||||
/// FIPS 180-4 §4.2.2 的 64 个轮常量。
|
||||
const K: [u32; 64] = [
|
||||
0x428a2f98, 0x71374491, 0xb5c0fbcf, 0xe9b5dba5, 0x3956c25b, 0x59f111f1, 0x923f82a4,
|
||||
0xab1c5ed5, 0xd807aa98, 0x12835b01, 0x243185be, 0x550c7dc3, 0x72be5d74, 0x80deb1fe,
|
||||
0x9bdc06a7, 0xc19bf174, 0xe49b69c1, 0xefbe4786, 0x0fc19dc6, 0x240ca1cc, 0x2de92c6f,
|
||||
0x4a7484aa, 0x5cb0a9dc, 0x76f988da, 0x983e5152, 0xa831c66d, 0xb00327c8, 0xbf597fc7,
|
||||
0xc6e00bf3, 0xd5a79147, 0x06ca6351, 0x14292967, 0x27b70a85, 0x2e1b2138, 0x4d2c6dfc,
|
||||
0x53380d13, 0x650a7354, 0x766a0abb, 0x81c2c92e, 0x92722c85, 0xa2bfe8a1, 0xa81a664b,
|
||||
0xc24b8b70, 0xc76c51a3, 0xd192e819, 0xd6990624, 0xf40e3585, 0x106aa070, 0x19a4c116,
|
||||
0x1e376c08, 0x2748774c, 0x34b0bcb5, 0x391c0cb3, 0x4ed8aa4a, 0x5b9cca4f, 0x682e6ff3,
|
||||
0x748f82ee, 0x78a5636f, 0x84c87814, 0x8cc70208, 0x90befffa, 0xa4506ceb, 0xbef9a3f7,
|
||||
0xc67178f2,
|
||||
];
|
||||
|
||||
const IV: [u32; 8] = [
|
||||
0x6a09e667, 0xbb67ae85, 0x3c6ef372, 0xa54ff53a, 0x510e527f, 0x9b05688c, 0x1f83d9ab,
|
||||
0x5be0cd19,
|
||||
];
|
||||
|
||||
/// SHA-256 摘要(32 字节)。
|
||||
pub fn sha256(data: &[u8]) -> [u8; 32] {
|
||||
let mut state = IV;
|
||||
// 填充:0x80 → 补 0 到 len ≡ 56 (mod 64) → 大端 64 位比特长度。
|
||||
let mut padded = Vec::with_capacity(data.len() + 72);
|
||||
padded.extend_from_slice(data);
|
||||
padded.push(0x80);
|
||||
while padded.len() % 64 != 56 {
|
||||
padded.push(0);
|
||||
}
|
||||
padded.extend_from_slice(&(data.len() as u64).wrapping_mul(8).to_be_bytes());
|
||||
for chunk in padded.chunks_exact(64) {
|
||||
let mut block = [0u8; 64];
|
||||
block.copy_from_slice(chunk);
|
||||
compress(&mut state, &block);
|
||||
}
|
||||
let mut out = [0u8; 32];
|
||||
for (i, word) in state.iter().enumerate() {
|
||||
out[4 * i..4 * i + 4].copy_from_slice(&word.to_be_bytes());
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
fn compress(state: &mut [u32; 8], block: &[u8; 64]) {
|
||||
let mut w = [0u32; 64];
|
||||
for (i, word) in w[..16].iter_mut().enumerate() {
|
||||
*word = u32::from_be_bytes([
|
||||
block[4 * i],
|
||||
block[4 * i + 1],
|
||||
block[4 * i + 2],
|
||||
block[4 * i + 3],
|
||||
]);
|
||||
}
|
||||
for i in 16..64 {
|
||||
let x = w[i - 15];
|
||||
let y = w[i - 2];
|
||||
let s0 = x.rotate_right(7) ^ x.rotate_right(18) ^ (x >> 3);
|
||||
let s1 = y.rotate_right(17) ^ y.rotate_right(19) ^ (y >> 10);
|
||||
w[i] = w[i - 16]
|
||||
.wrapping_add(s0)
|
||||
.wrapping_add(w[i - 7])
|
||||
.wrapping_add(s1);
|
||||
}
|
||||
let [mut a, mut b, mut c, mut d, mut e, mut f, mut g, mut h] = *state;
|
||||
for i in 0..64 {
|
||||
let big_s1 = e.rotate_right(6) ^ e.rotate_right(11) ^ e.rotate_right(25);
|
||||
let ch = (e & f) ^ ((!e) & g);
|
||||
let t1 = h
|
||||
.wrapping_add(big_s1)
|
||||
.wrapping_add(ch)
|
||||
.wrapping_add(K[i])
|
||||
.wrapping_add(w[i]);
|
||||
let big_s0 = a.rotate_right(2) ^ a.rotate_right(13) ^ a.rotate_right(22);
|
||||
let maj = (a & b) ^ (a & c) ^ (b & c);
|
||||
let t2 = big_s0.wrapping_add(maj);
|
||||
h = g;
|
||||
g = f;
|
||||
f = e;
|
||||
e = d.wrapping_add(t1);
|
||||
d = c;
|
||||
c = b;
|
||||
b = a;
|
||||
a = t1.wrapping_add(t2);
|
||||
}
|
||||
for (slot, value) in state
|
||||
.iter_mut()
|
||||
.zip([a, b, c, d, e, f, g, h].into_iter())
|
||||
{
|
||||
*slot = slot.wrapping_add(value);
|
||||
}
|
||||
}
|
||||
|
||||
/// 小写十六进制 SHA-256(平台 `dirDigest` 用的就是这种文本)。
|
||||
pub fn sha256_hex(data: &[u8]) -> String {
|
||||
hex::encode(sha256(data))
|
||||
}
|
||||
|
||||
/// 标准 base64(带 `=` 填充)—— 平台 `filesOf` 的 `contentBase64` 形状。
|
||||
pub fn base64_encode(data: &[u8]) -> String {
|
||||
const TABLE: &[u8; 64] =
|
||||
b"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";
|
||||
let mut out = String::with_capacity((data.len() + 2) / 3 * 4);
|
||||
for chunk in data.chunks(3) {
|
||||
let b0 = chunk[0] as u32;
|
||||
let b1 = chunk.get(1).copied().unwrap_or(0) as u32;
|
||||
let b2 = chunk.get(2).copied().unwrap_or(0) as u32;
|
||||
let n = (b0 << 16) | (b1 << 8) | b2;
|
||||
out.push(TABLE[((n >> 18) & 63) as usize] as char);
|
||||
out.push(TABLE[((n >> 12) & 63) as usize] as char);
|
||||
out.push(if chunk.len() > 1 {
|
||||
TABLE[((n >> 6) & 63) as usize] as char
|
||||
} else {
|
||||
'='
|
||||
});
|
||||
out.push(if chunk.len() > 2 {
|
||||
TABLE[(n & 63) as usize] as char
|
||||
} else {
|
||||
'='
|
||||
});
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
/// 目录摘要:逐行 `name\tsha256(content)`,按文件名排序,`\n` 连接,再取 SHA-256。
|
||||
///
|
||||
/// `BTreeMap` 的迭代序即字典序 ⇒ 与 Java `TreeMap` / Python `sorted(files.keys())` 一致。
|
||||
pub fn dir_digest(files: &BTreeMap<String, Vec<u8>>) -> String {
|
||||
let lines: Vec<String> = files
|
||||
.iter()
|
||||
.map(|(name, content)| format!("{}\t{}", name, sha256_hex(content)))
|
||||
.collect();
|
||||
sha256_hex(lines.join("\n").as_bytes())
|
||||
}
|
||||
|
||||
/// 状态文件名 / 执行 id 的安全化(不把 taskId 里的 `/` `:` 带进路径)。
|
||||
pub fn short_key(s: &str) -> String {
|
||||
sha256_hex(s.as_bytes())[..16].to_string()
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn sha256_matches_nist_vectors() {
|
||||
assert_eq!(
|
||||
sha256_hex(b""),
|
||||
"e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
|
||||
);
|
||||
assert_eq!(
|
||||
sha256_hex(b"abc"),
|
||||
"ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad"
|
||||
);
|
||||
// 跨 64 字节块边界(56 字节填充点是常见 bug)
|
||||
assert_eq!(
|
||||
sha256_hex(&b"a".repeat(1_000_000)),
|
||||
"cdc76e5c9914fb9281a1c7e284d73e67f1809a48a497200e046d39ccc7112cd0"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn base64_matches_rfc4648() {
|
||||
assert_eq!(base64_encode(b""), "");
|
||||
assert_eq!(base64_encode(b"f"), "Zg==");
|
||||
assert_eq!(base64_encode(b"fo"), "Zm8=");
|
||||
assert_eq!(base64_encode(b"foo"), "Zm9v");
|
||||
assert_eq!(base64_encode(b"foobar"), "Zm9vYmFy");
|
||||
}
|
||||
|
||||
/// 与 `python3 -c '...hashlib.sha256...'` 的实算值逐字对齐(跨平台回归锚点)。
|
||||
#[test]
|
||||
fn dir_digest_matches_platform_java_algorithm() {
|
||||
let mut files = BTreeMap::new();
|
||||
files.insert("algorithm.py".to_string(), b"import sys\nprint(1)\n".to_vec());
|
||||
files.insert("README.md".to_string(), b"hello".to_vec());
|
||||
assert_eq!(
|
||||
dir_digest(&files),
|
||||
"5cacaa8a0dfbdb801fed11346864ec241f3fda9f2b52ad5f1594fe7edfd9bab1"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn short_key_is_stable_and_path_safe() {
|
||||
let a = short_key("nax-1/vnet:a");
|
||||
assert_eq!(a, short_key("nax-1/vnet:a"));
|
||||
assert_ne!(a, short_key("nax-1/vnet:b"));
|
||||
assert!(a.chars().all(|c| c.is_ascii_hexdigit()));
|
||||
}
|
||||
}
|
||||
+791
-4
@@ -1,6 +1,793 @@
|
||||
//! 执行引擎占位(对位上游 `ssw.tdc.engine`)。
|
||||
//! 执行引擎:把一次平台 `Exec/Start` 串成「控制面对账 → ET 组网 → Ray 计算层 → MPC 点火」。
|
||||
//!
|
||||
//! 上游的隐私计算作业依赖 **Nax 引擎**与 **scard 安全平面** ⇒ 本地不可行,
|
||||
//! 本仓不实现(见 `CLAUDE.md`「明确不在本仓库范围」)。
|
||||
//! ## 从占位到实现(2026-11-13)
|
||||
//!
|
||||
//! 原 `_PLACEHOLDER` 声称「上游 Nax 执行引擎本地不可实现」。用户口径裁决后不再成立:
|
||||
//! **不起 Pod**,tdc-rs 在宿主机进程内承担执行,是统一接入点。本模块替代那个占位常量。
|
||||
//!
|
||||
//! ## 架构口径(用户裁决 + `decision` §6 三处分歧的处理)
|
||||
//!
|
||||
//! | 层 | 谁负责 | 本模块的落点 |
|
||||
//! |---|---|---|
|
||||
//! | 跨机网络 | EasyTier | `ensure_et`:`vnetName/vnetSecret` 幂等拉起,密钥落 0600 env 文件 |
|
||||
//! | 分布式计算 | Ray(**平台 nax 语义**:跨机 actor 放置与调度) | `ensure_ray`:head 起在使用方 vIP 侧 x86_64 宿主机,`rayHead` 端口按 GCS 解释;未启用时如实记 skipped,ZYNQ 侧走 `mpc cluster/agent` 等价路径 |
|
||||
//! | 算子执行 | `mpc` 二进制(计算数据不出 mpc) | `ensure_mpc`:`party→site` 映射 + `--peers` 点火 |
|
||||
//! | 控制面调度 | `:3020` 选址 / 起实例 / 拿端点 / 回报状态 | `control_plane_phase`:`GET /nodes` → `POST /desired` → `GET /sessions` → `GET /report` |
|
||||
//!
|
||||
//! 🔴 **绝不在 HTTP 请求里同步等 MPC 跑完**:真算分钟级,平台 `execStartTimeoutSeconds`
|
||||
//! 有上限。`Exec/Start` 立即回 `_code=200, ok=true`(只表示**已受理**),后台任务逐步
|
||||
//! 落状态;真实成败查 `Exec/Status`。「受理成功」与「执行成功」是两个不同事实,不合并汇报。
|
||||
//!
|
||||
//! 🔴 **fail-closed,不伪装**:缺 `TDC_EXEC_SITE_ENDPOINTS` / 缺二进制 / 缺物料 / 摘要不符
|
||||
//! ⇒ 状态 `FAILED` + 具体真因,绝不回落 K8s Service DNS、绝不假装跑过。
|
||||
//! `dry_run` 只落计划,状态是 `PLANNED`,**不是** `SUCCEEDED`。
|
||||
|
||||
pub const _PLACEHOLDER: &str = "占位模块:上游 Nax 执行引擎本地不可实现";
|
||||
pub mod config;
|
||||
pub mod digest;
|
||||
pub mod process;
|
||||
pub mod state;
|
||||
|
||||
use std::collections::BTreeMap;
|
||||
use std::path::Path;
|
||||
|
||||
use serde_json::{json, Value};
|
||||
|
||||
use crate::tdc::client::mpc_control_plane::{MpcControlPlaneClient, NOOP_MESSAGE};
|
||||
|
||||
pub use config::ExecConfig;
|
||||
pub use state::{ExecPhase, ExecRecord, ExecStore};
|
||||
|
||||
use digest::{base64_encode, dir_digest, short_key};
|
||||
use process::{
|
||||
ensure_single_instance, spawn_detached, write_secret_file, InstanceGuard, SpawnRequest,
|
||||
};
|
||||
|
||||
/// 执行层运行态(挂在 `TdcState` 上,后台任务持 `Arc` clone)。
|
||||
pub struct ExecRuntime {
|
||||
pub config: ExecConfig,
|
||||
pub store: ExecStore,
|
||||
pub control_plane: MpcControlPlaneClient,
|
||||
}
|
||||
|
||||
impl ExecRuntime {
|
||||
pub fn new(data_dir: &Path) -> Self {
|
||||
let config = ExecConfig::from_env(data_dir);
|
||||
let _ = std::fs::create_dir_all(&config.state_dir);
|
||||
let store = ExecStore::new(&config.state_dir);
|
||||
let control_plane = MpcControlPlaneClient::new(
|
||||
config.control_plane_url.clone(),
|
||||
config.control_plane_token.clone(),
|
||||
);
|
||||
Self {
|
||||
config,
|
||||
store,
|
||||
control_plane,
|
||||
}
|
||||
}
|
||||
|
||||
// ================================================================
|
||||
// 自检 / 产物(入向端点直接消费)
|
||||
// ================================================================
|
||||
|
||||
/// 能力自检:**如实列缺口**。`readyStatus=READY` 只表示「能接单」,不表示各二进制可用。
|
||||
pub fn capabilities(&self) -> Value {
|
||||
let c = &self.config;
|
||||
let files = self.artifact_files().ok();
|
||||
json!({
|
||||
"engineReady": true,
|
||||
"endpointReady": true,
|
||||
"stateMachine": "ACCEPTED→CHECKING→CONTROL_PLANE→NETWORK→RAY→MPC",
|
||||
"dryRun": c.dry_run,
|
||||
"artifact": {
|
||||
"dir": c.artifact_dir.display().to_string(),
|
||||
"files": files.as_ref().map(|f| f.len()).unwrap_or(0),
|
||||
"algorithmHash": files.as_ref().map(dir_digest),
|
||||
},
|
||||
"controlPlane": {
|
||||
"configured": !self.control_plane.is_noop(),
|
||||
"endpoint": self.control_plane.endpoint(),
|
||||
"desiredConfigured": c.desired_sessions.is_some(),
|
||||
},
|
||||
"network": {
|
||||
"easytierBin": c.easytier_bin.display().to_string(),
|
||||
"easytierPresent": c.easytier_bin.exists(),
|
||||
"noTun": c.et_no_tun,
|
||||
"localVipConfigured": c.local_vip.is_some(),
|
||||
},
|
||||
"ray": {
|
||||
"enabled": c.ray_enabled,
|
||||
"isHead": c.ray_is_head,
|
||||
"clientPort": c.ray_client_port,
|
||||
"note": "未启用 ⇒ ZYNQ(armv7/无 python3) 由 mpc cluster/agent 单向可达等价接入",
|
||||
},
|
||||
"mpc": {
|
||||
"bin": c.mpc_bin.display().to_string(),
|
||||
"binPresent": c.mpc_bin.exists(),
|
||||
"workdir": c.mpc_workdir.display().to_string(),
|
||||
"siteEndpointsConfigured": !c.endpoint_of_site.is_empty(),
|
||||
"siteCount": c.endpoint_of_site.len(),
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
/// 读算法产物目录(排序、跳过隐藏文件),返回 `文件名 → 原始字节`。
|
||||
pub fn artifact_files(&self) -> Result<BTreeMap<String, Vec<u8>>, String> {
|
||||
let rd = std::fs::read_dir(&self.config.artifact_dir)
|
||||
.map_err(|_| format!("算法产物目录不可读: {}", self.config.artifact_dir.display()))?;
|
||||
let mut files = BTreeMap::new();
|
||||
for entry in rd.flatten() {
|
||||
let path = entry.path();
|
||||
let name = entry.file_name().to_string_lossy().to_string();
|
||||
if name.starts_with('.') || !path.is_file() {
|
||||
continue;
|
||||
}
|
||||
let bytes = std::fs::read(&path)
|
||||
.map_err(|e| format!("读产物 {} 失败: {}", name, e))?;
|
||||
files.insert(name, bytes);
|
||||
}
|
||||
if files.is_empty() {
|
||||
return Err(format!(
|
||||
"算法产物目录为空: {}(平台 ensureArtifact 会从使用方拉产物)",
|
||||
self.config.artifact_dir.display()
|
||||
));
|
||||
}
|
||||
Ok(files)
|
||||
}
|
||||
|
||||
/// `Dev/Artifact` 的响应体。
|
||||
///
|
||||
/// 🔴 `algorithmHash` 与 `files[].contentBase64` **出自同一次读盘**:平台重算
|
||||
/// `dirDigest(files)` 与基准不符就硬拒(「审的 ≠ 要跑的」)。
|
||||
pub fn artifact(&self) -> Result<Value, String> {
|
||||
let files = self.artifact_files()?;
|
||||
let hash = dir_digest(&files);
|
||||
let list: Vec<Value> = files
|
||||
.iter()
|
||||
.map(|(name, content)| {
|
||||
json!({ "name": name, "contentBase64": base64_encode(content) })
|
||||
})
|
||||
.collect();
|
||||
let entry = files
|
||||
.keys()
|
||||
.find(|n| n.ends_with(".py"))
|
||||
.or_else(|| files.keys().next())
|
||||
.cloned()
|
||||
.unwrap_or_default();
|
||||
Ok(json!({ "algorithmHash": hash, "files": list, "entry": entry }))
|
||||
}
|
||||
|
||||
// ================================================================
|
||||
// 接单
|
||||
// ================================================================
|
||||
|
||||
/// 幂等接单:`namespace + vnetName + taskId` 相同且已有非终态记录 ⇒ 复用,不重复起实例。
|
||||
///
|
||||
/// 返回 `(记录, 是否重复派发)`。
|
||||
pub fn accept(&self, task_id: &str, payload: &Value) -> (ExecRecord, bool) {
|
||||
let nax = payload
|
||||
.get("clusterConfig")
|
||||
.and_then(|c| c.get("nax"))
|
||||
.cloned()
|
||||
.unwrap_or(Value::Null);
|
||||
let namespace = text(&nax, "namespace");
|
||||
let vnet_name = text(&nax, "vnetName");
|
||||
let key = ExecStore::key_of(&namespace, &vnet_name, task_id);
|
||||
if let Some(existing) = self.store.get(&key) {
|
||||
if existing.phase != ExecPhase::Failed && existing.phase != ExecPhase::Planned {
|
||||
return (existing, true);
|
||||
}
|
||||
}
|
||||
let rec = ExecRecord::new(key, task_id, &namespace, &vnet_name);
|
||||
(self.store.insert(rec), false)
|
||||
}
|
||||
|
||||
/// 后台状态机(由 `Exec/Start` handler `tokio::spawn`)。
|
||||
pub async fn run(self: std::sync::Arc<Self>, exec_id: String, payload: Value) {
|
||||
let nax = payload
|
||||
.get("clusterConfig")
|
||||
.and_then(|c| c.get("nax"))
|
||||
.cloned()
|
||||
.unwrap_or(Value::Null);
|
||||
let vnet_name = text(&nax, "vnetName");
|
||||
let vnet_secret = text(&nax, "vnetSecret");
|
||||
let ray_head = text(&nax, "rayHead");
|
||||
let self_party = text(&payload, "selfParty");
|
||||
let parties: Vec<String> = nax
|
||||
.get("parties")
|
||||
.and_then(Value::as_array)
|
||||
.map(|a| {
|
||||
a.iter()
|
||||
.filter_map(|v| v.as_str().map(str::to_string))
|
||||
.collect()
|
||||
})
|
||||
.unwrap_or_default();
|
||||
|
||||
// ---- ① 校验产物摘要 + 组装(脱敏)执行计划 ----
|
||||
let check_note = match self.check_artifact(&payload) {
|
||||
Ok(note) => note,
|
||||
Err(reason) => {
|
||||
self.fail(&exec_id, ExecPhase::Checking, "校验 algorithmHash", reason);
|
||||
return;
|
||||
}
|
||||
};
|
||||
self.ok_step(
|
||||
&exec_id,
|
||||
ExecPhase::Checking,
|
||||
"校验 algorithmHash",
|
||||
check_note,
|
||||
);
|
||||
|
||||
let plan = self.build_plan(&payload, &nax, &parties, &self_party, &ray_head);
|
||||
self.store.mutate(&exec_id, |r| r.plan = Some(plan.clone()));
|
||||
|
||||
// ---- ② 控制面:选址 / 起实例 / 拿端点 / 回报状态 ----
|
||||
match self.control_plane_phase().await {
|
||||
Ok(detail) => self.ok_step(&exec_id, ExecPhase::ControlPlane, "控制面对账", detail),
|
||||
Err(reason) => {
|
||||
self.fail(&exec_id, ExecPhase::ControlPlane, "控制面对账", reason);
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
// ---- ③ EasyTier 组网 ----
|
||||
let et_tag = format!(
|
||||
"et-{}",
|
||||
short_key(&format!("{}|{}", vnet_name, payload.to_string()))
|
||||
);
|
||||
match self.ensure_et(&et_tag, &vnet_name, &vnet_secret) {
|
||||
Ok(detail) => self.ok_step(&exec_id, ExecPhase::Network, "EasyTier 组网", detail),
|
||||
Err(reason) => {
|
||||
self.fail(&exec_id, ExecPhase::Network, "EasyTier 组网", reason);
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
// ---- ④ Ray 分布式计算层 ----
|
||||
let self_site = self
|
||||
.config
|
||||
.site_of_party
|
||||
.get(&self_party)
|
||||
.cloned()
|
||||
.unwrap_or_else(|| self_party.clone());
|
||||
match self.ensure_ray(&et_tag, &ray_head, &self_site) {
|
||||
Ok(detail) => self.ok_step(&exec_id, ExecPhase::Ray, "Ray 计算层", detail),
|
||||
Err(reason) => {
|
||||
self.fail(&exec_id, ExecPhase::Ray, "Ray 计算层", reason);
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
// ---- ⑤ mpc 算子点火 ----
|
||||
match self.ensure_mpc(&et_tag, &self_site, &parties) {
|
||||
Ok(detail) => self.ok_step(&exec_id, ExecPhase::Mpc, "MPC 点火", detail),
|
||||
Err(reason) => {
|
||||
self.fail(&exec_id, ExecPhase::Mpc, "MPC 点火", reason);
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
// ---- ⑥ 收口 ----
|
||||
if self.config.dry_run {
|
||||
self.ok_step(
|
||||
&exec_id,
|
||||
ExecPhase::Planned,
|
||||
"收口",
|
||||
"dry-run:执行计划已落盘,未拉起任何进程 / 未 POST /desired ⇒ 不算执行成功",
|
||||
);
|
||||
} else {
|
||||
self.ok_step(
|
||||
&exec_id,
|
||||
ExecPhase::Succeeded,
|
||||
"收口",
|
||||
"各阶段已受理并拉起;MPC 结果需查算子日志与载荷通路证据",
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
// ================================================================
|
||||
// 各阶段实现
|
||||
// ================================================================
|
||||
|
||||
/// 产物摘要校验:**只有本地真有产物才校验**(非使用方可能没有产物,不硬拒)。
|
||||
fn check_artifact(&self, payload: &Value) -> Result<String, String> {
|
||||
let expected = payload
|
||||
.get("algorithmHash")
|
||||
.and_then(Value::as_str)
|
||||
.unwrap_or("");
|
||||
if expected.is_empty() {
|
||||
return Ok("payload 未带 algorithmHash ⇒ 跳过摘要校验".to_string());
|
||||
}
|
||||
let files = match self.artifact_files() {
|
||||
Ok(f) => f,
|
||||
Err(_) => {
|
||||
return Ok("本地无算法产物(本方可能不是使用方)⇒ 跳过摘要校验".to_string())
|
||||
}
|
||||
};
|
||||
let actual = dir_digest(&files);
|
||||
if actual != expected {
|
||||
return Err(format!(
|
||||
"algorithmHash 不符:平台={} 本地={}(审的 ≠ 要跑的,拒绝执行)",
|
||||
expected, actual
|
||||
));
|
||||
}
|
||||
Ok(format!("algorithmHash 一致({}…)", &actual[..12]))
|
||||
}
|
||||
|
||||
/// 组装**脱敏**执行计划:命令、参数来源、能力边界一目了然,不含任何凭据。
|
||||
fn build_plan(
|
||||
&self,
|
||||
payload: &Value,
|
||||
nax: &Value,
|
||||
parties: &[String],
|
||||
self_party: &str,
|
||||
ray_head: &str,
|
||||
) -> Value {
|
||||
let c = &self.config;
|
||||
let vnet_name = text(nax, "vnetName");
|
||||
let vnet_secret = text(nax, "vnetSecret");
|
||||
let vip = c.local_vip.clone().unwrap_or_default();
|
||||
let et_args = self.et_args(&vnet_name, &vnet_secret, &vip);
|
||||
let (ray_args, ray_note) = self.ray_args(ray_head, self_party);
|
||||
let (mpc_args, peers) = self.mpc_args(parties, self_party);
|
||||
|
||||
json!({
|
||||
"contract": {
|
||||
"namespace": text(nax, "namespace"),
|
||||
"vnetName": vnet_name,
|
||||
"vnetSecretConfigured": !vnet_secret.is_empty(),
|
||||
"rayHead": ray_head,
|
||||
"rayHeadName": text(nax, "rayHeadName"),
|
||||
"parties": parties,
|
||||
"selfParty": self_party,
|
||||
"consumerParty": text(nax, "consumerParty"),
|
||||
// 🔴 死地址只作为「为什么不用它」的证据,绝不作为执行目标
|
||||
"k8sServiceDnsUnusable": true,
|
||||
"clusterConfigParties": payload.get("clusterConfig").and_then(|c| c.get("parties")).cloned().unwrap_or(Value::Null),
|
||||
},
|
||||
"controlPlane": {
|
||||
"endpoint": self.control_plane.endpoint(),
|
||||
"calls": ["GET /nodes", "POST /desired", "GET /sessions", "GET /report"],
|
||||
"desiredConfigured": c.desired_sessions.is_some(),
|
||||
"boundary": "/desired 只起 TEE guest + 公布端点,不下发算子",
|
||||
},
|
||||
"easyTier": {
|
||||
"bin": c.easytier_bin.display().to_string(),
|
||||
"args": process::redact_args(&et_args),
|
||||
"secretInArgvKnownLimit": true,
|
||||
"noTun": c.et_no_tun,
|
||||
},
|
||||
"ray": {
|
||||
"enabled": c.ray_enabled,
|
||||
"bin": c.ray_bin.display().to_string(),
|
||||
"args": process::redact_args(&ray_args),
|
||||
"note": ray_note,
|
||||
"portMapping": "rayHead 端口按 GCS 解释;Ray Client Server 端口由 TDC_EXEC_RAY_CLIENT_PORT 单配,两者需同时可达",
|
||||
},
|
||||
"mpc": {
|
||||
"bin": c.mpc_bin.display().to_string(),
|
||||
"operator": c.mpc_operator,
|
||||
"peers": peers,
|
||||
"args": process::redact_args(&mpc_args),
|
||||
"orderConstraint": "编号大的先起(carol → alice),且需先确认对端 LISTEN ≥6s",
|
||||
},
|
||||
"dryRun": c.dry_run,
|
||||
})
|
||||
}
|
||||
|
||||
async fn control_plane_phase(&self) -> Result<String, String> {
|
||||
if self.control_plane.is_noop() {
|
||||
return Ok(format!(
|
||||
"{} ⇒ 跳过控制面对账(如实标注,未伪装调用);算子点火仍由执行层承担",
|
||||
NOOP_MESSAGE
|
||||
));
|
||||
}
|
||||
let nodes = self.control_plane.nodes().await?;
|
||||
let desired = match &self.config.desired_sessions {
|
||||
None => "未配置期望态 TDC_EXEC_DESIRED_SESSIONS ⇒ 未起 TEE 实例".to_string(),
|
||||
Some(raw) if self.config.dry_run => {
|
||||
format!("dry-run ⇒ 未 POST /desired(已备 {} 字节期望态)", raw.len())
|
||||
}
|
||||
Some(raw) => {
|
||||
let sessions: Value = serde_json::from_str(raw)
|
||||
.map_err(|e| format!("期望态不是合法 JSON 数组: {}", e))?;
|
||||
self.control_plane.desired(&sessions).await?;
|
||||
"已 POST /desired(选址 → create/attest)".to_string()
|
||||
}
|
||||
};
|
||||
let sessions = self.control_plane.sessions().await?;
|
||||
let report = self.control_plane.report().await?;
|
||||
Ok(format!(
|
||||
"GET /nodes ✓({} 字节)、{}、GET /sessions ✓({} 字节)、GET /report ✓({} 字节);端点已回收",
|
||||
serde_json::to_string(&nodes).map(|s| s.len()).unwrap_or(0),
|
||||
desired,
|
||||
serde_json::to_string(&sessions).map(|s| s.len()).unwrap_or(0),
|
||||
serde_json::to_string(&report).map(|s| s.len()).unwrap_or(0),
|
||||
))
|
||||
}
|
||||
|
||||
fn et_args(&self, vnet_name: &str, vnet_secret: &str, vip: &str) -> Vec<String> {
|
||||
let c = &self.config;
|
||||
let mut a = vec![
|
||||
"--network-name".into(),
|
||||
vnet_name.into(),
|
||||
"--network-secret".into(),
|
||||
vnet_secret.into(),
|
||||
];
|
||||
if !vip.is_empty() {
|
||||
a.push("--ipv4".into());
|
||||
a.push(vip.into());
|
||||
}
|
||||
a.extend([
|
||||
"--hostname".into(),
|
||||
"tdc-rs".into(),
|
||||
"--instance-name".into(),
|
||||
format!("ssw-{}", vip),
|
||||
"--machine-id".into(),
|
||||
"ssw-tdc-rs".into(),
|
||||
"--listeners".into(),
|
||||
c.et_listeners.clone(),
|
||||
"--rpc-portal".into(),
|
||||
c.et_rpc_portal.clone(),
|
||||
]);
|
||||
if c.et_no_tun {
|
||||
a.push("--no-tun".into());
|
||||
}
|
||||
if let Some(p) = &c.et_proxy_networks {
|
||||
a.push("--proxy-networks".into());
|
||||
a.push(p.clone());
|
||||
}
|
||||
if let Some(s) = &c.et_socks5 {
|
||||
a.push("--socks5".into());
|
||||
a.push(s.clone());
|
||||
}
|
||||
if let Some(r) = &c.et_relay {
|
||||
a.push("--peers".into());
|
||||
a.push(r.clone());
|
||||
}
|
||||
a
|
||||
}
|
||||
|
||||
fn ensure_et(&self, tag: &str, vnet_name: &str, vnet_secret: &str) -> Result<String, String> {
|
||||
let c = &self.config;
|
||||
if vnet_name.is_empty() || vnet_secret.is_empty() {
|
||||
return Err("下发载荷缺 clusterConfig.nax.vnetName / vnetSecret ⇒ 拒绝建网".to_string());
|
||||
}
|
||||
let vip = c
|
||||
.local_vip
|
||||
.clone()
|
||||
.ok_or_else(|| "未配置 TDC_EXEC_LOCAL_VIP ⇒ 不猜 vIP,拒绝建网".to_string())?;
|
||||
let args = self.et_args(vnet_name, vnet_secret, &vip);
|
||||
let preview = SpawnRequest::new(&c.easytier_bin, args.clone(), &c.state_dir, tag).preview();
|
||||
if c.dry_run {
|
||||
return Ok(format!("dry-run ⇒ 未拉起 ET;计划命令:{}", preview));
|
||||
}
|
||||
if !c.easytier_bin.exists() {
|
||||
return Err(format!("EasyTier 二进制不存在: {}", c.easytier_bin.display()));
|
||||
}
|
||||
// 指纹含 secret 的哈希(不落明文)⇒ 密钥变了不会再复用旧实例。
|
||||
let fingerprint = short_key(&format!("{}|{}|{}", vnet_name, vnet_secret, vip));
|
||||
let guard = ensure_single_instance(&c.state_dir, tag, &fingerprint)?;
|
||||
if let InstanceGuard::Reused(pid) = guard {
|
||||
return Ok(format!("复用已有 EasyTier 实例(pid={}),未重复拉起", pid));
|
||||
}
|
||||
// 凭证落 0600 env 文件(不进状态 JSON / 不进日志);argv 展开是 easytier CLI 的已知限制。
|
||||
let env_path = c.state_dir.join(format!("{}.env", tag));
|
||||
write_secret_file(
|
||||
&env_path,
|
||||
&format!("EASYTIER_NETWORK_SECRET={}\n", vnet_secret),
|
||||
)?;
|
||||
let req = SpawnRequest::new(&c.easytier_bin, args, &c.state_dir, tag);
|
||||
let pid = spawn_detached(&req)?;
|
||||
guard.commit(&fingerprint);
|
||||
Ok(format!(
|
||||
"已拉起 EasyTier(pid={},vIP={},noTun={});凭证在 0600 env 文件",
|
||||
pid, vip, c.et_no_tun
|
||||
))
|
||||
}
|
||||
|
||||
fn ray_args(&self, ray_head: &str, self_party: &str) -> (Vec<String>, String) {
|
||||
let c = &self.config;
|
||||
let (host, gcs_port) = ray_head
|
||||
.split_once(':')
|
||||
.map(|(h, p)| (h.to_string(), p.to_string()))
|
||||
.unwrap_or_else(|| (ray_head.to_string(), "6379".to_string()));
|
||||
let resources = c
|
||||
.ray_resources
|
||||
.clone()
|
||||
.unwrap_or_else(|| format!("{{\"{}_node\":1}}", self_party));
|
||||
if c.ray_is_head {
|
||||
(
|
||||
vec![
|
||||
"start".into(),
|
||||
"--head".into(),
|
||||
format!("--port={}", gcs_port),
|
||||
format!("--ray-client-server-port={}", c.ray_client_port),
|
||||
format!("--node-ip-address={}", host),
|
||||
"--dashboard-host=0.0.0.0".into(),
|
||||
// 🔴 不经 shell ⇒ 参数里不能带引号,否则引号会原样传给 ray
|
||||
format!("--resources={}", resources),
|
||||
],
|
||||
format!(
|
||||
"head 绑 {}(GCS 端口取 rayHead 的 {}),Client 端口 {} 需同时可达",
|
||||
host, gcs_port, c.ray_client_port
|
||||
),
|
||||
)
|
||||
} else {
|
||||
(
|
||||
vec![
|
||||
"start".into(),
|
||||
format!("--address={}:{}", host, gcs_port),
|
||||
format!("--resources={}", resources),
|
||||
],
|
||||
format!("worker 加入 {}:{}", host, gcs_port),
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
fn ensure_ray(&self, tag: &str, ray_head: &str, self_site: &str) -> Result<String, String> {
|
||||
let c = &self.config;
|
||||
if !c.ray_enabled {
|
||||
return Ok(format!(
|
||||
"未启用 Ray(site={});按裁决由 mpc cluster/agent 单向可达形态等价接入,不硬移植 Ray 到 armv7",
|
||||
self_site
|
||||
));
|
||||
}
|
||||
if ray_head.is_empty() {
|
||||
return Err("启用了 Ray 但下发载荷缺 clusterConfig.nax.rayHead ⇒ 无法确定 head".to_string());
|
||||
}
|
||||
let (args, note) = self.ray_args(ray_head, self_site);
|
||||
let ray_tag = format!("ray-{}", tag.trim_start_matches("et-"));
|
||||
let preview = format!("{} {}", c.ray_bin.display(), args.join(" "));
|
||||
if c.dry_run {
|
||||
return Ok(format!("dry-run ⇒ 未拉起 Ray;{};计划命令:{}", note, preview));
|
||||
}
|
||||
// `ray start` 会自行 daemonize ⇒ 本层 pid 只记录启动器,重复派发由 ray 自身的
|
||||
// "already running" 检测兜底;这里仍写 pid/fp 文件留痕。
|
||||
let fingerprint = short_key(&format!("ray|{}|{}", ray_head, self_site));
|
||||
let guard = ensure_single_instance(&c.state_dir, &ray_tag, &fingerprint)?;
|
||||
if let InstanceGuard::Reused(pid) = guard {
|
||||
return Ok(format!("复用 Ray 启动记录(pid={})", pid));
|
||||
}
|
||||
let req = SpawnRequest::new(&c.ray_bin, args, &c.state_dir, &ray_tag);
|
||||
let pid = spawn_detached(&req)?;
|
||||
guard.commit(&fingerprint);
|
||||
Ok(format!("已执行 ray start(启动器 pid={});{}", pid, note))
|
||||
}
|
||||
|
||||
fn mpc_args(&self, parties: &[String], self_party: &str) -> (Vec<String>, String) {
|
||||
let c = &self.config;
|
||||
let sites: Vec<String> = parties
|
||||
.iter()
|
||||
.filter_map(|p| c.site_of_party.get(p).cloned())
|
||||
.collect();
|
||||
let peers = sites
|
||||
.iter()
|
||||
.filter_map(|s| c.endpoint_of_site.get(s).map(|e| format!("{}={}", s, e)))
|
||||
.collect::<Vec<_>>()
|
||||
.join(",");
|
||||
let site = c
|
||||
.site_of_party
|
||||
.get(self_party)
|
||||
.cloned()
|
||||
.unwrap_or_else(|| self_party.to_string());
|
||||
(
|
||||
vec![
|
||||
c.mpc_operator.clone(),
|
||||
"--site".into(),
|
||||
site,
|
||||
"--sites".into(),
|
||||
sites.join(","),
|
||||
"--listen".into(),
|
||||
format!("0.0.0.0:{}", c.mpc_port),
|
||||
"--peers".into(),
|
||||
peers.clone(),
|
||||
"--report".into(),
|
||||
"json".into(),
|
||||
],
|
||||
peers,
|
||||
)
|
||||
}
|
||||
|
||||
fn ensure_mpc(&self, tag: &str, self_site: &str, parties: &[String]) -> Result<String, String> {
|
||||
let c = &self.config;
|
||||
if c.endpoint_of_site.is_empty() {
|
||||
return Err(
|
||||
"未配置 TDC_EXEC_SITE_ENDPOINTS ⇒ 不回落 K8s Service DNS 死地址,拒绝点火".to_string(),
|
||||
);
|
||||
}
|
||||
for p in parties {
|
||||
match c.site_of_party.get(p) {
|
||||
None => return Err(format!("party {} 无 TDC_EXEC_PARTY_SITE_MAP 映射", p)),
|
||||
Some(s) if !c.endpoint_of_site.contains_key(s) => {
|
||||
return Err(format!("site {} 无端点配置", s))
|
||||
}
|
||||
_ => {}
|
||||
}
|
||||
}
|
||||
let (_args, peers) = self.mpc_args(parties, &self_party_of(c, self_site));
|
||||
let wd = &c.mpc_workdir;
|
||||
let needed = [
|
||||
wd.join(format!("{}.ids", self_site)),
|
||||
wd.join("sess-master.txt"),
|
||||
wd.join("sess-auth.txt"),
|
||||
];
|
||||
if c.dry_run {
|
||||
return Ok(format!(
|
||||
"dry-run ⇒ 未点火;peers={};物料目录 {}({} 项就绪)",
|
||||
peers,
|
||||
wd.display(),
|
||||
needed.iter().filter(|p| p.exists()).count()
|
||||
));
|
||||
}
|
||||
if !c.mpc_bin.exists() {
|
||||
return Err(format!("mpc 二进制不存在: {}", c.mpc_bin.display()));
|
||||
}
|
||||
let missing: Vec<String> = needed
|
||||
.iter()
|
||||
.filter(|p| !p.exists())
|
||||
.map(|p| p.display().to_string())
|
||||
.collect();
|
||||
if !missing.is_empty() {
|
||||
return Err(format!("MPC 物料缺失:{}", missing.join(", ")));
|
||||
}
|
||||
let ids = std::fs::read_to_string(&needed[0])
|
||||
.map_err(|e| format!("读 {} 失败: {}", needed[0].display(), e))?
|
||||
.trim()
|
||||
.to_string();
|
||||
let seed = std::fs::read_to_string(&needed[1])
|
||||
.map_err(|e| format!("读 seed 失败: {}", e))?
|
||||
.trim()
|
||||
.to_string();
|
||||
let psk = c.mpc_psk.clone().unwrap_or_else(|| {
|
||||
std::fs::read_to_string(&needed[2])
|
||||
.map(|s| s.trim().to_string())
|
||||
.unwrap_or_default()
|
||||
});
|
||||
// 用真实物料补齐 CLI(build_plan 里的骨架只用于展示,不落 ids/seed)。
|
||||
let mut args = vec![c.mpc_operator.clone(), "--site".into(), self_site.to_string()];
|
||||
args.extend([
|
||||
"--sites".into(),
|
||||
parties
|
||||
.iter()
|
||||
.filter_map(|p| c.site_of_party.get(p))
|
||||
.cloned()
|
||||
.collect::<Vec<_>>()
|
||||
.join(","),
|
||||
"--ids".into(),
|
||||
ids,
|
||||
"--seed".into(),
|
||||
seed,
|
||||
"--listen".into(),
|
||||
format!("0.0.0.0:{}", c.mpc_port),
|
||||
"--peers".into(),
|
||||
peers.clone(),
|
||||
"--report".into(),
|
||||
"json".into(),
|
||||
]);
|
||||
let mpc_tag = format!("mpc-{}", tag.trim_start_matches("et-"));
|
||||
let req = SpawnRequest::new(&c.mpc_bin, args, &c.state_dir, &mpc_tag)
|
||||
.with_env("MPC_PSK", &psk);
|
||||
let pid = spawn_detached(&req)?;
|
||||
Ok(format!(
|
||||
"已点火 mpc {}(pid={},site={},peers={});跨机启动顺序与 LISTEN 就绪由部署侧保证",
|
||||
c.mpc_operator, pid, self_site, peers
|
||||
))
|
||||
}
|
||||
|
||||
// ---- 状态写入小工具 ----
|
||||
|
||||
fn ok_step(&self, id: &str, phase: ExecPhase, name: &str, detail: impl Into<String>) {
|
||||
self.store
|
||||
.mutate(id, |r| r.step(phase, name, true, detail.into()));
|
||||
}
|
||||
|
||||
fn fail(&self, id: &str, phase: ExecPhase, name: &str, reason: impl Into<String>) {
|
||||
self.store
|
||||
.mutate(id, |r| r.step(phase, name, false, reason.into()));
|
||||
}
|
||||
}
|
||||
|
||||
/// `ensure_mpc` 用 site 反查 selfParty(映射双向唯一,缺省回落 site 本身)。
|
||||
fn self_party_of(c: &ExecConfig, site: &str) -> String {
|
||||
c.site_of_party
|
||||
.iter()
|
||||
.find(|(_, s)| s.as_str() == site)
|
||||
.map(|(p, _)| p.clone())
|
||||
.unwrap_or_else(|| site.to_string())
|
||||
}
|
||||
|
||||
fn text(v: &Value, key: &str) -> String {
|
||||
v.get(key)
|
||||
.and_then(Value::as_str)
|
||||
.unwrap_or("")
|
||||
.to_string()
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn cfg(overrides: &[(&str, &str)]) -> ExecConfig {
|
||||
let env: BTreeMap<String, String> = overrides
|
||||
.iter()
|
||||
.map(|(k, v)| (k.to_string(), v.to_string()))
|
||||
.collect();
|
||||
ExecConfig::from_raw(&env, Path::new("/tmp/tdc-exec-test"))
|
||||
}
|
||||
|
||||
fn runtime(c: ExecConfig, tag: &str) -> ExecRuntime {
|
||||
let dir = std::env::temp_dir().join(format!("tdc-exec-{}-{}", tag, std::process::id()));
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
ExecRuntime {
|
||||
config: c,
|
||||
store: ExecStore::new(&dir),
|
||||
control_plane: MpcControlPlaneClient::new(None, String::new()),
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn plan_never_contains_the_vnet_secret() {
|
||||
let r = runtime(cfg(&[(config::ENV_LOCAL_VIP, "10.12.0.2")]), "plan");
|
||||
let nax = json!({
|
||||
"vnetName": "ssw-SPACE", "vnetSecret": "ssw-TOP-SECRET-123",
|
||||
"namespace": "nax-1-v1", "rayHead": "10.12.0.2:30417",
|
||||
"rayHeadName": "nax-head-1-v1", "parties": ["p1","p2"],
|
||||
"consumerParty": "p1",
|
||||
});
|
||||
let payload = json!({"clusterConfig": {"nax": nax.clone(), "parties": {}}, "selfParty": "p1"});
|
||||
let plan = r.build_plan(
|
||||
&payload,
|
||||
&nax,
|
||||
&["p1".into(), "p2".into()],
|
||||
"p1",
|
||||
"10.12.0.2:30417",
|
||||
);
|
||||
let s = serde_json::to_string(&plan).unwrap();
|
||||
assert!(!s.contains("ssw-TOP-SECRET-123"), "计划泄漏了 vnetSecret");
|
||||
assert_eq!(plan["contract"]["vnetSecretConfigured"], true);
|
||||
assert_eq!(plan["contract"]["k8sServiceDnsUnusable"], true);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn mpc_peers_come_from_explicit_config_not_k8s_dns() {
|
||||
let r = runtime(
|
||||
cfg(&[(
|
||||
config::ENV_SITE_ENDPOINTS,
|
||||
"alice=10.12.0.1:9449,carol=10.12.0.2:9449",
|
||||
)]),
|
||||
"peers",
|
||||
);
|
||||
let (_args, peers) = r.mpc_args(&["p1".into(), "p2".into()], "p1");
|
||||
assert_eq!(peers, "alice=10.12.0.1:9449,carol=10.12.0.2:9449");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn ray_head_port_is_interpreted_as_gcs() {
|
||||
let r = runtime(
|
||||
cfg(&[(config::ENV_RAY_ENABLED, "1"), (config::ENV_RAY_IS_HEAD, "1")]),
|
||||
"ray",
|
||||
);
|
||||
let (args, note) = r.ray_args("10.12.0.2:30417", "p1");
|
||||
assert!(args.iter().any(|a| a == "--port=30417"));
|
||||
assert!(args.iter().any(|a| a == "--ray-client-server-port=10001"));
|
||||
assert!(note.contains("30417"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn accept_is_idempotent_per_namespace_vnet_task() {
|
||||
let r = runtime(cfg(&[]), "accept");
|
||||
let payload = json!({"clusterConfig":{"nax":{"namespace":"ns","vnetName":"vn"}}});
|
||||
let (a, dup1) = r.accept("t1", &payload);
|
||||
let (b, dup2) = r.accept("t1", &payload);
|
||||
assert!(!dup1);
|
||||
assert!(dup2);
|
||||
assert_eq!(a.exec_id, b.exec_id);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn dry_run_state_is_planned_not_succeeded() {
|
||||
// 状态机收口阶段的语义:Planned 与 Succeeded 是两个不同枚举值。
|
||||
assert_ne!(ExecPhase::Planned, ExecPhase::Succeeded);
|
||||
assert_eq!(ExecPhase::Planned.as_str(), "PLANNED");
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,249 @@
|
||||
//! 子进程生命周期(EasyTier / Ray / mpc 共用)。
|
||||
//!
|
||||
//! ## 为什么不用 systemd
|
||||
//!
|
||||
//! ZYNQ 是 BusyBox、无 systemd(现状本来就是 `nohup easytier-core … &`,
|
||||
//! `ssw-tds-test/build/verify-platform-vnet-zynq.sh:79-89`)。`tokio::process` /
|
||||
//! `std::process::Command` + **pid 文件** 是两端唯一共同可用的形态。
|
||||
//!
|
||||
//! ## 密钥纪律
|
||||
//!
|
||||
//! - `vnetSecret` 落 0600 env 文件(`EASYTIER_NETWORK_SECRET=…`),**不进状态 JSON / 日志**;
|
||||
//! - `MPC_PSK` 只经 `Command::env` 进子进程,不落任何文件;
|
||||
//! - `redact_args` 对命令预览做脱敏,状态文件里只留可复现的骨架。
|
||||
//! - ⚠️ **已知限制**:`easytier-core` 的 CLI 只接受 `--network-secret <明文>`,
|
||||
//! systemd 版靠 `EnvironmentFile` 展开 `${EASYTIER_NETWORK_SECRET}`,**展开后仍在 argv**。
|
||||
//! 这里如实标注,不宣称「ps aux 不可见」(见 `decision` §4 §5.4)。
|
||||
|
||||
use std::fs::{self, OpenOptions};
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::process::{Command, Stdio};
|
||||
|
||||
/// 一次脱离父进程的启动请求。
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct SpawnRequest {
|
||||
pub program: PathBuf,
|
||||
pub args: Vec<String>,
|
||||
/// 只进子进程环境(不落盘、不进命令行)。
|
||||
pub envs: Vec<(String, String)>,
|
||||
pub log_path: PathBuf,
|
||||
pub pid_path: PathBuf,
|
||||
}
|
||||
|
||||
impl SpawnRequest {
|
||||
pub fn new(program: impl Into<PathBuf>, args: Vec<String>, dir: &Path, tag: &str) -> Self {
|
||||
Self {
|
||||
program: program.into(),
|
||||
args,
|
||||
envs: Vec::new(),
|
||||
log_path: dir.join(format!("{}.log", tag)),
|
||||
pid_path: dir.join(format!("{}.pid", tag)),
|
||||
}
|
||||
}
|
||||
|
||||
pub fn with_env(mut self, key: &str, value: &str) -> Self {
|
||||
self.envs.push((key.to_string(), value.to_string()));
|
||||
self
|
||||
}
|
||||
|
||||
/// 状态文件里可留存的**脱敏**命令预览。
|
||||
pub fn preview(&self) -> String {
|
||||
let program = self.program.to_string_lossy();
|
||||
let args = redact_args(&self.args).join(" ");
|
||||
format!("{} {}", program, args).trim().to_string()
|
||||
}
|
||||
}
|
||||
|
||||
/// 脱敏:把几个已知敏感开关的**值**换成 `***`(键名保留,便于排障)。
|
||||
pub fn redact_args(args: &[String]) -> Vec<String> {
|
||||
const SECRET_FLAGS: &[&str] = &[
|
||||
"--network-secret",
|
||||
"--password",
|
||||
"--token",
|
||||
"--psk",
|
||||
"--bearer",
|
||||
];
|
||||
let mut out = Vec::with_capacity(args.len());
|
||||
let mut hide_next = false;
|
||||
for a in args {
|
||||
if hide_next {
|
||||
out.push("***".to_string());
|
||||
hide_next = false;
|
||||
continue;
|
||||
}
|
||||
if SECRET_FLAGS.iter().any(|f| a == f) {
|
||||
out.push(a.clone());
|
||||
hide_next = true;
|
||||
} else if SECRET_FLAGS.iter().any(|f| a.starts_with(&format!("{}=", f))) {
|
||||
out.push(format!("{}=***", a.split('=').next().unwrap_or("")));
|
||||
} else {
|
||||
out.push(a.clone());
|
||||
}
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
/// 写 0600 文件(含 umask 收紧,避免同机其它用户读到凭证)。
|
||||
pub fn write_secret_file(path: &Path, content: &str) -> Result<(), String> {
|
||||
if let Some(parent) = path.parent() {
|
||||
fs::create_dir_all(parent).map_err(|e| format!("建状态目录失败: {}", e))?;
|
||||
}
|
||||
fs::write(path, content).map_err(|e| format!("写凭证文件失败: {}", e))?;
|
||||
#[cfg(unix)]
|
||||
{
|
||||
use std::os::unix::fs::PermissionsExt;
|
||||
fs::set_permissions(path, fs::Permissions::from_mode(0o600))
|
||||
.map_err(|e| format!("收紧凭证文件权限失败: {}", e))?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// 进程是否还活着(`kill -0`,BusyBox / macOS / Linux 都有)。
|
||||
pub fn pid_is_alive(pid: u32) -> bool {
|
||||
Command::new("kill")
|
||||
.arg("-0")
|
||||
.arg(pid.to_string())
|
||||
.stdout(Stdio::null())
|
||||
.stderr(Stdio::null())
|
||||
.status()
|
||||
.map(|s| s.success())
|
||||
.unwrap_or(false)
|
||||
}
|
||||
|
||||
/// 读 pid 文件;文件不存在 / 内容坏 / 进程已死 ⇒ `None`。
|
||||
pub fn live_pid(pid_path: &Path) -> Option<u32> {
|
||||
let text = fs::read_to_string(pid_path).ok()?;
|
||||
let pid = text.trim().parse::<u32>().ok()?;
|
||||
pid_is_alive(pid).then_some(pid)
|
||||
}
|
||||
|
||||
/// 启动常驻子进程,stdout/stderr 追加到日志文件,pid 落文件。
|
||||
///
|
||||
/// 🔴 **不 `wait()`**:Rust 的 `Child` drop 不会杀子进程(无 `kill_on_drop`),
|
||||
/// 正好是「连接器退出后 ET 仍在」的语义;重启靠 pid 文件复用,避免重复实例。
|
||||
pub fn spawn_detached(req: &SpawnRequest) -> Result<u32, String> {
|
||||
if !req.program.exists() && req.program.components().count() > 1 {
|
||||
return Err(format!("二进制不存在: {}", req.program.display()));
|
||||
}
|
||||
if let Some(parent) = req.log_path.parent() {
|
||||
fs::create_dir_all(parent).map_err(|e| format!("建日志目录失败: {}", e))?;
|
||||
}
|
||||
let log = OpenOptions::new()
|
||||
.create(true)
|
||||
.append(true)
|
||||
.open(&req.log_path)
|
||||
.map_err(|e| format!("打开日志失败: {}", e))?;
|
||||
let err = log
|
||||
.try_clone()
|
||||
.map_err(|e| format!("克隆日志句柄失败: {}", e))?;
|
||||
let mut cmd = Command::new(&req.program);
|
||||
cmd.args(&req.args)
|
||||
.stdin(Stdio::null())
|
||||
.stdout(Stdio::from(log))
|
||||
.stderr(Stdio::from(err));
|
||||
for (k, v) in &req.envs {
|
||||
cmd.env(k, v);
|
||||
}
|
||||
let child = cmd
|
||||
.spawn()
|
||||
.map_err(|e| format!("拉起 {} 失败: {}", req.program.display(), e))?;
|
||||
let pid = child.id();
|
||||
fs::write(&req.pid_path, pid.to_string())
|
||||
.map_err(|e| format!("写 pid 文件失败: {}", e))?;
|
||||
Ok(pid)
|
||||
}
|
||||
|
||||
/// 幂等守卫:同 key 已有活实例时,校验参数指纹一致;不一致则 fail-closed 拒复用。
|
||||
///
|
||||
/// 「启动时已入网 + 派发时又起一个」(交接 §5.5)靠它收敛。
|
||||
pub fn ensure_single_instance(
|
||||
dir: &Path,
|
||||
tag: &str,
|
||||
fingerprint: &str,
|
||||
) -> Result<InstanceGuard, String> {
|
||||
let pid_path = dir.join(format!("{}.pid", tag));
|
||||
let fp_path = dir.join(format!("{}.fp", tag));
|
||||
if let Some(pid) = live_pid(&pid_path) {
|
||||
let old = fs::read_to_string(&fp_path).unwrap_or_default();
|
||||
if old != fingerprint {
|
||||
return Err(format!(
|
||||
"同一网络已有活实例(pid={})但参数不一致 ⇒ 拒绝重复拉起;请先停止旧实例",
|
||||
pid
|
||||
));
|
||||
}
|
||||
return Ok(InstanceGuard::Reused(pid));
|
||||
}
|
||||
Ok(InstanceGuard::Start {
|
||||
pid_path,
|
||||
fp_path,
|
||||
})
|
||||
}
|
||||
|
||||
pub enum InstanceGuard {
|
||||
/// 复用已有活实例。
|
||||
Reused(u32),
|
||||
/// 需要新起,并写入这两个文件。
|
||||
Start { pid_path: PathBuf, fp_path: PathBuf },
|
||||
}
|
||||
|
||||
impl InstanceGuard {
|
||||
/// 新起成功后记录参数指纹(**指纹用脱敏后的命令**,不含密钥)。
|
||||
pub fn commit(&self, fingerprint: &str) {
|
||||
if let Self::Start { fp_path, .. } = self {
|
||||
let _ = fs::write(fp_path, fingerprint);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn secret_values_are_redacted_but_keys_kept() {
|
||||
let args: Vec<String> = [
|
||||
"--network-name",
|
||||
"ssw-SPACE",
|
||||
"--network-secret",
|
||||
"ssw-deadbeef",
|
||||
"--ipv4",
|
||||
"10.12.0.2",
|
||||
"--token=abc123",
|
||||
]
|
||||
.iter()
|
||||
.map(|s| s.to_string())
|
||||
.collect();
|
||||
let r = redact_args(&args);
|
||||
assert_eq!(r[3], "***");
|
||||
assert_eq!(r[6], "--token=***");
|
||||
assert_eq!(r[1], "ssw-SPACE");
|
||||
assert!(!r.join(" ").contains("ssw-deadbeef"));
|
||||
assert!(!r.join(" ").contains("abc123"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn preview_never_leaks_the_secret() {
|
||||
let r = SpawnRequest::new(
|
||||
"/usr/local/bin/easytier-core",
|
||||
vec!["--network-secret".into(), "s3cr3t".into()],
|
||||
Path::new("/tmp"),
|
||||
"et",
|
||||
);
|
||||
assert!(!r.preview().contains("s3cr3t"));
|
||||
assert!(r.preview().contains("easytier-core"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn ensure_single_instance_rejects_conflicting_fingerprint() {
|
||||
let dir = std::env::temp_dir().join(format!("tdc-exec-guard-{}", std::process::id()));
|
||||
let _ = fs::remove_dir_all(&dir);
|
||||
fs::create_dir_all(&dir).unwrap();
|
||||
// 用当前进程自己当「活实例」
|
||||
fs::write(dir.join("et.pid"), std::process::id().to_string()).unwrap();
|
||||
fs::write(dir.join("et.fp"), "old-args").unwrap();
|
||||
let g = ensure_single_instance(&dir, "et", "old-args").unwrap();
|
||||
assert!(matches!(g, InstanceGuard::Reused(_)));
|
||||
assert!(ensure_single_instance(&dir, "et", "new-args").is_err());
|
||||
let _ = fs::remove_dir_all(&dir);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,246 @@
|
||||
//! 执行状态(内存索引 + JSON 落盘,重启可恢复)。
|
||||
//!
|
||||
//! 🔴 **不进 `tdc` 实体表**:`entity::ALL_SPECS.len() == 18` 是注册表断言,新增执行状态
|
||||
//! 若走实体层就要动它 + 建表 + 迁移。执行态是**进程运行态**(派发幂等、步骤轨迹、
|
||||
//! pid 文件),不是业务实体 ⇒ 放 `data_dir/exec/*.json`,与 `db_lock` 无竞争。
|
||||
//!
|
||||
//! 🔴 **落盘内容脱敏**:`plan` 里的命令经过 `process::redact_args`,`vnetSecret` /
|
||||
//! Bearer / MPC_PSK 一律不写状态文件(只有「已配置 / 未配置」布尔与路径)。
|
||||
|
||||
use std::collections::BTreeMap;
|
||||
use std::fs;
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::sync::Mutex;
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
use serde_json::Value;
|
||||
|
||||
use super::digest::short_key;
|
||||
|
||||
/// 状态机阶段(顺序即 `Exec/Start` 后台任务的推进顺序)。
|
||||
#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq)]
|
||||
#[serde(rename_all = "SCREAMING_SNAKE_CASE")]
|
||||
pub enum ExecPhase {
|
||||
/// 已受理(HTTP 已回 200,后台尚未开始)。
|
||||
Accepted,
|
||||
/// 校验 algorithmHash / 解析 nax 契约。
|
||||
Checking,
|
||||
/// 控制面:选址 → 起实例 → 拿端点 → 回报状态。
|
||||
ControlPlane,
|
||||
/// EasyTier 自动组网。
|
||||
Network,
|
||||
/// Ray head/worker(分布式计算层)。
|
||||
Ray,
|
||||
/// mpc 算子点火。
|
||||
Mpc,
|
||||
/// 全链完成。
|
||||
Succeeded,
|
||||
/// dry-run:只落执行计划,**不算成功**。
|
||||
Planned,
|
||||
/// 任一步失败(message 带真因)。
|
||||
Failed,
|
||||
}
|
||||
|
||||
impl ExecPhase {
|
||||
pub fn as_str(self) -> &'static str {
|
||||
match self {
|
||||
Self::Accepted => "ACCEPTED",
|
||||
Self::Checking => "CHECKING",
|
||||
Self::ControlPlane => "CONTROL_PLANE",
|
||||
Self::Network => "NETWORK",
|
||||
Self::Ray => "RAY",
|
||||
Self::Mpc => "MPC",
|
||||
Self::Succeeded => "SUCCEEDED",
|
||||
Self::Planned => "PLANNED",
|
||||
Self::Failed => "FAILED",
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct ExecStep {
|
||||
pub name: String,
|
||||
pub ok: bool,
|
||||
pub detail: String,
|
||||
pub at: String,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct ExecRecord {
|
||||
/// 幂等键的短哈希(安全文件名)。
|
||||
pub exec_id: String,
|
||||
pub task_id: String,
|
||||
pub namespace: String,
|
||||
pub vnet_name: String,
|
||||
pub phase: ExecPhase,
|
||||
pub ok: bool,
|
||||
pub message: String,
|
||||
pub steps: Vec<ExecStep>,
|
||||
/// 执行计划(dry-run 也会落,命令已脱敏)。
|
||||
pub plan: Option<Value>,
|
||||
pub created_at: String,
|
||||
pub updated_at: String,
|
||||
}
|
||||
|
||||
impl ExecRecord {
|
||||
pub fn new(exec_id: String, task_id: &str, namespace: &str, vnet_name: &str) -> Self {
|
||||
let now = now();
|
||||
Self {
|
||||
exec_id,
|
||||
task_id: task_id.to_string(),
|
||||
namespace: namespace.to_string(),
|
||||
vnet_name: vnet_name.to_string(),
|
||||
phase: ExecPhase::Accepted,
|
||||
ok: true,
|
||||
message: "已受理,等待后台执行".to_string(),
|
||||
steps: Vec::new(),
|
||||
plan: None,
|
||||
created_at: now.clone(),
|
||||
updated_at: now,
|
||||
}
|
||||
}
|
||||
|
||||
/// 推进阶段 + 追加一步轨迹。
|
||||
pub fn step(&mut self, phase: ExecPhase, name: &str, ok: bool, detail: impl Into<String>) {
|
||||
self.phase = phase;
|
||||
self.ok = ok;
|
||||
self.message = detail.into();
|
||||
self.steps.push(ExecStep {
|
||||
name: name.to_string(),
|
||||
ok,
|
||||
detail: self.message.clone(),
|
||||
at: now(),
|
||||
});
|
||||
self.updated_at = now();
|
||||
if !ok {
|
||||
self.phase = ExecPhase::Failed;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// 执行态存储。
|
||||
pub struct ExecStore {
|
||||
dir: PathBuf,
|
||||
inner: Mutex<BTreeMap<String, ExecRecord>>,
|
||||
}
|
||||
|
||||
impl ExecStore {
|
||||
pub fn new(dir: &Path) -> Self {
|
||||
let _ = fs::create_dir_all(dir);
|
||||
let mut map = BTreeMap::new();
|
||||
if let Ok(rd) = fs::read_dir(dir) {
|
||||
for entry in rd.flatten() {
|
||||
let p = entry.path();
|
||||
if p.extension().and_then(|e| e.to_str()) != Some("json") {
|
||||
continue;
|
||||
}
|
||||
if let Ok(text) = fs::read_to_string(&p) {
|
||||
if let Ok(rec) = serde_json::from_str::<ExecRecord>(&text) {
|
||||
map.insert(rec.exec_id.clone(), rec);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
Self {
|
||||
dir: dir.to_path_buf(),
|
||||
inner: Mutex::new(map),
|
||||
}
|
||||
}
|
||||
|
||||
/// 幂等键:`namespace` + `vnetName` + `taskId`(同一空间同一任务重复派发 ⇒ 复用同一记录)。
|
||||
pub fn key_of(namespace: &str, vnet_name: &str, task_id: &str) -> String {
|
||||
short_key(&format!("{}|{}|{}", namespace, vnet_name, task_id))
|
||||
}
|
||||
|
||||
pub fn get(&self, exec_id: &str) -> Option<ExecRecord> {
|
||||
self.inner.lock().ok()?.get(exec_id).cloned()
|
||||
}
|
||||
|
||||
pub fn list(&self) -> Vec<ExecRecord> {
|
||||
self.inner
|
||||
.lock()
|
||||
.map(|g| g.values().cloned().collect())
|
||||
.unwrap_or_default()
|
||||
}
|
||||
|
||||
/// 在同一把锁内改记录并落盘(**不跨 await**)。
|
||||
pub fn mutate<F>(&self, exec_id: &str, f: F) -> Option<ExecRecord>
|
||||
where
|
||||
F: FnOnce(&mut ExecRecord),
|
||||
{
|
||||
let mut guard = self.inner.lock().ok()?;
|
||||
let rec = guard.get_mut(exec_id)?;
|
||||
f(rec);
|
||||
let snapshot = rec.clone();
|
||||
self.persist(&snapshot);
|
||||
Some(snapshot)
|
||||
}
|
||||
|
||||
pub fn insert(&self, rec: ExecRecord) -> ExecRecord {
|
||||
self.persist(&rec);
|
||||
if let Ok(mut guard) = self.inner.lock() {
|
||||
guard.insert(rec.exec_id.clone(), rec.clone());
|
||||
}
|
||||
rec
|
||||
}
|
||||
|
||||
fn persist(&self, rec: &ExecRecord) {
|
||||
let path = self.dir.join(format!("{}.json", rec.exec_id));
|
||||
// 文件名来自 short_key(16 位 hex)⇒ 无路径穿越。
|
||||
let Ok(text) = serde_json::to_string_pretty(rec) else {
|
||||
return;
|
||||
};
|
||||
let tmp = self.dir.join(format!("{}.json.tmp", rec.exec_id));
|
||||
if fs::write(&tmp, text).is_ok() {
|
||||
let _ = fs::rename(tmp, path);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn now() -> String {
|
||||
chrono::Utc::now().to_rfc3339()
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn a_failed_step_marks_the_record_failed() {
|
||||
let mut r = ExecRecord::new("abc".into(), "t1", "ns", "vnet");
|
||||
r.step(ExecPhase::Checking, "校验产物", true, "ok");
|
||||
r.step(ExecPhase::Network, "起 ET", false, "缺二进制");
|
||||
assert_eq!(r.phase, ExecPhase::Failed);
|
||||
assert!(!r.ok);
|
||||
assert_eq!(r.steps.len(), 2);
|
||||
assert_eq!(r.message, "缺二进制");
|
||||
assert_eq!(ExecPhase::Succeeded.as_str(), "SUCCEEDED");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn store_roundtrips_and_recovers_from_disk() {
|
||||
let dir = std::env::temp_dir().join(format!("tdc-exec-store-{}", std::process::id()));
|
||||
let _ = fs::remove_dir_all(&dir);
|
||||
{
|
||||
let s = ExecStore::new(&dir);
|
||||
let mut r = ExecRecord::new("deadbeef".into(), "t1", "ns", "vnet");
|
||||
r.step(ExecPhase::Checking, "check", true, "ok");
|
||||
s.insert(r);
|
||||
}
|
||||
let s = ExecStore::new(&dir);
|
||||
let r = s.get("deadbeef").expect("重启后应能恢复");
|
||||
assert_eq!(r.task_id, "t1");
|
||||
assert_eq!(r.steps.len(), 1);
|
||||
let _ = fs::remove_dir_all(&dir);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn idempotency_key_is_stable_and_scoped() {
|
||||
let a = ExecStore::key_of("ns", "vnet", "t1");
|
||||
assert_eq!(a, ExecStore::key_of("ns", "vnet", "t1"));
|
||||
assert_ne!(a, ExecStore::key_of("ns", "vnet", "t2"));
|
||||
assert_eq!(a.len(), 16);
|
||||
}
|
||||
}
|
||||
@@ -31,6 +31,7 @@ use axum::extract::FromRef;
|
||||
use crate::platform::state::AppState;
|
||||
use crate::tdc::client::tds::TdsClient;
|
||||
use crate::tdc::config::TdcConfig;
|
||||
use crate::tdc::engine::ExecRuntime;
|
||||
|
||||
/// 首次启动引导出来的本地身份。
|
||||
///
|
||||
@@ -64,6 +65,12 @@ pub struct TdcState {
|
||||
/// 用 tdc 签发的 token 也能访问 `/api/oauth/CurrentUser`。
|
||||
/// `None` ⇒ 没接平台包(集成测试 / 独立运行),会话桥静默跳过。
|
||||
pub platform_sessions: Option<Arc<sspf::SessionStore>>,
|
||||
/// 执行层运行态(三入向端点 + 后台状态机共用;`MIGRATION.md` §1.7 的
|
||||
/// 「不实现交付执行」已被 2026-11-13 用户口径推翻,见 `docs/` 决策文档)。
|
||||
///
|
||||
/// 🔴 用 `Arc` ⇒ `tokio::spawn` 的后台任务与入向 handler 共享**同一份**执行状态,
|
||||
/// 不会各自建出第二张状态表 / 第二把 pid 幂等锁。
|
||||
pub exec: Arc<ExecRuntime>,
|
||||
}
|
||||
|
||||
impl TdcState {
|
||||
@@ -77,12 +84,15 @@ impl TdcState {
|
||||
platform.config.tds_connect_timeout,
|
||||
platform.config.tds_read_timeout,
|
||||
));
|
||||
// 🔴 必须在 `platform` 被 move 进 Self 之前取到数据目录(执行状态 / pid / 0600 env 落这里)。
|
||||
let exec = Arc::new(ExecRuntime::new(&platform.config.data_dir));
|
||||
Self {
|
||||
platform,
|
||||
tdc_config: Arc::new(tdc_config),
|
||||
bootstrap: Arc::new(bootstrap),
|
||||
tds,
|
||||
platform_sessions: None,
|
||||
exec,
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -40,6 +40,15 @@ pub const PUBLIC_PATHS: &[&str] = &[
|
||||
"/api/oauth/Login",
|
||||
"/api/ssw/tdc/Auth/Login",
|
||||
"/api/ssw/tdc/Auth/SsoConfig",
|
||||
// ---- 平台执行下发(2026-11-13,tdc-rs 执行链路)----
|
||||
// 🔴 ② 的 ⑦ 是**裸 curl 存活探测**(不带凭证):不放行的话端点只会从 404 变 401,
|
||||
// 无法证明「端点已存在」。`controller/exec.rs::guard` 在 handler 内做与中间件
|
||||
// 同口径的「内部令牌 + 协议版本」复核,仅 id=PROBE 的零副作用探测免凭证。
|
||||
// ⇒ 白名单放行 ≠ 免鉴权,真实下发仍 fail-closed。
|
||||
"/api/ssw/tdc/DeliveryTask/*/Internal/SelfCheckStatus",
|
||||
"/api/ssw/tdc/DeliveryTask/*/Dev/Artifact",
|
||||
"/api/ssw/tdc/DeliveryTask/*/Exec/Start",
|
||||
"/api/ssw/tdc/DeliveryTask/*/Exec/Status",
|
||||
];
|
||||
|
||||
/// 免鉴权的**平台端点**(健康检查)。
|
||||
@@ -60,8 +69,9 @@ pub fn all_public_paths() -> Vec<&'static str> {
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
/// 🔴 清单**字面钉住**:上游 `gateway.white-url` 去掉本仓无路由的死路径后,
|
||||
/// 只剩这 4 条(3 条业务端点 + 1 条平台端点)。
|
||||
/// 🔴 清单**字面钉住**:上游 `gateway.white-url` 去掉本仓无路由的死路径后剩 3 条,
|
||||
/// 加 1 条平台健康检查、4 条本批新增的执行端点 ⇒ 共 8 条。
|
||||
/// 新增的 4 条**不是免鉴权**:`controller/exec.rs::guard` 在 handler 内复核内部令牌。
|
||||
#[test]
|
||||
fn the_manifest_holds_exactly_the_live_endpoints() {
|
||||
assert_eq!(
|
||||
@@ -70,6 +80,10 @@ mod tests {
|
||||
"/api/oauth/Login",
|
||||
"/api/ssw/tdc/Auth/Login",
|
||||
"/api/ssw/tdc/Auth/SsoConfig",
|
||||
"/api/ssw/tdc/DeliveryTask/*/Internal/SelfCheckStatus",
|
||||
"/api/ssw/tdc/DeliveryTask/*/Dev/Artifact",
|
||||
"/api/ssw/tdc/DeliveryTask/*/Exec/Start",
|
||||
"/api/ssw/tdc/DeliveryTask/*/Exec/Status",
|
||||
"/api/health",
|
||||
]
|
||||
);
|
||||
|
||||
@@ -0,0 +1,311 @@
|
||||
//! 平台下发三端点的契约回归(对位 `build/fake-tdc-receiver.py` 与 ② 的 ⑦ 探测)。
|
||||
//!
|
||||
//! 🔴 验收锚点:`verify-contract-driven-tunnel.sh:410-413` 的裸 curl
|
||||
//! `POST .../DeliveryTask/PROBE/Exec/Start` 必须从 **404 → 200**(端点存在)。
|
||||
//!
|
||||
//! ⚠️ 每个测试用独立临时目录构造 `ExecRuntime`(直接替换 `state.exec`),
|
||||
//! **不设进程级环境变量** ⇒ 并行测试无 env 竞态。
|
||||
|
||||
use std::collections::BTreeMap;
|
||||
use std::fs;
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::sync::Arc;
|
||||
|
||||
use axum::body::Body;
|
||||
use axum::http::{Request, StatusCode};
|
||||
use axum::Router;
|
||||
use http_body_util::BodyExt;
|
||||
use pglite::PGlite;
|
||||
use serde_json::{json, Value};
|
||||
use tdc_connector::platform::auth::{auth_middleware, AuthState};
|
||||
use tdc_connector::platform::config::Config;
|
||||
use tdc_connector::platform::session::SessionStore;
|
||||
use tdc_connector::platform::state::AppState;
|
||||
use tdc_connector::tdc::client::mpc_control_plane::MpcControlPlaneClient;
|
||||
use tdc_connector::tdc::config::TdcConfig;
|
||||
use tdc_connector::tdc::controller::build_router;
|
||||
use tdc_connector::tdc::engine::config::{self, ExecConfig};
|
||||
use tdc_connector::tdc::engine::{ExecRuntime, ExecStore};
|
||||
use tdc_connector::tdc::state::{BootstrapIdentity, TdcState};
|
||||
use tdc_connector::tdc::support::public_paths;
|
||||
use tokio::sync::OnceCell;
|
||||
use tower::ServiceExt;
|
||||
|
||||
static DB: OnceCell<Arc<PGlite>> = OnceCell::const_new();
|
||||
|
||||
async fn shared_db() -> Arc<PGlite> {
|
||||
DB.get_or_init(|| async {
|
||||
let db = Arc::new(PGlite::open_temp().await.expect("打开临时 PGlite 失败"));
|
||||
tdc_connector::platform::db::ensure_schema(db.as_ref())
|
||||
.await
|
||||
.expect("建表失败");
|
||||
db
|
||||
})
|
||||
.await
|
||||
.clone()
|
||||
}
|
||||
|
||||
const ARTIFACT: &[u8] = b"import sys\nprint('fake algorithm for contract-driven tunnel test')\n";
|
||||
|
||||
/// 每个用例一个目录(含进程 id + 用例名),避免并行互踩。
|
||||
fn case_dir(name: &str) -> PathBuf {
|
||||
let d = std::env::temp_dir().join(format!("tdc-exec-it-{}-{}", std::process::id(), name));
|
||||
let _ = fs::remove_dir_all(&d);
|
||||
fs::create_dir_all(d.join("artifacts")).unwrap();
|
||||
fs::write(d.join("artifacts/algorithm.py"), ARTIFACT).unwrap();
|
||||
d
|
||||
}
|
||||
|
||||
fn exec_config(dir: &Path) -> ExecConfig {
|
||||
let env: BTreeMap<String, String> = [
|
||||
(config::ENV_ARTIFACT_DIR, dir.join("artifacts").to_string_lossy().to_string()),
|
||||
(config::ENV_STATE_DIR, dir.join("state").to_string_lossy().to_string()),
|
||||
(config::ENV_LOCAL_VIP, "10.12.0.2".to_string()),
|
||||
(config::ENV_SITE_ENDPOINTS, "alice=10.12.0.1:9449,carol=10.12.0.2:9449".to_string()),
|
||||
(config::ENV_DRY_RUN, "1".to_string()),
|
||||
]
|
||||
.into_iter()
|
||||
.map(|(k, v)| (k.to_string(), v))
|
||||
.collect();
|
||||
ExecConfig::from_raw(&env, dir)
|
||||
}
|
||||
|
||||
async fn build(name: &str) -> (Router, String, Arc<ExecRuntime>) {
|
||||
let dir = case_dir(name);
|
||||
let db = shared_db().await;
|
||||
let config = Config::from_env(std::env::current_dir().unwrap());
|
||||
let sessions = Arc::new(SessionStore::new());
|
||||
let token = config.internal_token.clone();
|
||||
let mut state = TdcState::new(
|
||||
AppState::new(config.clone(), db, sessions.clone()),
|
||||
TdcConfig::default(),
|
||||
BootstrapIdentity {
|
||||
identity_id: "TEST-OP".to_string(),
|
||||
identity_code: "TEST-CONNECTOR".to_string(),
|
||||
issued_secret: None,
|
||||
},
|
||||
);
|
||||
let cfg = exec_config(&dir);
|
||||
let exec = Arc::new(ExecRuntime {
|
||||
store: ExecStore::new(&cfg.state_dir),
|
||||
config: cfg,
|
||||
control_plane: MpcControlPlaneClient::new(None, String::new()),
|
||||
});
|
||||
state.exec = exec.clone();
|
||||
let auth_state = AuthState {
|
||||
internal_token: token.clone(),
|
||||
sessions,
|
||||
public_paths: Arc::new(public_paths::all_public_paths()),
|
||||
};
|
||||
let app: Router = build_router(state.clone())
|
||||
.layer(axum::middleware::from_fn_with_state(
|
||||
auth_state,
|
||||
auth_middleware,
|
||||
))
|
||||
.with_state(state);
|
||||
(app, token, exec)
|
||||
}
|
||||
|
||||
fn req(method: &str, uri: &str, body: &str, token: Option<&str>) -> Request<Body> {
|
||||
let mut b = Request::builder()
|
||||
.method(method)
|
||||
.uri(uri)
|
||||
.header("Content-Type", "application/json");
|
||||
if let Some(t) = token {
|
||||
b = b
|
||||
.header("X-Internal-Token", t)
|
||||
.header("X-Protocol-Version", "3.2")
|
||||
.header("X-Internal-Operator", "it");
|
||||
}
|
||||
b.body(Body::from(body.to_string())).unwrap()
|
||||
}
|
||||
|
||||
async fn call(app: Router, r: Request<Body>) -> (StatusCode, Value) {
|
||||
let resp = app.oneshot(r).await.unwrap();
|
||||
let status = resp.status();
|
||||
let bytes = resp.into_body().collect().await.unwrap().to_bytes();
|
||||
let v = serde_json::from_slice(&bytes).unwrap_or(Value::Null);
|
||||
(status, v)
|
||||
}
|
||||
|
||||
/// ⑦ 验收:**裸 curl 存活探测**从 404 → 200,且零副作用。
|
||||
#[tokio::test]
|
||||
async fn probe_is_200_without_any_credentials_and_has_no_side_effects() {
|
||||
let (app, _token, exec) = build("probe").await;
|
||||
let (status, v) = call(
|
||||
app,
|
||||
req(
|
||||
"POST",
|
||||
"/api/ssw/tdc/DeliveryTask/PROBE/Exec/Start",
|
||||
"",
|
||||
None,
|
||||
),
|
||||
)
|
||||
.await;
|
||||
assert_eq!(status, StatusCode::OK);
|
||||
assert_eq!(v["code"], 200);
|
||||
assert_eq!(v["data"]["ok"], true);
|
||||
assert_eq!(v["data"]["probe"], true);
|
||||
// 零副作用:不落任何执行记录
|
||||
assert!(exec.store.list().is_empty(), "PROBE 不得产生状态记录");
|
||||
}
|
||||
|
||||
/// 真实 taskId **仍需内部令牌**:白名单放行 ≠ 免鉴权。
|
||||
#[tokio::test]
|
||||
async fn real_dispatch_without_token_is_401() {
|
||||
let (app, _token, _exec) = build("unauth").await;
|
||||
let (status, v) = call(
|
||||
app,
|
||||
req(
|
||||
"POST",
|
||||
"/api/ssw/tdc/DeliveryTask/T1/Exec/Start",
|
||||
"{}",
|
||||
None,
|
||||
),
|
||||
)
|
||||
.await;
|
||||
assert_eq!(status, StatusCode::UNAUTHORIZED);
|
||||
assert_eq!(v["code"], 401);
|
||||
assert_eq!(v["success"], false);
|
||||
}
|
||||
|
||||
/// `SelfCheckStatus`:`readyStatus=READY` + 能力清单如实列出。
|
||||
#[tokio::test]
|
||||
async fn self_check_is_ready_and_reports_capabilities() {
|
||||
let (app, token, _exec) = build("selfcheck").await;
|
||||
let (status, v) = call(
|
||||
app,
|
||||
req(
|
||||
"POST",
|
||||
"/api/ssw/tdc/DeliveryTask/T1/Internal/SelfCheckStatus",
|
||||
"{}",
|
||||
Some(&token),
|
||||
),
|
||||
)
|
||||
.await;
|
||||
assert_eq!(status, StatusCode::OK);
|
||||
assert_eq!(v["code"], 200);
|
||||
assert_eq!(v["data"]["readyStatus"], "READY");
|
||||
assert_eq!(v["data"]["ok"], true);
|
||||
assert_eq!(v["data"]["capabilities"]["engineReady"], true);
|
||||
assert_eq!(v["data"]["capabilities"]["ray"]["enabled"], false);
|
||||
assert_eq!(v["data"]["capabilities"]["controlPlane"]["configured"], false);
|
||||
}
|
||||
|
||||
/// `Dev/Artifact`:`files` 是 **List**、`algorithmHash` 与 Python `dirDigest` 实算值一致。
|
||||
#[tokio::test]
|
||||
async fn artifact_is_a_list_with_self_computed_hash() {
|
||||
let (app, token, _exec) = build("artifact").await;
|
||||
let (status, v) = call(
|
||||
app,
|
||||
req(
|
||||
"GET",
|
||||
"/api/ssw/tdc/DeliveryTask/T1/Dev/Artifact?versionNo=1",
|
||||
"",
|
||||
Some(&token),
|
||||
),
|
||||
)
|
||||
.await;
|
||||
assert_eq!(status, StatusCode::OK);
|
||||
assert_eq!(v["code"], 200);
|
||||
let files = v["data"]["files"].as_array().expect("files 必须是 List");
|
||||
assert_eq!(files.len(), 1);
|
||||
assert_eq!(files[0]["name"], "algorithm.py");
|
||||
assert_eq!(files[0]["contentBase64"], "aW1wb3J0IHN5cwpwcmludCgnZmFrZSBhbGdvcml0aG0gZm9yIGNvbnRyYWN0LWRyaXZlbiB0dW5uZWwgdGVzdCcpCg==");
|
||||
// sha256( "algorithm.py\t" + sha256(ARTIFACT) )
|
||||
assert_eq!(
|
||||
v["data"]["algorithmHash"].as_str().unwrap().len(),
|
||||
64,
|
||||
"必须是 sha256 hex"
|
||||
);
|
||||
assert_eq!(v["data"]["entry"], "algorithm.py");
|
||||
}
|
||||
|
||||
/// `Exec/Start`:立即回执 + 后台状态机推进;`dry_run` 收口为 `PLANNED`(**不是 SUCCEEDED**)。
|
||||
#[tokio::test]
|
||||
async fn exec_start_accepts_and_background_state_machine_reaches_planned() {
|
||||
let (app, token, exec) = build("dispatch").await;
|
||||
let payload = json!({
|
||||
"algorithmHash": "ignored-when-local-artifacts-absent",
|
||||
"clusterConfig": {
|
||||
"nax": {
|
||||
"vnetName": "ssw-PORTAL-DEMO-SPACE",
|
||||
"vnetSecret": "ssw-IT-TEST-SECRET",
|
||||
"namespace": "nax-991794557079009427-v1",
|
||||
"rayHead": "10.12.0.2:30417",
|
||||
"rayHeadName": "nax-head-991794557079009427-v1",
|
||||
"parties": ["p1", "p2"],
|
||||
"consumerParty": "p1"
|
||||
},
|
||||
"parties": {
|
||||
"p1": {"address": "sf-99179455-p1.default.svc.cluster.local:20001"},
|
||||
"p2": {"address": "sf-99179455-p2.default.svc.cluster.local:20001"}
|
||||
}
|
||||
},
|
||||
"selfParty": "p1",
|
||||
"versionNo": 1
|
||||
});
|
||||
// 🔴 平台 payload 带 algorithmHash 时,本地产物 hash 必须一致;这里改成真实值。
|
||||
let mut payload = payload;
|
||||
let hash = {
|
||||
let mut files = BTreeMap::new();
|
||||
files.insert("algorithm.py".to_string(), ARTIFACT.to_vec());
|
||||
tdc_connector::tdc::engine::digest::dir_digest(&files)
|
||||
};
|
||||
payload["algorithmHash"] = json!(hash);
|
||||
|
||||
let (status, v) = call(
|
||||
app,
|
||||
req(
|
||||
"POST",
|
||||
"/api/ssw/tdc/DeliveryTask/T1/Exec/Start",
|
||||
&payload.to_string(),
|
||||
Some(&token),
|
||||
),
|
||||
)
|
||||
.await;
|
||||
assert_eq!(status, StatusCode::OK);
|
||||
assert_eq!(v["code"], 200);
|
||||
assert_eq!(v["data"]["ok"], true);
|
||||
assert_eq!(v["data"]["accepted"], true);
|
||||
let exec_id = v["data"]["execId"].as_str().unwrap().to_string();
|
||||
|
||||
// 等后台状态机跑完(纯本地、无网络调用)。
|
||||
let mut rec = None;
|
||||
for _ in 0..40 {
|
||||
if let Some(r) = exec.store.get(&exec_id) {
|
||||
if r.phase == tdc_connector::tdc::engine::ExecPhase::Planned
|
||||
|| r.phase == tdc_connector::tdc::engine::ExecPhase::Failed
|
||||
{
|
||||
rec = Some(r);
|
||||
break;
|
||||
}
|
||||
}
|
||||
tokio::time::sleep(std::time::Duration::from_millis(25)).await;
|
||||
}
|
||||
let rec = rec.expect("后台状态机未在 1s 内收口");
|
||||
assert_eq!(
|
||||
rec.phase,
|
||||
tdc_connector::tdc::engine::ExecPhase::Planned,
|
||||
"dry-run 必须是 PLANNED;实际 message={}",
|
||||
rec.message
|
||||
);
|
||||
assert_eq!(rec.steps.len(), 6, "六个阶段各留一步轨迹");
|
||||
// 计划脱敏:不得出现 vnetSecret 明文
|
||||
let plan = serde_json::to_string(&rec.plan).unwrap();
|
||||
assert!(!plan.contains("ssw-IT-TEST-SECRET"), "计划泄漏 vnetSecret");
|
||||
assert!(plan.contains("k8sServiceDnsUnusable"));
|
||||
}
|
||||
|
||||
/// 重复派发(同 namespace / vnet / taskId)复用同一记录,不重复起实例。
|
||||
#[tokio::test]
|
||||
async fn duplicate_dispatch_is_idempotent() {
|
||||
let (app, token, exec) = build("dup").await;
|
||||
let body = json!({"clusterConfig":{"nax":{"namespace":"ns","vnetName":"vn"}}}).to_string();
|
||||
let uri = "/api/ssw/tdc/DeliveryTask/T1/Exec/Start";
|
||||
let (_, a) = call(app.clone(), req("POST", uri, &body, Some(&token))).await;
|
||||
let (_, b) = call(app, req("POST", uri, &body, Some(&token))).await;
|
||||
assert_eq!(a["data"]["execId"], b["data"]["execId"]);
|
||||
assert_eq!(b["data"]["duplicate"], true);
|
||||
assert_eq!(exec.store.list().len(), 1);
|
||||
}
|
||||
Reference in new issue
Block a user