test: apix 纳入上线后验证(tests/apix_check.py + live_check.sh 三步)
apix 发版后最容易静默失效:location 没挂 / shm 没装载都不报错,只是 404,
而回归打假上游、agent.sh verify 又不发真实请求 —— 之前完全没有覆盖。
新增 tests/apix_check.py:
- 从 /api/v1/me(mdm host + X-API-Key 头)拉能力目录,逐个探 enabled 条目;
- vendor → GET /apix/<slug>?<p>=<占位>;passthrough → GET /apixp/<slug>/health;
- **无效 Key 预检**:证明 location 挂着 + 鉴权生效(与上游是否抽风无关);
- 判定按错误体归属:网关体({"error":{...}} / {"code":..,"msg":..})→ 失败;
上游体({"detail":...} / WAF 页)→ 只告警,不染红发版验证。
实测踩到的三个坑(已写进 README):
1. urllib 默认 UA `Python-urllib/3.x` 被供应商边缘 Cloudflare 拦(403 error code:
1010),看着像 IP 被封,换 curl UA 立刻 200;
2. passthrough tail 是 /apixp/<slug> 之后的全部(上游 /health → /apixp/ocr/health),
usage_hint 里的 /<tail> 要换成 slug,照抄是上游 404;
3. vendor 参数要放白名单值:accel 给 example.com 是 400 target host not allowed
(链路通,属告警),换 raw.githubusercontent 小文件即 200。
live_check.sh 改为三步(LLM 真实流量 / apix 探针 / 环境验收),加 --apix-only。
线上实测:LLM 8/8 + apix 3/3 + 验收全绿。
This commit is contained in:
1 parent
81f83f6f50
commit
cde14e1f1b
3 files changed
+344
-39
No files matched your search
@@ -26,6 +26,16 @@ Key 文件可以是裸 key,也可以是 `sk=<key>` 形式(带 `#` 注释行
|
||||
`sk=sk_xxx` → 全量 401**,脚本内部已处理;全程不打印 Key 原文。
|
||||
延迟参考:deepseek/qwen/minimax 1~2.6s,**glm 7~11s**,别拿 glm 当超时基线。
|
||||
|
||||
三个 apix 探针的固定坑(写脚本时踩过,改条目时留意):
|
||||
|
||||
1. **必须伪装 UA**:供应商边缘(souyi 走 Cloudflare)会拦 `Python-urllib/3.x` →
|
||||
`403 error code: 1010`;换 curl UA 立刻 200。看着像 IP 被封,其实是 UA。
|
||||
2. **passthrough 的 tail 是 `/apixp/<slug>` 之后的全部**:上游 `/health` 对应
|
||||
`/apixp/ocr/health`;`usage_hint` 里的 `/<tail>` 要换成 slug(`/ocr/base64` →
|
||||
`/apixp/ocr/ocr/base64`),照抄会得到上游 404。
|
||||
3. **vendor 的参数只放白名单值**:accel 只放行 GitHub/Gitee/GitLab,给
|
||||
`example.com` 会 `400 target host not allowed`(链路其实是通的,属 ⚠️)。
|
||||
|
||||
## 目录
|
||||
|
||||
| 路径 | 作用 |
|
||||
@@ -33,6 +43,7 @@ Key 文件可以是裸 key,也可以是 `sk=<key>` 形式(带 `#` 注释行
|
||||
| `run.sh` | 统一入口:编译 → 渲染配置 → 起进程 → 跑断言 → 收尾 |
|
||||
| `live_check.sh` | **上线后验证**:真实流量(`test_models.py`)+ 正式环境验收(`deploy/agent.sh verify`) |
|
||||
| `test_models.py` | 真实流量探针:4 slot × OpenAI/Anthropic 调用 + `/v1/models` 模型发现(打的是线上 `lm.mayishangshu.cn:88`,**不是本机夹具**;失败退出码 1) |
|
||||
| `apix_check.py` | apix 探针:`/apix/<slug>`(vendor) + `/apixp/<slug>`(passthrough) 是否真转发。**只有「网关侧没生效」才算失败**(无效 Key 预检 + 错误体归属判定),上游业务拒绝/供应商 WAF 只告警 |
|
||||
| `gateway/nginx.conf.in` | 域名网关测试配置**模板**(占位符 `@MODULE@` / `@REPO@` / `@RUN@`) |
|
||||
| `gateway/e2e.sh` | **108 条断言**:鉴权会话 / 账号 / 申请审核 / 免审 / 越权隔离 / Key 重置 / **自动子域名(`i`+8位+`dm`)与点击复制** / 管理台与清理 |
|
||||
| `gateway/backend/index.html` | 转发靶机内容(断言里检查 `"from":"backend"`) |
|
||||
|
||||
@@ -0,0 +1,280 @@
|
||||
#!/usr/bin/env python3
|
||||
"""apix 上线后探针 —— 验证 /apix/<slug>(vendor)与 /apixp/<slug>(passthrough)真能转发。
|
||||
|
||||
为什么要单独一个脚本:LLM 回归(tests/run.sh)打的是**假上游**,`agent.sh verify`
|
||||
只验管理台,两边都不碰 apix 链路。发版后 apix 配不生效(location 没挂 / shm 没装载)
|
||||
在这里才会暴露。
|
||||
|
||||
判定的关键在 **404 是谁给的**(实测):
|
||||
* 网关风格 `{"error":{"code":...}}` → ❌ 失败(location 没挂或条目停用,正是要抓的)
|
||||
* 上游风格 `{"detail":...}` → ⚠️ 告警(链路通,只是路径/参数不对)
|
||||
同理 4xx 业务拒绝(如 accel 的 `target host not allowed`)算 ⚠️ 不算失败 ——
|
||||
那说明鉴权、拼体、转发全通了,只是探针参数不合上游业务规则。
|
||||
|
||||
已知现象(2026-09-17 实测):**连续**探测 vendor 条目(accel / search)会触发
|
||||
souyi 边缘 Cloudflare 的 1010 拦截,返回 `error code: 1010`;间隔约 60s 后自动恢复
|
||||
200。所以连续跑出 ⚠️ 属预期,不是发版问题 —— 看「鉴权✓」就知道网关侧是好的。
|
||||
|
||||
探针怎么构造:
|
||||
vendor → `GET /apix/<slug>?<p>=<占位>`,参数名取 `params[0]`,没有就从
|
||||
`desc` / `usage_hint` 里正则找第一个 `?xx=`(如 accel 的 `?url=`)。
|
||||
passthrough → `GET /apixp/<slug>/health`(轻量、不触发 GPU 懒加载等业务开销)。
|
||||
注意 tail 是 `/apixp/<slug>` **之后**的全部,所以上游 `/health`
|
||||
对应 `/apixp/ocr/health`,而 usage_hint 里的 `/<tail>` 要换成 slug
|
||||
(`/ocr/base64` → `/apixp/ocr/ocr/base64`),别照抄。
|
||||
|
||||
用法:
|
||||
python3 tests/apix_check.py <key> # 探针全部 enabled 条目
|
||||
python3 tests/apix_check.py <key> --slug ocr # 只探某几个(逗号分隔)
|
||||
python3 tests/apix_check.py <key> --timeout 40
|
||||
IPASS_MDM_HOST=... IPASS_LM_HOST=... python3 tests/apix_check.py <key>
|
||||
|
||||
退出码:有 ❌ 失败 → 1;只有 ✅/⚠️ → 0。
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.parse
|
||||
import urllib.request
|
||||
|
||||
MDM = os.environ.get("IPASS_MDM_HOST", "https://mdm.mayishangshu.cn:88")
|
||||
LM = os.environ.get("IPASS_LM_HOST", "https://lm.mayishangshu.cn:88")
|
||||
|
||||
# 参数占位值:让请求「格式合法」即可,不追求业务成功
|
||||
PLACEHOLDER = {
|
||||
"q": "ping",
|
||||
"query": "ping",
|
||||
"keyword": "ping",
|
||||
# accel 这类加速服务只放行白名单 host(GitHub/Gitee/GitLab):
|
||||
# 用最小的 raw 文件(~485B),既能拿到 200 又不吃掉额度
|
||||
"url": "https://raw.githubusercontent.com/octocat/Hello-World/master/README",
|
||||
"text": "ping",
|
||||
}
|
||||
|
||||
PASS, WARN, FAIL = "pass", "warn", "fail"
|
||||
|
||||
# 见 http() 里的说明:供应商边缘会拦 `Python-urllib/3.x`,必须伪装成常见客户端
|
||||
DEFAULT_UA = "curl/8.4.0"
|
||||
|
||||
# 无效 Key 预检用:形态合法(sk_ 开头)但查不到,网关应返 401
|
||||
BAD_KEY = "sk_" + "0" * 48
|
||||
|
||||
# ⚠️ 重试前的等待(兜底用;注意 1010 的真实诱因是 UA 不是频率,见 DEFAULT_UA 注释)
|
||||
RETRY_WAIT_SEC = 15
|
||||
# 只有「疑似限流/网络抖动」值得重试;404(路径不存在)重试没意义
|
||||
RETRYABLE = (403, 429, 0)
|
||||
|
||||
|
||||
def http(method: str, url: str, key: str, timeout: int, body: bytes | None = None,
|
||||
ctype: str | None = None, auth: str = "Bearer"):
|
||||
"""发请求,返回 (code, body_text, elapsed)。网络层异常返回 (0, err, elapsed)。
|
||||
|
||||
`auth` 决定鉴权头形态:网关统一收 `Authorization: Bearer <sk_>`,
|
||||
但 **`/api/v1/me` 只认 `X-API-Key`**(2026-09-17 实测:带 Bearer 会 401)。
|
||||
"""
|
||||
if auth == "X-API-Key":
|
||||
headers = {"X-API-Key": key}
|
||||
else:
|
||||
headers = {"Authorization": f"Bearer {key}"}
|
||||
# ⚠️ 必须带一个「正常」UA:供应商边缘(souyi 走 Cloudflare)会拦
|
||||
# `Python-urllib/3.x` → 返回 403 `error code: 1010`(2026-09-17 实测,
|
||||
# 同一请求换 curl UA 立刻 200)。可被 IPASS_UA 覆盖。
|
||||
headers["User-Agent"] = os.environ.get("IPASS_UA", DEFAULT_UA)
|
||||
if ctype:
|
||||
headers["Content-Type"] = ctype
|
||||
req = urllib.request.Request(url, data=body, headers=headers, method=method)
|
||||
start = time.time()
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=timeout) as resp:
|
||||
return resp.status, resp.read().decode("utf-8", "replace"), time.time() - start
|
||||
except urllib.error.HTTPError as e:
|
||||
raw = e.read().decode("utf-8", "replace") if e.fp else ""
|
||||
return e.code, raw, time.time() - start
|
||||
except Exception as e: # 超时 / DNS / TLS
|
||||
return 0, str(e), time.time() - start
|
||||
|
||||
|
||||
def fetch_catalog(key: str, timeout: int) -> list | None:
|
||||
"""拉 /api/v1/me 的能力目录(mdm host + X-API-Key 头)。失败返回 None。"""
|
||||
code, raw, _ = http("GET", f"{MDM}/api/v1/me", key, timeout, auth="X-API-Key")
|
||||
if code != 200:
|
||||
print(f"✗ 拉能力目录失败:HTTP {code} {raw[:200]}")
|
||||
return None
|
||||
try:
|
||||
return json.loads(raw).get("data", {}).get("apix") or []
|
||||
except json.JSONDecodeError:
|
||||
print(f"✗ 能力目录不是 JSON:{raw[:200]}")
|
||||
return None
|
||||
|
||||
|
||||
def is_gateway_error(raw: str) -> bool:
|
||||
"""是网关自己返的错误体,还是转发之后上游返的(实测两种网关形态):
|
||||
|
||||
* apix / 管理接口:`{"code":401,"msg":"API Key 无效或已失效"}`(扁平)
|
||||
* LLM 侧:`{"error":{"code":"invalid_api_key",...}}`(嵌套)
|
||||
|
||||
上游与边缘都不是这两种:FastAPI 是 `{"detail":...}`,Cloudflare 拦截页是
|
||||
HTML/纯文本(`error code: 1010`),所以能可靠区分「网关没生效」与「上游拒单」。
|
||||
"""
|
||||
try:
|
||||
d = json.loads(raw)
|
||||
except json.JSONDecodeError:
|
||||
return False
|
||||
if not isinstance(d, dict):
|
||||
return False
|
||||
if isinstance(d.get("error"), dict):
|
||||
return True
|
||||
return "code" in d and "msg" in d
|
||||
|
||||
|
||||
def guess_param(item: dict) -> str | None:
|
||||
"""vendor 条目要带哪个 query 参数名。"""
|
||||
params = item.get("params") or []
|
||||
if params:
|
||||
return str(params[0])
|
||||
hint = f"{item.get('desc') or ''} {item.get('usage_hint') or ''}"
|
||||
m = re.search(r"[?&](\w+)=", hint)
|
||||
return m.group(1) if m else None
|
||||
|
||||
|
||||
def classify(code: int, raw: str) -> tuple[str, str]:
|
||||
"""把响应分成 pass / warn / fail。
|
||||
|
||||
核心原则:**「网关侧没生效」才是发版要抓的失败**,上游业务拒绝(含供应商 WAF)
|
||||
只能算告警 —— 否则供应商一次抽风就把发版验证染红(实测:souyi 边缘
|
||||
Cloudflare 1010 拦 IP,同一请求 20 分钟前还是 200)。
|
||||
"""
|
||||
if code == 0:
|
||||
return FAIL, "网络错误/超时"
|
||||
if 200 <= code < 300:
|
||||
return PASS, ""
|
||||
if code in (401, 403):
|
||||
if is_gateway_error(raw):
|
||||
return FAIL, "网关拒绝:Key 无效 / 不在白名单(location 是活的)"
|
||||
# 非网关体 = 转发之后被上游或边缘(WAF / Cloudflare)拒的
|
||||
return WARN, f"上游/边缘拒绝 HTTP {code}(非网关体,多为 WAF,需人工确认)"
|
||||
if code == 404:
|
||||
if is_gateway_error(raw):
|
||||
return FAIL, "网关 404:条目未启用或 location 未生效"
|
||||
return WARN, "上游 404:链路通但路径不存在"
|
||||
if 400 <= code < 500:
|
||||
return WARN, "上游 4xx:链路通,业务参数被拒"
|
||||
if is_gateway_error(raw):
|
||||
return FAIL, f"网关 {code}"
|
||||
return WARN, f"上游 {code}(多为探针参数或上游故障,需人工确认)"
|
||||
|
||||
|
||||
def probe(item: dict, key: str, timeout: int) -> dict:
|
||||
slug = item.get("slug") or "?"
|
||||
mode = item.get("mode") or "vendor"
|
||||
if mode == "passthrough":
|
||||
url = f"{LM}/apixp/{slug}/health"
|
||||
method, body, ctype = "GET", None, None
|
||||
else:
|
||||
p = guess_param(item)
|
||||
query = f"?{urllib.parse.urlencode({p: PLACEHOLDER.get(p, 'test')})}" if p else ""
|
||||
url = f"{LM}/apix/{slug}{query}"
|
||||
method, body, ctype = "GET", None, None
|
||||
|
||||
code, raw, elapsed = http(method, url, key, timeout, body, ctype)
|
||||
status, why = classify(code, raw)
|
||||
|
||||
# 无效 Key 预检:证明「location 挂着 + 鉴权生效」(与上游/WAF 是否抽风无关)。
|
||||
# 期望 401 且是网关错误体 —— 若连无效 Key 都没被网关拦下,说明 location 压根没生效。
|
||||
# 401 不计用量(网关只对实际转发记账),所以这一步没有副作用。
|
||||
acode, araw, _ = http(method, url, BAD_KEY, timeout, body, ctype)
|
||||
auth_ok = (acode == 401 and is_gateway_error(araw))
|
||||
if not auth_ok:
|
||||
status = FAIL
|
||||
why = f"网关鉴权未生效(无效 Key 返回 HTTP {acode},期望 401 网关体)"
|
||||
|
||||
return {
|
||||
"slug": slug, "mode": mode, "url": url, "code": code, "status": status,
|
||||
"why": why, "elapsed": elapsed, "raw": raw[:160].replace("\n", " "),
|
||||
"auth_ok": auth_ok, "auth_code": acode,
|
||||
}
|
||||
|
||||
|
||||
def parse_args(argv):
|
||||
key, timeout, only, retry = None, 30, None, True
|
||||
i = 1
|
||||
while i < len(argv):
|
||||
a = argv[i]
|
||||
if a.startswith("--timeout"):
|
||||
timeout = int(a.split("=", 1)[1]) if "=" in a else int(argv[i + 1])
|
||||
i += 1 if "=" not in a else 0
|
||||
elif a == "--no-retry":
|
||||
retry = False
|
||||
elif a.startswith("--slug"):
|
||||
raw = a.split("=", 1)[1] if "=" in a else argv[i + 1]
|
||||
i += 1 if "=" not in a else 0
|
||||
only = set(raw.split(","))
|
||||
elif key is None:
|
||||
key = a
|
||||
i += 1
|
||||
if key is None:
|
||||
print(__doc__)
|
||||
sys.exit(1)
|
||||
return key, timeout, only, retry
|
||||
|
||||
|
||||
def main():
|
||||
key, timeout, only, retry = parse_args(sys.argv)
|
||||
items = fetch_catalog(key, timeout)
|
||||
if items is None:
|
||||
# 拉不到目录本身就是故障:不能「静默无事可做」,否则发版验证会假绿
|
||||
sys.exit(1)
|
||||
if not items:
|
||||
print("· 能力目录里没有 apix 条目 —— 无事可做")
|
||||
sys.exit(0)
|
||||
|
||||
todo = [it for it in items if it.get("enabled")]
|
||||
if only:
|
||||
todo = [it for it in todo if it.get("slug") in only]
|
||||
disabled = [it.get("slug") for it in items if not it.get("enabled")]
|
||||
|
||||
print(f"🌐 {LM} ⏱ timeout={timeout}s 📦 apix 条目 {len(todo)} 个待探"
|
||||
+ (f"(停用跳过:{', '.join(disabled)})" if disabled else ""))
|
||||
print("-" * 66)
|
||||
|
||||
results = []
|
||||
for it in todo:
|
||||
print(f"⏳ {it.get('slug'):<10} ... ", end="", flush=True)
|
||||
r = probe(it, key, timeout)
|
||||
# ⚠️ 多半是边缘限流(见文件头的 1010 实测):隔一会儿重试一次,
|
||||
# 把「临时抽风」和「真的有问题」分开,避免发版验证被噪音污染。
|
||||
if retry and r["status"] == WARN and r["code"] in RETRYABLE:
|
||||
time.sleep(RETRY_WAIT_SEC)
|
||||
r2 = probe(it, key, timeout)
|
||||
if r2["status"] != WARN:
|
||||
r2["retried"] = True
|
||||
r = r2
|
||||
results.append(r)
|
||||
icon = {PASS: "✅", WARN: "⚠️ ", FAIL: "❌"}[r["status"]]
|
||||
mark = "鉴权✓" if r["auth_ok"] else f"鉴权✗(HTTP {r['auth_code']})"
|
||||
msg = f"{icon} [{r['mode']:<11}] {r['slug']:<10} {r['elapsed']:6.2f}s HTTP {r['code']} {mark}"
|
||||
if r.get("retried"):
|
||||
msg += " (重试后)"
|
||||
if r["why"]:
|
||||
msg += f" | {r['why']}"
|
||||
if r["code"] not in (0,) and r["raw"]:
|
||||
msg += f"\n └ {r['raw']}"
|
||||
print(msg)
|
||||
|
||||
print("-" * 66)
|
||||
n_fail = sum(1 for r in results if r["status"] == FAIL)
|
||||
n_warn = sum(1 for r in results if r["status"] == WARN)
|
||||
n_pass = sum(1 for r in results if r["status"] == PASS)
|
||||
print(f"📊 ✅{n_pass} ⚠️ {n_warn} ❌{n_fail} (共 {len(results)} 项)")
|
||||
if n_warn:
|
||||
print(" ⚠️ = 链路通但上游拒单/路径不存在,不是发版问题;按需人工确认")
|
||||
|
||||
sys.exit(1 if n_fail else 0)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
+53
-39
@@ -2,19 +2,23 @@
|
||||
# ---------------------------------------------------------------------------
|
||||
# 上线后验证(live smoke)—— 发版流程的最后一步
|
||||
#
|
||||
# 本机回归(tests/run.sh)跑的是**假上游**,验不到真实链路:`bash deploy/agent.sh verify`
|
||||
# 又只验管理台页面与后端常量,**不发一个真实 LLM 请求**。真正确认「新版在
|
||||
# 线上能干活」需要两半合起来:
|
||||
# 本机回归(tests/run.sh)跑的是**假上游**,验不到真实链路;`deploy/agent.sh verify`
|
||||
# 又只验管理台页面与后端常量,**一个真实请求都不发**。真正确认「新版在线上能干活」
|
||||
# 需要三步合起来:
|
||||
#
|
||||
# 1. 真实流量:tests/test_models.py —— 4 个 slot × OpenAI/Anthropic 双格式真实调用
|
||||
# + `/v1/models` 模型发现(下游 Key 鉴权、slot 路由、Key 替换、出站头都在这条链上)
|
||||
# 2. 环境验收:deploy/agent.sh verify —— 登录、页面新标记、subdomain_rule、趋势、
|
||||
# 1. LLM 真实流量:tests/test_models.py —— 4 slot × OpenAI/Anthropic 真实调用
|
||||
# + `/v1/models`(下游 Key 鉴权、slot 路由、Key 替换、出站头都在这条链上)
|
||||
# 2. apix 探针:tests/apix_check.py —— `/apix/<slug>`(vendor) 与 `/apixp/<slug>`
|
||||
# (passthrough) 是否真的转发(**发版后 apix 最容易静默失效**:location 没挂、
|
||||
# shm 没装载都不会报错,只是 404)
|
||||
# 3. 环境验收:deploy/agent.sh verify —— 登录、页面新标记、subdomain_rule、趋势、
|
||||
# 账号级 load(确认跑的确实是新镜像,而不是旧容器没换)
|
||||
#
|
||||
# 用法:
|
||||
# bash tests/live_check.sh # 两步都跑(key 默认 ~/ssw.txt)
|
||||
# bash tests/live_check.sh # 三步都跑(key 默认 ~/ssw.txt)
|
||||
# bash tests/live_check.sh ~/.keys/ipass.txt # 指定 key 文件
|
||||
# bash tests/live_check.sh --models-only # 只跑真实流量
|
||||
# bash tests/live_check.sh --models-only # 只跑 LLM 真实流量
|
||||
# bash tests/live_check.sh --apix-only # 只跑 apix 探针
|
||||
# bash tests/live_check.sh --verify-only # 只跑环境验收
|
||||
# IPASS_LIVE_KEY_FILE=~/ssw.txt bash tests/live_check.sh
|
||||
#
|
||||
@@ -22,7 +26,7 @@
|
||||
# **取整行会拿到 `sk=sk_xxx` → 全量 401 invalid_api_key**。这里统一:
|
||||
# 跳过空行/注释行 → 有 `=` 取第一个 `=` 之后 → 去引号与空白 → 只认 `sk_` 开头。
|
||||
#
|
||||
# ⚠️ 全程不打印 Key 原文(test_models.py 自带 mask,只显示 `sk_2df...2dae`)。
|
||||
# ⚠️ 全程不打印 Key 原文(子脚本自带 mask,只显示 `sk_2df...2dae`)。
|
||||
# ---------------------------------------------------------------------------
|
||||
set -uo pipefail
|
||||
|
||||
@@ -31,18 +35,22 @@ REPO=$(cd "$HERE/.." && pwd)
|
||||
|
||||
KEY_FILE="${IPASS_LIVE_KEY_FILE:-$HOME/ssw.txt}"
|
||||
RUN_MODELS=1
|
||||
RUN_APIX=1
|
||||
RUN_VERIFY=1
|
||||
|
||||
while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
--models-only) RUN_VERIFY=0; shift ;;
|
||||
--verify-only) RUN_MODELS=0; shift ;;
|
||||
-h|--help) sed -n '2,30p' "${BASH_SOURCE[0]}"; exit 0 ;;
|
||||
--models-only) RUN_APIX=0; RUN_VERIFY=0; shift ;;
|
||||
--apix-only) RUN_MODELS=0; RUN_VERIFY=0; shift ;;
|
||||
--verify-only) RUN_MODELS=0; RUN_APIX=0; shift ;;
|
||||
-h|--help) sed -n '2,32p' "${BASH_SOURCE[0]}"; exit 0 ;;
|
||||
-*) echo "✗ 未知参数:$1(用 -h 看用法)" >&2; exit 2 ;;
|
||||
*) KEY_FILE="$1"; shift ;;
|
||||
esac
|
||||
done
|
||||
|
||||
TOTAL=$((RUN_MODELS + RUN_APIX + RUN_VERIFY))
|
||||
|
||||
# python3:优先隔离的托管运行时,其次系统的(与 tests/run.sh 同一套选择逻辑)
|
||||
PY=""
|
||||
for c in /Users/xs/.workbuddy/binaries/python/versions/3.13.12/bin/python3 python3; do
|
||||
@@ -67,48 +75,54 @@ pick_key() {
|
||||
|
||||
rc=0
|
||||
step=0
|
||||
KEY=""
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 1/2 真实流量
|
||||
# ---------------------------------------------------------------------------
|
||||
if [ "$RUN_MODELS" -eq 1 ]; then
|
||||
step=$((step + 1))
|
||||
echo "== $step/2 真实流量(test_models.py:4 slot × OpenAI/Anthropic) =="
|
||||
|
||||
# 要发真实请求的两步都得先有 Key
|
||||
if [ "$RUN_MODELS" -eq 1 ] || [ "$RUN_APIX" -eq 1 ]; then
|
||||
if [ ! -f "$KEY_FILE" ]; then
|
||||
echo "✗ 找不到 Key 文件:$KEY_FILE"
|
||||
echo " 传路径或用 IPASS_LIVE_KEY_FILE 指定;文件内容可以是裸 key 或 sk=<key>"
|
||||
rc=1
|
||||
else
|
||||
KEY="$(pick_key "$KEY_FILE")"
|
||||
if [ -z "$KEY" ]; then
|
||||
echo "✗ $KEY_FILE 里没解析到 Key(期望一行 sk_ 开头,或 sk=<key> 形式)"
|
||||
rc=1
|
||||
else
|
||||
"$PY" "$HERE/test_models.py" "$KEY" --api both --concurrent --timeout 60
|
||||
mrc=$?
|
||||
[ "$mrc" -ne 0 ] && rc=1
|
||||
if [ "$mrc" -eq 0 ]; then
|
||||
# test_models.py 自己会根据结果退出;这里只做「全绿」提示,判定仍以它为准
|
||||
echo "· 真实流量步骤完成"
|
||||
fi
|
||||
fi
|
||||
exit 1
|
||||
fi
|
||||
KEY="$(pick_key "$KEY_FILE")"
|
||||
if [ -z "$KEY" ]; then
|
||||
echo "✗ $KEY_FILE 里没解析到 Key(期望一行 sk_ 开头,或 sk=<key> 形式)"
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 1/3 LLM 真实流量
|
||||
# ---------------------------------------------------------------------------
|
||||
if [ "$RUN_MODELS" -eq 1 ]; then
|
||||
step=$((step + 1))
|
||||
echo "== $step/$TOTAL LLM 真实流量(test_models.py:4 slot × OpenAI/Anthropic) =="
|
||||
"$PY" "$HERE/test_models.py" "$KEY" --api both --concurrent --timeout 60 || rc=1
|
||||
echo
|
||||
fi
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 2/2 正式环境验收
|
||||
# 2/3 apix 探针
|
||||
# ---------------------------------------------------------------------------
|
||||
if [ "$RUN_APIX" -eq 1 ]; then
|
||||
step=$((step + 1))
|
||||
echo "== $step/$TOTAL apix 探针(apix_check.py:vendor + passthrough) =="
|
||||
# ⚠️ 判定口径见脚本头:只有「网关侧没生效」才算失败,
|
||||
# 上游业务拒绝 / 供应商 WAF(如 Cloudflare 1010)只告警,不染红发版验证。
|
||||
"$PY" "$HERE/apix_check.py" "$KEY" --timeout 40 || rc=1
|
||||
echo
|
||||
fi
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 3/3 正式环境验收
|
||||
# ---------------------------------------------------------------------------
|
||||
if [ "$RUN_VERIFY" -eq 1 ]; then
|
||||
step=$((step + 1))
|
||||
echo "== $step/2 正式环境验收(deploy/agent.sh verify) =="
|
||||
echo "== $step/$TOTAL 正式环境验收(deploy/agent.sh verify) =="
|
||||
if [ ! -f "$REPO/.env" ]; then
|
||||
echo "· 跳过:仓库根没有 .env(控制面凭据只从这里读,没有就跑不了验收)"
|
||||
else
|
||||
bash "$REPO/deploy/agent.sh" verify
|
||||
vrc=$?
|
||||
[ "$vrc" -ne 0 ] && rc=1
|
||||
bash "$REPO/deploy/agent.sh" verify || rc=1
|
||||
fi
|
||||
echo
|
||||
fi
|
||||
|
||||
Reference in new issue
Block a user