feat: 合并 Docker Compose、标注表格优化与部署文档
将 platform + CVAT 合并为单文件 docker-compose.yml,完善 .env 与 init/dev_up 脚本; 新增 docs/DEPLOY.md 与更新 README 以支持新机器部署;含数据湖示例、车队地图、 紧凑表格 UI、ADAS det_7cls 路径与批次台账等近期改动。 Co-authored-by: Cursor <cursoragent@cursor.com>
34
.env.example
@@ -1,26 +1,42 @@
|
||||
# HSAP (Huaxu Sentinel Active Safety Platform) — Docker Compose 环境变量
|
||||
# 复制为 .env 后按部署机器修改路径与公网地址
|
||||
|
||||
# 平台端口
|
||||
# ── 服务端口(宿主机映射)──
|
||||
AS_PLATFORM_PORT=8787
|
||||
# 宿主机映射端口(与旧 active-safety 的 as-postgres:5432 / as-redis:6379 冲突时可改为 5433 / 6380)
|
||||
# 与旧环境冲突时可改为 5433 / 6380
|
||||
AS_DB_PORT=5433
|
||||
AS_REDIS_PORT=6380
|
||||
AS_WEB_DEV_PORT=5173
|
||||
CVAT_PORT=8080
|
||||
|
||||
# 外部 workspace(含 DMS/Lane 大文件数据时填写绝对路径;无则留默认占位目录)
|
||||
# ── 浏览器访问地址(部署到他人电脑 / 局域网时必改)──
|
||||
# 使用实际 IP 或域名,例如 http://192.168.1.100:8787
|
||||
AS_FRONTEND_URL=http://127.0.0.1:8787
|
||||
FEISHU_REDIRECT_URI=http://127.0.0.1:8787/api/v1/auth/feishu/callback
|
||||
# CVAT 画布对外地址(HSAP iframe 嵌入用)
|
||||
CVAT_PUBLIC_URL=http://127.0.0.1:8080
|
||||
# 容器内 CVAT API(一般无需改)
|
||||
CVAT_HOST=http://hsap-cvat-server:8080
|
||||
|
||||
# ── 数据挂载(宿主机绝对路径,按机器填写)──
|
||||
# DMS/Lane 大文件 workspace → 容器 /data/workspace
|
||||
# AS_WORKSPACE_ROOT=/path/to/DATA/workspace
|
||||
# 送标数据湖根目录 → 容器 /data/data(其下应有 送标/adas、送标/dms 等)
|
||||
# AS_DATA_LAKE_HOST=/path/to/DATA/data
|
||||
|
||||
# 车队地图 / T-Box 演示
|
||||
# ── 车队地图 / T-Box 演示 ──
|
||||
AS_FLEET_MAP_ENABLED=1
|
||||
AS_FLEET_MOCK_SEED=1
|
||||
AS_FLEET_MOCK_SIMULATE=1
|
||||
AS_FLEET_SIM_INTERVAL_SEC=8
|
||||
AS_TBOX_INGEST_TOKEN=hsap-demo-tbox-token
|
||||
# 高德底图 Key(国内推荐 gaode;无 Key 时部分瓦片可能受限)
|
||||
# AS_AMAP_KEY=
|
||||
# 车队地图底图:gaode(国内推荐)| osm
|
||||
AS_MAP_TILE_PROVIDER=gaode
|
||||
|
||||
# CVAT 标注引擎(随 make up 一并启动,补丁在 vendor/cvat/patches/)
|
||||
# CVAT_HOST=http://hsap-cvat-server:8080
|
||||
# CVAT_PUBLIC_URL=http://127.0.0.1:8080
|
||||
# CVAT_PORT=8080
|
||||
# ── 可选 MinIO 暂存(docker compose --profile minio up -d)──
|
||||
# MINIO_ROOT_USER=minioadmin
|
||||
# MINIO_ROOT_PASSWORD=minioadmin_change_me
|
||||
# MINIO_API_PORT=9000
|
||||
# MINIO_CONSOLE_PORT=9001
|
||||
# MINIO_BUCKET=hsap-staging
|
||||
|
||||
13
Makefile
@@ -1,17 +1,17 @@
|
||||
.PHONY: up down dev logs build ps health
|
||||
.PHONY: up down dev logs build ps health config
|
||||
|
||||
up:
|
||||
bash scripts/dev_up.sh
|
||||
|
||||
dev:
|
||||
docker compose --profile dev up -d --build web-dev
|
||||
@echo "Vite: http://127.0.0.1:5173"
|
||||
@echo "web-dev 服务已移除;请使用 platform/web 源码 + bash scripts/build_web.sh 后 restart platform"
|
||||
@exit 1
|
||||
|
||||
down:
|
||||
docker compose -f docker-compose.yml -f docker-compose.cvat.yml --profile dev down
|
||||
docker compose down
|
||||
|
||||
logs:
|
||||
docker compose logs -f platform worker
|
||||
docker compose logs -f platform worker cvat_server
|
||||
|
||||
ps:
|
||||
docker compose ps
|
||||
@@ -19,5 +19,8 @@ ps:
|
||||
build:
|
||||
docker compose build
|
||||
|
||||
config:
|
||||
docker compose config
|
||||
|
||||
health:
|
||||
@curl -s http://127.0.0.1:8787/api/v1/health 2>/dev/null | python3 -m json.tool || echo "平台未启动"
|
||||
|
||||
171
README.md
@@ -1,8 +1,10 @@
|
||||
# HSAP — Huaxu Sentinel Active Safety Platform
|
||||
|
||||
**华胥 Sentinel 主动安全算法迭代平台**(HSAP):卡车主动安全(DMS / Lane / ADAS)算法迭代平台,含 **Web UI + API + 审核 + Job 队列 + CVAT 标注 + 算法适配器**。
|
||||
**华胥 Sentinel 主动安全算法迭代平台**(HSAP):卡车主动安全(DMS / Lane / ADAS)算法迭代平台,含 **Web UI + API + 审核 + Job 队列 + CVAT 标注 + 算法适配器 + 车队地图**。
|
||||
|
||||
克隆本仓库即可运行平台;**算法源码与数据集脚手架已内嵌**,无需依赖本地软链。大文件(图像、训练权重)通过外部 `workspace` 或 **送标数据湖** 挂载,不进入 Git。
|
||||
克隆本仓库即可运行;**算法源码与数据集脚手架已内嵌**。大文件(图像、训练权重)通过外部 `workspace` 或 **送标数据湖** 挂载,不进入 Git。
|
||||
|
||||
**新机器部署请直接看 → [docs/DEPLOY.md](docs/DEPLOY.md)**
|
||||
|
||||
---
|
||||
|
||||
@@ -14,77 +16,83 @@ HSAP/
|
||||
├── platform/
|
||||
│ ├── as_platform/ # FastAPI 后端
|
||||
│ │ └── labeling/ # CVAT 客户端、格式转换、任务分配
|
||||
│ └── web/ # React (Vite) 前端
|
||||
│ └── web/ # React (Vite) 前端 → build 到 ui-hsap/dist
|
||||
├── algorithms/
|
||||
│ ├── dms_yolo/code/ # DMS YOLO 训练代码
|
||||
│ └── lane_ufld/code/ # Lane UFLD 代码
|
||||
│ ├── dms_yolo/code/ # DMS YOLO
|
||||
│ └── lane_ufld/code/ # Lane UFLD
|
||||
├── datasets/
|
||||
│ ├── dms/ # DMS 配置/脚本
|
||||
│ ├── lane/ # Lane 列表与脚本
|
||||
│ ├── adas -> ../../data/送标/adas # ADAS 送标数据湖(符号链接)
|
||||
│ └── labeling.registry.yaml # 标注 profile(cuboid_7cls 等)
|
||||
│ ├── lane/
|
||||
│ ├── adas -> ../../data/送标/adas # ADAS 送标数据湖(符号链接,clone 后可手动创建)
|
||||
│ └── labeling.registry.yaml
|
||||
├── lake/lake_example/ # 数据湖 inbox 目录示例与 manifest
|
||||
├── vendor/cvat/patches/ # CVAT no_auth + iframe 补丁
|
||||
├── scripts/ # dev_up、build_web、reset_labeling 等
|
||||
├── scripts/ # dev_up、build_web、init_after_clone 等
|
||||
├── docs/DEPLOY.md # 新机器部署(推荐阅读)
|
||||
├── docs/HANDOVER.md # 详细交接文档
|
||||
├── docker-compose.yml
|
||||
├── docker-compose.cvat.yml # 内置 CVAT 全套服务
|
||||
├── docker-compose.yml # 平台 + CVAT(单文件,已合并)
|
||||
└── manifests/
|
||||
├── feishu.env.example
|
||||
└── feishu.env # 本地生成,不入 Git
|
||||
```
|
||||
|
||||
| 内容 | 是否在 Git 中 | 说明 |
|
||||
|------|---------------|------|
|
||||
| 平台 + 前端 + 适配器 | ✅ | 直接提交 |
|
||||
| 平台 + 前端源码 + 适配器 | ✅ | 直接提交 |
|
||||
| CVAT 补丁 / compose | ✅ | `vendor/cvat/patches/` |
|
||||
| 算法 Python 源码 | ✅ | `scripts/vendor_workspace.sh` 同步 |
|
||||
| 数据集 yaml/脚本 | ✅ | 不含图像 |
|
||||
| 送标图像 / 标定 / 标注 | ❌ | 数据湖 `DATA/data/送标/`,volume 挂载 |
|
||||
| 图像 / 视频 / 权重 | ❌ | `.gitignore` 排除;用 workspace 挂载 |
|
||||
| 前端构建产物 `ui-hsap/dist` | ❌ | `bash scripts/build_web.sh` 生成 |
|
||||
| 算法 Python 源码 | ✅ | `scripts/vendor_workspace.sh` 可同步 |
|
||||
| 送标图像 / 标定 / 标注 | ❌ | 数据湖挂载 |
|
||||
| 图像 / 视频 / 权重 | ❌ | `.gitignore` |
|
||||
|
||||
---
|
||||
|
||||
## 快速开始(Docker,推荐)
|
||||
|
||||
**依赖:** Docker 20+、Docker Compose v2
|
||||
**依赖:** Docker 20+、Docker Compose v2、Node 18+(仅构建前端)
|
||||
|
||||
```bash
|
||||
git clone https://git.sanyele.com/ChengFang.LU/HSAP.git
|
||||
cd HSAP
|
||||
|
||||
bash scripts/init_after_clone.sh # 生成 .env / feishu.env
|
||||
bash scripts/init_after_clone.sh # 生成 .env / feishu.env,构建前端
|
||||
# 编辑 .env:AS_WORKSPACE_ROOT、AS_DATA_LAKE_HOST、AS_FRONTEND_URL
|
||||
bash scripts/dev_up.sh # 或: make up
|
||||
```
|
||||
|
||||
`make up` 会同时启动 **HSAP 平台** 与 **内置 CVAT**(无需单独部署 BK2/cvat,无需 CVAT 账号登录)。
|
||||
`make up` / `dev_up.sh` 一次启动 **HSAP 平台 + 内置 CVAT**(无需单独部署 CVAT,无需 CVAT 账号登录)。
|
||||
|
||||
| 服务 | 地址 | 说明 |
|
||||
|------|------|------|
|
||||
| HSAP 平台 UI + API | http://127.0.0.1:8787 | 登录、送标、标注、审核 |
|
||||
| CVAT 标注画布 | http://127.0.0.1:8080 | 由 HSAP iframe 嵌入;内部引擎 no_auth |
|
||||
| PostgreSQL | localhost:5433 | 默认映射端口(见 `.env`) |
|
||||
| HSAP 平台 UI + API | http://127.0.0.1:8787 | 登录、送标、标注、审核、车队地图 |
|
||||
| CVAT 标注画布 | http://127.0.0.1:8080 | HSAP iframe 嵌入;内部 no_auth |
|
||||
| PostgreSQL | localhost:5433 | 见 `.env` `AS_DB_PORT` |
|
||||
| Redis | localhost:6380 | Job 队列 |
|
||||
|
||||
```bash
|
||||
make logs # platform / worker 日志
|
||||
make down # 停止平台 + CVAT
|
||||
make dev # 额外启动 Vite 热更新 :5173
|
||||
make logs # platform / worker / cvat 日志
|
||||
make down # 停止全部服务
|
||||
make health # API 健康检查
|
||||
```
|
||||
|
||||
**开发登录:** 默认 `manifests/feishu.env` 中 `AS_DEV_AUTH=true`,无需飞书即可登录。
|
||||
**开发登录:** `manifests/feishu.env` 中 `AS_DEV_AUTH=true` 时无需飞书即可登录。
|
||||
|
||||
**改 HSAP 前端后(源码在 `platform/web/`):** 须执行 `bash scripts/build_web.sh` 并 `docker compose restart platform`,否则 8787 仍为旧静态包。
|
||||
**改前端后:** 须 `bash scripts/build_web.sh` 并 `docker compose restart platform`。
|
||||
|
||||
**部署到他人电脑 / 局域网:** 修改 `.env` 中 `AS_FRONTEND_URL`、`CVAT_PUBLIC_URL`、`FEISHU_REDIRECT_URI` 为实际访问地址,详见 [docs/DEPLOY.md](docs/DEPLOY.md)。
|
||||
|
||||
---
|
||||
|
||||
## 标注与送标
|
||||
|
||||
HSAP 统一使用 **CVAT** 作为标注画布;账号、任务分配、权限均在 HSAP 管理,CVAT 仅作内部引擎。
|
||||
HSAP 统一使用 **CVAT** 作为标注画布;账号、任务分配、权限在 HSAP 管理,CVAT 仅作内部引擎。
|
||||
|
||||
### 流程概览
|
||||
|
||||
```text
|
||||
数据湖登记批次 → 协调员「开标」→ CVAT 自动建 Task / 上传图片
|
||||
→ 协调员分配任务(飞书通讯录选人 + 可选 DM 通知)
|
||||
→ 标注员「我的标注」进入 → CVAT 画框/ Cuboid → 保存 → 同步回数据湖
|
||||
→ 协调员分配任务(飞书通讯录 + 可选 DM)
|
||||
→ 标注员「我的标注」→ CVAT 画框 / Cuboid → 保存 → 同步回数据湖
|
||||
→ 协调员提交批次 → 质检 → 导出入库
|
||||
```
|
||||
|
||||
@@ -93,7 +101,7 @@ HSAP 统一使用 **CVAT** 作为标注画布;账号、任务分配、权限
|
||||
| 角色 | 典型页面 |
|
||||
|------|----------|
|
||||
| 协调员 (engineer) | 送标工作台、标注进度、任务分配 |
|
||||
| 标注员 (labeler) | **我的标注**(默认落地页)、标注页 |
|
||||
| 标注员 (labeler) | **我的标注**、标注页 |
|
||||
| 审核员 (reviewer) | 审核队列 |
|
||||
|
||||
### 支持的标注类型
|
||||
@@ -102,56 +110,61 @@ HSAP 统一使用 **CVAT** 作为标注画布;账号、任务分配、权限
|
||||
|---------|------|-----------|
|
||||
| `ddaw` / `addw` 等 | DMS 2D 检测 | Rectangle |
|
||||
| `lane__lane_v1` | 车道线 | Polyline |
|
||||
| `cuboid_7cls` | ADAS 单目 3D Cuboid(7 类) | Cuboid |
|
||||
| `det_7cls` | ADAS 2D 七类检测 | Rectangle |
|
||||
| `cuboid_7cls` | ADAS 单目 3D Cuboid | Cuboid |
|
||||
|
||||
ADAS 7 类:`car`, `pedestrian`, `truck`, `bus`, `motorcycle`, `tricycle`, `traffic cone`
|
||||
|
||||
### 送标数据湖(ADAS Cuboid 示例)
|
||||
### 送标数据湖路径
|
||||
|
||||
宿主机目录(与仓库平级):
|
||||
宿主机(默认 `AS_DATA_LAKE_HOST` = 仓库上级 `../data`):
|
||||
|
||||
```text
|
||||
DATA/data/送标/adas/inbox/cuboid_7cls/<batch>/
|
||||
├── batch.meta.yaml
|
||||
├── images/
|
||||
├── calib/ # 相机内参(可选)
|
||||
│ └── cam0_front_6mm.yaml
|
||||
└── labels/
|
||||
└── quaternion_json/ # 3D 预标注 JSON(可选)
|
||||
```
|
||||
| 类型 | inbox 路径 | project | task |
|
||||
|------|----------|---------|------|
|
||||
| ADAS 2D | `送标/adas/inbox/det_7cls/<batch>/` | `adas` | `det_7cls` |
|
||||
| ADAS 3D | `送标/adas/inbox/cuboid_7cls/<batch>/` | `adas` | `cuboid_7cls` |
|
||||
| DMS | `送标/dms/inbox/<profile>/<batch>/` | `dms` | 见 profile |
|
||||
|
||||
`.env` / `docker-compose.yml` 默认将 `../data` 挂载为容器内 `/data/data`。登记与开标:
|
||||
批次目录示例见 `lake/lake_example/`。登记:**批次台账 → 扫描数据湖 → 登记**。
|
||||
|
||||
```bash
|
||||
# API 或平台 UI:register-batch → 标注进度 → 开标
|
||||
# 批次出现在「标注进度」后即可分配、标注
|
||||
```
|
||||
清空送标/标注 DB(保留账号):`bash scripts/reset_labeling.sh`
|
||||
|
||||
清空送标/标注 DB 记录(保留账号):`bash scripts/reset_labeling.sh`
|
||||
---
|
||||
|
||||
## 环境变量(`.env`)
|
||||
|
||||
| 变量 | 默认 | 说明 |
|
||||
|------|------|------|
|
||||
| `AS_PLATFORM_PORT` | 8787 | HSAP 端口 |
|
||||
| `CVAT_PORT` | 8080 | CVAT Traefik 端口 |
|
||||
| `AS_DB_PORT` | 5433 | PostgreSQL 宿主机端口 |
|
||||
| `AS_REDIS_PORT` | 6380 | Redis 宿主机端口 |
|
||||
| `AS_WORKSPACE_ROOT` | 占位目录 | DMS/Lane 大文件 |
|
||||
| `AS_DATA_LAKE_HOST` | `../data` | 送标数据湖根 |
|
||||
| `AS_FRONTEND_URL` | `http://127.0.0.1:8787` | 浏览器访问 HSAP |
|
||||
| `CVAT_PUBLIC_URL` | `http://127.0.0.1:8080` | 浏览器访问 CVAT |
|
||||
|
||||
完整列表见 [.env.example](.env.example)。
|
||||
|
||||
---
|
||||
|
||||
## 数据挂载
|
||||
|
||||
### 外部 workspace(DMS / Lane 大文件)
|
||||
|
||||
```bash
|
||||
# .env 中设置
|
||||
# .env
|
||||
AS_WORKSPACE_ROOT=/path/to/DATA/workspace
|
||||
AS_DATA_LAKE_HOST=/path/to/DATA/data
|
||||
|
||||
bash scripts/setup_links.sh
|
||||
docker compose -f docker-compose.yml -f docker-compose.cvat.yml up -d --build
|
||||
bash scripts/setup_links.sh # 可选:切换为 workspace 软链模式
|
||||
docker compose up -d --build
|
||||
```
|
||||
|
||||
### 送标数据湖(ADAS 等)
|
||||
确保 `datasets/adas` 指向数据湖(若缺失):
|
||||
|
||||
```bash
|
||||
# .env 可选覆盖宿主机路径(默认 ../data)
|
||||
AS_DATA_LAKE_HOST=/path/to/DATA/data
|
||||
ln -sfn ../../data/送标/adas datasets/adas
|
||||
```
|
||||
|
||||
确保 `HSAP/datasets/adas` 符号链接指向 `data/送标/adas`(clone 后若缺失可手动 `ln -sfn ../../data/送标/adas datasets/adas`)。
|
||||
|
||||
---
|
||||
|
||||
## 本机直跑(不用 Docker 跑 Platform)
|
||||
@@ -162,16 +175,11 @@ bash scripts/init_after_clone.sh
|
||||
bash scripts/run_local.sh
|
||||
```
|
||||
|
||||
- Job 默认 `AS_JOB_EXECUTOR=thread`(无需 Redis)
|
||||
- 无 PostgreSQL 时自动回退 SQLite(`manifests/platform.db`)
|
||||
- 标注功能需 CVAT 可达(`CVAT_HOST`)
|
||||
- Job 默认 `AS_JOB_EXECUTOR=thread`
|
||||
- 无 PostgreSQL 时回退 SQLite
|
||||
- 标注需 CVAT 可达
|
||||
|
||||
仅基础设施用 Docker:
|
||||
|
||||
```bash
|
||||
docker compose up -d postgres redis
|
||||
bash scripts/run_local.sh
|
||||
```
|
||||
仅基础设施用 Docker:`docker compose up -d postgres redis cvat_traefik cvat_server ...`
|
||||
|
||||
---
|
||||
|
||||
@@ -179,19 +187,16 @@ bash scripts/run_local.sh
|
||||
|
||||
```bash
|
||||
cp manifests/feishu.env.example manifests/feishu.env
|
||||
# 填入 FEISHU_APP_ID / FEISHU_APP_SECRET / AS_JWT_SECRET
|
||||
```
|
||||
|
||||
| 角色 | 权限 |
|
||||
|------|------|
|
||||
| admin | 全部 + 用户管理 |
|
||||
| reviewer | 审核批准/驳回 |
|
||||
| reviewer | 审核 |
|
||||
| engineer | 送标开标、任务分配、build/train |
|
||||
| labeler | 我的标注、CVAT 画布保存 |
|
||||
| labeler | 我的标注、CVAT 保存 |
|
||||
| viewer | 只读 |
|
||||
|
||||
**飞书任务分配** 需应用权限:`contact:contact:readonly_as_app`、`im:message:send_as_bot` 等(详见 `docs/HANDOVER.md`)。
|
||||
|
||||
---
|
||||
|
||||
## CLI 示例
|
||||
@@ -200,7 +205,6 @@ cp manifests/feishu.env.example manifests/feishu.env
|
||||
python as.py pending
|
||||
python as.py add dms dam --src /path/to/batch ...
|
||||
python as.py train dms dam --track local
|
||||
python as.py train dms dam --track platform
|
||||
```
|
||||
|
||||
---
|
||||
@@ -219,14 +223,12 @@ git commit -m "chore: refresh vendored algorithm scaffolds"
|
||||
|
||||
```bash
|
||||
bash scripts/sync_to_server.sh user@host:/opt/HSAP
|
||||
bash scripts/sync_to_server.sh user@host:/opt/HSAP --code-only
|
||||
```
|
||||
|
||||
### 推送到 Git 远端
|
||||
|
||||
```bash
|
||||
cd HSAP
|
||||
git status # 确认无 .env、feishu.env、node_modules、*.pt、送标图像
|
||||
git status # 确认无 .env、feishu.env、*.pt、送标图像
|
||||
git push origin main
|
||||
```
|
||||
|
||||
@@ -238,9 +240,10 @@ git push origin main
|
||||
|
||||
| 文档 | 说明 |
|
||||
|------|------|
|
||||
| [docs/HANDOVER.md](docs/HANDOVER.md) | **项目交接**:架构、API、运维、排障 |
|
||||
| [docs/DEPLOY.md](docs/DEPLOY.md) | **新机器部署** |
|
||||
| [docs/HANDOVER.md](docs/HANDOVER.md) | 架构、API、运维、排障 |
|
||||
| [docs/DEVELOPMENT_GUIDE.md](docs/DEVELOPMENT_GUIDE.md) | 开发指南 |
|
||||
| [docs/DATA_LAKE_CHECKLIST.md](docs/DATA_LAKE_CHECKLIST.md) | 数据湖检查清单 |
|
||||
| [lake/lake_example/README.md](lake/lake_example/README.md) | 数据湖 inbox 示例 |
|
||||
|
||||
---
|
||||
|
||||
@@ -248,11 +251,5 @@ git push origin main
|
||||
|
||||
| `AS_JOB_EXECUTOR` | 行为 |
|
||||
|-------------------|------|
|
||||
| `worker` | Docker 默认:API 入队 Redis,worker 执行 |
|
||||
| `thread` | 本机调试:API 进程内线程执行 |
|
||||
|
||||
GPU 训练机:
|
||||
|
||||
```bash
|
||||
PYTHONPATH=platform AS_JOB_EXECUTOR=worker python scripts/worker.py
|
||||
```
|
||||
| `worker` | Docker 默认:Redis 队列 + worker |
|
||||
| `thread` | 本机调试:API 进程内线程 |
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
# addw — packs: dms_v1
|
||||
path: /home/chengfanglu/DATA/workspace/DMS/DATASET/packs/dms_v1/addw
|
||||
train: /home/chengfanglu/DATA/workspace/DMS/DATASET/packs/dms_v1/addw/images/train
|
||||
val: /home/chengfanglu/DATA/workspace/DMS/DATASET/packs/dms_v1/addw/images/val
|
||||
path: /data/hsap/datasets/dms/packs/dms_v1/addw
|
||||
train: /data/hsap/datasets/dms/packs/dms_v1/addw/images/train
|
||||
val: /data/hsap/datasets/dms/packs/dms_v1/addw/images/val
|
||||
|
||||
nc: 4
|
||||
names: ["face", "eye_open", "nod_face", "nod_eye"]
|
||||
|
||||
@@ -370,11 +370,11 @@ def append_log(root: Path, record: dict) -> None:
|
||||
f.write(json.dumps(record, ensure_ascii=False) + "\n")
|
||||
|
||||
|
||||
def run_refresh(root: Path) -> None:
|
||||
subprocess.run(
|
||||
[sys.executable, str(SCRIPT_DIR / "refresh_yaml.py"), "--root", str(root)],
|
||||
check=True,
|
||||
)
|
||||
def run_refresh(root: Path, task: str | None = None) -> None:
|
||||
cmd = [sys.executable, str(SCRIPT_DIR / "refresh_yaml.py"), "--root", str(root)]
|
||||
if task:
|
||||
cmd.extend(["--task", task])
|
||||
subprocess.run(cmd, check=True)
|
||||
|
||||
|
||||
def ingest_one(
|
||||
@@ -588,7 +588,7 @@ def promote_inbox_batch(
|
||||
if not dry_run:
|
||||
append_log(root.resolve(), {"src": str(src), "pack": pack, **result})
|
||||
if refresh:
|
||||
run_refresh(root.resolve())
|
||||
run_refresh(root.resolve(), task=task)
|
||||
return result
|
||||
|
||||
|
||||
|
||||
@@ -36,7 +36,18 @@ def resolve_pack_dir(root: Path, pack_name: str) -> Path:
|
||||
name = reg.get("aliases", {}).get(pack_name, pack_name)
|
||||
for item in reg.get("packs", []):
|
||||
if item.get("name") == name:
|
||||
return (root / item["path"]).resolve()
|
||||
candidate = root / item["path"]
|
||||
if candidate.is_symlink():
|
||||
try:
|
||||
resolved = candidate.resolve()
|
||||
if resolved.is_dir():
|
||||
return resolved
|
||||
except OSError:
|
||||
pass
|
||||
if not candidate.exists():
|
||||
candidate.unlink()
|
||||
candidate.mkdir(parents=True, exist_ok=True)
|
||||
return candidate.resolve()
|
||||
candidate = root / name
|
||||
if candidate.is_dir():
|
||||
return candidate.resolve()
|
||||
|
||||
186
datasets/dms/versions/v3.json
Normal file
@@ -0,0 +1,186 @@
|
||||
{
|
||||
"version_id": "v3",
|
||||
"project": "dms",
|
||||
"created_at": "2026-06-16T02:16:28.125198+00:00",
|
||||
"description": "自动快照 · build dms/addw",
|
||||
"author": "system",
|
||||
"parent_version": "v2",
|
||||
"summary": {
|
||||
"packs_count": 3,
|
||||
"total_images": 138852,
|
||||
"total_labels": 2003,
|
||||
"batches_count": 60
|
||||
},
|
||||
"packs": {
|
||||
"dms_v1": {
|
||||
"train_images": 8794,
|
||||
"val_images": 1053,
|
||||
"test_images": 2129,
|
||||
"class_counts": {},
|
||||
"tasks": [
|
||||
"ddaw",
|
||||
"addw",
|
||||
"addw_face",
|
||||
"adas"
|
||||
]
|
||||
},
|
||||
"dms_v2": {
|
||||
"train_images": 1,
|
||||
"val_images": 0,
|
||||
"test_images": 0,
|
||||
"class_counts": {},
|
||||
"tasks": [
|
||||
"ddaw",
|
||||
"addw",
|
||||
"addw_face",
|
||||
"adas"
|
||||
]
|
||||
},
|
||||
"adas_v1": {
|
||||
"train_images": 116103,
|
||||
"val_images": 12901,
|
||||
"test_images": 0,
|
||||
"class_counts": {},
|
||||
"tasks": [
|
||||
"ddaw",
|
||||
"addw",
|
||||
"addw_face",
|
||||
"adas"
|
||||
]
|
||||
}
|
||||
},
|
||||
"batches": [
|
||||
"20260525_pilot",
|
||||
"e2e_2img_20260616",
|
||||
"once_test_test_cam01_000034",
|
||||
"once_test_test_cam01_000077",
|
||||
"once_test_test_cam01_000168",
|
||||
"once_test_test_cam01_000200",
|
||||
"once_test_test_cam01_000273",
|
||||
"once_test_test_cam01_000275",
|
||||
"once_test_test_cam01_000303",
|
||||
"once_test_test_cam01_000318",
|
||||
"once_test_test_cam01_000322",
|
||||
"once_test_test_cam01_000334",
|
||||
"once_test_test_cam09_000034",
|
||||
"once_test_test_cam09_000077",
|
||||
"once_test_test_cam09_000168",
|
||||
"once_test_test_cam09_000200",
|
||||
"once_test_test_cam09_000273",
|
||||
"once_test_test_cam09_000275",
|
||||
"once_test_test_cam09_000303",
|
||||
"once_test_test_cam09_000318",
|
||||
"once_test_test_cam09_000322",
|
||||
"once_test_test_cam09_000334",
|
||||
"once_train_train_cam01_000076",
|
||||
"once_train_train_cam01_000080",
|
||||
"once_train_train_cam01_000092",
|
||||
"once_train_train_cam01_000104",
|
||||
"once_train_train_cam01_000113",
|
||||
"once_train_train_cam01_000121",
|
||||
"once_train_train_cam03_000076",
|
||||
"once_train_train_cam03_000080",
|
||||
"once_train_train_cam03_000092",
|
||||
"once_train_train_cam03_000104",
|
||||
"once_train_train_cam03_000113",
|
||||
"once_train_train_cam03_000121",
|
||||
"once_train_train_cam09_000076",
|
||||
"once_train_train_cam09_000080",
|
||||
"once_train_train_cam09_000092",
|
||||
"once_train_train_cam09_000104",
|
||||
"once_train_train_cam09_000113",
|
||||
"once_train_train_cam09_000121",
|
||||
"once_val_val_cam01_000027",
|
||||
"once_val_val_cam01_000028",
|
||||
"once_val_val_cam01_000112",
|
||||
"once_val_val_cam01_000201",
|
||||
"once_val_val_cam03_000027",
|
||||
"once_val_val_cam03_000028",
|
||||
"once_val_val_cam03_000112",
|
||||
"once_val_val_cam03_000201",
|
||||
"wf_batch001_202511_fov100_2d_obj_det_data",
|
||||
"wf_batch003_202601_0109_0113_fov75_2d_obj_det_data",
|
||||
"wf_batch006_20240102_near_missing_datas",
|
||||
"wf_batch007_202512_1202_1209_data_with_prelabel",
|
||||
"wf_batch009_202601_0118_0120_truck425_data",
|
||||
"wf_batch011_202601_0122_0124_truck425_data",
|
||||
"wf_batch012_202601_0118_0121_truck_588_282_data",
|
||||
"wf_batch013_202601_0121_0124_truck_588_282_data",
|
||||
"wf_batch014_202601_0125_0131_data",
|
||||
"wf_batch017_202509_0901_0910_data",
|
||||
"wf_batch021_zhongka_bev_biaozhu_front_240710_240718",
|
||||
"wf_batch025_zhongka_bev_biaozhu_front_241213_241225"
|
||||
],
|
||||
"diff": {
|
||||
"added_packs": [
|
||||
"adas_v1"
|
||||
],
|
||||
"removed_packs": [],
|
||||
"added_batches": [
|
||||
"e2e_2img_20260616",
|
||||
"once_test_test_cam01_000034",
|
||||
"once_test_test_cam01_000077",
|
||||
"once_test_test_cam01_000168",
|
||||
"once_test_test_cam01_000200",
|
||||
"once_test_test_cam01_000273",
|
||||
"once_test_test_cam01_000275",
|
||||
"once_test_test_cam01_000303",
|
||||
"once_test_test_cam01_000318",
|
||||
"once_test_test_cam01_000322",
|
||||
"once_test_test_cam01_000334",
|
||||
"once_test_test_cam09_000034",
|
||||
"once_test_test_cam09_000077",
|
||||
"once_test_test_cam09_000168",
|
||||
"once_test_test_cam09_000200",
|
||||
"once_test_test_cam09_000273",
|
||||
"once_test_test_cam09_000275",
|
||||
"once_test_test_cam09_000303",
|
||||
"once_test_test_cam09_000318",
|
||||
"once_test_test_cam09_000322",
|
||||
"once_test_test_cam09_000334",
|
||||
"once_train_train_cam01_000076",
|
||||
"once_train_train_cam01_000080",
|
||||
"once_train_train_cam01_000092",
|
||||
"once_train_train_cam01_000104",
|
||||
"once_train_train_cam01_000113",
|
||||
"once_train_train_cam01_000121",
|
||||
"once_train_train_cam03_000076",
|
||||
"once_train_train_cam03_000080",
|
||||
"once_train_train_cam03_000092",
|
||||
"once_train_train_cam03_000104",
|
||||
"once_train_train_cam03_000113",
|
||||
"once_train_train_cam03_000121",
|
||||
"once_train_train_cam09_000076",
|
||||
"once_train_train_cam09_000080",
|
||||
"once_train_train_cam09_000092",
|
||||
"once_train_train_cam09_000104",
|
||||
"once_train_train_cam09_000113",
|
||||
"once_train_train_cam09_000121",
|
||||
"once_val_val_cam01_000027",
|
||||
"once_val_val_cam01_000028",
|
||||
"once_val_val_cam01_000112",
|
||||
"once_val_val_cam01_000201",
|
||||
"once_val_val_cam03_000027",
|
||||
"once_val_val_cam03_000028",
|
||||
"once_val_val_cam03_000112",
|
||||
"once_val_val_cam03_000201",
|
||||
"wf_batch001_202511_fov100_2d_obj_det_data",
|
||||
"wf_batch003_202601_0109_0113_fov75_2d_obj_det_data",
|
||||
"wf_batch006_20240102_near_missing_datas",
|
||||
"wf_batch007_202512_1202_1209_data_with_prelabel",
|
||||
"wf_batch009_202601_0118_0120_truck425_data",
|
||||
"wf_batch011_202601_0122_0124_truck425_data",
|
||||
"wf_batch012_202601_0118_0121_truck_588_282_data",
|
||||
"wf_batch013_202601_0121_0124_truck_588_282_data",
|
||||
"wf_batch014_202601_0125_0131_data",
|
||||
"wf_batch017_202509_0901_0910_data",
|
||||
"wf_batch021_zhongka_bev_biaozhu_front_240710_240718",
|
||||
"wf_batch025_zhongka_bev_biaozhu_front_241213_241225"
|
||||
],
|
||||
"removed_batches": [
|
||||
"addw_face",
|
||||
"ddaw",
|
||||
"labels"
|
||||
]
|
||||
}
|
||||
}
|
||||
@@ -33,10 +33,11 @@ profiles:
|
||||
scope_key: "lane:lane_v1"
|
||||
export_default: lane_gt_txt
|
||||
ml_adapter: lane_ufld
|
||||
cuboid_7cls:
|
||||
scope_key: "adas:cuboid_7cls"
|
||||
export_default: cvat_cuboid
|
||||
det_7cls:
|
||||
scope_key: "adas:det_7cls"
|
||||
export_default: yolo
|
||||
ml_adapter: adas_yolo26
|
||||
cvat_label_type: rectangle
|
||||
cvat_labels:
|
||||
- pedestrian
|
||||
- car
|
||||
@@ -45,3 +46,13 @@ profiles:
|
||||
- motorcycle
|
||||
- tricycle
|
||||
- traffic cone
|
||||
cuboid_7cls:
|
||||
scope_key: "adas:cuboid_7cls"
|
||||
export_default: cvat_cuboid
|
||||
ml_adapter: adas_yolo26
|
||||
cvat_label_type: cuboid
|
||||
adas:
|
||||
scope_key: "dms:adas"
|
||||
export_default: yolo
|
||||
ml_adapter: adas_yolo26
|
||||
cvat_label_type: rectangle
|
||||
|
||||
@@ -1,236 +0,0 @@
|
||||
# HSAP 内置 CVAT 标注引擎(与 platform 同 compose 项目、同网络)
|
||||
# 用法: docker compose -f docker-compose.yml -f docker-compose.cvat.yml up -d
|
||||
# 补丁源码: vendor/cvat/patches/(no_auth + HSAP iframe 嵌入)
|
||||
|
||||
name: hsap
|
||||
|
||||
x-cvat-server-volumes: &cvat-server-volumes
|
||||
- cvat_data:/home/django/data
|
||||
- cvat_keys:/home/django/keys
|
||||
- cvat_logs:/home/django/logs
|
||||
- ${AS_WORKSPACE_ROOT:-./.workspace-stub}:/home/django/share/workspace:ro
|
||||
- ./vendor/cvat/patches/base.py:/home/django/cvat/settings/base.py:ro
|
||||
- ./vendor/cvat/patches/no_auth.py:/home/django/cvat/apps/iam/no_auth.py:ro
|
||||
- ./vendor/cvat/patches/no_auth_middleware.py:/home/django/cvat/apps/iam/no_auth_middleware.py:ro
|
||||
|
||||
x-cvat-backend-env: &cvat-backend-env
|
||||
CVAT_POSTGRES_HOST: cvat_db
|
||||
CVAT_REDIS_INMEM_HOST: cvat_redis_inmem
|
||||
CVAT_REDIS_INMEM_PORT: 6379
|
||||
CVAT_REDIS_ONDISK_HOST: cvat_redis_ondisk
|
||||
CVAT_REDIS_ONDISK_PORT: 6666
|
||||
CLICKHOUSE_HOST: cvat_clickhouse
|
||||
CLICKHOUSE_PORT: 8123
|
||||
CLICKHOUSE_DB: cvat
|
||||
CLICKHOUSE_USER: user
|
||||
CLICKHOUSE_PASSWORD: user
|
||||
IAM_OPA_URL: http://cvat_opa:8181
|
||||
CVAT_OPA_URL: http://cvat_opa:8181
|
||||
CVAT_ANALYTICS: "0"
|
||||
CVAT_ALLOW_STATIC_CACHE: "no"
|
||||
ALLOWED_HOSTS: "*"
|
||||
SMOKESCREEN_OPTS: ""
|
||||
no_proxy: clickhouse,grafana,vector,opa
|
||||
|
||||
services:
|
||||
cvat_db:
|
||||
image: postgres:15-alpine
|
||||
container_name: hsap-cvat-db
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
POSTGRES_USER: root
|
||||
POSTGRES_DB: cvat
|
||||
POSTGRES_HOST_AUTH_METHOD: trust
|
||||
volumes:
|
||||
- cvat_pgdata:/var/lib/postgresql/data
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "pg_isready -U root -d cvat"]
|
||||
interval: 5s
|
||||
timeout: 5s
|
||||
retries: 8
|
||||
networks:
|
||||
- default
|
||||
|
||||
cvat_redis_inmem:
|
||||
image: redis:7.2.11-alpine
|
||||
container_name: hsap-cvat-redis-inmem
|
||||
restart: unless-stopped
|
||||
command: ["redis-server", "--save", "60", "100", "--appendonly", "yes"]
|
||||
volumes:
|
||||
- cvat_inmem_db:/data
|
||||
networks:
|
||||
- default
|
||||
|
||||
cvat_redis_ondisk:
|
||||
image: apache/kvrocks:2.15.0
|
||||
container_name: hsap-cvat-redis-ondisk
|
||||
restart: unless-stopped
|
||||
init: true
|
||||
volumes:
|
||||
- cvat_cache_db:/var/lib/kvrocks
|
||||
command: ["--compact-cron", "0 3 * * *"]
|
||||
networks:
|
||||
- default
|
||||
|
||||
cvat_clickhouse:
|
||||
image: clickhouse/clickhouse-server:23.11-alpine
|
||||
container_name: hsap-cvat-clickhouse
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
CLICKHOUSE_DB: cvat
|
||||
CLICKHOUSE_USER: user
|
||||
CLICKHOUSE_PASSWORD: user
|
||||
volumes:
|
||||
- cvat_clickhouse:/var/lib/clickhouse
|
||||
- cvat_clickhouse_logs:/var/log/clickhouse-server
|
||||
networks:
|
||||
default:
|
||||
aliases:
|
||||
- clickhouse
|
||||
|
||||
cvat_opa:
|
||||
image: openpolicyagent/opa:1.12.2
|
||||
container_name: hsap-cvat-opa
|
||||
restart: unless-stopped
|
||||
command:
|
||||
- run
|
||||
- --server
|
||||
- --addr=:8181
|
||||
- --log-level=error
|
||||
- --set=services.cvat.url=http://cvat-server:8080
|
||||
- --set=bundles.cvat.service=cvat
|
||||
- --set=bundles.cvat.resource=/api/auth/rules
|
||||
- --set=bundles.cvat.polling.min_delay_seconds=5
|
||||
- --set=bundles.cvat.polling.max_delay_seconds=15
|
||||
networks:
|
||||
default:
|
||||
aliases:
|
||||
- opa
|
||||
depends_on:
|
||||
- cvat_server
|
||||
|
||||
cvat_server:
|
||||
image: cvat/server:dev
|
||||
container_name: hsap-cvat-server
|
||||
restart: unless-stopped
|
||||
command: init run server nginx
|
||||
environment:
|
||||
<<: *cvat-backend-env
|
||||
labels:
|
||||
- traefik.enable=true
|
||||
- traefik.http.services.cvat.loadbalancer.server.port=8080
|
||||
- traefik.http.routers.cvat.rule=Host(`localhost`)||Host(`127.0.0.1`)&&(PathPrefix(`/api/`)||PathPrefix(`/static/`)||PathPrefix(`/admin`))
|
||||
- traefik.http.routers.cvat.entrypoints=web
|
||||
volumes: *cvat-server-volumes
|
||||
depends_on:
|
||||
cvat_db:
|
||||
condition: service_healthy
|
||||
cvat_clickhouse:
|
||||
condition: service_started
|
||||
networks:
|
||||
default:
|
||||
aliases:
|
||||
- cvat-server
|
||||
|
||||
cvat_worker_import:
|
||||
image: cvat/server:dev
|
||||
container_name: hsap-cvat-worker-import
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
<<: *cvat-backend-env
|
||||
volumes: *cvat-server-volumes
|
||||
command: run worker import
|
||||
depends_on:
|
||||
- cvat_server
|
||||
networks:
|
||||
- default
|
||||
|
||||
cvat_worker_export:
|
||||
image: cvat/server:dev
|
||||
container_name: hsap-cvat-worker-export
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
<<: *cvat-backend-env
|
||||
volumes: *cvat-server-volumes
|
||||
command: run worker export
|
||||
depends_on:
|
||||
- cvat_server
|
||||
networks:
|
||||
- default
|
||||
|
||||
cvat_worker_annotation:
|
||||
image: cvat/server:dev
|
||||
container_name: hsap-cvat-worker-annotation
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
<<: *cvat-backend-env
|
||||
volumes: *cvat-server-volumes
|
||||
command: run worker annotation
|
||||
depends_on:
|
||||
- cvat_server
|
||||
networks:
|
||||
- default
|
||||
|
||||
cvat_ui:
|
||||
image: cvat/ui:dev
|
||||
container_name: hsap-cvat-ui
|
||||
restart: unless-stopped
|
||||
labels:
|
||||
- traefik.enable=true
|
||||
- traefik.http.services.cvat-ui.loadbalancer.server.port=8000
|
||||
- traefik.http.routers.cvat-ui.rule=Host(`localhost`)||Host(`127.0.0.1`)
|
||||
- traefik.http.routers.cvat-ui.entrypoints=web
|
||||
# HSAP :8787 iframe 嵌入 CVAT :8080(去掉 UI 默认的 X-Frame-Options: deny)
|
||||
- traefik.http.middlewares.cvat-ui-frame.headers.customResponseHeaders.X-Frame-Options=
|
||||
- traefik.http.middlewares.cvat-ui-frame.headers.customResponseHeaders.Content-Security-Policy=frame-ancestors 'self' http://127.0.0.1:8787 http://localhost:8787 http://127.0.0.1:8080 http://localhost:8080
|
||||
- traefik.http.routers.cvat-ui.middlewares=cvat-ui-frame
|
||||
depends_on:
|
||||
- cvat_server
|
||||
networks:
|
||||
- default
|
||||
|
||||
cvat_traefik:
|
||||
image: traefik:v3.6
|
||||
container_name: hsap-cvat-traefik
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
TRAEFIK_ENTRYPOINTS_web_ADDRESS: :8080
|
||||
TRAEFIK_PROVIDERS_DOCKER_EXPOSEDBYDEFAULT: "false"
|
||||
TRAEFIK_PROVIDERS_DOCKER_NETWORK: hsap_default
|
||||
ports:
|
||||
- "${CVAT_PORT:-8080}:8080"
|
||||
volumes:
|
||||
- /var/run/docker.sock:/var/run/docker.sock:ro
|
||||
depends_on:
|
||||
- cvat_ui
|
||||
- cvat_server
|
||||
networks:
|
||||
- default
|
||||
|
||||
# 让 platform 在 CVAT 网关就绪后再启动(合并 compose 时生效)
|
||||
platform:
|
||||
depends_on:
|
||||
cvat_traefik:
|
||||
condition: service_started
|
||||
|
||||
volumes:
|
||||
cvat_pgdata:
|
||||
name: hsap_cvat_pgdata
|
||||
external: true
|
||||
cvat_inmem_db:
|
||||
name: hsap_cvat_inmem_db
|
||||
external: true
|
||||
cvat_cache_db:
|
||||
name: hsap_cvat_cache_db
|
||||
external: true
|
||||
cvat_clickhouse:
|
||||
cvat_clickhouse_logs:
|
||||
cvat_data:
|
||||
name: hsap_cvat_data
|
||||
external: true
|
||||
cvat_keys:
|
||||
name: hsap_cvat_keys
|
||||
external: true
|
||||
cvat_logs:
|
||||
name: hsap_cvat_logs
|
||||
external: true
|
||||
@@ -1,7 +1,12 @@
|
||||
# HSAP — Huaxu Sentinel Active Safety Platform
|
||||
# 平台 + 内置 CVAT 标注引擎(单文件编排)
|
||||
#
|
||||
# 用法:
|
||||
# docker compose up -d # 仅平台
|
||||
# docker compose -f docker-compose.yml -f docker-compose.cvat.yml up -d # 平台 + CVAT 标注引擎
|
||||
# cp .env.example .env && cp manifests/feishu.env.example manifests/feishu.env
|
||||
# bash scripts/init_after_clone.sh
|
||||
# docker compose up -d --build
|
||||
#
|
||||
# 可选 MinIO: docker compose --profile minio up -d
|
||||
|
||||
name: hsap
|
||||
|
||||
@@ -16,20 +21,53 @@ x-platform-env: &platform-env
|
||||
AS_DEV_AUTH: "true"
|
||||
AS_FORCE_DEV_AUTH: "true"
|
||||
AS_JWT_SECRET: dev-docker-secret
|
||||
AS_FRONTEND_URL: http://127.0.0.1:8787
|
||||
FEISHU_REDIRECT_URI: http://127.0.0.1:8787/api/v1/auth/feishu/callback
|
||||
# 容器内 workspace 挂载点(宿主机路径由 compose volumes 的 AS_WORKSPACE_ROOT 决定)
|
||||
AS_FRONTEND_URL: ${AS_FRONTEND_URL:-http://127.0.0.1:8787}
|
||||
FEISHU_REDIRECT_URI: ${FEISHU_REDIRECT_URI:-http://127.0.0.1:8787/api/v1/auth/feishu/callback}
|
||||
AS_WORKSPACE_ROOT: /data/workspace
|
||||
AS_DATA_LAKE_ROOT: /data/data
|
||||
# CVAT 标注引擎(合并 docker-compose.cvat.yml 时生效;无需 CVAT 账号)
|
||||
CVAT_HOST: ${CVAT_HOST:-http://hsap-cvat-server:8080}
|
||||
CVAT_PUBLIC_URL: ${CVAT_PUBLIC_URL:-http://127.0.0.1:8080}
|
||||
AS_FLEET_MAP_ENABLED: ${AS_FLEET_MAP_ENABLED:-1}
|
||||
AS_FLEET_MOCK_SEED: ${AS_FLEET_MOCK_SEED:-1}
|
||||
AS_FLEET_MOCK_SIMULATE: ${AS_FLEET_MOCK_SIMULATE:-1}
|
||||
AS_FLEET_SIM_INTERVAL_SEC: ${AS_FLEET_SIM_INTERVAL_SEC:-8}
|
||||
AS_TBOX_INGEST_TOKEN: ${AS_TBOX_INGEST_TOKEN:-hsap-demo-tbox-token}
|
||||
AS_MAP_TILE_PROVIDER: ${AS_MAP_TILE_PROVIDER:-gaode}
|
||||
AS_AMAP_KEY: ${AS_AMAP_KEY:-}
|
||||
|
||||
x-platform-volumes: &platform-volumes
|
||||
- .:/data/hsap
|
||||
- ${AS_WORKSPACE_ROOT:-./.workspace-stub}:/data/workspace:ro
|
||||
- ${AS_DATA_LAKE_HOST:-../data}:/data/data
|
||||
|
||||
x-cvat-server-volumes: &cvat-server-volumes
|
||||
- cvat_data:/home/django/data
|
||||
- cvat_keys:/home/django/keys
|
||||
- cvat_logs:/home/django/logs
|
||||
- ${AS_WORKSPACE_ROOT:-./.workspace-stub}:/home/django/share/workspace:ro
|
||||
- ./vendor/cvat/patches/base.py:/home/django/cvat/settings/base.py:ro
|
||||
- ./vendor/cvat/patches/no_auth.py:/home/django/cvat/apps/iam/no_auth.py:ro
|
||||
- ./vendor/cvat/patches/no_auth_middleware.py:/home/django/cvat/apps/iam/no_auth_middleware.py:ro
|
||||
|
||||
x-cvat-backend-env: &cvat-backend-env
|
||||
CVAT_POSTGRES_HOST: cvat_db
|
||||
CVAT_REDIS_INMEM_HOST: cvat_redis_inmem
|
||||
CVAT_REDIS_INMEM_PORT: 6379
|
||||
CVAT_REDIS_ONDISK_HOST: cvat_redis_ondisk
|
||||
CVAT_REDIS_ONDISK_PORT: 6666
|
||||
CLICKHOUSE_HOST: cvat_clickhouse
|
||||
CLICKHOUSE_PORT: 8123
|
||||
CLICKHOUSE_DB: cvat
|
||||
CLICKHOUSE_USER: user
|
||||
CLICKHOUSE_PASSWORD: user
|
||||
IAM_OPA_URL: http://cvat_opa:8181
|
||||
CVAT_OPA_URL: http://cvat_opa:8181
|
||||
CVAT_ANALYTICS: "0"
|
||||
CVAT_ALLOW_STATIC_CACHE: "no"
|
||||
ALLOWED_HOSTS: "*"
|
||||
SMOKESCREEN_OPTS: ""
|
||||
no_proxy: clickhouse,grafana,vector,opa
|
||||
|
||||
services:
|
||||
postgres:
|
||||
image: postgres:16-alpine
|
||||
@@ -78,6 +116,8 @@ services:
|
||||
container_name: hsap-platform
|
||||
restart: unless-stopped
|
||||
env_file:
|
||||
- path: .env
|
||||
required: false
|
||||
- manifests/feishu.env
|
||||
ports:
|
||||
- "${AS_PLATFORM_PORT:-8787}:8787"
|
||||
@@ -89,6 +129,8 @@ services:
|
||||
condition: service_healthy
|
||||
redis:
|
||||
condition: service_healthy
|
||||
cvat_traefik:
|
||||
condition: service_started
|
||||
|
||||
worker:
|
||||
build:
|
||||
@@ -97,6 +139,8 @@ services:
|
||||
container_name: hsap-worker
|
||||
restart: unless-stopped
|
||||
env_file:
|
||||
- path: .env
|
||||
required: false
|
||||
- manifests/feishu.env
|
||||
environment:
|
||||
<<: *platform-env
|
||||
@@ -110,6 +154,180 @@ services:
|
||||
redis:
|
||||
condition: service_healthy
|
||||
|
||||
# ── CVAT 标注引擎(vendor/cvat/patches:no_auth + HSAP iframe 嵌入)──
|
||||
cvat_db:
|
||||
image: postgres:15-alpine
|
||||
container_name: hsap-cvat-db
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
POSTGRES_USER: root
|
||||
POSTGRES_DB: cvat
|
||||
POSTGRES_HOST_AUTH_METHOD: trust
|
||||
volumes:
|
||||
- cvat_pgdata:/var/lib/postgresql/data
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "pg_isready -U root -d cvat"]
|
||||
interval: 5s
|
||||
timeout: 5s
|
||||
retries: 8
|
||||
networks:
|
||||
- default
|
||||
|
||||
cvat_redis_inmem:
|
||||
image: redis:7.2.11-alpine
|
||||
container_name: hsap-cvat-redis-inmem
|
||||
restart: unless-stopped
|
||||
command: ["redis-server", "--save", "60", "100", "--appendonly", "yes"]
|
||||
volumes:
|
||||
- cvat_inmem_db:/data
|
||||
networks:
|
||||
- default
|
||||
|
||||
cvat_redis_ondisk:
|
||||
image: apache/kvrocks:2.15.0
|
||||
container_name: hsap-cvat-redis-ondisk
|
||||
restart: unless-stopped
|
||||
init: true
|
||||
volumes:
|
||||
- cvat_cache_db:/var/lib/kvrocks
|
||||
command: ["--compact-cron", "0 3 * * *"]
|
||||
networks:
|
||||
- default
|
||||
|
||||
cvat_clickhouse:
|
||||
image: clickhouse/clickhouse-server:23.11-alpine
|
||||
container_name: hsap-cvat-clickhouse
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
CLICKHOUSE_DB: cvat
|
||||
CLICKHOUSE_USER: user
|
||||
CLICKHOUSE_PASSWORD: user
|
||||
volumes:
|
||||
- cvat_clickhouse:/var/lib/clickhouse
|
||||
- cvat_clickhouse_logs:/var/log/clickhouse-server
|
||||
networks:
|
||||
default:
|
||||
aliases:
|
||||
- clickhouse
|
||||
|
||||
cvat_server:
|
||||
image: cvat/server:dev
|
||||
container_name: hsap-cvat-server
|
||||
restart: unless-stopped
|
||||
command: init run server nginx
|
||||
environment:
|
||||
<<: *cvat-backend-env
|
||||
labels:
|
||||
- traefik.enable=true
|
||||
- traefik.http.services.cvat.loadbalancer.server.port=8080
|
||||
- traefik.http.routers.cvat.rule=PathPrefix(`/api/`)||PathPrefix(`/static/`)||PathPrefix(`/admin`)
|
||||
- traefik.http.routers.cvat.entrypoints=web
|
||||
volumes: *cvat-server-volumes
|
||||
depends_on:
|
||||
cvat_db:
|
||||
condition: service_healthy
|
||||
cvat_clickhouse:
|
||||
condition: service_started
|
||||
networks:
|
||||
default:
|
||||
aliases:
|
||||
- cvat-server
|
||||
|
||||
cvat_opa:
|
||||
image: openpolicyagent/opa:1.12.2
|
||||
container_name: hsap-cvat-opa
|
||||
restart: unless-stopped
|
||||
command:
|
||||
- run
|
||||
- --server
|
||||
- --addr=:8181
|
||||
- --log-level=error
|
||||
- --set=services.cvat.url=http://cvat-server:8080
|
||||
- --set=bundles.cvat.service=cvat
|
||||
- --set=bundles.cvat.resource=/api/auth/rules
|
||||
- --set=bundles.cvat.polling.min_delay_seconds=5
|
||||
- --set=bundles.cvat.polling.max_delay_seconds=15
|
||||
networks:
|
||||
default:
|
||||
aliases:
|
||||
- opa
|
||||
depends_on:
|
||||
- cvat_server
|
||||
|
||||
cvat_worker_import:
|
||||
image: cvat/server:dev
|
||||
container_name: hsap-cvat-worker-import
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
<<: *cvat-backend-env
|
||||
volumes: *cvat-server-volumes
|
||||
command: run worker import
|
||||
depends_on:
|
||||
- cvat_server
|
||||
networks:
|
||||
- default
|
||||
|
||||
cvat_worker_export:
|
||||
image: cvat/server:dev
|
||||
container_name: hsap-cvat-worker-export
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
<<: *cvat-backend-env
|
||||
volumes: *cvat-server-volumes
|
||||
command: run worker export
|
||||
depends_on:
|
||||
- cvat_server
|
||||
networks:
|
||||
- default
|
||||
|
||||
cvat_worker_annotation:
|
||||
image: cvat/server:dev
|
||||
container_name: hsap-cvat-worker-annotation
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
<<: *cvat-backend-env
|
||||
volumes: *cvat-server-volumes
|
||||
command: run worker annotation
|
||||
depends_on:
|
||||
- cvat_server
|
||||
networks:
|
||||
- default
|
||||
|
||||
cvat_ui:
|
||||
image: cvat/ui:dev
|
||||
container_name: hsap-cvat-ui
|
||||
restart: unless-stopped
|
||||
labels:
|
||||
- traefik.enable=true
|
||||
- traefik.http.services.cvat-ui.loadbalancer.server.port=8000
|
||||
- traefik.http.routers.cvat-ui.rule=PathPrefix(`/`)
|
||||
- traefik.http.routers.cvat-ui.entrypoints=web
|
||||
- traefik.http.middlewares.cvat-ui-frame.headers.customResponseHeaders.X-Frame-Options=
|
||||
- traefik.http.middlewares.cvat-ui-frame.headers.customResponseHeaders.Content-Security-Policy=frame-ancestors 'self' ${AS_FRONTEND_URL:-http://127.0.0.1:8787} ${CVAT_PUBLIC_URL:-http://127.0.0.1:8080}
|
||||
- traefik.http.routers.cvat-ui.middlewares=cvat-ui-frame
|
||||
depends_on:
|
||||
- cvat_server
|
||||
networks:
|
||||
- default
|
||||
|
||||
cvat_traefik:
|
||||
image: traefik:v3.6
|
||||
container_name: hsap-cvat-traefik
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
TRAEFIK_ENTRYPOINTS_web_ADDRESS: :8080
|
||||
TRAEFIK_PROVIDERS_DOCKER_EXPOSEDBYDEFAULT: "false"
|
||||
TRAEFIK_PROVIDERS_DOCKER_NETWORK: hsap_default
|
||||
ports:
|
||||
- "${CVAT_PORT:-8080}:8080"
|
||||
volumes:
|
||||
- /var/run/docker.sock:/var/run/docker.sock:ro
|
||||
depends_on:
|
||||
- cvat_ui
|
||||
- cvat_server
|
||||
networks:
|
||||
- default
|
||||
|
||||
# 可选:S3 兼容暂存(联调)— docker compose --profile minio up -d
|
||||
minio:
|
||||
image: minio/minio:latest
|
||||
@@ -155,3 +373,17 @@ volumes:
|
||||
hsap_pgdata:
|
||||
hsap_redis:
|
||||
hsap_minio_data:
|
||||
cvat_pgdata:
|
||||
name: hsap_cvat_pgdata
|
||||
cvat_inmem_db:
|
||||
name: hsap_cvat_inmem_db
|
||||
cvat_cache_db:
|
||||
name: hsap_cvat_cache_db
|
||||
cvat_clickhouse:
|
||||
cvat_clickhouse_logs:
|
||||
cvat_data:
|
||||
name: hsap_cvat_data
|
||||
cvat_keys:
|
||||
name: hsap_cvat_keys
|
||||
cvat_logs:
|
||||
name: hsap_cvat_logs
|
||||
|
||||
173
docs/DEPLOY.md
Normal file
@@ -0,0 +1,173 @@
|
||||
# HSAP 部署指南(新机器 / 他人电脑)
|
||||
|
||||
本文档面向 **从 Git 克隆后在全新 Linux 机器上启动 HSAP**(平台 + CVAT + PostgreSQL + Redis)。
|
||||
|
||||
---
|
||||
|
||||
## 1. 环境要求
|
||||
|
||||
| 组件 | 版本 |
|
||||
|------|------|
|
||||
| Docker | 20.10+ |
|
||||
| Docker Compose | v2(`docker compose` 子命令) |
|
||||
| Node.js + npm | 18+(仅构建前端;运行期不需要) |
|
||||
| 磁盘 | 建议 ≥ 20GB(含 CVAT 镜像与数据卷) |
|
||||
| 内存 | 建议 ≥ 8GB |
|
||||
|
||||
Ubuntu 示例:
|
||||
|
||||
```bash
|
||||
sudo apt update
|
||||
sudo apt install -y docker.io docker-compose-v2 git nodejs npm
|
||||
sudo usermod -aG docker $USER # 重新登录后生效
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 2. 克隆与初始化
|
||||
|
||||
```bash
|
||||
git clone https://git.sanyele.com/ChengFang.LU/HSAP.git
|
||||
cd HSAP
|
||||
bash scripts/init_after_clone.sh
|
||||
```
|
||||
|
||||
脚本会:
|
||||
|
||||
- 从 `.env.example` 生成 `.env`
|
||||
- 从 `manifests/feishu.env.example` 生成 `manifests/feishu.env`
|
||||
- 若上级目录存在 `workspace/`、`data/`,自动写入挂载路径
|
||||
- 首次构建前端 → `platform/ui-hsap/dist/`
|
||||
|
||||
---
|
||||
|
||||
## 3. 必改配置
|
||||
|
||||
### 3.1 `.env`(Docker Compose)
|
||||
|
||||
```bash
|
||||
cp .env.example .env
|
||||
nano .env
|
||||
```
|
||||
|
||||
| 变量 | 说明 |
|
||||
|------|------|
|
||||
| `AS_WORKSPACE_ROOT` | DMS/Lane 大文件目录(宿主机绝对路径) |
|
||||
| `AS_DATA_LAKE_HOST` | 送标数据湖根目录,下有 `送标/adas`、`送标/dms` 等 |
|
||||
| `AS_FRONTEND_URL` | 浏览器访问 HSAP 的完整 URL(局域网用 `http://<IP>:8787`) |
|
||||
| `CVAT_PUBLIC_URL` | 浏览器访问 CVAT 的 URL(通常 `http://<IP>:8080`) |
|
||||
| `FEISHU_REDIRECT_URI` | 飞书回调,与 `AS_FRONTEND_URL` 同源 |
|
||||
| `AS_DB_PORT` / `AS_REDIS_PORT` | 宿主机端口冲突时修改(默认 5433 / 6380) |
|
||||
|
||||
### 3.2 `manifests/feishu.env`(认证)
|
||||
|
||||
**开发 / 内网演示**(无需飞书):
|
||||
|
||||
```env
|
||||
AS_DEV_AUTH=true
|
||||
AS_JWT_SECRET=请更换为随机长字符串
|
||||
```
|
||||
|
||||
**生产飞书登录**:填入 `FEISHU_APP_ID`、`FEISHU_APP_SECRET`,并将 `AS_DEV_AUTH=false`。
|
||||
|
||||
---
|
||||
|
||||
## 4. 数据目录布局
|
||||
|
||||
推荐与仓库平级:
|
||||
|
||||
```text
|
||||
DATA/
|
||||
├── HSAP/ # 本仓库
|
||||
├── workspace/ # AS_WORKSPACE_ROOT
|
||||
│ ├── DMS/
|
||||
│ └── LaneDection/
|
||||
└── data/ # AS_DATA_LAKE_HOST
|
||||
└── 送标/
|
||||
├── adas/
|
||||
│ └── inbox/
|
||||
│ ├── det_7cls/ # ADAS 2D 七类
|
||||
│ └── cuboid_7cls/ # ADAS 3D Cuboid
|
||||
└── dms/
|
||||
└── inbox/
|
||||
```
|
||||
|
||||
inbox 目录结构示例见仓库内 `lake/lake_example/`。
|
||||
|
||||
`.env` 示例:
|
||||
|
||||
```env
|
||||
AS_WORKSPACE_ROOT=/opt/DATA/workspace
|
||||
AS_DATA_LAKE_HOST=/opt/DATA/data
|
||||
AS_FRONTEND_URL=http://192.168.1.50:8787
|
||||
CVAT_PUBLIC_URL=http://192.168.1.50:8080
|
||||
FEISHU_REDIRECT_URI=http://192.168.1.50:8787/api/v1/auth/feishu/callback
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. 启动
|
||||
|
||||
```bash
|
||||
bash scripts/dev_up.sh
|
||||
# 或
|
||||
make up
|
||||
```
|
||||
|
||||
单文件编排:`docker-compose.yml` 已包含 **平台 + CVAT 全套服务**,无需 `-f` 多文件。
|
||||
|
||||
| 服务 | 默认地址 |
|
||||
|------|----------|
|
||||
| HSAP 平台 | http://127.0.0.1:8787 |
|
||||
| CVAT 画布 | http://127.0.0.1:8080 |
|
||||
| PostgreSQL | localhost:5433 |
|
||||
| Redis | localhost:6380 |
|
||||
|
||||
健康检查:
|
||||
|
||||
```bash
|
||||
make health
|
||||
docker compose ps
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. 前端更新
|
||||
|
||||
修改 `platform/web/` 后须重新构建并重启:
|
||||
|
||||
```bash
|
||||
bash scripts/build_web.sh
|
||||
docker compose restart platform
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7. 常用运维
|
||||
|
||||
```bash
|
||||
docker compose logs -f platform worker cvat_server
|
||||
docker compose down
|
||||
docker compose up -d --build # 拉代码后重建
|
||||
bash scripts/reset_labeling.sh # 清空标注 DB(保留账号)
|
||||
```
|
||||
|
||||
可选 MinIO 暂存:
|
||||
|
||||
```bash
|
||||
docker compose --profile minio up -d
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 8. 排障
|
||||
|
||||
| 现象 | 处理 |
|
||||
|------|------|
|
||||
| 8787 白屏 / 旧 UI | `bash scripts/build_web.sh && docker compose restart platform` |
|
||||
| 标注页 CVAT 无法嵌入 | 检查 `CVAT_PUBLIC_URL` 与 `AS_FRONTEND_URL` 是否为浏览器实际访问地址 |
|
||||
| `CVAT 标注引擎不可用` | `docker compose ps`,确认 `hsap-cvat-server`、`hsap-cvat-traefik` 为 Up |
|
||||
| 送标扫描不到批次 | 确认 `AS_DATA_LAKE_HOST` 挂载正确,`送标/adas/inbox/...` 路径存在 |
|
||||
| 端口冲突 | 修改 `.env` 中 `AS_PLATFORM_PORT`、`CVAT_PORT`、`AS_DB_PORT`、`AS_REDIS_PORT` |
|
||||
|
||||
更多架构与 API 说明见 [HANDOVER.md](HANDOVER.md)。
|
||||
281
docs/DMS_E2E_2IMG_TEST.md
Normal file
@@ -0,0 +1,281 @@
|
||||
# DMS 2 图端到端测试手册(上下文备忘)
|
||||
|
||||
> 更新:2026-06-16
|
||||
> 用途:避免多轮对话丢失状态;记录 E2E 批次、脚本、已知问题与后续待测项。
|
||||
|
||||
---
|
||||
|
||||
## 1. 测试目标
|
||||
|
||||
验证 **QA 门禁完整链路**(DMS / ADDW):
|
||||
|
||||
```text
|
||||
raw_pool → 开 Campaign 标注 → 提交质检 → 质检通过 → 导出 YOLO → 提交 build → 审核批准 → ingested(进训练包)
|
||||
```
|
||||
|
||||
与已完成的 **Unified Ingest SDK**、**ADAS MOON-3D** smoke 并列,本手册专指 **2 张图 DMS 小批次 E2E**。
|
||||
|
||||
---
|
||||
|
||||
## 2. 当前测试批次(已创建)
|
||||
|
||||
| 字段 | 值 |
|
||||
|------|-----|
|
||||
| project | `dms` |
|
||||
| task | `addw`(ADDW 分心检测,4 类 bbox) |
|
||||
| batch | `e2e_2img_20260616` |
|
||||
| pack(入库目标) | `dms_v1`(`workflow.registry.yaml` active_packs) |
|
||||
| campaign_id | `59f7c8fd8402c072bbbf` |
|
||||
| cvat_task_id | `7` |
|
||||
| cvat_job_id | `7` |
|
||||
| cvat_job_url | `http://127.0.0.1:8080/tasks/7/jobs/7` |
|
||||
|
||||
### 数据路径
|
||||
|
||||
```text
|
||||
HSAP/datasets/dms/inbox/addw/e2e_2img_20260616/
|
||||
images/train/
|
||||
eye_open__001_snap_dam_0416_cloudy_5_109.jpg
|
||||
eye_open__002_snap_dam_0522_smoke_phone_484.jpg
|
||||
labels/ls_annotations/ # CVAT 同步后写入
|
||||
labels/yolo/ # 同步时顺带写(可选)
|
||||
batch.meta.yaml # stage 随流程推进更新
|
||||
```
|
||||
|
||||
宿主机 inbox 目录为 **root 所有**(docker 创建);标注/同步由 **hsap-platform 容器** 写入,属正常。
|
||||
|
||||
图片来源:从 `addw/20260525_pilot/images/train/` 复制前 2 张。
|
||||
|
||||
### 平台入口
|
||||
|
||||
- 标注画布:`http://127.0.0.1:8787/labeling/annotate/59f7c8fd8402c072bbbf`
|
||||
- 标注进度:搜 `e2e_2img_20260616`
|
||||
- 导出与入库:`/labeling/export`
|
||||
- 审核队列:`/system/audit`
|
||||
|
||||
### 当前进度(2026-06-16 实测)
|
||||
|
||||
| 项 | 状态 |
|
||||
|----|------|
|
||||
| 落盘 2 图 | ✅ |
|
||||
| register-batch raw_pool | ✅ |
|
||||
| open Campaign | ✅ `out_for_labeling` |
|
||||
| CVAT 标注 + 同步 | ✅ API:`saved=2, shapes=2`;`labeled=2/2` |
|
||||
| 提交质检 | ✅ |
|
||||
| 质检通过 | ✅ → `labeling_submitted` |
|
||||
| 导出 YOLO | ✅ `written=2`(需 **重启 hsap-worker** 后 Job 才走新 runner) |
|
||||
| build + ingested | ✅ `packs/dms_v1/addw/` 含 2 条 labels;`stage=ingested` |
|
||||
|
||||
**全流程结论**:数据已 merge 进 `datasets/dms/packs/dms_v1/addw/`(`labels/train` + `labels/val` 各 1 条);`stage=ingested`。
|
||||
|
||||
---
|
||||
|
||||
## 3. 自动化脚本
|
||||
|
||||
### 3.1 Shell 入口
|
||||
|
||||
```bash
|
||||
cd /home/chengfanglu/DATA/HSAP
|
||||
|
||||
# 查看状态
|
||||
bash scripts/smoke_dms_e2e_2img.sh info
|
||||
# 或
|
||||
python3 platform/as_platform/tests/run_dms_e2e_pipeline.py info
|
||||
|
||||
# 标完后跑全流程(提交→质检→导出→build→校验 ingested)
|
||||
bash scripts/smoke_dms_e2e_2img.sh run
|
||||
|
||||
# 等待标注(最多 600s)再跑
|
||||
DMS_E2E_WAIT_LABEL_SEC=600 bash scripts/smoke_dms_e2e_2img.sh run-wait
|
||||
|
||||
# 重新 setup(会覆盖复制 2 图 + register + open campaign)
|
||||
bash scripts/smoke_dms_e2e_2img.sh setup
|
||||
```
|
||||
|
||||
### 3.2 Python 实现
|
||||
|
||||
`platform/as_platform/tests/run_dms_e2e_pipeline.py`
|
||||
|
||||
- `info` — 打印 campaign_id、stage、labeled 数
|
||||
- `setup` — 调 API open campaign
|
||||
- `run` — API 驱动:submit → review all good → export job → submit-build-batch → approve → 断言 `ingested` + pack sources
|
||||
|
||||
成功标志:stdout 末尾 `DMS_E2E_PIPELINE_OK`
|
||||
|
||||
### 3.3 环境变量
|
||||
|
||||
| 变量 | 默认 |
|
||||
|------|------|
|
||||
| `HSAP_API` | `http://127.0.0.1:8787` |
|
||||
| `DMS_E2E_BATCH` | `e2e_2img_20260616` |
|
||||
| `DMS_E2E_TASK` | `addw` |
|
||||
| `DMS_E2E_PACK` | `dms_v1` |
|
||||
| `DMS_E2E_MIN_IMAGES` | `2` |
|
||||
|
||||
---
|
||||
|
||||
## 4. 相关已完成工作(同会话 / 前序 commit)
|
||||
|
||||
### Commit `0b8ade0` — Unified Ingest SDK
|
||||
|
||||
- `platform/as_platform/data/promote/` — DMS/ADAS promote_batch
|
||||
- ADAS cuboid export / fit / `adas_moon3d_v1`
|
||||
- `platform/as_platform/tests/test_unified_ingest_sdk.py`(7 项单元测试)
|
||||
- `scripts/smoke_adas_promote.sh`
|
||||
- `scripts/smoke_labeling_api.sh` 已接入上述测试
|
||||
|
||||
### 离线测试(均已 PASS)
|
||||
|
||||
```bash
|
||||
bash scripts/smoke_labeling_api.sh # 含 API(platform :8787)
|
||||
bash scripts/smoke_adas_promote.sh
|
||||
python3 platform/as_platform/tests/test_unified_ingest_sdk.py
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. 已知问题与修复记录
|
||||
|
||||
### 5.1 「立即同步」点击无反馈(UI)
|
||||
|
||||
**现象**:用户点「立即同步」似乎没反应。
|
||||
|
||||
**根因(已确认)**:
|
||||
|
||||
1. **后端正常**:`POST /api/v1/labeling/cvat/sync/{campaign_id}` 返回 200,`saved=2, shapes=2`。
|
||||
2. **前端 UX**:
|
||||
- 自动同步每 45s 跑时 `syncInFlight=true`,手动点击被 **静默丢弃**(无提示)。
|
||||
- `shapes=0` 时(CVAT 未 Ctrl+S 保存)仅更新顶栏灰色小字,**无 alert**,易被误认为无效。
|
||||
|
||||
**修复**(`AnnotationPage.tsx`,待 `build_web.sh` 后生效):
|
||||
|
||||
- 同步进行中再点 → 顶栏提示「同步进行中,请稍候…」
|
||||
- 手动同步结束 → **始终 alert**(含「暂无新标注,请先 CVAT 保存」说明)
|
||||
|
||||
**操作提示**:CVAT 画框后必须 **Ctrl+S 保存**,再点「立即同步」。
|
||||
|
||||
### 5.2 inbox 目录权限
|
||||
|
||||
`datasets/dms/inbox/addw/*` 由 docker 创建为 root;宿主机直接写 `labels/` 会 Permission denied。应通过平台 API/容器操作。
|
||||
|
||||
### 5.3 `register-batch` 计数
|
||||
|
||||
`images=4` 偶发(扫描逻辑);实际仅 2 张 jpg,以 `progress total_tasks=2` 为准。
|
||||
|
||||
### 5.4 API `labeling/batches` limit
|
||||
|
||||
查询 `limit` 最大 **100**(非 200),E2E 脚本已修正。
|
||||
|
||||
### 5.5 `get_batch_export_stats` ORM
|
||||
|
||||
已在 session 外读 `camp.project` 修为先提取 `project`(commit `0b8ade0`)。
|
||||
|
||||
---
|
||||
|
||||
## 6. 平台复现步骤(协调员)
|
||||
|
||||
1. **送标工作台** → 待送标 → `e2e_2img_20260616` → 开始标注(若已开 Campaign 则跳过)
|
||||
2. **标注画布** → CVAT 画框 → **Ctrl+S** → **立即同步**(或等 45s 自动同步)
|
||||
3. **标注进度** → **提交质检**
|
||||
4. **标注质检** → 逐张 Good(或跑脚本自动全 Good)
|
||||
5. **导出与入库** → 待导出 → **执行导出** → 待 build → **提交 build**
|
||||
6. **审核队列** → 批准 `build_dms`
|
||||
7. **数据目录** → 训练包 `dms_v1` → `addw` → 见 `sources/e2e_2img_20260616`
|
||||
|
||||
---
|
||||
|
||||
### 5.10 质检详情看不到图 / 框(已修)
|
||||
|
||||
**原因 1(主因)**:`<img src="/api/.../review-image">` **不会带 Bearer Token** → 401,页面显示「图片加载失败」或空白。
|
||||
|
||||
**修复**:`QualityReviewPage` 改为 `fetchReviewImageBlob` + `URL.createObjectURL` 带鉴权加载。
|
||||
|
||||
**原因 2**:质检 overlay 只读 `labels/{stem}.txt`,CVAT 同步写在 `labels/yolo/`、`labels/ls_annotations/`。
|
||||
|
||||
**修复**:`review.py` 多路径解析 + 支持 ls_annotations JSON 画框。
|
||||
|
||||
**原因 3**:图片列表 `rglob *.jpg` 重复(大小写扩展名)→ 质检队列显示 4 张实为 2 张。
|
||||
|
||||
**修复**:改用 `_iter_batch_images` 去重。
|
||||
|
||||
**注意**:若 CVAT 框未保存或宽高为 0,底栏仍显示「标注: 无」——需画布内画有效框并 Ctrl+S。
|
||||
|
||||
---
|
||||
|
||||
`AS_JOB_EXECUTOR=worker`,Job 在 **hsap-worker** 进程执行。容器若长期未重启,可能仍跑旧版 `labeling_export`(误调 `as.py build`)。
|
||||
|
||||
```bash
|
||||
docker restart hsap-worker hsap-platform
|
||||
bash scripts/build_web.sh && docker restart hsap-platform # UI 变更后
|
||||
```
|
||||
|
||||
### 5.7 `build_dms` 校验过严(已修)
|
||||
|
||||
**原问题**:`validate_dms_task()` 要求整个 `packs/dms_v1/addw` 已存在(且常指向损坏的 workspace 软链)。
|
||||
|
||||
**修复**:`validate_dms_inbox_batch(batch_dir)` 只校验 inbox 批次已有 YOLO labels。
|
||||
|
||||
### 5.8 `dms_v1` 损坏软链(已修)
|
||||
|
||||
`datasets/dms/packs/dms_v1` 曾指向不存在的 `/data/workspace/DMS/DATASET/...`,`mkdir` 报 `File exists`。
|
||||
|
||||
**修复**:`pack_registry.resolve_pack_dir` / `DmsYoloPromoteAdapter` 检测断链并回退为 HSAP 内真实目录。
|
||||
|
||||
### 5.9 `refresh_yaml` 拖垮 build Job(已修)
|
||||
|
||||
**原问题**:`promote_inbox_batch` 内 `run_refresh(root)` 无 `--task`,因其他任务(如 ddaw)无 pack 目录而 exit 1,导致 Job 失败(数据其实已 promote)。
|
||||
|
||||
**修复**:`run_refresh(root, task=task)`;单批次 promote 只刷新对应任务 yaml。
|
||||
|
||||
### 分包给本人走「我的标注」(2026-06-16)
|
||||
|
||||
已用代码将 2 张均分给 **卢承方(user_id=5,飞书登录)**:
|
||||
|
||||
- `assigned=2`,`pending=2`,`completed=0`
|
||||
- Campaign 已重置为 `in_progress` / `out_for_labeling`,原标注 JSON 已清空,可重新画框
|
||||
|
||||
**我的标注入口**:http://127.0.0.1:8787/labeling/my-tasks?campaign=59f7c8fd8402c072bbbf
|
||||
|
||||
若用 **dev 登录**(user_id=9 同名账号),需另分配或改用飞书账号登录。
|
||||
|
||||
```bash
|
||||
# 再次分配(协调员 API)
|
||||
curl -X POST -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
|
||||
http://127.0.0.1:8787/api/v1/labeling/campaigns/59f7c8fd8402c072bbbf/assign-tasks \
|
||||
-d '{"mode":"even","user_ids":[5]}'
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
- [x] 跑通 submit → 质检 → 导出 → build → `ingested`
|
||||
- [ ] 前端 rebuild 后复测「立即同步」反馈(已改 `AnnotationPage.tsx`)
|
||||
- [ ] UI 手工与脚本结果对照
|
||||
- [ ] 将 `smoke_dms_e2e_2img.sh run` 接入 `smoke_labeling_api.sh`
|
||||
- [ ] 提交上述 bugfix commit
|
||||
|
||||
---
|
||||
|
||||
## 8. 服务与端口
|
||||
|
||||
| 服务 | 地址 |
|
||||
|------|------|
|
||||
| HSAP 平台 | `http://127.0.0.1:8787` |
|
||||
| CVAT(iframe) | `http://127.0.0.1:8080` |
|
||||
| API 健康 | `GET /api/v1/health` |
|
||||
|
||||
```bash
|
||||
docker compose up -d platform worker
|
||||
docker compose -f docker-compose.yml -f docker-compose.cvat.yml up -d # CVAT
|
||||
bash scripts/build_web.sh && docker restart hsap-platform # UI 更新后
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 9. 其他参考批次(非本 E2E)
|
||||
|
||||
| batch | campaign_id | 说明 |
|
||||
|-------|-------------|------|
|
||||
| `addw/20260525_pilot` | `149329641efe128c00f2` | 24 图,曾 in_review |
|
||||
| ADAS `val_front6mm_pilot` | — | `smoke_adas_promote.sh` CLI 已 PASS |
|
||||
|
||||
ADAS registry 在 **HSAP 仓库外**:`data/送标/adas/adas.registry.yaml`。
|
||||
BIN
docs/bg.png
Normal file
|
After Width: | Height: | Size: 1.9 MiB |
21
lake/lake_example/README.md
Normal file
@@ -0,0 +1,21 @@
|
||||
# 数据湖 inbox 落盘示例
|
||||
|
||||
本仓库镜像 HSAP 真实 inbox 路径。
|
||||
**详细树形说明见 [`datasets/README.md`](datasets/README.md)**,批次清单见 [`datasets/manifest.yaml`](datasets/manifest.yaml)。
|
||||
|
||||
## ADAS 2D / 3D(同在 adas 项目下)
|
||||
|
||||
| 类型 | 路径 | project | task |
|
||||
|------|------|---------|------|
|
||||
| **ADAS 2D 七类** | `datasets/adas/inbox/det_7cls/{批次}/` | `adas` | `det_7cls` |
|
||||
| **ADAS 3D MOON** | `datasets/adas/inbox/cuboid_7cls/{批次}/` | `adas` | `cuboid_7cls` |
|
||||
|
||||
DMS 舱内(addw/ddaw/dam 等)仍在 `datasets/dms/inbox/`。
|
||||
|
||||
## 复制到数据湖
|
||||
|
||||
```bash
|
||||
bash copy_to_inbox.sh
|
||||
```
|
||||
|
||||
然后:**批次台账 → 扫描数据湖 → 登记到台账**。
|
||||
24
lake/lake_example/copy_to_inbox.sh
Executable file
@@ -0,0 +1,24 @@
|
||||
#!/usr/bin/env bash
|
||||
# 将 lake_example 样例复制到 HSAP 数据湖 inbox(不覆盖已有批次)
|
||||
set -euo pipefail
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
HSAP_ROOT="${HSAP_ROOT:-$(cd "$SCRIPT_DIR/../HSAP" && pwd)}"
|
||||
SRC="$SCRIPT_DIR/datasets"
|
||||
|
||||
echo "HSAP_ROOT=$HSAP_ROOT"
|
||||
echo "源: $SRC"
|
||||
echo ""
|
||||
echo "业务线 → 目标路径:"
|
||||
echo " DMS cabin (addw/ddaw/...) → datasets/dms/inbox/"
|
||||
echo " ADAS 2D (adas/inbox/det_7cls) → datasets/adas/inbox/det_7cls/"
|
||||
echo " ADAS 3D (adas/inbox/cuboid) → datasets/adas/inbox/cuboid_7cls/"
|
||||
echo " 车道线 (lane/inbox) → datasets/lane/inbox/"
|
||||
echo ""
|
||||
|
||||
rsync -av --ignore-existing "$SRC/dms/inbox/" "$HSAP_ROOT/datasets/dms/inbox/"
|
||||
rsync -av --ignore-existing "$SRC/adas/inbox/" "$HSAP_ROOT/datasets/adas/inbox/"
|
||||
rsync -av --ignore-existing "$SRC/lane/inbox/" "$HSAP_ROOT/datasets/lane/inbox/"
|
||||
|
||||
echo ""
|
||||
echo "完成。请到 批次台账 → 扫描数据湖 → 登记。"
|
||||
echo "索引: $SRC/README.md | manifest: $SRC/manifest.yaml"
|
||||
39
lake/lake_example/datasets/README.md
Normal file
@@ -0,0 +1,39 @@
|
||||
# 数据湖 inbox 示例根目录
|
||||
|
||||
本目录 **镜像** HSAP 真实数据湖路径,复制后可直接被「扫描数据湖」识别。
|
||||
|
||||
## ADAS 2D / 3D 都在 `adas/` 下
|
||||
|
||||
| 路径 | 类型 | project | task |
|
||||
|------|------|---------|------|
|
||||
| **`adas/inbox/det_7cls/`** | ADAS **2D** 七类 | `adas` | `det_7cls` |
|
||||
| **`adas/inbox/cuboid_7cls/`** | ADAS **3D** MOON | `adas` | `cuboid_7cls` |
|
||||
|
||||
DMS 舱内数据仍在 `dms/inbox/`(addw、ddaw、dam 等),不含 ADAS 七类。
|
||||
|
||||
## 完整目录树
|
||||
|
||||
```text
|
||||
datasets/
|
||||
├── dms/inbox/ project=dms
|
||||
│ ├── addw/20260616_addw_pilot/
|
||||
│ ├── ddaw/20260616_ddaw_pilot/
|
||||
│ ├── addw_face/20260616_face_pilot/
|
||||
│ ├── dam/batch_0516/20260616_dam_wave/
|
||||
│ ├── forward/detect/20260616_fwd_det/
|
||||
│ └── forward/classify/20260616_fwd_cls/
|
||||
├── adas/inbox/ project=adas
|
||||
│ ├── det_7cls/20260616_adas2d_pilot/ ★ 2D 七类
|
||||
│ └── cuboid_7cls/20260616_3d_pilot/ ★ 3D MOON
|
||||
└── lane/inbox/20260616_lane_pilot/ project=lane
|
||||
```
|
||||
|
||||
## 样例批次清单
|
||||
|
||||
见 [`manifest.yaml`](manifest.yaml)。
|
||||
|
||||
## 复制
|
||||
|
||||
```bash
|
||||
bash ../copy_to_inbox.sh
|
||||
```
|
||||
10
lake/lake_example/datasets/adas/inbox/README.md
Normal file
@@ -0,0 +1,10 @@
|
||||
# ADAS inbox(project=adas)
|
||||
|
||||
2D 与 3D 都在本目录下,靠 **task 子目录** 区分。
|
||||
|
||||
| 子目录 | task | 说明 |
|
||||
|--------|------|------|
|
||||
| **det_7cls/** | det_7cls | 2D 七类矩形框 |
|
||||
| cuboid_7cls/ | cuboid_7cls | 3D MOON cuboid |
|
||||
|
||||
复制目标:`HSAP/datasets/adas/inbox/`(软链至 `data/送标/adas/inbox/`)
|
||||
@@ -0,0 +1,6 @@
|
||||
line: ADAS 3D MOON
|
||||
project: adas
|
||||
task: cuboid_7cls
|
||||
mode: null
|
||||
batch: 20260616_3d_pilot
|
||||
inbox_path: datasets/adas/inbox/cuboid_7cls/20260616_3d_pilot
|
||||
@@ -0,0 +1,23 @@
|
||||
类型: ADAS · 3D Cuboid 七类 (MOON-3D)
|
||||
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
★ 同属 project=adas;2D 在同级 det_7cls/ ★
|
||||
|
||||
API 登记:
|
||||
项目 (project): adas
|
||||
任务 (task): cuboid_7cls
|
||||
子模式: (空)
|
||||
|
||||
数据湖路径:
|
||||
datasets/adas/inbox/cuboid_7cls/{批次名}/
|
||||
images/
|
||||
calib/ ← 相机内参,建议必填
|
||||
|
||||
与 2D 的区别:
|
||||
- 3D 本目录 → task=cuboid_7cls, cuboid 标注, 训练包 adas_moon3d_v1
|
||||
- 2D 在 adas/inbox/det_7cls/ → task=det_7cls, 矩形框, 训练包 adas_v1
|
||||
|
||||
标完导出: labels/quaternion_json/*.json
|
||||
|
||||
复制示例:
|
||||
cp -a lake_example/datasets/adas/inbox/cuboid_7cls/20260616_3d_pilot \
|
||||
HSAP/datasets/adas/inbox/cuboid_7cls/
|
||||
@@ -0,0 +1,8 @@
|
||||
schema: huaxu-batch-v1
|
||||
project: adas
|
||||
task: cuboid_7cls
|
||||
batch: 20260616_3d_pilot
|
||||
stage: raw_pool
|
||||
location: inbox
|
||||
format: cvat_cuboid
|
||||
# 3D 批次:建议同目录提供 calib/*.yaml
|
||||
@@ -0,0 +1,18 @@
|
||||
schema: huaxu-camera-calib-v1
|
||||
camera: cam0
|
||||
model: front_6mm
|
||||
image_size: [1920, 1080]
|
||||
|
||||
# Intrinsic Matrix K (3x3)
|
||||
K:
|
||||
- [1189.69669, 0.0, 1007.54822]
|
||||
- [0.0, 1189.69669, 517.47476]
|
||||
- [0.0, 0.0, 1.0]
|
||||
|
||||
fx: 1189.69669
|
||||
fy: 1189.69669
|
||||
cx: 1007.54822
|
||||
cy: 517.47476
|
||||
|
||||
# Distortion Coefficients (k1,k2,p1,p2,k3,k4,k5,k6)
|
||||
distortion: [0.28289, -0.09676, -0.00014, 0.00002, -0.00215, 0.70977, -0.08417, -0.02529]
|
||||
|
After Width: | Height: | Size: 710 KiB |
28
lake/lake_example/datasets/adas/inbox/cuboid_7cls/README.md
Normal file
@@ -0,0 +1,28 @@
|
||||
# ADAS 3D Cuboid 七类(MOON-3D)
|
||||
|
||||
```
|
||||
adas/inbox/cuboid_7cls/{批次名}/
|
||||
images/
|
||||
calib/ ← 建议有,用于 3D 拟合
|
||||
```
|
||||
|
||||
## 台账 / API
|
||||
|
||||
| 字段 | 值 |
|
||||
|------|-----|
|
||||
| **project** | `adas` |
|
||||
| **task** | `cuboid_7cls` |
|
||||
|
||||
## 与 2D 的区别(同在 adas 项目下)
|
||||
|
||||
| | ADAS 2D | ADAS 3D(本目录) |
|
||||
|---|---------|-------------------|
|
||||
| 路径 | `adas/inbox/det_7cls/` | `adas/inbox/cuboid_7cls/` |
|
||||
| task | `det_7cls` | `cuboid_7cls` |
|
||||
| 标注 | 矩形框 bbox | cuboid 3D |
|
||||
| 导出 | YOLO | quaternion_json |
|
||||
| 训练包 | adas_v1 | adas_moon3d_v1 |
|
||||
|
||||
## 样例
|
||||
|
||||
- `20260616_3d_pilot/`
|
||||
@@ -0,0 +1,6 @@
|
||||
line: ADAS 2D
|
||||
project: adas
|
||||
task: det_7cls
|
||||
mode: null
|
||||
batch: 20260616_adas2d_pilot
|
||||
inbox_path: datasets/adas/inbox/det_7cls/20260616_adas2d_pilot
|
||||
@@ -0,0 +1,19 @@
|
||||
类型: ADAS · 2D 七类检测(YOLO)
|
||||
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
★ 与 3D 同属 project=adas,task 不同 ★
|
||||
|
||||
API 登记:
|
||||
项目 (project): adas
|
||||
任务 (task): det_7cls
|
||||
子模式: (空)
|
||||
|
||||
数据湖路径:
|
||||
datasets/adas/inbox/det_7cls/{批次名}/images/
|
||||
|
||||
与 3D 的区别:
|
||||
- 2D 本目录 → task=det_7cls, 矩形框标注, 训练包 adas_v1
|
||||
- 3D 在 adas/inbox/cuboid_7cls/ → task=cuboid_7cls, cuboid 标注, 训练包 adas_moon3d_v1
|
||||
|
||||
复制示例:
|
||||
cp -a lake_example/datasets/adas/inbox/det_7cls/20260616_adas2d_pilot \
|
||||
HSAP/datasets/adas/inbox/det_7cls/
|
||||
@@ -0,0 +1,7 @@
|
||||
schema: huaxu-batch-v1
|
||||
project: adas
|
||||
task: det_7cls
|
||||
batch: 20260616_adas2d_pilot
|
||||
stage: raw_pool
|
||||
location: inbox
|
||||
format: yolo
|
||||
|
After Width: | Height: | Size: 349 KiB |
25
lake/lake_example/datasets/adas/inbox/det_7cls/README.md
Normal file
@@ -0,0 +1,25 @@
|
||||
# ADAS 2D 七类检测(YOLO)
|
||||
|
||||
```
|
||||
adas/inbox/det_7cls/{批次名}/images/
|
||||
```
|
||||
|
||||
## 台账 / API
|
||||
|
||||
| 字段 | 值 |
|
||||
|------|-----|
|
||||
| **project** | `adas` |
|
||||
| **task** | `det_7cls` |
|
||||
|
||||
## 与 3D 同项目、不同任务
|
||||
|
||||
| | ADAS 2D(本目录) | ADAS 3D |
|
||||
|---|-------------------|---------|
|
||||
| 路径 | `adas/inbox/det_7cls/` | `adas/inbox/cuboid_7cls/` |
|
||||
| task | `det_7cls` | `cuboid_7cls` |
|
||||
| 标注 | 矩形框 bbox | cuboid 3D |
|
||||
| 训练包 | adas_v1 | adas_moon3d_v1 |
|
||||
|
||||
## 样例
|
||||
|
||||
- `20260616_adas2d_pilot/`
|
||||
14
lake/lake_example/datasets/dms/inbox/README.md
Normal file
@@ -0,0 +1,14 @@
|
||||
# DMS inbox 示例(project=dms)
|
||||
|
||||
| 子目录 | 任务 task | 子模式 mode | 示例批次 |
|
||||
|--------|-----------|-------------|----------|
|
||||
| addw/ | addw | — | 20260616_addw_pilot |
|
||||
| ddaw/ | ddaw | — | 20260616_ddaw_pilot |
|
||||
| addw_face/ | addw_face | — | 20260616_face_pilot |
|
||||
| dam/batch_0516/ | dam | batch_0516 | 20260616_dam_wave |
|
||||
| forward/detect/ | forward | detect | 20260616_fwd_det |
|
||||
| forward/classify/ | forward | classify | 20260616_fwd_cls |
|
||||
|
||||
ADAS 七类 2D/3D 见 `datasets/adas/inbox/`(不在 dms 下)。
|
||||
|
||||
复制目标:`HSAP/datasets/dms/inbox/`
|
||||
@@ -0,0 +1,5 @@
|
||||
line: DMS ADDW
|
||||
project: dms
|
||||
task: addw
|
||||
batch: 20260616_addw_pilot
|
||||
inbox_path: datasets/dms/inbox/addw/20260616_addw_pilot
|
||||
@@ -0,0 +1,5 @@
|
||||
类型: DMS · ADDW 分心检测 (2D)
|
||||
项目: dms | 任务: addw | 子模式: (空)
|
||||
新进: 仅 images/train/*.jpg,无 labels/
|
||||
复制到: datasets/dms/inbox/addw/20260616_addw_pilot/
|
||||
台账: 项目=dms 任务=addw 批次名=20260616_addw_pilot
|
||||
@@ -0,0 +1,8 @@
|
||||
schema: huaxu-batch-v1
|
||||
project: dms
|
||||
task: addw
|
||||
batch: 20260616_addw_pilot
|
||||
stage: raw_pool
|
||||
location: inbox
|
||||
format: yolo
|
||||
# 登记后由平台写入 counts 等字段;此为手工落盘参考模板
|
||||
|
After Width: | Height: | Size: 349 KiB |
@@ -0,0 +1,5 @@
|
||||
line: DMS ADDW Face
|
||||
project: dms
|
||||
task: addw_face
|
||||
batch: 20260616_face_pilot
|
||||
inbox_path: datasets/dms/inbox/addw_face/20260616_face_pilot
|
||||
@@ -0,0 +1,4 @@
|
||||
类型: DMS · ADDW 人脸关键点 (pose)
|
||||
项目: dms | 任务: addw_face
|
||||
导出: YOLO pose 37 关键点
|
||||
复制到: datasets/dms/inbox/addw_face/20260616_face_pilot/
|
||||
|
After Width: | Height: | Size: 294 KiB |
@@ -0,0 +1,6 @@
|
||||
line: DMS DAM
|
||||
project: dms
|
||||
task: dam
|
||||
mode: batch_0516
|
||||
batch: 20260616_dam_wave
|
||||
inbox_path: datasets/dms/inbox/dam/batch_0516/20260616_dam_wave
|
||||
@@ -0,0 +1,4 @@
|
||||
类型: DMS · DAM 驾驶员监控 (2D)
|
||||
项目: dms | 任务: dam | 子模式: batch_0516 (必填)
|
||||
注意: inbox 路径含 dam/batch_0516/ 两层
|
||||
复制到: datasets/dms/inbox/dam/batch_0516/20260616_dam_wave/
|
||||
|
After Width: | Height: | Size: 349 KiB |
@@ -0,0 +1,5 @@
|
||||
line: DMS DDAW
|
||||
project: dms
|
||||
task: ddaw
|
||||
batch: 20260616_ddaw_pilot
|
||||
inbox_path: datasets/dms/inbox/ddaw/20260616_ddaw_pilot
|
||||
@@ -0,0 +1,3 @@
|
||||
类型: DMS · DDAW 疲劳检测 (2D)
|
||||
项目: dms | 任务: ddaw
|
||||
复制到: datasets/dms/inbox/ddaw/20260616_ddaw_pilot/
|
||||
|
After Width: | Height: | Size: 349 KiB |
@@ -0,0 +1,6 @@
|
||||
line: ADAS Forward Classify
|
||||
project: dms
|
||||
task: forward
|
||||
mode: classify
|
||||
batch: 20260616_fwd_cls
|
||||
inbox_path: datasets/dms/inbox/forward/classify/20260616_fwd_cls
|
||||
@@ -0,0 +1,3 @@
|
||||
类型: ADAS · 前向交通标志 细分类
|
||||
项目: dms | 任务: forward | 子模式: classify
|
||||
复制到: datasets/dms/inbox/forward/classify/20260616_fwd_cls/
|
||||
|
After Width: | Height: | Size: 294 KiB |
@@ -0,0 +1,6 @@
|
||||
line: ADAS Forward Detect
|
||||
project: dms
|
||||
task: forward
|
||||
mode: detect
|
||||
batch: 20260616_fwd_det
|
||||
inbox_path: datasets/dms/inbox/forward/detect/20260616_fwd_det
|
||||
@@ -0,0 +1,3 @@
|
||||
类型: ADAS · 前向交通标志 粗检测 (2D)
|
||||
项目: dms | 任务: forward | 子模式: detect
|
||||
复制到: datasets/dms/inbox/forward/detect/20260616_fwd_det/
|
||||
|
After Width: | Height: | Size: 349 KiB |
@@ -0,0 +1,5 @@
|
||||
line: Lane UFLD
|
||||
project: lane
|
||||
task: lane_v1
|
||||
batch: 20260616_lane_pilot
|
||||
inbox_path: datasets/lane/inbox/20260616_lane_pilot
|
||||
@@ -0,0 +1,5 @@
|
||||
类型: 车道线 UFLD 分割
|
||||
项目: lane | 无 task 子目录
|
||||
新进: 仅 images/*.jpg|png
|
||||
标完导出: annotations/*.png + list/train_gt.txt
|
||||
复制到: datasets/lane/inbox/20260616_lane_pilot/
|
||||
@@ -0,0 +1,6 @@
|
||||
schema: huaxu-batch-v1
|
||||
project: lane
|
||||
batch: 20260616_lane_pilot
|
||||
stage: raw_pool
|
||||
location: inbox
|
||||
format: ufld_archive
|
||||
BIN
lake/lake_example/datasets/lane/inbox/20260616_lane_pilot/images/test_001.png
Executable file
|
After Width: | Height: | Size: 673 KiB |
BIN
lake/lake_example/datasets/lane/inbox/20260616_lane_pilot/images/test_002.png
Executable file
|
After Width: | Height: | Size: 593 KiB |
9
lake/lake_example/datasets/lane/inbox/README.md
Normal file
@@ -0,0 +1,9 @@
|
||||
# 车道线 inbox(project=lane)
|
||||
|
||||
```
|
||||
lane/inbox/{批次名}/images/
|
||||
```
|
||||
|
||||
无 task 子目录。台账 project=lane。
|
||||
|
||||
样例:`20260616_lane_pilot/`
|
||||
70
lake/lake_example/datasets/manifest.yaml
Normal file
@@ -0,0 +1,70 @@
|
||||
version: 1
|
||||
description: lake_example 各业务线落盘样例(镜像 HSAP/datasets/inbox)
|
||||
|
||||
examples:
|
||||
- id: dms_addw
|
||||
line: DMS · ADDW
|
||||
project: dms
|
||||
task: addw
|
||||
mode: null
|
||||
batch: 20260616_addw_pilot
|
||||
path: dms/inbox/addw/20260616_addw_pilot
|
||||
|
||||
- id: dms_ddaw
|
||||
line: DMS · DDAW
|
||||
project: dms
|
||||
task: ddaw
|
||||
batch: 20260616_ddaw_pilot
|
||||
path: dms/inbox/ddaw/20260616_ddaw_pilot
|
||||
|
||||
- id: dms_addw_face
|
||||
line: DMS · ADDW 人脸关键点
|
||||
project: dms
|
||||
task: addw_face
|
||||
batch: 20260616_face_pilot
|
||||
path: dms/inbox/addw_face/20260616_face_pilot
|
||||
|
||||
- id: dms_dam
|
||||
line: DMS · DAM
|
||||
project: dms
|
||||
task: dam
|
||||
mode: batch_0516
|
||||
batch: 20260616_dam_wave
|
||||
path: dms/inbox/dam/batch_0516/20260616_dam_wave
|
||||
|
||||
- id: dms_forward_detect
|
||||
line: ADAS · 前向粗检测
|
||||
project: dms
|
||||
task: forward
|
||||
mode: detect
|
||||
batch: 20260616_fwd_det
|
||||
path: dms/inbox/forward/detect/20260616_fwd_det
|
||||
|
||||
- id: dms_forward_classify
|
||||
line: ADAS · 前向细分类
|
||||
project: dms
|
||||
task: forward
|
||||
mode: classify
|
||||
batch: 20260616_fwd_cls
|
||||
path: dms/inbox/forward/classify/20260616_fwd_cls
|
||||
|
||||
- id: adas_2d
|
||||
line: ADAS · 2D 七类
|
||||
project: adas
|
||||
task: det_7cls
|
||||
batch: 20260616_adas2d_pilot
|
||||
path: adas/inbox/det_7cls/20260616_adas2d_pilot
|
||||
|
||||
- id: adas_3d
|
||||
line: ADAS · 3D MOON Cuboid
|
||||
project: adas
|
||||
task: cuboid_7cls
|
||||
batch: 20260616_3d_pilot
|
||||
path: adas/inbox/cuboid_7cls/20260616_3d_pilot
|
||||
|
||||
- id: lane
|
||||
line: 车道线 UFLD
|
||||
project: lane
|
||||
task: lane_v1
|
||||
batch: 20260616_lane_pilot
|
||||
path: lane/inbox/20260616_lane_pilot
|
||||
@@ -1,5 +1,5 @@
|
||||
{
|
||||
"layout": "workspace_symlinks",
|
||||
"workspace": "/data/workspace",
|
||||
"linked_at": "2026-06-16T01:56:18+00:00"
|
||||
"linked_at": "2026-06-16T08:03:05+00:00"
|
||||
}
|
||||
|
||||
@@ -45,6 +45,45 @@ class DeliveryPatchBody(BaseModel):
|
||||
owner_name: str | None = None
|
||||
|
||||
|
||||
@router.get("/scan")
|
||||
def api_scan_deliveries(
|
||||
_user: Annotated[User, Depends(require_any_permission("read:deliveries", "read:pending", "*"))],
|
||||
projects: str | None = Query(None, description="逗号分隔: dms,adas,lane"),
|
||||
) -> dict[str, Any]:
|
||||
from as_platform.deliveries.scan import scan_delivery_sources
|
||||
|
||||
projs = [p.strip() for p in projects.split(",") if p.strip()] if projects else None
|
||||
return scan_delivery_sources(projects=projs)
|
||||
|
||||
|
||||
class ScanRegisterBody(BaseModel):
|
||||
items: list[dict[str, Any]] = Field(default_factory=list)
|
||||
sync_workbench: bool = True
|
||||
|
||||
|
||||
@router.post("/scan/register")
|
||||
def api_register_scanned_deliveries(
|
||||
body: ScanRegisterBody,
|
||||
user: Annotated[User, Depends(require_any_permission("write:delivery_submit", "*"))],
|
||||
) -> dict[str, Any]:
|
||||
from as_platform.deliveries.scan import register_scanned_to_ledger
|
||||
|
||||
return register_scanned_to_ledger(body.items, user, sync_workbench=body.sync_workbench)
|
||||
|
||||
|
||||
@router.post("/{delivery_id}/sync-workbench")
|
||||
def api_sync_delivery_workbench(
|
||||
delivery_id: str,
|
||||
_user: Annotated[User, Depends(require_any_permission("write:delivery_submit", "*"))],
|
||||
) -> dict[str, Any]:
|
||||
from as_platform.deliveries.scan import bridge_delivery_to_workbench
|
||||
|
||||
try:
|
||||
return bridge_delivery_to_workbench(delivery_id)
|
||||
except ValueError as e:
|
||||
raise HTTPException(400, str(e)) from e
|
||||
|
||||
|
||||
@router.get("")
|
||||
def api_list_deliveries(
|
||||
_user: Annotated[User, Depends(require_any_permission("read:deliveries", "read:pending", "*"))],
|
||||
|
||||
@@ -128,10 +128,40 @@ def api_assign_campaign(
|
||||
def api_labeling_batches(
|
||||
_user: Annotated[User, Depends(require_permission("read:pending"))],
|
||||
stage: str | None = Query(None),
|
||||
stages: str | None = Query(None, description="逗号分隔多阶段,一次扫描返回"),
|
||||
offset: int = Query(0, ge=0),
|
||||
limit: int = Query(20, ge=1, le=100),
|
||||
refresh: bool = Query(False, description="true 时先重建索引再返回"),
|
||||
q: str | None = Query(None, description="搜索批次名/任务/项目"),
|
||||
) -> dict[str, Any]:
|
||||
return list_labeling_batches(stage=stage, offset=offset, limit=limit)
|
||||
stage_list = [s.strip() for s in stages.split(",")] if stages else None
|
||||
return list_labeling_batches(
|
||||
stage=stage, stages=stage_list, offset=offset, limit=limit, refresh=refresh, q=q,
|
||||
)
|
||||
|
||||
|
||||
@router.post("/api/v1/labeling/batches/rebuild-index")
|
||||
def api_rebuild_batch_index(
|
||||
_user: Annotated[User, Depends(require_permission("write:labeling_assign"))],
|
||||
) -> dict[str, Any]:
|
||||
from as_platform.labeling.batch_index import rebuild_batch_index
|
||||
|
||||
return rebuild_batch_index()
|
||||
|
||||
|
||||
@router.post("/api/v1/labeling/batches/{campaign_id}/archive")
|
||||
def api_archive_batch(
|
||||
campaign_id: str,
|
||||
_user: Annotated[User, Depends(require_permission("write:labeling_assign"))],
|
||||
) -> dict[str, Any]:
|
||||
from as_platform.labeling.batch_index import archive_batch
|
||||
|
||||
try:
|
||||
return archive_batch(campaign_id)
|
||||
except FileNotFoundError:
|
||||
raise HTTPException(404, "batch not found") from None
|
||||
except ValueError as e:
|
||||
raise HTTPException(400, str(e)) from e
|
||||
|
||||
|
||||
@router.post("/api/v1/labeling/campaigns/open")
|
||||
@@ -488,23 +518,7 @@ def api_review_image(
|
||||
from as_platform.audit.review import get_review_image
|
||||
import tempfile
|
||||
try:
|
||||
# Get class names from registry
|
||||
import yaml
|
||||
from as_platform.data.core import load_wf, proj_root
|
||||
wf = load_wf()
|
||||
root = proj_root(wf, "dms")
|
||||
reg = yaml.safe_load((root / wf["projects"]["dms"]["registry"]).read_text())
|
||||
# Build class_names dict from campaign scope
|
||||
class_names: dict[int, str] = {}
|
||||
# Try to get from the specific task
|
||||
tasks = reg.get("tasks", {})
|
||||
for task_cfg in tasks.values():
|
||||
names = task_cfg.get("names")
|
||||
if isinstance(names, list):
|
||||
for i, n in enumerate(names):
|
||||
class_names[i] = n
|
||||
|
||||
data = get_review_image(campaign_id, path, class_names)
|
||||
data = get_review_image(campaign_id, path)
|
||||
tmp = tempfile.NamedTemporaryFile(suffix=".jpg", delete=False)
|
||||
tmp.write(data)
|
||||
tmp.close()
|
||||
@@ -528,6 +542,16 @@ def api_review_submit(
|
||||
def api_review_progress(
|
||||
campaign_id: str,
|
||||
_user: Annotated[User, Depends(require_permission("read:pending"))],
|
||||
) -> dict[str, int]:
|
||||
) -> dict[str, Any]:
|
||||
from as_platform.audit.review import review_progress
|
||||
return review_progress(campaign_id)
|
||||
|
||||
|
||||
@router.get("/api/v1/labeling/review-progress")
|
||||
def api_review_progress_batch(
|
||||
_user: Annotated[User, Depends(require_permission("read:pending"))],
|
||||
campaign_ids: str = Query(..., description="逗号分隔 campaign id,最多 50 个"),
|
||||
) -> dict[str, Any]:
|
||||
from as_platform.audit.review import review_progress_batch
|
||||
ids = [x.strip() for x in campaign_ids.split(",") if x.strip()]
|
||||
return review_progress_batch(ids)
|
||||
|
||||
@@ -693,7 +693,8 @@ def api_scan_inbox(
|
||||
project: str = Query("dms"),
|
||||
) -> dict[str, Any]:
|
||||
"""扫描 inbox 目录,返回未登记的新批次。"""
|
||||
from as_platform.data.core import get_pending_report, load_wf, proj_root
|
||||
from as_platform.data.core import load_wf, proj_root
|
||||
from as_platform.labeling.batch_index import index_is_empty, rebuild_batch_index
|
||||
|
||||
wf = load_wf()
|
||||
root = proj_root(wf, project)
|
||||
@@ -701,8 +702,20 @@ def api_scan_inbox(
|
||||
if not inbox.is_dir():
|
||||
return {"project": project, "items": [], "inbox_path": str(inbox)}
|
||||
|
||||
report = get_pending_report()
|
||||
registered = {b.get("batch", "") for b in report.get("batches", [])}
|
||||
if index_is_empty():
|
||||
rebuild_batch_index(wf)
|
||||
|
||||
from as_platform.db.engine import session_scope
|
||||
from as_platform.db.models import BatchIndex
|
||||
|
||||
with session_scope() as db:
|
||||
registered = {
|
||||
(r.task or "", r.batch)
|
||||
for r in db.query(BatchIndex).filter(
|
||||
BatchIndex.project == project,
|
||||
BatchIndex.archived.is_(False),
|
||||
).all()
|
||||
}
|
||||
|
||||
items: list[dict[str, Any]] = []
|
||||
for task_dir in sorted(inbox.iterdir()):
|
||||
@@ -713,7 +726,7 @@ def api_scan_inbox(
|
||||
continue
|
||||
batch_name = batch_dir.name
|
||||
task_name = task_dir.name
|
||||
if batch_name in registered:
|
||||
if (task_name, batch_name) in registered:
|
||||
continue # 已登记
|
||||
|
||||
# Count images (含 images/ 子目录)
|
||||
|
||||
@@ -58,11 +58,107 @@ def _parse_labels(label_path: Path) -> list[dict[str, Any]]:
|
||||
results = []
|
||||
for line in label_path.read_text().strip().splitlines():
|
||||
ann = _parse_yolo_line(line)
|
||||
if ann:
|
||||
if ann and ann["bbox"][2] > 0 and ann["bbox"][3] > 0:
|
||||
results.append(ann)
|
||||
return results
|
||||
|
||||
|
||||
def _class_names_for_campaign(camp) -> dict[int, str]:
|
||||
"""campaign task → class_id → name。"""
|
||||
import yaml
|
||||
from as_platform.data.core import load_wf, proj_root
|
||||
|
||||
if not camp or camp.project != "dms":
|
||||
return {}
|
||||
wf = load_wf()
|
||||
root = proj_root(wf, "dms")
|
||||
reg = yaml.safe_load((root / wf["projects"]["dms"]["registry"]).read_text(encoding="utf-8")) or {}
|
||||
tcfg = (reg.get("tasks") or {}).get(camp.task) or {}
|
||||
if camp.mode and tcfg.get("type") == "multi":
|
||||
mcfg = (tcfg.get("modes") or {}).get(camp.mode) or {}
|
||||
names = mcfg.get("names")
|
||||
else:
|
||||
names = tcfg.get("names")
|
||||
if isinstance(names, list):
|
||||
return {i: str(n) for i, n in enumerate(names)}
|
||||
if isinstance(names, dict):
|
||||
return {int(k): str(v) for k, v in names.items()}
|
||||
return {}
|
||||
|
||||
|
||||
def _name_to_class_id(name: str, class_names: dict[int, str]) -> int:
|
||||
rev = {v.lower(): k for k, v in class_names.items()}
|
||||
return rev.get(name.lower(), 0)
|
||||
|
||||
|
||||
def _resolve_yolo_label_path(batch_dir: Path, img_path: Path) -> Path | None:
|
||||
stem = img_path.stem
|
||||
for rel in (
|
||||
f"labels/{stem}.txt",
|
||||
f"labels/train/{stem}.txt",
|
||||
f"labels/val/{stem}.txt",
|
||||
f"labels/yolo/{stem}.txt",
|
||||
):
|
||||
p = batch_dir / rel
|
||||
if p.is_file():
|
||||
return p
|
||||
return None
|
||||
|
||||
|
||||
def _parse_ls_annotations(path: Path, class_names: dict[int, str]) -> list[dict[str, Any]]:
|
||||
import json
|
||||
|
||||
try:
|
||||
data = json.loads(path.read_text(encoding="utf-8"))
|
||||
except (OSError, json.JSONDecodeError):
|
||||
return []
|
||||
out: list[dict[str, Any]] = []
|
||||
for item in data.get("result") or []:
|
||||
if item.get("type") not in ("rectanglelabels", "rectangle"):
|
||||
continue
|
||||
val = item.get("value") or {}
|
||||
w_pct = float(val.get("width") or 0)
|
||||
h_pct = float(val.get("height") or 0)
|
||||
if w_pct <= 0 or h_pct <= 0:
|
||||
continue
|
||||
x_pct = float(val.get("x") or 0)
|
||||
y_pct = float(val.get("y") or 0)
|
||||
labels = val.get("rectanglelabels") or val.get("labels") or []
|
||||
label = labels[0] if labels else "unknown"
|
||||
cid = _name_to_class_id(str(label), class_names)
|
||||
cx = (x_pct + w_pct / 2) / 100.0
|
||||
cy = (y_pct + h_pct / 2) / 100.0
|
||||
out.append({"class_id": cid, "bbox": (cx, cy, w_pct / 100.0, h_pct / 100.0)})
|
||||
return out
|
||||
|
||||
|
||||
def _load_image_annotations(
|
||||
batch_dir: Path,
|
||||
img_path: Path,
|
||||
class_names: dict[int, str],
|
||||
) -> list[dict[str, Any]]:
|
||||
yolo = _resolve_yolo_label_path(batch_dir, img_path)
|
||||
if yolo:
|
||||
anns = _parse_labels(yolo)
|
||||
if anns:
|
||||
return anns
|
||||
from as_platform.labeling.annotate import _task_id_for_image
|
||||
|
||||
ann_json = batch_dir / "labels" / "ls_annotations" / f"{_task_id_for_image(img_path, batch_dir)}.json"
|
||||
if ann_json.is_file():
|
||||
return _parse_ls_annotations(ann_json, class_names)
|
||||
return []
|
||||
|
||||
|
||||
def _image_has_labels(batch_dir: Path, img_path: Path, class_names: dict[int, str]) -> bool:
|
||||
return bool(_load_image_annotations(batch_dir, img_path, class_names))
|
||||
|
||||
|
||||
def _list_review_images(batch_dir: Path) -> list[Path]:
|
||||
from as_platform.labeling.annotate import _iter_batch_images
|
||||
|
||||
return list(_iter_batch_images(batch_dir))
|
||||
|
||||
# ── Optimized overlay render ──
|
||||
|
||||
PALETTE = [(220, 20, 60), (30, 144, 255), (50, 205, 50), (255, 165, 0), (186, 85, 211), (0, 206, 209)]
|
||||
@@ -70,7 +166,7 @@ PALETTE = [(220, 20, 60), (30, 144, 255), (50, 205, 50), (255, 165, 0), (186, 85
|
||||
|
||||
def render_review_overlay(
|
||||
image_path: Path,
|
||||
label_path: Path | None,
|
||||
batch_dir: Path,
|
||||
class_names: dict[int, str],
|
||||
*,
|
||||
max_size: int = 800,
|
||||
@@ -88,7 +184,7 @@ def render_review_overlay(
|
||||
font = _get_font(max(12, min(16, w // 50)))
|
||||
line_w = max(1, w // 400)
|
||||
|
||||
anns = _parse_labels(label_path) if label_path else []
|
||||
anns = _load_image_annotations(batch_dir, image_path, class_names)
|
||||
for ann in anns:
|
||||
cid = ann["class_id"]
|
||||
color = PALETTE[cid % len(PALETTE)]
|
||||
@@ -140,17 +236,14 @@ def get_review_queue(campaign_id: str, offset: int = 0, limit: int = 20) -> dict
|
||||
if not camp:
|
||||
return {"items": [], "total": 0, "hint": "Campaign 不存在"}
|
||||
batch_dir = resolve_campaign_batch_dir(camp)
|
||||
class_names = _class_names_for_campaign(camp)
|
||||
if not batch_dir or not batch_dir.is_dir():
|
||||
return {"items": [], "total": 0, "hint": "批次目录不存在"}
|
||||
|
||||
img_dir = batch_dir / "images"
|
||||
if not img_dir.is_dir():
|
||||
all_images = _list_review_images(batch_dir)
|
||||
if not all_images:
|
||||
return {"items": [], "total": 0, "hint": "无 images 目录"}
|
||||
|
||||
all_images: list[Path] = []
|
||||
for ext in IMAGE_EXTS:
|
||||
all_images.extend(sorted(img_dir.rglob(f"*{ext}")))
|
||||
|
||||
# Get existing reviews
|
||||
with session_scope() as db:
|
||||
reviewed = {
|
||||
@@ -160,26 +253,26 @@ def get_review_queue(campaign_id: str, offset: int = 0, limit: int = 20) -> dict
|
||||
|
||||
total = len(all_images)
|
||||
page = all_images[offset:offset + limit]
|
||||
score_counts = {"good": 0, "fine": 0, "bad": 0, "pending": 0}
|
||||
items = []
|
||||
for img in page:
|
||||
rel = str(img.relative_to(batch_dir))
|
||||
score = reviewed.get(rel, "pending")
|
||||
score_counts[score] += 1
|
||||
label_path = batch_dir / "labels" / (img.stem + ".txt")
|
||||
items.append({
|
||||
"id": rel, "image_path": rel,
|
||||
"fileName": img.name,
|
||||
"score": score,
|
||||
"has_label": label_path.is_file(),
|
||||
"has_label": _image_has_labels(batch_dir, img, class_names),
|
||||
})
|
||||
|
||||
# Fill remaining counts
|
||||
for img in all_images:
|
||||
rel = str(img.relative_to(batch_dir))
|
||||
s = reviewed.get(rel, "pending")
|
||||
if s not in score_counts:
|
||||
score_counts[s] = 0
|
||||
with session_scope() as db:
|
||||
db_counts = _review_db_counts(db, campaign_id)
|
||||
reviewed_n = sum(db_counts.values())
|
||||
score_counts = {
|
||||
"good": db_counts.get("good", 0),
|
||||
"fine": db_counts.get("fine", 0),
|
||||
"bad": db_counts.get("bad", 0),
|
||||
"pending": max(0, total - reviewed_n),
|
||||
}
|
||||
|
||||
return {
|
||||
"items": items, "total": total,
|
||||
@@ -188,7 +281,7 @@ def get_review_queue(campaign_id: str, offset: int = 0, limit: int = 20) -> dict
|
||||
}
|
||||
|
||||
|
||||
def get_review_image(campaign_id: str, image_rel_path: str, class_names: dict[int, str]) -> bytes:
|
||||
def get_review_image(campaign_id: str, image_rel_path: str) -> bytes:
|
||||
from as_platform.labeling.annotate import resolve_campaign_batch_dir
|
||||
from as_platform.db.engine import session_scope
|
||||
from as_platform.db.models import LabelingCampaign
|
||||
@@ -197,11 +290,13 @@ def get_review_image(campaign_id: str, image_rel_path: str, class_names: dict[in
|
||||
if not camp:
|
||||
raise FileNotFoundError("Campaign 不存在")
|
||||
batch_dir = resolve_campaign_batch_dir(camp)
|
||||
class_names = _class_names_for_campaign(camp)
|
||||
if not batch_dir:
|
||||
raise FileNotFoundError("批次不存在")
|
||||
img_path = batch_dir / image_rel_path
|
||||
lbl_path = batch_dir / "labels" / (img_path.stem + ".txt")
|
||||
return render_review_overlay(img_path, lbl_path if lbl_path.is_file() else None, class_names)
|
||||
if not img_path.is_file():
|
||||
raise FileNotFoundError(f"图片不存在: {image_rel_path}")
|
||||
return render_review_overlay(img_path, batch_dir, class_names)
|
||||
|
||||
|
||||
def submit_review_scores(
|
||||
@@ -251,22 +346,76 @@ def submit_review_scores(
|
||||
|
||||
reviewed = sum(counts.values())
|
||||
if reviewed >= total_images and total_images > 0:
|
||||
pass_rate = counts.get("good", 0) / max(total_images, 1)
|
||||
new_stage = "review_approved" if pass_rate >= 0.8 else "review_rejected"
|
||||
_update_campaign_stage(db, campaign_id, new_stage)
|
||||
new_stage = _effective_stage_from_review(
|
||||
counts.get("good", 0), counts.get("fine", 0), counts.get("bad", 0), total_images,
|
||||
)
|
||||
if new_stage and new_stage != "in_review":
|
||||
raw = "review_approved" if new_stage == "labeling_submitted" else new_stage
|
||||
_update_campaign_stage(db, campaign_id, raw)
|
||||
|
||||
return {"ok": True, "updated": updated, "auto_advanced": reviewed >= total_images if total_images > 0 else False}
|
||||
auto_advanced = reviewed >= total_images if total_images > 0 else False
|
||||
acceptable = counts.get("good", 0) + counts.get("fine", 0) if total_images > 0 else 0
|
||||
final_stage = None
|
||||
if auto_advanced and total_images > 0:
|
||||
eff = _effective_stage_from_review(
|
||||
counts.get("good", 0), counts.get("fine", 0), counts.get("bad", 0), total_images,
|
||||
)
|
||||
final_stage = "review_approved" if eff == "labeling_submitted" else eff
|
||||
return {
|
||||
"ok": True,
|
||||
"updated": updated,
|
||||
"auto_advanced": auto_advanced,
|
||||
"stage": final_stage,
|
||||
}
|
||||
|
||||
|
||||
def _review_db_counts(db, campaign_id: str) -> dict[str, int]:
|
||||
from sqlalchemy import func
|
||||
from collections import Counter
|
||||
rows = db.query(LabelingReview.score, func.count()).filter(
|
||||
LabelingReview.campaign_id == campaign_id
|
||||
).group_by(LabelingReview.score).all()
|
||||
return {score: cnt for score, cnt in rows}
|
||||
|
||||
|
||||
PASS_RATE_THRESHOLD = 0.8
|
||||
|
||||
|
||||
def _effective_stage_from_review(good: int, fine: int, bad: int, total: int) -> str | None:
|
||||
"""Return campaign status after QA is complete; None if images remain unreviewed."""
|
||||
if total <= 0:
|
||||
return None
|
||||
reviewed = good + fine + bad
|
||||
if reviewed < total:
|
||||
return "in_review"
|
||||
acceptable = good + fine
|
||||
approved = acceptable / total >= PASS_RATE_THRESHOLD
|
||||
return "labeling_submitted" if approved else "review_rejected"
|
||||
|
||||
|
||||
def reconcile_review_stage(campaign_id: str) -> str | None:
|
||||
"""Align stored campaign stage with current review scores (fixes stale rejections)."""
|
||||
summary = _review_summary(campaign_id)
|
||||
if not summary.get("complete"):
|
||||
return summary.get("stage")
|
||||
expected = _effective_stage_from_review(
|
||||
summary["good"], summary["fine"], summary["bad"], summary["total"],
|
||||
)
|
||||
if not expected:
|
||||
return summary.get("stage")
|
||||
with session_scope() as db:
|
||||
from as_platform.db.models import LabelingCampaign
|
||||
camp = db.get(LabelingCampaign, campaign_id)
|
||||
if not camp:
|
||||
return None
|
||||
if camp.status == expected:
|
||||
return expected
|
||||
camp.status = expected
|
||||
from as_platform.labeling.batch_stage import update_campaign_batch_meta_stage
|
||||
update_campaign_batch_meta_stage(camp, expected)
|
||||
db.commit()
|
||||
return expected
|
||||
|
||||
|
||||
def _update_campaign_stage(db, campaign_id: str, new_stage: str) -> None:
|
||||
from as_platform.db.models import LabelingCampaign
|
||||
from as_platform.labeling.batch_stage import update_campaign_batch_meta_stage
|
||||
@@ -278,10 +427,63 @@ def _update_campaign_stage(db, campaign_id: str, new_stage: str) -> None:
|
||||
update_campaign_batch_meta_stage(camp, effective)
|
||||
|
||||
|
||||
def review_progress(campaign_id: str) -> dict[str, int]:
|
||||
def _review_summary(campaign_id: str) -> dict[str, Any]:
|
||||
from as_platform.labeling.annotate import resolve_campaign_batch_dir
|
||||
from as_platform.db.models import LabelingCampaign
|
||||
|
||||
with session_scope() as db:
|
||||
rows = db.query(LabelingReview).filter(LabelingReview.campaign_id == campaign_id).all()
|
||||
counts = {"good": 0, "fine": 0, "bad": 0, "pending": 0}
|
||||
for r in rows:
|
||||
counts[r.score] = counts.get(r.score, 0) + 1
|
||||
return counts
|
||||
camp = db.get(LabelingCampaign, campaign_id)
|
||||
if not camp:
|
||||
return {"good": 0, "fine": 0, "bad": 0, "pending": 0, "total": 0, "reviewed": 0, "pass_rate": 0, "complete": False, "stage": ""}
|
||||
batch_dir = resolve_campaign_batch_dir(camp)
|
||||
stage = camp.status or ""
|
||||
if not batch_dir or not batch_dir.is_dir():
|
||||
counts = _review_db_counts(db, campaign_id)
|
||||
reviewed = sum(counts.values())
|
||||
return {
|
||||
**{k: counts.get(k, 0) for k in ("good", "fine", "bad")},
|
||||
"pending": 0,
|
||||
"total": reviewed,
|
||||
"reviewed": reviewed,
|
||||
"pass_rate": round((counts.get("good", 0) + counts.get("fine", 0)) / max(reviewed, 1) * 100),
|
||||
"complete": reviewed > 0,
|
||||
"stage": stage,
|
||||
}
|
||||
|
||||
all_images = _list_review_images(batch_dir)
|
||||
db_counts = _review_db_counts(db, campaign_id)
|
||||
|
||||
total = len(all_images)
|
||||
good = db_counts.get("good", 0)
|
||||
fine = db_counts.get("fine", 0)
|
||||
bad = db_counts.get("bad", 0)
|
||||
reviewed = good + fine + bad
|
||||
acceptable = good + fine
|
||||
return {
|
||||
"good": good,
|
||||
"fine": fine,
|
||||
"bad": bad,
|
||||
"pending": max(0, total - reviewed),
|
||||
"total": total,
|
||||
"reviewed": reviewed,
|
||||
"pass_rate": round(acceptable / max(total, 1) * 100),
|
||||
"complete": reviewed >= total and total > 0,
|
||||
"stage": stage,
|
||||
}
|
||||
|
||||
|
||||
def review_progress(campaign_id: str) -> dict[str, Any]:
|
||||
result = _review_summary(campaign_id)
|
||||
if result.get("complete"):
|
||||
reconciled = reconcile_review_stage(campaign_id)
|
||||
if reconciled:
|
||||
result["stage"] = reconciled
|
||||
return result
|
||||
|
||||
|
||||
def review_progress_batch(campaign_ids: list[str]) -> dict[str, Any]:
|
||||
ids = [c.strip() for c in campaign_ids if c and c.strip()][:50]
|
||||
items: dict[str, Any] = {}
|
||||
for cid in ids:
|
||||
items[cid] = review_progress(cid)
|
||||
return {"items": items}
|
||||
|
||||
@@ -1098,15 +1098,22 @@ def register_batch(
|
||||
|
||||
meta_path = write_meta(batch_dir, data)
|
||||
invalidate_catalog_cache()
|
||||
batch_row = enrich_batch(
|
||||
batch_dir,
|
||||
project=project,
|
||||
task=task,
|
||||
pack=pack,
|
||||
batch=batch,
|
||||
location=location,
|
||||
)
|
||||
try:
|
||||
from as_platform.labeling.batch_index import upsert_batch_dict
|
||||
|
||||
upsert_batch_dict(batch_row)
|
||||
except Exception:
|
||||
pass
|
||||
return {
|
||||
"ok": True,
|
||||
"meta_path": str(meta_path),
|
||||
"batch": enrich_batch(
|
||||
batch_dir,
|
||||
project=project,
|
||||
task=task,
|
||||
pack=pack,
|
||||
batch=batch,
|
||||
location=location,
|
||||
),
|
||||
"batch": batch_row,
|
||||
}
|
||||
|
||||
@@ -425,6 +425,11 @@ def promote_candidate_to_inbox(
|
||||
raise ValueError(f"任务 {task} 为 multi,须指定 mode(如 batch_0516)")
|
||||
dest = _resolve_dms_inbox_dest(root, reg, task, eff_mode, batch_name)
|
||||
reg_batch = eff_mode or batch_name
|
||||
elif project == "adas":
|
||||
if not task:
|
||||
raise ValueError("ADAS 晋级需要 task(det_7cls 或 cuboid_7cls)")
|
||||
dest = root / "inbox" / task / batch_name
|
||||
reg_batch = batch_name
|
||||
else:
|
||||
dest = root / "inbox" / batch_name
|
||||
reg_batch = batch_name
|
||||
@@ -439,7 +444,7 @@ def promote_candidate_to_inbox(
|
||||
row = enrich_batch(
|
||||
dest,
|
||||
project=project,
|
||||
task=task if project == "dms" else None,
|
||||
task=task,
|
||||
pack=None,
|
||||
batch=reg_batch,
|
||||
location="inbox",
|
||||
@@ -482,6 +487,12 @@ def promote_candidate_to_inbox(
|
||||
db.flush()
|
||||
|
||||
invalidate_catalog_cache()
|
||||
try:
|
||||
from as_platform.labeling.batch_index import upsert_batch_dict
|
||||
|
||||
upsert_batch_dict({**meta, "path": str(dest), "location": "inbox"})
|
||||
except Exception:
|
||||
pass
|
||||
return {
|
||||
"ok": True,
|
||||
"candidate_id": candidate_id,
|
||||
|
||||
@@ -6,20 +6,35 @@ from pathlib import Path
|
||||
|
||||
from as_platform.data.promote.base import PackPromoteAdapter, PromoteContext, PromoteResult
|
||||
from as_platform.data.promote.manifest import refresh_dms_yaml
|
||||
from as_platform.data.promote.validate.dms_yolo import validate_dms_task
|
||||
from as_platform.data.promote.validate.dms_yolo import validate_dms_inbox_batch
|
||||
|
||||
_DMS_SCRIPTS = Path(__file__).resolve().parents[4] / "datasets" / "dms" / "scripts"
|
||||
if str(_DMS_SCRIPTS) not in sys.path:
|
||||
sys.path.insert(0, str(_DMS_SCRIPTS))
|
||||
|
||||
|
||||
def _resolve_promote_pack_dir(project_root: Path, pack: str) -> Path:
|
||||
"""解析 pack 目录;损坏的 workspace 软链则回退为 HSAP 内真实目录。"""
|
||||
candidate = project_root / "packs" / pack
|
||||
if candidate.is_symlink():
|
||||
try:
|
||||
resolved = candidate.resolve()
|
||||
if resolved.is_dir():
|
||||
return resolved
|
||||
except OSError:
|
||||
pass
|
||||
candidate.unlink()
|
||||
candidate.mkdir(parents=True, exist_ok=True)
|
||||
return candidate
|
||||
|
||||
|
||||
class DmsYoloPromoteAdapter(PackPromoteAdapter):
|
||||
project = "dms"
|
||||
|
||||
def validate(self, ctx: PromoteContext) -> list[str]:
|
||||
if ctx.skip_validate:
|
||||
return []
|
||||
return validate_dms_task(ctx.task)
|
||||
return validate_dms_inbox_batch(ctx.batch_dir)
|
||||
|
||||
def promote(self, ctx: PromoteContext) -> PromoteResult:
|
||||
from ingest_incremental import promote_inbox_batch
|
||||
@@ -34,8 +49,7 @@ class DmsYoloPromoteAdapter(PackPromoteAdapter):
|
||||
warnings=[f"batch_dir missing: {ctx.batch_dir}"],
|
||||
)
|
||||
|
||||
pack_dir = ctx.project_root / "packs" / ctx.pack
|
||||
pack_dir.mkdir(parents=True, exist_ok=True)
|
||||
pack_dir = _resolve_promote_pack_dir(ctx.project_root, ctx.pack)
|
||||
|
||||
detail = promote_inbox_batch(
|
||||
root=ctx.project_root,
|
||||
@@ -46,7 +60,7 @@ class DmsYoloPromoteAdapter(PackPromoteAdapter):
|
||||
dry_run=ctx.dry_run,
|
||||
refresh=ctx.refresh and not ctx.dry_run,
|
||||
)
|
||||
if ctx.refresh and not ctx.dry_run and not ctx.skip_validate:
|
||||
if ctx.refresh and not ctx.dry_run:
|
||||
refresh_dms_yaml(task=ctx.task)
|
||||
|
||||
added = int(detail.get("added") or 0)
|
||||
|
||||
@@ -8,6 +8,17 @@ from pathlib import Path
|
||||
from as_platform.config import WORKSPACE
|
||||
|
||||
|
||||
def validate_dms_inbox_batch(batch_dir: Path) -> list[str]:
|
||||
"""Promote 前校验单个 inbox 批次(不要求 pack 目录已存在)。"""
|
||||
from as_platform.labeling.batch_stage import batch_has_yolo_labels
|
||||
|
||||
if not batch_dir.is_dir():
|
||||
return [f"batch_dir missing: {batch_dir}"]
|
||||
if not batch_has_yolo_labels(batch_dir):
|
||||
return [f"no YOLO labels under {batch_dir} (先执行 labeling_export)"]
|
||||
return []
|
||||
|
||||
|
||||
def validate_dms_task(task: str | None) -> list[str]:
|
||||
cmd = [sys.executable, str(WORKSPACE / "scripts" / "validate_dms_tasks.py")]
|
||||
if task:
|
||||
|
||||
@@ -28,7 +28,7 @@ ROLE_DEFS: dict[str, tuple[str, list[str]]] = {
|
||||
]),
|
||||
"engineer": ("算法工程师", [
|
||||
"read:catalog", "read:pending", "read:jobs", "read:audit", "read:fleet", "write:fleet",
|
||||
"write:approval_submit", "write:delivery_submit", "read:deliveries",
|
||||
"write:approval_submit", "write:approval_review", "write:delivery_submit", "read:deliveries",
|
||||
"write:labeling_vendor", "write:labeling_assign",
|
||||
]),
|
||||
"labeler": ("标注协调", [
|
||||
@@ -79,6 +79,7 @@ def init_database() -> None:
|
||||
_ensure_feishu_bitable_columns(db)
|
||||
_ensure_approval_columns(db)
|
||||
_ensure_operation_log_columns(db)
|
||||
_ensure_batch_index_columns(db)
|
||||
_seed_roles_permissions(db)
|
||||
_seed_fleet_demo(db)
|
||||
_import_jsonl_if_empty(db)
|
||||
@@ -168,18 +169,22 @@ def _import_jsonl_if_empty(db: Session) -> None:
|
||||
|
||||
|
||||
def assign_default_role(db: Session, user: User) -> None:
|
||||
"""新用户默认角色;支持 open_id / 部门白名单自动 admin。"""
|
||||
"""新用户默认角色;白名单用户每次登录也会补齐 admin。"""
|
||||
admin_role = db.query(Role).filter_by(code="admin").first()
|
||||
user_dept_ids = set(user.feishu_department_ids())
|
||||
if user.feishu_open_id and user.feishu_open_id in FEISHU_ADMIN_OPEN_IDS:
|
||||
role_code = "admin"
|
||||
elif user_dept_ids and user_dept_ids.intersection(FEISHU_ADMIN_DEPARTMENT_IDS):
|
||||
role_code = "admin"
|
||||
elif not user.roles:
|
||||
role_code = "engineer"
|
||||
else:
|
||||
if admin_role and admin_role not in user.roles:
|
||||
user.roles.append(admin_role)
|
||||
return
|
||||
if user_dept_ids and user_dept_ids.intersection(FEISHU_ADMIN_DEPARTMENT_IDS):
|
||||
if admin_role and admin_role not in user.roles:
|
||||
user.roles.append(admin_role)
|
||||
return
|
||||
if user.roles:
|
||||
return
|
||||
role_code = "engineer"
|
||||
role = db.query(Role).filter_by(code=role_code).first()
|
||||
if role and role not in user.roles:
|
||||
if role:
|
||||
user.roles.append(role)
|
||||
|
||||
|
||||
@@ -325,6 +330,10 @@ def _ensure_approval_columns(db: Session) -> None:
|
||||
_ensure_table_columns(db, "approvals", {"rejection_category": "VARCHAR(32) DEFAULT ''"})
|
||||
|
||||
|
||||
def _ensure_batch_index_columns(db: Session) -> None:
|
||||
_ensure_table_columns(db, "batch_index", {"archived": "BOOLEAN DEFAULT FALSE"})
|
||||
|
||||
|
||||
def _ensure_operation_log_columns(db: Session) -> None:
|
||||
"""确保 operation_logs 表存在并包含所有列。"""
|
||||
inspector_args = {}
|
||||
|
||||
@@ -319,6 +319,67 @@ class BatchDelivery(Base):
|
||||
}
|
||||
|
||||
|
||||
class BatchIndex(Base):
|
||||
"""批次列表索引:由扫盘/登记/阶段变更写入,列表 API 只读此表。"""
|
||||
|
||||
__tablename__ = "batch_index"
|
||||
|
||||
campaign_id = Column(String(64), primary_key=True)
|
||||
project = Column(String(32), nullable=False, index=True)
|
||||
task = Column(String(64), nullable=True, index=True)
|
||||
mode = Column(String(64), nullable=True)
|
||||
batch = Column(String(128), nullable=False, index=True)
|
||||
pack = Column(String(64), nullable=True)
|
||||
location = Column(String(32), nullable=False, default="inbox")
|
||||
stage = Column(String(32), nullable=False, index=True)
|
||||
batch_path = Column(String(1024), nullable=True)
|
||||
scope_key = Column(String(128), nullable=True)
|
||||
engineer = Column(String(128), nullable=True)
|
||||
format = Column(String(32), nullable=True)
|
||||
image_count = Column(Integer, nullable=False, default=0)
|
||||
label_count = Column(Integer, nullable=False, default=0)
|
||||
has_meta = Column(Boolean, nullable=False, default=False)
|
||||
registry_only = Column(Boolean, nullable=False, default=False)
|
||||
next_cli = Column(String(512), nullable=True)
|
||||
domain = Column(String(32), nullable=True)
|
||||
domain_label = Column(String(64), nullable=True)
|
||||
task_label = Column(String(128), nullable=True)
|
||||
mode_label = Column(String(128), nullable=True)
|
||||
labeling_profile = Column(String(128), nullable=True)
|
||||
export_default = Column(String(128), nullable=True)
|
||||
ml_adapter = Column(String(128), nullable=True)
|
||||
indexed_at = Column(DateTime(timezone=True), nullable=False)
|
||||
archived = Column(Boolean, nullable=False, default=False, index=True)
|
||||
|
||||
def to_list_row(self) -> dict:
|
||||
return {
|
||||
"project": self.project,
|
||||
"task": self.task,
|
||||
"mode": self.mode,
|
||||
"batch": self.batch,
|
||||
"pack": self.pack,
|
||||
"stage": self.stage,
|
||||
"location": self.location,
|
||||
"path": self.batch_path,
|
||||
"engineer": self.engineer,
|
||||
"format": self.format,
|
||||
"counts": {"images": self.image_count, "labels": self.label_count},
|
||||
"has_meta": self.has_meta,
|
||||
"next_cli": self.next_cli,
|
||||
"scope_key": self.scope_key,
|
||||
"domain": self.domain,
|
||||
"domain_label": self.domain_label,
|
||||
"task_label": self.task_label,
|
||||
"mode_label": self.mode_label,
|
||||
"labeling_profile": self.labeling_profile,
|
||||
"export_default": self.export_default,
|
||||
"ml_adapter": self.ml_adapter,
|
||||
"campaign_id": self.campaign_id,
|
||||
"registry_only": self.registry_only,
|
||||
"indexed_at": self.indexed_at.isoformat() if self.indexed_at else None,
|
||||
}
|
||||
|
||||
|
||||
class FeishuBitableLink(Base):
|
||||
"""HSAP 批次与飞书多维表格行的对应关系。"""
|
||||
|
||||
|
||||
230
platform/as_platform/deliveries/scan.py
Normal file
@@ -0,0 +1,230 @@
|
||||
"""扫描 inbox / 数据湖目录,与批次台账对齐。"""
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from as_platform.data.core import load_wf, proj_root, register_batch
|
||||
from as_platform.db.engine import session_scope
|
||||
from as_platform.db.models import BatchDelivery, BatchIndex, User
|
||||
from as_platform.deliveries.service import _new_delivery_id, _normalize_task
|
||||
|
||||
|
||||
def _utcnow() -> datetime:
|
||||
return datetime.now(timezone.utc)
|
||||
|
||||
|
||||
def _dir_mtime_iso(path: Path) -> str | None:
|
||||
try:
|
||||
ts = path.stat().st_mtime
|
||||
return datetime.fromtimestamp(ts, tz=timezone.utc).strftime("%Y-%m-%d")
|
||||
except OSError:
|
||||
return None
|
||||
|
||||
|
||||
def _scan_project_inbox(project: str, wf: dict | None = None) -> list[dict[str, Any]]:
|
||||
from as_platform.data.batch import count_images, count_label_files, dms_has_labels
|
||||
|
||||
wf = wf or load_wf()
|
||||
root = proj_root(wf, project)
|
||||
inbox = root / "inbox"
|
||||
if not inbox.is_dir():
|
||||
return []
|
||||
|
||||
with session_scope() as db:
|
||||
deliveries = {
|
||||
(r.project, r.task or "", r.mode or "", r.batch_name): r
|
||||
for r in db.query(BatchDelivery).filter(BatchDelivery.project == project).all()
|
||||
}
|
||||
indexed = {
|
||||
(r.task or "", r.batch)
|
||||
for r in db.query(BatchIndex).filter(
|
||||
BatchIndex.project == project,
|
||||
BatchIndex.archived.is_(False),
|
||||
).all()
|
||||
}
|
||||
|
||||
items: list[dict[str, Any]] = []
|
||||
for task_dir in sorted(inbox.iterdir()):
|
||||
if not task_dir.is_dir():
|
||||
continue
|
||||
for batch_dir in sorted(task_dir.iterdir()):
|
||||
if not batch_dir.is_dir():
|
||||
continue
|
||||
task_name = task_dir.name
|
||||
batch_name = batch_dir.name
|
||||
img_count = count_images(batch_dir)
|
||||
if not img_count and (batch_dir / "images").is_dir():
|
||||
img_count = count_images(batch_dir / "images")
|
||||
lbl_count = count_label_files(batch_dir / "labels") if (batch_dir / "labels").is_dir() else 0
|
||||
has_labels = lbl_count > 0 or dms_has_labels(batch_dir)
|
||||
stage_hint = "returned" if has_labels and lbl_count > 0 else "raw_pool"
|
||||
|
||||
key = (project, task_name, "", batch_name)
|
||||
delivery = deliveries.get(key)
|
||||
in_index = (task_name, batch_name) in indexed
|
||||
|
||||
items.append({
|
||||
"project": project,
|
||||
"task": task_name,
|
||||
"mode": None,
|
||||
"batch": batch_name,
|
||||
"batch_name": batch_name,
|
||||
"path": str(batch_dir),
|
||||
"data_path": str(batch_dir),
|
||||
"images": img_count,
|
||||
"labels": lbl_count,
|
||||
"has_labels": has_labels,
|
||||
"stage_hint": stage_hint,
|
||||
"source_type": "inbox_scan",
|
||||
"delivery_id": delivery.id if delivery else None,
|
||||
"delivery_status": delivery.status if delivery else None,
|
||||
"in_ledger": delivery is not None,
|
||||
"in_workbench": in_index,
|
||||
"collection_start": delivery.collection_start if delivery else _dir_mtime_iso(batch_dir),
|
||||
"collection_end": delivery.collection_end if delivery else None,
|
||||
"created_at": delivery.created_at.isoformat() if delivery and delivery.created_at else None,
|
||||
"needs_ledger": delivery is None,
|
||||
"needs_workbench": not in_index,
|
||||
})
|
||||
return items
|
||||
|
||||
|
||||
def scan_delivery_sources(*, projects: list[str] | None = None) -> dict[str, Any]:
|
||||
"""扫描 inbox,返回与台账、工作台对齐状态。"""
|
||||
projs = projects or ["dms", "adas", "lane"]
|
||||
wf = load_wf()
|
||||
items: list[dict[str, Any]] = []
|
||||
for p in projs:
|
||||
items.extend(_scan_project_inbox(p, wf))
|
||||
needs_ledger = sum(1 for i in items if i.get("needs_ledger"))
|
||||
needs_workbench = sum(1 for i in items if i.get("needs_workbench"))
|
||||
return {
|
||||
"items": items,
|
||||
"count": len(items),
|
||||
"needs_ledger": needs_ledger,
|
||||
"needs_workbench": needs_workbench,
|
||||
"scanned_at": _utcnow().isoformat(),
|
||||
}
|
||||
|
||||
|
||||
def register_scanned_to_ledger(
|
||||
items: list[dict[str, Any]],
|
||||
user: User,
|
||||
*,
|
||||
sync_workbench: bool = True,
|
||||
) -> dict[str, Any]:
|
||||
"""将扫描结果登记到台账;已在 inbox 的批次直接标为 in_lake 并同步工作台。"""
|
||||
created = 0
|
||||
updated = 0
|
||||
synced = 0
|
||||
out_items: list[dict[str, Any]] = []
|
||||
|
||||
for raw in items:
|
||||
project = (raw.get("project") or "dms").strip()
|
||||
task = _normalize_task(project, raw.get("task"))
|
||||
mode = (raw.get("mode") or "").strip() or None
|
||||
batch_name = (raw.get("batch_name") or raw.get("batch") or "").strip()
|
||||
data_path = (raw.get("data_path") or raw.get("path") or "").strip()
|
||||
if not batch_name or not data_path:
|
||||
continue
|
||||
if not Path(data_path).is_dir():
|
||||
continue
|
||||
|
||||
stage_hint = raw.get("stage_hint") or "raw_pool"
|
||||
collection_start = (raw.get("collection_start") or "").strip() or _dir_mtime_iso(Path(data_path))
|
||||
collection_end = (raw.get("collection_end") or "").strip() or None
|
||||
estimated = raw.get("images")
|
||||
if estimated is None:
|
||||
estimated = raw.get("estimated_count")
|
||||
|
||||
with session_scope() as db:
|
||||
rec = (
|
||||
db.query(BatchDelivery)
|
||||
.filter_by(project=project, task=task, mode=mode, batch_name=batch_name)
|
||||
.first()
|
||||
)
|
||||
if not rec:
|
||||
rec = BatchDelivery(
|
||||
id=_new_delivery_id(),
|
||||
project=project,
|
||||
task=task,
|
||||
mode=mode,
|
||||
batch_name=batch_name,
|
||||
source_type=(raw.get("source_type") or "inbox_scan"),
|
||||
collection_start=collection_start,
|
||||
collection_end=collection_end,
|
||||
data_path=data_path,
|
||||
estimated_count=int(estimated) if estimated not in (None, "") else None,
|
||||
status="in_lake",
|
||||
inbox_path=data_path,
|
||||
owner_user_id=user.id,
|
||||
owner_name=user.name,
|
||||
submitted_by_user_id=user.id,
|
||||
submitted_by_name=user.name,
|
||||
)
|
||||
db.add(rec)
|
||||
created += 1
|
||||
else:
|
||||
if rec.status in ("draft", "rejected", "ingest_failed"):
|
||||
rec.status = "in_lake"
|
||||
if not rec.inbox_path:
|
||||
rec.inbox_path = data_path
|
||||
if not rec.data_path:
|
||||
rec.data_path = data_path
|
||||
if collection_start and not rec.collection_start:
|
||||
rec.collection_start = collection_start
|
||||
if estimated not in (None, "") and not rec.estimated_count:
|
||||
rec.estimated_count = int(estimated)
|
||||
if not rec.source_type:
|
||||
rec.source_type = "inbox_scan"
|
||||
rec.updated_at = _utcnow()
|
||||
updated += 1
|
||||
db.flush()
|
||||
out_items.append(rec.to_dict())
|
||||
|
||||
if sync_workbench and stage_hint in ("raw_pool", "returned"):
|
||||
try:
|
||||
register_batch(
|
||||
None,
|
||||
project,
|
||||
task,
|
||||
batch_name,
|
||||
stage=stage_hint,
|
||||
location="inbox",
|
||||
)
|
||||
synced += 1
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
return {
|
||||
"ok": True,
|
||||
"created": created,
|
||||
"updated": updated,
|
||||
"synced_workbench": synced,
|
||||
"items": out_items,
|
||||
}
|
||||
|
||||
|
||||
def bridge_delivery_to_workbench(delivery_id: str) -> dict[str, Any]:
|
||||
"""台账 in_lake 后同步到送标工作台索引。"""
|
||||
with session_scope() as db:
|
||||
rec = db.get(BatchDelivery, delivery_id)
|
||||
if not rec:
|
||||
raise ValueError("送标申请不存在")
|
||||
if rec.status != "in_lake":
|
||||
raise ValueError(f"当前状态不可同步工作台: {rec.status}")
|
||||
project = rec.project
|
||||
task = rec.task
|
||||
batch_name = rec.batch_name
|
||||
inbox_path = rec.inbox_path or rec.data_path
|
||||
|
||||
stage = "raw_pool"
|
||||
if inbox_path:
|
||||
labels_dir = Path(inbox_path) / "labels"
|
||||
if labels_dir.is_dir() and any(labels_dir.iterdir()):
|
||||
stage = "returned"
|
||||
|
||||
result = register_batch(None, project, task, batch_name, stage=stage, location="inbox")
|
||||
return {"ok": True, "delivery_id": delivery_id, "batch": result.get("batch")}
|
||||
@@ -24,26 +24,43 @@ def _new_delivery_id() -> str:
|
||||
|
||||
|
||||
def _normalize_task(project: str, task: str | None) -> str | None:
|
||||
if project == "dms":
|
||||
return (task or "").strip() or None
|
||||
return None
|
||||
t = (task or "").strip() or None
|
||||
if project == "adas":
|
||||
return t or "cuboid_7cls"
|
||||
if project == "lane":
|
||||
return t
|
||||
return t
|
||||
|
||||
|
||||
def _enrich_delivery_dict(db, rec: BatchDelivery) -> dict[str, Any]:
|
||||
def _enrich_delivery_dict(db, rec: BatchDelivery, *, ap_map: dict | None = None, job_map: dict | None = None) -> dict[str, Any]:
|
||||
d = rec.to_dict()
|
||||
d["submitted_by"] = rec.submitted_by_name
|
||||
if rec.approval_id:
|
||||
ap = db.get(Approval, rec.approval_id)
|
||||
ap = (ap_map or {}).get(rec.approval_id) if ap_map is not None else db.get(Approval, rec.approval_id)
|
||||
if ap:
|
||||
d["approval_status"] = ap.status
|
||||
d["job_id"] = ap.job_id
|
||||
if ap.job_id:
|
||||
job = db.get(Job, ap.job_id)
|
||||
job = (job_map or {}).get(ap.job_id) if job_map is not None else db.get(Job, ap.job_id)
|
||||
if job:
|
||||
d["job_status"] = job.status
|
||||
return d
|
||||
|
||||
|
||||
def _bulk_approval_maps(db, rows: list[BatchDelivery]) -> tuple[dict[str, Approval], dict[str, Job]]:
|
||||
approval_ids = [r.approval_id for r in rows if r.approval_id]
|
||||
ap_map: dict[str, Approval] = {}
|
||||
job_map: dict[str, Job] = {}
|
||||
if approval_ids:
|
||||
aps = db.query(Approval).filter(Approval.id.in_(approval_ids)).all()
|
||||
ap_map = {a.id: a for a in aps}
|
||||
job_ids = [a.job_id for a in aps if a.job_id]
|
||||
if job_ids:
|
||||
jobs = db.query(Job).filter(Job.id.in_(job_ids)).all()
|
||||
job_map = {j.id: j for j in jobs}
|
||||
return ap_map, job_map
|
||||
|
||||
|
||||
def list_deliveries(
|
||||
*,
|
||||
status: str | None = None,
|
||||
@@ -65,8 +82,9 @@ def list_deliveries(
|
||||
q = q.filter(BatchDelivery.status.in_(("draft", "rejected", "ingest_failed")))
|
||||
total = q.count()
|
||||
rows = q.offset(max(0, offset)).limit(max(1, limit)).all()
|
||||
ap_map, job_map = _bulk_approval_maps(db, rows)
|
||||
return {
|
||||
"items": [_enrich_delivery_dict(db, r) for r in rows],
|
||||
"items": [_enrich_delivery_dict(db, r, ap_map=ap_map, job_map=job_map) for r in rows],
|
||||
"total": total,
|
||||
"offset": offset,
|
||||
"limit": limit,
|
||||
|
||||
@@ -377,9 +377,14 @@ def get_live_fleet(db: Session) -> dict[str, Any]:
|
||||
items = []
|
||||
for v in vehicles:
|
||||
active = get_active_run(db, v.id)
|
||||
row = v.to_dict()
|
||||
row["vehicle_id"] = row["id"]
|
||||
row["lat"] = row.get("last_lat")
|
||||
row["lng"] = row.get("last_lng")
|
||||
row["speed_kmh"] = row.get("last_speed_kmh")
|
||||
items.append(
|
||||
{
|
||||
**v.to_dict(),
|
||||
**row,
|
||||
"active_run_id": active.id if active else None,
|
||||
"active_mileage_km": active.mileage_km if active else None,
|
||||
"active_run_no": active.run_no if active else None,
|
||||
|
||||
@@ -35,6 +35,15 @@ def validate_delivery_fields(
|
||||
return f"数据路径不存在: {path}"
|
||||
if project == "dms" and p.name in ("train", "val", "test") and p.parent.name == "images":
|
||||
return f"请填批次根目录(例如 {p.parent.parent}),不要填到 images/train"
|
||||
if project == "adas" and task == "det_7cls":
|
||||
if "/inbox/det_7cls/" not in str(p).replace("\\", "/"):
|
||||
return "ADAS 2D 须落在 adas/inbox/det_7cls/{批次}(project=adas, task=det_7cls)"
|
||||
if project == "adas" and task in (None, "", "cuboid_7cls"):
|
||||
if "/inbox/cuboid_7cls/" not in str(p).replace("\\", "/"):
|
||||
return "ADAS 3D 须落在 adas/inbox/cuboid_7cls/{批次}(project=adas, task=cuboid_7cls)"
|
||||
if project == "dms" and task == "adas":
|
||||
if "/inbox/adas/" not in str(p).replace("\\", "/"):
|
||||
return "旧版 ADAS 2D 路径 dms/inbox/adas/{批次};新数据请用 adas/inbox/det_7cls/"
|
||||
return None
|
||||
|
||||
|
||||
@@ -59,7 +68,7 @@ def ingest_from_directory(
|
||||
raise ValueError(err)
|
||||
|
||||
src = Path(data_path.strip())
|
||||
task_eff = task if project == "dms" else None
|
||||
task_eff = task if project in ("dms", "adas") else None
|
||||
cand = create_directory_candidate(
|
||||
project=project,
|
||||
task=task_eff,
|
||||
@@ -117,4 +126,11 @@ def run_delivery_ingest(delivery_id: str) -> dict[str, Any]:
|
||||
rec.error_message = None
|
||||
db.flush()
|
||||
|
||||
try:
|
||||
from as_platform.deliveries.scan import bridge_delivery_to_workbench
|
||||
|
||||
bridge_delivery_to_workbench(delivery_id)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
return result
|
||||
|
||||
294
platform/as_platform/labeling/batch_index.py
Normal file
@@ -0,0 +1,294 @@
|
||||
"""批次索引:扫盘结果落库,列表页只查 DB(<200ms)。"""
|
||||
from __future__ import annotations
|
||||
|
||||
import time
|
||||
from datetime import datetime, timezone
|
||||
from typing import Any
|
||||
|
||||
from as_platform.data.core import get_pending_report, load_wf
|
||||
from as_platform.db.engine import session_scope
|
||||
from as_platform.db.models import BatchIndex, LabelingCampaign
|
||||
from as_platform.labeling.scope import enrich_batch_labels, load_dms_registry
|
||||
from as_platform.labeling.stage import STAGE_ALIASES, effective_stage
|
||||
|
||||
|
||||
def _utcnow() -> datetime:
|
||||
return datetime.now(timezone.utc)
|
||||
|
||||
|
||||
def campaign_id_for_row(project: str, task: str, mode: str | None, batch: str, location: str) -> str:
|
||||
from as_platform.labeling.service import _campaign_id
|
||||
|
||||
return _campaign_id(project, task, mode, batch, location)
|
||||
|
||||
|
||||
def _batch_dict_to_fields(b: dict[str, Any], reg: dict) -> dict[str, Any] | None:
|
||||
if b.get("registry_only") and not b.get("batch"):
|
||||
return None
|
||||
row = enrich_batch_labels(b, reg)
|
||||
if row.get("registry_only"):
|
||||
pass
|
||||
raw_stage = row.get("stage")
|
||||
eff = effective_stage(raw_stage) or raw_stage or "raw_pool"
|
||||
allowed = (
|
||||
"raw_pool", "out_for_labeling", "returned", "labeling_submitted",
|
||||
"in_review", "review_approved", "review_rejected", "ingested",
|
||||
)
|
||||
if eff not in allowed and raw_stage not in allowed:
|
||||
return None
|
||||
project = row.get("project") or "dms"
|
||||
task = row.get("task") or ""
|
||||
mode = row.get("mode")
|
||||
batch = row.get("batch") or ""
|
||||
location = row.get("location") or "inbox"
|
||||
counts = row.get("counts") or {}
|
||||
cid = campaign_id_for_row(project, task, mode, batch, location)
|
||||
return {
|
||||
"campaign_id": cid,
|
||||
"project": project,
|
||||
"task": task or None,
|
||||
"mode": mode,
|
||||
"batch": batch,
|
||||
"pack": row.get("pack"),
|
||||
"location": location,
|
||||
"stage": eff,
|
||||
"batch_path": row.get("path"),
|
||||
"scope_key": row.get("scope_key"),
|
||||
"engineer": row.get("engineer"),
|
||||
"format": row.get("format"),
|
||||
"image_count": int(counts.get("images") or 0),
|
||||
"label_count": int(counts.get("labels") or 0),
|
||||
"has_meta": bool(row.get("has_meta")),
|
||||
"registry_only": bool(row.get("registry_only")),
|
||||
"next_cli": row.get("next_cli"),
|
||||
"domain": row.get("domain"),
|
||||
"domain_label": row.get("domain_label"),
|
||||
"task_label": row.get("task_label"),
|
||||
"mode_label": row.get("mode_label"),
|
||||
"labeling_profile": row.get("labeling_profile"),
|
||||
"export_default": row.get("export_default"),
|
||||
"ml_adapter": row.get("ml_adapter"),
|
||||
"indexed_at": _utcnow(),
|
||||
}
|
||||
|
||||
|
||||
def upsert_batch_dict(b: dict[str, Any], *, reg: dict | None = None) -> str | None:
|
||||
"""登记/单批更新时写入索引。"""
|
||||
reg = reg or load_dms_registry()
|
||||
fields = _batch_dict_to_fields(b, reg)
|
||||
if not fields:
|
||||
return None
|
||||
cid = fields["campaign_id"]
|
||||
fields["archived"] = False
|
||||
with session_scope() as db:
|
||||
rec = db.get(BatchIndex, cid)
|
||||
if rec:
|
||||
for k, v in fields.items():
|
||||
setattr(rec, k, v)
|
||||
else:
|
||||
db.add(BatchIndex(**fields))
|
||||
db.commit()
|
||||
return cid
|
||||
|
||||
|
||||
def archive_batch(campaign_id: str) -> dict[str, Any]:
|
||||
"""软删除:仅从工作台隐藏,不删磁盘数据。"""
|
||||
with session_scope() as db:
|
||||
rec = db.get(BatchIndex, campaign_id)
|
||||
if not rec:
|
||||
raise FileNotFoundError(campaign_id)
|
||||
if rec.archived:
|
||||
return {"ok": True, "campaign_id": campaign_id, "already_archived": True}
|
||||
if rec.stage != "raw_pool":
|
||||
raise ValueError("仅「待送标」批次可移除")
|
||||
camp = db.get(LabelingCampaign, campaign_id)
|
||||
if camp and camp.status not in ("not_opened", ""):
|
||||
raise ValueError("已开标的批次不可移除,请先在标注进度中处理")
|
||||
rec.archived = True
|
||||
rec.indexed_at = _utcnow()
|
||||
db.commit()
|
||||
return {"ok": True, "campaign_id": campaign_id}
|
||||
|
||||
|
||||
def sync_index_stage(campaign_id: str, stage: str) -> None:
|
||||
eff = effective_stage(stage) or stage
|
||||
with session_scope() as db:
|
||||
rec = db.get(BatchIndex, campaign_id)
|
||||
if rec:
|
||||
rec.stage = eff
|
||||
rec.indexed_at = _utcnow()
|
||||
db.commit()
|
||||
|
||||
|
||||
def index_count() -> int:
|
||||
with session_scope() as db:
|
||||
return db.query(BatchIndex).count()
|
||||
|
||||
|
||||
def index_is_empty() -> bool:
|
||||
return index_count() == 0
|
||||
|
||||
|
||||
def get_batch_by_campaign_id(campaign_id: str) -> dict[str, Any] | None:
|
||||
with session_scope() as db:
|
||||
rec = db.get(BatchIndex, campaign_id)
|
||||
return rec.to_list_row() if rec else None
|
||||
|
||||
|
||||
def rebuild_batch_index(wf: dict | None = None) -> dict[str, Any]:
|
||||
"""全量扫盘并重建索引(刷新/首次加载/登记后批量同步)。"""
|
||||
from as_platform.labeling.service import _registry_fallback_batches
|
||||
|
||||
t0 = time.perf_counter()
|
||||
wf = wf or load_wf()
|
||||
reg = load_dms_registry()
|
||||
report = get_pending_report(wf)
|
||||
candidates: list[dict[str, Any]] = list(report.get("batches") or [])
|
||||
candidates.extend(_registry_fallback_batches(wf, reg))
|
||||
|
||||
seen: set[str] = set()
|
||||
upserted = 0
|
||||
with session_scope() as db:
|
||||
for b in candidates:
|
||||
fields = _batch_dict_to_fields(b, reg)
|
||||
if not fields:
|
||||
continue
|
||||
cid = fields["campaign_id"]
|
||||
seen.add(cid)
|
||||
rec = db.get(BatchIndex, cid)
|
||||
if rec and rec.archived:
|
||||
continue
|
||||
if rec:
|
||||
for k, v in fields.items():
|
||||
setattr(rec, k, v)
|
||||
else:
|
||||
db.add(BatchIndex(**fields))
|
||||
upserted += 1
|
||||
|
||||
if seen:
|
||||
db.query(BatchIndex).filter(
|
||||
BatchIndex.campaign_id.notin_(seen),
|
||||
BatchIndex.archived.is_(False),
|
||||
).delete(synchronize_session=False)
|
||||
db.commit()
|
||||
|
||||
elapsed_ms = round((time.perf_counter() - t0) * 1000)
|
||||
return {
|
||||
"ok": True,
|
||||
"count": upserted,
|
||||
"elapsed_ms": elapsed_ms,
|
||||
"updated_at": report.get("updated_at"),
|
||||
}
|
||||
|
||||
|
||||
def _expand_stage_filters_for_sql(stage_filters: list[str]) -> list[str]:
|
||||
"""将筛选阶段展开为索引表中的实际 stage 值(含 review_approved 等别名)。"""
|
||||
expanded: set[str] = set()
|
||||
for sf in stage_filters:
|
||||
expanded.add(sf)
|
||||
for alias, canonical in STAGE_ALIASES.items():
|
||||
if canonical == sf:
|
||||
expanded.add(alias)
|
||||
return list(expanded)
|
||||
|
||||
|
||||
def _index_query(db, *, stage_filters: list[str], q: str | None):
|
||||
from sqlalchemy import func, or_
|
||||
|
||||
from as_platform.config import IS_POSTGRES
|
||||
|
||||
query = db.query(BatchIndex).filter(
|
||||
BatchIndex.registry_only.is_(False),
|
||||
BatchIndex.archived.is_(False),
|
||||
)
|
||||
if stage_filters:
|
||||
query = query.filter(BatchIndex.stage.in_(_expand_stage_filters_for_sql(stage_filters)))
|
||||
text = (q or "").strip()
|
||||
if text:
|
||||
pattern = f"%{text}%"
|
||||
if IS_POSTGRES:
|
||||
query = query.filter(
|
||||
or_(
|
||||
BatchIndex.batch.ilike(pattern),
|
||||
BatchIndex.task.ilike(pattern),
|
||||
BatchIndex.project.ilike(pattern),
|
||||
)
|
||||
)
|
||||
else:
|
||||
lp = pattern.lower()
|
||||
query = query.filter(
|
||||
or_(
|
||||
func.lower(BatchIndex.batch).like(lp),
|
||||
func.lower(func.coalesce(BatchIndex.task, "")).like(lp),
|
||||
func.lower(BatchIndex.project).like(lp),
|
||||
)
|
||||
)
|
||||
return query
|
||||
|
||||
|
||||
def list_batches_from_index(
|
||||
*,
|
||||
stage: str | None = None,
|
||||
stages: list[str] | None = None,
|
||||
offset: int = 0,
|
||||
limit: int = 20,
|
||||
q: str | None = None,
|
||||
) -> dict[str, Any]:
|
||||
stage_filters = [s.strip() for s in (stages or []) if s and s.strip()]
|
||||
if stage and stage not in stage_filters:
|
||||
stage_filters.append(stage)
|
||||
|
||||
with session_scope() as db:
|
||||
base = _index_query(db, stage_filters=stage_filters, q=q)
|
||||
total = base.count()
|
||||
recs = (
|
||||
base.order_by(BatchIndex.indexed_at.desc())
|
||||
.offset(max(0, offset))
|
||||
.limit(max(1, limit))
|
||||
.all()
|
||||
)
|
||||
page = [rec.to_list_row() for rec in recs]
|
||||
latest_indexed = max((rec.indexed_at for rec in recs), default=None)
|
||||
|
||||
cids = [r["campaign_id"] for r in page if r.get("campaign_id")]
|
||||
|
||||
camp_map: dict[str, dict[str, Any]] = {}
|
||||
if cids:
|
||||
with session_scope() as db:
|
||||
camps = db.query(LabelingCampaign).filter(LabelingCampaign.id.in_(cids)).all()
|
||||
for c in camps:
|
||||
camp_map[c.id] = {
|
||||
"status": c.status,
|
||||
"assigned_to_user_id": c.assigned_to_user_id,
|
||||
"assigned_to_name": c.assigned_to_name,
|
||||
}
|
||||
|
||||
progress_stages = frozenset({"out_for_labeling", "in_progress"})
|
||||
out: list[dict[str, Any]] = []
|
||||
for row in page:
|
||||
cid = row.get("campaign_id")
|
||||
camp = camp_map.get(cid) if cid else None
|
||||
status = camp["status"] if camp else "not_opened"
|
||||
if camp:
|
||||
row["assigned_to_user_id"] = camp["assigned_to_user_id"]
|
||||
row["assigned_to_name"] = camp["assigned_to_name"]
|
||||
row["campaign_status"] = status
|
||||
eff_stage = row.get("stage") or ""
|
||||
if camp and (status in progress_stages or eff_stage in progress_stages):
|
||||
try:
|
||||
from as_platform.labeling.progress import campaign_progress_summary
|
||||
|
||||
row.update(campaign_progress_summary(cid))
|
||||
except Exception:
|
||||
row.update({"total_tasks": 0, "completed_tasks": 0, "assigned_tasks": 0})
|
||||
out.append(row)
|
||||
|
||||
updated_at = latest_indexed.isoformat() if latest_indexed else None
|
||||
return {
|
||||
"items": out,
|
||||
"total": total,
|
||||
"offset": offset,
|
||||
"limit": limit,
|
||||
"updated_at": updated_at,
|
||||
"source": "index",
|
||||
}
|
||||
@@ -70,6 +70,12 @@ def update_campaign_batch_meta_stage(camp: LabelingCampaign, stage: str) -> bool
|
||||
if camp.mode:
|
||||
meta.setdefault("mode", camp.mode)
|
||||
write_meta(batch_dir, meta)
|
||||
try:
|
||||
from as_platform.labeling.batch_index import sync_index_stage
|
||||
|
||||
sync_index_stage(camp.id, stage)
|
||||
except Exception:
|
||||
pass
|
||||
return True
|
||||
|
||||
|
||||
|
||||
@@ -107,7 +107,10 @@ def _labels_from_registry_profile(project: str, task: str, mode: str | None) ->
|
||||
prof = (load_labeling_registry().get("profiles") or {}).get(pk) or {}
|
||||
cvat_names = prof.get("cvat_labels")
|
||||
if cvat_names:
|
||||
return [{"name": str(n), "type": "cuboid"} for n in cvat_names]
|
||||
label_type = prof.get("cvat_label_type") or (
|
||||
"cuboid" if project == "adas" and task == "cuboid_7cls" else "rectangle"
|
||||
)
|
||||
return [{"name": str(n), "type": label_type} for n in cvat_names]
|
||||
return None
|
||||
|
||||
|
||||
@@ -126,6 +129,10 @@ def build_cvat_labels(
|
||||
if project == "adas":
|
||||
if task == "cuboid_7cls":
|
||||
return ADAS_CUBOID_7CLS_LABELS
|
||||
if task == "det_7cls":
|
||||
return [_rect_label(n) for n in [
|
||||
"pedestrian", "car", "truck", "bus", "motorcycle", "tricycle", "traffic cone",
|
||||
]]
|
||||
return ADAS_CUBOID_7CLS_LABELS
|
||||
|
||||
if project == "lane":
|
||||
@@ -145,4 +152,8 @@ def resolve_annotation_types(project: str, task: str | None = None, mode: str |
|
||||
}
|
||||
if project == "adas" and task == "cuboid_7cls":
|
||||
return ["cuboid"]
|
||||
if project == "adas" and task == "det_7cls":
|
||||
return ["bbox"]
|
||||
if project == "dms" and task == "adas":
|
||||
return ["bbox"]
|
||||
return mapping.get(project, ["bbox"])
|
||||
|
||||
@@ -108,66 +108,23 @@ def _registry_fallback_batches(wf: dict, reg: dict) -> list[dict[str, Any]]:
|
||||
def list_labeling_batches(
|
||||
*,
|
||||
stage: str | None = None,
|
||||
stages: list[str] | None = None,
|
||||
offset: int = 0,
|
||||
limit: int = 20,
|
||||
refresh: bool = False,
|
||||
q: str | None = None,
|
||||
) -> dict[str, Any]:
|
||||
wf = load_wf()
|
||||
report = get_pending_report(wf)
|
||||
reg = load_dms_registry()
|
||||
items: list[dict[str, Any]] = []
|
||||
seen: set[str] = set()
|
||||
allowed_stages = ("raw_pool", "out_for_labeling", "returned", "labeling_submitted", "in_review", "review_approved", "review_rejected")
|
||||
from as_platform.labeling.batch_index import (
|
||||
index_is_empty,
|
||||
list_batches_from_index,
|
||||
rebuild_batch_index,
|
||||
)
|
||||
|
||||
def _append(b: dict[str, Any]) -> None:
|
||||
if b.get("registry_only"):
|
||||
return
|
||||
raw_stage = b.get("stage")
|
||||
eff = effective_stage(raw_stage)
|
||||
if stage and not matches_stage_filter(raw_stage, stage):
|
||||
return
|
||||
if eff not in allowed_stages and raw_stage not in allowed_stages:
|
||||
return
|
||||
row = enrich_batch_labels(b, reg)
|
||||
row["stage"] = eff or raw_stage
|
||||
cid = _campaign_id(
|
||||
row["project"], row.get("task") or "", row.get("mode"), row["batch"], row.get("location") or "inbox"
|
||||
)
|
||||
key = f"{cid}"
|
||||
if key in seen:
|
||||
return
|
||||
seen.add(key)
|
||||
with session_scope() as db:
|
||||
camp = db.get(LabelingCampaign, cid)
|
||||
status = camp.status if camp else "not_opened"
|
||||
if camp:
|
||||
row["assigned_to_user_id"] = camp.assigned_to_user_id
|
||||
row["assigned_to_name"] = camp.assigned_to_name
|
||||
row["campaign_id"] = cid
|
||||
row["campaign_status"] = status
|
||||
if camp and status in ("in_progress", "labeling_submitted"):
|
||||
try:
|
||||
from as_platform.labeling.progress import campaign_progress_summary
|
||||
|
||||
row.update(campaign_progress_summary(cid))
|
||||
except Exception:
|
||||
row.update({"total_tasks": 0, "completed_tasks": 0, "assigned_tasks": 0})
|
||||
items.append(row)
|
||||
|
||||
for b in report.get("batches", []):
|
||||
_append(b)
|
||||
|
||||
for b in _registry_fallback_batches(wf, reg):
|
||||
_append(b)
|
||||
|
||||
total = len(items)
|
||||
page = items[max(0, offset) : max(0, offset) + max(1, limit)]
|
||||
return {
|
||||
"items": page,
|
||||
"total": total,
|
||||
"offset": offset,
|
||||
"limit": limit,
|
||||
"updated_at": report.get("updated_at"),
|
||||
}
|
||||
if refresh or index_is_empty():
|
||||
rebuild_batch_index()
|
||||
return list_batches_from_index(
|
||||
stage=stage, stages=stages, offset=offset, limit=limit, q=q,
|
||||
)
|
||||
|
||||
|
||||
def open_campaign(
|
||||
@@ -189,13 +146,14 @@ def open_campaign(
|
||||
|
||||
cvat = get_cvat_client()
|
||||
if not cvat.ping():
|
||||
raise ValueError("CVAT 标注引擎不可用,请执行: docker compose -f docker-compose.yml -f docker-compose.cvat.yml up -d")
|
||||
raise ValueError("CVAT 标注引擎不可用,请执行: docker compose up -d")
|
||||
|
||||
cvat_labels = build_cvat_labels(project, task, mode, ann_types)
|
||||
cvat_task = cvat.create_task(name=cid, labels=cvat_labels)
|
||||
cvat_task_id = cvat_task.id
|
||||
cvat_job_url = cvat_task.job_url
|
||||
|
||||
batch_dir = None
|
||||
with session_scope() as db:
|
||||
camp = db.get(LabelingCampaign, cid)
|
||||
if not camp:
|
||||
@@ -241,6 +199,26 @@ def open_campaign(
|
||||
reg = load_dms_registry() if project == "dms" else None
|
||||
row = enrich_batch_labels(out, reg)
|
||||
row["stage"] = "out_for_labeling"
|
||||
try:
|
||||
from as_platform.labeling.batch_index import upsert_batch_dict
|
||||
|
||||
if batch_dir and batch_dir.is_dir():
|
||||
from as_platform.data.batch import enrich_batch
|
||||
|
||||
upsert_batch_dict(
|
||||
enrich_batch(
|
||||
batch_dir,
|
||||
project=project,
|
||||
task=task,
|
||||
pack=pack,
|
||||
batch=batch,
|
||||
location=location,
|
||||
),
|
||||
)
|
||||
else:
|
||||
upsert_batch_dict(row)
|
||||
except Exception:
|
||||
pass
|
||||
return row
|
||||
|
||||
|
||||
@@ -376,23 +354,15 @@ def list_labeling_assignees() -> dict[str, Any]:
|
||||
|
||||
|
||||
def _find_batch_for_campaign_id(campaign_id: str) -> dict[str, Any] | None:
|
||||
"""由确定性 campaign_id 反查 pending / registry 批次行。"""
|
||||
wf = load_wf()
|
||||
reg = load_dms_registry()
|
||||
candidates: list[dict[str, Any]] = []
|
||||
report = get_pending_report(wf)
|
||||
candidates.extend(report.get("batches") or [])
|
||||
candidates.extend(_registry_fallback_batches(wf, reg))
|
||||
for b in candidates:
|
||||
cid = _campaign_id(
|
||||
b.get("project") or "dms",
|
||||
b.get("task") or "",
|
||||
b.get("mode"),
|
||||
b.get("batch") or "",
|
||||
b.get("location") or "inbox",
|
||||
)
|
||||
if cid == campaign_id:
|
||||
return b
|
||||
"""由 campaign_id 反查批次行(优先读索引)。"""
|
||||
from as_platform.labeling.batch_index import get_batch_by_campaign_id, index_is_empty, rebuild_batch_index
|
||||
|
||||
row = get_batch_by_campaign_id(campaign_id)
|
||||
if row:
|
||||
return row
|
||||
if index_is_empty():
|
||||
rebuild_batch_index()
|
||||
return get_batch_by_campaign_id(campaign_id)
|
||||
return None
|
||||
|
||||
|
||||
|
||||
280
platform/as_platform/tests/run_dms_e2e_pipeline.py
Executable file
@@ -0,0 +1,280 @@
|
||||
#!/usr/bin/env python3
|
||||
"""DMS 2 图 E2E:标完后自动 提交→质检→导出→build 入库。"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import hashlib
|
||||
import json
|
||||
import sys
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[3]
|
||||
PLATFORM = ROOT / "platform"
|
||||
if str(PLATFORM) not in sys.path:
|
||||
sys.path.insert(0, str(PLATFORM))
|
||||
if str(ROOT) not in sys.path:
|
||||
sys.path.insert(0, str(ROOT))
|
||||
|
||||
|
||||
def campaign_id(project: str, task: str, mode: str | None, batch: str, location: str = "inbox") -> str:
|
||||
from as_platform.labeling.scope import format_scope_key
|
||||
|
||||
sk = format_scope_key(project, task, mode)
|
||||
raw = f"{sk}:{batch}:{location}"
|
||||
return hashlib.sha256(raw.encode()).hexdigest()[:20]
|
||||
|
||||
|
||||
class ApiClient:
|
||||
def __init__(self, base: str, token: str) -> None:
|
||||
self.base = base.rstrip("/")
|
||||
self.token = token
|
||||
|
||||
def _request(self, method: str, path: str, body: dict | None = None) -> Any:
|
||||
data = json.dumps(body).encode() if body is not None else None
|
||||
req = urllib.request.Request(
|
||||
f"{self.base}{path}",
|
||||
data=data,
|
||||
headers={
|
||||
"Authorization": f"Bearer {self.token}",
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
method=method,
|
||||
)
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=120) as resp:
|
||||
raw = resp.read().decode()
|
||||
return json.loads(raw) if raw else {}
|
||||
except urllib.error.HTTPError as e:
|
||||
detail = e.read().decode()
|
||||
raise RuntimeError(f"{method} {path} -> {e.code}: {detail}") from e
|
||||
|
||||
def get(self, path: str) -> Any:
|
||||
return self._request("GET", path)
|
||||
|
||||
def post(self, path: str, body: dict | None = None) -> Any:
|
||||
return self._request("POST", path, body)
|
||||
|
||||
|
||||
def login(base: str, name: str = "e2e-runner") -> ApiClient:
|
||||
req = urllib.request.Request(
|
||||
f"{base.rstrip('/')}/api/v1/auth/dev/login",
|
||||
data=json.dumps({"name": name}).encode(),
|
||||
headers={"Content-Type": "application/json"},
|
||||
method="POST",
|
||||
)
|
||||
with urllib.request.urlopen(req, timeout=30) as resp:
|
||||
data = json.loads(resp.read().decode())
|
||||
return ApiClient(base, data["access_token"])
|
||||
|
||||
|
||||
def wait_job(api: ApiClient, job_id: str, timeout: int = 180) -> dict[str, Any]:
|
||||
deadline = time.time() + timeout
|
||||
while time.time() < deadline:
|
||||
job = api.get(f"/api/v1/jobs/{job_id}")
|
||||
st = job.get("status")
|
||||
if st in ("succeeded", "failed"):
|
||||
return job
|
||||
time.sleep(1)
|
||||
raise TimeoutError(f"job {job_id} not finished in {timeout}s")
|
||||
|
||||
|
||||
def labeled_count(batch_dir: Path) -> int:
|
||||
ann_dir = batch_dir / "labels" / "ls_annotations"
|
||||
if not ann_dir.is_dir():
|
||||
return 0
|
||||
from as_platform.labeling.progress import _annotation_has_result
|
||||
|
||||
n = 0
|
||||
for p in ann_dir.glob("*.json"):
|
||||
if _annotation_has_result(p):
|
||||
n += 1
|
||||
return n
|
||||
|
||||
|
||||
def batch_stage(batch_dir: Path) -> str:
|
||||
meta = batch_dir / "batch.meta.yaml"
|
||||
if not meta.is_file():
|
||||
return "raw_pool"
|
||||
import yaml
|
||||
|
||||
data = yaml.safe_load(meta.read_text(encoding="utf-8")) or {}
|
||||
return str(data.get("stage") or "raw_pool")
|
||||
|
||||
|
||||
def cmd_info(args: argparse.Namespace) -> None:
|
||||
cid = campaign_id(args.project, args.task, None, args.batch)
|
||||
print(f"campaign_id={cid}")
|
||||
print(f"annotate_url=/labeling/annotate/{cid}")
|
||||
print(f"batch_path=datasets/dms/inbox/{args.task}/{args.batch}")
|
||||
wf_root = ROOT / "datasets" / "dms"
|
||||
batch_dir = wf_root / "inbox" / args.task / args.batch
|
||||
if batch_dir.is_dir():
|
||||
print(f"stage={batch_stage(batch_dir)} labeled={labeled_count(batch_dir)}")
|
||||
try:
|
||||
api = login(args.api)
|
||||
row = next(
|
||||
(i for i in api.get("/api/v1/labeling/batches?limit=100").get("items", [])
|
||||
if i.get("batch") == args.batch and i.get("task") == args.task),
|
||||
None,
|
||||
)
|
||||
if row:
|
||||
print(f"platform_stage={row.get('stage')} status={row.get('campaign_status')}")
|
||||
print(f"progress={row.get('completed_tasks', '?')}/{row.get('total_tasks', '?')}")
|
||||
except Exception as e:
|
||||
print(f"api_skip={e}")
|
||||
|
||||
|
||||
def cmd_setup(args: argparse.Namespace) -> None:
|
||||
api = login(args.api)
|
||||
body = {
|
||||
"project": args.project,
|
||||
"task": args.task,
|
||||
"batch": args.batch,
|
||||
"location": "inbox",
|
||||
}
|
||||
row = api.post("/api/v1/labeling/campaigns/open", body)
|
||||
print(json.dumps({"campaign_id": row.get("id"), "stage": row.get("stage"), "cvat_task_id": row.get("cvat_task_id")}, ensure_ascii=False))
|
||||
|
||||
|
||||
def wait_labels(batch_dir: Path, min_images: int, wait_sec: int) -> None:
|
||||
if labeled_count(batch_dir) >= min_images:
|
||||
return
|
||||
if wait_sec <= 0:
|
||||
raise RuntimeError(
|
||||
f"仅标注 {labeled_count(batch_dir)}/{min_images} 张,请先在平台画框保存,再执行 run 或 run-wait"
|
||||
)
|
||||
print(f"等待标注 {min_images} 张 (最多 {wait_sec}s)...")
|
||||
deadline = time.time() + wait_sec
|
||||
while time.time() < deadline:
|
||||
n = labeled_count(batch_dir)
|
||||
if n >= min_images:
|
||||
print(f"labeled={n}")
|
||||
return
|
||||
time.sleep(3)
|
||||
raise TimeoutError(f"超时:仅 {labeled_count(batch_dir)}/{min_images} 张有标注")
|
||||
|
||||
|
||||
def cmd_run(args: argparse.Namespace) -> None:
|
||||
cid = campaign_id(args.project, args.task, None, args.batch)
|
||||
batch_dir = ROOT / "datasets" / "dms" / "inbox" / args.task / args.batch
|
||||
if not batch_dir.is_dir():
|
||||
raise FileNotFoundError(batch_dir)
|
||||
|
||||
wait_labels(batch_dir, args.min_images, args.wait_label_sec)
|
||||
api = login(args.api)
|
||||
|
||||
print("==> 1. 提交质检")
|
||||
api.post(f"/api/v1/labeling/campaigns/{cid}/submit")
|
||||
row = next(
|
||||
i for i in api.get("/api/v1/labeling/batches?limit=100").get("items", [])
|
||||
if i.get("campaign_id") == cid
|
||||
)
|
||||
assert row.get("stage") == "in_review", row
|
||||
|
||||
print("==> 2. 质检通过 (全部 good)")
|
||||
queue = api.get(f"/api/v1/labeling/campaigns/{cid}/review-queue?limit=50")
|
||||
items = queue.get("items") or []
|
||||
scores = [{"image_path": it["image_path"], "score": "good"} for it in items]
|
||||
res = api.post(
|
||||
f"/api/v1/labeling/campaigns/{cid}/review-submit",
|
||||
{"scores": scores},
|
||||
)
|
||||
print("review", res)
|
||||
row = next(
|
||||
i for i in api.get("/api/v1/labeling/batches?limit=100").get("items", [])
|
||||
if i.get("campaign_id") == cid
|
||||
)
|
||||
assert row.get("stage") == "labeling_submitted", row
|
||||
|
||||
print("==> 3. 执行导出")
|
||||
exp = api.post(f"/api/v1/labeling/campaigns/{cid}/export")
|
||||
job_id = (exp.get("job") or {}).get("id")
|
||||
assert job_id, exp
|
||||
job = wait_job(api, job_id)
|
||||
if job.get("status") != "succeeded":
|
||||
raise RuntimeError(f"export failed: {job}")
|
||||
print("export_job", job.get("result"))
|
||||
|
||||
row = next(
|
||||
i for i in api.get("/api/v1/labeling/batches?limit=100").get("items", [])
|
||||
if i.get("campaign_id") == cid
|
||||
)
|
||||
assert row.get("stage") == "returned", row
|
||||
yolo = list((batch_dir / "labels").rglob("*.txt"))
|
||||
assert yolo, "export 后应有 YOLO txt"
|
||||
|
||||
print("==> 4. 提交 build 审核")
|
||||
appr = api.post(
|
||||
"/api/v1/system/audit/submit-build-batch",
|
||||
{
|
||||
"project": args.project,
|
||||
"task": args.task,
|
||||
"batch": args.batch,
|
||||
"pack": args.pack,
|
||||
"location": "inbox",
|
||||
"note": f"E2E smoke {args.batch}",
|
||||
},
|
||||
)
|
||||
approval_id = appr.get("id")
|
||||
assert approval_id, appr
|
||||
print("approval_id", approval_id)
|
||||
|
||||
print("==> 5. 批准 build")
|
||||
done = api.post(f"/api/v1/system/audit/{approval_id}/approve", {"comment": "e2e auto approve"})
|
||||
build_job_id = done.get("job_id")
|
||||
if build_job_id:
|
||||
bjob = wait_job(api, build_job_id, timeout=300)
|
||||
if bjob.get("status") != "succeeded":
|
||||
raise RuntimeError(f"build failed: {bjob}")
|
||||
print("build_job", bjob.get("result"))
|
||||
|
||||
row = next(
|
||||
i for i in api.get("/api/v1/labeling/batches?limit=100").get("items", [])
|
||||
if i.get("campaign_id") == cid
|
||||
)
|
||||
assert row.get("stage") == "ingested", row
|
||||
assert batch_stage(batch_dir) == "ingested", batch_stage(batch_dir)
|
||||
|
||||
dest = ROOT / "datasets" / "dms" / "packs" / args.pack / args.task / "sources" / args.batch
|
||||
assert dest.is_dir(), f"missing pack source: {dest}"
|
||||
dest_labels = list(dest.rglob("labels/**/*.txt")) + list(dest.rglob("labels/*.txt"))
|
||||
assert dest_labels, f"pack 内应有 labels: {dest}"
|
||||
|
||||
print("DMS_E2E_PIPELINE_OK")
|
||||
print(json.dumps({
|
||||
"campaign_id": cid,
|
||||
"batch": args.batch,
|
||||
"pack": args.pack,
|
||||
"dest": str(dest),
|
||||
"yolo_in_inbox": len(yolo),
|
||||
"stage": row.get("stage"),
|
||||
}, ensure_ascii=False, indent=2))
|
||||
|
||||
|
||||
def main() -> None:
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("command", choices=("setup", "run", "info"))
|
||||
ap.add_argument("--api", default="http://127.0.0.1:8787")
|
||||
ap.add_argument("--project", default="dms")
|
||||
ap.add_argument("--task", default="addw")
|
||||
ap.add_argument("--batch", default="e2e_2img_20260616")
|
||||
ap.add_argument("--pack", default="dms_v1")
|
||||
ap.add_argument("--min-images", type=int, default=2)
|
||||
ap.add_argument("--wait-label-sec", type=int, default=0)
|
||||
ap.add_argument("--skip-files", action="store_true")
|
||||
args = ap.parse_args()
|
||||
|
||||
if args.command == "info":
|
||||
cmd_info(args)
|
||||
elif args.command == "setup":
|
||||
cmd_setup(args)
|
||||
elif args.command == "run":
|
||||
cmd_run(args)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
54
platform/as_platform/tests/test_batch_index.py
Normal file
@@ -0,0 +1,54 @@
|
||||
"""批次索引:列表走 DB,重建走扫盘。"""
|
||||
from __future__ import annotations
|
||||
|
||||
import time
|
||||
|
||||
from as_platform.labeling.batch_index import (
|
||||
index_is_empty,
|
||||
list_batches_from_index,
|
||||
rebuild_batch_index,
|
||||
)
|
||||
from as_platform.labeling.service import list_labeling_batches
|
||||
|
||||
|
||||
def test_rebuild_and_list_from_index():
|
||||
r = rebuild_batch_index()
|
||||
assert r["ok"] is True
|
||||
assert r["count"] >= 0
|
||||
|
||||
t0 = time.perf_counter()
|
||||
out = list_batches_from_index(limit=100)
|
||||
elapsed_ms = (time.perf_counter() - t0) * 1000
|
||||
assert out["source"] == "index"
|
||||
assert "items" in out
|
||||
assert elapsed_ms < 500, f"index list too slow: {elapsed_ms:.0f}ms"
|
||||
|
||||
|
||||
def test_list_labeling_batches_uses_index():
|
||||
if index_is_empty():
|
||||
rebuild_batch_index()
|
||||
t0 = time.perf_counter()
|
||||
out = list_labeling_batches(limit=50)
|
||||
elapsed_ms = (time.perf_counter() - t0) * 1000
|
||||
assert "items" in out
|
||||
assert elapsed_ms < 800, f"list_labeling_batches too slow: {elapsed_ms:.0f}ms"
|
||||
|
||||
|
||||
def test_archive_batch_hides_from_list():
|
||||
from as_platform.db.engine import session_scope
|
||||
from as_platform.db.models import BatchIndex
|
||||
from as_platform.labeling.batch_index import archive_batch, list_batches_from_index
|
||||
|
||||
with session_scope() as db:
|
||||
rec = (
|
||||
db.query(BatchIndex)
|
||||
.filter(BatchIndex.archived.is_(False), BatchIndex.stage == "raw_pool")
|
||||
.first()
|
||||
)
|
||||
if not rec:
|
||||
return
|
||||
cid = rec.campaign_id
|
||||
|
||||
archive_batch(cid)
|
||||
out = list_batches_from_index(stage="raw_pool", limit=500)
|
||||
assert all(r.get("campaign_id") != cid for r in out["items"])
|
||||
51
platform/web/package-lock.json
generated
@@ -8,11 +8,14 @@
|
||||
"name": "hsap-platform-web",
|
||||
"version": "1.1.0",
|
||||
"dependencies": {
|
||||
"leaflet": "^1.9.4",
|
||||
"react": "^18.3.1",
|
||||
"react-dom": "^18.3.1",
|
||||
"react-leaflet": "^4.2.1",
|
||||
"react-router-dom": "^5.3.4"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/leaflet": "^1.9.21",
|
||||
"@types/react": "^18.3.3",
|
||||
"@types/react-dom": "^18.3.0",
|
||||
"@types/react-router-dom": "^5.3.3",
|
||||
@@ -807,6 +810,17 @@
|
||||
"node": ">= 8"
|
||||
}
|
||||
},
|
||||
"node_modules/@react-leaflet/core": {
|
||||
"version": "2.1.0",
|
||||
"resolved": "https://registry.npmjs.org/@react-leaflet/core/-/core-2.1.0.tgz",
|
||||
"integrity": "sha512-Qk7Pfu8BSarKGqILj4x7bCSZ1pjuAPZ+qmRwH5S7mDS91VSbVVsJSrW4qA+GPrro8t69gFYVMWb1Zc4yFmPiVg==",
|
||||
"license": "Hippocratic-2.1",
|
||||
"peerDependencies": {
|
||||
"leaflet": "^1.9.0",
|
||||
"react": "^18.0.0",
|
||||
"react-dom": "^18.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@rolldown/pluginutils": {
|
||||
"version": "1.0.0-beta.27",
|
||||
"resolved": "https://registry.npmjs.org/@rolldown/pluginutils/-/pluginutils-1.0.0-beta.27.tgz",
|
||||
@@ -1216,6 +1230,13 @@
|
||||
"dev": true,
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/@types/geojson": {
|
||||
"version": "7946.0.16",
|
||||
"resolved": "https://registry.npmjs.org/@types/geojson/-/geojson-7946.0.16.tgz",
|
||||
"integrity": "sha512-6C8nqWur3j98U6+lXDfTUWIfgvZU+EumvpHKcYjujKH7woYyLj2sUmff0tRhrqM7BohUw7Pz3ZB1jj2gW9Fvmg==",
|
||||
"dev": true,
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/@types/history": {
|
||||
"version": "4.7.11",
|
||||
"resolved": "https://registry.npmjs.org/@types/history/-/history-4.7.11.tgz",
|
||||
@@ -1223,6 +1244,16 @@
|
||||
"dev": true,
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/@types/leaflet": {
|
||||
"version": "1.9.21",
|
||||
"resolved": "https://registry.npmjs.org/@types/leaflet/-/leaflet-1.9.21.tgz",
|
||||
"integrity": "sha512-TbAd9DaPGSnzp6QvtYngntMZgcRk+igFELwR2N99XZn7RXUdKgsXMR+28bUO0rPsWp8MIu/f47luLIQuSLYv/w==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@types/geojson": "*"
|
||||
}
|
||||
},
|
||||
"node_modules/@types/prop-types": {
|
||||
"version": "15.7.15",
|
||||
"resolved": "https://registry.npmjs.org/@types/prop-types/-/prop-types-15.7.15.tgz",
|
||||
@@ -1898,6 +1929,12 @@
|
||||
"node": ">=6"
|
||||
}
|
||||
},
|
||||
"node_modules/leaflet": {
|
||||
"version": "1.9.4",
|
||||
"resolved": "https://registry.npmjs.org/leaflet/-/leaflet-1.9.4.tgz",
|
||||
"integrity": "sha512-nxS1ynzJOmOlHp+iL3FyWqK89GtNL8U8rvlMOsQdTTssxZwCXh8N2NB3GDQOL+YR3XnWyZAxwQixURb+FA74PA==",
|
||||
"license": "BSD-2-Clause"
|
||||
},
|
||||
"node_modules/lilconfig": {
|
||||
"version": "3.1.3",
|
||||
"resolved": "https://registry.npmjs.org/lilconfig/-/lilconfig-3.1.3.tgz",
|
||||
@@ -2323,6 +2360,20 @@
|
||||
"integrity": "sha512-24e6ynE2H+OKt4kqsOvNd8kBpV65zoxbA4BVsEOB3ARVWQki/DHzaUoC5KuON/BiccDaCCTZBuOcfZs70kR8bQ==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/react-leaflet": {
|
||||
"version": "4.2.1",
|
||||
"resolved": "https://registry.npmjs.org/react-leaflet/-/react-leaflet-4.2.1.tgz",
|
||||
"integrity": "sha512-p9chkvhcKrWn/H/1FFeVSqLdReGwn2qmiobOQGO3BifX+/vV/39qhY8dGqbdcPh1e6jxh/QHriLXr7a4eLFK4Q==",
|
||||
"license": "Hippocratic-2.1",
|
||||
"dependencies": {
|
||||
"@react-leaflet/core": "^2.1.0"
|
||||
},
|
||||
"peerDependencies": {
|
||||
"leaflet": "^1.9.0",
|
||||
"react": "^18.0.0",
|
||||
"react-dom": "^18.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/react-refresh": {
|
||||
"version": "0.17.0",
|
||||
"resolved": "https://registry.npmjs.org/react-refresh/-/react-refresh-0.17.0.tgz",
|
||||
|
||||
@@ -9,11 +9,14 @@
|
||||
"preview": "vite preview"
|
||||
},
|
||||
"dependencies": {
|
||||
"leaflet": "^1.9.4",
|
||||
"react": "^18.3.1",
|
||||
"react-dom": "^18.3.1",
|
||||
"react-leaflet": "^4.2.1",
|
||||
"react-router-dom": "^5.3.4"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/leaflet": "^1.9.21",
|
||||
"@types/react": "^18.3.3",
|
||||
"@types/react-dom": "^18.3.0",
|
||||
"@types/react-router-dom": "^5.3.3",
|
||||
|
||||
BIN
platform/web/public/login-bg.png
Normal file
|
After Width: | Height: | Size: 1.9 MiB |
@@ -30,7 +30,7 @@ const MODULES: ModuleDef[] = [
|
||||
{ to: "/labeling/my-tasks", label: "我的标注", perm: "read:pending" },
|
||||
{ to: "/labeling/workbench", label: "送标工作台", perm: "read:pending" },
|
||||
{ to: "/labeling/campaigns", label: "标注进度", perm: "read:pending" },
|
||||
{ to: "/labeling/review", label: "标注质检", perm: "read:pending" },
|
||||
{ to: "/labeling/review", label: "标注质检", perm: "write:approval_review" },
|
||||
{ to: "/labeling/export", label: "导出与入库", perm: "read:pending" },
|
||||
{ to: "/labeling/deliveries", label: "批次台账", perm: "read:deliveries" },
|
||||
{ to: "/labeling/catalog", label: "数据目录", perm: "read:catalog" },
|
||||
@@ -148,7 +148,8 @@ export const Sidebar: React.FC = () => {
|
||||
{/* Brand */}
|
||||
<div className="sidebar-brand">
|
||||
<NavLink to="/" className="no-underline text-inherit">
|
||||
HSAP <span className="text-gray-400 font-normal text-sm">数据闭环平台</span>
|
||||
HSAP
|
||||
<span className="text-gray-400 font-normal text-[11px]">数据闭环平台</span>
|
||||
</NavLink>
|
||||
</div>
|
||||
|
||||
|
||||
@@ -76,15 +76,35 @@ export const hsapApi = {
|
||||
health: () => fetchJson<{ status: string; database: string }>(`${API_BASE}/api/v1/health`),
|
||||
|
||||
// ── Labeling ──
|
||||
labelingBatches: (opts?: { stage?: string; offset?: number; limit?: number }) => {
|
||||
labelingBatches: (opts?: {
|
||||
stage?: string;
|
||||
stages?: string[];
|
||||
offset?: number;
|
||||
limit?: number;
|
||||
refresh?: boolean;
|
||||
q?: string;
|
||||
}) => {
|
||||
const p = new URLSearchParams();
|
||||
if (opts?.stage) p.set("stage", opts.stage);
|
||||
if (opts?.stages?.length) p.set("stages", opts.stages.join(","));
|
||||
else if (opts?.stage) p.set("stage", opts.stage);
|
||||
if (opts?.offset != null) p.set("offset", String(opts.offset));
|
||||
if (opts?.limit != null) p.set("limit", String(opts.limit));
|
||||
const q = p.toString();
|
||||
return fetchJson<PagedResult<LabelingBatchRow>>(`${API_BASE}/api/v1/labeling/batches${q ? `?${q}` : ""}`);
|
||||
if (opts?.refresh) p.set("refresh", "true");
|
||||
if (opts?.q) p.set("q", opts.q);
|
||||
const qs = p.toString();
|
||||
return fetchJson<PagedResult<LabelingBatchRow> & { source?: string; updated_at?: string }>(
|
||||
`${API_BASE}/api/v1/labeling/batches${qs ? `?${qs}` : ""}`,
|
||||
);
|
||||
},
|
||||
|
||||
rebuildBatchIndex: () =>
|
||||
postJson<{ ok: boolean; count: number; elapsed_ms: number }>(`${API_BASE}/api/v1/labeling/batches/rebuild-index`),
|
||||
|
||||
archiveLabelingBatch: (campaignId: string) =>
|
||||
postJson<{ ok: boolean; campaign_id: string }>(
|
||||
`${API_BASE}/api/v1/labeling/batches/${encodeURIComponent(campaignId)}/archive`,
|
||||
),
|
||||
|
||||
openLabelingCampaign: (body: { project: string; task: string; batch: string; mode?: string | null; pack?: string | null; location?: string }) =>
|
||||
postJson(`${API_BASE}/api/v1/labeling/campaigns/open`, body),
|
||||
|
||||
@@ -165,6 +185,46 @@ export const hsapApi = {
|
||||
labelingExportStats: (campaignId: string) =>
|
||||
fetchJson<Record<string, unknown>>(`${API_BASE}/api/v1/labeling/campaigns/${campaignId}/export-stats`),
|
||||
|
||||
fetchReviewImageBlob: async (campaignId: string, imagePath: string) => {
|
||||
const q = new URLSearchParams({ path: imagePath });
|
||||
const res = await fetch(
|
||||
`${API_BASE}/api/v1/labeling/campaigns/${campaignId}/review-image?${q}`,
|
||||
{ headers: authHeaders(), cache: "no-store" },
|
||||
);
|
||||
if (!res.ok) throw new Error(await res.text());
|
||||
return URL.createObjectURL(await res.blob());
|
||||
},
|
||||
|
||||
reviewProgress: (campaignId: string) =>
|
||||
fetchJson<{
|
||||
good: number; fine: number; bad: number; pending: number;
|
||||
total: number; reviewed: number; pass_rate: number; complete: boolean; stage?: string;
|
||||
}>(`${API_BASE}/api/v1/labeling/campaigns/${campaignId}/review-progress`),
|
||||
|
||||
reviewProgressBatch: (campaignIds: string[]) =>
|
||||
fetchJson<{ items: Record<string, {
|
||||
good: number; fine: number; bad: number; pending: number;
|
||||
total: number; reviewed: number; pass_rate: number; complete: boolean; stage?: string;
|
||||
}> }>(`${API_BASE}/api/v1/labeling/review-progress?campaign_ids=${encodeURIComponent(campaignIds.join(","))}`),
|
||||
|
||||
reviewQueue: (campaignId: string, opts?: { offset?: number; limit?: number }) => {
|
||||
const p = new URLSearchParams();
|
||||
if (opts?.offset != null) p.set("offset", String(opts.offset));
|
||||
if (opts?.limit != null) p.set("limit", String(opts.limit));
|
||||
const q = p.toString();
|
||||
return fetchJson<{
|
||||
items: { id: string; image_path: string; fileName: string; score: string; has_label: boolean }[];
|
||||
total: number;
|
||||
scores: { good: number; fine: number; bad: number; pending: number };
|
||||
}>(`${API_BASE}/api/v1/labeling/campaigns/${campaignId}/review-queue${q ? `?${q}` : ""}`);
|
||||
},
|
||||
|
||||
reviewSubmit: (campaignId: string, scores: { image_path: string; score: string }[]) =>
|
||||
postJson<{ ok: boolean; auto_advanced?: boolean; stage?: string }>(
|
||||
`${API_BASE}/api/v1/labeling/campaigns/${campaignId}/review-submit`,
|
||||
{ scores },
|
||||
),
|
||||
|
||||
cuboidFit: (campaignId: string) =>
|
||||
postJson<Record<string, unknown>>(`${API_BASE}/api/v1/labeling/campaigns/${campaignId}/cuboid-fit`),
|
||||
|
||||
@@ -285,6 +345,25 @@ export const hsapApi = {
|
||||
registerBatch: (body: Record<string, unknown>) => postJson(`${API_BASE}/api/v1/register-batch`, body),
|
||||
|
||||
// ── Deliveries ──
|
||||
scanDeliveries: (projects?: string[]) => {
|
||||
const p = new URLSearchParams();
|
||||
if (projects?.length) p.set("projects", projects.join(","));
|
||||
const qs = p.toString();
|
||||
return fetchJson<{
|
||||
items: Record<string, unknown>[];
|
||||
count: number;
|
||||
needs_ledger: number;
|
||||
needs_workbench: number;
|
||||
scanned_at?: string;
|
||||
}>(`${API_BASE}/api/v1/deliveries/scan${qs ? `?${qs}` : ""}`);
|
||||
},
|
||||
|
||||
registerScannedDeliveries: (items: Record<string, unknown>[], syncWorkbench = true) =>
|
||||
postJson<{ ok: boolean; created: number; updated: number; synced_workbench: number }>(
|
||||
`${API_BASE}/api/v1/deliveries/scan/register`,
|
||||
{ items, sync_workbench: syncWorkbench },
|
||||
),
|
||||
|
||||
listDeliveries: (opts?: { status?: string; mine?: boolean; offset?: number; limit?: number }) => {
|
||||
const p = new URLSearchParams();
|
||||
if (opts?.status) p.set("status", opts.status);
|
||||
@@ -465,7 +544,7 @@ export const hsapApi = {
|
||||
},
|
||||
|
||||
setUserRoles: (userId: number, roles: string[]) =>
|
||||
putJson(`${API_BASE}/api/v1/auth/users/${userId}/roles`, { roles }),
|
||||
putJson(`${API_BASE}/api/v1/auth/users/${userId}/roles`, { role_codes: roles }),
|
||||
|
||||
syncFeishuUsers: () =>
|
||||
postJson<{ ok: boolean; created: number; updated: number; total: number }>(`${API_BASE}/api/v1/system/feishu/sync-users`),
|
||||
|
||||
25
platform/web/src/components/CompactTableShell.tsx
Normal file
@@ -0,0 +1,25 @@
|
||||
import React from "react";
|
||||
|
||||
type ColWidth = string | number;
|
||||
|
||||
type CompactTableShellProps = {
|
||||
children: React.ReactNode;
|
||||
colWidths?: ColWidth[];
|
||||
};
|
||||
|
||||
export const CompactTableShell: React.FC<CompactTableShellProps> = ({ children, colWidths }) => (
|
||||
<div className="rounded-xl border border-gray-200 bg-white overflow-hidden">
|
||||
<div className="overflow-hidden">
|
||||
<table className="table-auto table-fixed w-full min-w-0 [&_th]:py-1.5 [&_th]:px-2 [&_th]:text-center [&_td]:py-1.5 [&_td]:px-2 [&_td]:text-center">
|
||||
{colWidths && colWidths.length > 0 && (
|
||||
<colgroup>
|
||||
{colWidths.map((w, i) => (
|
||||
<col key={i} style={typeof w === "number" ? { width: w } : { width: w }} />
|
||||
))}
|
||||
</colgroup>
|
||||
)}
|
||||
{children}
|
||||
</table>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
23
platform/web/src/components/TruncatedText.tsx
Normal file
@@ -0,0 +1,23 @@
|
||||
import React from "react";
|
||||
|
||||
interface TruncatedTextProps {
|
||||
text: string;
|
||||
className?: string;
|
||||
maxWidthClass?: string;
|
||||
}
|
||||
|
||||
export const TruncatedText: React.FC<TruncatedTextProps> = ({
|
||||
text,
|
||||
className = "",
|
||||
maxWidthClass = "max-w-[12rem]",
|
||||
}) => {
|
||||
if (!text) return <span className={`${className} text-center`}>—</span>;
|
||||
return (
|
||||
<span
|
||||
className={`block truncate cursor-default text-center mx-auto ${maxWidthClass} ${className}`}
|
||||
title={text}
|
||||
>
|
||||
{text}
|
||||
</span>
|
||||
);
|
||||
};
|
||||
@@ -63,7 +63,11 @@ export const StatusBadge: React.FC<{ status: string }> = ({ status }) => {
|
||||
failed: { variant: "danger", label: "失败" },
|
||||
draft: { variant: "default", label: "草稿" },
|
||||
submitted: { variant: "info", label: "已提交" },
|
||||
pending_review: { variant: "warning", label: "待审核" },
|
||||
ingesting: { variant: "info", label: "入湖中" },
|
||||
in_lake: { variant: "success", label: "已入湖" },
|
||||
ingested: { variant: "success", label: "已入湖" },
|
||||
ingest_failed: { variant: "danger", label: "入湖失败" },
|
||||
cancelled: { variant: "default", label: "已取消" },
|
||||
};
|
||||
const m = map[status] || { variant: "default" as const, label: status };
|
||||
|
||||
131
platform/web/src/lib/labelingDisplay.ts
Normal file
@@ -0,0 +1,131 @@
|
||||
import type { LabelingBatchRow } from "./types";
|
||||
|
||||
/** ADAS 3D MOON:project=adas, task=cuboid_7cls */
|
||||
export function isAdas3dScope(b: LabelingBatchRow): boolean {
|
||||
const project = b.project || "dms";
|
||||
const task = b.task || "";
|
||||
return project === "adas" && task === "cuboid_7cls";
|
||||
}
|
||||
|
||||
/** ADAS 2D 七类:project=adas, task=det_7cls(兼容旧 dms+adas) */
|
||||
export function isAdas2dScope(b: LabelingBatchRow): boolean {
|
||||
const project = b.project || "dms";
|
||||
const task = b.task || "";
|
||||
if (project === "adas" && task === "det_7cls") return true;
|
||||
return project === "dms" && task === "adas";
|
||||
}
|
||||
|
||||
/** 前向 ADAS 相关(2D 七类、3D、交通标志 forward) */
|
||||
export function isAdasScope(b: LabelingBatchRow): boolean {
|
||||
if (isAdas3dScope(b) || isAdas2dScope(b)) return true;
|
||||
const project = b.project || "dms";
|
||||
if (project === "lane") return false;
|
||||
const task = b.task || "";
|
||||
return b.domain === "forward" || task === "forward";
|
||||
}
|
||||
|
||||
/** 列表「项目」列:DMS / ADAS / 车道线 */
|
||||
export function displayProject(b: LabelingBatchRow): string {
|
||||
const project = b.project || "dms";
|
||||
if (project === "lane") return "车道线";
|
||||
if (isAdasScope(b)) return "ADAS";
|
||||
return "DMS";
|
||||
}
|
||||
|
||||
export function displayProjectFields(row: {
|
||||
project?: string;
|
||||
task?: string;
|
||||
domain?: string;
|
||||
}): string {
|
||||
return displayProject(row as LabelingBatchRow);
|
||||
}
|
||||
|
||||
export function displayTaskFields(row: {
|
||||
project?: string;
|
||||
task?: string;
|
||||
mode?: string;
|
||||
domain?: string;
|
||||
task_label?: string;
|
||||
mode_label?: string;
|
||||
}): string {
|
||||
return displayTask(row as LabelingBatchRow);
|
||||
}
|
||||
|
||||
/** 列表「任务」列 */
|
||||
export function displayTask(b: LabelingBatchRow): string {
|
||||
const project = b.project || "dms";
|
||||
if (project === "lane") return "车道线";
|
||||
|
||||
if (isAdas3dScope(b)) return "3D 七类";
|
||||
if (isAdas2dScope(b)) return "2D 七类";
|
||||
|
||||
if (isAdasScope(b)) {
|
||||
if (b.task === "forward") {
|
||||
return b.mode_label || (b.mode === "classify" ? "细分类" : "粗检测");
|
||||
}
|
||||
return "2D";
|
||||
}
|
||||
|
||||
const task = b.task || "";
|
||||
const cabin: Record<string, string> = {
|
||||
addw: "ADDW",
|
||||
addw_face: "ADDW 人脸",
|
||||
ddaw: "DDAW",
|
||||
dam: "DAM",
|
||||
};
|
||||
if (cabin[task]) {
|
||||
if (task === "dam" && b.mode_label) return `DAM · ${b.mode_label}`;
|
||||
return cabin[task];
|
||||
}
|
||||
return b.task_label || task || "—";
|
||||
}
|
||||
|
||||
/** 台账表单:业务线 → API project/task */
|
||||
export type DeliveryLineKind =
|
||||
| "dms"
|
||||
| "adas_2d"
|
||||
| "adas_3d"
|
||||
| "forward"
|
||||
| "lane";
|
||||
|
||||
export function deliveryLineToApi(line: DeliveryLineKind): { project: string; task: string; mode: string } {
|
||||
switch (line) {
|
||||
case "adas_2d":
|
||||
return { project: "adas", task: "det_7cls", mode: "" };
|
||||
case "adas_3d":
|
||||
return { project: "adas", task: "cuboid_7cls", mode: "" };
|
||||
case "forward":
|
||||
return { project: "dms", task: "forward", mode: "" };
|
||||
case "lane":
|
||||
return { project: "lane", task: "lane_v1", mode: "" };
|
||||
default:
|
||||
return { project: "dms", task: "", mode: "" };
|
||||
}
|
||||
}
|
||||
|
||||
export function defaultInboxPath(line: DeliveryLineKind, batch: string, task?: string, mode?: string): string {
|
||||
const b = batch.trim();
|
||||
if (line === "adas_2d") return `/data/hsap/datasets/adas/inbox/det_7cls/${b}`;
|
||||
if (line === "adas_3d") return `/data/hsap/datasets/adas/inbox/cuboid_7cls/${b}`;
|
||||
if (line === "lane") return `/data/hsap/datasets/lane/inbox/${b}`;
|
||||
if (line === "forward" && mode) return `/data/hsap/datasets/dms/inbox/forward/${mode}/${b}`;
|
||||
if (task) return `/data/hsap/datasets/dms/inbox/${task}/${b}`;
|
||||
return `/data/hsap/datasets/dms/inbox/{task}/${b}`;
|
||||
}
|
||||
|
||||
/** @deprecated 用 deliveryLineToApi */
|
||||
export function inferDeliveryProject(task: string, explicitProject?: string): string {
|
||||
const t = (task || "").trim().toLowerCase();
|
||||
if (explicitProject === "lane" || t === "lane" || t === "lane_v1") return "lane";
|
||||
if (explicitProject === "adas" || t === "cuboid_7cls" || t.includes("cuboid") || t === "moon3d") return "adas";
|
||||
return "dms";
|
||||
}
|
||||
|
||||
/** 提交 build 时默认训练包 */
|
||||
export function defaultBuildPack(b: LabelingBatchRow): string {
|
||||
if (b.pack) return b.pack;
|
||||
if (isAdas3dScope(b)) return "adas_moon3d_v1";
|
||||
if (isAdas2dScope(b)) return "adas_v1";
|
||||
if (b.project === "lane") return "lane_v1";
|
||||
return "dms_v2";
|
||||
}
|
||||
@@ -31,6 +31,7 @@ export type BatchRecord = {
|
||||
|
||||
export type LabelingBatchRow = BatchRecord & {
|
||||
scope_key?: string;
|
||||
domain?: string;
|
||||
domain_label?: string;
|
||||
task_label?: string;
|
||||
mode_label?: string;
|
||||
|
||||
@@ -0,0 +1,67 @@
|
||||
import { useEffect, useState } from "react";
|
||||
import { TileLayer, useMap } from "react-leaflet";
|
||||
|
||||
export type TileDef = {
|
||||
url: string;
|
||||
subdomains: string[];
|
||||
attribution: string;
|
||||
id: string;
|
||||
};
|
||||
|
||||
const OSM_FALLBACK: TileDef = {
|
||||
id: "osm",
|
||||
url: "https://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png",
|
||||
subdomains: ["a", "b", "c"],
|
||||
attribution: "© OpenStreetMap",
|
||||
};
|
||||
|
||||
type Props = {
|
||||
primary: TileDef;
|
||||
onActiveChange?: (tile: TileDef) => void;
|
||||
};
|
||||
|
||||
function TileErrorWatcher({ primary, onFallback }: { primary: TileDef; onFallback: () => void }) {
|
||||
const map = useMap();
|
||||
useEffect(() => {
|
||||
let errors = 0;
|
||||
const onError = () => {
|
||||
errors += 1;
|
||||
if (errors >= 3 && primary.id !== "osm") onFallback();
|
||||
};
|
||||
map.on("tileerror", onError);
|
||||
return () => {
|
||||
map.off("tileerror", onError);
|
||||
};
|
||||
}, [map, primary.id, onFallback]);
|
||||
return null;
|
||||
}
|
||||
|
||||
export function FallbackTileLayer({ primary, onActiveChange }: Props) {
|
||||
const [active, setActive] = useState<TileDef>(primary);
|
||||
|
||||
useEffect(() => {
|
||||
setActive(primary);
|
||||
}, [primary.id, primary.url]);
|
||||
|
||||
useEffect(() => {
|
||||
onActiveChange?.(active);
|
||||
}, [active, onActiveChange]);
|
||||
|
||||
const fallback = () => {
|
||||
if (active.id !== "osm") setActive(OSM_FALLBACK);
|
||||
};
|
||||
|
||||
return (
|
||||
<>
|
||||
<TileLayer
|
||||
key={active.id}
|
||||
attribution={active.attribution}
|
||||
url={active.url}
|
||||
subdomains={active.subdomains}
|
||||
/>
|
||||
{active.id === primary.id && primary.id !== "osm" && (
|
||||
<TileErrorWatcher primary={primary} onFallback={fallback} />
|
||||
)}
|
||||
</>
|
||||
);
|
||||
}
|
||||
139
platform/web/src/modules/fleet/components/FleetMapPanel.tsx
Normal file
@@ -0,0 +1,139 @@
|
||||
import { useEffect, useMemo } from "react";
|
||||
import { CircleMarker, MapContainer, Polyline, Popup, useMap } from "react-leaflet";
|
||||
import L from "leaflet";
|
||||
import "leaflet/dist/leaflet.css";
|
||||
import { FallbackTileLayer, type TileDef } from "./FallbackTileLayer";
|
||||
import "./fleet-map.css";
|
||||
|
||||
export type LiveVehicle = {
|
||||
vehicle_id: number;
|
||||
plate_no?: string;
|
||||
lat?: number | null;
|
||||
lng?: number | null;
|
||||
last_lat?: number | null;
|
||||
last_lng?: number | null;
|
||||
status?: string;
|
||||
online?: boolean;
|
||||
};
|
||||
|
||||
export type MapConfig = {
|
||||
provider?: string;
|
||||
tileProvider?: string;
|
||||
amapKey?: string;
|
||||
};
|
||||
|
||||
function vehicleLat(v: LiveVehicle): number | null {
|
||||
const lat = v.lat ?? v.last_lat;
|
||||
return lat != null ? Number(lat) : null;
|
||||
}
|
||||
|
||||
function vehicleLng(v: LiveVehicle): number | null {
|
||||
const lng = v.lng ?? v.last_lng;
|
||||
return lng != null ? Number(lng) : null;
|
||||
}
|
||||
|
||||
function resolveTiles(cfg: MapConfig): TileDef {
|
||||
const p = (cfg.tileProvider || cfg.provider || "gaode").toLowerCase();
|
||||
if (p === "gaode") {
|
||||
const base =
|
||||
"https://webrd0{s}.is.autonavi.com/appmaptile?lang=zh_cn&size=1&scale=1&style=8&x={x}&y={y}&z={z}";
|
||||
const url = cfg.amapKey ? `${base}&key=${cfg.amapKey}` : base;
|
||||
return { id: "gaode", url, subdomains: ["1", "2", "3", "4"], attribution: "© 高德" };
|
||||
}
|
||||
return {
|
||||
id: "osm",
|
||||
url: "https://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png",
|
||||
subdomains: ["a", "b", "c"],
|
||||
attribution: "© OpenStreetMap",
|
||||
};
|
||||
}
|
||||
|
||||
function MapResizeFix() {
|
||||
const map = useMap();
|
||||
useEffect(() => {
|
||||
const fix = () => map.invalidateSize();
|
||||
fix();
|
||||
const t1 = window.setTimeout(fix, 100);
|
||||
const t2 = window.setTimeout(fix, 400);
|
||||
const ro = new ResizeObserver(fix);
|
||||
const el = map.getContainer().parentElement;
|
||||
if (el) ro.observe(el);
|
||||
return () => {
|
||||
window.clearTimeout(t1);
|
||||
window.clearTimeout(t2);
|
||||
ro.disconnect();
|
||||
};
|
||||
}, [map]);
|
||||
return null;
|
||||
}
|
||||
|
||||
function FitBounds({ points }: { points: [number, number][] }) {
|
||||
const map = useMap();
|
||||
useEffect(() => {
|
||||
if (points.length < 1) return;
|
||||
if (points.length === 1) {
|
||||
map.setView(points[0], 14);
|
||||
return;
|
||||
}
|
||||
map.fitBounds(L.latLngBounds(points), { padding: [40, 40], maxZoom: 15 });
|
||||
}, [map, points]);
|
||||
return null;
|
||||
}
|
||||
|
||||
type Props = {
|
||||
mapConfig: MapConfig | null;
|
||||
vehicles: LiveVehicle[];
|
||||
className?: string;
|
||||
};
|
||||
|
||||
export function FleetMapPanel({ mapConfig, vehicles, className }: Props) {
|
||||
const tiles = useMemo(() => resolveTiles(mapConfig || {}), [mapConfig]);
|
||||
|
||||
const points = useMemo(() => {
|
||||
const pts: [number, number][] = [];
|
||||
for (const v of vehicles) {
|
||||
const lat = vehicleLat(v);
|
||||
const lng = vehicleLng(v);
|
||||
if (lat != null && lng != null) pts.push([lat, lng]);
|
||||
}
|
||||
return pts;
|
||||
}, [vehicles]);
|
||||
|
||||
const center: [number, number] = points[0] || [28.2, 112.98];
|
||||
|
||||
return (
|
||||
<div
|
||||
className={`fleet-map-shell rounded-lg border border-gray-200 overflow-hidden min-h-[420px] ${className || ""}`}
|
||||
style={{ height: 420 }}
|
||||
>
|
||||
<MapContainer
|
||||
center={center}
|
||||
zoom={12}
|
||||
className="h-full w-full"
|
||||
style={{ height: "100%", minHeight: 420, width: "100%" }}
|
||||
scrollWheelZoom
|
||||
>
|
||||
<FallbackTileLayer primary={tiles} />
|
||||
<MapResizeFix />
|
||||
<FitBounds points={points} />
|
||||
{vehicles.map((v) => {
|
||||
const lat = vehicleLat(v);
|
||||
const lng = vehicleLng(v);
|
||||
if (lat == null || lng == null) return null;
|
||||
return (
|
||||
<CircleMarker
|
||||
key={v.vehicle_id}
|
||||
center={[lat, lng]}
|
||||
radius={8}
|
||||
pathOptions={{ color: "#2563eb", fillColor: "#3b82f6", fillOpacity: 0.9 }}
|
||||
>
|
||||
<Popup>
|
||||
{v.plate_no || v.vehicle_id} · {v.online ? "在线" : v.status || "—"}
|
||||
</Popup>
|
||||
</CircleMarker>
|
||||
);
|
||||
})}
|
||||
</MapContainer>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
14
platform/web/src/modules/fleet/components/fleet-map.css
Normal file
@@ -0,0 +1,14 @@
|
||||
.fleet-map-shell {
|
||||
position: relative;
|
||||
}
|
||||
|
||||
.fleet-map-shell .leaflet-container {
|
||||
height: 100% !important;
|
||||
width: 100% !important;
|
||||
font: inherit;
|
||||
}
|
||||
|
||||
.fleet-map-shell .leaflet-pane,
|
||||
.fleet-map-shell .leaflet-tile-pane {
|
||||
z-index: 1;
|
||||
}
|
||||
@@ -1,49 +1,72 @@
|
||||
import React, { useEffect, useState } from "react";
|
||||
import React, { useCallback, useEffect, useMemo, useState } from "react";
|
||||
import { hsapApi } from "@/app/hsap-api";
|
||||
import { PageQueryState } from "@/components/PageQueryState";
|
||||
import { Badge } from "@/components/ui/Badge";
|
||||
import { FleetMapPanel, type LiveVehicle } from "../components/FleetMapPanel";
|
||||
|
||||
type LiveResponse = {
|
||||
vehicles?: LiveVehicle[];
|
||||
stats?: Record<string, unknown>;
|
||||
};
|
||||
|
||||
export const LiveMapPage: React.FC = () => {
|
||||
const [live, setLive] = useState<Record<string, unknown> | null>(null);
|
||||
const [live, setLive] = useState<LiveResponse | null>(null);
|
||||
const [mapConfig, setMapConfig] = useState<Record<string, unknown> | null>(null);
|
||||
const [loading, setLoading] = useState(true);
|
||||
const [error, setError] = useState<string | null>(null);
|
||||
|
||||
const load = async () => {
|
||||
setLoading(true);
|
||||
const load = useCallback(async () => {
|
||||
setError(null);
|
||||
try {
|
||||
const [l, m] = await Promise.all([hsapApi.fleetLive(), hsapApi.fleetMapConfig()]);
|
||||
setLive(l);
|
||||
setLive(l as LiveResponse);
|
||||
setMapConfig(m);
|
||||
} catch (e) {
|
||||
setError(String(e));
|
||||
} finally {
|
||||
setLoading(false);
|
||||
}
|
||||
setLoading(false);
|
||||
};
|
||||
}, []);
|
||||
|
||||
useEffect(() => { load(); }, []);
|
||||
useEffect(() => {
|
||||
load();
|
||||
const pollSec = Number(mapConfig?.pollIntervalSec ?? 5);
|
||||
const ms = Math.max(3, pollSec) * 1000;
|
||||
const timer = window.setInterval(load, ms);
|
||||
return () => window.clearInterval(timer);
|
||||
}, [load, mapConfig?.pollIntervalSec]);
|
||||
|
||||
const vehicles = (live?.vehicles || []) as Record<string, unknown>[];
|
||||
const vehicles = useMemo(
|
||||
() =>
|
||||
((live?.vehicles || []) as Record<string, unknown>[]).map((v) => ({
|
||||
vehicle_id: Number(v.vehicle_id ?? v.id ?? 0),
|
||||
plate_no: String(v.plate_no ?? ""),
|
||||
lat: (v.lat ?? v.last_lat) as number | null | undefined,
|
||||
lng: (v.lng ?? v.last_lng) as number | null | undefined,
|
||||
last_lat: v.last_lat as number | null | undefined,
|
||||
last_lng: v.last_lng as number | null | undefined,
|
||||
speed_kmh: (v.speed_kmh ?? v.last_speed_kmh) as number | null | undefined,
|
||||
online: Boolean(v.online),
|
||||
status: String(v.status ?? ""),
|
||||
})),
|
||||
[live?.vehicles],
|
||||
);
|
||||
|
||||
const tileLabel = String(mapConfig?.tileProvider ?? mapConfig?.provider ?? "gaode");
|
||||
|
||||
return (
|
||||
<div className="page-container">
|
||||
<div className="page-header">
|
||||
<h1>实时地图</h1>
|
||||
<p>车辆 GPS 实时位置追踪</p>
|
||||
<p>车辆 GPS 实时位置追踪 · 底图 {tileLabel}</p>
|
||||
</div>
|
||||
|
||||
<PageQueryState loading={loading} error={error}>
|
||||
{/* Map placeholder — Leaflet/AMap will be integrated */}
|
||||
<div className="card mb-4" style={{ height: "400px", background: "#e5e7eb" }}>
|
||||
<div className="flex items-center justify-center h-full text-gray-400">
|
||||
<div className="text-center">
|
||||
<div className="text-4xl mb-2">🗺️</div>
|
||||
<p>地图组件 (Leaflet/高德)</p>
|
||||
<p className="text-xs mt-1">底图: {mapConfig?.tile_provider as string || "gaode"}</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<FleetMapPanel
|
||||
mapConfig={mapConfig as { provider?: string; tileProvider?: string; amapKey?: string } | null}
|
||||
vehicles={vehicles}
|
||||
className="mb-4"
|
||||
/>
|
||||
|
||||
<div className="card">
|
||||
<div className="card-header">在线车辆 ({vehicles.length})</div>
|
||||
@@ -60,20 +83,30 @@ export const LiveMapPage: React.FC = () => {
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
{vehicles.map((v, i) => (
|
||||
<tr key={i}>
|
||||
<td className="font-medium">{v.plate_no as string || "—"}</td>
|
||||
<td className="font-mono text-xs">
|
||||
{String(v.lat ?? "—")}, {String(v.lng ?? "—")}
|
||||
</td>
|
||||
<td>{v.speed_kmh != null ? `${v.speed_kmh} km/h` : "—"}</td>
|
||||
<td><Badge variant="success">在线</Badge></td>
|
||||
</tr>
|
||||
))}
|
||||
{vehicles.map((v) => {
|
||||
const lat = v.lat ?? v.last_lat;
|
||||
const lng = v.lng ?? v.last_lng;
|
||||
return (
|
||||
<tr key={v.vehicle_id}>
|
||||
<td className="font-medium">{v.plate_no || "—"}</td>
|
||||
<td className="font-mono text-xs">
|
||||
{lat != null && lng != null ? `${lat.toFixed(5)}, ${lng.toFixed(5)}` : "—, —"}
|
||||
</td>
|
||||
<td>{v.speed_kmh != null ? `${v.speed_kmh} km/h` : "—"}</td>
|
||||
<td>
|
||||
<Badge variant={v.online ? "success" : "default"}>
|
||||
{v.online ? "在线" : "离线"}
|
||||
</Badge>
|
||||
</td>
|
||||
</tr>
|
||||
);
|
||||
})}
|
||||
</tbody>
|
||||
</table>
|
||||
)}
|
||||
<button onClick={load} className="mt-3 text-sm text-blue-600 hover:underline">刷新</button>
|
||||
<button type="button" onClick={load} className="mt-3 text-sm text-blue-600 hover:underline">
|
||||
刷新
|
||||
</button>
|
||||
</div>
|
||||
</PageQueryState>
|
||||
</div>
|
||||
|
||||
@@ -29,6 +29,8 @@ export const LabelingShell: React.FC = () => {
|
||||
const { hasPermission } = useAuth();
|
||||
const isCoordinator = hasPermission("write:labeling_assign");
|
||||
const isAnnotate = /\/labeling\/annotate\//.test(location.pathname);
|
||||
const isReviewDetail = /\/labeling\/review\/[^/]+/.test(location.pathname);
|
||||
const isImmersive = isAnnotate || isReviewDetail;
|
||||
|
||||
const visibleTabs = TABS.filter((tab) => {
|
||||
if (tab.coordinatorOnly && !isCoordinator) return false;
|
||||
@@ -40,8 +42,8 @@ export const LabelingShell: React.FC = () => {
|
||||
|
||||
return (
|
||||
<ModuleGuard requiredPerms={["read:pending", "read:deliveries", "read:catalog", "write:delivery_submit"]}>
|
||||
<div className={isAnnotate ? "h-[calc(100vh-0px)] flex flex-col min-h-0" : undefined}>
|
||||
{!isAnnotate && (
|
||||
<div className={isImmersive ? "labeling-immersive h-full min-h-0 flex flex-col overflow-hidden" : undefined}>
|
||||
{!isImmersive && (
|
||||
<nav className="module-tabs">
|
||||
{visibleTabs.map((tab) => (
|
||||
<NavLink
|
||||
@@ -55,7 +57,8 @@ export const LabelingShell: React.FC = () => {
|
||||
))}
|
||||
</nav>
|
||||
)}
|
||||
<Switch>
|
||||
<div className={isImmersive ? "flex-1 min-h-0 flex flex-col overflow-hidden" : undefined}>
|
||||
<Switch>
|
||||
<Route exact path={`${path}`} render={() => <Redirect to={defaultPath} />} />
|
||||
<Route path={`${path}/annotate/:campaignId`} component={AnnotationPage} />
|
||||
<Route path={`${path}/my-tasks`} component={MyTasksPage} />
|
||||
@@ -67,7 +70,8 @@ export const LabelingShell: React.FC = () => {
|
||||
<Route path={`${path}/deliveries`} component={DeliveriesPage} />
|
||||
<Route path={`${path}/catalog`} component={CatalogPage} />
|
||||
<Route path={`${path}/simulate`} component={SimulationStudioPage} />
|
||||
</Switch>
|
||||
</Switch>
|
||||
</div>
|
||||
</div>
|
||||
</ModuleGuard>
|
||||
);
|
||||
|
||||
@@ -0,0 +1,124 @@
|
||||
import React from "react";
|
||||
import { StageBadge } from "@/components/ui/Badge";
|
||||
import { CompactTableShell } from "@/components/CompactTableShell";
|
||||
import { TruncatedText } from "@/components/TruncatedText";
|
||||
import { TableIconAction } from "./TableIconAction";
|
||||
import { displayProject, displayTask } from "@/lib/labelingDisplay";
|
||||
import type { LabelingBatchRow } from "@/lib/types";
|
||||
|
||||
type CampaignBatchTableProps = {
|
||||
batches: LabelingBatchRow[];
|
||||
expandCampaign: string | null;
|
||||
onToggleExpand: (campaignId: string) => void;
|
||||
onExport: (campaignId: string) => void;
|
||||
onSubmit: (campaignId: string) => void;
|
||||
renderExpand?: (b: LabelingBatchRow) => React.ReactNode;
|
||||
};
|
||||
|
||||
const COLS = ["31%", "7%", "6%", "8%", "9%", "10%", "12.5rem"];
|
||||
|
||||
export const CampaignBatchTable: React.FC<CampaignBatchTableProps> = ({
|
||||
batches,
|
||||
expandCampaign,
|
||||
onToggleExpand,
|
||||
onExport,
|
||||
onSubmit,
|
||||
renderExpand,
|
||||
}) => (
|
||||
<CompactTableShell colWidths={COLS}>
|
||||
<thead>
|
||||
<tr>
|
||||
<th className="py-2">批次</th>
|
||||
<th className="py-2">任务</th>
|
||||
<th className="py-2">项目</th>
|
||||
<th className="py-2">状态</th>
|
||||
<th className="py-2">进度</th>
|
||||
<th className="py-2">Campaign</th>
|
||||
<th className="py-2 w-[12.5rem]">操作</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
{batches.map((b) => {
|
||||
const cid = b.campaign_id || "";
|
||||
const isExpanded = expandCampaign === cid;
|
||||
const canExport = ["labeling_submitted", "returned"].includes(b.stage || "");
|
||||
const pct =
|
||||
b.total_tasks && b.total_tasks > 0
|
||||
? Math.round(((b.completed_tasks || 0) / b.total_tasks) * 100)
|
||||
: 0;
|
||||
|
||||
return (
|
||||
<React.Fragment key={cid || b.batch}>
|
||||
<tr className={`align-middle ${isExpanded ? "bg-blue-50/40" : ""}`}>
|
||||
<td className="py-2 max-w-0 overflow-hidden">
|
||||
<TruncatedText text={b.batch || "—"} className="font-medium text-gray-900" maxWidthClass="max-w-full" />
|
||||
</td>
|
||||
<td className="py-2 max-w-0 overflow-hidden">
|
||||
<TruncatedText text={displayTask(b)} maxWidthClass="max-w-full" />
|
||||
</td>
|
||||
<td className="py-2 whitespace-nowrap text-sm text-gray-700">{displayProject(b)}</td>
|
||||
<td className="py-2 whitespace-nowrap">
|
||||
<StageBadge stage={b.stage} />
|
||||
</td>
|
||||
<td className="py-2 whitespace-nowrap">
|
||||
{b.total_tasks != null && b.total_tasks > 0 ? (
|
||||
<div className="flex items-center justify-center gap-1 min-w-0">
|
||||
<div className="w-10 h-1.5 bg-gray-100 rounded-full overflow-hidden shrink-0">
|
||||
<div
|
||||
className={`h-full rounded-full ${pct >= 100 ? "bg-green-500" : "bg-blue-500"}`}
|
||||
style={{ width: `${pct}%` }}
|
||||
/>
|
||||
</div>
|
||||
<span className="text-[11px] text-gray-500 tabular-nums">
|
||||
{b.completed_tasks}/{b.total_tasks}
|
||||
</span>
|
||||
</div>
|
||||
) : (
|
||||
<span className="text-xs text-gray-400">—</span>
|
||||
)}
|
||||
</td>
|
||||
<td className="py-2 max-w-0 overflow-hidden">
|
||||
<TruncatedText text={cid || "—"} className="font-mono text-xs text-gray-500" maxWidthClass="max-w-full" />
|
||||
</td>
|
||||
<td className="py-2 px-0.5 align-middle w-[12.5rem]">
|
||||
{cid && (
|
||||
<div className="inline-flex flex-nowrap items-center justify-center gap-0.5">
|
||||
<TableIconAction
|
||||
dense
|
||||
to={`/labeling/annotate/${encodeURIComponent(cid)}`}
|
||||
tone="blue"
|
||||
>
|
||||
✏️ 标注
|
||||
</TableIconAction>
|
||||
<TableIconAction
|
||||
dense
|
||||
tone="gray"
|
||||
disabled={!canExport}
|
||||
title={canExport ? undefined : "质检通过后可导出"}
|
||||
onClick={() => onExport(cid)}
|
||||
>
|
||||
📤 导出
|
||||
</TableIconAction>
|
||||
<TableIconAction dense tone="green" onClick={() => onSubmit(cid)}>
|
||||
✅ 质检
|
||||
</TableIconAction>
|
||||
<TableIconAction dense tone="purple" onClick={() => onToggleExpand(cid)}>
|
||||
👥 {isExpanded ? "收起" : "分配"}
|
||||
</TableIconAction>
|
||||
</div>
|
||||
)}
|
||||
</td>
|
||||
</tr>
|
||||
{isExpanded && renderExpand && (
|
||||
<tr>
|
||||
<td colSpan={7} className="p-0 border-b border-gray-100 bg-gray-50/50">
|
||||
{renderExpand(b)}
|
||||
</td>
|
||||
</tr>
|
||||
)}
|
||||
</React.Fragment>
|
||||
);
|
||||
})}
|
||||
</tbody>
|
||||
</CompactTableShell>
|
||||
);
|
||||
@@ -0,0 +1,271 @@
|
||||
import React, { useEffect, useState } from "react";
|
||||
import { Button } from "@/components/ui/Button";
|
||||
import {
|
||||
defaultInboxPath,
|
||||
deliveryLineToApi,
|
||||
type DeliveryLineKind,
|
||||
} from "@/lib/labelingDisplay";
|
||||
|
||||
export type DeliveryFormValues = {
|
||||
line: DeliveryLineKind;
|
||||
project: string;
|
||||
task: string;
|
||||
mode: string;
|
||||
batch_name: string;
|
||||
data_path: string;
|
||||
source_type: string;
|
||||
collection_start: string;
|
||||
collection_end: string;
|
||||
estimated_count: string;
|
||||
vehicle_scene: string;
|
||||
remark: string;
|
||||
};
|
||||
|
||||
const EMPTY: DeliveryFormValues = {
|
||||
line: "dms",
|
||||
project: "dms",
|
||||
task: "",
|
||||
mode: "",
|
||||
batch_name: "",
|
||||
data_path: "",
|
||||
source_type: "platform_delivery",
|
||||
collection_start: "",
|
||||
collection_end: "",
|
||||
estimated_count: "",
|
||||
vehicle_scene: "",
|
||||
remark: "",
|
||||
};
|
||||
|
||||
const LINES: { id: DeliveryLineKind; label: string; desc: string }[] = [
|
||||
{ id: "dms", label: "DMS 舱内", desc: "addw / ddaw / dam / addw_face" },
|
||||
{ id: "adas_2d", label: "ADAS 2D 七类", desc: "project=adas · det_7cls · adas/inbox/det_7cls/" },
|
||||
{ id: "adas_3d", label: "ADAS 3D MOON", desc: "project=adas · cuboid_7cls · adas/inbox/" },
|
||||
{ id: "forward", label: "前向交通标志", desc: "project=dms · task=forward · 须填子模式" },
|
||||
{ id: "lane", label: "车道线", desc: "project=lane · lane/inbox/{批次}" },
|
||||
];
|
||||
|
||||
type DeliveryCreateModalProps = {
|
||||
open: boolean;
|
||||
saving: boolean;
|
||||
onClose: () => void;
|
||||
onSubmit: (values: DeliveryFormValues) => void;
|
||||
};
|
||||
|
||||
const fieldCls =
|
||||
"w-full rounded-lg border border-gray-200 px-3 py-2 text-sm focus:outline-none focus:ring-2 focus:ring-blue-200";
|
||||
const labelCls = "block text-xs font-medium text-gray-600 mb-1";
|
||||
|
||||
export const DeliveryCreateModal: React.FC<DeliveryCreateModalProps> = ({
|
||||
open,
|
||||
saving,
|
||||
onClose,
|
||||
onSubmit,
|
||||
}) => {
|
||||
const [form, setForm] = useState<DeliveryFormValues>(EMPTY);
|
||||
const [intake, setIntake] = useState<"nas" | "inbox">("nas");
|
||||
|
||||
useEffect(() => {
|
||||
if (!open) return;
|
||||
const api = deliveryLineToApi(form.line);
|
||||
setForm((f) => ({
|
||||
...f,
|
||||
project: api.project,
|
||||
task: f.line === "dms" || f.line === "forward" ? f.task : api.task,
|
||||
mode: f.line === "forward" ? f.mode : api.mode,
|
||||
}));
|
||||
}, [form.line, open]);
|
||||
|
||||
useEffect(() => {
|
||||
if (!open || intake !== "inbox" || !form.batch_name.trim()) return;
|
||||
if (form.data_path.trim()) return;
|
||||
const p = defaultInboxPath(
|
||||
form.line,
|
||||
form.batch_name.trim(),
|
||||
form.task || undefined,
|
||||
form.mode || undefined,
|
||||
);
|
||||
setForm((f) => ({ ...f, data_path: p }));
|
||||
}, [form.batch_name, form.line, form.mode, form.task, intake, open]);
|
||||
|
||||
if (!open) return null;
|
||||
|
||||
const set = (k: keyof DeliveryFormValues, v: string) => setForm((f) => ({ ...f, [k]: v }));
|
||||
|
||||
const onLineChange = (line: DeliveryLineKind) => {
|
||||
const api = deliveryLineToApi(line);
|
||||
setForm((f) => ({
|
||||
...f,
|
||||
line,
|
||||
project: api.project,
|
||||
task: line === "dms" ? "" : api.task,
|
||||
mode: line === "forward" ? f.mode : api.mode,
|
||||
data_path: "",
|
||||
}));
|
||||
};
|
||||
|
||||
const pathHint =
|
||||
intake === "nas"
|
||||
? "/data/nas/采集批次/..."
|
||||
: defaultInboxPath(form.line, "{batch}", form.task || undefined, form.mode || undefined);
|
||||
|
||||
const taskLocked = form.line === "adas_2d" || form.line === "adas_3d" || form.line === "lane";
|
||||
const modeRequired = form.line === "forward" || form.line === "dms";
|
||||
|
||||
const handleSubmit = (e: React.FormEvent) => {
|
||||
e.preventDefault();
|
||||
const api = deliveryLineToApi(form.line);
|
||||
onSubmit({
|
||||
...form,
|
||||
project: api.project,
|
||||
task: form.line === "dms" ? form.task.trim() : api.task,
|
||||
mode: form.line === "forward" || (form.line === "dms" && form.task === "dam") ? form.mode.trim() : api.mode,
|
||||
source_type: intake === "nas" ? "platform_delivery" : "inbox_scan",
|
||||
});
|
||||
};
|
||||
|
||||
const resetClose = () => {
|
||||
setForm(EMPTY);
|
||||
setIntake("nas");
|
||||
onClose();
|
||||
};
|
||||
|
||||
return (
|
||||
<div className="fixed inset-0 z-50 flex items-center justify-center bg-black/40 p-4" onClick={resetClose}>
|
||||
<div
|
||||
className="bg-white rounded-xl shadow-xl w-full max-w-2xl max-h-[90vh] overflow-y-auto"
|
||||
onClick={(e) => e.stopPropagation()}
|
||||
>
|
||||
<div className="px-6 py-4 border-b border-gray-100">
|
||||
<h3 className="text-lg font-semibold text-gray-900">新建送标申请</h3>
|
||||
<p className="text-sm text-gray-500 mt-0.5">
|
||||
ADAS 2D 与 3D 同在 <code className="text-xs">adas/inbox/</code>:2D 在{" "}
|
||||
<code className="text-xs">det_7cls</code>,3D 在{" "}
|
||||
<code className="text-xs">adas/inbox/cuboid_7cls</code>
|
||||
</p>
|
||||
</div>
|
||||
|
||||
<form onSubmit={handleSubmit} className="px-6 py-4 space-y-4">
|
||||
<div>
|
||||
<label className={labelCls}>业务线 *</label>
|
||||
<div className="grid grid-cols-2 sm:grid-cols-3 gap-2">
|
||||
{LINES.map((l) => (
|
||||
<button
|
||||
key={l.id}
|
||||
type="button"
|
||||
onClick={() => onLineChange(l.id)}
|
||||
className={`rounded-lg border p-2.5 text-left text-xs transition-colors ${
|
||||
form.line === l.id ? "border-blue-400 bg-blue-50" : "border-gray-200 hover:bg-gray-50"
|
||||
}`}
|
||||
>
|
||||
<div className="font-medium text-sm">{l.label}</div>
|
||||
<div className="text-gray-500 mt-0.5 leading-snug">{l.desc}</div>
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div className="flex gap-2">
|
||||
{([
|
||||
{ id: "nas" as const, label: "NAS / 外挂盘", desc: "审批后拷贝入湖" },
|
||||
{ id: "inbox" as const, label: "已在 inbox", desc: "建议用「扫描数据湖」" },
|
||||
]).map((t) => (
|
||||
<button
|
||||
key={t.id}
|
||||
type="button"
|
||||
onClick={() => setIntake(t.id)}
|
||||
className={`flex-1 rounded-lg border p-3 text-left transition-colors ${
|
||||
intake === t.id ? "border-blue-400 bg-blue-50" : "border-gray-200 hover:bg-gray-50"
|
||||
}`}
|
||||
>
|
||||
<div className="text-sm font-medium">{t.label}</div>
|
||||
<div className="text-xs text-gray-500 mt-0.5">{t.desc}</div>
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
|
||||
<div className="grid grid-cols-3 gap-3">
|
||||
<div>
|
||||
<label className={labelCls}>API 项目</label>
|
||||
<input className={`${fieldCls} bg-gray-50`} readOnly value={form.project} />
|
||||
</div>
|
||||
<div>
|
||||
<label className={labelCls}>任务 *</label>
|
||||
<input
|
||||
className={taskLocked ? `${fieldCls} bg-gray-50` : fieldCls}
|
||||
readOnly={taskLocked}
|
||||
placeholder={form.line === "dms" ? "addw / dam …" : ""}
|
||||
value={form.task}
|
||||
onChange={(e) => set("task", e.target.value)}
|
||||
required
|
||||
/>
|
||||
</div>
|
||||
<div>
|
||||
<label className={labelCls}>子模式{form.line === "forward" ? " *" : ""}</label>
|
||||
<input
|
||||
className={fieldCls}
|
||||
placeholder={form.line === "forward" ? "detect / classify" : "dam: batch_0516"}
|
||||
value={form.mode}
|
||||
onChange={(e) => set("mode", e.target.value)}
|
||||
required={form.line === "forward"}
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div>
|
||||
<label className={labelCls}>批次名称 *</label>
|
||||
<input
|
||||
className={fieldCls}
|
||||
placeholder="20260601_pilot(勿与任务名相同)"
|
||||
value={form.batch_name}
|
||||
onChange={(e) => set("batch_name", e.target.value)}
|
||||
required
|
||||
/>
|
||||
</div>
|
||||
|
||||
<div>
|
||||
<label className={labelCls}>数据路径 *</label>
|
||||
<input
|
||||
className={`${fieldCls} font-mono text-xs`}
|
||||
placeholder={pathHint}
|
||||
value={form.data_path}
|
||||
onChange={(e) => set("data_path", e.target.value)}
|
||||
required
|
||||
/>
|
||||
</div>
|
||||
|
||||
<div className="grid grid-cols-2 gap-3">
|
||||
<div>
|
||||
<label className={labelCls}>采集开始</label>
|
||||
<input type="date" className={fieldCls} value={form.collection_start} onChange={(e) => set("collection_start", e.target.value)} />
|
||||
</div>
|
||||
<div>
|
||||
<label className={labelCls}>采集结束</label>
|
||||
<input type="date" className={fieldCls} value={form.collection_end} onChange={(e) => set("collection_end", e.target.value)} />
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div className="grid grid-cols-2 gap-3">
|
||||
<div>
|
||||
<label className={labelCls}>预估张数</label>
|
||||
<input type="number" min={0} className={fieldCls} value={form.estimated_count} onChange={(e) => set("estimated_count", e.target.value)} />
|
||||
</div>
|
||||
<div>
|
||||
<label className={labelCls}>车辆 / 场景</label>
|
||||
<input className={fieldCls} value={form.vehicle_scene} onChange={(e) => set("vehicle_scene", e.target.value)} />
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div>
|
||||
<label className={labelCls}>备注</label>
|
||||
<textarea className={`${fieldCls} resize-none`} rows={2} value={form.remark} onChange={(e) => set("remark", e.target.value)} />
|
||||
</div>
|
||||
|
||||
<div className="flex gap-2 justify-end pt-2 border-t border-gray-100">
|
||||
<Button type="button" variant="default" size="small" onClick={resetClose}>取消</Button>
|
||||
<Button type="submit" variant="primary" size="small" loading={saving}>保存草稿</Button>
|
||||
</div>
|
||||
</form>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
};
|
||||
@@ -0,0 +1,72 @@
|
||||
import React from "react";
|
||||
import { Button } from "@/components/ui/Button";
|
||||
|
||||
type DeliveryIntakePanelProps = {
|
||||
total: number;
|
||||
inLake: number;
|
||||
pending: number;
|
||||
scanPending?: number;
|
||||
scanning: boolean;
|
||||
canCreate: boolean;
|
||||
onScan: () => void;
|
||||
onCreate: () => void;
|
||||
};
|
||||
|
||||
export const DeliveryIntakePanel: React.FC<DeliveryIntakePanelProps> = ({
|
||||
total,
|
||||
inLake,
|
||||
pending,
|
||||
scanPending,
|
||||
scanning,
|
||||
canCreate,
|
||||
onScan,
|
||||
onCreate,
|
||||
}) => (
|
||||
<div className="grid grid-cols-1 md:grid-cols-4 gap-3 mb-4">
|
||||
<div className="card p-4 border-blue-100 bg-blue-50/40">
|
||||
<div className="text-xs text-blue-600 font-medium mb-1">台账总批次</div>
|
||||
<div className="text-2xl font-semibold text-blue-900 tabular-nums">{total}</div>
|
||||
<div className="text-xs text-gray-500 mt-1">已入湖 {inLake} · 待处理 {pending}</div>
|
||||
</div>
|
||||
|
||||
<button
|
||||
type="button"
|
||||
onClick={onScan}
|
||||
className="card p-4 text-left border-emerald-100 bg-emerald-50/40 hover:border-emerald-300 transition-colors"
|
||||
>
|
||||
<div className="text-xs text-emerald-700 font-medium mb-1">① 扫描数据湖</div>
|
||||
<div className="text-sm font-semibold text-emerald-900">inbox 周期落盘</div>
|
||||
<div className="text-xs text-gray-500 mt-2">
|
||||
{scanPending != null ? `待登记 ${scanPending} 批` : "发现 inbox 新批次"}
|
||||
</div>
|
||||
<Button size="small" variant="primary" className="mt-3" loading={scanning} onClick={(e) => { e.stopPropagation(); onScan(); }}>
|
||||
立即扫描
|
||||
</Button>
|
||||
</button>
|
||||
|
||||
<button
|
||||
type="button"
|
||||
disabled={!canCreate}
|
||||
onClick={onCreate}
|
||||
className="card p-4 text-left border-amber-100 bg-amber-50/40 hover:border-amber-300 transition-colors disabled:opacity-50"
|
||||
>
|
||||
<div className="text-xs text-amber-700 font-medium mb-1">② NAS 外挂送标</div>
|
||||
<div className="text-sm font-semibold text-amber-900">挂载盘 → 审批入湖</div>
|
||||
<div className="text-xs text-gray-500 mt-2">填写路径与采集周期</div>
|
||||
{canCreate && (
|
||||
<Button size="small" variant="default" className="mt-3" onClick={(e) => { e.stopPropagation(); onCreate(); }}>
|
||||
新建申请
|
||||
</Button>
|
||||
)}
|
||||
</button>
|
||||
|
||||
<div className="card p-4 border-purple-100 bg-purple-50/40">
|
||||
<div className="text-xs text-purple-700 font-medium mb-1">③ 送标工作台</div>
|
||||
<div className="text-sm font-semibold text-purple-900">入湖后开标</div>
|
||||
<div className="text-xs text-gray-500 mt-2">台账登记 → 工作台待送标</div>
|
||||
<a href="/labeling/workbench" className="inline-block mt-3">
|
||||
<Button size="small" variant="default">去开标 →</Button>
|
||||
</a>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
114
platform/web/src/modules/labeling/components/DeliveryTable.tsx
Normal file
@@ -0,0 +1,114 @@
|
||||
import React from "react";
|
||||
import { Link } from "react-router-dom";
|
||||
import { Button } from "@/components/ui/Button";
|
||||
import { StatusBadge } from "@/components/ui/Badge";
|
||||
import { CompactTableShell } from "@/components/CompactTableShell";
|
||||
import { TruncatedText } from "@/components/TruncatedText";
|
||||
import { displayProjectFields, displayTaskFields } from "@/lib/labelingDisplay";
|
||||
import type { BatchDelivery } from "@/lib/types";
|
||||
|
||||
const SOURCE_LABELS: Record<string, string> = {
|
||||
inbox_scan: "inbox扫描",
|
||||
platform_delivery: "NAS送标",
|
||||
upload: "页面上传",
|
||||
feishu_bitable: "飞书",
|
||||
};
|
||||
|
||||
function formatSource(st?: string | null, d?: BatchDelivery): string {
|
||||
if (!st) return "—";
|
||||
if (d?.project === "adas" && d?.task === "det_7cls") return "ADAS 2D";
|
||||
if (d?.project === "dms" && d?.task === "adas") return "ADAS 2D";
|
||||
if (d?.project === "adas" && d?.task === "cuboid_7cls") return "ADAS 3D";
|
||||
return SOURCE_LABELS[st] || st;
|
||||
}
|
||||
|
||||
function formatDate(iso?: string | null): string {
|
||||
if (!iso) return "—";
|
||||
return iso.slice(0, 10);
|
||||
}
|
||||
|
||||
type DeliveryTableProps = {
|
||||
deliveries: BatchDelivery[];
|
||||
canSubmit: boolean;
|
||||
onSubmit: (id: string) => void;
|
||||
onDelete: (id: string) => void;
|
||||
};
|
||||
|
||||
const COLS = ["22%", "8%", "8%", "8%", "9%", "10%", "9%", "9%", "7.5rem"];
|
||||
|
||||
export const DeliveryTable: React.FC<DeliveryTableProps> = ({
|
||||
deliveries,
|
||||
canSubmit,
|
||||
onSubmit,
|
||||
onDelete,
|
||||
}) => (
|
||||
<CompactTableShell colWidths={COLS}>
|
||||
<thead>
|
||||
<tr>
|
||||
<th className="py-2">批次</th>
|
||||
<th className="py-2">来源</th>
|
||||
<th className="py-2">项目</th>
|
||||
<th className="py-2">任务</th>
|
||||
<th className="py-2">状态</th>
|
||||
<th className="py-2">采集周期</th>
|
||||
<th className="py-2">数量</th>
|
||||
<th className="py-2">登记时间</th>
|
||||
<th className="py-2">操作</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
{deliveries.map((d) => {
|
||||
const period = d.collection_start
|
||||
? `${formatDate(d.collection_start)}${d.collection_end ? ` ~ ${formatDate(d.collection_end)}` : ""}`
|
||||
: "—";
|
||||
return (
|
||||
<tr key={d.id} className="align-middle">
|
||||
<td className="py-2 max-w-0">
|
||||
<TruncatedText text={d.batch_name || "—"} className="font-medium text-gray-900" maxWidthClass="max-w-full" />
|
||||
</td>
|
||||
<td className="py-2 whitespace-nowrap text-xs text-gray-600">{formatSource(d.source_type, d)}</td>
|
||||
<td className="py-2 whitespace-nowrap text-sm text-gray-700">
|
||||
{displayProjectFields({ project: d.project, task: d.task || undefined })}
|
||||
</td>
|
||||
<td className="py-2 max-w-0">
|
||||
<TruncatedText
|
||||
text={displayTaskFields({ project: d.project, task: d.task || undefined })}
|
||||
maxWidthClass="max-w-full"
|
||||
/>
|
||||
</td>
|
||||
<td className="py-2 whitespace-nowrap">
|
||||
<StatusBadge status={d.status} />
|
||||
</td>
|
||||
<td className="py-2 whitespace-nowrap text-xs text-gray-500">{period}</td>
|
||||
<td className="py-2 text-sm text-gray-600 tabular-nums">{d.estimated_count ?? "—"}</td>
|
||||
<td className="py-2 whitespace-nowrap text-xs text-gray-500">{formatDate(d.created_at)}</td>
|
||||
<td className="py-2 px-2 whitespace-nowrap w-[7.5rem]">
|
||||
<div className="inline-flex items-center justify-center gap-0.5 flex-wrap">
|
||||
{d.status === "draft" && canSubmit && (
|
||||
<>
|
||||
<Button size="small" variant="primary" className="!px-2" onClick={() => onSubmit(d.id)}>
|
||||
提交
|
||||
</Button>
|
||||
<Button size="small" variant="danger" className="!px-2" onClick={() => onDelete(d.id)}>
|
||||
删除
|
||||
</Button>
|
||||
</>
|
||||
)}
|
||||
{d.status === "in_lake" && (
|
||||
<Link to="/labeling/workbench">
|
||||
<Button size="small" variant="success" className="!px-2">开标</Button>
|
||||
</Link>
|
||||
)}
|
||||
{d.approval_id && (
|
||||
<Link to={`/system/audit/${d.approval_id}`}>
|
||||
<Button size="small" variant="default" className="!px-2">审核</Button>
|
||||
</Link>
|
||||
)}
|
||||
</div>
|
||||
</td>
|
||||
</tr>
|
||||
);
|
||||
})}
|
||||
</tbody>
|
||||
</CompactTableShell>
|
||||
);
|
||||
@@ -0,0 +1,108 @@
|
||||
import React from "react";
|
||||
import { Link } from "react-router-dom";
|
||||
import { Button } from "@/components/ui/Button";
|
||||
import { Badge } from "@/components/ui/Badge";
|
||||
import { CompactTableShell } from "@/components/CompactTableShell";
|
||||
import { TruncatedText } from "@/components/TruncatedText";
|
||||
import { displayProject, displayTask } from "@/lib/labelingDisplay";
|
||||
import type { LabelingBatchRow } from "@/lib/types";
|
||||
|
||||
type ExportBatchTableProps = {
|
||||
batches: LabelingBatchRow[];
|
||||
importingId: string | null;
|
||||
buildingId: string | null;
|
||||
onExport: (campaignId: string) => void;
|
||||
onImportVendor: (campaignId: string) => void;
|
||||
onSubmitBuild: (batch: LabelingBatchRow) => void;
|
||||
};
|
||||
|
||||
const COLS = ["34%", "14%", "8%", "10%", "auto", "7.5rem"];
|
||||
|
||||
export const ExportBatchTable: React.FC<ExportBatchTableProps> = ({
|
||||
batches,
|
||||
importingId,
|
||||
buildingId,
|
||||
onExport,
|
||||
onImportVendor,
|
||||
onSubmitBuild,
|
||||
}) => (
|
||||
<CompactTableShell colWidths={COLS}>
|
||||
<thead>
|
||||
<tr>
|
||||
<th className="py-2">批次</th>
|
||||
<th className="py-2">任务</th>
|
||||
<th className="py-2">项目</th>
|
||||
<th className="py-2">状态</th>
|
||||
<th className="py-2">Campaign</th>
|
||||
<th className="py-2 w-[7.5rem]">操作</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
{batches.map((b) => {
|
||||
const cid = b.campaign_id || "";
|
||||
const isExport = b.stage === "labeling_submitted";
|
||||
return (
|
||||
<tr key={cid || b.batch} className="align-middle">
|
||||
<td className="py-2 max-w-0">
|
||||
<TruncatedText text={b.batch || "—"} className="font-medium text-gray-900" maxWidthClass="max-w-full" />
|
||||
</td>
|
||||
<td className="py-2 max-w-0">
|
||||
<TruncatedText text={displayTask(b)} maxWidthClass="max-w-full" />
|
||||
</td>
|
||||
<td className="py-2 whitespace-nowrap">
|
||||
<span className="text-sm text-gray-700">{displayProject(b)}</span>
|
||||
</td>
|
||||
<td className="py-2 whitespace-nowrap">
|
||||
<Badge variant="warning" size="small">
|
||||
{isExport ? "待导出" : "待入库"}
|
||||
</Badge>
|
||||
</td>
|
||||
<td className="py-2 max-w-0">
|
||||
<TruncatedText
|
||||
text={cid || "—"}
|
||||
className="font-mono text-xs text-gray-500"
|
||||
maxWidthClass="max-w-full"
|
||||
/>
|
||||
</td>
|
||||
<td className="py-2 px-2 whitespace-nowrap w-[7.5rem]">
|
||||
{cid && isExport && (
|
||||
<div className="inline-flex items-center justify-center gap-0.5">
|
||||
<Button size="small" variant="primary" className="!px-2" onClick={() => onExport(cid)}>
|
||||
导出
|
||||
</Button>
|
||||
<Button
|
||||
size="small"
|
||||
variant="default"
|
||||
className="!px-2"
|
||||
loading={importingId === cid}
|
||||
onClick={() => onImportVendor(cid)}
|
||||
>
|
||||
导入
|
||||
</Button>
|
||||
</div>
|
||||
)}
|
||||
{b.stage === "returned" && (
|
||||
<div className="inline-flex items-center justify-center gap-0.5">
|
||||
<Button
|
||||
size="small"
|
||||
variant="primary"
|
||||
className="!px-2"
|
||||
loading={buildingId === (cid || b.batch)}
|
||||
onClick={() => onSubmitBuild(b)}
|
||||
>
|
||||
入库
|
||||
</Button>
|
||||
<Link to="/system/audit">
|
||||
<Button size="small" variant="default" className="!px-2">
|
||||
审核
|
||||
</Button>
|
||||
</Link>
|
||||
</div>
|
||||
)}
|
||||
</td>
|
||||
</tr>
|
||||
);
|
||||
})}
|
||||
</tbody>
|
||||
</CompactTableShell>
|
||||
);
|
||||
@@ -0,0 +1,170 @@
|
||||
import React from "react";
|
||||
import { Link } from "react-router-dom";
|
||||
import { Badge } from "@/components/ui/Badge";
|
||||
|
||||
type ExportPipelineFlowProps = {
|
||||
pendingExport?: number;
|
||||
pendingBuild?: number;
|
||||
};
|
||||
|
||||
const Arrow: React.FC<{ className?: string }> = ({ className = "" }) => (
|
||||
<div className={`flex items-center justify-center shrink-0 text-gray-300 ${className}`} aria-hidden>
|
||||
<svg className="w-3.5 h-3.5 hidden lg:block" fill="none" viewBox="0 0 24 24" stroke="currentColor" strokeWidth={2}>
|
||||
<path strokeLinecap="round" strokeLinejoin="round" d="M13 7l5 5m0 0l-5 5m5-5H6" />
|
||||
</svg>
|
||||
<svg className="w-3.5 h-3.5 lg:hidden" fill="none" viewBox="0 0 24 24" stroke="currentColor" strokeWidth={2}>
|
||||
<path strokeLinecap="round" strokeLinejoin="round" d="M19 14l-7 7m0 0l-7-7m7 7V3" />
|
||||
</svg>
|
||||
</div>
|
||||
);
|
||||
|
||||
type StepProps = {
|
||||
step: number;
|
||||
title: string;
|
||||
subtitle: string;
|
||||
role: string;
|
||||
badge?: React.ReactNode;
|
||||
icon: React.ReactNode;
|
||||
highlight?: "current" | "done" | "upstream";
|
||||
link?: string;
|
||||
footnote?: string;
|
||||
};
|
||||
|
||||
const StepCard: React.FC<StepProps> = ({ step, title, subtitle, role, badge, icon, highlight = "upstream", link, footnote }) => {
|
||||
const shell =
|
||||
highlight === "current"
|
||||
? "border-amber-200 bg-gradient-to-br from-amber-50/90 to-orange-50/50 shadow-sm ring-1 ring-amber-100"
|
||||
: highlight === "done"
|
||||
? "border-emerald-200 bg-gradient-to-br from-emerald-50/80 to-teal-50/40 ring-1 ring-emerald-100"
|
||||
: "border-gray-200 bg-gray-50/60";
|
||||
|
||||
const content = (
|
||||
<div className={`relative flex flex-col rounded-xl border p-3 min-h-[6.75rem] flex-1 min-w-0 transition-shadow hover:shadow-md ${shell}`}>
|
||||
<div className="flex items-start justify-between gap-2 mb-1.5">
|
||||
<span
|
||||
className={`inline-flex h-5 w-5 items-center justify-center rounded-md text-[10px] font-bold ${
|
||||
highlight === "current"
|
||||
? "bg-amber-500 text-white"
|
||||
: highlight === "done"
|
||||
? "bg-emerald-500 text-white"
|
||||
: "bg-gray-200 text-gray-600"
|
||||
}`}
|
||||
>
|
||||
{step}
|
||||
</span>
|
||||
<span className="text-xs leading-none opacity-75 select-none">{icon}</span>
|
||||
</div>
|
||||
<h3 className="text-sm font-semibold text-gray-900 m-0">{title}</h3>
|
||||
<p className="text-[11px] text-gray-500 mt-1 leading-relaxed flex-1">{subtitle}</p>
|
||||
<div className="flex flex-wrap items-center gap-1.5 mt-2 pt-2 border-t border-black/5">
|
||||
<span className="text-[10px] uppercase tracking-wide text-gray-400 font-medium">{role}</span>
|
||||
{badge}
|
||||
</div>
|
||||
{footnote && <p className="text-[10px] text-gray-400 mt-1.5 m-0">{footnote}</p>}
|
||||
</div>
|
||||
);
|
||||
|
||||
if (link) {
|
||||
return (
|
||||
<Link to={link} className="flex-1 min-w-[9rem] max-w-[14rem] no-underline text-inherit group">
|
||||
<div className="group-hover:scale-[1.02] transition-transform">{content}</div>
|
||||
</Link>
|
||||
);
|
||||
}
|
||||
return <div className="flex-1 min-w-[9rem] max-w-[14rem]">{content}</div>;
|
||||
};
|
||||
|
||||
export const ExportPipelineFlow: React.FC<ExportPipelineFlowProps> = ({
|
||||
pendingExport = 0,
|
||||
pendingBuild = 0,
|
||||
}) => {
|
||||
return (
|
||||
<div className="mb-4 rounded-2xl border border-gray-200 bg-white overflow-hidden shadow-sm">
|
||||
<div className="px-4 py-3 border-b border-gray-100 bg-gradient-to-r from-slate-50 to-white flex flex-wrap items-center justify-between gap-2">
|
||||
<div>
|
||||
<h2 className="text-sm font-semibold text-gray-800 m-0">送标 → 入库 全流程</h2>
|
||||
<p className="text-xs text-gray-500 m-0 mt-0.5">本页负责第 3、4 步;前两步在其他模块完成</p>
|
||||
</div>
|
||||
<div className="flex items-center gap-2 text-xs">
|
||||
{pendingExport > 0 && (
|
||||
<span className="inline-flex items-center gap-1 rounded-full bg-amber-100 text-amber-800 px-2.5 py-1 font-medium">
|
||||
<span className="w-1.5 h-1.5 rounded-full bg-amber-500 animate-pulse" />
|
||||
{pendingExport} 待导出
|
||||
</span>
|
||||
)}
|
||||
{pendingBuild > 0 && (
|
||||
<span className="inline-flex items-center gap-1 rounded-full bg-orange-100 text-orange-800 px-2.5 py-1 font-medium">
|
||||
{pendingBuild} 待 build
|
||||
</span>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div className="p-4 lg:p-5">
|
||||
<div className="flex flex-col lg:flex-row lg:items-stretch gap-1 lg:gap-0">
|
||||
<StepCard
|
||||
step={1}
|
||||
title="提交质检"
|
||||
subtitle="标注员在标注进度页完成标注后提交"
|
||||
role="标注员"
|
||||
icon="📝"
|
||||
highlight="upstream"
|
||||
link="/labeling/campaigns"
|
||||
badge={<Badge variant="default" size="small">标注进度</Badge>}
|
||||
/>
|
||||
<Arrow className="py-1 lg:py-0 lg:px-1" />
|
||||
<StepCard
|
||||
step={2}
|
||||
title="质检通过"
|
||||
subtitle="协调员审核合格/可用,退回则返工"
|
||||
role="协调员"
|
||||
icon="✓"
|
||||
highlight="upstream"
|
||||
link="/labeling/review"
|
||||
badge={<Badge variant="warning" size="small">待导出</Badge>}
|
||||
/>
|
||||
<Arrow className="py-1 lg:py-0 lg:px-1" />
|
||||
<StepCard
|
||||
step={3}
|
||||
title="执行导出"
|
||||
subtitle="CVAT → 训练格式(DMS·YOLO / ADAS·quaternion_json)"
|
||||
role="本页操作"
|
||||
icon="📤"
|
||||
highlight="current"
|
||||
badge={<Badge variant="warning" size="small">待 build</Badge>}
|
||||
footnote={pendingExport > 0 ? `下方 ${pendingExport} 个批次可导出` : undefined}
|
||||
/>
|
||||
<Arrow className="py-1 lg:py-0 lg:px-1" />
|
||||
<StepCard
|
||||
step={4}
|
||||
title="提交 build"
|
||||
subtitle="审核队列批准后 merge 进训练包"
|
||||
role="审核员"
|
||||
icon="🏗"
|
||||
highlight="done"
|
||||
badge={<Badge variant="success" size="small">已入库</Badge>}
|
||||
footnote="待 build ≠ 已入库"
|
||||
/>
|
||||
</div>
|
||||
|
||||
<div className="mt-4 flex flex-wrap items-center gap-x-4 gap-y-2 rounded-xl bg-slate-50 border border-slate-100 px-3 py-2.5 text-[11px] text-gray-500">
|
||||
<span className="inline-flex items-center gap-1.5">
|
||||
<span className="w-2 h-2 rounded-full bg-gray-300" />
|
||||
上游步骤(可点击跳转)
|
||||
</span>
|
||||
<span className="inline-flex items-center gap-1.5">
|
||||
<span className="w-2 h-2 rounded-full bg-amber-400" />
|
||||
本页:格式转换 / 供应商回标
|
||||
</span>
|
||||
<span className="inline-flex items-center gap-1.5">
|
||||
<span className="w-2 h-2 rounded-full bg-emerald-500" />
|
||||
终点:ingested 不再显示于本列表
|
||||
</span>
|
||||
<Link to="/system/audit" className="ml-auto text-blue-600 hover:underline font-medium">
|
||||
审核队列 →
|
||||
</Link>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
};
|
||||
@@ -0,0 +1,59 @@
|
||||
import React from "react";
|
||||
|
||||
type FilterChip = { label: string; value: string };
|
||||
|
||||
type LabelingListToolbarProps = {
|
||||
search: string;
|
||||
onSearchChange: (value: string) => void;
|
||||
placeholder?: string;
|
||||
filters?: FilterChip[];
|
||||
filterValue?: string;
|
||||
onFilterChange?: (value: string) => void;
|
||||
total: number;
|
||||
extra?: React.ReactNode;
|
||||
};
|
||||
|
||||
export const LabelingListToolbar: React.FC<LabelingListToolbarProps> = ({
|
||||
search,
|
||||
onSearchChange,
|
||||
placeholder = "搜索批次/任务/项目...",
|
||||
filters,
|
||||
filterValue = "",
|
||||
onFilterChange,
|
||||
total,
|
||||
extra,
|
||||
}) => (
|
||||
<div className="bg-white rounded-xl border border-gray-200 p-3 mb-4">
|
||||
<div className="flex items-center gap-3 flex-wrap">
|
||||
<div className="flex-1 min-w-[200px] relative">
|
||||
<svg className="absolute left-3 top-1/2 -translate-y-1/2 w-4 h-4 text-gray-400" fill="none" stroke="currentColor" viewBox="0 0 24 24">
|
||||
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M21 21l-6-6m2-5a7 7 0 11-14 0 7 7 0 0114 0z" />
|
||||
</svg>
|
||||
<input
|
||||
className="w-full pl-9 pr-4 py-2 text-sm border border-gray-300 rounded-lg focus:ring-2 focus:ring-blue-500/20 focus:border-blue-500 outline-none"
|
||||
placeholder={placeholder}
|
||||
value={search}
|
||||
onChange={(e) => onSearchChange(e.target.value)}
|
||||
/>
|
||||
</div>
|
||||
{filters && onFilterChange && (
|
||||
<div className="flex gap-1.5">
|
||||
{filters.map((f) => (
|
||||
<button
|
||||
key={f.value}
|
||||
type="button"
|
||||
onClick={() => onFilterChange(f.value)}
|
||||
className={`px-3 py-1.5 rounded-lg text-xs font-medium transition-colors ${
|
||||
filterValue === f.value ? "bg-blue-600 text-white" : "bg-gray-100 text-gray-600 hover:bg-gray-200"
|
||||
}`}
|
||||
>
|
||||
{f.label}
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
)}
|
||||
{extra}
|
||||
<span className="text-xs text-gray-500 font-medium bg-gray-50 px-2.5 py-1 rounded-full">{total} 条</span>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
@@ -0,0 +1,89 @@
|
||||
import React from "react";
|
||||
import { Link } from "react-router-dom";
|
||||
import { Button } from "@/components/ui/Button";
|
||||
import { CompactTableShell } from "@/components/CompactTableShell";
|
||||
import { TruncatedText } from "@/components/TruncatedText";
|
||||
import { displayProjectFields, displayTaskFields } from "@/lib/labelingDisplay";
|
||||
|
||||
export type MyAssignmentRow = {
|
||||
campaign_id: string;
|
||||
batch: string;
|
||||
task: string;
|
||||
project?: string;
|
||||
status?: string;
|
||||
assigned: number;
|
||||
completed: number;
|
||||
pending: number;
|
||||
campaign_total?: number;
|
||||
};
|
||||
|
||||
type MyTasksTableProps = {
|
||||
items: MyAssignmentRow[];
|
||||
highlightId?: string | null;
|
||||
};
|
||||
|
||||
const COLS = ["30%", "12%", "8%", "10%", "14%", "auto", "6rem"];
|
||||
|
||||
export const MyTasksTable: React.FC<MyTasksTableProps> = ({ items, highlightId }) => (
|
||||
<CompactTableShell colWidths={COLS}>
|
||||
<thead>
|
||||
<tr>
|
||||
<th className="py-2">批次</th>
|
||||
<th className="py-2">任务</th>
|
||||
<th className="py-2">项目</th>
|
||||
<th className="py-2">待标</th>
|
||||
<th className="py-2">进度</th>
|
||||
<th className="py-2">Campaign</th>
|
||||
<th className="py-2">操作</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
{items.map((row) => {
|
||||
const pct = row.assigned > 0 ? Math.round((row.completed / row.assigned) * 100) : 0;
|
||||
const highlighted = highlightId === row.campaign_id;
|
||||
return (
|
||||
<tr
|
||||
key={row.campaign_id}
|
||||
className={`align-middle ${highlighted ? "bg-blue-50/60" : ""}`}
|
||||
>
|
||||
<td className="py-2 max-w-0">
|
||||
<TruncatedText text={row.batch || "—"} className="font-medium text-gray-900" maxWidthClass="max-w-full" />
|
||||
</td>
|
||||
<td className="py-2 max-w-0">
|
||||
<TruncatedText text={displayTaskFields(row)} maxWidthClass="max-w-full" />
|
||||
</td>
|
||||
<td className="py-2 whitespace-nowrap text-sm text-gray-700">{displayProjectFields(row)}</td>
|
||||
<td className="py-2 whitespace-nowrap">
|
||||
<span className="text-sm font-medium text-amber-600 tabular-nums">{row.pending}</span>
|
||||
</td>
|
||||
<td className="py-2 whitespace-nowrap">
|
||||
<div className="flex items-center justify-center gap-1.5">
|
||||
<div className="w-14 h-1.5 bg-gray-100 rounded-full overflow-hidden">
|
||||
<div
|
||||
className={`h-full rounded-full ${pct >= 100 ? "bg-green-500" : "bg-blue-500"}`}
|
||||
style={{ width: `${pct}%` }}
|
||||
/>
|
||||
</div>
|
||||
<span className="text-xs text-gray-500 tabular-nums">{row.completed}/{row.assigned}</span>
|
||||
</div>
|
||||
</td>
|
||||
<td className="py-2 max-w-0">
|
||||
<TruncatedText
|
||||
text={row.campaign_id || "—"}
|
||||
className="font-mono text-xs text-gray-500"
|
||||
maxWidthClass="max-w-full"
|
||||
/>
|
||||
</td>
|
||||
<td className="py-2 px-2 whitespace-nowrap w-[6rem]">
|
||||
<Link to={`/labeling/annotate/${encodeURIComponent(row.campaign_id)}`}>
|
||||
<Button size="small" variant="primary" className="!px-2">
|
||||
{row.pending > 0 ? "标注" : "查看"}
|
||||
</Button>
|
||||
</Link>
|
||||
</td>
|
||||
</tr>
|
||||
);
|
||||
})}
|
||||
</tbody>
|
||||
</CompactTableShell>
|
||||
);
|
||||
@@ -0,0 +1,110 @@
|
||||
import React from "react";
|
||||
import { Link } from "react-router-dom";
|
||||
import { Button } from "@/components/ui/Button";
|
||||
import { StageBadge } from "@/components/ui/Badge";
|
||||
import { CompactTableShell } from "@/components/CompactTableShell";
|
||||
import { TruncatedText } from "@/components/TruncatedText";
|
||||
import { displayProject, displayTask } from "@/lib/labelingDisplay";
|
||||
import type { LabelingBatchRow } from "@/lib/types";
|
||||
|
||||
type ReviewProgress = {
|
||||
total: number;
|
||||
reviewed: number;
|
||||
good: number;
|
||||
fine: number;
|
||||
bad: number;
|
||||
pass_rate: number;
|
||||
complete: boolean;
|
||||
stage?: string;
|
||||
};
|
||||
|
||||
type ReviewBatchTableProps = {
|
||||
batches: LabelingBatchRow[];
|
||||
progressMap: Record<string, ReviewProgress>;
|
||||
};
|
||||
|
||||
const COLS = ["30%", "12%", "8%", "10%", "14%", "auto", "7.5rem"];
|
||||
|
||||
export const ReviewBatchTable: React.FC<ReviewBatchTableProps> = ({ batches, progressMap }) => (
|
||||
<CompactTableShell colWidths={COLS}>
|
||||
<thead>
|
||||
<tr>
|
||||
<th className="py-2">批次</th>
|
||||
<th className="py-2">任务</th>
|
||||
<th className="py-2">项目</th>
|
||||
<th className="py-2">状态</th>
|
||||
<th className="py-2">进度</th>
|
||||
<th className="py-2">Campaign</th>
|
||||
<th className="py-2">操作</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
{batches.map((b) => {
|
||||
const cid = b.campaign_id || "";
|
||||
const prog = cid ? progressMap[cid] : undefined;
|
||||
const reviewPct = prog && prog.total > 0 ? Math.round((prog.reviewed / prog.total) * 100) : 0;
|
||||
const reviewDone = prog?.complete;
|
||||
const effectiveStage = reviewDone && prog?.stage ? prog.stage : b.stage;
|
||||
const passed = effectiveStage === "labeling_submitted" || (reviewDone && (prog?.pass_rate ?? 0) >= 80);
|
||||
return (
|
||||
<tr key={cid || b.batch} className="align-middle">
|
||||
<td className="py-2 max-w-0">
|
||||
<TruncatedText text={b.batch || "—"} className="font-medium text-gray-900" maxWidthClass="max-w-full" />
|
||||
</td>
|
||||
<td className="py-2 max-w-0">
|
||||
<TruncatedText text={displayTask(b)} maxWidthClass="max-w-full" />
|
||||
</td>
|
||||
<td className="py-2 whitespace-nowrap text-sm text-gray-700">{displayProject(b)}</td>
|
||||
<td className="py-2 whitespace-nowrap">
|
||||
<StageBadge stage={effectiveStage || ""} />
|
||||
</td>
|
||||
<td className="py-2 whitespace-nowrap">
|
||||
{prog && prog.total > 0 ? (
|
||||
<div className="flex items-center justify-center gap-1.5 min-w-0">
|
||||
<div className="w-14 h-1.5 bg-gray-100 rounded-full overflow-hidden shrink-0">
|
||||
<div
|
||||
className={`h-full rounded-full ${reviewDone ? "bg-green-500" : "bg-blue-500"}`}
|
||||
style={{ width: `${reviewPct}%` }}
|
||||
/>
|
||||
</div>
|
||||
<span className="text-xs text-gray-500 tabular-nums">{prog.reviewed}/{prog.total}</span>
|
||||
{reviewDone && (
|
||||
<span className={`text-xs ${passed ? "text-green-600" : "text-red-600"}`}>
|
||||
{prog.pass_rate}%
|
||||
</span>
|
||||
)}
|
||||
</div>
|
||||
) : (
|
||||
<span className="text-xs text-gray-400">—</span>
|
||||
)}
|
||||
</td>
|
||||
<td className="py-2 max-w-0">
|
||||
<TruncatedText text={cid || "—"} className="font-mono text-xs text-gray-500" maxWidthClass="max-w-full" />
|
||||
</td>
|
||||
<td className="py-2 px-2 whitespace-nowrap w-[7.5rem]">
|
||||
<div className="flex justify-center">
|
||||
{effectiveStage === "in_review" && cid && (
|
||||
<Link to={`/labeling/review/${cid}`}>
|
||||
<Button size="small" variant="primary" className="!px-2">
|
||||
{reviewDone ? "查看" : prog && prog.reviewed > 0 ? "继续" : "质检"}
|
||||
</Button>
|
||||
</Link>
|
||||
)}
|
||||
{effectiveStage === "labeling_submitted" && (
|
||||
<Link to="/labeling/export">
|
||||
<Button size="small" variant="success" className="!px-2">导出</Button>
|
||||
</Link>
|
||||
)}
|
||||
{effectiveStage === "review_rejected" && cid && (
|
||||
<Link to={`/labeling/review/${cid}`}>
|
||||
<Button size="small" variant="default" className="!px-2">记录</Button>
|
||||
</Link>
|
||||
)}
|
||||
</div>
|
||||
</td>
|
||||
</tr>
|
||||
);
|
||||
})}
|
||||
</tbody>
|
||||
</CompactTableShell>
|
||||
);
|
||||