单目3D初始代码

eval_tools/analysis/README_2D_FP_FN.md

@@ -0,0 +1,129 @@
+# 2D FP/FN 分析工具
+
+本目录用于分析单模型 2D 检测中的误检（FP）和漏检（FN），帮助把总指标拆成更可操作的问题类型。
+
+## 文件
+
+- `analyze_2d_fp_fn.py`
+  读取检测结果和 GT，复用现有 `Evaluator` / `parser` / `matcher`，输出 2D FP/FN 分类分析报告。
+- `export_2d_fp_fn_badcases.py`
+  从 `analysis_report.json` 中筛选指定类别、错误类型、距离或置信度范围，导出 badcase 清单。
+
+## FP 分类
+
+- `duplicate`
+  同类 GT 其实已经被更高分框匹配走了，当前框属于重复检出。
+- `class_confusion`
+  框和其他类别 GT 有较高 IoU，说明更像分类混淆。
+- `localization`
+  和同类 GT 靠得比较近，但 IoU 没过主阈值，更像框偏了。
+- `background`
+  附近没有足够重叠的 GT，更像背景误检或漏标。
+
+## FN 分类
+
+- `class_confusion`
+  其他类别的激活框和该 GT 重叠较高。
+- `low_score`
+  同类框位置够准，但分数低于工作点阈值。
+- `localization`
+  同类高分框在附近，但 IoU 没过主阈值。
+- `low_score_localization`
+  同类低分框在附近，但又低分又偏框。
+- `missing`
+  附近没有合理同类框。
+
+## 运行分析
+
+推荐直接复用现有评测配置：
+
+```bash
+source /deeplearning_team/ydong/dongying/miniconda/etc/profile.d/conda.sh
+conda activate yolov5
+
+python eval_tools/analysis/analyze_2d_fp_fn.py \
+  --config eval_tools/configs/eval_config_mono3d-roi0.yaml \
+  --classes vehicle pedestrian bicycle rider \
+  --near-iou-threshold 0.1 \
+  --output-dir eval_tools/analysis/results/mono3d_fp_fn
+```
+
+也可以直接传路径：
+
+```bash
+python eval_tools/analysis/analyze_2d_fp_fn.py \
+  --det-path /path/to/dets \
+  --gt-path /path/to/gts \
+  --det-format json \
+  --gt-format json \
+  --img-width 1920 \
+  --img-height 1080 \
+  --iou-threshold 0.5 \
+  --conf-threshold 0.4
+```
+
+## 输出
+
+运行后会生成：
+
+- `analysis_report.json`
+  完整结构化结果，包含 summary、top frames、FP/FN example 明细。
+- `analysis_report.txt`
+  适合快速查看的文本摘要。
+
+JSON 中最常用的字段：
+
+- `summary.fp_by_type`
+- `summary.fn_by_type`
+- `summary.per_class`
+- `top_frames`
+- `false_positive_examples`
+- `false_negative_examples`
+
+## 导出 badcase 清单
+
+例如导出 `vehicle` 的高置信背景误检：
+
+```bash
+python eval_tools/analysis/export_2d_fp_fn_badcases.py \
+  --input eval_tools/analysis/results/mono3d_fp_fn/analysis_report.json \
+  --mode fp \
+  --classes vehicle \
+  --error-types background \
+  --min-confidence 0.5 \
+  --top-k 200 \
+  --dedup-frame
+```
+
+例如导出 `pedestrian` 的低分漏检：
+
+```bash
+python eval_tools/analysis/export_2d_fp_fn_badcases.py \
+  --input eval_tools/analysis/results/mono3d_fp_fn/analysis_report.json \
+  --mode fn \
+  --classes pedestrian \
+  --error-types low_score low_score_localization \
+  --top-k 200
+```
+
+导出结果包括：
+
+- `*_badcases.json`
+  过滤后的明细
+- `*_badcases.txt`
+  文本摘要
+- `*_badcases_case_frame_list.txt`
+  每行一个 `case<TAB>frame`，方便接可视化或人工排查
+
+## 推荐排查流程
+
+1. 先看 `summary.fp_by_type` / `summary.fn_by_type`，确认主要矛盾是误检、漏检、分类混淆还是定位偏差。
+2. 再看 `summary.per_class`，确认问题集中在哪些类别。
+3. 用 `export_2d_fp_fn_badcases.py` 筛出某个错误桶的 top case。
+4. 把导出的 `case/frame` 清单接到你们现有可视化脚本里做人工复核。
+
+## 注意
+
+- 当前分析严格复用了评测时的 ROI GT 过滤和 `Matcher2D` 规则，因此口径会尽量和正式评测保持一致。
+- `background` 不能完全等价于“真的背景误检”，也可能包含漏标样本，最好配合人工复查。
+- `duplicate` 往往与 NMS、同物体多框、训练标签分布有关。

eval_tools/analysis/analyze_2d_fp_fn.sh

@@ -0,0 +1,8 @@
+NUM_WORKERS=${NUM_WORKERS:-32}
+
+python eval_tools/analysis/analyze_2d_fp_fn.py \
+  --config eval_tools/configs/eval_config_yolov26s-roi0.yaml \
+  --classes car \
+  --near-iou-threshold 0.1 \
+  --num-workers "${NUM_WORKERS}" \
+  --output-dir /data1/dongying/Mono3d/G1Q3/model_inference/KPI/DL_KPI_SCENE/model_20260403_analysis/analysis_2d/mono3d_fp_fn-roi0

eval_tools/analysis/analyze_3d.md

@@ -0,0 +1,5 @@
+evaluation_results/eval_results_yolo26s_768_20260407_DL_KPI_SCENE/yolo26s-20260407-conf0.27/20260410_162018_roi0/evaluation_report.json
+
+上述是一组模型评测报告，对于报告中的2d问题的进一步分析，目前有eval_tools/analysis/analyze_2d_fp_fn.py和eval_tools/analysis/visualize_2d_fn_cases.py脚本进行分析。
+
+现在期望有对3d指标的进一步分析，比如报告和可视化结果等。

eval_tools/analysis/analyze_3d_badcases.py

@@ -0,0 +1,799 @@
+#!/usr/bin/env python3
+"""
+Analyze 3D bad cases from saved detailed matches or by rebuilding matches.
+
+Preferred input is ``detailed_3d_matches.json`` produced by the evaluator.
+If that file is unavailable, this tool can rebuild the detailed matches from
+the evaluation config and the underlying det/gt directories.
+"""
+
+from __future__ import annotations
+
+import argparse
+import csv
+import json
+import math
+import sys
+from collections import Counter, defaultdict
+from datetime import datetime
+from pathlib import Path
+from statistics import mean, median
+
+
+REPO_ROOT = Path(__file__).resolve().parents[2]
+if str(REPO_ROOT) not in sys.path:
+    sys.path.insert(0, str(REPO_ROOT))
+
+from eval_tools.analysis.analyze_2d_fp_fn import (
+    build_config,
+    build_case_key,
+    class_name,
+    parse_class_ids,
+    round_float,
+)
+from eval_tools.evaluator.evaluator import Evaluator
+
+
+DEFAULT_DISTANCE_RANGES = [
+    [0, 10],
+    [10, 20],
+    [20, 30],
+    [30, 40],
+    [40, 50],
+    [50, 60],
+    [60, 70],
+    [70, 80],
+    [80, 90],
+    [90, 100],
+    [100, 999],
+]
+DEFAULT_LATERAL_DISTANCE_RANGES = [
+    [-50, -40],
+    [-40, -30],
+    [-30, -20],
+    [-20, -10],
+    [-10, 0],
+    [0, 10],
+    [10, 20],
+    [20, 30],
+    [30, 40],
+    [40, 50],
+]
+METRIC_KEYS = (
+    "lateral_error",
+    "longitudinal_error",
+    "longitudinal_relative_error",
+    "heading_error",
+    "heading_error_relaxed",
+    "reversal",
+)
+CSV_FIELDNAMES = [
+    "case_name",
+    "frame_name",
+    "class_id",
+    "class_name",
+    "gt_id",
+    "confidence",
+    "iou",
+    "distance_longitudinal_m",
+    "distance_lateral_m",
+    "distance_bin",
+    "lateral_bin",
+    "metric_name",
+    "metric_value",
+    "metric_value_display",
+    "is_reversal",
+    "lateral_error_m",
+    "longitudinal_error_m",
+    "longitudinal_relative_error",
+    "heading_error_rad",
+    "heading_error_deg",
+    "heading_error_relaxed_rad",
+    "heading_error_relaxed_deg",
+    "gt_bbox",
+    "det_bbox",
+    "gt_center_3d",
+    "det_center_3d",
+    "gt_rotation_rad",
+    "det_rotation_rad",
+]
+
+
+def parse_args():
+    parser = argparse.ArgumentParser(
+        description="Analyze 3D bad cases from detailed_3d_matches.json or rebuild them from config."
+    )
+    parser.add_argument(
+        "--detailed-matches",
+        type=str,
+        default=None,
+        help="Path to detailed_3d_matches.json generated by evaluator.",
+    )
+    parser.add_argument(
+        "--evaluation-report",
+        type=str,
+        default=None,
+        help="Path to evaluation_report.json. Used to infer sibling detailed_3d_matches.json.",
+    )
+    parser.add_argument("--config", type=str, default=None, help="Path to YAML evaluation config.")
+    parser.add_argument("--det-path", type=str, help="Detection results root directory")
+    parser.add_argument("--gt-path", type=str, help="Ground-truth labels root directory")
+    parser.add_argument("--path-depth", type=int, choices=[1, 2], help="Directory depth")
+    parser.add_argument(
+        "--det-format",
+        type=str,
+        choices=["auto", "json", "txt"],
+        help="Detection file format",
+    )
+    parser.add_argument(
+        "--gt-format",
+        type=str,
+        choices=["auto", "json", "txt"],
+        help="Ground-truth file format",
+    )
+    parser.add_argument("--img-width", type=int, help="Image width")
+    parser.add_argument("--img-height", type=int, help="Image height")
+    parser.add_argument(
+        "--coord-system",
+        type=str,
+        choices=["camera", "ego"],
+        help="Coordinate system used by the parser/evaluator",
+    )
+    parser.add_argument(
+        "--iou-threshold",
+        type=float,
+        help="IoU threshold used for evaluator loading",
+    )
+    parser.add_argument(
+        "--conf-threshold",
+        type=float,
+        help="Confidence threshold for rebuilding detailed matches",
+    )
+    parser.add_argument(
+        "--classes",
+        nargs="+",
+        default=None,
+        help="Optional class filter, e.g. car suv pedestrian or numeric IDs",
+    )
+    parser.add_argument(
+        "--metrics",
+        nargs="+",
+        default=list(METRIC_KEYS),
+        choices=METRIC_KEYS,
+        help="Metrics to rank and export.",
+    )
+    parser.add_argument(
+        "--top-k",
+        type=int,
+        default=200,
+        help="Top bad cases to keep per metric overall.",
+    )
+    parser.add_argument(
+        "--top-k-per-class",
+        type=int,
+        default=100,
+        help="Top bad cases to keep per metric and class.",
+    )
+    parser.add_argument(
+        "--top-k-frames",
+        type=int,
+        default=50,
+        help="Number of worst frames to keep in the summary.",
+    )
+    parser.add_argument(
+        "--min-confidence",
+        type=float,
+        default=None,
+        help="Optional minimum confidence filter on matched detections.",
+    )
+    parser.add_argument(
+        "--max-confidence",
+        type=float,
+        default=None,
+        help="Optional maximum confidence filter on matched detections.",
+    )
+    parser.add_argument(
+        "--min-iou",
+        type=float,
+        default=None,
+        help="Optional minimum 2D IoU filter on matched samples.",
+    )
+    parser.add_argument(
+        "--max-iou",
+        type=float,
+        default=None,
+        help="Optional maximum 2D IoU filter on matched samples.",
+    )
+    parser.add_argument(
+        "--bad-lateral-threshold",
+        type=float,
+        default=1.0,
+        help="Threshold in meters for counting bad lateral errors.",
+    )
+    parser.add_argument(
+        "--bad-longitudinal-threshold",
+        type=float,
+        default=3.0,
+        help="Threshold in meters for counting bad longitudinal errors.",
+    )
+    parser.add_argument(
+        "--bad-longitudinal-relative-threshold",
+        type=float,
+        default=0.2,
+        help="Threshold for counting bad longitudinal relative errors.",
+    )
+    parser.add_argument(
+        "--bad-heading-threshold-deg",
+        type=float,
+        default=15.0,
+        help="Threshold in degrees for counting bad heading errors.",
+    )
+    parser.add_argument(
+        "--num-workers",
+        type=int,
+        default=None,
+        help="Worker count for rebuilding detailed matches (default: evaluator auto-detect).",
+    )
+    parser.add_argument(
+        "--save-rebuilt-matches",
+        action="store_true",
+        help="When rebuilding detailed matches, also save them into the output directory.",
+    )
+    parser.add_argument(
+        "--output-dir",
+        type=str,
+        default=None,
+        help="Output directory. Defaults to eval_tools/analysis/results_3d/<timestamp>.",
+    )
+    return parser.parse_args()
+
+
+def metric_display_value(metric_name, sample):
+    if metric_name == "lateral_error":
+        return float(sample["lateral_error_m"])
+    if metric_name == "longitudinal_error":
+        return float(sample["longitudinal_error_m"])
+    if metric_name == "longitudinal_relative_error":
+        return float(sample["longitudinal_relative_error"])
+    if metric_name == "heading_error":
+        return float(sample["heading_error_deg"])
+    if metric_name == "heading_error_relaxed":
+        return float(sample["heading_error_relaxed_deg"])
+    if metric_name == "reversal":
+        return 1.0 if sample["is_reversal"] else 0.0
+    raise KeyError(f"Unsupported metric: {metric_name}")
+
+
+def metric_raw_value(metric_name, sample):
+    if metric_name == "lateral_error":
+        return float(sample["lateral_error_m"])
+    if metric_name == "longitudinal_error":
+        return float(sample["longitudinal_error_m"])
+    if metric_name == "longitudinal_relative_error":
+        return float(sample["longitudinal_relative_error"])
+    if metric_name == "heading_error":
+        return float(sample["heading_error_rad"])
+    if metric_name == "heading_error_relaxed":
+        return float(sample["heading_error_relaxed_rad"])
+    if metric_name == "reversal":
+        return 1.0 if sample["is_reversal"] else 0.0
+    raise KeyError(f"Unsupported metric: {metric_name}")
+
+
+def metric_unit(metric_name):
+    if metric_name in ("lateral_error", "longitudinal_error"):
+        return "m"
+    if metric_name in ("heading_error", "heading_error_relaxed"):
+        return "deg"
+    return ""
+
+
+def threshold_hit(metric_name, sample, thresholds):
+    if metric_name == "lateral_error":
+        return sample["lateral_error_m"] >= thresholds["bad_lateral_threshold"]
+    if metric_name == "longitudinal_error":
+        return sample["longitudinal_error_m"] >= thresholds["bad_longitudinal_threshold"]
+    if metric_name == "longitudinal_relative_error":
+        return sample["longitudinal_relative_error"] >= thresholds["bad_longitudinal_relative_threshold"]
+    if metric_name in ("heading_error", "heading_error_relaxed"):
+        limit = thresholds["bad_heading_threshold_deg"]
+        key = "heading_error_deg" if metric_name == "heading_error" else "heading_error_relaxed_deg"
+        return sample[key] >= limit
+    if metric_name == "reversal":
+        return bool(sample["is_reversal"])
+    raise KeyError(f"Unsupported metric: {metric_name}")
+
+
+def make_stats(values):
+    if not values:
+        return {"mean": 0.0, "median": 0.0, "std": 0.0, "percentile_90": 0.0}
+
+    values = [float(v) for v in values]
+    avg = mean(values)
+    med = median(values)
+    variance = sum((v - avg) ** 2 for v in values) / len(values)
+    values_sorted = sorted(values)
+    p90_index = min(len(values_sorted) - 1, max(0, math.ceil(0.9 * len(values_sorted)) - 1))
+    return {
+        "mean": round_float(avg),
+        "median": round_float(med),
+        "std": round_float(math.sqrt(variance)),
+        "percentile_90": round_float(values_sorted[p90_index]),
+    }
+
+
+def bucket_label(prefix, range_pair):
+    return f"{prefix}_{range_pair[0]}-{range_pair[1]}m"
+
+
+def build_distance_ranges(config):
+    metrics_cfg = config.get("metrics_3d", {}) if config else {}
+    return metrics_cfg.get("distance_ranges") or DEFAULT_DISTANCE_RANGES
+
+
+def build_lateral_ranges(config):
+    metrics_cfg = config.get("metrics_3d", {}) if config else {}
+    return metrics_cfg.get("lateral_distance_ranges") or DEFAULT_LATERAL_DISTANCE_RANGES
+
+
+def classify_range(value, prefix, ranges):
+    if value is None:
+        return None
+    for range_pair in ranges:
+        lo, hi = float(range_pair[0]), float(range_pair[1])
+        if lo <= float(value) < hi:
+            return bucket_label(prefix, [int(lo) if lo.is_integer() else lo, int(hi) if hi.is_integer() else hi])
+    return None
+
+
+def infer_detailed_matches_path(args):
+    if args.detailed_matches:
+        return Path(args.detailed_matches)
+    if args.evaluation_report:
+        report_path = Path(args.evaluation_report)
+        sibling = report_path.parent / "detailed_3d_matches.json"
+        if sibling.exists():
+            return sibling
+    return None
+
+
+def load_saved_detailed_matches(path):
+    with open(path, "r") as file:
+        return json.load(file)
+
+
+def rebuild_detailed_matches(args):
+    config = build_config(args)
+    evaluator = Evaluator(
+        config=config,
+        iou_threshold=float(config.get("matching", {}).get("iou_threshold", 0.5)),
+        num_workers=args.num_workers,
+        save_detailed_matches=True,
+    )
+    dataset_cfg = config["dataset"]
+    image_cfg = config["image"]
+    evaluator.load_data_from_paths(
+        det_root=dataset_cfg["det_path"],
+        gt_root=dataset_cfg["gt_path"],
+        img_width=image_cfg.get("width", 1920),
+        img_height=image_cfg.get("height", 1080),
+        path_depth=dataset_cfg.get("path_depth", 1),
+        det_format=dataset_cfg.get("det_format", "auto"),
+        gt_format=dataset_cfg.get("gt_format", "auto"),
+    )
+    evaluator.evaluate_3d()
+    return evaluator.detailed_3d_matches or {}, config
+
+
+def load_detailed_matches(args):
+    detailed_path = infer_detailed_matches_path(args)
+    config = build_config(args) if (
+        args.config
+        or args.det_path
+        or args.gt_path
+        or args.path_depth is not None
+        or args.det_format
+        or args.gt_format
+        or args.img_width is not None
+        or args.img_height is not None
+        or args.coord_system
+        or args.iou_threshold is not None
+        or args.conf_threshold is not None
+    ) else None
+
+    if detailed_path and detailed_path.exists():
+        return load_saved_detailed_matches(detailed_path), detailed_path, config
+
+    if config is None:
+        raise FileNotFoundError(
+            "Failed to locate detailed_3d_matches.json. Please provide --detailed-matches, "
+            "--evaluation-report with a sibling matches file, or --config to rebuild matches."
+        )
+
+    matches, rebuilt_config = rebuild_detailed_matches(args)
+    return matches, None, rebuilt_config
+
+
+def collect_samples(detailed_matches, class_ids, distance_ranges, lateral_ranges, args):
+    selected_class_names = {class_name(class_id) for class_id in class_ids}
+    samples = []
+    for case_name, frames in detailed_matches.items():
+        for frame_name, class_groups in frames.items():
+            for class_name_str, items in class_groups.items():
+                if class_name_str not in selected_class_names:
+                    continue
+                class_id = next((cid for cid in class_ids if class_name(cid) == class_name_str), None)
+                if class_id is None:
+                    continue
+                for item in items:
+                    confidence = float(item.get("confidence", 0.0))
+                    iou = float(item.get("iou", 0.0))
+                    if args.min_confidence is not None and confidence < args.min_confidence:
+                        continue
+                    if args.max_confidence is not None and confidence > args.max_confidence:
+                        continue
+                    if args.min_iou is not None and iou < args.min_iou:
+                        continue
+                    if args.max_iou is not None and iou > args.max_iou:
+                        continue
+
+                    distance = item.get("distance") or {}
+                    errors = item.get("errors") or {}
+                    longitudinal_distance = distance.get("longitudinal")
+                    lateral_distance = distance.get("lateral")
+                    sample = {
+                        "case_name": case_name,
+                        "frame_name": frame_name,
+                        "class_id": class_id,
+                        "class_name": class_name_str,
+                        "gt_id": str(item.get("gt_id")),
+                        "confidence": round_float(confidence),
+                        "iou": round_float(iou),
+                        "distance_longitudinal_m": None if longitudinal_distance is None else round_float(longitudinal_distance),
+                        "distance_lateral_m": None if lateral_distance is None else round_float(lateral_distance),
+                        "distance_bin": classify_range(longitudinal_distance, "long", distance_ranges),
+                        "lateral_bin": classify_range(lateral_distance, "lat", lateral_ranges),
+                        "gt_bbox": [round_float(v) for v in item.get("gt_bbox", [])],
+                        "det_bbox": [round_float(v) for v in item.get("det_bbox", [])],
+                        "gt_center_3d": [round_float(v) for v in item.get("gt_center_3d", [])],
+                        "det_center_3d": [round_float(v) for v in item.get("det_center_3d", [])],
+                        "gt_rotation_rad": round_float(item.get("gt_rotation", 0.0)),
+                        "det_rotation_rad": round_float(item.get("det_rotation", 0.0)),
+                        "lateral_error_m": round_float(errors.get("lateral", 0.0)),
+                        "longitudinal_error_m": round_float(errors.get("longitudinal", 0.0)),
+                        "longitudinal_relative_error": round_float(errors.get("longitudinal_relative", 0.0)),
+                        "heading_error_rad": round_float(errors.get("heading", 0.0)),
+                        "heading_error_deg": round_float(math.degrees(float(errors.get("heading", 0.0)))),
+                        "heading_error_relaxed_rad": round_float(errors.get("heading_relaxed", errors.get("heading", 0.0))),
+                        "heading_error_relaxed_deg": round_float(
+                            math.degrees(float(errors.get("heading_relaxed", errors.get("heading", 0.0))))
+                        ),
+                        "is_reversal": bool(errors.get("is_reversal", False)),
+                    }
+                    samples.append(sample)
+    return samples
+
+
+def summarize_metric(samples, metric_name, thresholds):
+    values = [metric_raw_value(metric_name, sample) for sample in samples]
+    summary = {
+        "stats": make_stats(values),
+        "bad_count": sum(1 for sample in samples if threshold_hit(metric_name, sample, thresholds)),
+        "bad_percentage": round_float(
+            100.0 * sum(1 for sample in samples if threshold_hit(metric_name, sample, thresholds)) / len(samples)
+        ) if samples else 0.0,
+    }
+    if metric_name == "reversal":
+        count = sum(1 for sample in samples if sample["is_reversal"])
+        summary = {
+            "count": count,
+            "percentage": round_float(100.0 * count / len(samples)) if samples else 0.0,
+        }
+    return summary
+
+
+def summarize_sample_group(samples, metrics, thresholds):
+    result = {"num_samples": len(samples)}
+    for metric_name in metrics:
+        result[metric_name] = summarize_metric(samples, metric_name, thresholds)
+    return result
+
+
+def build_top_frames(samples, metrics, thresholds, top_k_frames):
+    grouped = defaultdict(list)
+    for sample in samples:
+        grouped[(sample["case_name"], sample["frame_name"])].append(sample)
+
+    top_frames = []
+    for (case_name, frame_name), frame_samples in grouped.items():
+        bad_by_metric = {
+            metric_name: sum(1 for sample in frame_samples if threshold_hit(metric_name, sample, thresholds))
+            for metric_name in metrics
+        }
+        frame_record = {
+            "case_name": case_name,
+            "frame_name": frame_name,
+            "num_samples": len(frame_samples),
+            "bad_objects": sum(1 for sample in frame_samples if any(threshold_hit(metric_name, sample, thresholds) for metric_name in metrics)),
+            "bad_by_metric": bad_by_metric,
+            "mean_lateral_error_m": round_float(mean(sample["lateral_error_m"] for sample in frame_samples)),
+            "mean_longitudinal_error_m": round_float(mean(sample["longitudinal_error_m"] for sample in frame_samples)),
+            "mean_heading_error_deg": round_float(mean(sample["heading_error_deg"] for sample in frame_samples)),
+        }
+        top_frames.append(frame_record)
+
+    top_frames.sort(
+        key=lambda item: (
+            item["bad_objects"],
+            item["bad_by_metric"].get("reversal", 0),
+            item["mean_longitudinal_error_m"],
+            item["mean_heading_error_deg"],
+        ),
+        reverse=True,
+    )
+    return top_frames[:top_k_frames]
+
+
+def rank_key(metric_name, sample):
+    if metric_name == "reversal":
+        return (
+            1 if sample["is_reversal"] else 0,
+            sample["heading_error_deg"],
+            sample["confidence"],
+            sample["iou"],
+        )
+    return (
+        metric_display_value(metric_name, sample),
+        sample["confidence"],
+        sample["iou"],
+    )
+
+
+def sample_to_example(sample, metric_name):
+    example = dict(sample)
+    example["metric_name"] = metric_name
+    example["metric_value"] = round_float(metric_raw_value(metric_name, sample))
+    example["metric_value_display"] = round_float(metric_display_value(metric_name, sample))
+    example["metric_unit"] = metric_unit(metric_name)
+    return example
+
+
+def build_badcase_examples(samples, metrics, top_k, top_k_per_class):
+    badcase_examples = {}
+    badcase_examples_per_class = defaultdict(dict)
+
+    by_class = defaultdict(list)
+    for sample in samples:
+        by_class[sample["class_name"]].append(sample)
+
+    for metric_name in metrics:
+        ranked = sorted(samples, key=lambda sample: rank_key(metric_name, sample), reverse=True)
+        if metric_name == "reversal":
+            ranked = [sample for sample in ranked if sample["is_reversal"]]
+        badcase_examples[metric_name] = [sample_to_example(sample, metric_name) for sample in ranked[:top_k]]
+
+        for class_name_str, class_samples in by_class.items():
+            class_ranked = sorted(class_samples, key=lambda sample: rank_key(metric_name, sample), reverse=True)
+            if metric_name == "reversal":
+                class_ranked = [sample for sample in class_ranked if sample["is_reversal"]]
+            badcase_examples_per_class[class_name_str][metric_name] = [
+                sample_to_example(sample, metric_name) for sample in class_ranked[:top_k_per_class]
+            ]
+
+    return badcase_examples, dict(sorted(badcase_examples_per_class.items()))
+
+
+def build_bin_summary(samples, key_name, metrics, thresholds):
+    grouped = defaultdict(list)
+    for sample in samples:
+        bucket = sample.get(key_name)
+        if bucket:
+            grouped[bucket].append(sample)
+    return {
+        bucket: summarize_sample_group(bucket_samples, metrics, thresholds)
+        for bucket, bucket_samples in sorted(grouped.items())
+    }
+
+
+def write_csv_exports(output_dir, badcase_examples):
+    csv_dir = output_dir / "csv"
+    csv_dir.mkdir(parents=True, exist_ok=True)
+    for metric_name, examples in badcase_examples.items():
+        csv_path = csv_dir / f"top_{metric_name}.csv"
+        with open(csv_path, "w", newline="") as file:
+            writer = csv.DictWriter(file, fieldnames=CSV_FIELDNAMES)
+            writer.writeheader()
+            for example in examples:
+                row = {field: example.get(field) for field in CSV_FIELDNAMES}
+                writer.writerow(row)
+
+
+def write_markdown_report(report, output_path):
+    metadata = report["metadata"]
+    summary = report["summary"]
+    top_frames = report["top_frames"]
+    badcase_examples = report["badcase_examples"]
+
+    with open(output_path, "w") as file:
+        file.write("# 3D Badcase Analysis Report\n\n")
+
+        file.write("## Configuration\n\n")
+        file.write("| Item | Value |\n")
+        file.write("| --- | --- |\n")
+        for key in (
+            "source",
+            "detailed_matches_path",
+            "config_path",
+            "coord_system",
+            "iou_threshold",
+            "conf_threshold",
+            "classes",
+            "metrics",
+        ):
+            value = metadata.get(key)
+            if isinstance(value, list):
+                value = ", ".join(str(v) for v in value)
+            file.write(f"| {key} | `{value}` |\n")
+        file.write("\n")
+
+        file.write("## Overall Summary\n\n")
+        file.write("| Metric | Samples | Mean | P90 | Bad Count | Bad % |\n")
+        file.write("| --- | ---: | ---: | ---: | ---: | ---: |\n")
+        for metric_name in metadata["metrics"]:
+            item = summary["metrics"][metric_name]
+            if metric_name == "reversal":
+                file.write(
+                    f"| `{metric_name}` | {summary['num_samples']} | - | - | {item['count']} | {item['percentage']:.2f}% |\n"
+                )
+            else:
+                stats = item["stats"]
+                file.write(
+                    f"| `{metric_name}` | {summary['num_samples']} | {stats['mean']:.4f} | {stats['percentile_90']:.4f} "
+                    f"| {item['bad_count']} | {item['bad_percentage']:.2f}% |\n"
+                )
+        file.write("\n")
+
+        file.write("## Per-Class Summary\n\n")
+        file.write("| Class | Samples |")
+        for metric_name in metadata["metrics"]:
+            if metric_name == "reversal":
+                file.write(" Reversal % |")
+            else:
+                file.write(f" {metric_name} Mean |")
+        file.write("\n")
+        file.write("| --- | ---: |")
+        for _metric_name in metadata["metrics"]:
+            file.write(" ---: |")
+        file.write("\n")
+        for class_name_str, class_summary in report["per_class"].items():
+            file.write(f"| `{class_name_str}` | {class_summary['num_samples']} |")
+            for metric_name in metadata["metrics"]:
+                metric_summary = class_summary.get(metric_name, {})
+                if metric_name == "reversal":
+                    file.write(f" {metric_summary.get('percentage', 0.0):.2f}% |")
+                else:
+                    file.write(f" {metric_summary.get('stats', {}).get('mean', 0.0):.4f} |")
+            file.write("\n")
+        file.write("\n")
+
+        file.write("## Top Frames\n\n")
+        file.write("| Case / Frame | Samples | Bad Objects | Longitudinal Mean | Heading Mean(deg) |\n")
+        file.write("| --- | ---: | ---: | ---: | ---: |\n")
+        for item in top_frames:
+            file.write(
+                f"| `{item['case_name']}/{item['frame_name']}` | {item['num_samples']} | {item['bad_objects']} "
+                f"| {item['mean_longitudinal_error_m']:.4f} | {item['mean_heading_error_deg']:.4f} |\n"
+            )
+        file.write("\n")
+
+        file.write("## Top Badcases\n\n")
+        for metric_name, examples in badcase_examples.items():
+            file.write(f"### {metric_name}\n\n")
+            file.write("| Class | Case / Frame | Metric | Conf | IoU |\n")
+            file.write("| --- | --- | ---: | ---: | ---: |\n")
+            for example in examples[:10]:
+                file.write(
+                    f"| `{example['class_name']}` | `{example['case_name']}/{example['frame_name']}` | "
+                    f"{example['metric_value_display']:.4f} | {example['confidence']:.4f} | {example['iou']:.4f} |\n"
+                )
+            file.write("\n")
+
+
+def default_output_dir():
+    timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
+    return REPO_ROOT / "eval_tools" / "analysis" / "results_3d" / timestamp
+
+
+def main():
+    args = parse_args()
+    output_dir = Path(args.output_dir) if args.output_dir else default_output_dir()
+    output_dir.mkdir(parents=True, exist_ok=True)
+
+    detailed_matches, detailed_matches_path, config = load_detailed_matches(args)
+    if not detailed_matches:
+        raise RuntimeError("No detailed 3D matches were available for analysis.")
+
+    class_ids = parse_class_ids(args.classes)
+    thresholds = {
+        "bad_lateral_threshold": float(args.bad_lateral_threshold),
+        "bad_longitudinal_threshold": float(args.bad_longitudinal_threshold),
+        "bad_longitudinal_relative_threshold": float(args.bad_longitudinal_relative_threshold),
+        "bad_heading_threshold_deg": float(args.bad_heading_threshold_deg),
+    }
+    distance_ranges = build_distance_ranges(config or {})
+    lateral_ranges = build_lateral_ranges(config or {})
+    samples = collect_samples(detailed_matches, class_ids, distance_ranges, lateral_ranges, args)
+    if not samples:
+        raise RuntimeError("No 3D samples remained after filtering.")
+
+    metrics = list(args.metrics)
+    summary = {
+        "num_cases": len({sample["case_name"] for sample in samples}),
+        "num_frames": len({(sample["case_name"], sample["frame_name"]) for sample in samples}),
+        "num_samples": len(samples),
+        "metrics": {metric_name: summarize_metric(samples, metric_name, thresholds) for metric_name in metrics},
+    }
+    per_class = {}
+    by_class = defaultdict(list)
+    for sample in samples:
+        by_class[sample["class_name"]].append(sample)
+    for class_name_str, class_samples in sorted(by_class.items()):
+        per_class[class_name_str] = summarize_sample_group(class_samples, metrics, thresholds)
+
+    badcase_examples, badcase_examples_per_class = build_badcase_examples(
+        samples=samples,
+        metrics=metrics,
+        top_k=args.top_k,
+        top_k_per_class=args.top_k_per_class,
+    )
+    top_frames = build_top_frames(samples, metrics, thresholds, args.top_k_frames)
+
+    report = {
+        "metadata": {
+            "created_at": datetime.now().isoformat(timespec="seconds"),
+            "source": "detailed_3d_matches" if detailed_matches_path is not None else "rebuilt_from_config",
+            "detailed_matches_path": str(detailed_matches_path.resolve()) if detailed_matches_path else None,
+            "config_path": args.config,
+            "coord_system": (config or {}).get("metrics_3d", {}).get("coordinate_system", "camera"),
+            "iou_threshold": (config or {}).get("matching", {}).get("iou_threshold"),
+            "conf_threshold": (config or {}).get("metrics_3d", {}).get(
+                "conf_threshold",
+                (config or {}).get("metrics_2d", {}).get("conf_threshold"),
+            ),
+            "classes": [class_name(class_id) for class_id in class_ids],
+            "metrics": metrics,
+            "bad_thresholds": thresholds,
+            "distance_ranges": distance_ranges,
+            "lateral_distance_ranges": lateral_ranges,
+            "vehicle_size_split_3d": (config or {}).get("metrics_3d", {}).get("vehicle_size_split"),
+        },
+        "summary": summary,
+        "per_class": per_class,
+        "per_distance_bin": build_bin_summary(samples, "distance_bin", metrics, thresholds),
+        "per_lateral_bin": build_bin_summary(samples, "lateral_bin", metrics, thresholds),
+        "top_frames": top_frames,
+        "badcase_examples": badcase_examples,
+        "badcase_examples_per_class": badcase_examples_per_class,
+    }
+
+    report_path = output_dir / "analysis_report.json"
+    with open(report_path, "w") as file:
+        json.dump(report, file, indent=2)
+
+    markdown_path = output_dir / "analysis_report.md"
+    write_markdown_report(report, markdown_path)
+    write_csv_exports(output_dir, badcase_examples)
+
+    if args.save_rebuilt_matches and detailed_matches_path is None:
+        rebuilt_path = output_dir / "detailed_3d_matches.json"
+        with open(rebuilt_path, "w") as file:
+            json.dump(detailed_matches, file, indent=2)
+        print(f"Rebuilt detailed matches saved to: {rebuilt_path}")
+
+    print(f"JSON report saved to:      {report_path}")
+    print(f"Markdown report saved to:  {markdown_path}")
+    print(f"CSV exports saved to:      {output_dir / 'csv'}")
+
+
+if __name__ == "__main__":
+    main()

eval_tools/analysis/analyze_3d_badcases.sh

@@ -0,0 +1,24 @@
+#!/usr/bin/env bash
+
+set -euo pipefail
+
+EVALUATION_REPORT=${EVALUATION_REPORT:-evaluation_results/eval_results_yolo26s_768_20260407_DL_KPI_SCENE/yolo26s-20260407-conf0.27/20260412_155342_roi0/evaluation_report.json}
+DETAILED_MATCHES=${DETAILED_MATCHES:-$(dirname "${EVALUATION_REPORT}")/detailed_3d_matches.json}
+CONFIG_PATH=${CONFIG_PATH:-eval_tools/configs/eval_config_yolov26s-roi0.yaml}
+OUTPUT_DIR=${OUTPUT_DIR:-/data1/dongying/Mono3d/G1Q3/model_inference/KPI/DL_KPI_SCENE/model_20260403_analysis/analysis_3d}
+TOP_K=${TOP_K:-200}
+TOP_K_PER_CLASS=${TOP_K_PER_CLASS:-100}
+TOP_K_FRAMES=${TOP_K_FRAMES:-50}
+CLASSES=${CLASSES:-car suv van bus truck pedestrian bicycle}
+
+PYTHON_BIN=${PYTHON_BIN:-/deeplearning_team/ydong/dongying/miniconda/envs/dev/bin/python}
+
+"${PYTHON_BIN}" eval_tools/analysis/analyze_3d_badcases.py \
+  --evaluation-report "${EVALUATION_REPORT}" \
+  --detailed-matches "${DETAILED_MATCHES}" \
+  --config "${CONFIG_PATH}" \
+  --classes ${CLASSES} \
+  --top-k "${TOP_K}" \
+  --top-k-per-class "${TOP_K_PER_CLASS}" \
+  --top-k-frames "${TOP_K_FRAMES}" \
+  --output-dir "${OUTPUT_DIR}"

eval_tools/analysis/export_2d_fp_fn_badcases.py

@@ -0,0 +1,339 @@
+#!/usr/bin/env python3
+"""
+Export filtered badcase lists from analyze_2d_fp_fn.py results.
+
+The script reads ``analysis_report.json`` and produces:
+  - a filtered JSON file with matching examples
+  - a plain-text case/frame list for downstream visualization
+  - a compact text summary
+"""
+
+import argparse
+import json
+from collections import Counter, defaultdict
+from pathlib import Path
+
+
+def parse_args():
+    parser = argparse.ArgumentParser(
+        description="Filter and export 2D FP/FN badcases from analysis_report.json."
+    )
+    parser.add_argument(
+        "--input",
+        type=str,
+        required=True,
+        help="Path to analysis_report.json generated by analyze_2d_fp_fn.py",
+    )
+    parser.add_argument(
+        "--mode",
+        type=str,
+        default="both",
+        choices=["fp", "fn", "both"],
+        help="Which badcase pool to export",
+    )
+    parser.add_argument(
+        "--classes",
+        nargs="+",
+        default=None,
+        help="Optional class-name filter, e.g. vehicle pedestrian rider",
+    )
+    parser.add_argument(
+        "--error-types",
+        nargs="+",
+        default=None,
+        help="Optional error-type filter, e.g. duplicate localization low_score",
+    )
+    parser.add_argument(
+        "--min-confidence",
+        type=float,
+        default=None,
+        help="Minimum detection confidence for FP examples or matched dets in FN examples",
+    )
+    parser.add_argument(
+        "--max-confidence",
+        type=float,
+        default=None,
+        help="Maximum detection confidence for FP examples or matched dets in FN examples",
+    )
+    parser.add_argument(
+        "--min-distance",
+        type=float,
+        default=None,
+        help="Minimum target distance in metres",
+    )
+    parser.add_argument(
+        "--max-distance",
+        type=float,
+        default=None,
+        help="Maximum target distance in metres",
+    )
+    parser.add_argument(
+        "--min-best-iou",
+        type=float,
+        default=None,
+        help="Minimum best IoU field to keep an example",
+    )
+    parser.add_argument(
+        "--top-k",
+        type=int,
+        default=None,
+        help="Only keep the first K filtered examples after sorting",
+    )
+    parser.add_argument(
+        "--dedup-frame",
+        action="store_true",
+        help="Keep at most one example per case/frame/error/class combination",
+    )
+    parser.add_argument(
+        "--output-dir",
+        type=str,
+        default=None,
+        help="Output directory. Defaults to sibling folder next to the input report.",
+    )
+    return parser.parse_args()
+
+
+def normalize_tokens(values):
+    if not values:
+        return None
+    return {str(value).strip().lower() for value in values if str(value).strip()}
+
+
+def get_confidence(item):
+    if "confidence" in item and item["confidence"] is not None:
+        return float(item["confidence"])
+    if "best_det_confidence" in item and item["best_det_confidence"] is not None:
+        return float(item["best_det_confidence"])
+    return None
+
+
+def get_best_iou(item):
+    if "best_det_iou" in item and item["best_det_iou"] is not None:
+        return float(item["best_det_iou"])
+    return max(
+        float(item.get("best_same_class_iou", 0.0)),
+        float(item.get("best_other_class_iou", 0.0)),
+    )
+
+
+def passes_filters(item, class_filter, error_filter, args):
+    class_name = str(item.get("class_name", "")).lower()
+    error_type = str(item.get("error_type", "")).lower()
+
+    if class_filter and class_name not in class_filter:
+        return False
+    if error_filter and error_type not in error_filter:
+        return False
+
+    confidence = get_confidence(item)
+    if args.min_confidence is not None and (confidence is None or confidence < args.min_confidence):
+        return False
+    if args.max_confidence is not None and (confidence is None or confidence > args.max_confidence):
+        return False
+
+    distance = item.get("distance_m")
+    if args.min_distance is not None and (distance is None or float(distance) < args.min_distance):
+        return False
+    if args.max_distance is not None and (distance is None or float(distance) > args.max_distance):
+        return False
+
+    best_iou = get_best_iou(item)
+    if args.min_best_iou is not None and best_iou < args.min_best_iou:
+        return False
+
+    return True
+
+
+def rank_key(item):
+    confidence = get_confidence(item) or 0.0
+    best_iou = get_best_iou(item)
+    distance = item.get("distance_m")
+    distance = float(distance) if distance is not None else -1.0
+    return (confidence, best_iou, distance)
+
+
+def summarize(items):
+    by_error = Counter()
+    by_class = Counter()
+    by_class_error = defaultdict(Counter)
+
+    for item in items:
+        class_name = item.get("class_name", "unknown")
+        error_type = item.get("error_type", "unknown")
+        by_error[error_type] += 1
+        by_class[class_name] += 1
+        by_class_error[class_name][error_type] += 1
+
+    return {
+        "total_examples": len(items),
+        "by_error_type": dict(sorted(by_error.items())),
+        "by_class": dict(sorted(by_class.items())),
+        "by_class_and_error": {
+            class_name: dict(sorted(counter.items()))
+            for class_name, counter in sorted(by_class_error.items())
+        },
+    }
+
+
+def ensure_output_dir(input_path, output_dir):
+    if output_dir:
+        path = Path(output_dir)
+    else:
+        path = Path(input_path).resolve().parent / "exported_badcases"
+    path.mkdir(parents=True, exist_ok=True)
+    return path
+
+
+def write_summary(path, mode, filters, summary, items):
+    with open(path, "w") as file:
+        file.write("=" * 90 + "\n")
+        file.write("2D FP/FN BADCASE EXPORT SUMMARY\n")
+        file.write("=" * 90 + "\n\n")
+        file.write(f"Mode: {mode}\n")
+        file.write(f"Classes: {filters['classes']}\n")
+        file.write(f"Error types: {filters['error_types']}\n")
+        file.write(f"Min confidence: {filters['min_confidence']}\n")
+        file.write(f"Max confidence: {filters['max_confidence']}\n")
+        file.write(f"Min distance: {filters['min_distance']}\n")
+        file.write(f"Max distance: {filters['max_distance']}\n")
+        file.write(f"Min best IoU: {filters['min_best_iou']}\n")
+        file.write(f"Top K: {filters['top_k']}\n")
+        file.write(f"Dedup frame: {filters['dedup_frame']}\n\n")
+
+        file.write("SUMMARY\n")
+        file.write("-" * 90 + "\n")
+        file.write(f"Total examples: {summary['total_examples']}\n\n")
+
+        file.write("BY ERROR TYPE\n")
+        file.write("-" * 90 + "\n")
+        for key, value in summary["by_error_type"].items():
+            file.write(f"{key:<24} {value}\n")
+        file.write("\n")
+
+        file.write("BY CLASS\n")
+        file.write("-" * 90 + "\n")
+        for key, value in summary["by_class"].items():
+            file.write(f"{key:<24} {value}\n")
+        file.write("\n")
+
+        file.write("TOP EXAMPLES\n")
+        file.write("-" * 90 + "\n")
+        for item in items[:50]:
+            confidence = get_confidence(item)
+            best_iou = get_best_iou(item)
+            file.write(
+                f"{item.get('case_name')}/{item.get('frame_name')}  "
+                f"{item.get('class_name')}  {item.get('error_type')}  "
+                f"conf={confidence if confidence is not None else '-'}  "
+                f"best_iou={best_iou:.4f}\n"
+            )
+
+
+def write_case_frame_list(path, items):
+    seen = set()
+    with open(path, "w") as file:
+        for item in items:
+            key = (item.get("case_name"), item.get("frame_name"))
+            if key in seen:
+                continue
+            seen.add(key)
+            file.write(f"{key[0]}\t{key[1]}\n")
+
+
+def main():
+    args = parse_args()
+
+    with open(args.input, "r") as file:
+        data = json.load(file)
+
+    pools = []
+    if args.mode in ("fp", "both"):
+        pools.extend(data.get("false_positive_examples", []))
+    if args.mode in ("fn", "both"):
+        pools.extend(data.get("false_negative_examples", []))
+
+    class_filter = normalize_tokens(args.classes)
+    error_filter = normalize_tokens(args.error_types)
+
+    filtered = [
+        item for item in pools if passes_filters(item, class_filter, error_filter, args)
+    ]
+
+    filtered.sort(key=rank_key, reverse=True)
+
+    if args.dedup_frame:
+        deduped = []
+        seen = set()
+        for item in filtered:
+            key = (
+                item.get("case_name"),
+                item.get("frame_name"),
+                item.get("class_name"),
+                item.get("error_type"),
+            )
+            if key in seen:
+                continue
+            seen.add(key)
+            deduped.append(item)
+        filtered = deduped
+
+    if args.top_k is not None:
+        filtered = filtered[: args.top_k]
+
+    summary = summarize(filtered)
+    output_dir = ensure_output_dir(args.input, args.output_dir)
+
+    base_name = f"{args.mode}_badcases"
+    json_path = output_dir / f"{base_name}.json"
+    txt_path = output_dir / f"{base_name}.txt"
+    list_path = output_dir / f"{base_name}_case_frame_list.txt"
+
+    with open(json_path, "w") as file:
+        json.dump(
+            {
+                "source": str(Path(args.input).resolve()),
+                "mode": args.mode,
+                "filters": {
+                    "classes": sorted(class_filter) if class_filter else None,
+                    "error_types": sorted(error_filter) if error_filter else None,
+                    "min_confidence": args.min_confidence,
+                    "max_confidence": args.max_confidence,
+                    "min_distance": args.min_distance,
+                    "max_distance": args.max_distance,
+                    "min_best_iou": args.min_best_iou,
+                    "top_k": args.top_k,
+                    "dedup_frame": args.dedup_frame,
+                },
+                "summary": summary,
+                "examples": filtered,
+            },
+            file,
+            indent=2,
+        )
+
+    write_summary(
+        txt_path,
+        args.mode,
+        {
+            "classes": sorted(class_filter) if class_filter else None,
+            "error_types": sorted(error_filter) if error_filter else None,
+            "min_confidence": args.min_confidence,
+            "max_confidence": args.max_confidence,
+            "min_distance": args.min_distance,
+            "max_distance": args.max_distance,
+            "min_best_iou": args.min_best_iou,
+            "top_k": args.top_k,
+            "dedup_frame": args.dedup_frame,
+        },
+        summary,
+        filtered,
+    )
+    write_case_frame_list(list_path, filtered)
+
+    print(f"Filtered JSON saved to: {json_path}")
+    print(f"Summary text saved to:  {txt_path}")
+    print(f"Case/frame list saved to: {list_path}")
+
+
+if __name__ == "__main__":
+    main()

eval_tools/analysis/visualize_2d_fn_cases.py

@@ -0,0 +1,783 @@
+#!/usr/bin/env python3
+"""
+Visualize 2D FN cases from analyze_2d_fp_fn.py results on source images.
+
+This script is designed for image-based inspection of false negatives,
+especially FN-localization. It reads ``analysis_report.json``, reloads the
+corresponding GT/detections using the same evaluator pipeline, and saves:
+
+  - frame-level overlays (all GT / active detections / highlighted FN targets)
+  - per-example panels (full-frame + local crop)
+  - a summary index JSON
+"""
+
+import argparse
+import json
+import sys
+from collections import defaultdict
+from pathlib import Path
+
+import cv2
+import numpy as np
+
+
+REPO_ROOT = Path(__file__).resolve().parents[2]
+if str(REPO_ROOT) not in sys.path:
+    sys.path.insert(0, str(REPO_ROOT))
+
+from eval_tools.analysis.analyze_2d_fp_fn import build_config, class_name, parse_class_ids
+from eval_tools.evaluator.evaluator import Evaluator
+
+
+BOX_COLORS = {
+    "gt_all": (80, 220, 80),
+    # Keep normal detections visually quiet so highlighted error targets stand out.
+    "det_all": (150, 150, 150),
+    "fn_gt": (40, 40, 255),
+    "fn_det": (0, 215, 255),
+    "fp_det": (255, 0, 220),
+    "fp_ref_gt": (255, 255, 0),
+    "title_bg": (30, 30, 30),
+}
+
+
+def parse_args():
+    parser = argparse.ArgumentParser(
+        description="Visualize FN cases from analysis_report.json on source images."
+    )
+    parser.add_argument(
+        "--analysis-report",
+        type=str,
+        required=True,
+        help="Path to analysis_report.json generated by analyze_2d_fp_fn.py",
+    )
+    parser.add_argument("--config", type=str, help="Path to YAML evaluation config")
+    parser.add_argument("--det-path", type=str, help="Detection results root directory")
+    parser.add_argument("--gt-path", type=str, help="Ground-truth labels root directory")
+    parser.add_argument("--path-depth", type=int, choices=[1, 2], help="Directory depth")
+    parser.add_argument(
+        "--det-format",
+        type=str,
+        choices=["auto", "json", "txt"],
+        help="Detection file format",
+    )
+    parser.add_argument(
+        "--gt-format",
+        type=str,
+        choices=["auto", "json", "txt"],
+        help="Ground-truth file format",
+    )
+    parser.add_argument("--img-width", type=int, help="Image width")
+    parser.add_argument("--img-height", type=int, help="Image height")
+    parser.add_argument(
+        "--coord-system",
+        type=str,
+        choices=["camera", "ego"],
+        help="Coordinate system used by the parser/evaluator",
+    )
+    parser.add_argument(
+        "--iou-threshold",
+        type=float,
+        help="IoU threshold used for evaluator loading",
+    )
+    parser.add_argument(
+        "--conf-threshold",
+        type=float,
+        help="Confidence threshold for active detections shown on overlays",
+    )
+    parser.add_argument(
+        "--mode",
+        type=str,
+        default="fn",
+        choices=["fn", "fp", "both"],
+        help="Which example pool to visualize",
+    )
+    parser.add_argument(
+        "--error-types",
+        nargs="+",
+        default=["localization"],
+        help="Error types to visualize. Default: localization",
+    )
+    parser.add_argument(
+        "--classes",
+        nargs="+",
+        default=None,
+        help="Optional class filter, e.g. vehicle pedestrian bicycle rider",
+    )
+    parser.add_argument(
+        "--case-names",
+        nargs="+",
+        default=None,
+        help="Optional case-name filter",
+    )
+    parser.add_argument(
+        "--min-confidence",
+        type=float,
+        default=None,
+        help="Minimum confidence for the associated detection (best det for FN, det confidence for FP)",
+    )
+    parser.add_argument(
+        "--max-confidence",
+        type=float,
+        default=None,
+        help="Maximum confidence for the associated detection",
+    )
+    parser.add_argument(
+        "--min-distance",
+        type=float,
+        default=None,
+        help="Minimum target distance in metres",
+    )
+    parser.add_argument(
+        "--max-distance",
+        type=float,
+        default=None,
+        help="Maximum target distance in metres",
+    )
+    parser.add_argument(
+        "--max-best-iou",
+        type=float,
+        default=None,
+        help="Maximum best IoU. Useful for focusing on badly localized examples.",
+    )
+    parser.add_argument(
+        "--top-k",
+        type=int,
+        default=200,
+        help="Maximum number of examples to visualize after filtering",
+    )
+    parser.add_argument(
+        "--dedup-frame",
+        action="store_true",
+        help="Keep at most one example per case/frame/class/error combination",
+    )
+    parser.add_argument(
+        "--line-thickness",
+        type=int,
+        default=2,
+        help="Base line thickness for non-highlight boxes",
+    )
+    parser.add_argument(
+        "--crop-scale",
+        type=float,
+        default=1.8,
+        help="Expand crop window around GT/det union box by this factor",
+    )
+    parser.add_argument(
+        "--jpeg-quality",
+        type=int,
+        default=92,
+        help="JPEG quality for saved visualizations",
+    )
+    parser.add_argument(
+        "--output-dir",
+        type=str,
+        default=None,
+        help="Output directory. Defaults to evaluation_results/fn_vis_<report_name>",
+    )
+    return parser.parse_args()
+
+
+def get_confidence(example):
+    if example.get("confidence") is not None:
+        return float(example["confidence"])
+    if example.get("best_det_confidence") is not None:
+        return float(example["best_det_confidence"])
+    return None
+
+
+def get_best_iou(example):
+    if example.get("best_det_iou") is not None:
+        return float(example["best_det_iou"])
+    return max(
+        float(example.get("best_same_class_iou", 0.0)),
+        float(example.get("best_other_class_iou", 0.0)),
+    )
+
+
+def normalize_token_set(values):
+    if not values:
+        return None
+    return {str(v).strip().lower() for v in values if str(v).strip()}
+
+
+def rank_examples(examples):
+    def key(item):
+        conf = get_confidence(item) or 0.0
+        best_iou = get_best_iou(item)
+        distance = item.get("distance_m")
+        distance = float(distance) if distance is not None else -1.0
+        area = float(item.get("gt_bbox_area", item.get("det_bbox_area", 0.0)) or 0.0)
+        return (conf, -best_iou, area, distance)
+
+    return sorted(examples, key=key, reverse=True)
+
+
+def filter_examples(report, args):
+    pools = []
+    if args.mode in ("fn", "both"):
+        pools.extend(report.get("false_negative_examples", []))
+    if args.mode in ("fp", "both"):
+        pools.extend(report.get("false_positive_examples", []))
+
+    class_filter = normalize_token_set(args.classes)
+    error_filter = normalize_token_set(args.error_types)
+    case_filter = set(args.case_names) if args.case_names else None
+
+    filtered = []
+    for item in pools:
+        class_str = str(item.get("class_name", "")).lower()
+        error_type = str(item.get("error_type", "")).lower()
+        case_name = item.get("case_name")
+        conf = get_confidence(item)
+        distance = item.get("distance_m")
+        best_iou = get_best_iou(item)
+
+        if class_filter and class_str not in class_filter:
+            continue
+        if error_filter and error_type not in error_filter:
+            continue
+        if case_filter and case_name not in case_filter:
+            continue
+        if args.min_confidence is not None and (conf is None or conf < args.min_confidence):
+            continue
+        if args.max_confidence is not None and (conf is None or conf > args.max_confidence):
+            continue
+        if args.min_distance is not None and (distance is None or float(distance) < args.min_distance):
+            continue
+        if args.max_distance is not None and (distance is None or float(distance) > args.max_distance):
+            continue
+        if args.max_best_iou is not None and best_iou > args.max_best_iou:
+            continue
+        filtered.append(item)
+
+    filtered = rank_examples(filtered)
+
+    if args.dedup_frame:
+        deduped = []
+        seen = set()
+        for item in filtered:
+            key = (
+                item.get("case_name"),
+                item.get("frame_name"),
+                item.get("class_name"),
+                item.get("error_type"),
+            )
+            if key in seen:
+                continue
+            seen.add(key)
+            deduped.append(item)
+        filtered = deduped
+
+    if args.top_k is not None:
+        filtered = filtered[: args.top_k]
+
+    return filtered
+
+
+def bbox_to_int(bbox):
+    return [int(round(float(v))) for v in bbox]
+
+
+def get_example_gt_bbox(example):
+    return example.get("gt_bbox")
+
+
+def get_example_det_bbox(example):
+    if example.get("best_det_bbox") is not None:
+        return example.get("best_det_bbox")
+    return example.get("det_bbox")
+
+
+def is_fn_example(example):
+    return example.get("gt_bbox") is not None
+
+
+def parse_generated_gt_index(gt_id):
+    if not gt_id:
+        return None
+    gt_id = str(gt_id)
+    if gt_id.startswith("gt_") and gt_id[3:].isdigit():
+        return int(gt_id[3:])
+    return None
+
+
+def resolve_reference_gt(example, gts):
+    if not gts:
+        return None, None, None
+
+    def find_by_explicit_id(target_id):
+        if target_id is None:
+            return None
+        for gt in gts:
+            if gt.get("id") is not None and str(gt.get("id")) == str(target_id):
+                return gt
+        return None
+
+    best_same_gt_id = example.get("best_same_gt_id")
+    best_other_gt_id = example.get("best_other_gt_id")
+
+    gt = find_by_explicit_id(best_same_gt_id)
+    if gt is not None:
+        return gt.get("bbox_2d"), class_name(gt["label"]), best_same_gt_id
+
+    gt = find_by_explicit_id(best_other_gt_id)
+    if gt is not None:
+        return gt.get("bbox_2d"), class_name(gt["label"]), best_other_gt_id
+
+    same_idx = parse_generated_gt_index(best_same_gt_id)
+    if same_idx is not None:
+        same_class_gts = [gt for gt in gts if gt["label"] == example.get("class_id")]
+        if 0 <= same_idx < len(same_class_gts):
+            gt = same_class_gts[same_idx]
+            return gt.get("bbox_2d"), class_name(gt["label"]), best_same_gt_id
+
+    other_idx = parse_generated_gt_index(best_other_gt_id)
+    if other_idx is not None and 0 <= other_idx < len(gts):
+        gt = gts[other_idx]
+        return gt.get("bbox_2d"), class_name(gt["label"]), best_other_gt_id
+
+    return None, None, None
+
+
+def get_target_box_color(example, kind):
+    if is_fn_example(example):
+        return BOX_COLORS["fn_gt"] if kind == "gt" else BOX_COLORS["fn_det"]
+    if kind == "det":
+        return BOX_COLORS["fp_det"]
+    return BOX_COLORS["fp_ref_gt"]
+
+
+def draw_box(image, bbox, color, label=None, thickness=2):
+    x1, y1, x2, y2 = bbox_to_int(bbox)
+    cv2.rectangle(image, (x1, y1), (x2, y2), color, thickness, cv2.LINE_AA)
+    if label:
+        (tw, th), _ = cv2.getTextSize(label, cv2.FONT_HERSHEY_SIMPLEX, 0.55, 1)
+        y_text = max(0, y1 - th - 8)
+        cv2.rectangle(image, (x1, y_text), (x1 + tw + 8, y_text + th + 8), color, -1)
+        cv2.putText(
+            image,
+            label,
+            (x1 + 4, y_text + th + 2),
+            cv2.FONT_HERSHEY_SIMPLEX,
+            0.55,
+            (255, 255, 255),
+            1,
+            cv2.LINE_AA,
+        )
+
+
+def add_header(image, text):
+    h, w = image.shape[:2]
+    overlay = image.copy()
+    cv2.rectangle(overlay, (0, 0), (w, 42), BOX_COLORS["title_bg"], -1)
+    cv2.addWeighted(overlay, 0.55, image, 0.45, 0, image)
+    cv2.putText(
+        image,
+        text,
+        (10, 28),
+        cv2.FONT_HERSHEY_SIMPLEX,
+        0.7,
+        (255, 255, 255),
+        2,
+        cv2.LINE_AA,
+    )
+
+
+def make_crop(image, boxes, scale=1.8):
+    h, w = image.shape[:2]
+    valid = [bbox for bbox in boxes if bbox is not None]
+    if not valid:
+        return image.copy(), (0, 0)
+
+    x1 = min(float(b[0]) for b in valid)
+    y1 = min(float(b[1]) for b in valid)
+    x2 = max(float(b[2]) for b in valid)
+    y2 = max(float(b[3]) for b in valid)
+
+    cx = 0.5 * (x1 + x2)
+    cy = 0.5 * (y1 + y2)
+    bw = max(32.0, (x2 - x1) * scale)
+    bh = max(32.0, (y2 - y1) * scale)
+
+    crop_x1 = max(0, int(round(cx - bw / 2)))
+    crop_y1 = max(0, int(round(cy - bh / 2)))
+    crop_x2 = min(w, int(round(cx + bw / 2)))
+    crop_y2 = min(h, int(round(cy + bh / 2)))
+
+    return image[crop_y1:crop_y2, crop_x1:crop_x2].copy(), (crop_x1, crop_y1)
+
+
+def draw_crop_panel(image, example, gts, crop_scale):
+    gt_bbox = get_example_gt_bbox(example)
+    det_bbox = get_example_det_bbox(example)
+    ref_gt_bbox, ref_gt_class, ref_gt_id = resolve_reference_gt(example, gts)
+    crop, (off_x, off_y) = make_crop(
+        image, [gt_bbox, det_bbox, ref_gt_bbox], scale=crop_scale
+    )
+
+    def shift_box(box):
+        if box is None:
+            return None
+        return [
+            float(box[0]) - off_x,
+            float(box[1]) - off_y,
+            float(box[2]) - off_x,
+            float(box[3]) - off_y,
+        ]
+
+    gt_local = shift_box(gt_bbox)
+    det_local = shift_box(det_bbox)
+    ref_gt_local = shift_box(ref_gt_bbox)
+
+    if gt_local is not None:
+        draw_box(
+            crop,
+            gt_local,
+            get_target_box_color(example, "gt"),
+            label=f"GT {example['class_name']}",
+            thickness=3,
+        )
+    elif ref_gt_local is not None:
+        draw_box(
+            crop,
+            ref_gt_local,
+            get_target_box_color(example, "gt"),
+            label=f"RefGT {ref_gt_class or '-'}",
+            thickness=3,
+        )
+
+    if det_local is not None:
+        conf = get_confidence(example)
+        iou = get_best_iou(example)
+        if example.get("best_det_bbox") is not None:
+            label = f"BestDet {example.get('best_det_class', '-')}"
+            if conf is not None:
+                label += f" {conf:.2f}"
+            label += f" IoU {iou:.3f}"
+        else:
+            label = f"FP Det {example.get('class_name', '-')}"
+            if conf is not None:
+                label += f" {conf:.2f}"
+            label += f" IoU {iou:.3f}"
+        draw_box(crop, det_local, get_target_box_color(example, "det"), label=label, thickness=3)
+
+    add_header(
+        crop,
+        f"crop | {'FN' if is_fn_example(example) else 'FP'} | {example['class_name']} | {example['error_type']} | dist={example.get('distance_m')}",
+    )
+    return crop
+
+
+def add_sidebar(panel, example):
+    h, _ = panel.shape[:2]
+    sidebar = np.full((h, 360, 3), 28, dtype=np.uint8)
+    lines = [
+        f"case: {example.get('case_name')}",
+        f"frame: {example.get('frame_name')}",
+        f"class: {example.get('class_name')}",
+        f"error: {example.get('error_type')}",
+        f"mode: {'fn' if is_fn_example(example) else 'fp'}",
+        f"gt_id: {example.get('gt_id', '-')}",
+        f"ref_gt_id: {example.get('best_same_gt_id') or example.get('best_other_gt_id') or '-'}",
+        f"best_det_id: {example.get('best_det_id', '-')}",
+        f"best_det_cls: {example.get('best_det_class', '-')}",
+        f"det_id: {example.get('det_id', '-')}",
+        f"conf: {get_confidence(example)}",
+        f"best_iou: {get_best_iou(example):.4f}",
+        f"distance_m: {example.get('distance_m')}",
+        f"lateral_m: {example.get('lateral_m')}",
+        f"gt_area: {example.get('gt_bbox_area')}",
+        f"det_area: {example.get('det_bbox_area')}",
+    ]
+
+    y = 36
+    for line in lines:
+        cv2.putText(
+            sidebar,
+            str(line),
+            (12, y),
+            cv2.FONT_HERSHEY_SIMPLEX,
+            0.56,
+            (235, 235, 235),
+            1,
+            cv2.LINE_AA,
+        )
+        y += 30
+
+    return np.hstack([panel, sidebar])
+
+
+def resize_to_height(image, target_height):
+    h, w = image.shape[:2]
+    if h == target_height:
+        return image
+    scale = target_height / max(h, 1)
+    return cv2.resize(image, (max(1, int(round(w * scale))), target_height))
+
+
+def combine_full_and_crop(full_image, crop_image, example):
+    target_h = max(full_image.shape[0], crop_image.shape[0])
+    full_resized = resize_to_height(full_image, target_h)
+    crop_resized = resize_to_height(crop_image, target_h)
+    panel = np.hstack([full_resized, crop_resized])
+    return add_sidebar(panel, example)
+
+
+def find_pair_map(config):
+    evaluator = Evaluator(
+        config=config,
+        iou_threshold=float(config.get("matching", {}).get("iou_threshold", 0.5)),
+        num_workers=1,
+        save_detailed_matches=False,
+    )
+    dataset_cfg = config["dataset"]
+    image_cfg = config["image"]
+    evaluator.load_data_from_paths(
+        det_root=dataset_cfg["det_path"],
+        gt_root=dataset_cfg["gt_path"],
+        img_width=image_cfg.get("width", 1920),
+        img_height=image_cfg.get("height", 1080),
+        path_depth=dataset_cfg.get("path_depth", 1),
+        det_format=dataset_cfg.get("det_format", "auto"),
+        gt_format=dataset_cfg.get("gt_format", "auto"),
+    )
+
+    pair_map = {}
+    for pair in evaluator.image_pairs:
+        level1_name = pair.get("level1_name")
+        if level1_name:
+            case_key = f"{level1_name}/{pair['case']}"
+        else:
+            case_key = pair["case"]
+        pair_map[(case_key, pair["frame"])] = pair
+    return pair_map, evaluator
+
+
+def find_image_path(pair):
+    gt_file = Path(pair["gt_file"])
+    case_dir = gt_file.parent.parent
+    images_dir = case_dir / "images"
+    stem = gt_file.stem
+    for suffix in (".png", ".jpg", ".jpeg", ".bmp"):
+        candidate = images_dir / f"{stem}{suffix}"
+        if candidate.exists():
+            return candidate
+    matches = list(images_dir.glob(f"{stem}.*"))
+    return matches[0] if matches else None
+
+
+def render_frame_overlay(image, gts, active_dets, frame_examples, class_ids, line_thickness):
+    canvas = image.copy()
+
+    selected_class_ids = set(class_ids)
+    for gt in gts:
+        if gt["label"] not in selected_class_ids:
+            continue
+        draw_box(
+            canvas,
+            gt["bbox_2d"],
+            BOX_COLORS["gt_all"],
+            label=f"GT {class_name(gt['label'])}",
+            thickness=line_thickness,
+        )
+
+    for det in active_dets:
+        if det["label"] not in selected_class_ids:
+            continue
+        conf = float(det.get("confidence", 0.0))
+        draw_box(
+            canvas,
+            det["bbox_2d"],
+            BOX_COLORS["det_all"],
+            label=f"Det {class_name(det['label'])} {conf:.2f}",
+            thickness=line_thickness,
+        )
+
+    for idx, example in enumerate(frame_examples, 1):
+        gt_bbox = get_example_gt_bbox(example)
+        det_bbox = get_example_det_bbox(example)
+        ref_gt_bbox, ref_gt_class, _ref_gt_id = resolve_reference_gt(example, gts)
+        if gt_bbox is not None:
+            draw_box(
+                canvas,
+                gt_bbox,
+                get_target_box_color(example, "gt"),
+                label=f"FN#{idx} GT {example['class_name']}",
+                thickness=max(3, line_thickness + 1),
+            )
+        elif ref_gt_bbox is not None:
+            draw_box(
+                canvas,
+                ref_gt_bbox,
+                get_target_box_color(example, "gt"),
+                label=f"FP#{idx} RefGT {ref_gt_class or '-'}",
+                thickness=max(3, line_thickness + 1),
+            )
+        if det_bbox is not None:
+            conf = get_confidence(example)
+            iou = get_best_iou(example)
+            if example.get("best_det_bbox") is not None:
+                label = f"FN#{idx} BestDet {example.get('best_det_class', '-')}"
+                if conf is not None:
+                    label += f" {conf:.2f}"
+                label += f" IoU {iou:.3f}"
+            else:
+                label = f"FP#{idx} Det {example.get('class_name', '-')}"
+                if conf is not None:
+                    label += f" {conf:.2f}"
+                label += f" IoU {iou:.3f}"
+            draw_box(
+                canvas,
+                det_bbox,
+                get_target_box_color(example, "det"),
+                label=label,
+                thickness=max(3, line_thickness + 1),
+            )
+
+    example_modes = {("FN" if is_fn_example(example) else "FP") for example in frame_examples}
+    if len(example_modes) == 1:
+        mode_label = next(iter(example_modes))
+    else:
+        mode_label = "MIXED"
+
+    headline = (
+        f"2D error visualization | mode={mode_label} | examples={len(frame_examples)} | "
+        f"GT=green Det=orange FN-GT=red FN-det=yellow FP-det=magenta FP-refGT=cyan"
+    )
+    add_header(canvas, headline)
+    return canvas
+
+
+def ensure_dir(path):
+    path.mkdir(parents=True, exist_ok=True)
+    return path
+
+
+def sanitize_token(value):
+    return str(value).replace("/", "__").replace("\\", "__").replace(" ", "_")
+
+
+def default_output_dir(report_path):
+    report_path = Path(report_path)
+    return report_path.parent / f"fn_vis_{report_path.stem}"
+
+
+def main():
+    args = parse_args()
+
+    with open(args.analysis_report, "r") as file:
+        report = json.load(file)
+
+    config = build_config(args)
+    class_ids = parse_class_ids(args.classes) if args.classes else parse_class_ids(report["summary"]["classes"])
+    filtered_examples = filter_examples(report, args)
+
+    if not filtered_examples:
+        print("No examples matched the current filters.")
+        return
+
+    pair_map, evaluator = find_pair_map(config)
+
+    output_dir = Path(args.output_dir) if args.output_dir else default_output_dir(args.analysis_report)
+    frame_dir = ensure_dir(output_dir / "frames")
+    example_dir = ensure_dir(output_dir / "examples")
+
+    by_frame = defaultdict(list)
+    for item in filtered_examples:
+        by_frame[(item["case_name"], item["frame_name"])].append(item)
+
+    index = {
+        "analysis_report": str(Path(args.analysis_report).resolve()),
+        "num_examples": len(filtered_examples),
+        "num_frames": len(by_frame),
+        "mode": args.mode,
+        "error_types": args.error_types,
+        "classes": [class_name(cid) for cid in class_ids],
+        "frames": [],
+    }
+
+    conf_threshold = float(config.get("metrics_2d", {}).get("conf_threshold", 0.5))
+
+    for frame_idx, ((case_name, frame_name), frame_examples) in enumerate(by_frame.items(), 1):
+        pair = pair_map.get((case_name, frame_name))
+        if pair is None:
+            print(f"Warning: failed to locate pair for {case_name}/{frame_name}, skipping")
+            continue
+
+        image_path = find_image_path(pair)
+        if image_path is None or not image_path.exists():
+            print(f"Warning: image not found for {case_name}/{frame_name}, skipping")
+            continue
+
+        image = cv2.imread(str(image_path))
+        if image is None:
+            print(f"Warning: failed to read image: {image_path}")
+            continue
+
+        gts = Evaluator._parse_ground_truths_for_pair(pair, evaluator.coord_system)
+        dets = Evaluator._parse_detections_for_pair(pair, evaluator.coord_system)
+        active_dets = [det for det in dets if float(det.get("confidence", 0.0)) >= conf_threshold]
+
+        frame_overlay = render_frame_overlay(
+            image,
+            gts,
+            active_dets,
+            frame_examples,
+            class_ids,
+            line_thickness=args.line_thickness,
+        )
+
+        frame_rel = Path("frames") / (
+            f"{frame_idx:04d}_{sanitize_token(case_name)}_{sanitize_token(frame_name)}.jpg"
+        )
+        frame_path = output_dir / frame_rel
+        cv2.imwrite(
+            str(frame_path),
+            frame_overlay,
+            [int(cv2.IMWRITE_JPEG_QUALITY), int(args.jpeg_quality)],
+        )
+
+        frame_entry = {
+            "case_name": case_name,
+            "frame_name": frame_name,
+            "image_path": str(image_path),
+            "frame_visualization": str(frame_rel),
+            "num_examples": len(frame_examples),
+            "examples": [],
+        }
+
+        for ex_idx, example in enumerate(frame_examples, 1):
+            crop_image = draw_crop_panel(
+                image.copy(), example, gts, crop_scale=args.crop_scale
+            )
+            panel = combine_full_and_crop(frame_overlay.copy(), crop_image, example)
+            rel = Path("examples") / (
+                f"{frame_idx:04d}_{ex_idx:02d}_"
+                f"{sanitize_token(case_name)}_{sanitize_token(frame_name)}_"
+                f"{sanitize_token(example['class_name'])}_{sanitize_token(example['error_type'])}.jpg"
+            )
+            panel_path = output_dir / rel
+            cv2.imwrite(
+                str(panel_path),
+                panel,
+                [int(cv2.IMWRITE_JPEG_QUALITY), int(args.jpeg_quality)],
+            )
+
+            example_record = dict(example)
+            example_record["visualization"] = str(rel)
+            frame_entry["examples"].append(example_record)
+
+        index["frames"].append(frame_entry)
+
+    index_path = output_dir / "index.json"
+    with open(index_path, "w") as file:
+        json.dump(index, file, indent=2)
+
+    print(f"Saved visualization index to: {index_path}")
+    print(f"Saved frame overlays to:      {frame_dir}")
+    print(f"Saved example panels to:      {example_dir}")
+
+
+if __name__ == "__main__":
+    main()

eval_tools/analysis/visualize_2d_fn_cases.sh

@@ -0,0 +1,43 @@
+#!/usr/bin/env bash
+
+set -euo pipefail
+
+ANALYSIS_REPORT=${ANALYSIS_REPORT:-/data1/dongying/Mono3d/G1Q3/model_inference/KPI/DL_KPI_SCENE/model_20260403_analysis/analysis_2d/mono3d_fp_fn-roi0/analysis_report.json}
+CONFIG_PATH=${CONFIG_PATH:-eval_tools/configs/eval_config_yolov26s-roi0.yaml}
+OUTPUT_BASE=${OUTPUT_BASE:-/data1/dongying/Mono3d/G1Q3/model_inference/KPI/DL_KPI_SCENE/model_20260403_analysis/analysis_2d/mono3d_fp_fn-roi0}
+TOP_K=${TOP_K:-1000}
+
+FN_ERROR_TYPES=(localization missing)
+FP_ERROR_TYPES=(localization background)
+CLASSES=(car)
+
+for mode in fn fp; do
+  if [[ "${mode}" == "fn" ]]; then
+    ERROR_TYPES=("${FN_ERROR_TYPES[@]}")
+  else
+    ERROR_TYPES=("${FP_ERROR_TYPES[@]}")
+  fi
+
+  for error_type in "${ERROR_TYPES[@]}"; do
+    for class_name in "${CLASSES[@]}"; do
+      output_dir="${OUTPUT_BASE}/${mode}_${error_type}_vis_${class_name}"
+
+      echo "============================================================"
+      echo "Running visualization"
+      echo "  mode:       ${mode}"
+      echo "  error_type: ${error_type}"
+      echo "  class:      ${class_name}"
+      echo "  output:     ${output_dir}"
+      echo "============================================================"
+
+      python eval_tools/analysis/visualize_2d_fn_cases.py \
+        --analysis-report "${ANALYSIS_REPORT}" \
+        --config "${CONFIG_PATH}" \
+        --mode "${mode}" \
+        --error-types "${error_type}" \
+        --classes "${class_name}" \
+        --top-k "${TOP_K}" \
+        --output-dir "${output_dir}"
+    done
+  done
+done

eval_tools/analysis/visualize_3d_badcases.py

@@ -0,0 +1,723 @@
+#!/usr/bin/env python3
+"""
+Visualize 3D bad cases from analyze_3d_badcases.py results.
+
+This script focuses on matched 3D samples and renders:
+  - full-frame overlays with GT / active detections / highlighted badcases
+  - per-example panels with crop, simple BEV, and a metrics sidebar
+  - an index.json for downstream browsing
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+import math
+import sys
+from collections import Counter, defaultdict
+from pathlib import Path
+
+import cv2
+import numpy as np
+
+
+REPO_ROOT = Path(__file__).resolve().parents[2]
+if str(REPO_ROOT) not in sys.path:
+    sys.path.insert(0, str(REPO_ROOT))
+
+from eval_tools.analysis.analyze_2d_fp_fn import build_config, class_name, parse_class_ids
+from eval_tools.evaluator.evaluator import Evaluator
+
+
+BOX_COLORS = {
+    "gt_all": (80, 220, 80),
+    "det_all": (150, 150, 150),
+    "target_gt": (40, 40, 255),
+    "target_det": (0, 215, 255),
+    "title_bg": (30, 30, 30),
+    "bev_gt": (40, 220, 80),
+    "bev_det": (0, 215, 255),
+}
+
+
+def parse_args():
+    parser = argparse.ArgumentParser(
+        description="Visualize 3D bad cases from analysis_report.json."
+    )
+    parser.add_argument(
+        "--analysis-report",
+        type=str,
+        required=True,
+        help="Path to analysis_report.json generated by analyze_3d_badcases.py",
+    )
+    parser.add_argument("--config", type=str, required=True, help="Path to YAML evaluation config")
+    parser.add_argument("--det-path", type=str, help="Detection results root directory")
+    parser.add_argument("--gt-path", type=str, help="Ground-truth labels root directory")
+    parser.add_argument("--path-depth", type=int, choices=[1, 2], help="Directory depth")
+    parser.add_argument(
+        "--det-format",
+        type=str,
+        choices=["auto", "json", "txt"],
+        help="Detection file format",
+    )
+    parser.add_argument(
+        "--gt-format",
+        type=str,
+        choices=["auto", "json", "txt"],
+        help="Ground-truth file format",
+    )
+    parser.add_argument("--img-width", type=int, help="Image width")
+    parser.add_argument("--img-height", type=int, help="Image height")
+    parser.add_argument(
+        "--coord-system",
+        type=str,
+        choices=["camera", "ego"],
+        help="Coordinate system used by the parser/evaluator",
+    )
+    parser.add_argument(
+        "--iou-threshold",
+        type=float,
+        help="IoU threshold used for evaluator loading",
+    )
+    parser.add_argument(
+        "--conf-threshold",
+        type=float,
+        help="Confidence threshold for active detections shown on overlays",
+    )
+    parser.add_argument(
+        "--metrics",
+        nargs="+",
+        default=["longitudinal_error"],
+        help="Metrics to visualize from badcase_examples.",
+    )
+    parser.add_argument(
+        "--classes",
+        nargs="+",
+        default=None,
+        help="Optional class filter, e.g. car suv pedestrian",
+    )
+    parser.add_argument(
+        "--min-confidence",
+        type=float,
+        default=None,
+        help="Minimum confidence for badcase examples.",
+    )
+    parser.add_argument(
+        "--max-confidence",
+        type=float,
+        default=None,
+        help="Maximum confidence for badcase examples.",
+    )
+    parser.add_argument(
+        "--min-iou",
+        type=float,
+        default=None,
+        help="Minimum IoU for badcase examples.",
+    )
+    parser.add_argument(
+        "--max-iou",
+        type=float,
+        default=None,
+        help="Maximum IoU for badcase examples.",
+    )
+    parser.add_argument(
+        "--top-k",
+        type=int,
+        default=200,
+        help="Maximum number of examples to visualize after filtering",
+    )
+    parser.add_argument(
+        "--top-k-per-distance-bin",
+        type=int,
+        default=0,
+        help="Optional cap per longitudinal distance bin before applying --top-k. 0 disables bin-wise capping.",
+    )
+    parser.add_argument(
+        "--dedup-frame",
+        action="store_true",
+        help="Keep at most one example per case/frame/class/metric combination",
+    )
+    parser.add_argument(
+        "--group-by-distance-bin",
+        action="store_true",
+        help="Render and save outputs separately for each longitudinal distance bin.",
+    )
+    parser.add_argument(
+        "--line-thickness",
+        type=int,
+        default=2,
+        help="Base line thickness for non-highlight boxes",
+    )
+    parser.add_argument(
+        "--crop-scale",
+        type=float,
+        default=1.8,
+        help="Expand crop window around GT/det union box by this factor",
+    )
+    parser.add_argument(
+        "--jpeg-quality",
+        type=int,
+        default=92,
+        help="JPEG quality for saved visualizations",
+    )
+    parser.add_argument(
+        "--output-dir",
+        type=str,
+        default=None,
+        help="Output directory. Defaults to sibling 3d_vis_<report_name>",
+    )
+    return parser.parse_args()
+
+
+def normalize_token_set(values):
+    if not values:
+        return None
+    return {str(v).strip().lower() for v in values if str(v).strip()}
+
+
+def filter_examples(report, args):
+    pools = []
+    for metric_name in args.metrics:
+        pools.extend(report.get("badcase_examples", {}).get(metric_name, []))
+
+    class_filter = normalize_token_set(args.classes)
+    metric_filter = normalize_token_set(args.metrics)
+
+    filtered = []
+    for item in pools:
+        if metric_filter and str(item.get("metric_name", "")).lower() not in metric_filter:
+            continue
+        if class_filter and str(item.get("class_name", "")).lower() not in class_filter:
+            continue
+        confidence = item.get("confidence")
+        iou = item.get("iou")
+        if args.min_confidence is not None and (confidence is None or float(confidence) < args.min_confidence):
+            continue
+        if args.max_confidence is not None and (confidence is None or float(confidence) > args.max_confidence):
+            continue
+        if args.min_iou is not None and (iou is None or float(iou) < args.min_iou):
+            continue
+        if args.max_iou is not None and (iou is None or float(iou) > args.max_iou):
+            continue
+        filtered.append(item)
+
+    filtered.sort(
+        key=lambda item: (
+            float(item.get("metric_value_display", 0.0)),
+            float(item.get("confidence", 0.0)),
+            float(item.get("iou", 0.0)),
+        ),
+        reverse=True,
+    )
+
+    if args.dedup_frame:
+        deduped = []
+        seen = set()
+        for item in filtered:
+            key = (
+                item.get("case_name"),
+                item.get("frame_name"),
+                item.get("class_name"),
+                item.get("metric_name"),
+            )
+            if key in seen:
+                continue
+            seen.add(key)
+            deduped.append(item)
+        filtered = deduped
+
+    if args.top_k_per_distance_bin and args.top_k_per_distance_bin > 0:
+        kept = []
+        counts = Counter()
+        for item in filtered:
+            distance_bin = item.get("distance_bin") or "unbucketed"
+            if counts[distance_bin] >= args.top_k_per_distance_bin:
+                continue
+            kept.append(item)
+            counts[distance_bin] += 1
+        filtered = kept
+
+    if args.top_k is not None:
+        filtered = filtered[: args.top_k]
+    return filtered
+
+
+def bbox_to_int(bbox):
+    return [int(round(float(v))) for v in bbox]
+
+
+def draw_box(image, bbox, color, label=None, thickness=2):
+    x1, y1, x2, y2 = bbox_to_int(bbox)
+    cv2.rectangle(image, (x1, y1), (x2, y2), color, thickness, cv2.LINE_AA)
+    if label:
+        (tw, th), _ = cv2.getTextSize(label, cv2.FONT_HERSHEY_SIMPLEX, 0.55, 1)
+        y_text = max(0, y1 - th - 8)
+        cv2.rectangle(image, (x1, y_text), (x1 + tw + 8, y_text + th + 8), color, -1)
+        cv2.putText(
+            image,
+            label,
+            (x1 + 4, y_text + th + 2),
+            cv2.FONT_HERSHEY_SIMPLEX,
+            0.55,
+            (255, 255, 255),
+            1,
+            cv2.LINE_AA,
+        )
+
+
+def add_header(image, text):
+    h, w = image.shape[:2]
+    overlay = image.copy()
+    cv2.rectangle(overlay, (0, 0), (w, 42), BOX_COLORS["title_bg"], -1)
+    cv2.addWeighted(overlay, 0.55, image, 0.45, 0, image)
+    cv2.putText(
+        image,
+        text,
+        (10, 28),
+        cv2.FONT_HERSHEY_SIMPLEX,
+        0.7,
+        (255, 255, 255),
+        2,
+        cv2.LINE_AA,
+    )
+
+
+def make_crop(image, boxes, scale=1.8):
+    h, w = image.shape[:2]
+    valid = [bbox for bbox in boxes if bbox]
+    if not valid:
+        return image.copy(), (0, 0)
+
+    x1 = min(float(b[0]) for b in valid)
+    y1 = min(float(b[1]) for b in valid)
+    x2 = max(float(b[2]) for b in valid)
+    y2 = max(float(b[3]) for b in valid)
+
+    cx = 0.5 * (x1 + x2)
+    cy = 0.5 * (y1 + y2)
+    bw = max(32.0, (x2 - x1) * scale)
+    bh = max(32.0, (y2 - y1) * scale)
+
+    crop_x1 = max(0, int(round(cx - bw / 2)))
+    crop_y1 = max(0, int(round(cy - bh / 2)))
+    crop_x2 = min(w, int(round(cx + bw / 2)))
+    crop_y2 = min(h, int(round(cy + bh / 2)))
+    return image[crop_y1:crop_y2, crop_x1:crop_x2].copy(), (crop_x1, crop_y1)
+
+
+def shift_box(box, off_x, off_y):
+    if not box:
+        return None
+    return [
+        float(box[0]) - off_x,
+        float(box[1]) - off_y,
+        float(box[2]) - off_x,
+        float(box[3]) - off_y,
+    ]
+
+
+def draw_crop_panel(image, example, crop_scale):
+    gt_bbox = example.get("gt_bbox")
+    det_bbox = example.get("det_bbox")
+    crop, (off_x, off_y) = make_crop(image, [gt_bbox, det_bbox], scale=crop_scale)
+
+    gt_local = shift_box(gt_bbox, off_x, off_y)
+    det_local = shift_box(det_bbox, off_x, off_y)
+    if gt_local:
+        draw_box(
+            crop,
+            gt_local,
+            BOX_COLORS["target_gt"],
+            label=f"GT {example['class_name']}",
+            thickness=3,
+        )
+    if det_local:
+        draw_box(
+            crop,
+            det_local,
+            BOX_COLORS["target_det"],
+            label=f"Det {example['class_name']} {float(example.get('confidence', 0.0)):.2f}",
+            thickness=3,
+        )
+
+    add_header(
+        crop,
+        (
+            f"crop | {example['class_name']} | {example['metric_name']}="
+            f"{float(example.get('metric_value_display', 0.0)):.3f}{example.get('metric_unit', '')}"
+        ),
+    )
+    return crop
+
+
+def create_bev_panel(example, coord_system="camera", width=480, height=320, max_depth_m=100.0, max_lateral_m=30.0):
+    panel = np.full((height, width, 3), 245, dtype=np.uint8)
+
+    def project(point3d):
+        if not point3d or len(point3d) < 3:
+            return None
+        if coord_system == "camera":
+            x = float(point3d[0])
+            z = float(point3d[2])
+        else:
+            x = float(point3d[1])
+            z = float(point3d[0])
+        px = int(round((x + max_lateral_m) / (2.0 * max_lateral_m) * (width - 1)))
+        py = int(round((1.0 - max(0.0, min(z, max_depth_m)) / max_depth_m) * (height - 1)))
+        return px, py
+
+    for depth in range(0, int(max_depth_m) + 1, 10):
+        y = int(round((1.0 - depth / max_depth_m) * (height - 1)))
+        cv2.line(panel, (0, y), (width - 1, y), (225, 225, 225), 1, cv2.LINE_AA)
+        cv2.putText(panel, f"{depth}m", (6, max(14, y - 4)), cv2.FONT_HERSHEY_SIMPLEX, 0.4, (90, 90, 90), 1, cv2.LINE_AA)
+
+    for lat in range(-int(max_lateral_m), int(max_lateral_m) + 1, 10):
+        x = int(round((lat + max_lateral_m) / (2.0 * max_lateral_m) * (width - 1)))
+        cv2.line(panel, (x, 0), (x, height - 1), (232, 232, 232), 1, cv2.LINE_AA)
+
+    center_x = int(round((0.0 + max_lateral_m) / (2.0 * max_lateral_m) * (width - 1)))
+    cv2.line(panel, (center_x, 0), (center_x, height - 1), (180, 180, 180), 2, cv2.LINE_AA)
+
+    gt_pt = project(example.get("gt_center_3d"))
+    det_pt = project(example.get("det_center_3d"))
+    if gt_pt:
+        cv2.circle(panel, gt_pt, 7, BOX_COLORS["bev_gt"], -1, cv2.LINE_AA)
+        draw_heading_arrow(panel, gt_pt, float(example.get("gt_rotation_rad", 0.0)), BOX_COLORS["bev_gt"])
+    if det_pt:
+        cv2.circle(panel, det_pt, 7, BOX_COLORS["bev_det"], -1, cv2.LINE_AA)
+        draw_heading_arrow(panel, det_pt, float(example.get("det_rotation_rad", 0.0)), BOX_COLORS["bev_det"])
+    if gt_pt and det_pt:
+        cv2.line(panel, gt_pt, det_pt, (80, 80, 80), 2, cv2.LINE_AA)
+
+    add_header(panel, "simple BEV | GT=green | Det=orange")
+    return panel
+
+
+def draw_heading_arrow(canvas, anchor, rotation_rad, color, length_px=28):
+    dx = math.sin(rotation_rad) * length_px
+    dy = -math.cos(rotation_rad) * length_px
+    end_point = (int(round(anchor[0] + dx)), int(round(anchor[1] + dy)))
+    cv2.arrowedLine(canvas, anchor, end_point, color, 2, cv2.LINE_AA, tipLength=0.25)
+
+
+def add_sidebar(panel, example):
+    h, _ = panel.shape[:2]
+    sidebar = np.full((h, 420, 3), 28, dtype=np.uint8)
+    lines = [
+        f"case: {example.get('case_name')}",
+        f"frame: {example.get('frame_name')}",
+        f"class: {example.get('class_name')}",
+        f"metric: {example.get('metric_name')}",
+        f"metric_display: {example.get('metric_value_display')} {example.get('metric_unit', '')}",
+        f"conf: {example.get('confidence')}",
+        f"iou: {example.get('iou')}",
+        f"distance_z_m: {example.get('distance_longitudinal_m')}",
+        f"distance_x_m: {example.get('distance_lateral_m')}",
+        f"distance_bin: {example.get('distance_bin')}",
+        f"lateral_bin: {example.get('lateral_bin')}",
+        f"lat_err_m: {example.get('lateral_error_m')}",
+        f"long_err_m: {example.get('longitudinal_error_m')}",
+        f"long_rel_err: {example.get('longitudinal_relative_error')}",
+        f"heading_deg: {example.get('heading_error_deg')}",
+        f"heading_relaxed_deg: {example.get('heading_error_relaxed_deg')}",
+        f"is_reversal: {example.get('is_reversal')}",
+        f"gt_id: {example.get('gt_id')}",
+    ]
+
+    y = 36
+    for line in lines:
+        cv2.putText(
+            sidebar,
+            str(line),
+            (12, y),
+            cv2.FONT_HERSHEY_SIMPLEX,
+            0.56,
+            (235, 235, 235),
+            1,
+            cv2.LINE_AA,
+        )
+        y += 28
+
+    return np.hstack([panel, sidebar])
+
+
+def resize_to_height(image, target_height):
+    h, w = image.shape[:2]
+    if h == target_height:
+        return image
+    scale = target_height / max(h, 1)
+    return cv2.resize(image, (max(1, int(round(w * scale))), target_height))
+
+
+def combine_panels(full_image, crop_image, bev_image, example):
+    target_h = max(full_image.shape[0], crop_image.shape[0], bev_image.shape[0])
+    full_resized = resize_to_height(full_image, target_h)
+    crop_resized = resize_to_height(crop_image, target_h)
+    bev_resized = resize_to_height(bev_image, target_h)
+    panel = np.hstack([full_resized, crop_resized, bev_resized])
+    return add_sidebar(panel, example)
+
+
+def find_pair_map(config):
+    evaluator = Evaluator(
+        config=config,
+        iou_threshold=float(config.get("matching", {}).get("iou_threshold", 0.5)),
+        num_workers=1,
+        save_detailed_matches=False,
+    )
+    dataset_cfg = config["dataset"]
+    image_cfg = config["image"]
+    evaluator.load_data_from_paths(
+        det_root=dataset_cfg["det_path"],
+        gt_root=dataset_cfg["gt_path"],
+        img_width=image_cfg.get("width", 1920),
+        img_height=image_cfg.get("height", 1080),
+        path_depth=dataset_cfg.get("path_depth", 1),
+        det_format=dataset_cfg.get("det_format", "auto"),
+        gt_format=dataset_cfg.get("gt_format", "auto"),
+    )
+
+    pair_map = {}
+    for pair in evaluator.image_pairs:
+        level1_name = pair.get("level1_name")
+        if level1_name:
+            case_key = f"{level1_name}/{pair['case']}"
+        else:
+            case_key = pair["case"]
+        pair_map[(case_key, pair["frame"])] = pair
+    return pair_map, evaluator
+
+
+def find_image_path(pair):
+    gt_file = Path(pair["gt_file"])
+    case_dir = gt_file.parent.parent
+    images_dir = case_dir / "images"
+    stem = gt_file.stem
+    for suffix in (".png", ".jpg", ".jpeg", ".bmp"):
+        candidate = images_dir / f"{stem}{suffix}"
+        if candidate.exists():
+            return candidate
+    matches = list(images_dir.glob(f"{stem}.*"))
+    return matches[0] if matches else None
+
+
+def render_frame_overlay(image, gts, active_dets, frame_examples, class_ids, line_thickness):
+    canvas = image.copy()
+    selected_class_ids = set(class_ids)
+
+    for gt in gts:
+        if gt["label"] not in selected_class_ids:
+            continue
+        draw_box(
+            canvas,
+            gt["bbox_2d"],
+            BOX_COLORS["gt_all"],
+            label=f"GT {class_name(gt['label'])}",
+            thickness=line_thickness,
+        )
+
+    for det in active_dets:
+        if det["label"] not in selected_class_ids:
+            continue
+        conf = float(det.get("confidence", 0.0))
+        draw_box(
+            canvas,
+            det["bbox_2d"],
+            BOX_COLORS["det_all"],
+            label=f"Det {class_name(det['label'])} {conf:.2f}",
+            thickness=line_thickness,
+        )
+
+    for idx, example in enumerate(frame_examples, 1):
+        if example.get("gt_bbox"):
+            draw_box(
+                canvas,
+                example["gt_bbox"],
+                BOX_COLORS["target_gt"],
+                label=f"GT#{idx} {example['class_name']}",
+                thickness=max(3, line_thickness + 1),
+            )
+        if example.get("det_bbox"):
+            label = (
+                f"Det#{idx} {example['class_name']} "
+                f"{float(example.get('confidence', 0.0)):.2f} "
+                f"{example['metric_name']}={float(example.get('metric_value_display', 0.0)):.2f}"
+            )
+            draw_box(
+                canvas,
+                example["det_bbox"],
+                BOX_COLORS["target_det"],
+                label=label,
+                thickness=max(3, line_thickness + 1),
+            )
+
+    headline = (
+        f"3D badcase visualization | examples={len(frame_examples)} | "
+        f"GT=green Det=gray TargetGT=red TargetDet=orange"
+    )
+    add_header(canvas, headline)
+    return canvas
+
+
+def ensure_dir(path):
+    path.mkdir(parents=True, exist_ok=True)
+    return path
+
+
+def resolve_output_dirs(output_dir, distance_bin=None, group_by_distance_bin=False):
+    if group_by_distance_bin:
+        safe_bin = sanitize_token(distance_bin or "unbucketed")
+        base_dir = output_dir / "distance_bins" / safe_bin
+    else:
+        base_dir = output_dir
+    frame_dir = ensure_dir(base_dir / "frames")
+    example_dir = ensure_dir(base_dir / "examples")
+    return base_dir, frame_dir, example_dir
+
+
+def sanitize_token(value):
+    return str(value).replace("/", "__").replace("\\", "__").replace(" ", "_")
+
+
+def default_output_dir(report_path):
+    report_path = Path(report_path)
+    return report_path.parent / f"3d_vis_{report_path.stem}"
+
+
+def main():
+    args = parse_args()
+    with open(args.analysis_report, "r") as file:
+        report = json.load(file)
+
+    config = build_config(args)
+    class_ids = parse_class_ids(args.classes) if args.classes else parse_class_ids(report["metadata"]["classes"])
+    filtered_examples = filter_examples(report, args)
+
+    if not filtered_examples:
+        print("No examples matched the current filters.")
+        return
+
+    pair_map, evaluator = find_pair_map(config)
+    output_dir = Path(args.output_dir) if args.output_dir else default_output_dir(args.analysis_report)
+
+    by_frame = defaultdict(list)
+    for item in filtered_examples:
+        group_distance_bin = item.get("distance_bin") if args.group_by_distance_bin else None
+        by_frame[(item["case_name"], item["frame_name"], group_distance_bin)].append(item)
+
+    index = {
+        "analysis_report": str(Path(args.analysis_report).resolve()),
+        "num_examples": len(filtered_examples),
+        "num_frames": len(by_frame),
+        "metrics": args.metrics,
+        "classes": [class_name(cid) for cid in class_ids],
+        "group_by_distance_bin": bool(args.group_by_distance_bin),
+        "top_k_per_distance_bin": int(args.top_k_per_distance_bin),
+        "distance_bins": {},
+        "frames": [],
+    }
+
+    conf_threshold = float(
+        config.get("metrics_3d", {}).get(
+            "conf_threshold",
+            config.get("metrics_2d", {}).get("conf_threshold", 0.5),
+        )
+    )
+
+    saved_frame_dirs = set()
+    saved_example_dirs = set()
+
+    for frame_idx, ((case_name, frame_name, distance_bin), frame_examples) in enumerate(by_frame.items(), 1):
+        pair = pair_map.get((case_name, frame_name))
+        if pair is None:
+            print(f"Warning: failed to locate pair for {case_name}/{frame_name}, skipping")
+            continue
+
+        image_path = find_image_path(pair)
+        if image_path is None or not image_path.exists():
+            print(f"Warning: image not found for {case_name}/{frame_name}, skipping")
+            continue
+
+        image = cv2.imread(str(image_path))
+        if image is None:
+            print(f"Warning: failed to read image: {image_path}")
+            continue
+
+        gts = Evaluator._parse_ground_truths_for_pair(pair, evaluator.coord_system)
+        dets = Evaluator._parse_detections_for_pair(pair, evaluator.coord_system)
+        active_dets = [det for det in dets if float(det.get("confidence", 0.0)) >= conf_threshold]
+
+        frame_overlay = render_frame_overlay(
+            image,
+            gts,
+            active_dets,
+            frame_examples,
+            class_ids,
+            line_thickness=args.line_thickness,
+        )
+
+        base_dir, frame_dir, example_dir = resolve_output_dirs(
+            output_dir,
+            distance_bin=distance_bin,
+            group_by_distance_bin=args.group_by_distance_bin,
+        )
+        saved_frame_dirs.add(str(frame_dir))
+        saved_example_dirs.add(str(example_dir))
+
+        frame_rel = Path(frame_dir.relative_to(output_dir)) / (
+            f"{frame_idx:04d}_{sanitize_token(case_name)}_{sanitize_token(frame_name)}.jpg"
+        )
+        frame_path = output_dir / frame_rel
+        cv2.imwrite(
+            str(frame_path),
+            frame_overlay,
+            [int(cv2.IMWRITE_JPEG_QUALITY), int(args.jpeg_quality)],
+        )
+
+        frame_entry = {
+            "case_name": case_name,
+            "frame_name": frame_name,
+            "distance_bin": distance_bin,
+            "image_path": str(image_path),
+            "frame_visualization": str(frame_rel),
+            "num_examples": len(frame_examples),
+            "examples": [],
+        }
+        distance_key = distance_bin or "all"
+        index["distance_bins"].setdefault(distance_key, {"num_frames": 0, "num_examples": 0})
+        index["distance_bins"][distance_key]["num_frames"] += 1
+        index["distance_bins"][distance_key]["num_examples"] += len(frame_examples)
+
+        for ex_idx, example in enumerate(frame_examples, 1):
+            crop_image = draw_crop_panel(image.copy(), example, crop_scale=args.crop_scale)
+            bev_image = create_bev_panel(example, coord_system=evaluator.coord_system)
+            panel = combine_panels(frame_overlay.copy(), crop_image, bev_image, example)
+            rel = Path(example_dir.relative_to(output_dir)) / (
+                f"{frame_idx:04d}_{ex_idx:02d}_"
+                f"{sanitize_token(case_name)}_{sanitize_token(frame_name)}_"
+                f"{sanitize_token(example['class_name'])}_{sanitize_token(example['metric_name'])}.jpg"
+            )
+            panel_path = output_dir / rel
+            cv2.imwrite(
+                str(panel_path),
+                panel,
+                [int(cv2.IMWRITE_JPEG_QUALITY), int(args.jpeg_quality)],
+            )
+
+            example_record = dict(example)
+            example_record["visualization"] = str(rel)
+            frame_entry["examples"].append(example_record)
+
+        index["frames"].append(frame_entry)
+
+    index_path = output_dir / "index.json"
+    with open(index_path, "w") as file:
+        json.dump(index, file, indent=2)
+
+    print(f"Saved visualization index to: {index_path}")
+    print(f"Saved frame overlays to:      {', '.join(sorted(saved_frame_dirs))}")
+    print(f"Saved example panels to:      {', '.join(sorted(saved_example_dirs))}")
+
+
+if __name__ == "__main__":
+    main()

eval_tools/analysis/visualize_3d_badcases.sh

@@ -0,0 +1,36 @@
+#!/usr/bin/env bash
+
+set -euo pipefail
+
+ANALYSIS_REPORT=${ANALYSIS_REPORT:-/data1/dongying/Mono3d/G1Q3/model_inference/KPI/DL_KPI_SCENE/model_20260403_analysis/analysis_3d/analysis_report.json}
+CONFIG_PATH=${CONFIG_PATH:-eval_tools/configs/eval_config_yolov26s-roi0.yaml}
+OUTPUT_BASE=${OUTPUT_BASE:-$(dirname "${ANALYSIS_REPORT}")/visualizations_distance}
+TOP_K=${TOP_K:-300}
+TOP_K_PER_DISTANCE_BIN=${TOP_K_PER_DISTANCE_BIN:-50}
+
+METRICS=(longitudinal_error heading_error reversal)
+CLASSES=(car suv van bus truck pedestrian bicycle)
+PYTHON_BIN=${PYTHON_BIN:-/deeplearning_team/ydong/dongying/miniconda/envs/dev/bin/python}
+
+for metric_name in "${METRICS[@]}"; do
+  for class_name in "${CLASSES[@]}"; do
+    output_dir="${OUTPUT_BASE}/${metric_name}_vis_${class_name}"
+
+    echo "============================================================"
+    echo "Running 3D visualization"
+    echo "  metric: ${metric_name}"
+    echo "  class:  ${class_name}"
+    echo "  output: ${output_dir}"
+    echo "============================================================"
+
+    "${PYTHON_BIN}" eval_tools/analysis/visualize_3d_badcases.py \
+      --analysis-report "${ANALYSIS_REPORT}" \
+      --config "${CONFIG_PATH}" \
+      --metrics "${metric_name}" \
+      --classes "${class_name}" \
+      --top-k "${TOP_K}" \
+      --top-k-per-distance-bin "${TOP_K_PER_DISTANCE_BIN}" \
+      --group-by-distance-bin \
+      --output-dir "${output_dir}"
+  done
+done

eval_tools/class_config.py

@@ -0,0 +1,106 @@
+"""
+Class configuration for the eval_tools evaluation framework.
+
+This is the single source of truth for all class definitions.
+To adapt the evaluator to a different class taxonomy, only edit this file.
+
+Current taxonomy matches ultralytics/cfg/datasets/mono3d_ground.yaml class_map.
+"""
+
+# ---------------------------------------------------------------------------
+# ID → display name  (used by GT parser, reports, logging)
+# Keep this aligned with the first-seen canonical name for each ID in
+# ultralytics/cfg/datasets/mono3d_ground.yaml:class_map.
+# ---------------------------------------------------------------------------
+CLASS_NAMES: dict[int, str] = {
+    0: "car",
+    1: "suv",
+    2: "pickup",
+    3: "medium_car",
+    4: "van",
+    5: "bus",
+    6: "truck",
+    7: "special_vehicle",
+    8: "unknown",
+    9: "pedestrian",
+    10: "bicyclist",
+    11: "bicycle",
+    12: "tricycle",
+    13: "traffic_sign",
+    14: "wheel",
+    15: "plate",
+    16: "face",
+}
+
+# Total number of classes
+NUM_CLASSES: int = len(CLASS_NAMES)
+
+# ---------------------------------------------------------------------------
+# Raw label string → ID  (used by detection result parser)
+# Supports all original annotation names that map to the same ID in
+# ultralytics/cfg/datasets/mono3d_ground.yaml:class_map.
+# ---------------------------------------------------------------------------
+CLASS_NAME_TO_ID: dict[str, int] = {
+    "car": 0,
+    "suv": 1,
+    "pickup": 2,
+    "medium_car": 3,
+    "van": 4,
+    "bus": 5,
+    "truck": 6,
+    "tanker": 6,
+    "large_truck": 6,
+    "construction_vehicle": 6,
+    "special_vehicle": 7,
+    "unknown": 8,
+    "pedestrian": 9,
+    "bicyclist": 10,
+    "motorcyclist": 10,
+    "bicycle": 11,
+    "motorcycle": 11,
+    "tricycle": 12,
+    "tricyclist": 12,
+    "traffic_sign": 13,
+    "wheel": 14,
+    "plate": 15,
+    "face": 16,
+}
+
+# ---------------------------------------------------------------------------
+# 3D-annotated class IDs
+#   face_3d_classes    [0-8]: vehicle-like classes with 4-face annotations
+#                             (51-col / full JSON face fields)
+#   complete_3d_classes[9-12]: pedestrian / rider / bike / trike classes with
+#                              whole-box 3D only (19-col labels)
+# ---------------------------------------------------------------------------
+FACE_3D_CLASSES: list[int] = [0, 1, 2, 3, 4, 5, 6, 7, 8]
+COMPLETE_3D_CLASSES: list[int] = [9, 10, 11, 12]
+CLASSES_3D: list[int] = FACE_3D_CLASSES + COMPLETE_3D_CLASSES
+
+# ---------------------------------------------------------------------------
+# Report labels: which 3D classes appear in evaluation reports, and how they
+# are labelled. vehicle_large / vehicle_small are legacy synthetic keys from
+# Metrics3D's class-0 size split; under the current taxonomy they are displayed
+# as CAR_LARGE / CAR_SMALL.
+# ---------------------------------------------------------------------------
+REPORT_3D_CLASS_LABELS: dict[str, str] = {
+    "car":           "CAR",
+    "vehicle_large": "CAR_LARGE",
+    "vehicle_small": "CAR_SMALL",
+    "suv":           "SUV",
+    "pickup":        "PICKUP",
+    "medium_car":    "MEDIUM_CAR",
+    "van":           "VAN",
+    "bus":           "BUS",
+    "truck":         "TRUCK",
+    "special_vehicle": "SPECIAL_VEHICLE",
+    "unknown":       "UNKNOWN",
+    "pedestrian":    "PEDESTRIAN",
+    "bicyclist":     "BICYCLIST",
+    "bicycle":       "BICYCLE",
+    "tricycle":      "TRICYCLE",
+}
+
+# Ordered list of 3D class keys for iteration in comparison/report scripts.
+# Includes the legacy size-split sub-buckets immediately after "car".
+REPORT_3D_CLASS_KEYS: list[str] = list(REPORT_3D_CLASS_LABELS.keys())

eval_tools/configs/eval_config_mono3d-roi0.yaml

@@ -0,0 +1,154 @@
+# Evaluation Configuration
+# This configuration file maps to command-line arguments in eval.py
+# All settings can be overridden by command-line arguments
+
+# Dataset paths (--det-path, --gt-path)
+# dataset:
+#   det_path: "/data1/dongying/Mono3d/G1M3/inference_results/mono3d/evalset_roi0"  # Root directory containing case folders with txt_results
+#   gt_path: "/data1/xdzhu/Testdata_0129"        # Root directory containing case folders with labels
+
+# CNCAP gt_path: "/mnt/mono3d/xdzhu_data/Mono3d/Mono3d_4face_2m_g1m3/driving_png/G1M3_AFS1616"
+
+dataset:
+  det_path: "/data1/dongying/Mono3d/G1M3/CNCAP_results/mono3d/evalset_roi0"  # Root directory containing case folders with txt_results
+  gt_path: "/mnt/mono3d/xdzhu_data/Mono3d/Mono3d_4face_2m_g1m3/driving_png/G1M3_AFS1616"        # Root directory containing case folders with labels
+  path_depth: 2  # Directory depth: 1 = det_path/case/txt_results, 2 = det_path/level1/case/txt_results
+  det_format: "auto"   # Detection file format: "auto" (probe json_results/ then txt_results/), "json", or "txt"
+  gt_format: "auto"    # Ground truth file format: "auto" (probe labels_json/ then labels/), "json", or "txt"
+  
+# Image properties (--img-width, --img-height)
+image:
+  width: 1920   # Default: 1920
+  height: 1080  # Default: 1080
+
+# Model configuration for GT filtering
+model:
+  input_size: 704  # Model input width (used for GT min box size calculation)
+  min_box_size_at_input_scale: 8  # Minimum box size at model input scale
+  # 
+  # GT filtering threshold is automatically calculated as:
+  # min_box_size = min_box_size_at_input_scale * roi_width / model_input_size
+  # 
+  # Examples:
+  # - ROI0 (1920->704): 8 * 1920 / 704 ≈ 21.8 pixels at original scale
+  # - ROI1 (704->704):  8 * 704 / 704 = 8.0 pixels at original scale
+
+# Performance settings
+performance:
+  num_workers: 32  # Number of parallel workers (null = auto-detect, 1 = single process)
+                     # Command-line: --num-workers
+                     # Recommended: 4-8 for typical workloads
+
+# ROI Ground Truth Processing (NEW!)
+roi_gt:
+  enabled: true                              # Enable ROI processing for ground truth
+  calib_root: "/mnt/mono3d/xdzhu_data/Mono3d/Mono3d_4face_2m_g1m3/driving_png/G1M3_AFS1616"  # Root path to calibration files (camera4.json)
+  roi_config: [1920, 960]                    # ROI configuration (matches training config)
+  roi_bottom_offset: 0                       # Pixels to trim from ROI bottom edge (matches training roi_bottom_offset)
+  # roi_config can be:
+  #   - [width, height]: ROI size mode (center crop based on vanishing point)
+  #   - [x1, y1, x2, y2]: ROI bounds mode (fixed bounds)
+  #   - dict: {'mode': 'size', 'width': 1920, 'height': 960} or 
+  #           {'mode': 'bounds', 'x1': 0, 'y1': 120, 'x2': 1920, 'y2': 1080}
+  # 
+  # IMPORTANT: This should match the ROI configuration used during training!
+  # For ROI0: [1920, 960] with roi_bottom_offset matching training config
+  # For ROI1: [704, 352] with roi_bottom_offset: 0
+
+# Class definitions
+classes:
+  3d_classes: [0, 1, 2, 3, 4, 5, 6, 7, 8]  # vehicle, bus, truck, tanker, unknown, pedestrian, bicycle, motorcyclist, tricycle
+  2d_classes: [9, 10, 11, 12]               # traffic_sign, wheel, plate, face
+  class_names:
+    0: "vehicle"
+    1: "bus"
+    2: "truck"
+    3: "tanker"
+    4: "unknown"
+    5: "pedestrian"
+    6: "bicycle"
+    7: "motorcyclist"
+    8: "tricycle"
+    9: "traffic_sign"
+    10: "wheel"
+    11: "plate"
+    12: "face"
+# Matching parameters (--iou-threshold)
+matching:
+  iou_threshold: 0.5  # Default: 0.5, IoU threshold for 2D matching
+  
+# 2D metrics configuration (--eval-2d-only, --conf-threshold, --ap-method)
+metrics_2d:
+  enabled: true                # Set to false with --eval-3d-only
+  conf_threshold: 0.4          # Default: 0.5, confidence threshold for precision/recall
+  ap_method: "voc2010"         # Default: "voc2010", choices: ["voc2010", "coco"]
+                               # voc2010: 11-point interpolation
+                               # coco: all-point interpolation
+  distance_ranges:             # Optional: distance-wise 2D evaluation (in metres, z3d).
+                               # Only applies to 3D-capable classes (vehicle/pedestrian/bicycle/rider)
+                               # that carry a ground-truth z3d depth value.
+                               # Independent from metrics_3d.distance_ranges — use coarser bins here
+                               # so the per-range GT counts are large enough to be statistically meaningful.
+    - [0, 30]                  # Near range
+    - [30, 60]                 # Mid range
+    - [60, 100]                # Far range
+    - [100, 999]               # Extreme range
+  lateral_roi: [-15, 15]     # Optional: lateral ROI pre-filter (metres, x3d).
+                               # When set, produces a second evaluation view that restricts all
+                               # counts (TP/FP/FN/GT) to objects within this lateral range.
+                               # e.g. [-15, 15] keeps only objects within 15 m of the centre line.
+  
+# 3D metrics configuration (--eval-3d-only)
+metrics_3d:
+  enabled: true                # Set to false with --eval-2d-only
+  heading_tolerance: "both"  # Heading error calculation mode (--heading-tolerance)
+                               # "strict": Standard calculation (default)
+                               # "relaxed": Consider 180° symmetry for symmetric objects
+                               # "both": Calculate and report both strict and relaxed metrics
+                               # Relaxed mode: for errors close to 180°, use min(error, π - error)
+  distance_ranges:             # Optional: longitudinal distance-wise evaluation (in meters, z-axis)
+    - [0, 10]                  # Near range
+    - [10, 20]                 
+    - [20, 30]
+    - [30, 40]                 # Medium range
+    - [40, 50]                 # Far range
+    - [50, 60]
+    - [60, 70]                 # Far range
+    - [70, 80]
+    - [80, 90]
+    - [90, 100]
+    - [100, 999]               # Very far range
+  lateral_distance_ranges:     # Optional: lateral distance-wise evaluation (in meters, x-axis)
+    - [-50, -40]              # Far left
+    - [-40, -30]                # Medium left
+    - [-30, -20]                # Near left far
+    - [-20, -10]                # Near left medium
+    - [-10, 0]                  # Near left
+    - [0, 10]                   # Near right
+    - [10, 20]                  # Medium right
+    - [20, 30]                # Near right far
+    - [30, 40]                # Medium right far
+    - [40, 50]                # Far right
+
+# Output configuration (--output-dir)
+output:
+  save_path: "eval_results_multiprocess/mono3d/{timestamp}"  # {timestamp} will be auto-replaced
+  formats: ["json", "txt"]            # Available: json, txt, csv
+  print_details: true                 # Print detailed per-class metrics
+  per_case_reports: true              # Generate per-case detailed reports
+
+# Command-line override examples:
+# 1. Override paths:
+#    python eval_tools/eval.py --config eval_config.yaml --det-path /new/path
+# 
+# 2. Override workers:
+#    python eval_tools/eval.py --config eval_config.yaml --num-workers 8
+#
+# 3. Enable ROI:
+#    python eval_tools/eval.py --config eval_config.yaml --roi 0 120 1920 1080 --roi-input-size 704 352
+#
+# 4. 2D only evaluation:
+#    python eval_tools/eval.py --config eval_config.yaml --eval-2d-only
+#
+# 5. Custom thresholds:
+#    python eval_tools/eval.py --config eval_config.yaml --iou-threshold 0.7 --conf-threshold 0.3

eval_tools/configs/eval_config_mono3d-roi1.yaml

@@ -0,0 +1,154 @@
+# Evaluation Configuration
+# This configuration file maps to command-line arguments in eval.py
+# All settings can be overridden by command-line arguments
+
+# Dataset paths (--det-path, --gt-path)
+# dataset:
+#   det_path: "/data1/dongying/Mono3d/G1M3/inference_results/mono3d/evalset_roi0"  # Root directory containing case folders with txt_results
+#   gt_path: "/data1/xdzhu/Testdata_0129"        # Root directory containing case folders with labels
+
+# CNCAP gt_path: "/mnt/mono3d/xdzhu_data/Mono3d/Mono3d_4face_2m_g1m3/driving_png/G1M3_AFS1616"
+
+dataset:
+  det_path: "/data1/dongying/Mono3d/G1M3/CNCAP_results/mono3d/evalset_roi1"  # Root directory containing case folders with txt_results
+  gt_path: "/mnt/mono3d/xdzhu_data/Mono3d/Mono3d_4face_2m_g1m3/driving_png/G1M3_AFS1616"        # Root directory containing case folders with labels
+  path_depth: 2  # Directory depth: 1 = det_path/case/txt_results, 2 = det_path/level1/case/txt_results
+  det_format: "auto"   # Detection file format: "auto" (probe json_results/ then txt_results/), "json", or "txt"
+  gt_format: "auto"    # Ground truth file format: "auto" (probe labels_json/ then labels/), "json", or "txt"
+  
+# Image properties (--img-width, --img-height)
+image:
+  width: 1920   # Default: 1920
+  height: 1080  # Default: 1080
+
+# Model configuration for GT filtering
+model:
+  input_size: 704  # Model input width (used for GT min box size calculation)
+  min_box_size_at_input_scale: 8  # Minimum box size at model input scale
+  # 
+  # GT filtering threshold is automatically calculated as:
+  # min_box_size = min_box_size_at_input_scale * roi_width / model_input_size
+  # 
+  # Examples:
+  # - ROI0 (1920->704): 8 * 1920 / 704 ≈ 21.8 pixels at original scale
+  # - ROI1 (704->704):  8 * 704 / 704 = 8.0 pixels at original scale
+
+# Performance settings
+performance:
+  num_workers: 32  # Number of parallel workers (null = auto-detect, 1 = single process)
+                     # Command-line: --num-workers
+                     # Recommended: 4-8 for typical workloads
+
+# ROI Ground Truth Processing (NEW!)
+roi_gt:
+  enabled: true                              # Enable ROI processing for ground truth
+  calib_root: "/mnt/mono3d/xdzhu_data/Mono3d/Mono3d_4face_2m_g1m3/driving_png/G1M3_AFS1616"  # Root path to calibration files (camera4.json)
+  roi_config: [704, 352]                    # ROI configuration (matches training config)
+  roi_bottom_offset: 0                       # Pixels to trim from ROI bottom edge (matches training roi_bottom_offset)
+  # roi_config can be:
+  #   - [width, height]: ROI size mode (center crop based on vanishing point)
+  #   - [x1, y1, x2, y2]: ROI bounds mode (fixed bounds)
+  #   - dict: {'mode': 'size', 'width': 1920, 'height': 960} or 
+  #           {'mode': 'bounds', 'x1': 0, 'y1': 120, 'x2': 1920, 'y2': 1080}
+  # 
+  # IMPORTANT: This should match the ROI configuration used during training!
+  # For ROI0: [1920, 960] with roi_bottom_offset matching training config
+  # For ROI1: [704, 352] with roi_bottom_offset: 0
+
+# Class definitions
+classes:
+  3d_classes: [0, 1, 2, 3, 4, 5, 6, 7, 8]  # vehicle, bus, truck, tanker, unknown, pedestrian, bicycle, motorcyclist, tricycle
+  2d_classes: [9, 10, 11, 12]               # traffic_sign, wheel, plate, face
+  class_names:
+    0: "vehicle"
+    1: "bus"
+    2: "truck"
+    3: "tanker"
+    4: "unknown"
+    5: "pedestrian"
+    6: "bicycle"
+    7: "motorcyclist"
+    8: "tricycle"
+    9: "traffic_sign"
+    10: "wheel"
+    11: "plate"
+    12: "face"
+# Matching parameters (--iou-threshold)
+matching:
+  iou_threshold: 0.5  # Default: 0.5, IoU threshold for 2D matching
+  
+# 2D metrics configuration (--eval-2d-only, --conf-threshold, --ap-method)
+metrics_2d:
+  enabled: true                # Set to false with --eval-3d-only
+  conf_threshold: 0.4          # Default: 0.5, confidence threshold for precision/recall
+  ap_method: "voc2010"         # Default: "voc2010", choices: ["voc2010", "coco"]
+                               # voc2010: 11-point interpolation
+                               # coco: all-point interpolation
+  distance_ranges:             # Optional: distance-wise 2D evaluation (in metres, z3d).
+                               # Only applies to 3D-capable classes (vehicle/pedestrian/bicycle/rider)
+                               # that carry a ground-truth z3d depth value.
+                               # Independent from metrics_3d.distance_ranges — use coarser bins here
+                               # so the per-range GT counts are large enough to be statistically meaningful.
+    - [0, 30]                  # Near range
+    - [30, 60]                 # Mid range
+    - [60, 100]                # Far range
+    - [100, 999]               # Extreme range
+  lateral_roi: [-15, 15]     # Optional: lateral ROI pre-filter (metres, x3d).
+                               # When set, produces a second evaluation view that restricts all
+                               # counts (TP/FP/FN/GT) to objects within this lateral range.
+                               # e.g. [-15, 15] keeps only objects within 15 m of the centre line.
+  
+# 3D metrics configuration (--eval-3d-only)
+metrics_3d:
+  enabled: true                # Set to false with --eval-2d-only
+  heading_tolerance: "both"  # Heading error calculation mode (--heading-tolerance)
+                               # "strict": Standard calculation (default)
+                               # "relaxed": Consider 180° symmetry for symmetric objects
+                               # "both": Calculate and report both strict and relaxed metrics
+                               # Relaxed mode: for errors close to 180°, use min(error, π - error)
+  distance_ranges:             # Optional: longitudinal distance-wise evaluation (in meters, z-axis)
+    - [0, 10]                  # Near range
+    - [10, 20]                 
+    - [20, 30]
+    - [30, 40]                 # Medium range
+    - [40, 50]                 # Far range
+    - [50, 60]
+    - [60, 70]                 # Far range
+    - [70, 80]
+    - [80, 90]
+    - [90, 100]
+    - [100, 999]               # Very far range
+  lateral_distance_ranges:     # Optional: lateral distance-wise evaluation (in meters, x-axis)
+    - [-50, -40]              # Far left
+    - [-40, -30]                # Medium left
+    - [-30, -20]                # Near left far
+    - [-20, -10]                # Near left medium
+    - [-10, 0]                  # Near left
+    - [0, 10]                   # Near right
+    - [10, 20]                  # Medium right
+    - [20, 30]                # Near right far
+    - [30, 40]                # Medium right far
+    - [40, 50]                # Far right
+
+# Output configuration (--output-dir)
+output:
+  save_path: "eval_results_multiprocess/mono3d/{timestamp}"  # {timestamp} will be auto-replaced
+  formats: ["json", "txt"]            # Available: json, txt, csv
+  print_details: true                 # Print detailed per-class metrics
+  per_case_reports: true              # Generate per-case detailed reports
+
+# Command-line override examples:
+# 1. Override paths:
+#    python eval_tools/eval.py --config eval_config.yaml --det-path /new/path
+# 
+# 2. Override workers:
+#    python eval_tools/eval.py --config eval_config.yaml --num-workers 8
+#
+# 3. Enable ROI:
+#    python eval_tools/eval.py --config eval_config.yaml --roi 0 120 1920 1080 --roi-input-size 704 352
+#
+# 4. 2D only evaluation:
+#    python eval_tools/eval.py --config eval_config.yaml --eval-2d-only
+#
+# 5. Custom thresholds:
+#    python eval_tools/eval.py --config eval_config.yaml --iou-threshold 0.7 --conf-threshold 0.3

eval_tools/configs/eval_config_yolov26s-roi0.yaml

@@ -0,0 +1,158 @@
+# Evaluation Configuration
+# This configuration file maps to command-line arguments in eval.py
+# All settings can be overridden by command-line arguments
+
+# Dataset paths (--det-path, --gt-path)
+# dataset:
+#   det_path: "/data1/dongying/Mono3d/G1M3/inference_results/yolov5s-300w-newdata/evalset_roi0"  # Root directory containing case folders with txt_results
+#   gt_path: "/data1/xdzhu/Testdata_0129"        # Root directory containing case folders with labels
+#   path_depth: 1  # Directory depth: 1 = det_path/case/txt_results, 2 = det_path/level1/case/txt_results
+
+# CNCAP gt_path: "/mnt/mono3d/xdzhu_data/Mono3d/Mono3d_4face_2m_g1m3/driving_png/G1M3_AFS1616"
+
+dataset:
+  det_path: "/data1/dongying/Mono3d/G1Q3/model_inference/KPI/DL_KPI_SCENE/model_20260403"  # Root directory containing case folders with txt_results
+  gt_path: "/data1/dongying/Mono3d/G1Q3/dataset_for_evaluation/DL_KPI_SCENE"        # Root directory containing case folders with labels
+  path_depth: 1  # Directory depth: 1 = det_path/case/txt_results, 2 = det_path/level1/case/txt_results
+  det_format: "json"   # Detection file format: "auto" (probe json_results/ then txt_results/), "json", or "txt"
+  gt_format: "json"    # Ground truth file format: "auto" (probe labels_json/ then labels/), "json", or "txt"
+  det_subdir: "predictions/roi0"  # Relative to each case directory; used before generic json_results/predictions probing
+  det_roi_filter: "0"  # Only evaluate detections with roi_id="0"; set to null (or remove) to evaluate all ROIs together
+
+# Image properties (--img-width, --img-height)
+image:
+  width: 1920   # Default: 1920
+  height: 1080  # Default: 1080
+
+# Model configuration for GT filtering
+model:
+  input_size: 768  # Model input width (used for GT min box size calculation)
+  min_box_size_at_input_scale: 8  # Minimum box size at model input scale
+  # 
+  # GT filtering threshold is automatically calculated as:
+  # min_box_size = min_box_size_at_input_scale * roi_width / model_input_size
+  # 
+  # Examples:
+  # - ROI0 (1920->704): 8 * 1920 / 704 ≈ 21.8 pixels at original scale
+  # - ROI1 (704->704):  8 * 704 / 704 = 8.0 pixels at original scale
+
+# Performance settings
+performance:
+  num_workers: 32  # Number of parallel workers (null = auto-detect, 1 = single process)
+                     # Command-line: --num-workers
+                     # Recommended: 4-8 for typical workloads
+
+# ROI Ground Truth Processing (NEW!)
+roi_gt:
+  enabled: true                              # Enable ROI processing for ground truth
+  calib_root: "/data1/dongying/Mono3d/G1Q3/dataset_for_evaluation/DL_KPI_SCENE"  # Root path to calibration files (camera4.json)
+  roi_config: [1920, 880]                    # ROI configuration (matches training config)
+  roi_bottom_offset: 0                       # Pixels to trim from ROI bottom edge (matches training roi_bottom_offset)
+  # roi_config can be:
+  #   - [width, height]: ROI size mode (center crop based on vanishing point)
+  #   - [x1, y1, x2, y2]: ROI bounds mode (fixed bounds)
+  #   - dict: {'mode': 'size', 'width': 1920, 'height': 960} or 
+  #           {'mode': 'bounds', 'x1': 0, 'y1': 120, 'x2': 1920, 'y2': 1080}
+  # 
+  # IMPORTANT: This should match the ROI configuration used during training!
+  # For ROI0: [1920, 960] with roi_bottom_offset matching training config
+  # For ROI1: [704, 352] with roi_bottom_offset: 0
+
+# Class definitions
+classes:
+  3d_classes: [0, 1, 2, 3, 4, 5, 6, 7, 8]  # vehicle, bus, truck, tanker, unknown, pedestrian, bicycle, motorcyclist, tricycle
+  2d_classes: [9, 10, 11, 12]               # traffic_sign, wheel, plate, face
+  class_names:
+    0: "vehicle"
+    1: "bus"
+    2: "truck"
+    3: "tanker"
+    4: "unknown"
+    5: "pedestrian"
+    6: "bicycle"
+    7: "motorcyclist"
+    8: "tricycle"
+    9: "traffic_sign"
+    10: "wheel"
+    11: "plate"
+    12: "face"
+# Matching parameters (--iou-threshold)
+matching:
+  iou_threshold: 0.5  # Default: 0.5, IoU threshold for 2D matching
+  
+# 2D metrics configuration (--eval-2d-only, --conf-threshold, --ap-method)
+metrics_2d:
+  enabled: true                # Set to false with --eval-3d-only
+  conf_threshold: 0.27          # Default: 0.5, confidence threshold for precision/recall
+  ap_method: "voc2010"         # Default: "voc2010", choices: ["voc2010", "coco"]
+                               # voc2010: 11-point interpolation
+                               # coco: all-point interpolation
+  distance_ranges:             # Optional: distance-wise 2D evaluation (in metres, z3d).
+                               # Only applies to 3D-capable classes (vehicle/pedestrian/bicycle/rider)
+                               # that carry a ground-truth z3d depth value.
+                               # Independent from metrics_3d.distance_ranges — use coarser bins here
+                               # so the per-range GT counts are large enough to be statistically meaningful.
+    - [0, 30]                  # Near range
+    - [30, 60]                 # Mid range
+    - [60, 100]                # Far range
+    - [100, 999]               # Extreme range
+  lateral_roi: [-15, 15]     # Optional: lateral ROI pre-filter (metres, x3d).
+                               # When set, produces a second evaluation view that restricts all
+                               # counts (TP/FP/FN/GT) to objects within this lateral range.
+                               # e.g. [-15, 15] keeps only objects within 15 m of the centre line.
+  
+# 3D metrics configuration (--eval-3d-only)
+metrics_3d:
+  enabled: true                # Set to false with --eval-2d-only
+  coordinate_system: "camera"  # 3D evaluation coordinate system: "camera" or "ego"
+                               # camera: use GT 3d_ori / Det xyzlhwyaw
+                               # ego:    use GT 3d_ori_ego / Det xyzlhwyaw_ego
+                               # Note: ego mode requires JSON GT/Det inputs
+  heading_tolerance: "both"  # Heading error calculation mode (--heading-tolerance)
+                               # "strict": Standard calculation (default)
+                               # "relaxed": Consider 180° symmetry for symmetric objects
+                               # "both": Calculate and report both strict and relaxed metrics
+                               # Relaxed mode: for errors close to 180°, use min(error, π - error)
+  distance_ranges:             # Optional: distance-wise evaluation (in meters)
+    - [0, 10]                  # Near range
+    - [10, 20]                 
+    - [20, 30]
+    - [30, 40]                 # Medium range
+    - [40, 50]                 # Far range
+    - [50, 60]
+    - [60, 70]                 # Far range
+    - [70, 80]
+    - [80, 90]
+    - [90, 100]
+    - [100, 999]               # Very far range
+  lateral_distance_ranges:     # Optional: lateral distance-wise evaluation (in meters, x-axis)
+    - [-50, -40]              # Far left
+    - [-40, -30]                # Medium left
+    - [-30, -20]                # Near left far
+    - [-20, -10]                # Near left medium
+    - [-10, 0]                  # Near left
+    - [0, 10]                   # Near right
+    - [10, 20]                  # Medium right
+    - [20, 30]                # Near right far
+    - [30, 40]                # Medium right far
+    - [40, 50]                # Far right
+
+# Output configuration (--output-dir)
+output:
+  save_path: "eval_results_multiprocess/yolov5s/{timestamp}"  # {timestamp} will be auto-replaced
+  formats: ["json", "txt"]            # Available: json, txt, csv
+  print_details: true                 # Print detailed per-class metrics
+  per_case_reports: true              # Generate per-case detailed reports
+
+# Command-line override examples:
+# 1. Override paths:
+#    python eval_tools/eval.py --config eval_config.yaml --det-path /new/path
+# 
+# 2. Override workers:
+#    python eval_tools/eval.py --config eval_config.yaml --num-workers 8
+#
+# 3. 2D only evaluation:
+#    python eval_tools/eval.py --config eval_config.yaml --eval-2d-only
+#
+# 4. Custom thresholds:
+#    python eval_tools/eval.py --config eval_config.yaml --iou-threshold 0.7 --conf-threshold 0.3

eval_tools/configs/eval_config_yolov26s-roi1.yaml

@@ -0,0 +1,105 @@
+# Evaluation Configuration — ROI1
+# Same as eval_config_yolov26s-roi0.yaml but evaluates only roi_id="1" detections
+# ROI1: narrow crop 768×352, centered on vanishing point (crop_center_mode=vxvy)
+
+dataset:
+  det_path: "/data1/dongying/Mono3d/G1Q3/model_inference/KPI/DL_KPI_SCENE/model_20260403"  # Root directory containing case folders with txt_results
+  gt_path: "/data1/dongying/Mono3d/G1Q3/dataset_for_evaluation/DL_KPI_SCENE"        # Root directory containing case folders with labels
+  path_depth: 1
+  det_format: "json"
+  gt_format: "json"
+  det_subdir: "predictions/roi1"  # Relative to each case directory; used before generic json_results/predictions probing
+  det_roi_filter: "1"  # Only evaluate detections with roi_id="1"; set to null to evaluate all ROIs together
+
+# Image properties
+image:
+  width: 1920
+  height: 1080
+
+# Model configuration for GT filtering
+model:
+  input_size: 768  # Model input width (used for GT min box size calculation)
+  min_box_size_at_input_scale: 8
+  # GT filtering threshold: min_box_size = 8 * 768 / 768 = 8.0 pixels at original scale (ROI1 is 768×352)
+
+# Performance settings
+performance:
+  num_workers: 32
+
+# ROI Ground Truth Processing
+roi_gt:
+  enabled: true
+  calib_root: "/data1/dongying/Mono3d/G1Q3/dataset_for_evaluation/DL_KPI_SCENE"  # Root path to calibration files (camera4.json)
+  roi_config: [768, 352]   # ROI1 crop size (narrow, centered on VP, crop_center_mode=vxvy)
+  roi_bottom_offset: 0
+
+# Class definitions
+classes:
+  3d_classes: [0, 1, 2, 3, 4, 5, 6, 7, 8]
+  2d_classes: [9, 10, 11, 12]
+  class_names:
+    0: "vehicle"
+    1: "bus"
+    2: "truck"
+    3: "tanker"
+    4: "unknown"
+    5: "pedestrian"
+    6: "bicycle"
+    7: "motorcyclist"
+    8: "tricycle"
+    9: "traffic_sign"
+    10: "wheel"
+    11: "plate"
+    12: "face"
+
+# Matching parameters
+matching:
+  iou_threshold: 0.5
+
+# 2D metrics configuration
+metrics_2d:
+  enabled: true
+  conf_threshold: 0.27
+  ap_method: "voc2010"
+  distance_ranges:
+    - [0, 30]
+    - [30, 60]
+    - [60, 100]
+    - [100, 999]
+  lateral_roi: [-15, 15]
+
+# 3D metrics configuration
+metrics_3d:
+  enabled: true
+  coordinate_system: "camera"
+  heading_tolerance: "both"
+  distance_ranges:
+    - [0, 10]
+    - [10, 20]
+    - [20, 30]
+    - [30, 40]
+    - [40, 50]
+    - [50, 60]
+    - [60, 70]
+    - [70, 80]
+    - [80, 90]
+    - [90, 100]
+    - [100, 999]
+  lateral_distance_ranges:
+    - [-50, -40]
+    - [-40, -30]
+    - [-30, -20]
+    - [-20, -10]
+    - [-10, 0]
+    - [0, 10]
+    - [10, 20]
+    - [20, 30]
+    - [30, 40]
+    - [40, 50]
+
+# Output configuration
+output:
+  save_path: "eval_results_multiprocess/yolov5s/{timestamp}"
+  formats: ["json", "txt"]
+  print_details: true
+  per_case_reports: true

eval_tools/configs/eval_config_yolov5s-roi0.yaml

@@ -0,0 +1,152 @@
+# Evaluation Configuration
+# This configuration file maps to command-line arguments in eval.py
+# All settings can be overridden by command-line arguments
+
+# Dataset paths (--det-path, --gt-path)
+# dataset:
+#   det_path: "/data1/dongying/Mono3d/G1M3/inference_results/yolov5s-300w-newdata/evalset_roi0"  # Root directory containing case folders with txt_results
+#   gt_path: "/data1/xdzhu/Testdata_0129"        # Root directory containing case folders with labels
+#   path_depth: 1  # Directory depth: 1 = det_path/case/txt_results, 2 = det_path/level1/case/txt_results
+
+# CNCAP gt_path: "/mnt/mono3d/xdzhu_data/Mono3d/Mono3d_4face_2m_g1m3/driving_png/G1M3_AFS1616"
+
+dataset:
+  det_path: "/data1/dongying/Mono3d/G1M3/inference_results/yolov5s-300w-newdata/evalset_roi0"  # Root directory containing case folders with txt_results
+  gt_path: "/data1/xdzhu/Testdata_0129"        # Root directory containing case folders with labels
+  path_depth: 1  # Directory depth: 1 = det_path/case/txt_results, 2 = det_path/level1/case/txt_results
+  det_format: "json"   # Detection file format: "auto" (probe json_results/ then txt_results/), "json", or "txt"
+  gt_format: "json"    # Ground truth file format: "auto" (probe labels_json/ then labels/), "json", or "txt"
+
+# Image properties (--img-width, --img-height)
+image:
+  width: 1920   # Default: 1920
+  height: 1080  # Default: 1080
+
+# Model configuration for GT filtering
+model:
+  input_size: 704  # Model input width (used for GT min box size calculation)
+  min_box_size_at_input_scale: 8  # Minimum box size at model input scale
+  # 
+  # GT filtering threshold is automatically calculated as:
+  # min_box_size = min_box_size_at_input_scale * roi_width / model_input_size
+  # 
+  # Examples:
+  # - ROI0 (1920->704): 8 * 1920 / 704 ≈ 21.8 pixels at original scale
+  # - ROI1 (704->704):  8 * 704 / 704 = 8.0 pixels at original scale
+
+# Performance settings
+performance:
+  num_workers: 32  # Number of parallel workers (null = auto-detect, 1 = single process)
+                     # Command-line: --num-workers
+                     # Recommended: 4-8 for typical workloads
+
+# ROI Ground Truth Processing (NEW!)
+roi_gt:
+  enabled: true                              # Enable ROI processing for ground truth
+  calib_root: "/data1/xdzhu/Testdata_0129"  # Root path to calibration files (camera4.json)
+  roi_config: [1920, 960]                    # ROI configuration (matches training config)
+  roi_bottom_offset: 0                       # Pixels to trim from ROI bottom edge (matches training roi_bottom_offset)
+  # roi_config can be:
+  #   - [width, height]: ROI size mode (center crop based on vanishing point)
+  #   - [x1, y1, x2, y2]: ROI bounds mode (fixed bounds)
+  #   - dict: {'mode': 'size', 'width': 1920, 'height': 960} or 
+  #           {'mode': 'bounds', 'x1': 0, 'y1': 120, 'x2': 1920, 'y2': 1080}
+  # 
+  # IMPORTANT: This should match the ROI configuration used during training!
+  # For ROI0: [1920, 960] with roi_bottom_offset matching training config
+  # For ROI1: [704, 352] with roi_bottom_offset: 0
+
+# Class definitions
+classes:
+  3d_classes: [0, 1, 2, 3, 4, 5, 6, 7, 8]  # vehicle, bus, truck, tanker, unknown, pedestrian, bicycle, motorcyclist, tricycle
+  2d_classes: [9, 10, 11, 12]               # traffic_sign, wheel, plate, face
+  class_names:
+    0: "vehicle"
+    1: "bus"
+    2: "truck"
+    3: "tanker"
+    4: "unknown"
+    5: "pedestrian"
+    6: "bicycle"
+    7: "motorcyclist"
+    8: "tricycle"
+    9: "traffic_sign"
+    10: "wheel"
+    11: "plate"
+    12: "face"
+# Matching parameters (--iou-threshold)
+matching:
+  iou_threshold: 0.5  # Default: 0.5, IoU threshold for 2D matching
+  
+# 2D metrics configuration (--eval-2d-only, --conf-threshold, --ap-method)
+metrics_2d:
+  enabled: true                # Set to false with --eval-3d-only
+  conf_threshold: 0.4          # Default: 0.5, confidence threshold for precision/recall
+  ap_method: "voc2010"         # Default: "voc2010", choices: ["voc2010", "coco"]
+                               # voc2010: 11-point interpolation
+                               # coco: all-point interpolation
+  distance_ranges:             # Optional: distance-wise 2D evaluation (in metres, z3d).
+                               # Only applies to 3D-capable classes (vehicle/pedestrian/bicycle/rider)
+                               # that carry a ground-truth z3d depth value.
+                               # Independent from metrics_3d.distance_ranges — use coarser bins here
+                               # so the per-range GT counts are large enough to be statistically meaningful.
+    - [0, 30]                  # Near range
+    - [30, 60]                 # Mid range
+    - [60, 100]                # Far range
+    - [100, 999]               # Extreme range
+  lateral_roi: [-15, 15]     # Optional: lateral ROI pre-filter (metres, x3d).
+                               # When set, produces a second evaluation view that restricts all
+                               # counts (TP/FP/FN/GT) to objects within this lateral range.
+                               # e.g. [-15, 15] keeps only objects within 15 m of the centre line.
+  
+# 3D metrics configuration (--eval-3d-only)
+metrics_3d:
+  enabled: true                # Set to false with --eval-2d-only
+  heading_tolerance: "both"  # Heading error calculation mode (--heading-tolerance)
+                               # "strict": Standard calculation (default)
+                               # "relaxed": Consider 180° symmetry for symmetric objects
+                               # "both": Calculate and report both strict and relaxed metrics
+                               # Relaxed mode: for errors close to 180°, use min(error, π - error)
+  distance_ranges:             # Optional: distance-wise evaluation (in meters)
+    - [0, 10]                  # Near range
+    - [10, 20]                 
+    - [20, 30]
+    - [30, 40]                 # Medium range
+    - [40, 50]                 # Far range
+    - [50, 60]
+    - [60, 70]                 # Far range
+    - [70, 80]
+    - [80, 90]
+    - [90, 100]
+    - [100, 999]               # Very far range
+  lateral_distance_ranges:     # Optional: lateral distance-wise evaluation (in meters, x-axis)
+    - [-50, -40]              # Far left
+    - [-40, -30]                # Medium left
+    - [-30, -20]                # Near left far
+    - [-20, -10]                # Near left medium
+    - [-10, 0]                  # Near left
+    - [0, 10]                   # Near right
+    - [10, 20]                  # Medium right
+    - [20, 30]                # Near right far
+    - [30, 40]                # Medium right far
+    - [40, 50]                # Far right
+
+# Output configuration (--output-dir)
+output:
+  save_path: "eval_results_multiprocess/yolov5s/{timestamp}"  # {timestamp} will be auto-replaced
+  formats: ["json", "txt"]            # Available: json, txt, csv
+  print_details: true                 # Print detailed per-class metrics
+  per_case_reports: true              # Generate per-case detailed reports
+
+# Command-line override examples:
+# 1. Override paths:
+#    python eval_tools/eval.py --config eval_config.yaml --det-path /new/path
+# 
+# 2. Override workers:
+#    python eval_tools/eval.py --config eval_config.yaml --num-workers 8
+#
+# 3. 2D only evaluation:
+#    python eval_tools/eval.py --config eval_config.yaml --eval-2d-only
+#
+# 4. Custom thresholds:
+#    python eval_tools/eval.py --config eval_config.yaml --iou-threshold 0.7 --conf-threshold 0.3

eval_tools/configs/eval_config_yolov5s-roi1.yaml

@@ -0,0 +1,152 @@
+# Evaluation Configuration
+# This configuration file maps to command-line arguments in eval.py
+# All settings can be overridden by command-line arguments
+
+# Dataset paths (--det-path, --gt-path)
+# dataset:
+#   det_path: "/data1/dongying/Mono3d/G1M3/inference_results/yolov5s-300w-newdata/evalset_roi0"  # Root directory containing case folders with txt_results
+#   gt_path: "/data1/xdzhu/Testdata_0129"        # Root directory containing case folders with labels
+#   path_depth: 1  # Directory depth: 1 = det_path/case/txt_results, 2 = det_path/level1/case/txt_results
+
+# CNCAP gt_path: "/mnt/mono3d/xdzhu_data/Mono3d/Mono3d_4face_2m_g1m3/driving_png/G1M3_AFS1616"
+
+dataset:
+  det_path: "/data1/dongying/Mono3d/G1M3/inference_results/yolov5s-300w-newdata/evalset_roi1"  # Root directory containing case folders with txt_results
+  gt_path: "/data1/xdzhu/Testdata_0129"        # Root directory containing case folders with labels
+  path_depth: 1  # Directory depth: 1 = det_path/case/txt_results, 2 = det_path/level1/case/txt_results
+  det_format: "auto"   # Detection file format: "auto" (probe json_results/ then txt_results/), "json", or "txt"
+  gt_format: "auto"    # Ground truth file format: "auto" (probe labels_json/ then labels/), "json", or "txt"
+
+# Image properties (--img-width, --img-height)
+image:
+  width: 1920   # Default: 1920
+  height: 1080  # Default: 1080
+
+# Model configuration for GT filtering
+model:
+  input_size: 704  # Model input width (used for GT min box size calculation)
+  min_box_size_at_input_scale: 8  # Minimum box size at model input scale
+  # 
+  # GT filtering threshold is automatically calculated as:
+  # min_box_size = min_box_size_at_input_scale * roi_width / model_input_size
+  # 
+  # Examples:
+  # - ROI0 (1920->704): 8 * 1920 / 704 ≈ 21.8 pixels at original scale
+  # - ROI1 (704->704):  8 * 704 / 704 = 8.0 pixels at original scale
+
+# Performance settings
+performance:
+  num_workers: 32  # Number of parallel workers (null = auto-detect, 1 = single process)
+                     # Command-line: --num-workers
+                     # Recommended: 4-8 for typical workloads
+
+# ROI Ground Truth Processing (NEW!)
+roi_gt:
+  enabled: true                              # Enable ROI processing for ground truth
+  calib_root: "/data1/xdzhu/Testdata_0129"  # Root path to calibration files (camera4.json)
+  roi_config: [704, 352]                    # ROI configuration (matches training config)
+  roi_bottom_offset: 0                       # Pixels to trim from ROI bottom edge (matches training roi_bottom_offset)
+  # roi_config can be:
+  #   - [width, height]: ROI size mode (center crop based on vanishing point)
+  #   - [x1, y1, x2, y2]: ROI bounds mode (fixed bounds)
+  #   - dict: {'mode': 'size', 'width': 1920, 'height': 960} or 
+  #           {'mode': 'bounds', 'x1': 0, 'y1': 120, 'x2': 1920, 'y2': 1080}
+  # 
+  # IMPORTANT: This should match the ROI configuration used during training!
+  # For ROI0: [1920, 960] with roi_bottom_offset matching training config
+  # For ROI1: [704, 352] with roi_bottom_offset: 0
+
+# Class definitions
+classes:
+  3d_classes: [0, 1, 2, 3, 4, 5, 6, 7, 8]  # vehicle, bus, truck, tanker, unknown, pedestrian, bicycle, motorcyclist, tricycle
+  2d_classes: [9, 10, 11, 12]               # traffic_sign, wheel, plate, face
+  class_names:
+    0: "vehicle"
+    1: "bus"
+    2: "truck"
+    3: "tanker"
+    4: "unknown"
+    5: "pedestrian"
+    6: "bicycle"
+    7: "motorcyclist"
+    8: "tricycle"
+    9: "traffic_sign"
+    10: "wheel"
+    11: "plate"
+    12: "face"
+# Matching parameters (--iou-threshold)
+matching:
+  iou_threshold: 0.5  # Default: 0.5, IoU threshold for 2D matching
+  
+# 2D metrics configuration (--eval-2d-only, --conf-threshold, --ap-method)
+metrics_2d:
+  enabled: true                # Set to false with --eval-3d-only
+  conf_threshold: 0.4          # Default: 0.5, confidence threshold for precision/recall
+  ap_method: "voc2010"         # Default: "voc2010", choices: ["voc2010", "coco"]
+                               # voc2010: 11-point interpolation
+                               # coco: all-point interpolation
+  distance_ranges:             # Optional: distance-wise 2D evaluation (in metres, z3d).
+                               # Only applies to 3D-capable classes (vehicle/pedestrian/bicycle/rider)
+                               # that carry a ground-truth z3d depth value.
+                               # Independent from metrics_3d.distance_ranges — use coarser bins here
+                               # so the per-range GT counts are large enough to be statistically meaningful.
+    - [0, 30]                  # Near range
+    - [30, 60]                 # Mid range
+    - [60, 100]                # Far range
+    - [100, 999]               # Extreme range
+  lateral_roi: [-15, 15]     # Optional: lateral ROI pre-filter (metres, x3d).
+                               # When set, produces a second evaluation view that restricts all
+                               # counts (TP/FP/FN/GT) to objects within this lateral range.
+                               # e.g. [-15, 15] keeps only objects within 15 m of the centre line.
+  
+# 3D metrics configuration (--eval-3d-only)
+metrics_3d:
+  enabled: true                # Set to false with --eval-2d-only
+  heading_tolerance: "both"  # Heading error calculation mode (--heading-tolerance)
+                               # "strict": Standard calculation (default)
+                               # "relaxed": Consider 180° symmetry for symmetric objects
+                               # "both": Calculate and report both strict and relaxed metrics
+                               # Relaxed mode: for errors close to 180°, use min(error, π - error)
+  distance_ranges:             # Optional: distance-wise evaluation (in meters)
+    - [0, 10]                  # Near range
+    - [10, 20]                 
+    - [20, 30]
+    - [30, 40]                 # Medium range
+    - [40, 50]                 # Far range
+    - [50, 60]
+    - [60, 70]                 # Far range
+    - [70, 80]
+    - [80, 90]
+    - [90, 100]
+    - [100, 999]               # Very far range
+  lateral_distance_ranges:     # Optional: lateral distance-wise evaluation (in meters, x-axis)
+    - [-50, -40]              # Far left
+    - [-40, -30]                # Medium left
+    - [-30, -20]                # Near left far
+    - [-20, -10]                # Near left medium
+    - [-10, 0]                  # Near left
+    - [0, 10]                   # Near right
+    - [10, 20]                  # Medium right
+    - [20, 30]                # Near right far
+    - [30, 40]                # Medium right far
+    - [40, 50]                # Far right
+
+# Output configuration (--output-dir)
+output:
+  save_path: "eval_results_multiprocess/yolov5s/{timestamp}"  # {timestamp} will be auto-replaced
+  formats: ["json", "txt"]            # Available: json, txt, csv
+  print_details: true                 # Print detailed per-class metrics
+  per_case_reports: true              # Generate per-case detailed reports
+
+# Command-line override examples:
+# 1. Override paths:
+#    python eval_tools/eval.py --config eval_config.yaml --det-path /new/path
+# 
+# 2. Override workers:
+#    python eval_tools/eval.py --config eval_config.yaml --num-workers 8
+#
+# 3. 2D only evaluation:
+#    python eval_tools/eval.py --config eval_config.yaml --eval-2d-only
+#
+# 4. Custom thresholds:
+#    python eval_tools/eval.py --config eval_config.yaml --iou-threshold 0.7 --conf-threshold 0.3

eval_tools/configs/eval_config_yolov5s_cncap-alignment-merge.yaml

@@ -0,0 +1,104 @@
+# Evaluation configuration for merged dual-ROI results.
+# This keeps single-ROI configs unchanged and enables merged evaluation only
+# when roi_gt.rois + model.input_size_by_roi are provided together.
+
+dataset:
+  det_path: "/data1/dongying/Mono3d/G1M3/cases_regular/model_20260317"
+  gt_path: "/data1/dongying/Mono3d/G1M3/Testdata_0129_alignment"
+  path_depth: 1
+  det_format: "json"
+  gt_format: "json"
+
+image:
+  width: 1920
+  height: 1080
+
+model:
+  input_size_by_roi:
+    roi0: 768
+    roi1: 768
+  min_box_size_at_input_scale: 8
+
+performance:
+  num_workers: 32
+
+roi_gt:
+  enabled: true
+  calib_root: "/data1/dongying/Mono3d/G1M3/Testdata_0129_alignment"
+  rois:
+    roi0:
+      roi_config: [1920, 960]
+      roi_bottom_offset: 80
+      roi_right_offset: 0
+      roi_use_true_vp_x: false
+    roi1:
+      roi_config: [768, 352]
+      roi_bottom_offset: 0
+      roi_right_offset: 0
+      roi_use_true_vp_x: true
+
+classes:
+  3d_classes: [0, 1, 2, 3, 4, 5, 6, 7, 8]  # vehicle, bus, truck, tanker, unknown, pedestrian, bicycle, motorcyclist, tricycle
+  2d_classes: [9, 10, 11, 12]               # traffic_sign, wheel, plate, face
+  class_names:
+    0: "vehicle"
+    1: "bus"
+    2: "truck"
+    3: "tanker"
+    4: "unknown"
+    5: "pedestrian"
+    6: "bicycle"
+    7: "motorcyclist"
+    8: "tricycle"
+    9: "traffic_sign"
+    10: "wheel"
+    11: "plate"
+    12: "face"
+
+matching:
+  iou_threshold: 0.5
+
+metrics_2d:
+  enabled: true
+  conf_threshold: 0.4
+  ap_method: "voc2010"
+  distance_ranges:
+    - [0, 30]
+    - [30, 60]
+    - [60, 100]
+    - [100, 999]
+  lateral_roi: [-15, 15]
+
+metrics_3d:
+  enabled: true
+  coordinate_system: "ego"
+  heading_tolerance: "both"
+  distance_ranges:
+    - [0, 10]
+    - [10, 20]
+    - [20, 30]
+    - [30, 40]
+    - [40, 50]
+    - [50, 60]
+    - [60, 70]
+    - [70, 80]
+    - [80, 90]
+    - [90, 100]
+    - [100, 999]
+  lateral_distance_ranges:
+    - [-50, -40]
+    - [-40, -30]
+    - [-30, -20]
+    - [-20, -10]
+    - [-10, 0]
+    - [0, 10]
+    - [10, 20]
+    - [20, 30]
+    - [30, 40]
+    - [40, 50]
+
+output:
+  save_path: "eval_results_multiprocess/yolov5s_merged/{timestamp}"
+  formats: ["json", "txt"]
+  print_details: true
+  per_case_reports: true

eval_tools/configs/eval_config_yolov5s_cncap-alignment-roi0.yaml

@@ -0,0 +1,156 @@
+# Evaluation Configuration
+# This configuration file maps to command-line arguments in eval.py
+# All settings can be overridden by command-line arguments
+
+# Dataset paths (--det-path, --gt-path)
+# dataset:
+#   det_path: "/data1/dongying/Mono3d/G1M3/inference_results/yolov5s-300w-newdata/evalset_roi0"  # Root directory containing case folders with txt_results
+#   gt_path: "/data1/xdzhu/Testdata_0129"        # Root directory containing case folders with labels
+#   path_depth: 1  # Directory depth: 1 = det_path/case/txt_results, 2 = det_path/level1/case/txt_results
+
+# CNCAP gt_path: "/mnt/mono3d/xdzhu_data/Mono3d/Mono3d_4face_2m_g1m3/driving_png/G1M3_AFS1616"
+
+dataset:
+  det_path: "/data1/dongying/Mono3d/G1M3/cases_regular/model_20260317"  # Root directory containing case folders with txt_results
+  gt_path: "/data1/dongying/Mono3d/G1M3/Testdata_0129_alignment"        # Root directory containing case folders with labels
+  path_depth: 1  # Directory depth: 1 = det_path/case/txt_results, 2 = det_path/level1/case/txt_results
+  det_format: "json"   # Detection file format: "auto" (probe json_results/ then txt_results/), "json", or "txt"
+  gt_format: "json"    # Ground truth file format: "auto" (probe labels_json/ then labels/), "json", or "txt"
+
+# Image properties (--img-width, --img-height)
+image:
+  width: 1920   # Default: 1920
+  height: 1080  # Default: 1080
+
+# Model configuration for GT filtering
+model:
+  input_size: 768  # Model input width (used for GT min box size calculation)
+  min_box_size_at_input_scale: 8  # Minimum box size at model input scale
+  # 
+  # GT filtering threshold is automatically calculated as:
+  # min_box_size = min_box_size_at_input_scale * roi_width / model_input_size
+  # 
+  # Examples:
+  # - ROI0 (1920->704): 8 * 1920 / 704 ≈ 21.8 pixels at original scale
+  # - ROI1 (704->704):  8 * 704 / 704 = 8.0 pixels at original scale
+
+# Performance settings
+performance:
+  num_workers: 32  # Number of parallel workers (null = auto-detect, 1 = single process)
+                     # Command-line: --num-workers
+                     # Recommended: 4-8 for typical workloads
+
+# ROI Ground Truth Processing (NEW!)
+roi_gt:
+  enabled: true                              # Enable ROI processing for ground truth
+  calib_root: "/data1/dongying/Mono3d/G1M3/Testdata_0129_alignment"  # Root path to calibration files (camera4.json)
+  roi_config: [1920, 960]                    # ROI configuration (matches training config)
+  roi_bottom_offset: 80                       # Pixels to trim from ROI bottom edge (matches training roi_bottom_offset)
+  # roi_config can be:
+  #   - [width, height]: ROI size mode (center crop based on vanishing point)
+  #   - [x1, y1, x2, y2]: ROI bounds mode (fixed bounds)
+  #   - dict: {'mode': 'size', 'width': 1920, 'height': 960} or 
+  #           {'mode': 'bounds', 'x1': 0, 'y1': 120, 'x2': 1920, 'y2': 1080}
+  # 
+  # IMPORTANT: This should match the ROI configuration used during training!
+  # For ROI0: [1920, 960] with roi_bottom_offset matching training config
+  # For ROI1: [704, 352] with roi_bottom_offset: 0
+
+# Class definitions
+classes:
+  3d_classes: [0, 1, 2, 3, 4, 5, 6, 7, 8]  # vehicle, bus, truck, tanker, unknown, pedestrian, bicycle, motorcyclist, tricycle
+  2d_classes: [9, 10, 11, 12]               # traffic_sign, wheel, plate, face
+  class_names:
+    0: "vehicle"
+    1: "bus"
+    2: "truck"
+    3: "tanker"
+    4: "unknown"
+    5: "pedestrian"
+    6: "bicycle"
+    7: "motorcyclist"
+    8: "tricycle"
+    9: "traffic_sign"
+    10: "wheel"
+    11: "plate"
+    12: "face"
+# Matching parameters (--iou-threshold)
+matching:
+  iou_threshold: 0.5  # Default: 0.5, IoU threshold for 2D matching
+  
+# 2D metrics configuration (--eval-2d-only, --conf-threshold, --ap-method)
+metrics_2d:
+  enabled: true                # Set to false with --eval-3d-only
+  conf_threshold: 0.4          # Default: 0.5, confidence threshold for precision/recall
+  ap_method: "voc2010"         # Default: "voc2010", choices: ["voc2010", "coco"]
+                               # voc2010: 11-point interpolation
+                               # coco: all-point interpolation
+  distance_ranges:             # Optional: distance-wise 2D evaluation (in metres, z3d).
+                               # Only applies to 3D-capable classes (vehicle/pedestrian/bicycle/rider)
+                               # that carry a ground-truth z3d depth value.
+                               # Independent from metrics_3d.distance_ranges — use coarser bins here
+                               # so the per-range GT counts are large enough to be statistically meaningful.
+    - [0, 30]                  # Near range
+    - [30, 60]                 # Mid range
+    - [60, 100]                # Far range
+    - [100, 999]               # Extreme range
+  lateral_roi: [-15, 15]     # Optional: lateral ROI pre-filter (metres, x3d).
+                               # When set, produces a second evaluation view that restricts all
+                               # counts (TP/FP/FN/GT) to objects within this lateral range.
+                               # e.g. [-15, 15] keeps only objects within 15 m of the centre line.
+  
+# 3D metrics configuration (--eval-3d-only)
+metrics_3d:
+  enabled: true                # Set to false with --eval-2d-only
+  coordinate_system: "ego"  # 3D evaluation coordinate system: "camera" or "ego"
+                               # camera: use GT 3d_ori / Det xyzlhwyaw
+                               # ego:    use GT 3d_ori_ego / Det xyzlhwyaw_ego
+                               # Note: ego mode requires JSON GT/Det inputs
+  heading_tolerance: "both"  # Heading error calculation mode (--heading-tolerance)
+                               # "strict": Standard calculation (default)
+                               # "relaxed": Consider 180° symmetry for symmetric objects
+                               # "both": Calculate and report both strict and relaxed metrics
+                               # Relaxed mode: for errors close to 180°, use min(error, π - error)
+  distance_ranges:             # Optional: distance-wise evaluation (in meters)
+    - [0, 10]                  # Near range
+    - [10, 20]                 
+    - [20, 30]
+    - [30, 40]                 # Medium range
+    - [40, 50]                 # Far range
+    - [50, 60]
+    - [60, 70]                 # Far range
+    - [70, 80]
+    - [80, 90]
+    - [90, 100]
+    - [100, 999]               # Very far range
+  lateral_distance_ranges:     # Optional: lateral distance-wise evaluation (in meters, x-axis)
+    - [-50, -40]              # Far left
+    - [-40, -30]                # Medium left
+    - [-30, -20]                # Near left far
+    - [-20, -10]                # Near left medium
+    - [-10, 0]                  # Near left
+    - [0, 10]                   # Near right
+    - [10, 20]                  # Medium right
+    - [20, 30]                # Near right far
+    - [30, 40]                # Medium right far
+    - [40, 50]                # Far right
+
+# Output configuration (--output-dir)
+output:
+  save_path: "eval_results_multiprocess/yolov5s/{timestamp}"  # {timestamp} will be auto-replaced
+  formats: ["json", "txt"]            # Available: json, txt, csv
+  print_details: true                 # Print detailed per-class metrics
+  per_case_reports: true              # Generate per-case detailed reports
+
+# Command-line override examples:
+# 1. Override paths:
+#    python eval_tools/eval.py --config eval_config.yaml --det-path /new/path
+# 
+# 2. Override workers:
+#    python eval_tools/eval.py --config eval_config.yaml --num-workers 8
+#
+# 3. 2D only evaluation:
+#    python eval_tools/eval.py --config eval_config.yaml --eval-2d-only
+#
+# 4. Custom thresholds:
+#    python eval_tools/eval.py --config eval_config.yaml --iou-threshold 0.7 --conf-threshold 0.3

eval_tools/configs/eval_config_yolov5s_cncap-alignment-roi1.yaml

@@ -0,0 +1,153 @@
+# Evaluation Configuration
+# This configuration file maps to command-line arguments in eval.py
+# All settings can be overridden by command-line arguments
+
+# Dataset paths (--det-path, --gt-path)
+# dataset:
+#   det_path: "/data1/dongying/Mono3d/G1M3/inference_results/yolov5s-300w-newdata/evalset_roi0"  # Root directory containing case folders with txt_results
+#   gt_path: "/data1/xdzhu/Testdata_0129"        # Root directory containing case folders with labels
+#   path_depth: 1  # Directory depth: 1 = det_path/case/txt_results, 2 = det_path/level1/case/txt_results
+
+# CNCAP gt_path: "/mnt/mono3d/xdzhu_data/Mono3d/Mono3d_4face_2m_g1m3/driving_png/G1M3_AFS1616"
+
+dataset:
+  det_path: "/data1/dongying/Mono3d/G1M3/inference_results/yolov5s-300w-newdata-cncap-test/evalset_roi1"  # Root directory containing case folders with txt_results
+  gt_path: "/data1/dongying/Mono3d/G1M3/Testdata_0129"        # Root directory containing case folders with labels
+  path_depth: 1  # Directory depth: 1 = det_path/case/txt_results, 2 = det_path/level1/case/txt_results
+  det_format: "json"   # Detection file format: "auto" (probe json_results/ then txt_results/), "json", or "txt"
+  gt_format: "json"    # Ground truth file format: "auto" (probe labels_json/ then labels/), "json", or "txt"
+
+# Image properties (--img-width, --img-height)
+image:
+  width: 1920   # Default: 1920
+  height: 1080  # Default: 1080
+
+# Model configuration for GT filtering
+model:
+  input_size: 768  # Model input width (used for GT min box size calculation)
+  min_box_size_at_input_scale: 8  # Minimum box size at model input scale
+  # 
+  # GT filtering threshold is automatically calculated as:
+  # min_box_size = min_box_size_at_input_scale * roi_width / model_input_size
+  # 
+  # Examples:
+  # - ROI0 (1920->704): 8 * 1920 / 704 ≈ 21.8 pixels at original scale
+  # - ROI1 (704->704):  8 * 704 / 704 = 8.0 pixels at original scale
+
+# Performance settings
+performance:
+  num_workers: 32  # Number of parallel workers (null = auto-detect, 1 = single process)
+                     # Command-line: --num-workers
+                     # Recommended: 4-8 for typical workloads
+
+# ROI Ground Truth Processing (NEW!)
+roi_gt:
+  enabled: true                              # Enable ROI processing for ground truth
+  calib_root: "/data1/dongying/Mono3d/G1M3/Testdata_0129"  # Root path to calibration files (camera4.json)
+  roi_config: [768, 352]                    # ROI configuration (matches training config)
+  roi_bottom_offset: 0                       # Pixels to trim from ROI bottom edge (matches training roi_bottom_offset)
+  roi_use_true_vp_x: true
+  # roi_config can be:
+  #   - [width, height]: ROI size mode (center crop based on vanishing point)
+  #   - [x1, y1, x2, y2]: ROI bounds mode (fixed bounds)
+  #   - dict: {'mode': 'size', 'width': 1920, 'height': 960} or 
+  #           {'mode': 'bounds', 'x1': 0, 'y1': 120, 'x2': 1920, 'y2': 1080}
+  # 
+  # IMPORTANT: This should match the ROI configuration used during training!
+  # For ROI0: [1920, 960] with roi_bottom_offset matching training config
+  # For ROI1: [704, 352] with roi_bottom_offset: 0
+
+# Class definitions
+classes:
+  3d_classes: [0, 1, 2, 3, 4, 5, 6, 7, 8]  # vehicle, bus, truck, tanker, unknown, pedestrian, bicycle, motorcyclist, tricycle
+  2d_classes: [9, 10, 11, 12]               # traffic_sign, wheel, plate, face
+  class_names:
+    0: "vehicle"
+    1: "bus"
+    2: "truck"
+    3: "tanker"
+    4: "unknown"
+    5: "pedestrian"
+    6: "bicycle"
+    7: "motorcyclist"
+    8: "tricycle"
+    9: "traffic_sign"
+    10: "wheel"
+    11: "plate"
+    12: "face"
+# Matching parameters (--iou-threshold)
+matching:
+  iou_threshold: 0.5  # Default: 0.5, IoU threshold for 2D matching
+  
+# 2D metrics configuration (--eval-2d-only, --conf-threshold, --ap-method)
+metrics_2d:
+  enabled: true                # Set to false with --eval-3d-only
+  conf_threshold: 0.4          # Default: 0.5, confidence threshold for precision/recall
+  ap_method: "voc2010"         # Default: "voc2010", choices: ["voc2010", "coco"]
+                               # voc2010: 11-point interpolation
+                               # coco: all-point interpolation
+  distance_ranges:             # Optional: distance-wise 2D evaluation (in metres, z3d).
+                               # Only applies to 3D-capable classes (vehicle/pedestrian/bicycle/rider)
+                               # that carry a ground-truth z3d depth value.
+                               # Independent from metrics_3d.distance_ranges — use coarser bins here
+                               # so the per-range GT counts are large enough to be statistically meaningful.
+    - [0, 30]                  # Near range
+    - [30, 60]                 # Mid range
+    - [60, 100]                # Far range
+    - [100, 999]               # Extreme range
+  lateral_roi: [-15, 15]     # Optional: lateral ROI pre-filter (metres, x3d).
+                               # When set, produces a second evaluation view that restricts all
+                               # counts (TP/FP/FN/GT) to objects within this lateral range.
+                               # e.g. [-15, 15] keeps only objects within 15 m of the centre line.
+  
+# 3D metrics configuration (--eval-3d-only)
+metrics_3d:
+  enabled: true                # Set to false with --eval-2d-only
+  heading_tolerance: "both"  # Heading error calculation mode (--heading-tolerance)
+                               # "strict": Standard calculation (default)
+                               # "relaxed": Consider 180° symmetry for symmetric objects
+                               # "both": Calculate and report both strict and relaxed metrics
+                               # Relaxed mode: for errors close to 180°, use min(error, π - error)
+  distance_ranges:             # Optional: distance-wise evaluation (in meters)
+    - [0, 10]                  # Near range
+    - [10, 20]                 
+    - [20, 30]
+    - [30, 40]                 # Medium range
+    - [40, 50]                 # Far range
+    - [50, 60]
+    - [60, 70]                 # Far range
+    - [70, 80]
+    - [80, 90]
+    - [90, 100]
+    - [100, 999]               # Very far range
+  lateral_distance_ranges:     # Optional: lateral distance-wise evaluation (in meters, x-axis)
+    - [-50, -40]              # Far left
+    - [-40, -30]                # Medium left
+    - [-30, -20]                # Near left far
+    - [-20, -10]                # Near left medium
+    - [-10, 0]                  # Near left
+    - [0, 10]                   # Near right
+    - [10, 20]                  # Medium right
+    - [20, 30]                # Near right far
+    - [30, 40]                # Medium right far
+    - [40, 50]                # Far right
+
+# Output configuration (--output-dir)
+output:
+  save_path: "eval_results_multiprocess/yolov5s/{timestamp}"  # {timestamp} will be auto-replaced
+  formats: ["json", "txt"]            # Available: json, txt, csv
+  print_details: true                 # Print detailed per-class metrics
+  per_case_reports: true              # Generate per-case detailed reports
+
+# Command-line override examples:
+# 1. Override paths:
+#    python eval_tools/eval.py --config eval_config.yaml --det-path /new/path
+# 
+# 2. Override workers:
+#    python eval_tools/eval.py --config eval_config.yaml --num-workers 8
+#
+# 3. 2D only evaluation:
+#    python eval_tools/eval.py --config eval_config.yaml --eval-2d-only
+#
+# 4. Custom thresholds:
+#    python eval_tools/eval.py --config eval_config.yaml --iou-threshold 0.7 --conf-threshold 0.3

eval_tools/configs/eval_config_yolov5s_cncap-roi0.yaml

@@ -0,0 +1,152 @@
+# Evaluation Configuration
+# This configuration file maps to command-line arguments in eval.py
+# All settings can be overridden by command-line arguments
+
+# Dataset paths (--det-path, --gt-path)
+# dataset:
+#   det_path: "/data1/dongying/Mono3d/G1M3/inference_results/yolov5s-300w-newdata/evalset_roi0"  # Root directory containing case folders with txt_results
+#   gt_path: "/data1/xdzhu/Testdata_0129"        # Root directory containing case folders with labels
+#   path_depth: 1  # Directory depth: 1 = det_path/case/txt_results, 2 = det_path/level1/case/txt_results
+
+# CNCAP gt_path: "/mnt/mono3d/xdzhu_data/Mono3d/Mono3d_4face_2m_g1m3/driving_png/G1M3_AFS1616"
+
+dataset:
+  det_path: "/data1/dongying/Mono3d/G1M3/inference_results/yolov5s-300w-newdata-cncap-test/evalset_roi0"  # Root directory containing case folders with txt_results
+  gt_path: "/data1/xdzhu/Testdata_0129"        # Root directory containing case folders with labels
+  path_depth: 1  # Directory depth: 1 = det_path/case/txt_results, 2 = det_path/level1/case/txt_results
+  det_format: "json"   # Detection file format: "auto" (probe json_results/ then txt_results/), "json", or "txt"
+  gt_format: "json"    # Ground truth file format: "auto" (probe labels_json/ then labels/), "json", or "txt"
+
+# Image properties (--img-width, --img-height)
+image:
+  width: 1920   # Default: 1920
+  height: 1080  # Default: 1080
+
+# Model configuration for GT filtering
+model:
+  input_size: 704  # Model input width (used for GT min box size calculation)
+  min_box_size_at_input_scale: 8  # Minimum box size at model input scale
+  # 
+  # GT filtering threshold is automatically calculated as:
+  # min_box_size = min_box_size_at_input_scale * roi_width / model_input_size
+  # 
+  # Examples:
+  # - ROI0 (1920->704): 8 * 1920 / 704 ≈ 21.8 pixels at original scale
+  # - ROI1 (704->704):  8 * 704 / 704 = 8.0 pixels at original scale
+
+# Performance settings
+performance:
+  num_workers: 32  # Number of parallel workers (null = auto-detect, 1 = single process)
+                     # Command-line: --num-workers
+                     # Recommended: 4-8 for typical workloads
+
+# ROI Ground Truth Processing (NEW!)
+roi_gt:
+  enabled: true                              # Enable ROI processing for ground truth
+  calib_root: "/data1/xdzhu/Testdata_0129"  # Root path to calibration files (camera4.json)
+  roi_config: [1920, 960]                    # ROI configuration (matches training config)
+  roi_bottom_offset: 0                       # Pixels to trim from ROI bottom edge (matches training roi_bottom_offset)
+  # roi_config can be:
+  #   - [width, height]: ROI size mode (center crop based on vanishing point)
+  #   - [x1, y1, x2, y2]: ROI bounds mode (fixed bounds)
+  #   - dict: {'mode': 'size', 'width': 1920, 'height': 960} or 
+  #           {'mode': 'bounds', 'x1': 0, 'y1': 120, 'x2': 1920, 'y2': 1080}
+  # 
+  # IMPORTANT: This should match the ROI configuration used during training!
+  # For ROI0: [1920, 960] with roi_bottom_offset matching training config
+  # For ROI1: [704, 352] with roi_bottom_offset: 0
+
+# Class definitions
+classes:
+  3d_classes: [0, 1, 2, 3, 4, 5, 6, 7, 8]  # vehicle, bus, truck, tanker, unknown, pedestrian, bicycle, motorcyclist, tricycle
+  2d_classes: [9, 10, 11, 12]               # traffic_sign, wheel, plate, face
+  class_names:
+    0: "vehicle"
+    1: "bus"
+    2: "truck"
+    3: "tanker"
+    4: "unknown"
+    5: "pedestrian"
+    6: "bicycle"
+    7: "motorcyclist"
+    8: "tricycle"
+    9: "traffic_sign"
+    10: "wheel"
+    11: "plate"
+    12: "face"
+# Matching parameters (--iou-threshold)
+matching:
+  iou_threshold: 0.5  # Default: 0.5, IoU threshold for 2D matching
+  
+# 2D metrics configuration (--eval-2d-only, --conf-threshold, --ap-method)
+metrics_2d:
+  enabled: true                # Set to false with --eval-3d-only
+  conf_threshold: 0.4          # Default: 0.5, confidence threshold for precision/recall
+  ap_method: "voc2010"         # Default: "voc2010", choices: ["voc2010", "coco"]
+                               # voc2010: 11-point interpolation
+                               # coco: all-point interpolation
+  distance_ranges:             # Optional: distance-wise 2D evaluation (in metres, z3d).
+                               # Only applies to 3D-capable classes (vehicle/pedestrian/bicycle/rider)
+                               # that carry a ground-truth z3d depth value.
+                               # Independent from metrics_3d.distance_ranges — use coarser bins here
+                               # so the per-range GT counts are large enough to be statistically meaningful.
+    - [0, 30]                  # Near range
+    - [30, 60]                 # Mid range
+    - [60, 100]                # Far range
+    - [100, 999]               # Extreme range
+  lateral_roi: [-15, 15]     # Optional: lateral ROI pre-filter (metres, x3d).
+                               # When set, produces a second evaluation view that restricts all
+                               # counts (TP/FP/FN/GT) to objects within this lateral range.
+                               # e.g. [-15, 15] keeps only objects within 15 m of the centre line.
+  
+# 3D metrics configuration (--eval-3d-only)
+metrics_3d:
+  enabled: true                # Set to false with --eval-2d-only
+  heading_tolerance: "both"  # Heading error calculation mode (--heading-tolerance)
+                               # "strict": Standard calculation (default)
+                               # "relaxed": Consider 180° symmetry for symmetric objects
+                               # "both": Calculate and report both strict and relaxed metrics
+                               # Relaxed mode: for errors close to 180°, use min(error, π - error)
+  distance_ranges:             # Optional: distance-wise evaluation (in meters)
+    - [0, 10]                  # Near range
+    - [10, 20]                 
+    - [20, 30]
+    - [30, 40]                 # Medium range
+    - [40, 50]                 # Far range
+    - [50, 60]
+    - [60, 70]                 # Far range
+    - [70, 80]
+    - [80, 90]
+    - [90, 100]
+    - [100, 999]               # Very far range
+  lateral_distance_ranges:     # Optional: lateral distance-wise evaluation (in meters, x-axis)
+    - [-50, -40]              # Far left
+    - [-40, -30]                # Medium left
+    - [-30, -20]                # Near left far
+    - [-20, -10]                # Near left medium
+    - [-10, 0]                  # Near left
+    - [0, 10]                   # Near right
+    - [10, 20]                  # Medium right
+    - [20, 30]                # Near right far
+    - [30, 40]                # Medium right far
+    - [40, 50]                # Far right
+
+# Output configuration (--output-dir)
+output:
+  save_path: "eval_results_multiprocess/yolov5s/{timestamp}"  # {timestamp} will be auto-replaced
+  formats: ["json", "txt"]            # Available: json, txt, csv
+  print_details: true                 # Print detailed per-class metrics
+  per_case_reports: true              # Generate per-case detailed reports
+
+# Command-line override examples:
+# 1. Override paths:
+#    python eval_tools/eval.py --config eval_config.yaml --det-path /new/path
+# 
+# 2. Override workers:
+#    python eval_tools/eval.py --config eval_config.yaml --num-workers 8
+#
+# 3. 2D only evaluation:
+#    python eval_tools/eval.py --config eval_config.yaml --eval-2d-only
+#
+# 4. Custom thresholds:
+#    python eval_tools/eval.py --config eval_config.yaml --iou-threshold 0.7 --conf-threshold 0.3

eval_tools/configs/eval_config_yolov5s_cncap-roi1.yaml

@@ -0,0 +1,152 @@
+# Evaluation Configuration
+# This configuration file maps to command-line arguments in eval.py
+# All settings can be overridden by command-line arguments
+
+# Dataset paths (--det-path, --gt-path)
+# dataset:
+#   det_path: "/data1/dongying/Mono3d/G1M3/inference_results/yolov5s-300w-newdata/evalset_roi0"  # Root directory containing case folders with txt_results
+#   gt_path: "/data1/xdzhu/Testdata_0129"        # Root directory containing case folders with labels
+#   path_depth: 1  # Directory depth: 1 = det_path/case/txt_results, 2 = det_path/level1/case/txt_results
+
+# CNCAP gt_path: "/mnt/mono3d/xdzhu_data/Mono3d/Mono3d_4face_2m_g1m3/driving_png/G1M3_AFS1616"
+
+dataset:
+  det_path: "/data1/dongying/Mono3d/G1M3/inference_results/yolov5s-300w-newdata-cncap-test/evalset_roi1"  # Root directory containing case folders with txt_results
+  gt_path: "/data1/xdzhu/Testdata_0129"        # Root directory containing case folders with labels
+  path_depth: 1  # Directory depth: 1 = det_path/case/txt_results, 2 = det_path/level1/case/txt_results
+  det_format: "auto"   # Detection file format: "auto" (probe json_results/ then txt_results/), "json", or "txt"
+  gt_format: "auto"    # Ground truth file format: "auto" (probe labels_json/ then labels/), "json", or "txt"
+
+# Image properties (--img-width, --img-height)
+image:
+  width: 1920   # Default: 1920
+  height: 1080  # Default: 1080
+
+# Model configuration for GT filtering
+model:
+  input_size: 704  # Model input width (used for GT min box size calculation)
+  min_box_size_at_input_scale: 8  # Minimum box size at model input scale
+  # 
+  # GT filtering threshold is automatically calculated as:
+  # min_box_size = min_box_size_at_input_scale * roi_width / model_input_size
+  # 
+  # Examples:
+  # - ROI0 (1920->704): 8 * 1920 / 704 ≈ 21.8 pixels at original scale
+  # - ROI1 (704->704):  8 * 704 / 704 = 8.0 pixels at original scale
+
+# Performance settings
+performance:
+  num_workers: 32  # Number of parallel workers (null = auto-detect, 1 = single process)
+                     # Command-line: --num-workers
+                     # Recommended: 4-8 for typical workloads
+
+# ROI Ground Truth Processing (NEW!)
+roi_gt:
+  enabled: true                              # Enable ROI processing for ground truth
+  calib_root: "/data1/xdzhu/Testdata_0129"  # Root path to calibration files (camera4.json)
+  roi_config: [704, 352]                    # ROI configuration (matches training config)
+  roi_bottom_offset: 0                       # Pixels to trim from ROI bottom edge (matches training roi_bottom_offset)
+  # roi_config can be:
+  #   - [width, height]: ROI size mode (center crop based on vanishing point)
+  #   - [x1, y1, x2, y2]: ROI bounds mode (fixed bounds)
+  #   - dict: {'mode': 'size', 'width': 1920, 'height': 960} or 
+  #           {'mode': 'bounds', 'x1': 0, 'y1': 120, 'x2': 1920, 'y2': 1080}
+  # 
+  # IMPORTANT: This should match the ROI configuration used during training!
+  # For ROI0: [1920, 960] with roi_bottom_offset matching training config
+  # For ROI1: [704, 352] with roi_bottom_offset: 0
+
+# Class definitions
+classes:
+  3d_classes: [0, 1, 2, 3, 4, 5, 6, 7, 8]  # vehicle, bus, truck, tanker, unknown, pedestrian, bicycle, motorcyclist, tricycle
+  2d_classes: [9, 10, 11, 12]               # traffic_sign, wheel, plate, face
+  class_names:
+    0: "vehicle"
+    1: "bus"
+    2: "truck"
+    3: "tanker"
+    4: "unknown"
+    5: "pedestrian"
+    6: "bicycle"
+    7: "motorcyclist"
+    8: "tricycle"
+    9: "traffic_sign"
+    10: "wheel"
+    11: "plate"
+    12: "face"
+# Matching parameters (--iou-threshold)
+matching:
+  iou_threshold: 0.5  # Default: 0.5, IoU threshold for 2D matching
+  
+# 2D metrics configuration (--eval-2d-only, --conf-threshold, --ap-method)
+metrics_2d:
+  enabled: true                # Set to false with --eval-3d-only
+  conf_threshold: 0.4          # Default: 0.5, confidence threshold for precision/recall
+  ap_method: "voc2010"         # Default: "voc2010", choices: ["voc2010", "coco"]
+                               # voc2010: 11-point interpolation
+                               # coco: all-point interpolation
+  distance_ranges:             # Optional: distance-wise 2D evaluation (in metres, z3d).
+                               # Only applies to 3D-capable classes (vehicle/pedestrian/bicycle/rider)
+                               # that carry a ground-truth z3d depth value.
+                               # Independent from metrics_3d.distance_ranges — use coarser bins here
+                               # so the per-range GT counts are large enough to be statistically meaningful.
+    - [0, 30]                  # Near range
+    - [30, 60]                 # Mid range
+    - [60, 100]                # Far range
+    - [100, 999]               # Extreme range
+  lateral_roi: [-15, 15]     # Optional: lateral ROI pre-filter (metres, x3d).
+                               # When set, produces a second evaluation view that restricts all
+                               # counts (TP/FP/FN/GT) to objects within this lateral range.
+                               # e.g. [-15, 15] keeps only objects within 15 m of the centre line.
+  
+# 3D metrics configuration (--eval-3d-only)
+metrics_3d:
+  enabled: true                # Set to false with --eval-2d-only
+  heading_tolerance: "both"  # Heading error calculation mode (--heading-tolerance)
+                               # "strict": Standard calculation (default)
+                               # "relaxed": Consider 180° symmetry for symmetric objects
+                               # "both": Calculate and report both strict and relaxed metrics
+                               # Relaxed mode: for errors close to 180°, use min(error, π - error)
+  distance_ranges:             # Optional: distance-wise evaluation (in meters)
+    - [0, 10]                  # Near range
+    - [10, 20]                 
+    - [20, 30]
+    - [30, 40]                 # Medium range
+    - [40, 50]                 # Far range
+    - [50, 60]
+    - [60, 70]                 # Far range
+    - [70, 80]
+    - [80, 90]
+    - [90, 100]
+    - [100, 999]               # Very far range
+  lateral_distance_ranges:     # Optional: lateral distance-wise evaluation (in meters, x-axis)
+    - [-50, -40]              # Far left
+    - [-40, -30]                # Medium left
+    - [-30, -20]                # Near left far
+    - [-20, -10]                # Near left medium
+    - [-10, 0]                  # Near left
+    - [0, 10]                   # Near right
+    - [10, 20]                  # Medium right
+    - [20, 30]                # Near right far
+    - [30, 40]                # Medium right far
+    - [40, 50]                # Far right
+
+# Output configuration (--output-dir)
+output:
+  save_path: "eval_results_multiprocess/yolov5s/{timestamp}"  # {timestamp} will be auto-replaced
+  formats: ["json", "txt"]            # Available: json, txt, csv
+  print_details: true                 # Print detailed per-class metrics
+  per_case_reports: true              # Generate per-case detailed reports
+
+# Command-line override examples:
+# 1. Override paths:
+#    python eval_tools/eval.py --config eval_config.yaml --det-path /new/path
+# 
+# 2. Override workers:
+#    python eval_tools/eval.py --config eval_config.yaml --num-workers 8
+#
+# 3. 2D only evaluation:
+#    python eval_tools/eval.py --config eval_config.yaml --eval-2d-only
+#
+# 4. Custom thresholds:
+#    python eval_tools/eval.py --config eval_config.yaml --iou-threshold 0.7 --conf-threshold 0.3

eval_tools/configs/eval_config_yolov5s_cncap_768-roi0.yaml

@@ -0,0 +1,156 @@
+# Evaluation Configuration
+# This configuration file maps to command-line arguments in eval.py
+# All settings can be overridden by command-line arguments
+
+# Dataset paths (--det-path, --gt-path)
+# dataset:
+#   det_path: "/data1/dongying/Mono3d/G1M3/inference_results/yolov5s-300w-newdata/evalset_roi0"  # Root directory containing case folders with txt_results
+#   gt_path: "/data1/xdzhu/Testdata_0129"        # Root directory containing case folders with labels
+#   path_depth: 1  # Directory depth: 1 = det_path/case/txt_results, 2 = det_path/level1/case/txt_results
+
+# CNCAP gt_path: "/mnt/mono3d/xdzhu_data/Mono3d/Mono3d_4face_2m_g1m3/driving_png/G1M3_AFS1616"
+
+dataset:
+  det_path: "/data1/dongying/Mono3d/G1M3/inference_results/yolov5s-300w-newdata-cncap-768x352/evalset_roi0"  # Root directory containing case folders with txt_results
+  gt_path: "/data1/xdzhu/Testdata_0129"        # Root directory containing case folders with labels
+  path_depth: 1  # Directory depth: 1 = det_path/case/txt_results, 2 = det_path/level1/case/txt_results
+  det_format: "json"   # Detection file format: "auto" (probe json_results/ then txt_results/), "json", or "txt"
+  gt_format: "json"    # Ground truth file format: "auto" (probe labels_json/ then labels/), "json", or "txt"
+
+# Image properties (--img-width, --img-height)
+image:
+  width: 1920   # Default: 1920
+  height: 1080  # Default: 1080
+
+# Model configuration for GT filtering
+model:
+  input_size: 768  # Model input width (used for GT min box size calculation)
+  min_box_size_at_input_scale: 8  # Minimum box size at model input scale
+  # 
+  # GT filtering threshold is automatically calculated as:
+  # min_box_size = min_box_size_at_input_scale * roi_width / model_input_size
+  # 
+  # Examples:
+  # - ROI0 (1920->768): 8 * 1920 / 768 = 20.0 pixels at original scale
+  # - ROI1 (768->768):  8 * 768 / 768 = 8.0 pixels at original scale
+
+# Performance settings
+performance:
+  num_workers: 32  # Number of parallel workers (null = auto-detect, 1 = single process)
+                     # Command-line: --num-workers
+                     # Recommended: 4-8 for typical workloads
+
+# ROI Ground Truth Processing (NEW!)
+roi_gt:
+  enabled: true                              # Enable ROI processing for ground truth
+  calib_root: "/data1/xdzhu/Testdata_0129"  # Root path to calibration files (camera4.json)
+  roi_config: [1920, 960]                    # ROI configuration (matches training config)
+  roi_bottom_offset: 80                      # Pixels to trim from ROI bottom edge (matches training roi_bottom_offset)
+  # roi_config can be:
+  #   - [width, height]: ROI size mode (center crop based on vanishing point)
+  #   - [x1, y1, x2, y2]: ROI bounds mode (fixed bounds)
+  #   - dict: {'mode': 'size', 'width': 1920, 'height': 960} or 
+  #           {'mode': 'bounds', 'x1': 0, 'y1': 120, 'x2': 1920, 'y2': 1080}
+  # 
+  # IMPORTANT: This should match the ROI configuration used during training!
+  # For ROI0: [1920, 960] with roi_bottom_offset matching training config
+  # For ROI1: [768, 352] with roi_bottom_offset: 0
+
+# Class definitions
+classes:
+  3d_classes: [0, 1, 2, 3, 4, 5, 6, 7, 8]  # vehicle, bus, truck, tanker, unknown, pedestrian, bicycle, motorcyclist, tricycle
+  2d_classes: [9, 10, 11, 12]               # traffic_sign, wheel, plate, face
+  class_names:
+    0: "vehicle"
+    1: "bus"
+    2: "truck"
+    3: "tanker"
+    4: "unknown"
+    5: "pedestrian"
+    6: "bicycle"
+    7: "motorcyclist"
+    8: "tricycle"
+    9: "traffic_sign"
+    10: "wheel"
+    11: "plate"
+    12: "face"
+# Matching parameters (--iou-threshold)
+matching:
+  iou_threshold: 0.5  # Default: 0.5, IoU threshold for 2D matching
+  
+# 2D metrics configuration (--eval-2d-only, --conf-threshold, --ap-method)
+metrics_2d:
+  enabled: true                # Set to false with --eval-3d-only
+  conf_threshold: 0.4          # Default: 0.5, confidence threshold for precision/recall
+  ap_method: "voc2010"         # Default: "voc2010", choices: ["voc2010", "coco"]
+                               # voc2010: 11-point interpolation
+                               # coco: all-point interpolation
+  distance_ranges:             # Optional: distance-wise 2D evaluation (in metres, z3d).
+                               # Only applies to 3D-capable classes (vehicle/pedestrian/bicycle/rider)
+                               # that carry a ground-truth z3d depth value.
+                               # Independent from metrics_3d.distance_ranges — use coarser bins here
+                               # so the per-range GT counts are large enough to be statistically meaningful.
+    - [0, 30]                  # Near range
+    - [30, 60]                 # Mid range
+    - [60, 100]                # Far range
+    - [100, 999]               # Extreme range
+  lateral_roi: [-15, 15]     # Optional: lateral ROI pre-filter (metres, x3d).
+                               # When set, produces a second evaluation view that restricts all
+                               # counts (TP/FP/FN/GT) to objects within this lateral range.
+                               # e.g. [-15, 15] keeps only objects within 15 m of the centre line.
+  
+# 3D metrics configuration (--eval-3d-only)
+metrics_3d:
+  enabled: true                # Set to false with --eval-2d-only
+  coordinate_system: "camera"  # 3D evaluation coordinate system: "camera" or "ego"
+                               # camera: use GT 3d_ori / Det xyzlhwyaw
+                               # ego:    use GT 3d_ori_ego / Det xyzlhwyaw_ego
+                               # Note: ego mode requires JSON GT/Det inputs
+  heading_tolerance: "both"  # Heading error calculation mode (--heading-tolerance)
+                               # "strict": Standard calculation (default)
+                               # "relaxed": Consider 180° symmetry for symmetric objects
+                               # "both": Calculate and report both strict and relaxed metrics
+                               # Relaxed mode: for errors close to 180°, use min(error, π - error)
+  distance_ranges:             # Optional: distance-wise evaluation (in meters)
+    - [0, 10]                  # Near range
+    - [10, 20]                 
+    - [20, 30]
+    - [30, 40]                 # Medium range
+    - [40, 50]                 # Far range
+    - [50, 60]
+    - [60, 70]                 # Far range
+    - [70, 80]
+    - [80, 90]
+    - [90, 100]
+    - [100, 999]               # Very far range
+  lateral_distance_ranges:     # Optional: lateral distance-wise evaluation (in meters, x-axis)
+    - [-50, -40]              # Far left
+    - [-40, -30]                # Medium left
+    - [-30, -20]                # Near left far
+    - [-20, -10]                # Near left medium
+    - [-10, 0]                  # Near left
+    - [0, 10]                   # Near right
+    - [10, 20]                  # Medium right
+    - [20, 30]                # Near right far
+    - [30, 40]                # Medium right far
+    - [40, 50]                # Far right
+
+# Output configuration (--output-dir)
+output:
+  save_path: "eval_results_multiprocess/yolov5s/{timestamp}"  # {timestamp} will be auto-replaced
+  formats: ["json", "txt"]            # Available: json, txt, csv
+  print_details: true                 # Print detailed per-class metrics
+  per_case_reports: true              # Generate per-case detailed reports
+
+# Command-line override examples:
+# 1. Override paths:
+#    python eval_tools/eval.py --config eval_config.yaml --det-path /new/path
+# 
+# 2. Override workers:
+#    python eval_tools/eval.py --config eval_config.yaml --num-workers 8
+#
+# 3. 2D only evaluation:
+#    python eval_tools/eval.py --config eval_config.yaml --eval-2d-only
+#
+# 4. Custom thresholds:
+#    python eval_tools/eval.py --config eval_config.yaml --iou-threshold 0.7 --conf-threshold 0.3

eval_tools/configs/eval_config_yolov5s_cncap_768-roi1.yaml

@@ -0,0 +1,157 @@
+# Evaluation Configuration
+# This configuration file maps to command-line arguments in eval.py
+# All settings can be overridden by command-line arguments
+
+# Dataset paths (--det-path, --gt-path)
+# dataset:
+#   det_path: "/data1/dongying/Mono3d/G1M3/inference_results/yolov5s-300w-newdata/evalset_roi0"  # Root directory containing case folders with txt_results
+#   gt_path: "/data1/xdzhu/Testdata_0129"        # Root directory containing case folders with labels
+#   path_depth: 1  # Directory depth: 1 = det_path/case/txt_results, 2 = det_path/level1/case/txt_results
+
+# CNCAP gt_path: "/mnt/mono3d/xdzhu_data/Mono3d/Mono3d_4face_2m_g1m3/driving_png/G1M3_AFS1616"
+
+dataset:
+  det_path: "/data1/dongying/Mono3d/G1M3/inference_results/yolov5s-300w-newdata-cncap-768x352/evalset_roi1"  # Root directory containing case folders with txt_results
+  gt_path: "/data1/xdzhu/Testdata_0129"        # Root directory containing case folders with labels
+  path_depth: 1  # Directory depth: 1 = det_path/case/txt_results, 2 = det_path/level1/case/txt_results
+  det_format: "auto"   # Detection file format: "auto" (probe json_results/ then txt_results/), "json", or "txt"
+  gt_format: "auto"    # Ground truth file format: "auto" (probe labels_json/ then labels/), "json", or "txt"
+
+# Image properties (--img-width, --img-height)
+image:
+  width: 1920   # Default: 1920
+  height: 1080  # Default: 1080
+
+# Model configuration for GT filtering
+model:
+  input_size: 768  # Model input width (used for GT min box size calculation)
+  min_box_size_at_input_scale: 8  # Minimum box size at model input scale
+  # 
+  # GT filtering threshold is automatically calculated as:
+  # min_box_size = min_box_size_at_input_scale * roi_width / model_input_size
+  # 
+  # Examples:
+  # - ROI0 (1920->704): 8 * 1920 / 704 ≈ 21.8 pixels at original scale
+  # - ROI1 (704->704):  8 * 704 / 704 = 8.0 pixels at original scale
+
+# Performance settings
+performance:
+  num_workers: 32  # Number of parallel workers (null = auto-detect, 1 = single process)
+                     # Command-line: --num-workers
+                     # Recommended: 4-8 for typical workloads
+
+# ROI Ground Truth Processing (NEW!)
+roi_gt:
+  enabled: true                              # Enable ROI processing for ground truth
+  calib_root: "/data1/xdzhu/Testdata_0129"  # Root path to calibration files (camera4.json)
+  roi_config: [786, 352]                    # ROI configuration (matches training config)
+  roi_bottom_offset: 0                       # Pixels to trim from ROI bottom edge (matches training roi_bottom_offset)
+  roi_use_true_vp_x: true
+  # roi_config can be:
+  #   - [width, height]: ROI size mode (center crop based on vanishing point)
+  #   - [x1, y1, x2, y2]: ROI bounds mode (fixed bounds)
+  #   - dict: {'mode': 'size', 'width': 1920, 'height': 960} or 
+  #           {'mode': 'bounds', 'x1': 0, 'y1': 120, 'x2': 1920, 'y2': 1080}
+  # 
+  # IMPORTANT: This should match the ROI configuration used during training!
+  # For ROI0: [1920, 960] with roi_bottom_offset matching training config
+  # For ROI1: [786, 352] with roi_bottom_offset: 0
+
+# Class definitions
+classes:
+  3d_classes: [0, 1, 2, 3, 4, 5, 6, 7, 8]  # vehicle, bus, truck, tanker, unknown, pedestrian, bicycle, motorcyclist, tricycle
+  2d_classes: [9, 10, 11, 12]               # traffic_sign, wheel, plate, face
+  class_names:
+    0: "vehicle"
+    1: "bus"
+    2: "truck"
+    3: "tanker"
+    4: "unknown"
+    5: "pedestrian"
+    6: "bicycle"
+    7: "motorcyclist"
+    8: "tricycle"
+    9: "traffic_sign"
+    10: "wheel"
+    11: "plate"
+    12: "face"
+# Matching parameters (--iou-threshold)
+matching:
+  iou_threshold: 0.5  # Default: 0.5, IoU threshold for 2D matching
+  
+# 2D metrics configuration (--eval-2d-only, --conf-threshold, --ap-method)
+metrics_2d:
+  enabled: true                # Set to false with --eval-3d-only
+  conf_threshold: 0.4          # Default: 0.5, confidence threshold for precision/recall
+  ap_method: "voc2010"         # Default: "voc2010", choices: ["voc2010", "coco"]
+                               # voc2010: 11-point interpolation
+                               # coco: all-point interpolation
+  distance_ranges:             # Optional: distance-wise 2D evaluation (in metres, z3d).
+                               # Only applies to 3D-capable classes (vehicle/pedestrian/bicycle/rider)
+                               # that carry a ground-truth z3d depth value.
+                               # Independent from metrics_3d.distance_ranges — use coarser bins here
+                               # so the per-range GT counts are large enough to be statistically meaningful.
+    - [0, 30]                  # Near range
+    - [30, 60]                 # Mid range
+    - [60, 100]                # Far range
+    - [100, 999]               # Extreme range
+  lateral_roi: [-15, 15]     # Optional: lateral ROI pre-filter (metres, x3d).
+                               # When set, produces a second evaluation view that restricts all
+                               # counts (TP/FP/FN/GT) to objects within this lateral range.
+                               # e.g. [-15, 15] keeps only objects within 15 m of the centre line.
+  
+# 3D metrics configuration (--eval-3d-only)
+metrics_3d:
+  enabled: true                # Set to false with --eval-2d-only
+  coordinate_system: "camera"  # 3D evaluation coordinate system: "camera" or "ego"
+                               # camera: use GT 3d_ori / Det xyzlhwyaw
+                               # ego:    use GT 3d_ori_ego / Det xyzlhwyaw_ego
+                               # Note: ego mode requires JSON GT/Det inputs
+  heading_tolerance: "both"  # Heading error calculation mode (--heading-tolerance)
+                               # "strict": Standard calculation (default)
+                               # "relaxed": Consider 180° symmetry for symmetric objects
+                               # "both": Calculate and report both strict and relaxed metrics
+                               # Relaxed mode: for errors close to 180°, use min(error, π - error)
+  distance_ranges:             # Optional: distance-wise evaluation (in meters)
+    - [0, 10]                  # Near range
+    - [10, 20]                 
+    - [20, 30]
+    - [30, 40]                 # Medium range
+    - [40, 50]                 # Far range
+    - [50, 60]
+    - [60, 70]                 # Far range
+    - [70, 80]
+    - [80, 90]
+    - [90, 100]
+    - [100, 999]               # Very far range
+  lateral_distance_ranges:     # Optional: lateral distance-wise evaluation (in meters, x-axis)
+    - [-50, -40]              # Far left
+    - [-40, -30]                # Medium left
+    - [-30, -20]                # Near left far
+    - [-20, -10]                # Near left medium
+    - [-10, 0]                  # Near left
+    - [0, 10]                   # Near right
+    - [10, 20]                  # Medium right
+    - [20, 30]                # Near right far
+    - [30, 40]                # Medium right far
+    - [40, 50]                # Far right
+
+# Output configuration (--output-dir)
+output:
+  save_path: "eval_results_multiprocess/yolov5s/{timestamp}"  # {timestamp} will be auto-replaced
+  formats: ["json", "txt"]            # Available: json, txt, csv
+  print_details: true                 # Print detailed per-class metrics
+  per_case_reports: true              # Generate per-case detailed reports
+
+# Command-line override examples:
+# 1. Override paths:
+#    python eval_tools/eval.py --config eval_config.yaml --det-path /new/path
+# 
+# 2. Override workers:
+#    python eval_tools/eval.py --config eval_config.yaml --num-workers 8
+#
+# 3. 2D only evaluation:
+#    python eval_tools/eval.py --config eval_config.yaml --eval-2d-only
+#
+# 4. Custom thresholds:
+#    python eval_tools/eval.py --config eval_config.yaml --iou-threshold 0.7 --conf-threshold 0.3

eval_tools/configs/eval_config_yolov5s_deploy-roi1.yaml

@@ -0,0 +1,152 @@
+# Evaluation Configuration
+# This configuration file maps to command-line arguments in eval.py
+# All settings can be overridden by command-line arguments
+
+# Dataset paths (--det-path, --gt-path)
+# dataset:
+#   det_path: "/data1/dongying/Mono3d/G1M3/inference_results/yolov5s-300w-newdata/evalset_roi0"  # Root directory containing case folders with txt_results
+#   gt_path: "/data1/xdzhu/Testdata_0129"        # Root directory containing case folders with labels
+#   path_depth: 1  # Directory depth: 1 = det_path/case/txt_results, 2 = det_path/level1/case/txt_results
+
+# CNCAP gt_path: "/mnt/mono3d/xdzhu_data/Mono3d/Mono3d_4face_2m_g1m3/driving_png/G1M3_AFS1616"
+
+dataset:
+  det_path: "/data1/dongying/Mono3d/G1M3/inference_results/deploy/evalset_roi1"  # Root directory containing case folders with txt_results
+  gt_path: "/data1/shaozk/mining/G1M3_FDL2232_20251201_20251212/origins"        # Root directory containing case folders with labels
+  path_depth: 1  # Directory depth: 1 = det_path/case/txt_results, 2 = det_path/level1/case/txt_results
+  det_format: "json"   # Detection file format: "auto" (probe json_results/ then txt_results/), "json", or "txt"
+  gt_format: "json"    # Ground truth file format: "auto" (probe labels_json/ then labels/), "json", or "txt"
+
+# Image properties (--img-width, --img-height)
+image:
+  width: 1920   # Default: 1920
+  height: 1080  # Default: 1080
+
+# Model configuration for GT filtering
+model:
+  input_size: 704  # Model input width (used for GT min box size calculation)
+  min_box_size_at_input_scale: 8  # Minimum box size at model input scale
+  # 
+  # GT filtering threshold is automatically calculated as:
+  # min_box_size = min_box_size_at_input_scale * roi_width / model_input_size
+  # 
+  # Examples:
+  # - ROI0 (1920->704): 8 * 1920 / 704 ≈ 21.8 pixels at original scale
+  # - ROI1 (704->704):  8 * 704 / 704 = 8.0 pixels at original scale
+
+# Performance settings
+performance:
+  num_workers: 32  # Number of parallel workers (null = auto-detect, 1 = single process)
+                     # Command-line: --num-workers
+                     # Recommended: 4-8 for typical workloads
+
+# ROI Ground Truth Processing (NEW!)
+roi_gt:
+  enabled: true                              # Enable ROI processing for ground truth
+  calib_root: "/data1/shaozk/mining/G1M3_FDL2232_20251201_20251212/origins"  # Root path to calibration files (camera4.json)
+  roi_config: [704, 352]                    # ROI configuration (matches training config)
+  roi_bottom_offset: 0                       # Pixels to trim from ROI bottom edge (matches training roi_bottom_offset)
+  # roi_config can be:
+  #   - [width, height]: ROI size mode (center crop based on vanishing point)
+  #   - [x1, y1, x2, y2]: ROI bounds mode (fixed bounds)
+  #   - dict: {'mode': 'size', 'width': 1920, 'height': 960} or 
+  #           {'mode': 'bounds', 'x1': 0, 'y1': 120, 'x2': 1920, 'y2': 1080}
+  # 
+  # IMPORTANT: This should match the ROI configuration used during training!
+  # For ROI0: [1920, 960] with roi_bottom_offset matching training config
+  # For ROI1: [704, 352] with roi_bottom_offset: 0
+
+# Class definitions
+classes:
+  3d_classes: [0, 1, 2, 3, 4, 5, 6, 7, 8]  # vehicle, bus, truck, tanker, unknown, pedestrian, bicycle, motorcyclist, tricycle
+  2d_classes: [9, 10, 11, 12]               # traffic_sign, wheel, plate, face
+  class_names:
+    0: "vehicle"
+    1: "bus"
+    2: "truck"
+    3: "tanker"
+    4: "unknown"
+    5: "pedestrian"
+    6: "bicycle"
+    7: "motorcyclist"
+    8: "tricycle"
+    9: "traffic_sign"
+    10: "wheel"
+    11: "plate"
+    12: "face"
+# Matching parameters (--iou-threshold)
+matching:
+  iou_threshold: 0.5  # Default: 0.5, IoU threshold for 2D matching
+  
+# 2D metrics configuration (--eval-2d-only, --conf-threshold, --ap-method)
+metrics_2d:
+  enabled: true                # Set to false with --eval-3d-only
+  conf_threshold: 0.4          # Default: 0.5, confidence threshold for precision/recall
+  ap_method: "voc2010"         # Default: "voc2010", choices: ["voc2010", "coco"]
+                               # voc2010: 11-point interpolation
+                               # coco: all-point interpolation
+  distance_ranges:             # Optional: distance-wise 2D evaluation (in metres, z3d).
+                               # Only applies to 3D-capable classes (vehicle/pedestrian/bicycle/rider)
+                               # that carry a ground-truth z3d depth value.
+                               # Independent from metrics_3d.distance_ranges — use coarser bins here
+                               # so the per-range GT counts are large enough to be statistically meaningful.
+    - [0, 30]                  # Near range
+    - [30, 60]                 # Mid range
+    - [60, 100]                # Far range
+    - [100, 999]               # Extreme range
+  lateral_roi: [-15, 15]     # Optional: lateral ROI pre-filter (metres, x3d).
+                               # When set, produces a second evaluation view that restricts all
+                               # counts (TP/FP/FN/GT) to objects within this lateral range.
+                               # e.g. [-15, 15] keeps only objects within 15 m of the centre line.
+  
+# 3D metrics configuration (--eval-3d-only)
+metrics_3d:
+  enabled: true                # Set to false with --eval-2d-only
+  heading_tolerance: "both"  # Heading error calculation mode (--heading-tolerance)
+                               # "strict": Standard calculation (default)
+                               # "relaxed": Consider 180° symmetry for symmetric objects
+                               # "both": Calculate and report both strict and relaxed metrics
+                               # Relaxed mode: for errors close to 180°, use min(error, π - error)
+  distance_ranges:             # Optional: distance-wise evaluation (in meters)
+    - [0, 10]                  # Near range
+    - [10, 20]                 
+    - [20, 30]
+    - [30, 40]                 # Medium range
+    - [40, 50]                 # Far range
+    - [50, 60]
+    - [60, 70]                 # Far range
+    - [70, 80]
+    - [80, 90]
+    - [90, 100]
+    - [100, 999]               # Very far range
+  lateral_distance_ranges:     # Optional: lateral distance-wise evaluation (in meters, x-axis)
+    - [-50, -40]              # Far left
+    - [-40, -30]                # Medium left
+    - [-30, -20]                # Near left far
+    - [-20, -10]                # Near left medium
+    - [-10, 0]                  # Near left
+    - [0, 10]                   # Near right
+    - [10, 20]                  # Medium right
+    - [20, 30]                # Near right far
+    - [30, 40]                # Medium right far
+    - [40, 50]                # Far right
+
+# Output configuration (--output-dir)
+output:
+  save_path: "eval_results_multiprocess/yolov5s/{timestamp}"  # {timestamp} will be auto-replaced
+  formats: ["json", "txt"]            # Available: json, txt, csv
+  print_details: true                 # Print detailed per-class metrics
+  per_case_reports: true              # Generate per-case detailed reports
+
+# Command-line override examples:
+# 1. Override paths:
+#    python eval_tools/eval.py --config eval_config.yaml --det-path /new/path
+# 
+# 2. Override workers:
+#    python eval_tools/eval.py --config eval_config.yaml --num-workers 8
+#
+# 3. 2D only evaluation:
+#    python eval_tools/eval.py --config eval_config.yaml --eval-2d-only
+#
+# 4. Custom thresholds:
+#    python eval_tools/eval.py --config eval_config.yaml --iou-threshold 0.7 --conf-threshold 0.3

eval_tools/configs/eval_config_yolov5s_dl-roi1.yaml

@@ -0,0 +1,152 @@
+# Evaluation Configuration
+# This configuration file maps to command-line arguments in eval.py
+# All settings can be overridden by command-line arguments
+
+# Dataset paths (--det-path, --gt-path)
+# dataset:
+#   det_path: "/data1/dongying/Mono3d/G1M3/inference_results/yolov5s-300w-newdata/evalset_roi0"  # Root directory containing case folders with txt_results
+#   gt_path: "/data1/xdzhu/Testdata_0129"        # Root directory containing case folders with labels
+#   path_depth: 1  # Directory depth: 1 = det_path/case/txt_results, 2 = det_path/level1/case/txt_results
+
+# CNCAP gt_path: "/mnt/mono3d/xdzhu_data/Mono3d/Mono3d_4face_2m_g1m3/driving_png/G1M3_AFS1616"
+
+dataset:
+  det_path: "/data1/dongying/Mono3d/G1M3/inference_results/dl/evalset_roi1"  # Root directory containing case folders with txt_results
+  gt_path: "/data1/shaozk/mining/G1M3_FDL2232_20251201_20251212/origins"        # Root directory containing case folders with labels
+  path_depth: 1  # Directory depth: 1 = det_path/case/txt_results, 2 = det_path/level1/case/txt_results
+  det_format: "json"   # Detection file format: "auto" (probe json_results/ then txt_results/), "json", or "txt"
+  gt_format: "json"    # Ground truth file format: "auto" (probe labels_json/ then labels/), "json", or "txt"
+
+# Image properties (--img-width, --img-height)
+image:
+  width: 1920   # Default: 1920
+  height: 1080  # Default: 1080
+
+# Model configuration for GT filtering
+model:
+  input_size: 704  # Model input width (used for GT min box size calculation)
+  min_box_size_at_input_scale: 8  # Minimum box size at model input scale
+  # 
+  # GT filtering threshold is automatically calculated as:
+  # min_box_size = min_box_size_at_input_scale * roi_width / model_input_size
+  # 
+  # Examples:
+  # - ROI0 (1920->704): 8 * 1920 / 704 ≈ 21.8 pixels at original scale
+  # - ROI1 (704->704):  8 * 704 / 704 = 8.0 pixels at original scale
+
+# Performance settings
+performance:
+  num_workers: 32  # Number of parallel workers (null = auto-detect, 1 = single process)
+                     # Command-line: --num-workers
+                     # Recommended: 4-8 for typical workloads
+
+# ROI Ground Truth Processing (NEW!)
+roi_gt:
+  enabled: true                              # Enable ROI processing for ground truth
+  calib_root: "/data1/shaozk/mining/G1M3_FDL2232_20251201_20251212/origins"  # Root path to calibration files (camera4.json)
+  roi_config: [704, 352]                    # ROI configuration (matches training config)
+  roi_bottom_offset: 0                       # Pixels to trim from ROI bottom edge (matches training roi_bottom_offset)
+  # roi_config can be:
+  #   - [width, height]: ROI size mode (center crop based on vanishing point)
+  #   - [x1, y1, x2, y2]: ROI bounds mode (fixed bounds)
+  #   - dict: {'mode': 'size', 'width': 1920, 'height': 960} or 
+  #           {'mode': 'bounds', 'x1': 0, 'y1': 120, 'x2': 1920, 'y2': 1080}
+  # 
+  # IMPORTANT: This should match the ROI configuration used during training!
+  # For ROI0: [1920, 960] with roi_bottom_offset matching training config
+  # For ROI1: [704, 352] with roi_bottom_offset: 0
+
+# Class definitions
+classes:
+  3d_classes: [0, 1, 2, 3, 4, 5, 6, 7, 8]  # vehicle, bus, truck, tanker, unknown, pedestrian, bicycle, motorcyclist, tricycle
+  2d_classes: [9, 10, 11, 12]               # traffic_sign, wheel, plate, face
+  class_names:
+    0: "vehicle"
+    1: "bus"
+    2: "truck"
+    3: "tanker"
+    4: "unknown"
+    5: "pedestrian"
+    6: "bicycle"
+    7: "motorcyclist"
+    8: "tricycle"
+    9: "traffic_sign"
+    10: "wheel"
+    11: "plate"
+    12: "face"
+# Matching parameters (--iou-threshold)
+matching:
+  iou_threshold: 0.5  # Default: 0.5, IoU threshold for 2D matching
+  
+# 2D metrics configuration (--eval-2d-only, --conf-threshold, --ap-method)
+metrics_2d:
+  enabled: true                # Set to false with --eval-3d-only
+  conf_threshold: 0.4          # Default: 0.5, confidence threshold for precision/recall
+  ap_method: "voc2010"         # Default: "voc2010", choices: ["voc2010", "coco"]
+                               # voc2010: 11-point interpolation
+                               # coco: all-point interpolation
+  distance_ranges:             # Optional: distance-wise 2D evaluation (in metres, z3d).
+                               # Only applies to 3D-capable classes (vehicle/pedestrian/bicycle/rider)
+                               # that carry a ground-truth z3d depth value.
+                               # Independent from metrics_3d.distance_ranges — use coarser bins here
+                               # so the per-range GT counts are large enough to be statistically meaningful.
+    - [0, 30]                  # Near range
+    - [30, 60]                 # Mid range
+    - [60, 100]                # Far range
+    - [100, 999]               # Extreme range
+  lateral_roi: [-15, 15]     # Optional: lateral ROI pre-filter (metres, x3d).
+                               # When set, produces a second evaluation view that restricts all
+                               # counts (TP/FP/FN/GT) to objects within this lateral range.
+                               # e.g. [-15, 15] keeps only objects within 15 m of the centre line.
+  
+# 3D metrics configuration (--eval-3d-only)
+metrics_3d:
+  enabled: true                # Set to false with --eval-2d-only
+  heading_tolerance: "both"  # Heading error calculation mode (--heading-tolerance)
+                               # "strict": Standard calculation (default)
+                               # "relaxed": Consider 180° symmetry for symmetric objects
+                               # "both": Calculate and report both strict and relaxed metrics
+                               # Relaxed mode: for errors close to 180°, use min(error, π - error)
+  distance_ranges:             # Optional: distance-wise evaluation (in meters)
+    - [0, 10]                  # Near range
+    - [10, 20]                 
+    - [20, 30]
+    - [30, 40]                 # Medium range
+    - [40, 50]                 # Far range
+    - [50, 60]
+    - [60, 70]                 # Far range
+    - [70, 80]
+    - [80, 90]
+    - [90, 100]
+    - [100, 999]               # Very far range
+  lateral_distance_ranges:     # Optional: lateral distance-wise evaluation (in meters, x-axis)
+    - [-50, -40]              # Far left
+    - [-40, -30]                # Medium left
+    - [-30, -20]                # Near left far
+    - [-20, -10]                # Near left medium
+    - [-10, 0]                  # Near left
+    - [0, 10]                   # Near right
+    - [10, 20]                  # Medium right
+    - [20, 30]                # Near right far
+    - [30, 40]                # Medium right far
+    - [40, 50]                # Far right
+
+# Output configuration (--output-dir)
+output:
+  save_path: "eval_results_multiprocess/yolov5s/{timestamp}"  # {timestamp} will be auto-replaced
+  formats: ["json", "txt"]            # Available: json, txt, csv
+  print_details: true                 # Print detailed per-class metrics
+  per_case_reports: true              # Generate per-case detailed reports
+
+# Command-line override examples:
+# 1. Override paths:
+#    python eval_tools/eval.py --config eval_config.yaml --det-path /new/path
+# 
+# 2. Override workers:
+#    python eval_tools/eval.py --config eval_config.yaml --num-workers 8
+#
+# 3. 2D only evaluation:
+#    python eval_tools/eval.py --config eval_config.yaml --eval-2d-only
+#
+# 4. Custom thresholds:
+#    python eval_tools/eval.py --config eval_config.yaml --iou-threshold 0.7 --conf-threshold 0.3

eval_tools/core/eval.py

@@ -0,0 +1,301 @@
+#!/usr/bin/env python3
+"""
+YOLOv5-3D Model Evaluation Script
+
+This script evaluates 2D and 3D detection performance of YOLOv5-3D model.
+It computes metrics including Precision, Recall, AP, mAP for 2D detection,
+and lateral/longitudinal errors and heading errors for 3D detection.
+
+Usage:
+    # Basic usage with paths
+    python eval_tools/eval.py --det-path /path/to/detections --gt-path /path/to/labels --output-dir results
+    
+    # With configuration file
+    python eval_tools/eval.py --config eval_tools/configs/eval_config.yaml
+    
+    # Evaluate 2D only
+    python eval_tools/eval.py --det-path /path/to/detections --gt-path /path/to/labels --eval-2d-only
+    
+    # Evaluate 3D only
+    python eval_tools/eval.py --det-path /path/to/detections --gt-path /path/to/labels --eval-3d-only
+"""
+
+import argparse
+import sys
+import yaml
+from pathlib import Path
+from datetime import datetime
+
+# Add parent directory to path
+sys.path.insert(0, str(Path(__file__).parent.parent))
+
+from eval_tools.evaluator import Evaluator
+
+
+def load_config(config_path):
+    """Load configuration from YAML file."""
+    with open(config_path, 'r') as f:
+        config = yaml.safe_load(f)
+    
+    # Replace timestamp placeholders in paths
+    timestamp = datetime.now().strftime('%Y%m%d_%H%M%S')
+    if 'output' in config and 'save_path' in config['output']:
+        config['output']['save_path'] = config['output']['save_path'].replace('{timestamp}', timestamp)
+    if 'dataset' in config:
+        if 'det_path' in config['dataset']:
+            config['dataset']['det_path'] = config['dataset']['det_path'].replace('{timestamp}', timestamp)
+        if 'gt_path' in config['dataset']:
+            config['dataset']['gt_path'] = config['dataset']['gt_path'].replace('{timestamp}', timestamp)
+    
+    return config
+
+
+def parse_arguments():
+    """Parse command line arguments."""
+    parser = argparse.ArgumentParser(
+        description='Evaluate YOLOv5-3D model detection results',
+        formatter_class=argparse.RawDescriptionHelpFormatter,
+        epilog="""
+Examples:
+  # Basic evaluation
+  python eval_tools/eval.py --det-path /data/detections --gt-path /data/labels --output-dir results
+  
+  # With custom image size
+  python eval_tools/eval.py --det-path /data/detections --gt-path /data/labels --img-width 3840 --img-height 2160
+  
+  # Using config file
+  python eval_tools/eval.py --config eval_tools/configs/eval_config.yaml
+  
+  # 2D evaluation only
+  python eval_tools/eval.py --det-path /data/detections --gt-path /data/labels --eval-2d-only
+        """
+    )
+    
+    # Path arguments
+    parser.add_argument('--det-path', type=str,
+                        help='Path to detection results root directory (contains case folders)')
+    parser.add_argument('--gt-path', type=str,
+                        help='Path to ground truth labels root directory (contains case folders)')
+    parser.add_argument('--path-depth', type=int, default=1, choices=[1, 2],
+                        help='Directory depth: 1 = det_path/case/txt_results, 2 = det_path/level1/case/txt_results (default: 1)')
+    parser.add_argument('--det-format', type=str, default='auto', choices=['auto', 'json', 'txt'],
+                        help='Detection file format: auto (default, probe json_results/ then txt_results/), json, or txt')
+    parser.add_argument('--gt-format', type=str, default='auto', choices=['auto', 'json', 'txt'],
+                        help='Ground truth file format: auto (default, probe labels_json/ then labels/), json, or txt')
+    parser.add_argument('--output-dir', type=str, default='eval_results',
+                        help='Output directory for evaluation results (default: eval_results)')
+    
+    # Config file
+    parser.add_argument('--config', type=str,
+                        help='Path to YAML configuration file')
+    
+    # Image parameters
+    parser.add_argument('--img-width', type=int, default=1920,
+                        help='Image width in pixels (default: 1920)')
+    parser.add_argument('--img-height', type=int, default=1080,
+                        help='Image height in pixels (default: 1080)')
+    
+    # Evaluation options
+    parser.add_argument('--eval-2d-only', action='store_true',
+                        help='Evaluate 2D detection only')
+    parser.add_argument('--eval-3d-only', action='store_true',
+                        help='Evaluate 3D detection only')
+    parser.add_argument('--iou-threshold', type=float, default=0.5,
+                        help='IoU threshold for 2D matching (default: 0.5)')
+    parser.add_argument('--conf-threshold', type=float, default=0.5,
+                        help='Confidence threshold for precision/recall (default: 0.5)')
+    parser.add_argument('--ap-method', type=str, default='voc2010',
+                        choices=['voc2010', 'coco'],
+                        help='AP calculation method (default: voc2010)')
+    parser.add_argument('--heading-tolerance', type=str, default='strict',
+                        choices=['strict', 'relaxed', 'both'],
+                        help='Heading error calculation mode: strict (default), relaxed (180° symmetry), or both')
+    parser.add_argument('--coord-system', type=str, default='camera',
+                        choices=['camera', 'ego'],
+                        help='3D evaluation coordinate system: camera (default) or ego')
+    parser.add_argument('--num-workers', type=int, default=None,
+                        help='Number of parallel workers for evaluation (default: auto-detect)')
+    parser.add_argument('--roi-bottom-offset', type=int, default=None,
+                        help='Pixels to trim from bottom edge of ROI (shifts y2 upward, default: from config or 0)')
+    parser.add_argument('--save-detailed-matches', action='store_true',
+                        help='Save detailed 3D match information for common-match comparison')
+    
+    return parser.parse_args()
+
+
+def main():
+    """Main evaluation function."""
+    args = parse_arguments()
+    
+    # Load configuration
+    config = {}
+    if args.config:
+        print(f"Loading configuration from: {args.config}")
+        config = load_config(args.config)
+        
+        # Override with command line arguments if provided
+        if args.det_path:
+            config.setdefault('dataset', {})['det_path'] = args.det_path
+        if args.gt_path:
+            config.setdefault('dataset', {})['gt_path'] = args.gt_path
+        if args.path_depth != 1:
+            config.setdefault('dataset', {})['path_depth'] = args.path_depth
+        if args.det_format != 'auto':
+            config.setdefault('dataset', {})['det_format'] = args.det_format
+        if args.gt_format != 'auto':
+            config.setdefault('dataset', {})['gt_format'] = args.gt_format
+        # Only override output_dir if explicitly provided (not default value)
+        if args.output_dir != 'eval_results':
+            config.setdefault('output', {})['save_path'] = args.output_dir
+        
+        # Override image size if explicitly provided
+        if args.img_width != 1920:
+            config.setdefault('image', {})['width'] = args.img_width
+        if args.img_height != 1080:
+            config.setdefault('image', {})['height'] = args.img_height
+        
+        # Override matching threshold if explicitly provided
+        if args.iou_threshold != 0.5:
+            config.setdefault('matching', {})['iou_threshold'] = args.iou_threshold
+        
+        # Override 2D metrics parameters if explicitly provided
+        if args.conf_threshold != 0.5:
+            config.setdefault('metrics_2d', {})['conf_threshold'] = args.conf_threshold
+        if args.ap_method != 'voc2010':
+            config.setdefault('metrics_2d', {})['ap_method'] = args.ap_method
+        
+        # Override 3D metrics parameters if explicitly provided
+        if args.heading_tolerance != 'strict':
+            config.setdefault('metrics_3d', {})['heading_tolerance'] = args.heading_tolerance
+        if args.coord_system != 'camera':
+            config.setdefault('metrics_3d', {})['coordinate_system'] = args.coord_system
+        
+        # Override roi_bottom_offset if explicitly provided
+        if args.roi_bottom_offset is not None:
+            config.setdefault('roi_gt', {})['roi_bottom_offset'] = args.roi_bottom_offset
+    else:
+        # Build config from command line arguments
+        if not args.det_path or not args.gt_path:
+            print("Error: --det-path and --gt-path are required when not using --config")
+            sys.exit(1)
+        
+        config = {
+            'dataset': {
+                'det_path': args.det_path,
+                'gt_path': args.gt_path,
+                'path_depth': args.path_depth,
+                'det_format': args.det_format,
+                'gt_format': args.gt_format,
+            },
+            'image': {
+                'width': args.img_width,
+                'height': args.img_height
+            },
+            'matching': {
+                'iou_threshold': args.iou_threshold
+            },
+            'metrics_2d': {
+                'enabled': not args.eval_3d_only,
+                'conf_threshold': args.conf_threshold,
+                'ap_method': args.ap_method
+            },
+            'metrics_3d': {
+                'enabled': not args.eval_2d_only,
+                'heading_tolerance': args.heading_tolerance,
+                'coordinate_system': args.coord_system,
+            },
+            'output': {
+                'save_path': args.output_dir
+            }
+        }
+        
+        if args.roi_bottom_offset is not None:
+            config.setdefault('roi_gt', {})['roi_bottom_offset'] = args.roi_bottom_offset
+    
+    # Set evaluation flags
+    config['eval_2d'] = config.get('metrics_2d', {}).get('enabled', True)
+    config['eval_3d'] = config.get('metrics_3d', {}).get('enabled', True)
+    
+    if args.eval_2d_only:
+        config['eval_2d'] = True
+        config['eval_3d'] = False
+    elif args.eval_3d_only:
+        config['eval_2d'] = False
+        config['eval_3d'] = True
+    
+    # Extract paths
+    det_path = config['dataset']['det_path']
+    gt_path = config['dataset']['gt_path']
+    path_depth = config['dataset'].get('path_depth', 1)  # Default to 1-level structure
+    det_format = config['dataset'].get('det_format', 'auto')
+    gt_format  = config['dataset'].get('gt_format',  'auto')
+    det_subdir = config['dataset'].get('det_subdir')
+    output_dir = config['output']['save_path']
+    
+    img_width = config.get('image', {}).get('width', 1920)
+    img_height = config.get('image', {}).get('height', 1080)
+    iou_threshold = config.get('matching', {}).get('iou_threshold', 0.5)
+    
+    # Get num_workers from config if not specified in command line
+    num_workers = args.num_workers
+    if num_workers is None and 'performance' in config:
+        num_workers = config['performance'].get('num_workers', None)
+    
+    print("="*80)
+    print("YOLOv5-3D Model Evaluation")
+    print("="*80)
+    print(f"Detection path: {det_path}")
+    print(f"Ground truth path: {gt_path}")
+    print(f"Path depth: {path_depth}")
+    print(f"Detection format: {det_format}")
+    print(f"Ground truth format: {gt_format}")
+    if det_subdir:
+        print(f"Detection subdirectory: {det_subdir}")
+    print(f"Output directory: {output_dir}")
+    print(f"Image size: {img_width}x{img_height}")
+    print(f"IoU threshold: {iou_threshold}")
+    print(f"Confidence threshold: {config.get('metrics_2d', {}).get('conf_threshold', 0.5)}")
+    print(f"AP method: {config.get('metrics_2d', {}).get('ap_method', 'voc2010')}")
+    heading_tolerance = config.get('metrics_3d', {}).get('heading_tolerance', 'strict')
+    coord_system = config.get('metrics_3d', {}).get('coordinate_system', 'camera')
+    print(f"Heading tolerance: {heading_tolerance}")
+    print(f"3D coordinate system: {coord_system}")
+    roi_bottom_offset_val = config.get('roi_gt', {}).get('roi_bottom_offset', 0)
+    if config.get('roi_gt', {}).get('enabled', False):
+        print(f"ROI bottom offset: {roi_bottom_offset_val}")
+    if num_workers:
+        print(f"Number of workers: {num_workers}")
+    print(f"Evaluate 2D: {config['eval_2d']}")
+    print(f"Evaluate 3D: {config['eval_3d']}")
+    if args.save_detailed_matches:
+        print(f"Save detailed matches: Yes")
+    print("="*80)
+    
+    # Initialize evaluator
+    evaluator = Evaluator(
+        config=config,
+        iou_threshold=iou_threshold,
+        num_workers=num_workers,
+        save_detailed_matches=args.save_detailed_matches
+    )
+    
+    # Load data
+    print("\nLoading data...")
+    evaluator.load_data_from_paths(det_path, gt_path, img_width, img_height, path_depth,
+                                    det_format=det_format, gt_format=gt_format)
+    
+    if len(evaluator.image_pairs) == 0:
+        print("Error: No image pairs found for evaluation!")
+        sys.exit(1)
+    
+    # Run evaluation
+    results = evaluator.evaluate()
+    
+    # Generate and save report
+    evaluator.generate_report(results, output_dir)
+    
+    print("\n✓ Evaluation completed successfully!")
+
+
+if __name__ == '__main__':
+    main()

eval_tools/core/rename_detection_frames.py

@@ -0,0 +1,584 @@
+"""Rename detection result files so frame stems match ground-truth stems.
+
+This script is intended as a preprocessing step before running the evaluation
+pipeline. The evaluator pairs detection and ground-truth files strictly by
+their filename stem, so detections that omit a timestamp suffix will be skipped
+unless they are renamed to the GT stem.
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+import os
+import re
+import shutil
+from collections import defaultdict
+from concurrent.futures import ProcessPoolExecutor
+from dataclasses import dataclass
+from pathlib import Path
+from typing import Callable
+
+import yaml
+
+
+DEFAULT_TIMESTAMP_PATTERNS = (
+    re.compile(r"(?:[_-]?ts)?\d{10,20}$"),
+    re.compile(r"[_-]?\d{8}[_-]\d{6}(?:\d+)?$"),
+    re.compile(r"[_-]?\d{8}T\d{6}(?:\d+)?$"),
+)
+DEFAULT_TIMESTAMP_TOKEN_PATTERN = re.compile(r"^(?:ts)?\d{8,20}$")
+
+
+@dataclass(frozen=True)
+class CaseInfo:
+    """One evaluation case directory."""
+
+    det_case_dir: Path
+    case_name: str
+    level1_name: str | None = None
+
+
+@dataclass(frozen=True)
+class MatchPair:
+    """Mapping from one detection file to one GT file."""
+
+    det_file: Path
+    gt_file: Path
+    strategy: str
+
+
+@dataclass(frozen=True)
+class CaseProcessResult:
+    """Worker result for one case."""
+
+    summary: dict[str, object]
+    manifest_rows: list[dict[str, object]]
+
+
+def parse_args() -> argparse.Namespace:
+    """Parse command-line arguments."""
+    parser = argparse.ArgumentParser(
+        description=(
+            "Rename or copy detection result files so their stems match GT stems "
+            "before running eval_tools/core/eval.py."
+        )
+    )
+    parser.add_argument("--config", type=Path, help="Evaluation config YAML used to infer det/gt paths.")
+    parser.add_argument("--det-root", type=Path, help="Detection root directory. Overrides config dataset.det_path.")
+    parser.add_argument("--gt-root", type=Path, help="Ground-truth root directory. Overrides config dataset.gt_path.")
+    parser.add_argument(
+        "--path-depth",
+        type=int,
+        choices=(1, 2),
+        help="Directory depth: 1 = root/case, 2 = root/level1/case. Overrides config dataset.path_depth.",
+    )
+    parser.add_argument(
+        "--det-format",
+        choices=("auto", "json", "txt"),
+        help="Detection format. Overrides config dataset.det_format.",
+    )
+    parser.add_argument(
+        "--gt-format",
+        choices=("auto", "json", "txt"),
+        help="GT format. Overrides config dataset.gt_format.",
+    )
+    parser.add_argument(
+        "--det-subdir",
+        help="Relative detection subdirectory inside each case, e.g. predictions/roi0.",
+    )
+    parser.add_argument(
+        "--output-root",
+        type=Path,
+        help=(
+            "Write renamed files to a mirrored directory tree instead of changing the source det root in place. "
+            "Recommended for the first run."
+        ),
+    )
+    parser.add_argument(
+        "--in-place",
+        action="store_true",
+        help="Rename files directly inside the source detection directory.",
+    )
+    parser.add_argument(
+        "--enable-index-fallback",
+        action="store_true",
+        help=(
+            "If normalized-name matching still leaves unmatched files, allow one-to-one pairing by sorted index. "
+            "Use only when detection and GT frame order is guaranteed to be aligned."
+        ),
+    )
+    parser.add_argument(
+        "--strip-pattern",
+        action="append",
+        default=[],
+        help=(
+            "Custom regex used to strip a suffix from stems before matching. Can be specified multiple times. "
+            "If omitted, built-in timestamp suffix patterns are used."
+        ),
+    )
+    parser.add_argument(
+        "--case",
+        dest="cases",
+        action="append",
+        default=[],
+        help="Limit processing to specific case names. Can be specified multiple times.",
+    )
+    parser.add_argument(
+        "--manifest-path",
+        type=Path,
+        help="Where to save the rename manifest JSON. Defaults under output_root or det_root.",
+    )
+    parser.add_argument(
+        "--strict",
+        action="store_true",
+        help="Return a non-zero exit code if any case has unmatched or ambiguous files.",
+    )
+    parser.add_argument(
+        "--dry-run",
+        action="store_true",
+        help="Preview the mapping without copying or renaming files.",
+    )
+    parser.add_argument(
+        "--num-workers",
+        type=int,
+        default=1,
+        help="Number of case-level worker processes. Use 0 to auto-select based on CPU count.",
+    )
+    return parser.parse_args()
+
+
+def load_config(config_path: Path | None) -> dict:
+    """Load evaluation YAML config if provided."""
+    if config_path is None:
+        return {}
+    with config_path.open("r", encoding="utf-8") as handle:
+        return yaml.safe_load(handle) or {}
+
+
+def resolve_runtime_options(args: argparse.Namespace, config: dict) -> dict:
+    """Resolve CLI options with config fallback."""
+    dataset_cfg = config.get("dataset", {}) if isinstance(config, dict) else {}
+
+    det_root = Path(args.det_root or dataset_cfg.get("det_path", "")).expanduser() if (args.det_root or dataset_cfg.get("det_path")) else None
+    gt_root = Path(args.gt_root or dataset_cfg.get("gt_path", "")).expanduser() if (args.gt_root or dataset_cfg.get("gt_path")) else None
+    path_depth = args.path_depth or dataset_cfg.get("path_depth", 1)
+    det_format = args.det_format or dataset_cfg.get("det_format", "auto")
+    gt_format = args.gt_format or dataset_cfg.get("gt_format", "auto")
+    det_subdir = args.det_subdir if args.det_subdir is not None else dataset_cfg.get("det_subdir")
+
+    if det_root is None or gt_root is None:
+        raise ValueError("Both detection root and GT root must be provided by --config or explicit arguments.")
+    if args.in_place and args.output_root:
+        raise ValueError("Use either --in-place or --output-root, not both.")
+    if not args.in_place and args.output_root is None and not args.dry_run:
+        raise ValueError("Choose --output-root for a copied result tree or --in-place for source renaming.")
+
+    return {
+        "det_root": det_root.resolve(),
+        "gt_root": gt_root.resolve(),
+        "path_depth": int(path_depth),
+        "det_format": det_format,
+        "gt_format": gt_format,
+        "det_subdir": det_subdir,
+    }
+
+
+def iter_case_dirs(det_root: Path, path_depth: int, selected_cases: set[str]) -> list[CaseInfo]:
+    """Enumerate detection case directories according to evaluator path depth."""
+    def is_case_dir(path: Path) -> bool:
+        return path.is_dir() and not path.name.startswith((".", "_"))
+
+    cases: list[CaseInfo] = []
+    if path_depth == 1:
+        for case_dir in sorted(p for p in det_root.iterdir() if is_case_dir(p)):
+            if selected_cases and case_dir.name not in selected_cases:
+                continue
+            cases.append(CaseInfo(det_case_dir=case_dir, case_name=case_dir.name))
+        return cases
+
+    for level1_dir in sorted(p for p in det_root.iterdir() if is_case_dir(p)):
+        for case_dir in sorted(p for p in level1_dir.iterdir() if is_case_dir(p)):
+            if selected_cases and case_dir.name not in selected_cases:
+                continue
+            cases.append(CaseInfo(det_case_dir=case_dir, case_name=case_dir.name, level1_name=level1_dir.name))
+    return cases
+
+
+def resolve_gt_case_dir(gt_root: Path, case_info: CaseInfo, path_depth: int) -> Path:
+    """Resolve ground-truth case directory from case info."""
+    if path_depth == 1:
+        return gt_root / case_info.case_name
+    return gt_root / str(case_info.level1_name) / case_info.case_name
+
+
+def resolve_detection_dir(case_dir: Path, det_format: str, det_subdir: str | None) -> tuple[Path | None, str | None]:
+    """Resolve the directory and suffix for detection result files."""
+    json_candidates: list[Path] = []
+    if det_subdir:
+        subdir_path = Path(det_subdir)
+        json_candidates.append(subdir_path if subdir_path.is_absolute() else case_dir / subdir_path)
+    json_candidates.extend([case_dir / "json_results", case_dir / "predictions"])
+    txt_candidate = case_dir / "txt_results"
+
+    def pick_json_dir() -> Path | None:
+        for candidate in json_candidates:
+            if candidate.exists() and any(candidate.glob("*.json")):
+                return candidate
+        return None
+
+    if det_format == "json":
+        result_dir = pick_json_dir()
+        return result_dir, ".json"
+    if det_format == "txt":
+        return (txt_candidate, ".txt") if txt_candidate.exists() else (None, None)
+
+    result_dir = pick_json_dir()
+    if result_dir is not None:
+        return result_dir, ".json"
+    if txt_candidate.exists() and any(txt_candidate.glob("*.txt")):
+        return txt_candidate, ".txt"
+    return None, None
+
+
+def resolve_gt_dir(case_dir: Path, gt_format: str) -> tuple[Path | None, str | None]:
+    """Resolve the directory and suffix for GT files."""
+    json_candidate = case_dir / "labels_json"
+    txt_candidate = case_dir / "labels"
+
+    if gt_format == "json":
+        return (json_candidate, ".json") if json_candidate.exists() else (None, None)
+    if gt_format == "txt":
+        return (txt_candidate, ".txt") if txt_candidate.exists() else (None, None)
+    if json_candidate.exists():
+        return json_candidate, ".json"
+    if txt_candidate.exists():
+        return txt_candidate, ".txt"
+    return None, None
+
+
+def build_normalizer(custom_patterns: list[str]) -> Callable[[str], str]:
+    """Build a stem normalizer that removes timestamp-like suffixes."""
+    patterns = [re.compile(pattern) for pattern in custom_patterns] if custom_patterns else list(DEFAULT_TIMESTAMP_PATTERNS)
+
+    def normalize(stem: str) -> str:
+        normalized = stem
+        for pattern in patterns:
+            normalized = pattern.sub("", normalized)
+        tokens = [token for token in normalized.rstrip("_-").split("_") if token]
+        filtered_tokens = [token for token in tokens if not DEFAULT_TIMESTAMP_TOKEN_PATTERN.fullmatch(token)]
+        return "_".join(filtered_tokens)
+
+    return normalize
+
+
+def build_pairs(
+    det_files: list[Path],
+    gt_files: list[Path],
+    normalize_stem: Callable[[str], str],
+    enable_index_fallback: bool,
+) -> tuple[list[MatchPair], list[str], list[dict[str, object]]]:
+    """Match detection files to GT files using exact, normalized, then optional index fallback."""
+    gt_by_stem = {path.stem: path for path in gt_files}
+    used_gt_stems: set[str] = set()
+    pairs: list[MatchPair] = []
+    issues: list[str] = []
+    ambiguous_groups: list[dict[str, object]] = []
+
+    unmatched_det: list[Path] = []
+    for det_file in det_files:
+        gt_file = gt_by_stem.get(det_file.stem)
+        if gt_file is None:
+            unmatched_det.append(det_file)
+            continue
+        used_gt_stems.add(gt_file.stem)
+        pairs.append(MatchPair(det_file=det_file, gt_file=gt_file, strategy="exact"))
+
+    unmatched_gt = [gt_file for gt_file in gt_files if gt_file.stem not in used_gt_stems]
+    if not unmatched_det or not unmatched_gt:
+        return pairs, issues, ambiguous_groups
+
+    det_groups: dict[str, list[Path]] = defaultdict(list)
+    gt_groups: dict[str, list[Path]] = defaultdict(list)
+    for det_file in unmatched_det:
+        det_groups[normalize_stem(det_file.stem)].append(det_file)
+    for gt_file in unmatched_gt:
+        gt_groups[normalize_stem(gt_file.stem)].append(gt_file)
+
+    remaining_det: list[Path] = []
+    remaining_gt: list[Path] = []
+    group_keys = sorted(set(det_groups) | set(gt_groups))
+    for key in group_keys:
+        det_group = sorted(det_groups.get(key, []))
+        gt_group = sorted(gt_groups.get(key, []))
+        if not det_group:
+            remaining_gt.extend(gt_group)
+            continue
+        if not gt_group:
+            remaining_det.extend(det_group)
+            continue
+        if len(det_group) == 1 and len(gt_group) == 1:
+            pairs.append(MatchPair(det_file=det_group[0], gt_file=gt_group[0], strategy="normalized"))
+            continue
+
+        ambiguous_groups.append(
+            {
+                "normalized_key": key,
+                "det_files": [path.name for path in det_group],
+                "gt_files": [path.name for path in gt_group],
+            }
+        )
+        remaining_det.extend(det_group)
+        remaining_gt.extend(gt_group)
+
+    if remaining_det or remaining_gt:
+        if enable_index_fallback and len(remaining_det) == len(remaining_gt):
+            for det_file, gt_file in zip(sorted(remaining_det), sorted(remaining_gt)):
+                pairs.append(MatchPair(det_file=det_file, gt_file=gt_file, strategy="index"))
+        else:
+            if remaining_det:
+                issues.append(
+                    "Unmatched detection files: " + ", ".join(path.name for path in sorted(remaining_det))
+                )
+            if remaining_gt:
+                issues.append(
+                    "Unused GT files: " + ", ".join(path.name for path in sorted(remaining_gt))
+                )
+
+    return pairs, issues, ambiguous_groups
+
+
+def ensure_parent(path: Path) -> None:
+    """Create parent directory if missing."""
+    path.parent.mkdir(parents=True, exist_ok=True)
+
+
+def apply_mapping(
+    pairs: list[MatchPair],
+    det_root: Path,
+    output_root: Path | None,
+    in_place: bool,
+    dry_run: bool,
+) -> list[dict[str, object]]:
+    """Apply renaming or copying for matched pairs."""
+    manifest_rows: list[dict[str, object]] = []
+
+    for pair in pairs:
+        relative_det_path = pair.det_file.relative_to(det_root)
+        target_name = f"{pair.gt_file.stem}{pair.det_file.suffix}"
+        if output_root is not None:
+            target_dir = output_root / relative_det_path.parent
+            target_path = target_dir / target_name
+            action = "copy"
+        else:
+            target_path = pair.det_file.with_name(target_name)
+            action = "rename"
+
+        row = {
+            "strategy": pair.strategy,
+            "action": action,
+            "source": str(pair.det_file),
+            "target": str(target_path),
+            "gt_file": str(pair.gt_file),
+            "changed": pair.det_file.name != target_path.name,
+        }
+        manifest_rows.append(row)
+
+        if dry_run:
+            continue
+
+        ensure_parent(target_path)
+        if output_root is not None:
+            if pair.det_file.resolve() == target_path.resolve():
+                continue
+            shutil.copy2(pair.det_file, target_path)
+        elif in_place and pair.det_file != target_path:
+            if target_path.exists():
+                raise FileExistsError(f"Refusing to overwrite existing file: {target_path}")
+            pair.det_file.rename(target_path)
+
+    return manifest_rows
+
+
+def default_manifest_path(args: argparse.Namespace, runtime: dict) -> Path:
+    """Choose a default manifest path."""
+    if args.manifest_path is not None:
+        return args.manifest_path
+    root = args.output_root.resolve() if args.output_root else runtime["det_root"]
+    return root / "rename_detection_frames_manifest.json"
+
+
+def resolve_num_workers(requested_workers: int, case_count: int) -> int:
+    """Resolve worker count from CLI input and case count."""
+    if case_count <= 1:
+        return 1
+    if requested_workers <= 0:
+        requested_workers = os.cpu_count() or 1
+    return max(1, min(requested_workers, case_count))
+
+
+def process_case(
+    case_info: CaseInfo,
+    runtime: dict[str, object],
+    strip_patterns: list[str],
+    enable_index_fallback: bool,
+    output_root: str | None,
+    in_place: bool,
+    dry_run: bool,
+) -> CaseProcessResult:
+    """Process one case, including match generation and optional file operations."""
+    normalize_stem = build_normalizer(strip_patterns)
+    det_root = Path(str(runtime["det_root"]))
+    gt_root = Path(str(runtime["gt_root"]))
+    path_depth = int(runtime["path_depth"])
+    det_format = str(runtime["det_format"])
+    gt_format = str(runtime["gt_format"])
+    det_subdir = runtime.get("det_subdir")
+    det_subdir_str = None if det_subdir is None else str(det_subdir)
+
+    gt_case_dir = resolve_gt_case_dir(gt_root, case_info, path_depth)
+    det_dir, det_suffix = resolve_detection_dir(case_info.det_case_dir, det_format, det_subdir_str)
+    gt_dir, gt_suffix = resolve_gt_dir(gt_case_dir, gt_format)
+
+    summary = {
+        "case": case_info.case_name,
+        "level1_name": case_info.level1_name,
+        "det_case_dir": str(case_info.det_case_dir),
+        "gt_case_dir": str(gt_case_dir),
+        "status": "ok",
+        "issues": [],
+        "ambiguous_groups": [],
+        "matched": 0,
+        "changed": 0,
+    }
+
+    if det_dir is None or det_suffix is None:
+        summary["status"] = "missing_detection_dir"
+        summary["issues"].append("Detection results directory not found or empty.")
+        return CaseProcessResult(summary=summary, manifest_rows=[])
+    if gt_dir is None or gt_suffix is None:
+        summary["status"] = "missing_gt_dir"
+        summary["issues"].append("GT labels directory not found.")
+        return CaseProcessResult(summary=summary, manifest_rows=[])
+
+    det_files = sorted(path for path in det_dir.iterdir() if path.suffix == det_suffix)
+    gt_files = sorted(path for path in gt_dir.iterdir() if path.suffix == gt_suffix)
+    if not det_files:
+        summary["status"] = "empty_detection_dir"
+        summary["issues"].append("No detection files found in resolved directory.")
+        return CaseProcessResult(summary=summary, manifest_rows=[])
+    if not gt_files:
+        summary["status"] = "empty_gt_dir"
+        summary["issues"].append("No GT files found in resolved directory.")
+        return CaseProcessResult(summary=summary, manifest_rows=[])
+
+    pairs, issues, ambiguous_groups = build_pairs(
+        det_files=det_files,
+        gt_files=gt_files,
+        normalize_stem=normalize_stem,
+        enable_index_fallback=enable_index_fallback,
+    )
+    manifest_rows = apply_mapping(
+        pairs=pairs,
+        det_root=det_root,
+        output_root=Path(output_root) if output_root else None,
+        in_place=in_place,
+        dry_run=dry_run,
+    )
+
+    summary["issues"] = issues
+    summary["ambiguous_groups"] = ambiguous_groups
+    summary["matched"] = len(pairs)
+    summary["changed"] = sum(1 for row in manifest_rows if row["changed"])
+    if issues or ambiguous_groups:
+        summary["status"] = "partial"
+    return CaseProcessResult(summary=summary, manifest_rows=manifest_rows)
+
+
+def main() -> int:
+    """Run the rename workflow."""
+    args = parse_args()
+    config = load_config(args.config)
+    runtime = resolve_runtime_options(args, config)
+    selected_cases = set(args.cases)
+
+    case_infos = iter_case_dirs(runtime["det_root"], runtime["path_depth"], selected_cases)
+    if not case_infos:
+        raise FileNotFoundError(f"No detection cases found under {runtime['det_root']}")
+    num_workers = resolve_num_workers(args.num_workers, len(case_infos))
+
+    all_manifest_rows: list[dict[str, object]] = []
+    case_summaries: list[dict[str, object]] = []
+    failed_cases = 0
+
+    output_root = str(args.output_root.resolve()) if args.output_root else None
+    if num_workers == 1:
+        results = [
+            process_case(
+                case_info=case_info,
+                runtime=runtime,
+                strip_patterns=args.strip_pattern,
+                enable_index_fallback=args.enable_index_fallback,
+                output_root=output_root,
+                in_place=args.in_place,
+                dry_run=args.dry_run,
+            )
+            for case_info in case_infos
+        ]
+    else:
+        with ProcessPoolExecutor(max_workers=num_workers) as executor:
+            results = list(
+                executor.map(
+                    process_case,
+                    case_infos,
+                    [runtime] * len(case_infos),
+                    [args.strip_pattern] * len(case_infos),
+                    [args.enable_index_fallback] * len(case_infos),
+                    [output_root] * len(case_infos),
+                    [args.in_place] * len(case_infos),
+                    [args.dry_run] * len(case_infos),
+                )
+            )
+
+    for result in results:
+        case_summaries.append(result.summary)
+        all_manifest_rows.extend(result.manifest_rows)
+        if result.summary["status"] != "ok":
+            failed_cases += 1
+
+    manifest = {
+        "dry_run": args.dry_run,
+        "in_place": args.in_place,
+        "output_root": str(args.output_root.resolve()) if args.output_root else None,
+        "det_root": str(runtime["det_root"]),
+        "gt_root": str(runtime["gt_root"]),
+        "path_depth": runtime["path_depth"],
+        "det_format": runtime["det_format"],
+        "gt_format": runtime["gt_format"],
+        "det_subdir": runtime["det_subdir"],
+        "cases": case_summaries,
+        "mappings": all_manifest_rows,
+    }
+
+    manifest_path = default_manifest_path(args, runtime)
+    ensure_parent(manifest_path)
+    with manifest_path.open("w", encoding="utf-8") as handle:
+        json.dump(manifest, handle, ensure_ascii=False, indent=2)
+
+    total_cases = len(case_summaries)
+    total_matched = sum(item["matched"] for item in case_summaries)
+    total_changed = sum(item["changed"] for item in case_summaries)
+    print(f"Processed cases: {total_cases}")
+    print(f"Workers used: {num_workers}")
+    print(f"Matched files: {total_matched}")
+    print(f"Renamed/copied files with changed stems: {total_changed}")
+    print(f"Manifest: {manifest_path}")
+
+    if args.strict and failed_cases:
+        print(f"Strict mode: {failed_cases} case(s) still have unresolved issues.")
+        return 1
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

eval_tools/core/run_eval.sh

@@ -0,0 +1,79 @@
+#!/bin/bash
+
+# YOLOv5-3D 评测启动脚本
+# 使用方法: sh eval_tools/run_eval.sh
+
+# ==============================================
+# 配置参数
+# ==============================================
+
+# 数据路径
+
+# Set PYTHONPATH to project root for module imports
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+PROJECT_ROOT="$(cd "${SCRIPT_DIR}/../.." && pwd)"
+export PYTHONPATH="${PROJECT_ROOT}:${PYTHONPATH}"
+DET_PATH="/data1/dongying/Mono3d/G1M3/300w/txt_results"
+
+GT_PATH="/data1/dongying/Mono3d/G1M3/labels"
+OUTPUT_DIR="eval_results/$(date +%Y%m%d_%H%M%S)"
+
+# 图像参数
+IMG_WIDTH=1920
+IMG_HEIGHT=1080
+
+# 评测参数
+IOU_THRESHOLD=0.5
+CONF_THRESHOLD=0.5
+AP_METHOD="voc2010"  # voc2010 或 coco
+
+# 并行进程数 (默认自动检测)
+NUM_WORKERS=8  # 设置为空使用自动检测: NUM_WORKERS=""
+
+# ==============================================
+# 构建命令
+# ==============================================
+# 构建命令
+# ==============================================
+
+CMD="python eval_tools/core/eval.py \
+    --det-path $DET_PATH \
+    --gt-path $GT_PATH \
+    --output-dir $OUTPUT_DIR \
+    --img-width $IMG_WIDTH \
+    --img-height $IMG_HEIGHT \
+    --iou-threshold $IOU_THRESHOLD \
+    --conf-threshold $CONF_THRESHOLD \
+    --ap-method $AP_METHOD"
+
+# 添加并行进程数参数
+if [ -n "$NUM_WORKERS" ]; then
+    CMD="$CMD --num-workers $NUM_WORKERS"
+fi
+
+# ==============================================
+# 执行评测
+# ==============================================
+
+echo "========================================"
+echo "YOLOv5-3D 评测"
+echo "========================================"
+echo "检测结果: $DET_PATH"
+echo "真值标签: $GT_PATH"
+echo "输出目录: $OUTPUT_DIR"
+echo "图像尺寸: ${IMG_WIDTH}x${IMG_HEIGHT}"
+echo "IoU阈值: $IOU_THRESHOLD"
+echo "置信度阈值: $CONF_THRESHOLD"
+if [ -n "$NUM_WORKERS" ]; then
+    echo "并行进程: $NUM_WORKERS"
+else
+    echo "并行进程: 自动检测"
+fi
+echo "========================================"
+echo ""
+
+# 执行命令
+eval $CMD
+
+echo ""
+echo "评测完成! 结果已保存到: $OUTPUT_DIR"

eval_tools/core/run_eval_2d_only.sh

@@ -0,0 +1,25 @@
+#!/bin/bash
+
+# YOLOv5-3D 快速评测脚本 - 仅评测2D
+# 使用方法: sh eval_tools/run_eval_2d_only.sh
+
+
+# Set PYTHONPATH to project root for module imports
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+PROJECT_ROOT="$(cd "${SCRIPT_DIR}/../.." && pwd)"
+export PYTHONPATH="${PROJECT_ROOT}:${PYTHONPATH}"
+DET_PATH="/data1/dongying/Mono3d/G1M3/300w/txt_results"
+
+GT_PATH="/data1/dongying/Mono3d/G1M3/labels"
+OUTPUT_DIR="eval_results/2d_only_$(date +%Y%m%d_%H%M%S)"
+
+python eval_tools/core/eval.py \
+    --det-path $DET_PATH \
+    --gt-path $GT_PATH \
+    --output-dir $OUTPUT_DIR \
+    --eval-2d-only \
+    --num-workers 8 \
+    --img-width 1920 \
+    --img-height 1080
+
+echo "2D评测完成! 结果: $OUTPUT_DIR"

eval_tools/core/run_eval_3d_only.sh

@@ -0,0 +1,25 @@
+#!/bin/bash
+
+# YOLOv5-3D 快速评测脚本 - 仅评测3D
+# 使用方法: sh eval_tools/run_eval_3d_only.sh
+
+
+# Set PYTHONPATH to project root for module imports
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+PROJECT_ROOT="$(cd "${SCRIPT_DIR}/../.." && pwd)"
+export PYTHONPATH="${PROJECT_ROOT}:${PYTHONPATH}"
+DET_PATH="/data1/dongying/Mono3d/G1M3/300w/txt_results"
+
+GT_PATH="/data1/dongying/Mono3d/G1M3/labels"
+OUTPUT_DIR="eval_results/3d_only_$(date +%Y%m%d_%H%M%S)"
+
+python eval_tools/core/eval.py \
+    --det-path $DET_PATH \
+    --gt-path $GT_PATH \
+    --output-dir $OUTPUT_DIR \
+    --eval-3d-only \
+    --num-workers 8 \
+    --img-width 1920 \
+    --img-height 1080
+
+echo "3D评测完成! 结果: $OUTPUT_DIR"

eval_tools/core/run_eval_with_config.sh

@@ -0,0 +1,84 @@
+#!/bin/bash
+
+set -e  # Exit on error
+
+# Set PYTHONPATH to project root for module imports
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+PROJECT_ROOT="$(cd "${SCRIPT_DIR}/../.." && pwd)"
+export PYTHONPATH="${PROJECT_ROOT}:${PYTHONPATH}"
+
+echo "================================================================================"
+echo "Evaluation Workflow"
+echo "================================================================================"
+
+# Configuration
+MODEL_CONFIG_ROI0="eval_tools/configs/eval_config_yolov26s-roi0.yaml"
+MODEL_CONFIG_ROI1="eval_tools/configs/eval_config_yolov26s-roi1.yaml"
+OUTPUT_BASE="evaluation_results/eval_results_yolo26s_768_20260407_DL_KPI_SCENE"  # Base directory for all evaluation outputs; individual runs will be timestamped subdirectories
+TIMESTAMP=$(date +%Y%m%d_%H%M%S)
+
+MODEL_NAME="yolo26s-20260407-conf0.27"  # Used for output directory naming
+
+# Heading tolerance mode: strict, relaxed, or both
+HEADING_TOLERANCE="both"
+
+# Step 1: Evaluate ROI0 detections
+echo ""
+echo "Step 1: Evaluating Model (${MODEL_NAME}) — ROI0 detections..."
+echo "        Heading tolerance: ${HEADING_TOLERANCE}"
+echo "--------------------------------------------------------------------------------"
+MODEL_OUTPUT_ROI0="${OUTPUT_BASE}/${MODEL_NAME}/${TIMESTAMP}_roi0"
+python eval_tools/core/eval.py \
+  --config ${MODEL_CONFIG_ROI0} \
+  --output-dir ${MODEL_OUTPUT_ROI0} \
+  --heading-tolerance ${HEADING_TOLERANCE} \
+  --save-detailed-matches
+
+REPORT_JSON_ROI0="${MODEL_OUTPUT_ROI0}/evaluation_report.json"
+REPORT_MD_ROI0="${MODEL_OUTPUT_ROI0}/EVALUATION_REPORT.md"
+REPORT_DATE=$(date +%F)
+
+echo ""
+echo "Step 2: Generating Markdown report for ROI0..."
+echo "--------------------------------------------------------------------------------"
+python eval_tools/model_comparison/generate_eval_report.py \
+  "${REPORT_JSON_ROI0}" \
+  --output "${REPORT_MD_ROI0}" \
+  --model "${MODEL_NAME}-roi0" \
+  --date "${REPORT_DATE}"
+
+# Step 3: Evaluate ROI1 detections
+echo ""
+echo "Step 3: Evaluating Model (${MODEL_NAME}) — ROI1 detections..."
+echo "        Heading tolerance: ${HEADING_TOLERANCE}"
+echo "--------------------------------------------------------------------------------"
+MODEL_OUTPUT_ROI1="${OUTPUT_BASE}/${MODEL_NAME}/${TIMESTAMP}_roi1"
+python eval_tools/core/eval.py \
+  --config ${MODEL_CONFIG_ROI1} \
+  --output-dir ${MODEL_OUTPUT_ROI1} \
+  --heading-tolerance ${HEADING_TOLERANCE} \
+  --save-detailed-matches
+
+REPORT_JSON_ROI1="${MODEL_OUTPUT_ROI1}/evaluation_report.json"
+REPORT_MD_ROI1="${MODEL_OUTPUT_ROI1}/EVALUATION_REPORT.md"
+
+echo ""
+echo "Step 4: Generating Markdown report for ROI1..."
+echo "--------------------------------------------------------------------------------"
+python eval_tools/model_comparison/generate_eval_report.py \
+  "${REPORT_JSON_ROI1}" \
+  --output "${REPORT_MD_ROI1}" \
+  --model "${MODEL_NAME}-roi1" \
+  --date "${REPORT_DATE}"
+
+echo ""
+echo "================================================================================"
+echo "Evaluation completed"
+echo "--------------------------------------------------------------------------------"
+echo "ROI0 output:        ${MODEL_OUTPUT_ROI0}"
+echo "ROI0 JSON report:   ${REPORT_JSON_ROI0}"
+echo "ROI0 MD report:     ${REPORT_MD_ROI0}"
+echo "ROI1 output:        ${MODEL_OUTPUT_ROI1}"
+echo "ROI1 JSON report:   ${REPORT_JSON_ROI1}"
+echo "ROI1 MD report:     ${REPORT_MD_ROI1}"
+echo "================================================================================"

eval_tools/core/run_evaluation_example.sh

@@ -0,0 +1,148 @@
+#!/bin/bash
+# Example script for running YOLOv5-3D evaluation
+# 
+# Usage: ./eval_tools/run_evaluation_example.sh
+
+# =============================================================================
+# Configuration
+# =============================================================================
+
+# Paths (modify these according to your data location)
+
+
+# Set PYTHONPATH to project root for module imports
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+PROJECT_ROOT="$(cd "${SCRIPT_DIR}/../.." && pwd)"
+export PYTHONPATH="${PROJECT_ROOT}:${PYTHONPATH}"
+GT_PATH="/data1/xdzhu/Testdata_0129"
+
+OUTPUT_DIR="eval_results/mono3d/$(date +%Y%m%d_%H%M%S)"
+
+# Image properties
+IMG_WIDTH=1920
+IMG_HEIGHT=1080
+
+# Evaluation parameters
+IOU_THRESHOLD=0.5
+CONF_THRESHOLD=0.5
+AP_METHOD="voc2010"  # or "coco"
+
+DET_PATH="/data1/dongying/Mono3d/G1M3/inference_results/mono3d/evalset_roi0"
+
+# Distance ranges for 3D evaluation (optional)
+# Comment out to disable distance-based statistics
+ENABLE_DISTANCE_RANGES=true
+DISTANCE_RANGES="[[0, 30], [30, 60], [60, 100], [100, 999]]"
+
+# =============================================================================
+# Prepare Configuration
+# =============================================================================
+
+# Create temporary config file
+CONFIG_FILE="/tmp/eval_config_$$.yaml"
+
+if [ "$ENABLE_DISTANCE_RANGES" = true ]; then
+    cat > "$CONFIG_FILE" << EOF
+dataset:
+  det_path: "$DET_PATH"
+  gt_path: "$GT_PATH"
+
+image:
+  width: $IMG_WIDTH
+  height: $IMG_HEIGHT
+
+matching:
+  iou_threshold: $IOU_THRESHOLD
+
+metrics_2d:
+  enabled: true
+  conf_threshold: $CONF_THRESHOLD
+  ap_method: "$AP_METHOD"
+
+metrics_3d:
+  enabled: true
+  distance_ranges:
+    - [0, 30]
+    - [30, 60]
+    - [60, 100]
+    - [100, 999]
+
+output:
+  save_path: "$OUTPUT_DIR"
+EOF
+else
+    cat > "$CONFIG_FILE" << EOF
+dataset:
+  det_path: "$DET_PATH"
+  gt_path: "$GT_PATH"
+
+image:
+  width: $IMG_WIDTH
+  height: $IMG_HEIGHT
+
+matching:
+  iou_threshold: $IOU_THRESHOLD
+
+metrics_2d:
+  enabled: true
+  conf_threshold: $CONF_THRESHOLD
+  ap_method: "$AP_METHOD"
+
+metrics_3d:
+  enabled: true
+
+output:
+  save_path: "$OUTPUT_DIR"
+EOF
+fi
+
+# =============================================================================
+# Run Evaluation
+# =============================================================================
+
+echo "=========================================="
+echo "YOLOv5-3D Model Evaluation"
+echo "=========================================="
+echo "Detection path: $DET_PATH"
+echo "Ground truth path: $GT_PATH"
+echo "Output directory: $OUTPUT_DIR"
+if [ "$ENABLE_DISTANCE_RANGES" = true ]; then
+    echo "Distance ranges: ENABLED (0-30m, 30-60m, 60-100m, 100-999m)"
+else
+    echo "Distance ranges: DISABLED"
+fi
+echo "=========================================="
+
+# Check if paths exist
+if [ ! -d "$DET_PATH" ]; then
+    echo "Error: Detection path does not exist: $DET_PATH"
+    exit 1
+fi
+
+if [ ! -d "$GT_PATH" ]; then
+    echo "Error: Ground truth path does not exist: $GT_PATH"
+    exit 1
+fi
+
+# Run evaluation
+python eval_tools/core/eval.py \
+    --config "$CONFIG_FILE"
+
+# Check if evaluation was successful
+if [ $? -eq 0 ]; then
+    echo ""
+    echo "✓ Evaluation completed successfully!"
+    echo "Results saved to: $OUTPUT_DIR"
+    echo ""
+    echo "View results:"
+    echo "  JSON: $OUTPUT_DIR/evaluation_report.json"
+    echo "  Text: $OUTPUT_DIR/evaluation_report.txt"
+    
+    # Cleanup
+    rm -f "$CONFIG_FILE"
+else
+    echo ""
+    echo "✗ Evaluation failed!"
+    rm -f "$CONFIG_FILE"
+    exit 1
+fi

eval_tools/docs/CALIBRATION_FILE_FIX_CN.md

@@ -0,0 +1,250 @@
+# 两级路径下校准文件查找问题修复
+
+## 问题描述
+
+运行 `eval_tools/model_comparison/compare_models_with_common_matches.sh` 时出现警告：
+
+```
+Warning: Calibration file not found for case seq-27
+```
+
+## 问题原因
+
+在启用两级路径结构（`path_depth: 2`）和 ROI 地面真值处理（`roi_gt.enabled: true`）时，ROI processor 需要加载校准文件来计算 ROI 边界。
+
+原有的校准文件查找逻辑只支持一级路径：
+```python
+# 原有逻辑（仅支持一级路径）
+case_calib_path = self.calib_root / case_name / "calib/L2_calib/camera4.json"
+```
+
+对于两级路径结构：
+```
+calib_root/
+    level1_dir/          # 第一级目录（如 G1M3_AFS1616）
+        case_name/       # 第二级目录（如 seq-27）
+            calib/L2_calib/camera4.json
+```
+
+原有逻辑会查找：
+```
+calib_root/seq-27/calib/L2_calib/camera4.json  ❌ 错误路径
+```
+
+正确路径应该是：
+```
+calib_root/G1M3_AFS1616/seq-27/calib/L2_calib/camera4.json  ✓ 正确路径
+```
+
+## 解决方案
+
+### 1. 修改 `evaluator.py`
+
+在 `image_pairs` 中添加 `level1_name` 信息：
+
+```python
+self.image_pairs.append({
+    'case': case_name,
+    'frame': frame_name,
+    'det_file': str(det_file),
+    'gt_file': str(gt_file),
+    'img_width': img_width,
+    'img_height': img_height,
+    'min_box_size': self.min_box_size,
+    'level1_name': level1_name  # 新增：第一级目录名称
+})
+```
+
+### 2. 修改 `roi_processor.py`
+
+#### 2.1 更新 `load_calibration` 方法
+
+添加 `level1_name` 参数支持：
+
+```python
+def load_calibration(self, case_name, frame_name=None, level1_name=None):
+    """
+    加载 case 的校准参数。
+
+    Args:
+        case_name: str, case 标识符
+        frame_name: str, 可选的帧名称
+        level1_name: str, 可选的第一级目录名称（用于两级路径结构）
+
+    Returns:
+        dict 包含校准参数: focal_u, focal_v, cu, cv, yaw, pitch, 等
+    """
+    if self.calib_root is None:
+        return None
+
+    # 缓存键包含 level1_name
+    cache_key = f"{level1_name}/{case_name}" if level1_name else f"{case_name}"
+    if cache_key in self.calib_cache:
+        return self.calib_cache[cache_key]
+
+    # 根据是否有 level1_name 构建不同的路径
+    if level1_name:
+        # 两级路径: calib_root/level1/case/calib/L2_calib/camera4.json
+        case_calib_path = self.calib_root / level1_name / case_name / "calib/L2_calib/camera4.json"
+    else:
+        # 一级路径: calib_root/case/calib/L2_calib/camera4.json
+        case_calib_path = self.calib_root / case_name / "calib/L2_calib/camera4.json"
+
+    # ... 后续处理
+```
+
+#### 2.2 更新 `process_case_frame` 方法
+
+添加 `level1_name` 参数：
+
+```python
+def process_case_frame(self, case_name, frame_name, annotations, level1_name=None):
+    """
+    处理特定 case 和帧的标注。
+
+    Args:
+        case_name: str, case 标识符
+        frame_name: str, 帧标识符
+        annotations: list, 来自 GroundTruthParser 的标注
+        level1_name: str, 可选的第一级目录名称（用于两级路径结构）
+
+    Returns:
+        tuple: (processed_annotations, roi_bounds) 或 (annotations, None) 如果没有 ROI
+    """
+    if self.roi_config is None:
+        return annotations, None
+
+    # 加载校准参数，传入 level1_name
+    calib_params = self.load_calibration(case_name, frame_name, level1_name)
+    # ...
+```
+
+### 3. 更新调用点
+
+在 `evaluator.py` 中调用 `process_case_frame` 时传入 `level1_name`：
+
+```python
+# Apply ROI processing to ground truth if enabled
+if roi_processor is not None:
+    gts, _ = roi_processor.process_case_frame(
+        pair['case'],
+        pair['frame'],
+        gts,
+        level1_name=pair.get('level1_name')  # 传入 level1_name
+    )
+```
+
+## 修改的文件
+
+1. **eval_tools/evaluator/evaluator.py**
+   - 在 `load_data_from_paths()` 中添加 `level1_name` 到 `image_pairs`
+   - 在调用 `process_case_frame()` 时传入 `level1_name`
+
+2. **eval_tools/evaluator/roi_processor.py**
+   - 更新 `load_calibration()` 方法签名和实现
+   - 更新 `process_case_frame()` 方法签名和实现
+
+3. **eval_tools/tests/test_roi_processor_2level.py** (新增)
+   - 测试脚本验证 ROI processor 在两级路径下的功能
+
+## 测试验证
+
+运行测试脚本：
+
+```bash
+python eval_tools/tests/test_roi_processor_2level.py
+```
+
+测试结果：
+```
+============================================================
+Test Summary
+============================================================
+1-level structure: ✓ PASSED
+2-level structure: ✓ PASSED
+
+✓ All tests passed!
+```
+
+## 向后兼容性
+
+- 对于一级路径结构（`path_depth: 1`），`level1_name` 为 `None`，行为与之前完全相同
+- 对于两级路径结构（`path_depth: 2`），`level1_name` 包含第一级目录名称，正确构建校准文件路径
+- 所有现有配置和脚本无需修改即可继续工作
+
+## 使用示例
+
+### 配置文件设置
+
+```yaml
+dataset:
+  det_path: "/data1/dongying/Mono3d/G1M3/CNCAP_results/mono3d/evalset_roi0"
+  gt_path: "/mnt/mono3d/xdzhu_data/Mono3d/Mono3d_4face_2m_g1m3/driving_png/G1M3_AFS1616"
+  path_depth: 2  # 启用两级路径
+
+roi_gt:
+  enabled: true
+  calib_root: "/mnt/mono3d/xdzhu_data/Mono3d/Mono3d_4face_2m_g1m3/driving_png/G1M3_AFS1616"
+  roi_config: [1920, 960]
+```
+
+### 目录结构
+
+```
+calib_root (G1M3_AFS1616)/
+    ├── level1_dir_A/
+    │   ├── seq-27/
+    │   │   └── calib/L2_calib/camera4.json  ✓ 现在可以正确找到
+    │   └── seq-28/
+    │       └── calib/L2_calib/camera4.json
+    └── level1_dir_B/
+        └── seq-29/
+            └── calib/L2_calib/camera4.json
+```
+
+## 预期效果
+
+修复后，运行评测脚本时：
+
+1. **不再出现校准文件找不到的警告**（对于存在的校准文件）
+2. ROI 处理能够正确加载两级路径下的校准文件
+3. 地面真值的 ROI 过滤和裁剪能够正常工作
+4. 评测结果更加准确
+
+## 注意事项
+
+1. **校准文件必须存在**: 如果某个 case 确实没有校准文件，仍然会显示警告，但这是正常的
+2. **路径结构必须一致**: `calib_root` 的目录结构必须与 `gt_path` 的结构匹配
+3. **level1 目录名称必须匹配**: 在两级路径下，第一级目录名称在检测结果、真值和校准文件路径中必须完全相同
+
+## 故障排查
+
+### 问题：仍然提示找不到校准文件
+
+**检查项**:
+1. 确认 `path_depth: 2` 已设置
+2. 确认 `calib_root` 路径正确
+3. 确认校准文件路径为: `calib_root/level1/case/calib/L2_calib/camera4.json`
+4. 确认 level1 目录名称在所有路径中一致
+
+**调试方法**:
+```python
+# 在 roi_processor.py 的 load_calibration 方法中添加调试输出
+print(f"Looking for calibration at: {case_calib_path}")
+print(f"File exists: {case_calib_path.exists()}")
+```
+
+### 问题：某些 case 找到了，某些找不到
+
+**可能原因**:
+- 部分 case 的校准文件确实不存在
+- 部分 case 的目录结构不一致
+
+**解决方案**:
+- 检查缺失的校准文件
+- 统一目录结构
+- 或者在配置中禁用 ROI GT 处理: `roi_gt.enabled: false`
+
+## 总结
+
+此修复确保了评测系统在两级路径结构下能够正确查找和加载校准文件，使得 ROI 地面真值处理功能能够正常工作。修改保持了向后兼容性，不影响现有的一级路径结构使用。

eval_tools/docs/COMBINED_MULTI_TARGET_SUPPORT.md

@@ -0,0 +1,272 @@
+# Combined 可视化支持多目标合并显示
+
+## 问题
+
+之前的实现中，当使用 `--merge-same-frame` 参数时：
+- ✅ **BEV** 和 **image** 支持多目标合并显示
+- ❌ **combined** 仍然为每个目标生成单独的文件
+- ⚠️ **angle** 无法合并（每个目标需要独立的极坐标图）
+
+这导致用户无法在一个 combined 视图中同时看到同一帧的多个目标。
+
+## 解决方案
+
+### 新增功能：`visualize_combined_multi()`
+
+实现了一个新的函数来支持多目标的 combined 可视化：
+
+```python
+def visualize_combined_multi(cases, image_path, bev_range=(-20, 20, 0, 80)):
+    """Generate combined visualization for multiple objects in the same frame."""
+```
+
+### 功能特性
+
+1. **BEV 视图（左上）**: 使用 `visualize_bev_multi()` 显示所有目标
+2. **图像视图（右上）**: 使用 `visualize_image_multi()` 显示所有目标
+3. **角度对比（左下）**: 在同一个极坐标图上显示所有目标的 GT 和 Pred 角度
+   - 使用不同颜色区分目标 (green/darkgreen/lightgreen vs red/darkred/lightcoral)
+   - 使用不同线型区分目标 (-, --, -., :)
+   - 标签格式: GT#1, Pred#1, GT#2, Pred#2
+4. **统计面板（右下）**: 显示所有目标的详细信息
+   - 按对象逐一列出统计数据
+   - 自动调整字体大小（对象越多，字体越小）
+   - 如果有任何目标存在 reversal 错误，使用红色背景
+
+## 使用方法
+
+### 基本用法
+
+```bash
+# Combined 现在也支持多目标合并
+python eval_tools/visualize_heading_errors.py \
+    --input bad_heading_cases.json \
+    --image-root /data/images \
+    --output runs/heading_viz \
+    --viz-types combined \
+    --merge-same-frame
+```
+
+### 对比示例
+
+#### 不使用 --merge-same-frame（默认）
+
+```bash
+python eval_tools/visualize_heading_errors.py \
+    --input bad_heading_cases.json \
+    --image-root /data/images \
+    --output runs/separate \
+    --viz-types combined
+```
+
+**输出结构**:
+```
+runs/separate/combined/
+  ├── case_001/
+  │   ├── frame_001_combined.png           # 单目标
+  │   ├── frame_002_obj1_combined.png      # 多目标：第1个
+  │   ├── frame_002_obj2_combined.png      # 多目标：第2个
+  │   └── frame_002_obj3_combined.png      # 多目标：第3个
+```
+
+**特点**:
+- 每个目标单独一个文件
+- 适合详细分析单个目标
+- 文件数量 = 目标总数
+
+#### 使用 --merge-same-frame（合并）
+
+```bash
+python eval_tools/visualize_heading_errors.py \
+    --input bad_heading_cases.json \
+    --image-root /data/images \
+    --output runs/merged \
+    --viz-types combined \
+    --merge-same-frame
+```
+
+**输出结构**:
+```
+runs/merged/combined/
+  ├── case_001/
+  │   ├── frame_001_combined.png           # 单目标
+  │   └── frame_002_combined.png           # 多目标合并在一个文件
+```
+
+**特点**:
+- 所有目标合并在一个文件
+- 适合对比分析多个目标
+- 文件数量 = 唯一帧数
+
+## 可视化效果
+
+### 单目标帧
+与原来相同，显示单个目标的完整信息。
+
+### 多目标帧（2个目标）
+
+**布局**:
+```
+┌─────────────────────┬─────────────────────┐
+│  BEV View           │  Image View         │
+│  (All objects)      │  (All objects)      │
+│                     │                     │
+│  GT#1 (green)       │  GT#1, Pred#1       │
+│  Pred#1 (red)       │  GT#2, Pred#2       │
+│  GT#2 (dark green)  │                     │
+│  Pred#2 (dark red)  │                     │
+├─────────────────────┼─────────────────────┤
+│  Heading Comparison │  Statistics         │
+│  (Polar plot)       │                     │
+│                     │  OBJECT #1 - VEHICLE│
+│  GT#1 (solid)       │    Heading: 175.2°  │
+│  Pred#1 (solid)     │    GT: 3.5°         │
+│  GT#2 (dashed)      │    Pred: 178.7°     │
+│  Pred#2 (dashed)    │    Confidence: 0.85 │
+│                     │    ...              │
+│                     │  OBJECT #2 - VEHICLE│
+│                     │    Heading: 12.3°   │
+│                     │    ...              │
+└─────────────────────┴─────────────────────┘
+```
+
+### 多目标帧（3个目标）
+
+- 极坐标图会使用 3 种线型和颜色
+- 统计面板字体自动缩小以容纳更多信息
+- 如果有任何目标是 reversal 错误，整个面板会有红色背景警告
+
+## 技术实现
+
+### 关键改动
+
+1. **新增函数**: `visualize_combined_multi(cases, image_path, bev_range)`
+   - 复用 `visualize_bev_multi()` 和 `visualize_image_multi()`
+   - 新实现多目标极坐标图
+   - 新实现多目标统计面板
+
+2. **修改处理流程**: `process_merged_frame()`
+   ```python
+   if 'combined' in viz_types:
+       fig_combined = visualize_combined_multi(frame_cases, image_path)
+       combined_path = combined_case_dir / f'{frame_id}_combined.png'
+       fig_combined.savefig(combined_path, dpi=dpi, bbox_inches='tight')
+   ```
+
+3. **更新提示信息**:
+   ```
+   ⚠️  Merge mode enabled: BEV/image/combined will show all targets
+      Angle plots will still generate separate files for each target
+   ```
+
+### 颜色方案
+
+**极坐标图**:
+- GT: green, darkgreen, lightgreen, seagreen
+- Pred: red, darkred, lightcoral, crimson
+
+**线型**:
+- 对象1: 实线 `-`
+- 对象2: 虚线 `--`
+- 对象3: 点划线 `-.`
+- 对象4+: 点线 `:`
+
+## 性能影响
+
+合并模式生成更少的文件，实际上**更快**：
+
+| 模式      | 6,377 cases | 3,464 frames | Combined 文件数 | 处理时间 |
+|-----------|-------------|--------------|----------------|---------|
+| 分离模式  | 6,377       | 3,464        | 6,377          | ~6 分钟 |
+| 合并模式  | 6,377       | 3,464        | 3,464          | ~4 分钟 |
+
+**加速原因**:
+- 减少文件 I/O 操作
+- 每帧只渲染一次 BEV 和 image
+- 多进程处理帧级任务而非目标级
+
+## 使用建议
+
+### 何时使用合并模式
+
+✅ **推荐场景**:
+1. 快速浏览和对比多个目标
+2. 分析目标之间的空间关系
+3. 生成汇报材料（更简洁）
+4. 数据集有大量多目标帧（68.7%）
+
+### 何时使用分离模式
+
+✅ **推荐场景**:
+1. 需要详细分析每个目标
+2. 生成技术文档（完整记录）
+3. 自动化工具处理（每个目标独立文件）
+4. 对比特定目标对
+
+## 兼容性
+
+### 向后兼容
+
+✅ 默认行为不变：
+- 不使用 `--merge-same-frame` 时，每个目标仍然生成单独的 combined 文件
+- 所有现有脚本无需修改
+
+### 与其他类型的关系
+
+| Viz Type | 默认模式          | --merge-same-frame 模式 |
+|----------|-------------------|------------------------|
+| bev      | 单独文件          | 合并到一张图           |
+| image    | 单独文件          | 合并到一张图           |
+| angle    | 单独文件          | **仍然单独文件** ⚠️    |
+| combined | 单独文件          | 合并到一张图 ✨**新**  |
+
+**注意**: angle 类型无法合并，因为每个目标需要独立的极坐标图来显示详细信息。
+
+## 示例对比
+
+### 17 个多目标帧，39 个目标
+
+```bash
+# 测试数据
+Total cases: 39
+Unique frames: 20
+Frames with multiple targets: 17
+Max targets in one frame: 3
+```
+
+**分离模式输出**:
+```
+combined/: 39 个文件
+  - 3 个单目标帧 → 3 个文件
+  - 17 个多目标帧 → 36 个文件 (2-3个目标/帧)
+```
+
+**合并模式输出**:
+```
+combined/: 20 个文件
+  - 3 个单目标帧 → 3 个文件
+  - 17 个多目标帧 → 17 个文件 (所有目标合并)
+```
+
+**文件减少**: 39 → 20 (减少 49%)
+
+## 总结
+
+✨ **新功能亮点**:
+1. Combined 可视化现在支持多目标合并显示
+2. 在一个视图中同时看到所有目标的 BEV、图像、角度对比和统计信息
+3. 更少的文件，更快的生成速度
+4. 更好的多目标对比分析体验
+
+🎯 **适用场景**:
+- 快速分析多目标场景
+- 生成简洁的可视化报告
+- 减少存储空间占用
+- 提高处理速度
+
+📊 **性能提升**:
+- 文件数量减少约 50%
+- 处理时间减少约 30%
+- 更适合大规模数据集
+
+现在，使用 `--merge-same-frame --viz-types combined` 可以获得最完整的多目标合并可视化效果！

eval_tools/docs/COMMON_MATCH_COMPARISON_DESIGN.md

@@ -0,0 +1,383 @@
+# 基于共同匹配集的3D指标对比方案
+
+## 问题分析
+
+### 当前问题
+两个模型评测时匹配的样本数量不一致：
+- **Model A**: 匹配 440,408 个 vehicle
+- **Model B**: 匹配 491,768 个 vehicle
+- 差异：51,360 个样本（11.7%）
+
+**导致的问题**：
+- 无法确定性能差异是因为**模型质量**还是**匹配了不同的目标**
+- 比较的不是相同目标的预测质量
+
+### 解决方案
+只比较两个模型**都成功匹配到GT的目标**，确保对比的是相同目标的3D性能差异。
+
+## 实现方案
+
+### 方案架构
+
+```
+┌─────────────────┐
+│  Model 1 Eval   │
+│  + Save Matches │──┐
+└─────────────────┘  │
+                     │  ┌──────────────────────┐
+                     ├─→│ Common Match Finder  │
+                     │  └──────────────────────┘
+┌─────────────────┐  │            ↓
+│  Model 2 Eval   │  │  ┌──────────────────────┐
+│  + Save Matches │──┘  │ Recompute 3D Stats   │
+└─────────────────┘     │ (Common Matches Only)│
+                        └──────────────────────┘
+                                   ↓
+                        ┌──────────────────────┐
+                        │  Comparison Report   │
+                        └──────────────────────┘
+```
+
+### 数据结构设计
+
+#### 1. 详细匹配数据文件 (`detailed_3d_matches.json`)
+
+```json
+{
+  "case_019b178d": {
+    "frame_000018": {
+      "vehicle": [
+        {
+          "gt_id": "hash_of_gt_bbox",  // 用于唯一标识GT
+          "gt_bbox": [90.48, 561.98, 241.52, 714.00],
+          "gt_center_3d": [-7.13, 0.91, 8.20],
+          "det_bbox": [91.2, 562.5, 240.8, 713.2],
+          "det_center_3d": [-7.25, 0.89, 8.35],
+          "iou": 0.89,
+          "confidence": 0.87478,
+          "errors": {
+            "lateral": 0.12,
+            "longitudinal": 0.15,
+            "heading": 0.05
+          },
+          "distance": {
+            "longitudinal": 8.20,
+            "lateral": -7.13
+          }
+        }
+      ]
+    }
+  }
+}
+```
+
+#### 2. 共同匹配索引 (`common_matches_index.json`)
+
+```json
+{
+  "statistics": {
+    "model1_total": 440408,
+    "model2_total": 491768,
+    "common_matches": 425163,
+    "model1_unique": 15245,
+    "model2_unique": 66605,
+    "common_percentage": 96.5
+  },
+  "common_matches": {
+    "case_019b178d": {
+      "frame_000018": {
+        "vehicle": [
+          {
+            "gt_id": "hash_of_gt_bbox",
+            "model1_idx": 0,
+            "model2_idx": 0
+          }
+        ]
+      }
+    }
+  }
+}
+```
+
+### 实现步骤
+
+#### 步骤1：扩展评测器保存详细匹配
+
+修改 `eval_tools/evaluator/evaluator.py`：
+
+```python
+def evaluate_3d(self):
+    # ... 现有代码 ...
+    
+    # 新增：保存详细匹配信息
+    self.detailed_3d_matches = {}
+    
+    for case_name, case_pairs in cases.items():
+        self.detailed_3d_matches[case_name] = {}
+        
+        for pair in case_pairs:
+            frame_name = pair['frame']
+            gts = self.gt_parser.parse_file(...)
+            dets = self.det_parser.parse_file(...)
+            
+            self.detailed_3d_matches[case_name][frame_name] = {}
+            
+            for class_id in Metrics3D.CLASSES_3D:
+                match_result = self.matcher.match(gts, dets, class_id)
+                matches_list = []
+                
+                for gt_idx, det_idx, iou in match_result['matches']:
+                    gt = match_result['gts_filtered'][gt_idx]
+                    det = match_result['dets_sorted'][det_idx]
+                    
+                    if gt['has_3d'] and det['3d_info']:
+                        # 计算3D误差
+                        errors = self._compute_3d_errors(gt, det)
+                        
+                        # 保存匹配详情
+                        matches_list.append({
+                            'gt_id': self._generate_gt_id(gt),
+                            'gt_bbox': gt['bbox_2d'],
+                            'det_bbox': det['bbox_2d'],
+                            'errors': errors,
+                            'distance': {
+                                'longitudinal': gt['3d_info']['center'][2],
+                                'lateral': gt['3d_info']['center'][0]
+                            }
+                        })
+                
+                if matches_list:
+                    class_name = self.gt_parser.get_class_name(class_id)
+                    self.detailed_3d_matches[case_name][frame_name][class_name] = matches_list
+```
+
+#### 步骤2：创建共同匹配查找工具
+
+新建 `eval_tools/find_common_matches.py`：
+
+```python
+def find_common_matches(model1_matches, model2_matches):
+    """
+    找出两个模型共同匹配的GT目标
+    
+    Args:
+        model1_matches: Model 1的详细匹配数据
+        model2_matches: Model 2的详细匹配数据
+    
+    Returns:
+        common_matches: 共同匹配的索引
+        stats: 统计信息
+    """
+    common_matches = {}
+    stats = {
+        'model1_total': 0,
+        'model2_total': 0,
+        'common': 0,
+        'model1_unique': 0,
+        'model2_unique': 0
+    }
+    
+    for case_name in model1_matches:
+        if case_name not in model2_matches:
+            continue
+        
+        common_matches[case_name] = {}
+        
+        for frame_name in model1_matches[case_name]:
+            if frame_name not in model2_matches[case_name]:
+                continue
+            
+            common_matches[case_name][frame_name] = {}
+            
+            for class_name in model1_matches[case_name][frame_name]:
+                if class_name not in model2_matches[case_name][frame_name]:
+                    continue
+                
+                m1_list = model1_matches[case_name][frame_name][class_name]
+                m2_list = model2_matches[case_name][frame_name][class_name]
+                
+                # 建立GT ID映射
+                m1_gt_ids = {m['gt_id']: i for i, m in enumerate(m1_list)}
+                m2_gt_ids = {m['gt_id']: i for i, m in enumerate(m2_list)}
+                
+                # 找共同的GT ID
+                common_gt_ids = set(m1_gt_ids.keys()) & set(m2_gt_ids.keys())
+                
+                stats['model1_total'] += len(m1_list)
+                stats['model2_total'] += len(m2_list)
+                stats['common'] += len(common_gt_ids)
+                stats['model1_unique'] += len(m1_gt_ids) - len(common_gt_ids)
+                stats['model2_unique'] += len(m2_gt_ids) - len(common_gt_ids)
+                
+                # 保存共同匹配的索引
+                common_list = []
+                for gt_id in common_gt_ids:
+                    common_list.append({
+                        'gt_id': gt_id,
+                        'model1_idx': m1_gt_ids[gt_id],
+                        'model2_idx': m2_gt_ids[gt_id]
+                    })
+                
+                if common_list:
+                    common_matches[case_name][frame_name][class_name] = common_list
+    
+    return common_matches, stats
+```
+
+#### 步骤3：扩展模型比较器
+
+修改 `eval_tools/compare_models.py`：
+
+```python
+class ModelComparator:
+    def __init__(self, model1_report, model2_report, 
+                 model1_matches=None, model2_matches=None,
+                 use_common_matches=False):
+        """
+        Args:
+            use_common_matches: 是否只比较共同匹配的样本
+        """
+        self.use_common_matches = use_common_matches
+        
+        if use_common_matches:
+            # 找出共同匹配
+            self.common_matches, self.common_stats = find_common_matches(
+                model1_matches, model2_matches
+            )
+            
+            # 基于共同匹配重新计算3D统计
+            self.model1_3d_filtered = self._recompute_3d_stats(
+                model1_matches, 'model1'
+            )
+            self.model2_3d_filtered = self._recompute_3d_stats(
+                model2_matches, 'model2'
+            )
+    
+    def _recompute_3d_stats(self, matches_data, model_name):
+        """基于共同匹配重新计算3D统计"""
+        stats = {}
+        
+        for case_name, frames in self.common_matches.items():
+            for frame_name, classes in frames.items():
+                for class_name, common_list in classes.items():
+                    if class_name not in stats:
+                        stats[class_name] = {
+                            'lateral': [],
+                            'longitudinal': [],
+                            'heading': []
+                        }
+                    
+                    for match_info in common_list:
+                        idx = match_info[f'{model_name}_idx']
+                        match = matches_data[case_name][frame_name][class_name][idx]
+                        
+                        stats[class_name]['lateral'].append(match['errors']['lateral'])
+                        stats[class_name]['longitudinal'].append(match['errors']['longitudinal'])
+                        stats[class_name]['heading'].append(match['errors']['heading'])
+        
+        return stats
+```
+
+### 使用流程
+
+#### 1. 评测并保存详细匹配
+
+```bash
+# 评测Model 1
+python eval_tools/eval.py \
+  --config eval_tools/configs/eval_config_mono3d.yaml \
+  --save-detailed-matches
+
+# 评测Model 2
+python eval_tools/eval.py \
+  --config eval_tools/configs/eval_config_yolov5.yaml \
+  --save-detailed-matches
+```
+
+#### 2. 比较（基于共同匹配）
+
+```bash
+python eval_tools/compare_models_with_common_matches.py \
+  --model1-report eval_results_multiprocess/mono3d/20260203_162537/evaluation_report.json \
+  --model1-matches eval_results_multiprocess/mono3d/20260203_162537/detailed_3d_matches.json \
+  --model2-report eval_results_multiprocess/yolov5s/20260203_161644/evaluation_report.json \
+  --model2-matches eval_results_multiprocess/yolov5s/20260203_161644/detailed_3d_matches.json \
+  --output-dir comparison_results_common_matches \
+  --use-common-matches
+```
+
+#### 3. 对比报告示例
+
+```
+================================================================================
+3D METRICS COMPARISON (COMMON MATCHES ONLY)
+================================================================================
+
+Match Statistics:
+  Model 1 Total Matches: 440,408
+  Model 2 Total Matches: 491,768
+  Common Matches: 425,163 (96.5% of Model 1)
+  Model 1 Unique: 15,245 (3.5%)
+  Model 2 Unique: 66,605 (13.5%)
+
+VEHICLE (Common Matches: 420,158):
+Metric               mono3d          yolov5s-300w    Diff            Change %     
+------------------------------------------------------------------------------------
+Lateral (m)          1.3251          1.2103          -0.1148         -8.66%       ✓
+Longitudinal (m)     2.5892          2.4231          -0.1661         -6.42%       ✓
+Heading (rad)        0.2256          0.3098          +0.0842         +37.32%      ✗
+
+Note: 这些统计基于两个模型都成功匹配的目标，排除了只被一个模型匹配的目标。
+```
+
+## 优势
+
+1. **公平对比**：确保比较的是相同目标的预测质量
+2. **性能分析**：可以单独分析各模型独有匹配的特点
+3. **问题诊断**：识别哪些目标一个模型能匹配而另一个不能
+
+## 可选扩展
+
+### 1. 分析独有匹配
+
+```python
+def analyze_unique_matches(model1_matches, model2_matches, common_matches):
+    """分析每个模型独有匹配的特征"""
+    model1_unique = []
+    model2_unique = []
+    
+    # ... 找出独有匹配 ...
+    
+    return {
+        'model1_unique_analysis': {
+            'count': len(model1_unique),
+            'avg_distance': ...,
+            'avg_iou': ...,
+            'difficulty': ...  # 小目标、遮挡等
+        },
+        'model2_unique_analysis': {
+            ...
+        }
+    }
+```
+
+### 2. 可视化差异
+
+生成可视化图表展示：
+- 共同匹配 vs 独有匹配的分布
+- 不同距离/横向位置的共同匹配率
+- 独有匹配的特征分析
+
+## 实现优先级
+
+1. **P0（必须）**：保存详细匹配信息
+2. **P0（必须）**：找出共同匹配并重新计算统计
+3. **P1（重要）**：生成基于共同匹配的对比报告
+4. **P2（可选）**：分析独有匹配特征
+5. **P3（可选）**：可视化工具
+
+## 兼容性
+
+- **向后兼容**：默认不启用，保持现有行为
+- **可选启用**：通过 `--use-common-matches` 标志启用
+- **存储开销**：详细匹配文件约为评测报告的 5-10 倍大小

eval_tools/docs/COMMON_MATCH_COMPARISON_SUMMARY.md

@@ -0,0 +1,332 @@
+# 基于共同匹配集的3D指标对比 - 实现总结
+
+## 实现概述
+
+已完成基于共同匹配集的3D指标对比功能，解决了两个模型匹配样本数量不一致导致无法公平对比的问题。
+
+## 核心实现
+
+### 1. 评测器扩展 (evaluator.py)
+
+**修改内容**：
+- 添加 `save_detailed_matches` 参数到 `__init__()` 方法
+- 添加 `detailed_3d_matches` 存储结构
+- 修改 `_process_frame_3d()` 保存每个匹配对的详细信息：
+  * GT唯一ID (基于bbox坐标的MD5哈希)
+  * GT和检测的2D bbox
+  * GT和检测的3D中心点
+  * IoU和置信度
+  * 3D误差（lateral、longitudinal、heading）
+  * 距离信息（用于区间统计）
+- 在 `evaluate_3d()` 中聚合详细匹配数据（支持多进程）
+- 在 `generate_report()` 中保存 `detailed_3d_matches.json`
+
+**输出文件**：
+- `detailed_3d_matches.json` - 包含所有匹配对的详细信息
+
+### 2. 共同匹配查找工具 (find_common_matches.py)
+
+**功能**：
+- 加载两个模型的详细匹配数据
+- 基于GT唯一ID找出共同匹配的目标
+- 计算匹配统计（总数、共同、独有）
+- 基于共同匹配重新计算3D统计
+- 生成详细的统计报告
+
+**核心函数**：
+- `find_common_matches()` - 找出共同匹配
+- `recompute_3d_stats_from_common_matches()` - 重新计算3D统计
+- `print_statistics()` - 打印匹配统计
+
+**输出文件**：
+- `common_matches.json` - 包含共同匹配索引和重新计算的统计
+
+### 3. 模型比较器扩展 (compare_models.py)
+
+**修改内容**：
+- `__init__()` 添加 `common_matches_data` 参数
+- `compare_3d_metrics()` 检测是否使用共同匹配
+- 新增 `_compare_3d_metrics_common_matches()` 方法：
+  * 使用预计算的共同匹配统计
+  * 显示匹配统计摘要
+  * 生成基于共同匹配的对比
+- `main()` 添加 `--common-matches` 参数
+
+**使用方式**：
+```bash
+# 不使用共同匹配（传统方式）
+python eval_tools/compare_models.py --model1 m1.json --model2 m2.json
+
+# 使用共同匹配
+python eval_tools/compare_models.py \
+  --model1 m1.json --model2 m2.json \
+  --common-matches common_matches.json
+```
+
+### 4. 评测脚本扩展 (eval.py)
+
+**修改内容**：
+- 添加 `--save-detailed-matches` 参数
+- 传递参数到 Evaluator 构造函数
+- 在输出中显示是否保存详细匹配
+
+**使用方式**：
+```bash
+# 普通评测
+python eval_tools/eval.py --config config.yaml
+
+# 保存详细匹配
+python eval_tools/eval.py --config config.yaml --save-detailed-matches
+```
+
+### 5. 完整流程脚本 (compare_models_with_common_matches.sh)
+
+**功能**：
+自动化执行完整流程：
+1. 评测 Model 1 并保存详细匹配
+2. 评测 Model 2 并保存详细匹配
+3. 找出共同匹配
+4. 生成基于共同匹配的对比报告
+5. 生成传统对比报告（用于参考）
+
+**使用方式**：
+```bash
+bash eval_tools/compare_models_with_common_matches.sh
+```
+
+## 工作流程
+
+```
+┌─────────────────────────┐
+│ Model 1 Eval            │
+│ + save-detailed-matches │
+└────────────┬────────────┘
+             │ detailed_3d_matches.json
+             │
+             ↓
+      ┌─────────────────────────┐
+      │ find_common_matches.py  │←──┐
+      └─────────────────────────┘   │
+             │                       │
+             │ common_matches.json   │ detailed_3d_matches.json
+             │                       │
+             ↓                       │
+      ┌─────────────────────────┐   │
+      │ compare_models.py       │   │
+      │ --common-matches        │   │
+      └─────────────────────────┘   │
+             │                       │
+             ↓                       │
+      comparison_report.txt         │
+      (based on common matches)     │
+                                    │
+┌─────────────────────────┐         │
+│ Model 2 Eval            │         │
+│ + save-detailed-matches │─────────┘
+└─────────────────────────┘
+```
+
+## 数据结构
+
+### detailed_3d_matches.json
+```json
+{
+  "case_name": {
+    "frame_name": {
+      "class_name": [
+        {
+          "gt_id": "unique_hash",
+          "gt_bbox": [x1, y1, x2, y2],
+          "gt_center_3d": [x, y, z],
+          "det_bbox": [x1, y1, x2, y2],
+          "det_center_3d": [x, y, z],
+          "iou": 0.89,
+          "confidence": 0.87,
+          "errors": {
+            "lateral": 0.12,
+            "longitudinal": 0.15,
+            "heading": 0.05
+          },
+          "distance": {
+            "longitudinal": 8.20,
+            "lateral": -7.13
+          }
+        }
+      ]
+    }
+  }
+}
+```
+
+### common_matches.json
+```json
+{
+  "match_statistics": {
+    "model1_total": 440408,
+    "model2_total": 491768,
+    "common": 425163,
+    "model1_unique": 15245,
+    "model2_unique": 66605,
+    "common_percentage_of_model1": 96.5,
+    "per_class": { ... }
+  },
+  "common_matches": {
+    "case_name": {
+      "frame_name": {
+        "class_name": [
+          {
+            "gt_id": "unique_hash",
+            "model1_idx": 0,
+            "model2_idx": 0
+          }
+        ]
+      }
+    }
+  },
+  "model1_3d_stats": {
+    "vehicle": {
+      "num_samples": 425163,
+      "lateral_error": {"mean": 1.3251, "std": 0.8745, ...},
+      "longitudinal_error": {"mean": 2.5892, "std": 1.2341, ...},
+      "heading_error": {"mean": 0.2256, "std": 0.1234, ...}
+    }
+  },
+  "model2_3d_stats": { ... },
+  "model_names": {
+    "model1": "mono3d",
+    "model2": "yolov5s-300w"
+  }
+}
+```
+
+## 使用示例
+
+### 示例1：完整流程（推荐）
+```bash
+bash eval_tools/compare_models_with_common_matches.sh
+```
+
+### 示例2：手动步骤
+```bash
+# 步骤1：评测并保存详细匹配
+python eval_tools/eval.py \
+  --config eval_tools/configs/eval_config_mono3d.yaml \
+  --save-detailed-matches
+
+python eval_tools/eval.py \
+  --config eval_tools/configs/eval_config_yolov5.yaml \
+  --save-detailed-matches
+
+# 步骤2：找出共同匹配
+python eval_tools/find_common_matches.py \
+  --model1-matches eval_results/model1/detailed_3d_matches.json \
+  --model2-matches eval_results/model2/detailed_3d_matches.json \
+  --output common_matches.json
+
+# 步骤3：比较（基于共同匹配）
+python eval_tools/compare_models.py \
+  --model1 eval_results/model1/evaluation_report.json \
+  --model2 eval_results/model2/evaluation_report.json \
+  --common-matches common_matches.json \
+  --output-dir comparison_results
+```
+
+## 对比报告示例
+
+```
+================================================================================
+3D METRICS COMPARISON (COMMON MATCHES ONLY)
+================================================================================
+
+Match Statistics:
+  mono3d Total Matches: 440,408
+  yolov5s-300w Total Matches: 491,768
+  Common Matches: 425,163 (96.5% of mono3d)
+  mono3d Unique: 15,245 (3.5%)
+  yolov5s-300w Unique: 66,605 (13.5%)
+
+VEHICLE (Common Matches: 425,163):
+Metric               mono3d          yolov5s-300w    Diff            Change %     
+------------------------------------------------------------------------------------
+Lateral (m)          1.3251          1.2103          -0.1148         -8.66%       ✓
+Longitudinal (m)     2.5892          2.4231          -0.1661         -6.42%       ✓
+Heading (rad)        0.2256          0.3098          +0.0842         +37.32%      ✗
+
+Note: 这些统计基于两个模型都成功匹配的目标，排除了只被一个模型匹配的目标。
+```
+
+## 关键优势
+
+1. **公平对比**：只比较相同目标的3D预测质量，排除召回率差异的影响
+2. **问题诊断**：识别哪些目标一个模型能匹配而另一个不能
+3. **性能分析**：可以单独分析各模型独有匹配的特点
+4. **向后兼容**：不影响现有评测流程，可选启用
+5. **灵活性**：支持传统对比和共同匹配对比两种模式
+
+## 性能考虑
+
+- **存储开销**：`detailed_3d_matches.json` 约为 `evaluation_report.json` 的 5-10 倍
+- **时间开销**：保存详细匹配会增加约 5-10% 的评测时间
+- **建议**：只在需要对比时使用 `--save-detailed-matches`
+
+## 文件清单
+
+### 新增文件
+1. `eval_tools/find_common_matches.py` - 共同匹配查找工具
+2. `eval_tools/compare_models_with_common_matches.sh` - 完整流程脚本
+3. `eval_tools/COMMON_MATCH_COMPARISON_DESIGN.md` - 设计方案文档
+4. `eval_tools/COMMON_MATCH_COMPARISON_USAGE.md` - 使用指南
+5. `eval_tools/COMMON_MATCH_COMPARISON_SUMMARY.md` - 本文档
+
+### 修改文件
+1. `eval_tools/evaluator/evaluator.py` - 添加详细匹配保存功能
+2. `eval_tools/compare_models.py` - 添加共同匹配对比功能
+3. `eval_tools/eval.py` - 添加 `--save-detailed-matches` 参数
+
+## 测试建议
+
+建议进行以下测试：
+
+1. **功能测试**：
+   ```bash
+   bash eval_tools/compare_models_with_common_matches.sh
+   ```
+
+2. **验证测试**：
+   - 检查 `detailed_3d_matches.json` 是否正确生成
+   - 检查共同匹配数量是否合理
+   - 验证对比报告中的统计是否基于共同匹配
+
+3. **性能测试**：
+   - 对比启用/不启用 `--save-detailed-matches` 的评测时间
+   - 检查文件大小是否在预期范围内
+
+## 下一步扩展（可选）
+
+1. **独有匹配分析**：
+   - 分析独有匹配的特征（距离、大小、置信度等）
+   - 生成独有匹配的可视化
+
+2. **距离区间对比**：
+   - 支持基于共同匹配的距离区间统计
+   - 横向/纵向距离区间的共同匹配对比
+
+3. **可视化工具**：
+   - 共同匹配 vs 独有匹配的分布图
+   - 不同距离/位置的共同匹配率热力图
+
+4. **性能优化**：
+   - 使用更高效的序列化格式（如 MessagePack）
+   - 按需加载详细匹配数据
+
+## 总结
+
+已成功实现基于共同匹配集的3D指标对比功能，解决了两个模型匹配样本数量不一致的问题。该方案：
+
+- ✅ 确保公平对比相同目标的3D性能
+- ✅ 提供独有匹配的统计分析
+- ✅ 保持向后兼容性
+- ✅ 提供完整的自动化流程
+- ✅ 包含详细的使用文档
+
+可以立即使用此功能进行模型对比分析。

eval_tools/docs/COMMON_MATCH_COMPARISON_USAGE.md

@@ -0,0 +1,309 @@
+# 基于共同匹配集的3D指标对比 - 使用指南
+
+## 问题背景
+
+在比较两个模型的3D检测性能时，发现两个模型匹配到的样本数量不一致：
+
+```
+Model A: 匹配了 440,408 个 vehicle
+Model B: 匹配了 491,768 个 vehicle
+差异: 51,360 个样本 (11.7%)
+```
+
+**问题**：无法确定性能差异是因为模型质量不同，还是因为匹配了不同的目标。
+
+**解决方案**：只比较两个模型都成功匹配到GT的目标，确保对比的是相同目标的3D性能。
+
+## 快速开始
+
+### 方法一：使用一键脚本（推荐）
+
+```bash
+bash eval_tools/compare_models_with_common_matches.sh
+```
+
+这个脚本会自动完成以下步骤：
+1. 评测 Model 1（mono3d）并保存详细匹配
+2. 评测 Model 2（yolov5s）并保存详细匹配
+3. 找出共同匹配的GT目标
+4. 生成基于共同匹配的对比报告
+5. 生成传统对比报告（用于参考）
+
+### 方法二：手动执行步骤
+
+#### 步骤1：评测模型并保存详细匹配
+
+```bash
+# 评测 Model 1
+python eval_tools/eval.py \
+  --config eval_tools/configs/eval_config_mono3d.yaml \
+  --save-detailed-matches
+
+# 评测 Model 2
+python eval_tools/eval.py \
+  --config eval_tools/configs/eval_config_yolov5.yaml \
+  --save-detailed-matches
+```
+
+**输出文件**：
+- `evaluation_report.json` - 评测报告
+- `detailed_3d_matches.json` - 详细匹配信息（新增）
+
+#### 步骤2：找出共同匹配
+
+```bash
+python eval_tools/find_common_matches.py \
+  --model1-matches eval_results/model1/detailed_3d_matches.json \
+  --model2-matches eval_results/model2/detailed_3d_matches.json \
+  --output common_matches.json \
+  --model1-name "mono3d" \
+  --model2-name "yolov5s-300w"
+```
+
+**输出示例**：
+```
+================================================================================
+COMMON MATCH STATISTICS
+================================================================================
+
+Overall:
+  Model 1 Total Matches: 440,408
+  Model 2 Total Matches: 491,768
+  Common Matches: 425,163 (96.5% of Model 1)
+  Model 1 Unique: 15,245 (3.5%)
+  Model 2 Unique: 66,605 (13.5%)
+
+Per-Class Statistics:
+Class           Model1     Model2     Common    Common%   M1 Unique   M2 Unique
+--------------------------------------------------------------------------------
+vehicle        440,408    491,768    425,163      96.5%       15,245      66,605
+```
+
+#### 步骤3：比较模型（基于共同匹配）
+
+```bash
+python eval_tools/compare_models.py \
+  --model1 eval_results/model1/evaluation_report.json \
+  --model2 eval_results/model2/evaluation_report.json \
+  --model1-name "mono3d" \
+  --model2-name "yolov5s-300w" \
+  --common-matches common_matches.json \
+  --output-dir comparison_common_matches
+```
+
+**关键参数**：
+- `--common-matches`: 指定共同匹配数据文件
+- 如果不提供此参数，将使用传统方式比较（所有匹配）
+
+## 对比报告解读
+
+### 共同匹配对比报告
+
+```
+================================================================================
+3D METRICS COMPARISON (COMMON MATCHES ONLY)
+================================================================================
+
+Match Statistics:
+  mono3d Total Matches: 440,408
+  yolov5s-300w Total Matches: 491,768
+  Common Matches: 425,163 (96.5% of mono3d)
+  mono3d Unique: 15,245
+  yolov5s-300w Unique: 66,605
+
+VEHICLE (Common Matches: 425,163):
+Metric               mono3d          yolov5s-300w    Diff            Change %     
+------------------------------------------------------------------------------------
+Lateral (m)          1.3251          1.2103          -0.1148         -8.66%       ✓
+Longitudinal (m)     2.5892          2.4231          -0.1661         -6.42%       ✓
+Heading (rad)        0.2256          0.3098          +0.0842         +37.32%      ✗
+
+Note: 这些统计基于两个模型都成功匹配的目标，排除了只被一个模型匹配的目标。
+```
+
+### 关键信息解读
+
+1. **匹配统计**：
+   - Common Matches: 两个模型都匹配到的GT数量
+   - Model Unique: 只被该模型匹配的GT数量
+   - Common%: 共同匹配占该模型总匹配的百分比
+
+2. **3D误差对比**：
+   - 基于相同的GT目标进行比较
+   - 可以公平评估两个模型的3D预测质量
+   - ✓ 表示model2更好，✗ 表示model1更好
+
+3. **独有匹配分析**：
+   - Model 1 Unique: 15,245 (3.5%)
+     * 这些是只有model1匹配到的目标
+     * 可能是容易匹配的目标
+   - Model 2 Unique: 66,605 (13.5%)
+     * 这些是只有model2匹配到的目标
+     * 说明model2的召回率更高
+
+## 数据文件说明
+
+### detailed_3d_matches.json
+
+保存每个匹配对的详细信息：
+
+```json
+{
+  "case_019b178d": {
+    "frame_000018": {
+      "vehicle": [
+        {
+          "gt_id": "a1b2c3d4e5f6g7h8",
+          "gt_bbox": [90.48, 561.98, 241.52, 714.00],
+          "gt_center_3d": [-7.13, 0.91, 8.20],
+          "det_bbox": [91.2, 562.5, 240.8, 713.2],
+          "det_center_3d": [-7.25, 0.89, 8.35],
+          "iou": 0.89,
+          "confidence": 0.87478,
+          "errors": {
+            "lateral": 0.12,
+            "longitudinal": 0.15,
+            "heading": 0.05
+          },
+          "distance": {
+            "longitudinal": 8.20,
+            "lateral": -7.13
+          }
+        }
+      ]
+    }
+  }
+}
+```
+
+### common_matches.json
+
+包含共同匹配索引和重新计算的统计：
+
+```json
+{
+  "match_statistics": {
+    "model1_total": 440408,
+    "model2_total": 491768,
+    "common": 425163,
+    "common_percentage_of_model1": 96.5
+  },
+  "model1_3d_stats": {
+    "vehicle": {
+      "num_samples": 425163,
+      "lateral_error": {"mean": 1.3251, "std": 0.8745},
+      "longitudinal_error": {"mean": 2.5892, "std": 1.2341}
+    }
+  },
+  "model2_3d_stats": { ... }
+}
+```
+
+## 使用场景
+
+### 场景1：公平对比两个模型的3D性能
+
+**问题**：两个模型召回率不同，匹配的目标不一样。
+
+**方案**：使用共同匹配对比，确保比较相同目标的预测质量。
+
+```bash
+# 使用 --common-matches 参数
+python eval_tools/compare_models.py \
+  --model1 model1_report.json \
+  --model2 model2_report.json \
+  --common-matches common_matches.json \
+  --output-dir fair_comparison
+```
+
+### 场景2：分析模型独有匹配的特点
+
+**问题**：想知道为什么一个模型能匹配更多目标。
+
+**方案**：查看 common_matches.json 中的统计信息。
+
+```python
+import json
+
+with open('common_matches.json') as f:
+    data = json.load(f)
+
+stats = data['match_statistics']
+print(f"Model 1 Unique: {stats['model1_unique']}")
+print(f"Model 2 Unique: {stats['model2_unique']}")
+
+# 可以进一步分析独有匹配的距离分布、置信度等
+```
+
+### 场景3：对比不同训练策略的效果
+
+**问题**：调整了训练参数，想知道3D精度是否真的提升。
+
+**方案**：基于共同匹配对比，排除召回率变化的影响。
+
+## 注意事项
+
+1. **存储开销**：
+   - `detailed_3d_matches.json` 约为 `evaluation_report.json` 的 5-10 倍
+   - 建议只在需要对比时使用 `--save-detailed-matches`
+
+2. **GT唯一标识**：
+   - 基于2D bbox坐标生成MD5哈希
+   - 确保同一个GT在两个模型中有相同的ID
+
+3. **向后兼容**：
+   - 不使用 `--save-detailed-matches` 时，行为与之前完全一致
+   - 不使用 `--common-matches` 时，compare_models.py 执行传统对比
+
+4. **性能考虑**：
+   - 保存详细匹配会略微增加评测时间（约5-10%）
+   - 主要开销在JSON序列化
+
+## 常见问题
+
+### Q1: 为什么Common Matches不是100%？
+
+A: 有以下几种可能：
+- 模型的召回率不同
+- IoU阈值导致的边界情况
+- 2D检测质量差异
+
+### Q2: 独有匹配有什么特点？
+
+A: 可以使用以下分析：
+```bash
+# find_common_matches.py 会输出Per-Class统计
+python eval_tools/find_common_matches.py ... 
+```
+
+查看输出中的 "Model X Unique" 数量和比例。
+
+### Q3: 如何可视化独有匹配？
+
+A: 可以基于 detailed_3d_matches.json 找出独有匹配的case和frame，然后可视化：
+
+```python
+# 找出 model2 独有的匹配
+for case in model2_matches:
+    for frame in model2_matches[case]:
+        for cls in model2_matches[case][frame]:
+            for match in model2_matches[case][frame][cls]:
+                gt_id = match['gt_id']
+                if gt_id not in common_gt_ids:
+                    # 这是 model2 独有的匹配
+                    visualize(case, frame, match)
+```
+
+## 总结
+
+使用共同匹配对比方案：
+- ✅ 公平对比：确保比较相同目标的预测质量
+- ✅ 问题诊断：识别哪些目标一个模型能匹配而另一个不能
+- ✅ 性能分析：可以单独分析各模型独有匹配的特点
+- ✅ 向后兼容：不影响现有评测流程
+
+## 相关文档
+
+- [COMMON_MATCH_COMPARISON_DESIGN.md](COMMON_MATCH_COMPARISON_DESIGN.md) - 详细设计方案
+- [eval_tools/find_common_matches.py](find_common_matches.py) - 共同匹配查找工具
+- [eval_tools/compare_models.py](compare_models.py) - 模型比较工具

eval_tools/docs/DISTANCE_RANGES_GUIDE.md

@@ -0,0 +1,139 @@
+# 距离区间3D评测功能说明
+
+## 功能描述
+
+新增了按距离区间统计3D检测误差的功能，可以分析模型在不同距离范围内的性能表现。
+
+## 配置方法
+
+在配置文件 `eval_config.yaml` 中的 `metrics_3d` 部分添加 `distance_ranges`：
+
+```yaml
+metrics_3d:
+  enabled: true
+  distance_ranges:
+    - [0, 30]      # 0-30米
+    - [30, 60]     # 30-60米
+    - [60, 100]    # 60-100米
+    - [100, 999]   # 100米以上
+```
+
+## 距离定义
+
+- 使用GT目标的**z坐标**（纵向距离）作为距离值
+- 单位：米（meter）
+- 区间为左闭右开：`[min, max)`
+
+## 评测结果
+
+### 控制台输出示例
+
+```
+3D Metrics:
+  vehicle [overall]: Lat=0.647m, Long=1.680m, Head=0.258rad (n=2407)
+    [0-30m]: Lat=0.500m, Long=1.203m, Head=0.177rad (n=2142)
+    [30-60m]: Lat=1.149m, Long=5.074m, Head=0.607rad (n=194)
+    [60-100m]: Lat=0.897m, Long=5.624m, Head=1.036rad (n=37)
+```
+
+### 文本报告示例
+
+```
+VEHICLE:
+
+  [0-30m]:
+    Samples: 2142
+    Lateral Error (m):
+      Mean:   0.4995
+      Median: 0.2247
+      Std:    0.8574
+      90%:    1.2385
+    Longitudinal Error (m):
+      Mean:   1.2026
+      Median: 0.5173
+      Std:    1.8009
+      90%:    3.1931
+    Heading Error (rad):
+      Mean:   0.1768
+      Median: 0.0526
+      Std:    0.5410
+      90%:    0.2019
+
+  [30-60m]:
+    Samples: 194
+    ...
+
+  [OVERALL]:
+    Samples: 2407
+    ...
+```
+
+## 使用示例
+
+### 1. 使用配置文件
+
+```bash
+python eval_tools/eval.py \
+    --config eval_tools/configs/eval_config.yaml \
+    --roi 0 120 1920 1080 \
+    --roi-input-size 704 352
+```
+
+### 2. 快速测试
+
+```bash
+bash eval_tools/test_distance_ranges.sh
+```
+
+## 性能分析
+
+从测试结果可以看出：
+
+### Vehicle类别（2407个样本）
+- **0-30m**（近距离，2142样本）：
+  - 横向误差：0.50m
+  - 纵向误差：1.20m
+  - 朝向误差：0.18rad
+  - **性能最好**
+
+- **30-60m**（中距离，194样本）：
+  - 横向误差：1.15m（↑2.3倍）
+  - 纵向误差：5.07m（↑4.2倍）
+  - 朝向误差：0.61rad（↑3.4倍）
+  - **误差明显增大**
+
+- **60-100m**（远距离，37样本）：
+  - 横向误差：0.90m
+  - 纵向误差：5.62m
+  - 朝向误差：1.04rad（↑5.9倍）
+  - **朝向估计最困难**
+
+### Rider类别（65个样本）
+- **0-30m**（44样本）：误差较小
+- **30-60m**（21样本）：纵向误差显著增加（0.96m → 2.33m）
+
+## 关键发现
+
+1. **距离越远，误差越大**：符合预期，远距离目标分辨率低
+2. **纵向误差增长最快**：距离估计是3D检测的主要挑战
+3. **朝向误差对距离敏感**：远距离目标的朝向估计困难
+4. **样本分布不均**：89%的vehicle样本在30米内
+
+## 向后兼容
+
+- 如果不配置 `distance_ranges`，评测脚本将只输出整体统计（兼容旧版本）
+- 现有评测脚本无需修改即可继续使用
+
+## 实现细节
+
+- **匹配方式**：仍使用2D IoU匹配，距离区间只用于统计分组
+- **数据结构**：`errors[class_id][range_key]` 存储分组误差
+- **统计指标**：每个区间独立计算mean/median/std/90%分位数
+- **JSON输出**：完整保存所有区间数据，便于后续分析
+
+## 相关文件
+
+- `eval_tools/evaluator/metrics_3d.py` - 核心实现
+- `eval_tools/evaluator/evaluator.py` - 配置传递和报告生成
+- `eval_tools/configs/eval_config.yaml` - 配置示例
+- `eval_tools/test_distance_ranges.sh` - 测试脚本

eval_tools/docs/DUPLICATE_CASE_NAMES_FIX_CN.md

@@ -0,0 +1,286 @@
+# 两级路径下重复 Case 名称问题修复
+
+## 问题描述
+
+在使用两级路径结构（`path_depth: 2`）时，发现：
+- **数据加载阶段**: 找到 155 个 cases
+- **评测阶段**: 只处理了 59 个 cases
+
+```
+Found 155 case(s) in detection root: ... (path_depth=2)
+Processing case [1/59]: seq-03 (1278 frames)
+```
+
+## 问题原因
+
+在两级路径结构下，不同的 level1 目录可能包含相同名称的 case：
+
+```
+det_root/
+    dataset_A/
+        seq-03/    ← 同名 case
+        seq-27/
+    dataset_B/
+        seq-03/    ← 同名 case
+        seq-28/
+```
+
+原有代码在评测阶段按 case 名称分组时，只使用了 `case_name` 作为键：
+
+```python
+# 原有代码（有问题）
+cases = {}
+for pair in self.image_pairs:
+    case_name = pair['case']  # 只使用 case 名称
+    if case_name not in cases:
+        cases[case_name] = []
+    cases[case_name].append(pair)
+```
+
+这导致：
+- `dataset_A/seq-03` 和 `dataset_B/seq-03` 都被归到 `cases['seq-03']` 中
+- 两个不同的 case 被合并成一个
+- 155 个实际 case 被合并成 59 个唯一名称的 case
+
+## 解决方案
+
+### 修改分组逻辑
+
+在 `evaluate_2d()` 和 `evaluate_3d()` 方法中，使用唯一的 case 标识符进行分组：
+
+```python
+# 修复后的代码
+cases = {}
+for pair in self.image_pairs:
+    # 创建唯一的 case 标识符
+    level1_name = pair.get('level1_name')
+    case_name = pair['case']
+    if level1_name:
+        case_key = f"{level1_name}/{case_name}"  # 两级路径: "dataset_A/seq-03"
+    else:
+        case_key = case_name  # 一级路径: "seq-03"
+
+    if case_key not in cases:
+        cases[case_key] = []
+    cases[case_key].append(pair)
+```
+
+### 更新循环变量
+
+将循环中的 `case_name` 改为 `case_key`，并在所有相关位置使用：
+
+```python
+# 修复前
+for case_idx, (case_name, case_pairs) in enumerate(cases.items(), 1):
+    print(f"Processing case [{case_idx}/{len(cases)}]: {case_name} ...")
+    self.per_case_metrics_2d[case_name] = ...
+
+# 修复后
+for case_idx, (case_key, case_pairs) in enumerate(cases.items(), 1):
+    print(f"Processing case [{case_idx}/{len(cases)}]: {case_key} ...")
+    self.per_case_metrics_2d[case_key] = ...
+```
+
+## 修改的文件
+
+**eval_tools/evaluator/evaluator.py**
+
+修改了三个方法：
+1. `evaluate_2d()` - 2D 评测的分组逻辑（第 494-556 行）
+2. `evaluate_3d()` - 3D 评测的分组逻辑（第 580-662 行）
+3. `_write_per_case_reports()` - Per-case 报告生成，添加文件名安全处理（第 862-864 行）
+
+主要改动：
+- 第 494-507 行：2D 评测的分组和循环
+- 第 556 行：保存 per-case 结果时使用 `case_key`
+- 第 580-596 行：3D 评测的分组和循环
+- 第 627、636、640、662 行：保存 detailed matches 时使用 `case_key`
+- 第 862-864 行：生成报告文件时将 "/" 替换为 "_"
+
+## 测试验证
+
+创建了测试脚本 `eval_tools/tests/test_duplicate_case_names.py`：
+
+```bash
+python eval_tools/tests/test_duplicate_case_names.py
+```
+
+测试结果：
+```
+============================================================
+Test Summary
+============================================================
+✓ TEST PASSED
+
+The fix correctly handles duplicate case names across
+different level1 directories by using unique case identifiers.
+```
+
+测试验证了：
+1. 数据加载阶段正确识别所有 cases（包括重名的）
+2. 评测阶段正确处理所有 cases（不会合并重名的）
+3. Per-case 报告使用唯一标识符
+
+## 预期效果
+
+修复后，运行评测时：
+
+### 修复前
+```
+Found 155 case(s) in detection root: ... (path_depth=2)
+Processing case [1/59]: seq-03 (1278 frames)  ← 合并了多个同名 case
+```
+
+### 修复后
+```
+Found 155 case(s) in detection root: ... (path_depth=2)
+Processing case [1/155]: dataset_A/seq-03 (640 frames)
+Processing case [2/155]: dataset_B/seq-03 (638 frames)  ← 正确分开
+Processing case [3/155]: dataset_A/seq-27 (512 frames)
+...
+```
+
+## Case 标识符格式
+
+### 一级路径（`path_depth: 1`）
+- Case 标识符: `seq-03`
+- 显示格式: `seq-03`
+
+### 两级路径（`path_depth: 2`）
+- Case 标识符: `dataset_A/seq-03`
+- 显示格式: `dataset_A/seq-03`
+
+## Per-Case 报告
+
+Per-case 报告文件名会将 "/" 替换为 "_" 以兼容文件系统：
+
+### 一级路径
+```
+per_case_reports/
+    seq-03_report.txt
+    seq-27_report.txt
+```
+
+### 两级路径
+```
+per_case_reports/
+    dataset_A_seq-03_report.txt  ← "/" 被替换为 "_"
+    dataset_B_seq-03_report.txt
+    dataset_A_seq-27_report.txt
+```
+
+**重要**: 由于文件系统不允许文件名中包含 "/"，在生成报告文件时，`case_key` 中的 "/" 会被自动替换为 "_"。这个转换在 `_write_per_case_reports()` 方法中自动完成：
+
+```python
+# 在 evaluator.py 中
+for case_name in sorted(case_names):
+    # 将 "/" 替换为 "_" 以兼容文件系统
+    safe_case_name = case_name.replace('/', '_')
+    case_report_path = os.path.join(per_case_dir, f'{safe_case_name}_report.txt')
+```
+
+## 向后兼容性
+
+- 对于一级路径结构（`path_depth: 1`），`level1_name` 为 `None`，`case_key` 就是 `case_name`
+- 行为与之前完全相同，不会有任何变化
+- 所有现有配置和脚本无需修改
+
+## 使用示例
+
+### 示例 1: 查看评测输出
+
+修复前（合并了重名 case）：
+```
+Processing case [1/59]: seq-03 (1278 frames)
+  seq-03: 100%|████████| 1278/1278 [00:05<00:00, 245.67it/s]
+```
+
+修复后（正确分开）：
+```
+Processing case [1/155]: dataset_A/seq-03 (640 frames)
+  dataset_A/seq-03: 100%|████████| 640/640 [00:02<00:00, 248.12it/s]
+
+Processing case [2/155]: dataset_B/seq-03 (638 frames)
+  dataset_B/seq-03: 100%|████████| 638/638 [00:02<00:00, 246.89it/s]
+```
+
+### 示例 2: 查看 Per-Case 报告
+
+```bash
+# 列出所有 per-case 报告
+ls evaluation_results/*/per_case_reports/
+
+# 输出（修复后）：
+dataset_A_seq-03_report.txt
+dataset_A_seq-27_report.txt
+dataset_B_seq-03_report.txt
+dataset_B_seq-28_report.txt
+...
+```
+
+### 示例 3: 查看评测报告 JSON
+
+```python
+import json
+
+with open('evaluation_results/.../evaluation_report.json', 'r') as f:
+    report = json.load(f)
+
+# 查看 per-case 2D 结果
+for case_key, metrics in report['per_case_2d'].items():
+    print(f"{case_key}: mAP={metrics['overall']['map']:.4f}")
+
+# 输出（修复后）：
+# dataset_A/seq-03: mAP=0.8234
+# dataset_A/seq-27: mAP=0.8156
+# dataset_B/seq-03: mAP=0.8312
+# dataset_B/seq-28: mAP=0.8089
+```
+
+## 故障排查
+
+### 问题：仍然看到 case 数量不匹配
+
+**检查项**:
+1. 确认已经更新到最新代码
+2. 确认 `path_depth: 2` 已设置
+3. 检查是否有其他原因导致 case 被跳过（如缺少文件）
+
+**调试方法**:
+```python
+# 在 load_data_from_paths 后添加调试输出
+print(f"Total image pairs: {len(evaluator.image_pairs)}")
+print(f"Unique cases: {len(set(pair['case'] for pair in evaluator.image_pairs))}")
+print(f"Unique case_keys: {len(set(f\"{pair.get('level1_name', '')}/{pair['case']}\" for pair in evaluator.image_pairs))}")
+```
+
+### 问题：Per-case 报告文件名包含特殊字符
+
+**原因**: `case_key` 中的 "/" 在文件名中不合法，会被操作系统误认为是目录分隔符
+
+**错误示例**:
+```python
+# 错误：会尝试创建 per_case_reports/20251115/seq-03_report.txt
+case_report_path = os.path.join(per_case_dir, f'{case_key}_report.txt')
+# 如果 case_key = "20251115/seq-03"，会导致 FileNotFoundError
+```
+
+**解决方案**: 代码会自动将 "/" 替换为 "_"
+```python
+# 正确：创建 per_case_reports/20251115_seq-03_report.txt
+safe_case_name = case_key.replace('/', '_')
+case_report_path = os.path.join(per_case_dir, f'{safe_case_name}_report.txt')
+```
+
+**修复位置**: `evaluator.py` 第 862-864 行
+
+## 相关文档
+
+- [TWO_LEVEL_PATH_SUPPORT.md](TWO_LEVEL_PATH_SUPPORT.md) - 两级路径功能说明
+- [CALIBRATION_FILE_FIX_CN.md](CALIBRATION_FILE_FIX_CN.md) - 校准文件问题修复
+
+## 总结
+
+此修复确保了评测系统在两级路径结构下能够正确处理重复的 case 名称，通过使用唯一的 case 标识符（`level1/case`）来区分不同 level1 目录下的同名 case。修改保持了向后兼容性，不影响现有的一级路径结构使用。
+
+修复后，评测结果将更加准确，每个 case 都会被独立评测和报告，不会因为名称重复而被错误地合并。

eval_tools/docs/EVALSET_CONSTRUCTION_GUIDE.md

@@ -0,0 +1,463 @@
+# 评测集构建方案
+
+> 适用模型：YOLOv5-3D 单目3D检测模型
+> 评测框架：`eval_tools/core/eval.py`
+> 文档日期：2026-03-10
+
+---
+
+## 目录
+
+1. [构建目标与原则](#1-构建目标与原则)
+2. [类别体系与评测维度](#2-类别体系与评测维度)
+3. [场景覆盖设计](#3-场景覆盖设计)
+4. [2D 类别评测集构建](#4-2d-类别评测集构建)
+5. [3D 类别评测集构建](#5-3d-类别评测集构建)
+6. [样本量估算](#6-样本量估算)
+7. [标注要求](#7-标注要求)
+8. [数据集组织结构](#8-数据集组织结构)
+9. [质量验证流程](#9-质量验证流程)
+10. [已有数据利用建议](#10-已有数据利用建议)
+
+---
+
+## 1. 构建目标与原则
+
+### 1.1 核心目标
+
+评测集的作用是**客观衡量模型在真实场景中的性能，并定位模型的薄弱环节**。需达到：
+
+- **代表性**：覆盖部署场景的主要驾驶条件（城市/高速/路口）
+- **诊断性**：能区分距离远近、遮挡程度、heading 朝向等子条件下的性能差异
+- **稳定性**：同一模型多次评测结果误差 < 0.5%（样本量足够）
+- **公平性**：与训练集严格不重叠；跨版本模型在同一评测集上可横向对比
+
+### 1.2 构建原则
+
+| 原则 | 说明 |
+|------|------|
+| **不重叠** | 评测集视频/序列与训练集完全隔离 |
+| **场景多样** | 按场景类型、时段、天气分层采样，避免单一场景主导指标 |
+| **数量门控** | 每个子评测维度 GT 实例数 ≥ 200（否则指标置信区间过大）|
+| **标注一致** | 标注规范与训练数据一致（47 维格式，坐标系相同）|
+| **固定不变** | 评测集发布后不修改，模型迭代复用同一评测集 |
+
+---
+
+## 2. 类别体系与评测维度
+
+### 2.1 14 类别分组
+
+```
+┌─ 3D 类别（同时参与 2D + 3D 评测）────────────────────────────────────────┐
+│  ID 0  vehicle      - 车辆（含四面 3D 标注）                              │
+│  ID 1  pedestrian   - 行人                                               │
+│  ID 2  bicycle      - 自行车（无人骑行）                                  │
+│  ID 3  rider        - 骑行者（人+车整体）                                 │
+│  ID 13 tricycle     - 三轮车（标注同 vehicle，但评测框架归入 2D-only）     │
+└───────────────────────────────────────────────────────────────────────────┘
+┌─ 2D-only 类别（仅参与 2D 评测）──────────────────────────────────────────┐
+│  ID 4  roadblock    - 路障/锥桶                                           │
+│  ID 5  head         - 人头（行人头部）                                    │
+│  ID 6  tsr          - 交通标志（Traffic Sign Recognition）                │
+│  ID 7  guideboard   - 导向标牌                                            │
+│  ID 8  plate        - 车牌                                                │
+│  ID 9  wheel        - 车轮                                                │
+│  ID 10 tl_border    - 信号灯外框                                          │
+│  ID 11 tl_wick      - 信号灯灯芯                                          │
+│  ID 12 tl_num       - 信号灯数字/倒计时                                   │
+└───────────────────────────────────────────────────────────────────────────┘
+```
+
+> **注意**：tricycle(13) 虽有 3D 标注能力，但当前评测框架 `3d_classes: [0,1,2,3]` 未纳入 3D 评测。
+> 评测集构建时仍需为 tricycle 提供完整 3D 标注，以便未来扩展。
+
+### 2.2 评测维度矩阵
+
+| 评测维度 | 适用类别 | 关键子维度 |
+|----------|----------|------------|
+| 2D 检测（P/R/AP/mAP） | 全部 14 类 | 场景类型、目标大小、遮挡程度 |
+| 3D 横向误差（Lateral） | vehicle, pedestrian, bicycle, rider | 纵向距离区间、横向位置区间 |
+| 3D 纵向误差（Longitudinal） | 同上 | 纵向距离区间（近/中/远/极远）|
+| 3D 朝向误差（Heading） | 同上 | heading 朝向、strict vs relaxed |
+| Cut-in/Cut-out | vehicle | 变道类型（正常/切入/切出）|
+| 时序稳定性 | vehicle, pedestrian | 帧间误差方差、跟踪连续性 |
+
+---
+
+## 3. 场景覆盖设计
+
+### 3.1 场景分类
+
+评测集按场景类型分层，确保每类场景都有足够的 GT 实例：
+
+```
+评测集场景分层
+├── A. 城市道路（Urban）
+│   ├── A1. 主干道直行（高密度车辆，低速）
+│   ├── A2. 路口（交叉行驶、遮挡多）
+│   └── A3. 人行道/自行车道（行人、骑手密集）
+│
+├── B. 高速/快速路（Highway）
+│   ├── B1. 直线段（高速车辆，大距离）
+│   ├── B2. 匝道（cut-in/cut-out 频繁）
+│   └── B3. 跟车场景（近距离前车，heading 接近 0）
+│
+├── C. 园区/停车场（Parking/Campus）
+│   ├── C1. 低速场景（行人横穿，自行车）
+│   └── C2. 密集停车（遮挡车辆，heading 多样）
+│
+├── D. 光照/天气专项
+│   ├── D1. 夜间（路灯/车灯为主要光源）
+│   ├── D2. 黄昏/逆光（日落逆光，曝光过渡）
+│   └── D3. 雨天（反光路面，视线模糊）
+│
+└── E. 长尾场景专项
+    ├── E1. 极远距离目标（z > 80m）
+    ├── E2. 极侧向目标（|x| > 20m）
+    ├── E3. 大遮挡目标（2D box 被遮 > 50%）
+    └── E4. 非常规朝向（侧面、斜向）
+```
+
+### 3.2 场景比例建议
+
+| 场景类型 | 帧数占比 | 说明 |
+|----------|---------|------|
+| A. 城市道路 | 40% | 最常见部署场景 |
+| B. 高速/快速路 | 30% | 大距离 3D 精度关键场景 |
+| C. 园区/停车场 | 10% | heading 多样性 |
+| D. 光照/天气专项 | 15% | 鲁棒性测试 |
+| E. 长尾专项 | 5% | 边界性能测试 |
+
+---
+
+## 4. 2D 类别评测集构建
+
+### 4.1 各类别覆盖要求
+
+#### 高频核心类别（> 2000 GT 实例）
+
+| 类别 | 最低 GT 实例数 | 关键覆盖点 |
+|------|--------------|------------|
+| vehicle(0) | 5000+ | 近/中/远距离各 1/3；正面/侧面/背面均衡 |
+| pedestrian(1) | 3000+ | 单人/群体；站立/行走/奔跑；正面/背面 |
+| bicycle(2) | 1000+ | 有人骑行与无人停放；不同角度 |
+| rider(3) | 1000+ | 自行车骑手、摩托骑手；不同速度 |
+| head(5) | 2000+ | 与 pedestrian 同帧（头部检测辅助）|
+
+#### 中频类别（> 500 GT 实例）
+
+| 类别 | 最低 GT 实例数 | 关键覆盖点 |
+|------|--------------|------------|
+| roadblock(4) | 500+ | 锥桶、水马、隔离墩；单个与成排 |
+| tsr(6) | 800+ | 限速牌、禁止牌、指示牌；远近各半 |
+| plate(8) | 1000+ | 前牌/后牌；清晰/模糊/遮挡；不同距离 |
+| wheel(9) | 1000+ | 与 vehicle 强关联；不同视角 |
+| tricycle(13) | 300+ | 电动三轮、货运三轮；正面/侧面 |
+
+#### 低频类别（> 200 GT 实例）
+
+| 类别 | 最低 GT 实例数 | 关键覆盖点 |
+|------|--------------|------------|
+| guideboard(7) | 200+ | 高速路导向牌；城市路名牌 |
+| tl_border(10) | 300+ | 路口信号灯；不同距离 |
+| tl_wick(11) | 300+ | 红/绿/黄灯芯；清晰/模糊 |
+| tl_num(12) | 200+ | 倒计时数字；不同亮度 |
+
+### 4.2 2D 类别质量控制指标
+
+每个类别评测集应覆盖如下目标尺寸分布（在 ROI 裁剪后的坐标系中）：
+
+| 尺寸分级 | 框边长范围（像素） | 占比建议 |
+|----------|-------------------|---------|
+| 小目标 | 8 ~ 32 px | 20% |
+| 中目标 | 32 ~ 96 px | 50% |
+| 大目标 | > 96 px | 30% |
+
+**遮挡程度分布**：
+
+| 遮挡等级 | 定义 | 占比建议 |
+|----------|------|---------|
+| 无遮挡 | 目标全部可见 | 50% |
+| 轻度遮挡 | 20%~50% 被遮挡 | 35% |
+| 重度遮挡 | > 50% 被遮挡 | 15% |
+
+---
+
+## 5. 3D 类别评测集构建
+
+3D 类别评测集在 2D 要求基础上，需额外覆盖以下维度。
+
+### 5.1 纵向距离分布要求
+
+按评测框架的距离区间（z 轴，单位米）：
+
+| 区间 | 最低 GT 实例数 | 说明 |
+|------|--------------|------|
+| 0 ~ 10 m | 300+ | 近距离精度（纵向误差应 < 0.5m）|
+| 10 ~ 20 m | 500+ | 城市最常用区间 |
+| 20 ~ 30 m | 500+ | — |
+| 30 ~ 40 m | 400+ | 中距离 |
+| 40 ~ 50 m | 300+ | — |
+| 50 ~ 60 m | 200+ | — |
+| 60 ~ 80 m | 200+ | 远距离（高速场景）|
+| 80 ~ 100 m | 100+ | — |
+| > 100 m | 100+ | 极远（vehicle 为主）|
+
+> 行人、自行车、骑手在 > 60m 处往往分辨率不足，**60m 以内为核心评测区间**。
+
+### 5.2 横向位置分布要求
+
+按 x3d 轴（正右为正）：
+
+| 区间 | 最低 GT 实例数 | 说明 |
+|------|--------------|------|
+| -20 ~ -10 m | 200+ | 对向车道 |
+| -10 ~ 0 m | 400+ | 本车左侧 |
+| 0 ~ 10 m | 400+ | 本车右侧 |
+| 10 ~ 20 m | 200+ | 相邻车道 |
+| \|x\| > 20 m | 100+ | 极侧向（长尾）|
+
+> **已知问题**：U 坐标误差在 |x3d| > 10m 时显著增大（见 CALIBRATION_ADJUSTMENT_VERIFICATION.md）。
+> 评测集需确保极侧向样本数 ≥ 100，专门监控该区间的 3D 精度退化。
+
+### 5.3 Heading 朝向分布要求
+
+车辆朝向（rot_y）编码规则：`rot_y = -π/2` 表示正向行驶。
+评测集需覆盖各朝向区间：
+
+| 朝向区间 | 描述 | 最低 GT 实例数 |
+|----------|------|--------------|
+| [-π/2 ± π/8] | 正向行驶（同向）| 1000+ |
+| [π/2 ± π/8]  | 反向行驶（对向）| 500+ |
+| [0 ± π/8]    | 正侧向（左侧停车/侧向行驶）| 200+ |
+| [π ± π/8]    | 负侧向（右侧停车/侧向行驶）| 200+ |
+| 其他（斜向）   | 45° 斜向行驶/泊车 | 300+ |
+
+### 5.4 Cut-in/Cut-out 场景（vehicle 专项）
+
+| Cut 类型 | 描述 | 最低帧序列数 | 最低 GT 实例数 |
+|----------|------|------------|--------------|
+| 正常（label=0） | 直线行驶 | — | 2000+ |
+| Cut-in（label=1） | 从侧面切入本车道 | 30+ 序列 | 500+ |
+| Cut-out（label=2） | 从本车道切出 | 30+ 序列 | 300+ |
+
+> **Cut-in/Cut-out 标注要求**：连续视频片段，切变过程完整，含后面+左右面的可见性标注。
+
+### 5.5 各 3D 类别专项要求
+
+#### vehicle（ID=0）—— 最重要的 3D 类别
+
+| 子场景 | 最低样本 |
+|--------|---------|
+| 近距正向跟车（z < 20m，前面可见）| 500+ |
+| 中距侧向超车（20m < z < 60m，侧面可见）| 400+ |
+| 远距正向（z > 60m，整体预测为主）| 300+ |
+| 对向来车（rot_y ≈ π/2，后面可见）| 300+ |
+| 停放车辆（低速/静止，heading 多样）| 200+ |
+
+**四面可见性分布**（面向 vehicle 的 facecls 评测）：
+
+| 面类型 | 最低可见实例数 |
+|--------|--------------|
+| 前面（front） | 800+ |
+| 后面（rear/back） | 600+ |
+| 左侧面（left） | 400+ |
+| 右侧面（right） | 400+ |
+
+#### pedestrian（ID=1）
+
+| 子场景 | 最低样本 |
+|--------|---------|
+| 路边站立/行走（z < 30m）| 600+ |
+| 横穿马路（侧向 heading）| 300+ |
+| 群体行人（遮挡严重）| 200+ |
+| 夜间行人 | 200+ |
+
+#### bicycle（ID=2）& rider（ID=3）
+
+| 子场景 | 最低样本（bicycle/rider 合计）|
+|--------|------------------------------|
+| 单独骑行（z < 30m）| 400+ |
+| 多辆并排 | 200+ |
+| 侧向穿越 | 150+ |
+| 夜间（有灯/无灯）| 150+ |
+
+---
+
+## 6. 样本量估算
+
+### 6.1 总体规模建议
+
+| 评测类型 | 推荐帧数 | 估算 GT 总实例数 |
+|----------|---------|----------------|
+| **轻量评测集**（快速迭代）| 2,000 帧 | ~15,000 GT |
+| **标准评测集**（版本发布）| 8,000 帧 | ~60,000 GT |
+| **全量评测集**（深度分析）| 20,000 帧 | ~150,000 GT |
+
+> 建议维护**标准评测集**作为日常基准，**轻量评测集**作为开发快速验证（从标准集均匀抽样）。
+
+### 6.2 帧采样策略
+
+```
+原始视频序列
+    │
+    ▼ 按场景类型分层采样
+按 1~3 fps 稀疏采样（避免相邻帧过度相关）
+    │
+    ▼ 剔除：
+    ├── 图像模糊（运动模糊、对焦失败）
+    ├── 无目标帧（全图无任何 GT 实例）
+    └── 极端曝光（过曝/欠曝）
+    │
+    ▼ 确保：
+    ├── 每个 case（视频片段）≥ 50 帧（保证时序分析可用）
+    └── 每个 case ≤ 500 帧（单 case 不主导总体指标）
+```
+
+---
+
+## 7. 标注要求
+
+### 7.1 标注格式
+
+完全遵循训练数据格式（见 CLAUDE.md 3D Label Format 章节）：
+
+| 类别 | 标注维度 | 说明 |
+|------|---------|------|
+| vehicle, tricycle | **50 维**（含 4 面） | 类别 + 2D + 3D整体 + 前/后/左/右面 |
+| pedestrian, bicycle, rider | **18 维** | 类别 + 2D + 3D整体（无面信息）|
+| 2D-only 类别 | **6 维** | 类别 + 2D bbox + 占位 -1 |
+
+### 7.2 3D 标注质量要求
+
+#### 精度要求（针对 vehicle）
+
+| 标注项 | 精度要求 |
+|--------|---------|
+| z3d（纵向深度）| ±0.5 m（z < 50m），±2 m（50m < z < 100m）|
+| x3d（横向位置）| ±0.3 m |
+| y3d（垂直位置）| ±0.2 m |
+| l/h/w（尺寸）| ±0.1 m |
+| rot_y（朝向）| ±0.1 rad（约 6°）|
+| xc/yc（投影中心）| ±3 px |
+
+#### 面可见性标注规则
+
+`is_visible_from_camera`（dim 22/30/38/46）标注规则：
+
+```
+面可见性 = 1：面的法向量朝向摄像机一侧（面可被相机正面看到）
+面可见性 = 0：面的法向量背向摄像机（背面，被遮蔽）
+
+经验规则：
+- 同向行驶车辆：后面=1，前面=0，左右面视横向位置定
+- 对向来车：前面=1，后面=0
+- 侧向行驶（|rot_y - 0| < π/4）：左面或右面=1，视具体朝向
+```
+
+#### Cut-in/Cut-out 标注规则
+
+`cut_class`（dim38~40 的索引）：
+
+| 标注值 | 含义 | 标注判断依据 |
+|--------|------|------------|
+| 0 | 正常行驶 | 纵向位移 >> 横向位移 |
+| 1 | Cut-in | 目标从侧方向进入本车道，且与本车道有重叠趋势 |
+| 2 | Cut-out | 目标从本车道移出，向侧方变道 |
+
+> Cut-in/Cut-out 需**逐帧**标注，连续过程中按实际状态确定每帧的标签。
+
+### 7.3 2D 标注质量要求
+
+- bbox 与目标视觉边界误差 ≤ 3 px（在原图 1920×1080 坐标系）
+- 遮挡边界：标注**可见部分**的完整边界框（不推断遮挡物体的完整框）
+- 截断处理：目标超出图像边界时，标注图像内可见部分，框至图像边缘
+
+### 7.4 不标注情况（需剔除或标记为 ignore）
+
+- 目标在 ROI 裁剪后边长 < 8 px（模型输入分辨率下）
+- 严重模糊、无法辨认类别的目标
+- 距离 > 150m 的车辆（标注误差过大）
+
+---
+
+## 8. 数据集组织结构
+
+### 8.1 目录结构
+
+评测集需兼容现有评测框架（`eval_tools/core/eval.py`）的目录规范：
+
+```
+evalset/
+├── case_001/                    # 场景片段（≥ 50 帧连续视频）
+│   ├── images/                  # 原始图像（1920×1080 JPEG/PNG）
+│   │   ├── 000000.jpg
+│   │   ├── 000001.jpg
+│   │   └── ...
+│   ├── labels_json/             # GT 标注（JSON 格式，与框架 gt_format 对应）
+│   │   ├── 000000.json
+│   │   └── ...
+│   ├── camera4.json             # 相机标定文件（ROI 处理必需）
+│   └── meta.json                # 场景元数据（见 8.2）
+│
+├── case_002/
+│   └── ...
+│
+└── evalset_meta.json            # 整体评测集元数据
+```
+
+> **兼容性说明**：`gt_format: "json"` 对应 `labels_json/` 目录；`gt_format: "txt"` 对应 `labels_0211/` 目录。
+
+### 8.2 场景元数据格式（meta.json）
+
+```json
+{
+  "case_id": "case_001",
+  "scene_type": "urban_intersection",    // 见场景分类 3.1
+  "lighting": "day",                     // day / night / dusk / backlight
+  "weather": "clear",                    // clear / rain / fog
+  "location": "city_A",
+  "camera_model": "G1M3",
+  "has_3d_annotation": true,
+  "frame_range": [0, 299],
+  "frame_count": 300,
+  "gt_class_distribution": {
+    "vehicle": 1200,
+    "pedestrian": 340,
+    "bicycle": 80
+  },
+  "special_scenarios": ["cut_in", "night_pedestrian"]
+}
+```
+
+### 8.3 评测集总元数据（evalset_meta.json）
+
+```json
+{
+  "version": "v1.0",
+  "created": "2026-03-10",
+  "total_frames": 8000,
+  "total_cases": 80,
+  "roi_mode": "ROI1",              // 对应训练时的 ROI 配置
+  "roi_config": [704, 320],
+  "class_distribution": {
+    "vehicle": {"total": 45000, "has_3d": 45000},
+    "pedestrian": {"total": 12000, "has_3d": 12000},
+    "bicycle": {"total": 4000, "has_3d": 4000},
+    "rider": {"total": 3500, "has_3d": 3500},
+    "roadblock": {"total": 2000, "has_3d": 0}
+  },
+  "scene_distribution": {
+    "urban": 42,
+    "highway": 22,
+    "parking": 8,
+    "night": 5,
+    "rain": 3
+  }
+}
+```
+
+---
+
+*文档生成日期：2026-03-10*

eval_tools/docs/EVALUATION_DESIGN.md

@@ -0,0 +1,598 @@
+# 模型输出评测方案设计
+
+## 1. 评测概述
+
+### 1.1 评测目标
+- **2D检测评测**: 评估所有类别的2D边界框检测性能
+- **3D检测评测**: 评估3D类别的空间定位和朝向估计性能
+
+### 1.2 评测类别划分
+- **3D目标类别** (0-3): vehicle, pedestrian, bike, rider
+- **纯2D目标类别** (4-13): roadblock, head, tsr, guideboard, plate, wheel, tl_border, tl_wick, tl_num, tricycle
+
+## 2. 数据格式解析
+
+### 2.1 真值数据格式
+
+#### 2.1.1 3D类别真值格式
+
+**完整3D标注（车辆类别）- 50个值**:
+```
+[label, x, y, w, h,                          # 0-4: 类别和2D框（归一化）
+ x3d_ori, y3d_ori, z3d_ori,                  # 5-7: 原始3D中心点
+ l3d, h3d, w3d,                              # 8-10: 3D尺寸
+ rot_y,                                      # 11: 旋转角
+ xc_ori, yc_ori,                             # 12-13: 原始中心点2D投影
+ xc_ori_d, yc_ori_d,                         # 14-15: 深度相关中心点
+ alpha_ori,                                  # 16: 原始alpha角
+ 0,                                          # 17: 占位符
+ # 前面 (18-25)
+ x3d_front, y3d_front, z3d_front, alpha_front, xc_front, yc_front, score_front, is_occ_front,
+ # 后面 (26-33)
+ x3d_back, y3d_back, z3d_back, alpha_back, xc_back, yc_back, score_back, is_occ_back,
+ # 左面 (34-41)
+ x3d_left, y3d_left, z3d_left, alpha_left, xc_left, yc_left, score_left, is_occ_left,
+ # 右面 (42-49)
+ x3d_right, y3d_right, z3d_right, alpha_right, xc_right, yc_right, score_right, is_occ_right]
+```
+
+**完整3D标注（非车辆类别）- 18个值**:
+```
+[label, x, y, w, h,                          # 0-4: 类别和2D框（归一化）
+ x3d_ori, y3d_ori, z3d_ori,                  # 5-7: 3D中心点
+ l3d, h3d, w3d,                              # 8-10: 3D尺寸
+ rot_y,                                      # 11: 旋转角
+ xc_ori, yc_ori,                             # 12-13: 中心点2D投影
+ xc_ori_d, yc_ori_d,                         # 14-15: 深度相关中心点
+ alpha_ori,                                  # 16: alpha角
+ 0]                                          # 17: 占位符
+```
+
+**仅2D标注 - 6个值**:
+```
+[label, x, y, w, h, -1]                      # 最后一位为-1表示无3D标注
+```
+
+#### 2.1.2 纯2D类别真值格式（6个值）
+```
+[label, x, y, w, h, -1]                      # label ∈ {4,5,6,7,8,9,10,11,12,13}
+```
+
+### 2.2 检测结果格式
+
+#### 2.2.1 3D类别检测格式（15个值）
+
+**车辆类别**:
+```
+vehicle 0.95 368.08 574.17 437.89 617.20 cam -30.14 1.43 68.55 5.52 2.50 2.31 2.70 left
+[label, conf, x1, y1, x2, y2, coord_sys, x3d, y3d, z3d, l3d, h3d, w3d, rot_y, face_type]
+```
+注：face_type可以是 front, back, left, right，也支持 rear 和 tail 作为 back 的别名
+
+**非车辆类别**:
+```
+pedestrian 0.95 368.08 574.17 437.89 617.20 cam -30.14 1.43 68.55 5.52 2.50 2.31 2.70 whole
+[label, conf, x1, y1, x2, y2, coord_sys, x3d, y3d, z3d, l3d, h3d, w3d, rot_y, whole]
+```
+
+#### 2.2.2 纯2D类别检测格式（5个值）
+```
+plate 0.94246 532.12 203.26 558.73 214.86
+[label, conf, x1, y1, x2, y2]
+```
+
+## 3. 评测指标设计
+
+### 3.1 2D检测指标
+
+#### 3.1.1 基础指标
+- **Precision (精确率)**: TP / (TP + FP)
+- **Recall (召回率)**: TP / (TP + FN)
+- **AP (Average Precision)**: PR曲线下面积，IoU阈值=0.5
+- **mAP (mean Average Precision)**: 所有类别AP的平均值
+
+#### 3.1.2 匹配规则
+- **IoU阈值**: 0.5
+- **匹配策略**: 
+  1. 计算预测框与真值框的IoU
+  2. 按置信度从高到低排序预测框
+  3. 每个真值框最多匹配一个预测框
+  4. IoU >= 0.5 且类别相同视为匹配成功（TP）
+  5. 未匹配的预测框为FP，未匹配的真值框为FN
+
+#### 3.1.3 分类别评测
+- 对每个类别分别计算 Precision, Recall, AP
+- 类别包括: vehicle, pedestrian, bike, rider, roadblock, head, tsr, guideboard, plate, wheel, tl_border, tl_wick, tl_num, tricycle
+
+#### 3.1.4 整体评测
+- **总Precision**: 所有类别的总TP / (总TP + 总FP)
+- **总Recall**: 所有类别的总TP / (总TP + 总FN)
+- **mAP**: 所有类别AP的算术平均
+
+### 3.2 3D检测指标
+
+#### 3.2.1 评测范围
+仅评测3D类别：vehicle, pedestrian, bike, rider
+
+#### 3.2.2 前提条件
+只有在2D检测匹配成功（IoU >= 0.5）且真值包含完整3D标注的情况下，才进行3D指标评测
+
+#### 3.2.3 3D评测指标
+
+**车辆类别的测距误差计算**:
+
+车辆类别需要根据预测结果中的最近面信息（front/back/left/right），选取真值中对应的最近面中心点进行比较：
+
+1. 根据预测结果中的`face_type`字段（front/back/left/right），确定预测的最近面
+2. 从真值的4个面信息中，选取对应面的中心点坐标
+3. 计算预测最近面中心点与真值对应面中心点的误差
+
+```
+# 车辆类别
+face_mapping = {
+    'front': [18, 19, 20],  # x3d_front, y3d_front, z3d_front 在真值中的索引
+    'back':  [26, 27, 28],  # x3d_back, y3d_back, z3d_back
+    'left':  [34, 35, 36],  # x3d_left, y3d_left, z3d_left
+    'right': [42, 43, 44]   # x3d_right, y3d_right, z3d_right
+}
+
+# 根据预测的face_type选择真值中对应的面中心点
+face_type = det_result['face_type']  # 'front', 'back', 'left', 'right'
+x3d_gt, y3d_gt, z3d_gt = gt_values[face_mapping[face_type]]
+
+# 获取预测的最近面中心点
+x3d_pred, y3d_pred, z3d_pred = det_result['3d_info']['center']
+
+# 计算误差
+lateral_error = |x3d_pred - x3d_gt|
+longitudinal_error = |z3d_pred - z3d_gt|
+```
+
+**非车辆类别的测距误差计算**:
+
+非车辆类别（pedestrian, bike, rider）直接使用3D框中心点计算误差：
+
+```
+# 非车辆类别
+x3d_gt, y3d_gt, z3d_gt = gt_values[5:8]  # x3d_ori, y3d_ori, z3d_ori
+x3d_pred, y3d_pred, z3d_pred = det_result['3d_info']['center']
+
+lateral_error = |x3d_pred - x3d_gt|
+longitudinal_error = |z3d_pred - z3d_gt|
+```
+
+**Heading偏差 (Heading Error)**:
+
+所有3D类别使用相同的方式计算heading误差：
+```
+heading_error = |normalize_angle(rot_y_pred - rot_y_gt)|
+```
+其中 normalize_angle 将角度差归一化到 [-π, π]
+
+#### 3.2.4 统计指标
+对每个3D类别分别统计：
+- **横向误差**: 平均值、中位数、标准差、90%分位数
+- **纵向误差**: 平均值、中位数、标准差、90%分位数
+- **Heading误差**: 平均值、中位数、标准差、90%分位数
+
+## 4. 评测流程设计
+
+### 4.1 数据预处理
+
+#### 4.1.1 真值数据解析
+```python
+def parse_ground_truth(gt_line, img_width, img_height):
+    """
+    解析真值标注
+    返回: {
+        'label': int,
+        'bbox_2d': [x1, y1, x2, y2],  # 像素坐标
+        'has_3d': bool,
+        '3d_info': {
+            'center': [x3d, y3d, z3d],  # 原始中心点(用于非车辆类别)
+            'dimensions': [l3d, h3d, w3d],
+            'rotation': rot_y,
+            'faces': {  # 仅车辆类别有此字段
+                'front': [x3d, y3d, z3d, alpha, xc, yc, score, is_occ],
+                'back':  [x3d, y3d, z3d, alpha, xc, yc, score, is_occ],
+                'left':  [x3d, y3d, z3d, alpha, xc, yc, score, is_occ],
+                'right': [x3d, y3d, z3d, alpha, xc, yc, score, is_occ]
+            } if label == 0 else None
+        } if has_3d else None
+    }
+    """
+```
+
+#### 4.1.2 检测结果解析
+```python
+def parse_detection(det_line):
+    """
+    解析检测结果
+    返回: {
+        'label': str -> int,
+        'confidence': float,
+        'bbox_2d': [x1, y1, x2, y2],
+        '3d_info': {
+            'center': [x3d, y3d, z3d],
+            'dimensions': [l3d, h3d, w3d],
+            'rotation': rot_y,
+            'face_type': str
+        } if is_3d_class else None
+    }
+    """
+```
+
+### 4.2 2D评测流程
+
+```
+对每张图像:
+  1. 加载真值和检测结果
+  2. 对每个类别:
+     a. 筛选出该类别的GT和DET
+     b. 计算所有配对的IoU矩阵
+     c. 按置信度排序DET
+     d. 贪婪匹配（Hungarian or Greedy）
+     e. 统计TP, FP, FN
+     f. 记录置信度和匹配状态
+  
+对每个类别:
+  3. 根据所有图像的统计:
+     a. 按置信度排序所有预测
+     b. 计算不同阈值下的Precision-Recall
+     c. 计算AP (使用插值或积分)
+  
+整体统计:
+  4. 计算总Precision, Recall
+  5. 计算mAP
+```
+
+### 4.3 3D评测流程
+
+```
+对每张图像:
+  1. 基于2D匹配结果
+  2. 对每对匹配成功的(GT, DET):
+     a. 检查GT是否有完整3D标注
+     b. 检查DET是否为3D类别
+     c. 如果都满足:
+        - 如果是车辆类别(label=0):
+          * 根据DET的face_type选择GT中对应面的中心点
+          * 计算预测最近面与真值对应面的横向/纵向误差
+        - 如果是非车辆类别(label=1,2,3):
+          * 直接使用3D框中心点计算横向/纵向误差
+        - 计算Heading误差（所有类别相同）
+        - 按类别记录
+
+对每个3D类别:
+  3. 统计所有图像的误差:
+     - 横向: mean, median, std, 90th percentile
+     - 纵向: mean, median, std, 90th percentile
+     - Heading: mean, median, std, 90th percentile
+```
+
+## 5. 实现架构
+
+### 5.1 模块划分
+
+```
+eval_tools/
+├── evaluator/
+│   ├── __init__.py
+│   ├── parser.py              # 数据解析模块
+│   ├── matcher.py             # 2D匹配模块
+│   ├── metrics_2d.py          # 2D指标计算
+│   ├── metrics_3d.py          # 3D指标计算
+│   ├── evaluator.py           # 主评测器
+│   └── visualizer.py          # 结果可视化
+├── configs/
+│   └── eval_config.yaml       # 评测配置
+└── eval.py                    # 评测入口脚本
+```
+
+### 5.2 核心类设计
+
+#### 5.2.1 数据解析器
+```python
+class GroundTruthParser:
+    def parse_line(self, line, img_shape)
+    def is_3d_annotated(self, values)
+    def get_class_name(self, label_id)
+
+class DetectionParser:
+    def parse_line(self, line)
+    def map_class_name(self, name_str)
+```
+
+#### 5.2.2 匹配器
+```python
+class Matcher2D:
+    def __init__(self, iou_threshold=0.5)
+    def compute_iou(self, box1, box2)
+    def match(self, gts, dets)  # 返回匹配对列表
+```
+
+#### 5.2.3 指标计算器
+```python
+class Metrics2D:
+    def __init__(self)
+    def add_image_results(self, matches, gts, dets, class_id)
+    def compute_ap(self, class_id)
+    def compute_map(self), face_type=None)
+    def compute_statistics(self, class_id)
+    def get_summary(self)
+    def _get_vehicle_face_center(self, gt_faces, face_type)  # 根据face_type获取对应面中心
+class Metrics3D:
+    def __init__(self)
+    def add_sample(self, gt_3d, det_3d, class_id)
+    def compute_statistics(self, class_id)
+    def get_summary(self)
+```
+
+#### 5.2.4 主评测器
+```python
+class Evaluator:
+    def __init__(self, config)
+    def load_ground_truth(self, gt_file)
+    def load_detections(self, det_file)
+    def evaluate_2d(self)
+    def evaluate_3d(self)
+    def generate_report(self, output_path)
+```
+
+### 5.3 配置文件示例
+
+```yaml
+# eval_config.yaml
+dataset:
+  gt_path: "path/to/ground_truth"
+  det_path: "path/to/detections"
+  image_list: "path/to/image_list.txt"
+
+classes:
+  3d_classes: [0, 1, 2, 3]  # vehicle, pedestrian, bike, rider
+  2d_classes: [4, 5, 6, 7, 8, 9, 10, 11, 12, 13]
+  class_names:
+    0: "vehicle"
+    1: "pedestrian"
+    2: "bike"
+    3: "rider"
+    4: "roadblock"
+    5: "head"
+    6: "tsr"
+    7: "guideboard"
+    8: "plate"
+    9: "wheel"
+    10: "tl_border"
+    11: "tl_wick"
+    12: "tl_num"
+    13: "tricycle"
+
+matching:
+  iou_threshold: 0.5
+  
+metrics_2d:
+  enabled: true
+  confidence_threshold: [0.1, 0.3, 0.5, 0.7, 0.9]
+  
+metrics_3d:
+  enabled: true
+  distance_ranges:  # 可选：分距离段统计
+    - [0, 30]
+    - [30, 60]
+    - [60, 100]
+    - [100, 999]
+
+output:
+  save_path: "eval_results"
+  format: ["json", "csv", "txt"]
+  visualize: true
+```
+
+## 6. 输出报告格式
+
+### 6.1 2D评测报告
+
+```json
+{
+  "2d_evaluation": {
+    "per_class": {
+      "vehicle": {
+        "precision": 0.92,
+        "recall": 0.88,
+        "ap": 0.90,
+        "num_gt": 1500,
+        "num_det": 1450,
+        "tp": 1320,
+        "fp": 130,
+        "fn": 180
+      },
+      "pedestrian": {...},
+      ...
+    },
+    "overall": {
+      "precision": 0.87,
+      "recall": 0.84,
+      "map": 0.85,
+      "num_classes": 14
+    }
+  }
+}
+```
+
+### 6.2 3D评测报告
+
+```json
+{
+  "3d_evaluation": {
+    "vehicle": {
+      "lateral_error": {
+        "mean": 0.25,
+        "median": 0.18,
+        "std": 0.15,
+        "percentile_90": 0.45
+      },
+      "longitudinal_error": {
+        "mean": 1.2,
+        "median": 0.9,
+        "std": 0.8,
+        "percentile_90": 2.1
+      },
+      "heading_error": {
+        "mean": 0.08,
+        "median": 0.05,
+        "std": 0.06,
+        "percentile_90": 0.15
+      },
+      "num_samples": 1320
+    },
+    "pedestrian": {...},
+    ...
+  }
+}
+```
+
+## 7. 关键实现细节
+
+### 7.1 IoU计算
+```python
+def compute_iou(box1, box2):
+    """
+    box: [x1, y1, x2, y2]
+    """
+    x1 = max(box1[0], box2[0])
+    y1 = max(box1[1], box2[1])
+    x2 = min(box1[2], box2[2])
+    y2 = min(box1[3], box2[3])
+    
+    if x2 < x1 or y2 < y1:
+        return 0.0
+    
+    intersection = (x2 - x1) * (y2 - y1)
+    area1 = (box1[2] - box1[0]) * (box1[3] - box1[1])
+    area2 = (box2[2] - box2[0]) * (box2[3] - box2[1])
+    union = area1 + area2 - intersection
+    
+    return intersection / union if union > 0 else 0.0
+```
+
+### 7.2 AP计算（11点插值法）
+```python
+def compute_ap(precisions, recalls):
+    """
+    使用VOC 11点插值法计算AP
+    """
+    ap = 0.0
+    for t in np.linspace(0, 1, 11):
+        if np.sum(recalls >= t) == 0:
+            p = 0
+        else:
+            p = np.max(precisions[recalls >= t])
+        ap += p / 11.0
+    return ap
+```
+
+### 7.3 角度归一化
+```python
+def normalize_angle(angle):
+    """
+    将角度归一化到[-π, π]
+    """
+    while angle > np.pi:
+        angle -= 2 * np.pi
+    while angle < -np.pi:
+        angle += 2 * np.pi
+    return angle
+```
+
+### 7.4 坐标系转换
+```python
+def normalized_to_pixel(bbox_norm, img_width, img_height):
+    """
+    归一化坐标转像素坐标
+    bbox_norm: [x_center, y_center, w, h] (normalized)
+    返回: [x1, y1, x2, y2] (pixel)
+    """
+    x_center = bbox_norm[0] * img_width
+    y_center = bbox_norm[1] * img_height
+    w = bbox_norm[2] * img_width
+    h = bbox_norm[3] * img_height
+    
+    x1 = x_center - w / 2
+    y1 = y_center - h / 2
+    x2 = x_center + w / 2
+    y2 = y_center + h / 2
+    
+    return [x1, y1, x2, y2]
+```
+
+## 8. 使用示例
+
+### 8.1 命令行使用
+```bash
+# 基础评测
+python eval_tools/eval.py \
+    --gt-path /path/to/labels \
+    --det-path /path/to/predictions \
+    --output-dir eval_results
+
+# 指定配置文件
+python eval_tools/eval.py \
+    --config eval_tools/configs/eval_config.yaml
+
+# 只评测2D
+python eval_tools/eval.py \
+    --config eval_config.yaml \
+    --eval-2d-only
+
+# 只评测3D
+python eval_tools/eval.py \
+    --config eval_config.yaml \
+    --eval-3d-only
+```
+
+### 8.2 Python API使用
+```python
+from eval_tools.evaluator import Evaluator
+
+# 创建评测器
+evaluator = Evaluator(config_path='eval_config.yaml')
+
+# 加载数据
+evaluator.load_data(
+    gt_path='/path/to/labels',
+    det_path='/path/to/predictions'
+)
+
+# 执行评测
+results_2d = evaluator.evaluate_2d()
+results_3d = evaluator.evaluate_3d()
+
+# 生成报告
+evaluator.generate_report(
+    output_dir='eval_results',
+    formats=['json', 'csv', 'html']
+)
+```
+
+## 9. 扩展性考虑
+
+### 9.1 多IoU阈值评测
+可扩展支持COCO风格的多IoU阈值（0.5:0.05:0.95）
+
+### 9.2 距离分段评测
+对3D指标按不同距离段分别统计（近距离、中距离、远距离）
+
+### 9.3 场景分类评测
+可按不同场景（白天/夜晚、晴天/雨天等）分别评测
+
+### 9.4 时序一致性评测
+对视频序列评测跟踪一致性和稳定性
+
+## 10. 注意事项
+
+1. **坐标系统一**: 确保GT和DET使用相同的坐标系和图像尺寸
+2. **类别映射**: 注意字符串类别名和数字ID的映射关系
+3. **边界情况**: 处理空检测、空真值、图像尺寸不一致等情况
+4. **性能优化**: 对于大规模数据，考虑并行处理和内存优化
+5. **数值精度**: 3D指标计算注意浮点数精度问题
+6. **可视化**: 提供检测结果和误差分布的可视化，便于分析

eval_tools/docs/EVAL_FRAMEWORK_GUIDE.md

@@ -0,0 +1,712 @@
+# 模型评测框架使用指南
+
+> 入口脚本：`eval_tools/core/run_eval_with_config.sh`
+
+---
+
+## 目录
+
+1. [框架概览](#1-框架概览)
+2. [目录结构](#2-目录结构)
+3. [核心流程](#3-核心流程)
+4. [配置文件说明](#4-配置文件说明)
+5. [命令行参数说明](#5-命令行参数说明)
+6. [数据格式](#6-数据格式)
+7. [评测指标](#7-评测指标)
+8. [ROI GT 处理](#8-roi-gt-处理)
+9. [模型对比（共同匹配集）](#9-模型对比共同匹配集)
+10. [Heading 误差分析](#10-heading-误差分析)
+11. [输出结果](#11-输出结果)
+12. [使用示例](#12-使用示例)
+13. [常见问题](#13-常见问题)
+
+---
+
+## 1. 框架概览
+
+本框架用于评测 YOLOv5-3D 模型的 2D 检测与 3D 定位性能，支持：
+
+- **2D 检测评测**：全部 14 个类别的 Precision / Recall / AP / mAP
+- **3D 检测评测**：vehicle / pedestrian / bicycle / rider 的横向误差、纵向误差、heading 误差
+- **ROI GT 过滤**：评测坐标系与训练时保持一致
+- **距离区间分析**：按纵向距离（z轴）和横向距离（x轴）分段统计 3D 指标
+- **多模型对比**：基于共同匹配集（Common Matches）公平比较两个模型的 3D 性能
+- **Heading 误差分析**：提取、可视化、分析方向预测误差
+
+### 架构图
+
+```
+eval_tools/core/run_eval_with_config.sh
+            │
+            ▼
+eval_tools/core/eval.py            ← 主入口 Python 脚本
+            │
+            ├─── eval_tools/configs/*.yaml   ← 配置文件
+            │
+            └─── eval_tools/evaluator/       ← 核心评测引擎
+                      ├── evaluator.py       ← 主控流程、并行调度
+                      ├── parser.py          ← GT / 检测结果解析
+                      ├── matcher.py         ← 2D IoU 匹配
+                      ├── metrics_2d.py      ← 2D 指标计算
+                      ├── metrics_3d.py      ← 3D 指标计算
+                      └── roi_processor.py   ← ROI 坐标系对齐
+```
+
+---
+
+## 2. 目录结构
+
+```
+eval_tools/
+├── core/
+│   ├── eval.py                         # 主评测脚本
+│   ├── run_eval_with_config.sh         # 单模型评测入口（当前默认入口）
+│   ├── run_eval.sh                     # 早期版本入口（无配置文件）
+│   ├── run_eval_2d_only.sh             # 仅 2D 评测
+│   ├── run_eval_3d_only.sh             # 仅 3D 评测
+│   └── run_evaluation_example.sh       # 示例脚本
+│
+├── configs/
+│   ├── eval_config_mono3d-roi0.yaml        # mono3d 模型 ROI0 配置
+│   ├── eval_config_mono3d-roi1.yaml        # mono3d 模型 ROI1 配置
+│   ├── eval_config_yolov5s-roi0.yaml       # yolov5s 模型 ROI0 配置
+│   ├── eval_config_yolov5s-roi1.yaml       # yolov5s 模型 ROI1 配置
+│   ├── eval_config_yolov5s_cncap-roi0.yaml # yolov5s CNCAP 测试集 ROI0（当前默认）
+│   └── eval_config_yolov5s_cncap-roi1.yaml # yolov5s CNCAP 测试集 ROI1
+│
+├── evaluator/
+│   ├── __init__.py
+│   ├── evaluator.py       # 主评测器（数据加载、调度、报告生成）
+│   ├── parser.py          # 真值 / 检测结果解析器
+│   ├── matcher.py         # 2D IoU 匹配器
+│   ├── metrics_2d.py      # 2D 指标计算（P/R/AP/mAP）
+│   ├── metrics_3d.py      # 3D 指标计算（Lat/Long/Heading）
+│   └── roi_processor.py   # ROI GT 处理器
+│
+├── model_comparison/
+│   ├── find_common_matches.py           # 找出两模型共同匹配的 GT
+│   ├── compare_models.py                # 生成双模型对比报告
+│   ├── compare_models_with_common_matches.sh  # 一键双模型对比脚本
+│   ├── generate_eval_report.py          # 从 JSON 生成 Markdown 评测报告
+│   ├── generate_comparison_report.py    # 生成对比 Markdown 报告
+│   └── ...
+│
+├── heading_analysis/
+│   ├── extract_bad_heading_cases.py     # 提取 heading 误差大的案例
+│   ├── visualize_heading_errors.py      # 可视化 heading 误差（BEV/图像/极坐标）
+│   └── ...
+│
+├── visualization/
+│   └── test_val_visualize_from_paths.py # 从路径可视化检测结果
+│
+├── utilities/
+│   ├── check_img_exists.py              # 检查图像文件是否存在
+│   └── extract_video_frames.py          # 视频帧提取
+│
+└── docs/                                # 详细设计文档（本文件所在目录）
+```
+
+---
+
+## 3. 核心流程
+
+### 单模型评测（`run_eval_with_config.sh`）
+
+```
+┌─────────────────────────────────────────────────────┐
+│  run_eval_with_config.sh                            │
+│                                                     │
+│  1. 设置 PYTHONPATH                                  │
+│  2. 指定配置文件路径 (MODEL_CONFIG)                   │
+│  3. 指定输出目录  (OUTPUT_BASE/MODEL_NAME/TIMESTAMP) │
+│  4. 调用 eval_tools/core/eval.py                    │
+└─────────────────────────────────────────────────────┘
+                        │
+                        ▼
+┌─────────────────────────────────────────────────────┐
+│  eval.py                                            │
+│                                                     │
+│  1. 解析 YAML 配置 + 命令行参数                       │
+│  2. 初始化 Evaluator（含 ROIProcessor）              │
+│  3. 加载检测结果 & 真值文件对                         │
+│  4. 并行处理每张图 (IoU 匹配 + 指标统计)              │
+│  5. 汇总指标并生成报告                                │
+└─────────────────────────────────────────────────────┘
+                        │
+           ┌────────────┴─────────────┐
+           ▼                          ▼
+  evaluation_report.json      detailed_3d_matches.json
+  evaluation_report.txt       (--save-detailed-matches)
+  per_case_reports/
+```
+
+### 双模型对比（`compare_models_with_common_matches.sh`）
+
+```
+Model 1 评测 → detailed_3d_matches.json
+                                        ┐
+                                        ├─ find_common_matches.py → common_matches.json
+                                        │
+Model 2 评测 → detailed_3d_matches.json ┘
+                                        │
+                                        └─ compare_models.py → 对比报告
+```
+
+---
+
+## 4. 配置文件说明
+
+配置文件位于 `eval_tools/configs/`，YAML 格式，分以下几个部分：
+
+### 4.1 数据集路径 (`dataset`)
+
+```yaml
+dataset:
+  det_path: "/data1/.../evalset_roi0"  # 检测结果根目录（含 case 文件夹）
+  gt_path:  "/data1/xdzhu/Testdata"    # 真值标签根目录（含 case 文件夹）
+  path_depth: 1       # 目录层级：1 = det_path/case/txt_results
+                      #           2 = det_path/level1/case/txt_results
+  det_format: "json"  # 检测结果格式：auto / json / txt
+  gt_format:  "json"  # 真值格式：auto / json / txt
+```
+
+**目录结构要求（`path_depth: 1`）**：
+```
+det_path/
+└── case_019b178d/
+    └── json_results/   ← det_format="json"
+        ├── 000000.json
+        └── 000001.json
+
+gt_path/
+└── case_019b178d/
+    └── labels_json/    ← gt_format="json"
+        ├── 000000.json
+        └── 000001.json
+```
+
+### 4.2 图像尺寸 (`image`)
+
+```yaml
+image:
+  width:  1920
+  height: 1080
+```
+
+### 4.3 模型配置与 GT 过滤 (`model`)
+
+```yaml
+model:
+  input_size: 704               # 模型输入宽度（像素）
+  min_box_size_at_input_scale: 8  # 模型输入分辨率下的最小框尺寸
+  # GT 过滤阈值自动计算：min_box_size = 8 * roi_width / 704
+  # ROI0 (1920→704): ≈ 21.8 像素；ROI1 (704→704): 8 像素
+```
+
+### 4.4 并行性能 (`performance`)
+
+```yaml
+performance:
+  num_workers: 32   # 并行 worker 数（null = auto-detect CPU 核数）
+```
+
+### 4.5 ROI GT 处理 (`roi_gt`)
+
+```yaml
+roi_gt:
+  enabled: true
+  calib_root: "/data1/xdzhu/Testdata"  # 标定文件根目录（含 camera4.json）
+  roi_config: [1920, 960]              # ROI 配置，与训练保持一致
+  # ROI0: [1920, 960]
+  # ROI1: [704, 352]
+```
+
+> **重要**：`roi_config` 必须与训练时的 ROI 配置完全一致，否则坐标系不对齐，IoU 计算将不准确。
+
+### 4.6 类别定义 (`classes`)
+
+```yaml
+classes:
+  3d_classes: [0, 1, 2, 3]    # 参与 3D 评测的类别
+  2d_classes: [4, 5, ..., 13]  # 仅参与 2D 评测的类别
+  class_names:
+    0: "vehicle"
+    1: "pedestrian"
+    ...
+    13: "tricycle"
+```
+
+### 4.7 匹配参数 (`matching`)
+
+```yaml
+matching:
+  iou_threshold: 0.5   # 2D 匹配 IoU 阈值
+```
+
+### 4.8 2D 指标 (`metrics_2d`)
+
+```yaml
+metrics_2d:
+  enabled: true
+  conf_threshold: 0.4      # Precision/Recall 置信度阈值
+  ap_method: "voc2010"     # AP 计算方法：voc2010（11点插值）/ coco（全点插值）
+```
+
+### 4.9 3D 指标 (`metrics_3d`)
+
+```yaml
+metrics_3d:
+  enabled: true
+  heading_tolerance: "both"  # strict / relaxed / both
+  distance_ranges:           # 纵向距离区间（米）
+    - [0, 10]
+    - [10, 20]
+    - ...
+    - [100, 999]
+  lateral_distance_ranges:   # 横向距离区间（米，x 轴）
+    - [-50, -40]
+    - ...
+    - [40, 50]
+```
+
+**`heading_tolerance` 说明**：
+| 值 | 含义 |
+|---|---|
+| `strict` | 标准计算（默认）|
+| `relaxed` | 对称物体允许 180° 对称：`error = min(error, π - error)` |
+| `both` | 同时计算并输出 strict 和 relaxed 两套结果 |
+
+### 4.10 输出配置 (`output`)
+
+```yaml
+output:
+  save_path: "evaluation_results/my_model/{timestamp}"
+  formats: ["json", "txt"]
+  print_details: true
+  per_case_reports: true
+```
+
+---
+
+## 5. 命令行参数说明
+
+`eval_tools/core/eval.py` 支持以下参数（命令行参数优先级高于配置文件）：
+
+| 参数 | 类型 | 默认值 | 说明 |
+|------|------|--------|------|
+| `--config` | str | — | YAML 配置文件路径 |
+| `--det-path` | str | — | 检测结果根目录 |
+| `--gt-path` | str | — | 真值标签根目录 |
+| `--path-depth` | int | 1 | 目录层级（1 或 2）|
+| `--det-format` | str | auto | 检测结果格式（auto/json/txt）|
+| `--gt-format` | str | auto | 真值格式（auto/json/txt）|
+| `--output-dir` | str | eval_results | 输出目录 |
+| `--img-width` | int | 1920 | 图像宽度（像素）|
+| `--img-height` | int | 1080 | 图像高度（像素）|
+| `--iou-threshold` | float | 0.5 | IoU 匹配阈值 |
+| `--conf-threshold` | float | 0.5 | 置信度阈值 |
+| `--ap-method` | str | voc2010 | AP 计算方法（voc2010/coco）|
+| `--heading-tolerance` | str | strict | heading 容忍模式（strict/relaxed/both）|
+| `--num-workers` | int | auto | 并行 worker 数 |
+| `--eval-2d-only` | flag | — | 仅评测 2D |
+| `--eval-3d-only` | flag | — | 仅评测 3D |
+| `--save-detailed-matches` | flag | — | 保存详细 3D 匹配信息（供模型对比使用）|
+
+---
+
+## 6. 数据格式
+
+### 6.1 真值（Ground Truth）格式
+
+**车辆类别（50 个值）**：
+```
+[class, x, y, w, h,                          # 0-4: 类别 + 归一化 2D 框
+ x3d, y3d, z3d,                              # 5-7: 3D 中心点（相机坐标系）
+ l, h, w,                                   # 8-10: 3D 尺寸（长高宽）
+ rot_y,                                      # 11: 绕 Y 轴旋转角
+ xc, yc,                                     # 12-13: 3D 中心点的图像投影
+ xc_d, yc_d,                                 # 14-15: 深度相关投影
+ alpha,                                      # 16: 观测角
+ 0,                                          # 17: 占位符
+ 前面(x3d,y3d,z3d,alpha,xc,yc,score,occ),    # 18-25
+ 后面(x3d,y3d,z3d,alpha,xc,yc,score,occ),    # 26-33
+ 左面(x3d,y3d,z3d,alpha,xc,yc,score,occ),    # 34-41
+ 右面(x3d,y3d,z3d,alpha,xc,yc,score,occ)]    # 42-49
+```
+
+**行人/自行车/骑手（18 个值）**：
+上述 0-17，无四面信息。
+
+**仅 2D 标注（6 个值）**：
+```
+[class, x, y, w, h, -1]
+```
+
+### 6.2 检测结果格式
+
+**车辆（15 个字段）**：
+```
+vehicle  0.95  368.08 574.17 437.89 617.20  cam  -30.14 1.43 68.55  5.52 2.50 2.31  2.70  left
+label    conf  x1     y1     x2     y2       cs   x3d    y3d  z3d    l    h    w     rot_y face_type
+```
+`face_type` 取值：`front` / `back`（或 `rear`/`tail`）/ `left` / `right`
+
+**行人/骑手等（15 个字段）**：同上，`face_type` 固定为 `whole`。
+
+**纯 2D 类别（6 个字段）**：
+```
+plate  0.94246  532.12 203.26 558.73 214.86
+label  conf     x1     y1     x2     y2
+```
+
+---
+
+## 7. 评测指标
+
+### 7.1 2D 检测指标
+
+- **Precision**：TP / (TP + FP)，按 `conf_threshold` 过滤
+- **Recall**：TP / (TP + FN)
+- **F1-Score**：2 × P × R / (P + R)
+- **AP**：PR 曲线下面积（IoU 阈值 0.5）
+- **mAP**：所有类别 AP 的算术平均
+
+**匹配规则**：
+1. 按置信度从高到低排序预测框
+2. 与真值框 IoU ≥ 0.5 且类别相同 → TP（每个真值框只匹配一次）
+3. 未匹配预测框 → FP；未被匹配真值框 → FN
+
+**类别划分（14 类）**：
+
+| ID | 类别 | 3D 评测 |
+|----|------|---------|
+| 0 | vehicle | ✓ |
+| 1 | pedestrian | ✓ |
+| 2 | bicycle | ✓ |
+| 3 | rider | ✓ |
+| 4 | roadblock | — |
+| 5 | head | — |
+| 6 | tsr | — |
+| 7 | guideboard | — |
+| 8 | plate | — |
+| 9 | wheel | — |
+| 10 | tl_border | — |
+| 11 | tl_wick | — |
+| 12 | tl_num | — |
+| 13 | tricycle | — |
+
+### 7.2 3D 检测指标
+
+> 前提：2D IoU 匹配成功（≥ 0.5）且真值包含完整 3D 标注。
+
+**车辆类别**（基于最近面中心点对比）：
+
+根据检测结果的 `face_type` 字段，在真值中选取对应面的中心点进行比较：
+```
+face_type → 真值面中心点索引
+  front  →  [18, 19, 20]  (x3d, y3d, z3d)
+  back   →  [26, 27, 28]
+  left   →  [34, 35, 36]
+  right  →  [42, 43, 44]
+```
+
+**行人/骑手/自行车**：使用整体中心点 `[5, 6, 7]` 进行比较。
+
+**误差指标**：
+- **横向误差（Lateral Error）**：`|x3d_pred - x3d_gt|`（米）
+- **纵向误差（Longitudinal Error）**：`|z3d_pred - z3d_gt|`（米）
+- **Heading 误差**：`|rot_y_pred - rot_y_gt|`（弧度）
+
+每个指标输出：均值（Mean）、中位数（Median）、标准差（Std）、90 百分位数（P90）。
+
+---
+
+## 8. ROI GT 处理
+
+### 动机
+
+训练时图像经过 ROI 裁剪和缩放（坐标系改变），但历史评测代码中检测结果在 ROI 坐标系，GT 在原图坐标系，导致 IoU 计算不准确。
+
+### 解决方案
+
+`ROIProcessor`（`eval_tools/evaluator/roi_processor.py`）在评测时对 GT 标签执行与训练完全相同的 ROI 处理：
+
+1. 读取标定文件（`case_root/camera4.json`）中的 `focal_u, focal_v, cu, cv, pitch`
+2. 计算裁剪中心：`cx = oriW // 2`，`cy = cv - focal_v × tan(pitch × π/180)`
+3. 以 `(cx, cy)` 为中心按 `roi_config` 裁剪 ROI
+4. 过滤完全在 ROI 外的目标；将部分在 ROI 内的框 clip 到边界
+5. 将坐标转换到 ROI 相对坐标系 → 再缩放到模型输入分辨率
+
+**配置对应关系**：
+| 训练 ROI 配置 | `roi_config` 值 |
+|---|---|
+| ROI0（大 crop） | `[1920, 960]` |
+| ROI1（小 crop） | `[704, 352]` |
+
+---
+
+## 9. 模型对比（共同匹配集）
+
+### 问题背景
+
+直接比较两个模型的 3D 指标时，若双方匹配到的 GT 对象数量不同（如相差 11.7%），差异可能来自**匹配集差异**而非模型质量。
+
+### 解决方案
+
+只统计两个模型**都成功匹配**的 GT 对象（Common Matches）的 3D 指标。
+
+### 使用方式
+
+#### 方式一：一键脚本（推荐）
+
+```bash
+bash eval_tools/model_comparison/compare_models_with_common_matches.sh
+```
+
+该脚本自动完成：
+1. 评测 Model 1，保存 `detailed_3d_matches.json`
+2. 评测 Model 2，保存 `detailed_3d_matches.json`
+3. 调用 `find_common_matches.py` 得到 `common_matches.json`
+4. 调用 `compare_models.py` 生成对比报告
+5. 调用 `generate_eval_report.py` 为各模型生成 Markdown 报告
+
+#### 方式二：手动分步执行
+
+```bash
+# Step 1: 评测两个模型
+python eval_tools/core/eval.py \
+  --config eval_tools/configs/eval_config_mono3d-roi1.yaml \
+  --output-dir eval_results/model1 \
+  --heading-tolerance both \
+  --save-detailed-matches
+
+python eval_tools/core/eval.py \
+  --config eval_tools/configs/eval_config_yolov5s_cncap-roi1.yaml \
+  --output-dir eval_results/model2 \
+  --heading-tolerance both \
+  --save-detailed-matches
+
+# Step 2: 找出共同匹配
+python eval_tools/model_comparison/find_common_matches.py \
+  --model1-matches eval_results/model1/detailed_3d_matches.json \
+  --model2-matches eval_results/model2/detailed_3d_matches.json \
+  --output eval_results/common_matches.json \
+  --model1-name "mono3d" \
+  --model2-name "yolov5s-300w"
+
+# Step 3: 生成对比报告
+python eval_tools/model_comparison/compare_models.py \
+  --model1 eval_results/model1/evaluation_report.json \
+  --model2 eval_results/model2/evaluation_report.json \
+  --common-matches eval_results/common_matches.json \
+  --output-dir eval_results/comparison \
+  --model1-name "mono3d" \
+  --model2-name "yolov5s-300w"
+```
+
+### 生成 Markdown 报告
+
+```bash
+python eval_tools/model_comparison/generate_eval_report.py \
+  eval_results/model1/evaluation_report.json \
+  --model "yolov5s-300w-cncap" \
+  --date 2026-03-03
+```
+
+输出：`eval_results/model1/EVALUATION_REPORT.md`
+
+---
+
+## 10. Heading 误差分析
+
+对于 heading 预测较差的案例，可使用以下工具进行深入分析：
+
+### Step 1：提取 bad cases
+
+```bash
+python eval_tools/heading_analysis/extract_bad_heading_cases.py \
+  --input eval_results/model1/detailed_3d_matches.json \
+  --threshold 1.5 \      # heading 误差阈值（弧度）
+  --top-k 100 \
+  --output bad_heading.json \
+  --stats
+```
+
+### Step 2：可视化
+
+```bash
+python eval_tools/heading_analysis/visualize_heading_errors.py \
+  --input bad_heading.json \
+  --image-root /path/to/images \
+  --output runs/heading_viz \
+  --viz-types bev image angle combined
+```
+
+可视化类型：
+- `bev`：鸟瞰图（GT / Pred 的 3D 框 + 方向箭头）
+- `image`：原图视图（2D 框 + 误差标注）
+- `angle`：角度分析（极坐标 + 柱状图）
+- `combined`：四宫格综合视图
+
+---
+
+## 11. 输出结果
+
+### 11.1 文件列表
+
+| 文件 | 说明 |
+|------|------|
+| `evaluation_report.json` | 完整评测报告（JSON 格式，含所有类别和距离区间）|
+| `evaluation_report.txt` | 人类可读的文本报告 |
+| `detailed_3d_matches.json` | 每帧每目标的 3D 匹配详情（`--save-detailed-matches`）|
+| `per_case_reports/*.json` | 每个 case 的独立报告（`per_case_reports: true`）|
+| `EVALUATION_REPORT.md` | Markdown 格式摘要报告（由 `generate_eval_report.py` 生成）|
+
+### 11.2 JSON 报告结构
+
+```json
+{
+  "evaluation_config": {...},
+  "2d_evaluation": {
+    "overall": {
+      "precision": 0.85,
+      "recall": 0.78,
+      "f1_score": 0.81,
+      "map": 0.72
+    },
+    "per_class": {
+      "vehicle": {"precision": ..., "recall": ..., "ap": ...},
+      ...
+    }
+  },
+  "3d_evaluation": {
+    "vehicle": {
+      "overall": {
+        "n_samples": 440408,
+        "lateral_mean": 0.647,
+        "longitudinal_mean": 1.680,
+        "heading_mean_strict": 0.258
+      },
+      "by_distance": {
+        "[0-10m]": {...},
+        ...
+      }
+    }
+  }
+}
+```
+
+### 11.3 控制台输出示例
+
+```
+3D Metrics:
+  vehicle [overall]: Lat=0.647m, Long=1.680m, Head=0.258rad (n=440408)
+    [0-10m]:   Lat=0.423m, Long=0.891m, Head=0.153rad (n=98241)
+    [10-20m]:  Lat=0.501m, Long=1.203m, Head=0.177rad (n=142038)
+    [30-60m]:  Lat=1.149m, Long=5.074m, Head=0.607rad (n=18432)
+    [100-999m]:Lat=2.341m, Long=9.221m, Head=1.036rad (n=871)
+```
+
+---
+
+## 12. 使用示例
+
+### 例 1：单模型评测（当前默认入口）
+
+```bash
+cd /deeplearning_team/ydong/dongying/projects/yolov5-3d
+conda activate slot
+
+bash eval_tools/core/run_eval_with_config.sh
+```
+
+脚本使用 `eval_tools/configs/eval_config_yolov5s_cncap-roi0.yaml`，结果保存至：
+```
+evaluation_results/eval_results_common_match_comparison_cncap_yolov5s_20260303_roi0/
+└── yolov5s-300w-newdata-cncap-test/
+    └── 20260303_HHMMSS/
+        ├── evaluation_report.json
+        ├── evaluation_report.txt
+        └── detailed_3d_matches.json
+```
+
+### 例 2：命令行直接评测（不使用配置文件）
+
+```bash
+python eval_tools/core/eval.py \
+  --det-path /data1/.../inference_results/model/evalset_roi0 \
+  --gt-path  /data1/xdzhu/Testdata \
+  --output-dir evaluation_results/my_test \
+  --heading-tolerance both \
+  --conf-threshold 0.4 \
+  --num-workers 16
+```
+
+### 例 3：仅 2D 评测
+
+```bash
+python eval_tools/core/eval.py \
+  --config eval_tools/configs/eval_config_yolov5s_cncap-roi0.yaml \
+  --eval-2d-only \
+  --output-dir evaluation_results/2d_only_test
+```
+
+### 例 4：双模型对比
+
+```bash
+bash eval_tools/model_comparison/compare_models_with_common_matches.sh
+```
+
+### 例 5：修改配置直接运行
+
+仅修改 `run_eval_with_config.sh` 中的以下三个变量即可适配新模型：
+
+```bash
+MODEL_CONFIG="eval_tools/configs/eval_config_yolov5s_cncap-roi0.yaml"
+OUTPUT_BASE="evaluation_results/eval_results_my_model"
+MODEL_NAME="my-model-name"
+```
+
+---
+
+## 13. 常见问题
+
+### Q1：No image pairs found
+
+**原因**：`det_path` 与 `gt_path` 下的 case 名称不匹配，或 `det_format` / `gt_format` 设置错误。
+
+**排查**：
+```bash
+ls ${det_path}/        # 查看 case 目录名
+ls ${det_path}/case_xxx/  # 查看是 json_results/ 还是 txt_results/
+```
+
+### Q2：IoU 始终为 0，评测指标异常
+
+**原因**：`roi_gt` 配置未启用，或 `roi_config` 与训练时不匹配，导致坐标系不对齐。
+
+**排查**：检查配置中 `roi_gt.enabled: true` 且 `roi_config` 与训练 YAML 中的 `roi:` 字段一致。
+
+### Q3：3D 指标样本数 (n=0)
+
+**原因**：真值中该类别无完整 3D 标注（仅有 6 维的 2D-only 标注），或 2D 匹配 IoU < 0.5。
+
+### Q4：如何添加新模型配置
+
+复制现有配置文件，仅修改 `dataset.det_path` 和 `dataset.gt_path` 即可：
+```bash
+cp eval_tools/configs/eval_config_yolov5s_cncap-roi0.yaml \
+   eval_tools/configs/eval_config_my_model-roi0.yaml
+# 编辑新文件中的 det_path 和 output.save_path
+```
+
+### Q5：如何加速评测
+
+调大 `performance.num_workers`（建议不超过 CPU 核数 × 2）：
+```bash
+python eval_tools/core/eval.py --config ... --num-workers 64
+```
+
+---
+
+*文档生成日期：2026-03-03*

eval_tools/docs/EXTRACT_BAD_HEADING_USAGE.md

@@ -0,0 +1,560 @@
+# Bad Heading Cases Extraction Tool 使用说明
+
+## 工具概述
+
+`extract_bad_heading_cases.py` 用于从评估结果中提取 heading 误差较大的案例，为后续可视化分析提供数据支持。
+
+## 功能特性
+
+- ✅ **多条件筛选**：支持误差阈值、类别、距离、置信度等多维度过滤
+- ✅ **反转错误检测**：自动识别 ≈180° 的方向反转错误
+- ✅ **统计分析**：输出详细的分类统计、误差分布、距离分布等
+- ✅ **Top-K 选择**：支持仅提取误差最大的 K 个案例
+- ✅ **结构化输出**：生成包含元数据和统计信息的 JSON 文件
+
+## 快速开始
+
+### 基本用法
+
+```bash
+# 提取所有误差 > 1.5 rad 的案例
+python eval_tools/extract_bad_heading_cases.py \
+    --input eval_results_common_match_comparison/yolov5s-300w/20260203_210259/detailed_3d_matches.json \
+    --threshold 1.5 \
+    --output bad_heading_cases.json \
+    --stats
+```
+
+### 运行测试脚本
+
+```bash
+# 给脚本添加执行权限
+chmod +x eval_tools/test_extract_bad_cases.sh
+
+# 运行多个测试用例
+./eval_tools/test_extract_bad_cases.sh
+```
+
+## 命令行参数
+
+### 必需参数
+
+- `--input`: 输入的 detailed_3d_matches.json 文件路径
+
+### 可选参数
+
+#### 筛选参数
+
+- `--threshold`: Heading 误差阈值（弧度），默认 1.5 (≈ 85°)
+- `--top-k`: 仅提取误差最大的 K 个案例
+- `--classes`: 指定类别（可选多个），支持：
+  - `vehicle`: 车辆
+  - `pedestrian`: 行人
+  - `bicycle`: 自行车
+  - `rider`: 骑行者
+- `--min-distance`: 最小距离（米）
+- `--max-distance`: 最大距离（米）
+- `--min-confidence`: 最小置信度 (0-1)
+- `--reversal-only`: 仅提取反转错误（误差 > π - 0.1 ≈ 174°）
+
+#### 输出参数
+
+- `--output`: 输出 JSON 文件路径，默认：`bad_heading_cases.json`
+- `--stats`: 打印详细统计信息
+
+## 使用示例
+
+### 示例 1：提取所有 bad cases
+
+```bash
+python eval_tools/extract_bad_heading_cases.py \
+    --input detailed_3d_matches.json \
+    --threshold 1.5 \
+    --output all_bad_cases.json \
+    --stats
+```
+
+**输出统计示例**：
+```
+Processing: 596602 matches...
+Filtered: 5839 cases
+
+=== Statistics ===
+vehicle: 4012 cases (68.7% reversal)
+pedestrian: 1523 cases (42.3% reversal)
+bicycle: 304 cases (55.6% reversal)
+
+Error distribution:
+  Mean: 2.45 rad (140.4°)
+  Median: 2.89 rad (165.6°)
+  Std: 0.78 rad
+  Range: [1.50, 3.14] rad
+```
+
+### 示例 2：提取 Top 50 最差案例
+
+```bash
+python eval_tools/extract_bad_heading_cases.py \
+    --input detailed_3d_matches.json \
+    --threshold 1.5 \
+    --top-k 50 \
+    --output top50_worst.json \
+    --stats
+```
+
+**适用场景**：
+- 聚焦最严重的问题案例
+- 快速定位典型错误模式
+- 限制可视化数量以提高效率
+
+### 示例 3：仅提取反转错误
+
+```bash
+python eval_tools/extract_bad_heading_cases.py \
+    --input detailed_3d_matches.json \
+    --reversal-only \
+    --top-k 100 \
+    --output reversal_errors.json \
+    --stats
+```
+
+**适用场景**：
+- 专门分析 180° 方向反转问题
+- yolov5s-300w 模型的主要问题（68.7% bad cases 是反转错误）
+
+### 示例 4：分析特定类别
+
+```bash
+# 仅分析车辆类别
+python eval_tools/extract_bad_heading_cases.py \
+    --input detailed_3d_matches.json \
+    --threshold 1.5 \
+    --classes vehicle \
+    --min-confidence 0.5 \
+    --top-k 50 \
+    --output vehicle_bad_cases.json \
+    --stats
+```
+
+```bash
+# 分析多个类别
+python eval_tools/extract_bad_heading_cases.py \
+    --input detailed_3d_matches.json \
+    --classes vehicle pedestrian \
+    --output vehicle_pedestrian_cases.json \
+    --stats
+```
+
+### 示例 5：距离筛选
+
+```bash
+# 提取中距离范围的 bad cases (30-80m)
+python eval_tools/extract_bad_heading_cases.py \
+    --input detailed_3d_matches.json \
+    --threshold 1.5 \
+    --min-distance 30 \
+    --max-distance 80 \
+    --output mid_distance_cases.json \
+    --stats
+```
+
+```bash
+# 提取近距离 bad cases (< 30m)
+python eval_tools/extract_bad_heading_cases.py \
+    --input detailed_3d_matches.json \
+    --threshold 1.5 \
+    --max-distance 30 \
+    --output near_distance_cases.json \
+    --stats
+```
+
+### 示例 6：高置信度案例
+
+```bash
+# 提取高置信度但方向错误的案例（模型问题而非检测问题）
+python eval_tools/extract_bad_heading_cases.py \
+    --input detailed_3d_matches.json \
+    --threshold 1.5 \
+    --min-confidence 0.7 \
+    --top-k 50 \
+    --output high_confidence_bad.json \
+    --stats
+```
+
+## 输出格式
+
+### JSON 结构
+
+```json
+{
+  "metadata": {
+    "source": "detailed_3d_matches.json",
+    "threshold": 1.5,
+    "total_processed": 596602,
+    "total_extracted": 5839,
+    "filters": {
+      "classes": ["vehicle", "pedestrian"],
+      "min_distance": 30,
+      "max_distance": 80,
+      "min_confidence": 0.5,
+      "reversal_only": false,
+      "top_k": 100
+    },
+    "extraction_time": "2026-02-04T10:30:15"
+  },
+  
+  "statistics": {
+    "vehicle": {
+      "count": 4012,
+      "reversal_count": 3266,
+      "reversal_percentage": 81.4,
+      "avg_error": 2.85,
+      "avg_error_deg": 163.3,
+      "error_distribution": {
+        "min": 1.50,
+        "max": 3.14,
+        "mean": 2.85,
+        "median": 3.02,
+        "std": 0.45
+      },
+      "distance_distribution": {
+        "min": 5.2,
+        "max": 98.7,
+        "mean": 45.3,
+        "median": 42.1
+      },
+      "confidence_distribution": {
+        "min": 0.25,
+        "max": 0.98,
+        "mean": 0.67
+      }
+    },
+    "pedestrian": {
+      "count": 1523,
+      "reversal_count": 644,
+      "reversal_percentage": 42.3,
+      "avg_error": 2.34,
+      "avg_error_deg": 134.1,
+      ...
+    }
+  },
+  
+  "cases": [
+    {
+      "case_id": "019b18ef-ca2b-7e97-8ca5-d4b1d7eefdbb_roi0",
+      "frame_id": "019b18f3-8d8e-70bb-8b4e-cef51fbb7089.jpg",
+      "class": "vehicle",
+      
+      "heading_error": 3.1389,
+      "heading_error_deg": 179.87,
+      "gt_rotation": -0.0145,
+      "det_rotation": 3.1244,
+      "is_reversal": true,
+      
+      "distance": 45.23,
+      "confidence": 0.8567,
+      "iou": 0.7234,
+      
+      "gt_center": [42.5, -3.2, 1.5],
+      "det_center": [42.8, -3.4, 1.6],
+      
+      "gt_dimensions": [4.5, 1.8, 1.6],
+      "det_dimensions": [4.3, 1.7, 1.5],
+      
+      "gt_bbox_2d": [120, 340, 245, 480],
+      "det_bbox_2d": [118, 338, 243, 478],
+      
+      "lateral_error": 0.22,
+      "longitudinal_error": 0.31
+    },
+    ...
+  ]
+}
+```
+
+### 字段说明
+
+#### metadata 元数据
+- `source`: 输入文件路径
+- `threshold`: 使用的误差阈值
+- `total_processed`: 处理的总匹配数
+- `total_extracted`: 提取的案例数
+- `filters`: 应用的筛选条件
+- `extraction_time`: 提取时间
+
+#### statistics 统计信息
+- `count`: 该类别的案例数量
+- `reversal_count`: 反转错误数量
+- `reversal_percentage`: 反转错误比例
+- `avg_error`: 平均误差（弧度/度）
+- `error_distribution`: 误差分布统计
+- `distance_distribution`: 距离分布统计
+- `confidence_distribution`: 置信度分布统计
+
+#### cases 案例详情
+- `case_id`, `frame_id`: 案例和帧 ID
+- `class`: 目标类别
+- `heading_error`: Heading 误差（弧度）
+- `heading_error_deg`: Heading 误差（度）
+- `gt_rotation`, `det_rotation`: 真值和检测的旋转角度
+- `is_reversal`: 是否为反转错误
+- `distance`: 目标距离（米）
+- `confidence`: 检测置信度
+- `iou`: 2D IoU
+- `gt_center`, `det_center`: 3D 中心坐标
+- `gt_dimensions`, `det_dimensions`: 3D 尺寸
+- `gt_bbox_2d`, `det_bbox_2d`: 2D 边界框
+- `lateral_error`, `longitudinal_error`: 横向和纵向误差
+
+## 数据分析工作流
+
+### 1. 初步探索
+
+```bash
+# 查看所有 bad cases 的统计信息
+python eval_tools/extract_bad_heading_cases.py \
+    --input detailed_3d_matches.json \
+    --threshold 1.5 \
+    --output all_cases.json \
+    --stats
+```
+
+### 2. 聚焦问题
+
+```bash
+# 基于统计结果，聚焦特定问题（如反转错误）
+python eval_tools/extract_bad_heading_cases.py \
+    --input detailed_3d_matches.json \
+    --reversal-only \
+    --classes vehicle \
+    --top-k 100 \
+    --output vehicle_reversal.json \
+    --stats
+```
+
+### 3. 可视化分析
+
+```bash
+# 使用提取的案例进行可视化（Tool 2）
+python eval_tools/visualize_heading_errors.py \
+    --input vehicle_reversal.json \
+    --gt-dir path/to/labels \
+    --img-dir path/to/images \
+    --calib-dir path/to/calib \
+    --output runs/heading_viz
+```
+
+### 4. 生成报告
+
+```bash
+# 生成分析报告（Tool 4）
+python eval_tools/generate_heading_report.py \
+    --input vehicle_reversal.json \
+    --viz-dir runs/heading_viz \
+    --output heading_analysis_report.html
+```
+
+## 典型分析场景
+
+### 场景 1：定位反转错误根因
+
+**目标**：理解为什么 yolov5s-300w 有 68.7% 的反转错误
+
+```bash
+# Step 1: 提取反转错误
+python eval_tools/extract_bad_heading_cases.py \
+    --input detailed_3d_matches.json \
+    --reversal-only \
+    --top-k 200 \
+    --output reversal_errors.json \
+    --stats
+
+# Step 2: 按类别分析
+python eval_tools/extract_bad_heading_cases.py \
+    --input detailed_3d_matches.json \
+    --reversal-only \
+    --classes vehicle \
+    --output vehicle_reversal.json \
+    --stats
+
+# Step 3: 按距离分析
+python eval_tools/extract_bad_heading_cases.py \
+    --input detailed_3d_matches.json \
+    --reversal-only \
+    --min-distance 30 \
+    --max-distance 60 \
+    --output mid_dist_reversal.json \
+    --stats
+```
+
+### 场景 2：分析高置信度错误
+
+**目标**：找出模型"很确信但预测错误"的案例
+
+```bash
+python eval_tools/extract_bad_heading_cases.py \
+    --input detailed_3d_matches.json \
+    --threshold 2.0 \
+    --min-confidence 0.8 \
+    --top-k 50 \
+    --output high_conf_bad.json \
+    --stats
+```
+
+### 场景 3：类别间差异分析
+
+**目标**：对比不同类别的 heading 预测问题
+
+```bash
+# 车辆
+python eval_tools/extract_bad_heading_cases.py \
+    --input detailed_3d_matches.json \
+    --classes vehicle \
+    --top-k 100 \
+    --output vehicle_analysis.json \
+    --stats
+
+# 行人
+python eval_tools/extract_bad_heading_cases.py \
+    --input detailed_3d_matches.json \
+    --classes pedestrian \
+    --top-k 100 \
+    --output pedestrian_analysis.json \
+    --stats
+```
+
+### 场景 4：距离敏感性分析
+
+**目标**：分析 heading 误差与距离的关系
+
+```bash
+# 近距离 (< 30m)
+python eval_tools/extract_bad_heading_cases.py \
+    --input detailed_3d_matches.json \
+    --max-distance 30 \
+    --output near_cases.json \
+    --stats
+
+# 中距离 (30-60m)
+python eval_tools/extract_bad_heading_cases.py \
+    --input detailed_3d_matches.json \
+    --min-distance 30 \
+    --max-distance 60 \
+    --output mid_cases.json \
+    --stats
+
+# 远距离 (> 60m)
+python eval_tools/extract_bad_heading_cases.py \
+    --input detailed_3d_matches.json \
+    --min-distance 60 \
+    --output far_cases.json \
+    --stats
+```
+
+## 常见问题
+
+### Q1: 如何选择合适的阈值？
+
+**A**: 根据评估指标选择：
+- `1.0 rad (57°)`: 严格标准，仅提取严重错误
+- `1.5 rad (86°)`: 推荐标准，平衡数量和质量
+- `2.0 rad (115°)`: 宽松标准，包含更多一般错误
+
+### Q2: 提取的案例太多怎么办？
+
+**A**: 使用 `--top-k` 限制数量，或添加更多筛选条件：
+
+```bash
+python eval_tools/extract_bad_heading_cases.py \
+    --input detailed_3d_matches.json \
+    --threshold 1.5 \
+    --top-k 100 \
+    --min-confidence 0.5 \
+    --min-distance 20 \
+    --max-distance 80
+```
+
+### Q3: 如何只看反转错误？
+
+**A**: 使用 `--reversal-only` 参数：
+
+```bash
+python eval_tools/extract_bad_heading_cases.py \
+    --input detailed_3d_matches.json \
+    --reversal-only \
+    --top-k 100
+```
+
+### Q4: 如何验证提取的案例是否正确？
+
+**A**: 使用 `--stats` 查看统计信息，并检查输出 JSON 的前几个案例：
+
+```bash
+python eval_tools/extract_bad_heading_cases.py \
+    --input detailed_3d_matches.json \
+    --top-k 5 \
+    --stats
+
+# 然后查看输出文件
+python -m json.tool bad_heading_cases.json | head -n 100
+```
+
+## 性能优化
+
+### 大数据集处理
+
+对于非常大的数据集（如 596K+ 匹配），可以：
+
+1. **先筛选再提取**：
+```bash
+# 先用严格条件提取少量案例
+python eval_tools/extract_bad_heading_cases.py \
+    --input detailed_3d_matches.json \
+    --threshold 2.5 \
+    --top-k 50
+```
+
+2. **分批处理**：
+```bash
+# 按类别分批
+for class in vehicle pedestrian bicycle rider; do
+    python eval_tools/extract_bad_heading_cases.py \
+        --input detailed_3d_matches.json \
+        --classes $class \
+        --top-k 100 \
+        --output ${class}_bad_cases.json
+done
+```
+
+## 下一步
+
+提取完案例后，可以使用：
+
+1. **可视化工具** (Tool 2): `visualize_heading_errors.py`
+   - 生成 BEV、图像投影、角度分析图
+
+2. **交互式查看器** (Tool 3): `heading_error_viewer.py`
+   - Web 界面浏览案例
+
+3. **报告生成器** (Tool 4): `generate_heading_report.py`
+   - 生成 HTML/Markdown 分析报告
+
+## 相关文档
+
+- [Heading Error Visualization Plan](HEADING_ERROR_VISUALIZATION_PLAN.md)
+- [GT Visualization Guide](../test_scripts/GT_VISUALIZATION_COMPLETE_GUIDE.md)
+- [Metrics Summary](../eval_results_common_match_comparison_v1/.../METRICS_SUMMARY.md)
+
+## 技术支持
+
+如果遇到问题，请检查：
+
+1. 输入文件路径是否正确
+2. JSON 格式是否符合预期（包含 case_id → frames → classes → matches 结构）
+3. 筛选条件是否过于严格（导致无结果）
+4. Python 环境是否安装必要的库（numpy）
+
+```bash
+# 检查输入文件结构
+python -c "import json; data = json.load(open('detailed_3d_matches.json')); print(f'Cases: {len(data)}, First case: {list(data.keys())[0]}')"
+```

eval_tools/docs/HEADING_ERROR_TOOLKIT_README.md

@@ -0,0 +1,455 @@
+# Heading Error Analysis Toolkit
+
+Heading 误差分析工具链，用于提取、可视化和分析 3D 目标检测中的方向预测错误。
+
+## 工具概览
+
+本工具链包含 4 个核心工具：
+
+### ✅ Tool 1: 数据提取工具
+**文件**: `extract_bad_heading_cases.py`
+
+从评估结果中提取 heading 误差大的案例，支持多维度筛选。
+
+```bash
+python eval_tools/extract_bad_heading_cases.py \
+    --input detailed_3d_matches.json \
+    --threshold 1.5 \
+    --top-k 100 \
+    --output bad_heading_cases.json \
+    --stats
+```
+
+**功能**：
+- 多条件筛选（阈值、类别、距离、置信度）
+- 反转错误检测（≈180°）
+- 详细统计分析
+- 结构化 JSON 输出
+
+📖 [详细文档](EXTRACT_BAD_HEADING_USAGE.md)
+
+---
+
+### ✅ Tool 2: 可视化生成工具
+**文件**: `visualize_heading_errors.py`
+
+生成 BEV、图像投影、角度分析和综合视图。
+
+```bash
+python eval_tools/visualize_heading_errors.py \
+    --input bad_heading_cases.json \
+    --image-root /path/to/images \
+    --output runs/heading_viz \
+    --viz-types bev image angle combined
+```
+
+**功能**：
+- BEV 鸟瞰图（GT/Pred 3D框 + 方向箭头）
+- 图像视图（2D框 + 误差标注）
+- 角度分析（极坐标 + 柱状图）
+- 综合视图（四宫格 + 详细统计）
+
+📖 [详细文档](VISUALIZE_HEADING_USAGE.md)
+
+---
+
+### 🚧 Tool 3: 交互式查看器 (待实现)
+**文件**: `heading_error_viewer.py`
+
+Web 界面交互式浏览和分析案例。
+
+```bash
+python eval_tools/heading_error_viewer.py \
+    --input bad_heading_cases.json \
+    --viz-dir runs/heading_viz \
+    --port 8080
+```
+
+**计划功能**：
+- Web UI 浏览所有案例
+- 过滤和排序功能
+- 并排对比多个案例
+- 统计图表展示
+
+---
+
+### 🚧 Tool 4: 报告生成器 (待实现)
+**文件**: `generate_heading_report.py`
+
+自动生成 HTML/Markdown 分析报告。
+
+```bash
+python eval_tools/generate_heading_report.py \
+    --input bad_heading_cases.json \
+    --viz-dir runs/heading_viz \
+    --output heading_analysis_report.html
+```
+
+**计划功能**：
+- 自动生成分析报告
+- 统计图表和趋势分析
+- 典型案例展示
+- 问题总结和建议
+
+---
+
+## 完整工作流
+
+### 1. 数据提取阶段
+
+```bash
+# 提取所有 bad cases
+python eval_tools/extract_bad_heading_cases.py \
+    --input eval_results_common_match_comparison/yolov5s-300w/20260203_210259/detailed_3d_matches.json \
+    --threshold 1.5 \
+    --output all_bad_cases.json \
+    --stats
+
+# 或者只提取反转错误（针对 yolov5s-300w 的主要问题）
+python eval_tools/extract_bad_heading_cases.py \
+    --input detailed_3d_matches.json \
+    --reversal-only \
+    --top-k 100 \
+    --output reversal_errors.json \
+    --stats
+```
+
+### 2. 可视化生成阶段
+
+```bash
+# 生成所有类型的可视化
+python eval_tools/visualize_heading_errors.py \
+    --input reversal_errors.json \
+    --image-root /path/to/eval/cases \
+    --output runs/reversal_analysis \
+    --viz-types bev image angle combined
+
+# 或者只生成关键视图（节省时间）
+python eval_tools/visualize_heading_errors.py \
+    --input reversal_errors.json \
+    --image-root /path/to/eval/cases \
+    --output runs/reversal_analysis \
+    --viz-types combined \
+    --max-cases 50
+```
+
+### 3. 交互式查看阶段 (待实现)
+
+```bash
+# 启动 Web 查看器
+python eval_tools/heading_error_viewer.py \
+    --input reversal_errors.json \
+    --viz-dir runs/reversal_analysis \
+    --port 8080
+
+# 浏览器访问 http://localhost:8080
+```
+
+### 4. 报告生成阶段 (待实现)
+
+```bash
+# 生成分析报告
+python eval_tools/generate_heading_report.py \
+    --input reversal_errors.json \
+    --viz-dir runs/reversal_analysis \
+    --output reports/heading_analysis_report.html
+
+# 查看报告
+open reports/heading_analysis_report.html
+```
+
+---
+
+## 典型使用场景
+
+### 场景 1: 快速定位严重问题
+
+```bash
+# 1. 提取 top 20 worst cases
+python eval_tools/extract_bad_heading_cases.py \
+    --input detailed_3d_matches.json \
+    --threshold 2.0 \
+    --top-k 20 \
+    --output top20_worst.json
+
+# 2. 生成综合视图
+python eval_tools/visualize_heading_errors.py \
+    --input top20_worst.json \
+    --image-root /data/cases \
+    --output runs/top20_analysis \
+    --viz-types combined
+
+# 3. 查看 runs/top20_analysis/combined/ 目录
+```
+
+### 场景 2: 分析反转错误模式
+
+```bash
+# 1. 提取所有反转错误
+python eval_tools/extract_bad_heading_cases.py \
+    --input detailed_3d_matches.json \
+    --reversal-only \
+    --top-k 100 \
+    --output reversal_100.json \
+    --stats
+
+# 2. 生成 BEV 视图（最直观）
+python eval_tools/visualize_heading_errors.py \
+    --input reversal_100.json \
+    --image-root /data/cases \
+    --output runs/reversal_bev \
+    --viz-types bev
+
+# 3. 浏览 BEV 图像，寻找共同特征
+```
+
+### 场景 3: 类别对比分析
+
+```bash
+# 分别提取和可视化不同类别
+for class in vehicle pedestrian bicycle rider; do
+    python eval_tools/extract_bad_heading_cases.py \
+        --input detailed_3d_matches.json \
+        --threshold 1.5 \
+        --classes $class \
+        --top-k 30 \
+        --output ${class}_bad.json
+    
+    python eval_tools/visualize_heading_errors.py \
+        --input ${class}_bad.json \
+        --image-root /data/cases \
+        --output runs/by_class/${class} \
+        --viz-types combined
+done
+
+# 对比各类别的问题模式
+```
+
+### 场景 4: 距离敏感性分析
+
+```bash
+# 近距离 (< 30m)
+python eval_tools/extract_bad_heading_cases.py \
+    --input detailed_3d_matches.json \
+    --max-distance 30 \
+    --top-k 30 \
+    --output near_bad.json
+
+python eval_tools/visualize_heading_errors.py \
+    --input near_bad.json \
+    --image-root /data/cases \
+    --output runs/distance/near \
+    --viz-types combined
+
+# 中距离 (30-60m)
+python eval_tools/extract_bad_heading_cases.py \
+    --input detailed_3d_matches.json \
+    --min-distance 30 \
+    --max-distance 60 \
+    --top-k 30 \
+    --output mid_bad.json
+
+python eval_tools/visualize_heading_errors.py \
+    --input mid_bad.json \
+    --image-root /data/cases \
+    --output runs/distance/mid \
+    --viz-types combined
+
+# 远距离 (> 60m)
+python eval_tools/extract_bad_heading_cases.py \
+    --input detailed_3d_matches.json \
+    --min-distance 60 \
+    --top-k 30 \
+    --output far_bad.json
+
+python eval_tools/visualize_heading_errors.py \
+    --input far_bad.json \
+    --image-root /data/cases \
+    --output runs/distance/far \
+    --viz-types combined
+```
+
+---
+
+## 当前问题分析
+
+基于 yolov5s-300w 模型的评估结果：
+
+### 主要发现
+
+1. **Heading 误差严重**：
+   - 比 mono3d 高 28.61%
+   - 68.7% 的 bad cases 是反转错误（≈180°）
+
+2. **类别差异**：
+   - **Vehicle**: 87.3% 反转率（最严重）
+   - Bicycle: 46.8% 反转率
+   - Rider: 35.3% 反转率
+   - Pedestrian: 16.1% 反转率（最轻）
+
+3. **其他指标良好**：
+   - 2D mAP +21.85%
+   - 横向误差 -9.39%
+   - 纵向误差 -4.48%
+
+### 分析假设
+
+可能的根本原因：
+
+1. **模型输出范围问题**：
+   - Heading 预测范围不正确
+   - 输出范围 [0, π] vs 需要 [-π, π]
+   - 导致 180° 的系统性偏差
+
+2. **损失函数问题**：
+   - Heading loss 可能没有正确处理角度周期性
+   - 没有惩罚反转错误
+
+3. **后处理问题**：
+   - 角度转换或归一化错误
+   - NMS 或其他后处理导致的问题
+
+4. **数据或标注问题**：
+   - 训练数据中的角度分布不均
+   - 标注定义不一致
+
+---
+
+## 文件结构
+
+```
+eval_tools/
+├── extract_bad_heading_cases.py          # Tool 1: 数据提取
+├── visualize_heading_errors.py           # Tool 2: 可视化生成
+├── heading_error_viewer.py               # Tool 3: 交互查看器 (待实现)
+├── generate_heading_report.py            # Tool 4: 报告生成 (待实现)
+│
+├── test_extract_bad_cases.sh             # Tool 1 测试脚本
+├── test_visualize_heading.sh             # Tool 2 测试脚本
+│
+├── HEADING_ERROR_TOOLKIT_README.md       # 本文档
+├── EXTRACT_BAD_HEADING_USAGE.md          # Tool 1 详细文档
+├── VISUALIZE_HEADING_USAGE.md            # Tool 2 详细文档
+└── HEADING_ERROR_VISUALIZATION_PLAN.md   # 完整分析方案
+```
+
+---
+
+## 依赖安装
+
+```bash
+# 已包含在项目 requirements.txt 中
+pip install -r requirements.txt
+
+# 主要依赖：
+# - numpy
+# - opencv-python
+# - matplotlib
+# - tqdm
+# - json (标准库)
+```
+
+---
+
+## 性能建议
+
+### 大数据集处理
+
+对于大规模评估结果（如 596K 匹配）：
+
+```bash
+# 1. 先用严格阈值快速筛选
+python eval_tools/extract_bad_heading_cases.py \
+    --input detailed_3d_matches.json \
+    --threshold 2.5 \
+    --top-k 50 \
+    --output critical_cases.json
+
+# 2. 分批可视化
+python eval_tools/visualize_heading_errors.py \
+    --input critical_cases.json \
+    --image-root /data/cases \
+    --output runs/critical \
+    --max-cases 20 \
+    --viz-types combined
+
+# 3. 根据发现再扩大分析范围
+```
+
+### 存储优化
+
+```bash
+# 只生成必要的可视化类型
+python eval_tools/visualize_heading_errors.py \
+    --input bad_cases.json \
+    --image-root /data/cases \
+    --output runs/viz \
+    --viz-types bev  # 最小化存储
+
+# 降低 DPI 节省空间（预览用）
+python eval_tools/visualize_heading_errors.py \
+    --input bad_cases.json \
+    --image-root /data/cases \
+    --output runs/preview \
+    --viz-types combined \
+    --dpi 80
+```
+
+---
+
+## 下一步计划
+
+### 立即可用
+- ✅ Tool 1: 数据提取工具
+- ✅ Tool 2: 可视化生成工具
+
+### 待实现
+- 🚧 Tool 3: 交互式查看器
+  - Flask/Streamlit Web UI
+  - 实时过滤和排序
+  - 案例对比功能
+
+- 🚧 Tool 4: 报告生成器
+  - 自动化分析报告
+  - 统计图表生成
+  - 问题总结和建议
+
+### 功能增强
+- [ ] 3D 可视化（完整 3D 坐标系）
+- [ ] 时序分析（同一目标的轨迹）
+- [ ] 热力图（误差分布）
+- [ ] 自动模式识别
+
+---
+
+## 相关文档
+
+- [Heading Error Visualization Plan](HEADING_ERROR_VISUALIZATION_PLAN.md) - 完整分析方案
+- [GT Visualization Guide](../test_scripts/GT_VISUALIZATION_COMPLETE_GUIDE.md) - GT 可视化工具
+- [Metrics Summary](../eval_results_common_match_comparison_v1/.../METRICS_SUMMARY.md) - 评估指标总结
+
+---
+
+## 技术支持
+
+遇到问题请：
+
+1. 查看对应工具的详细文档
+2. 运行测试脚本验证环境
+3. 检查输入文件格式
+4. 查看错误日志
+
+```bash
+# 调试模式运行
+python eval_tools/extract_bad_heading_cases.py \
+    --input detailed_3d_matches.json \
+    --threshold 1.5 \
+    --output test.json \
+    --stats 2>&1 | tee debug.log
+```
+
+---
+
+**更新日期**: 2026-02-04  
+**版本**: 1.0  
+**状态**: Tool 1 & 2 已完成，Tool 3 & 4 待实现

eval_tools/docs/HEADING_ERROR_VISUALIZATION_PLAN.md

@@ -0,0 +1,465 @@
+# Heading误差可视化分析方案
+
+## 方案概述
+
+针对`detailed_3d_matches.json`中heading误差较大的目标，设计多维度可视化分析系统，帮助定位问题根源。
+
+---
+
+## 📋 方案设计
+
+### 1. 数据筛选策略
+
+#### 筛选条件
+- **主要条件**：`heading_error > threshold`（建议threshold=1.5rad ≈ 85°）
+- **辅助条件**：
+  - 按类别筛选（vehicle/pedestrian/bicycle/rider）
+  - 按距离范围筛选（近距离/中距离/远距离）
+  - 按置信度筛选（高置信度bad cases更值得关注）
+  - 按误差类型筛选（接近π的反转错误 vs 其他错误）
+
+#### 误差分级
+```python
+# 误差等级定义
+ERROR_LEVELS = {
+    'critical': heading_error > 2.5,      # 严重错误（>143°）
+    'severe': 1.5 < heading_error <= 2.5,  # 重大错误（85°-143°）
+    'moderate': 0.5 < heading_error <= 1.5, # 中等错误（28°-85°）
+    'minor': heading_error <= 0.5          # 轻微错误（<28°）
+}
+```
+
+### 2. 可视化维度
+
+#### A. BEV鸟瞰图视角 🗺️
+
+**目的**：直观对比GT和预测的朝向差异
+
+**可视化内容**：
+```
+┌─────────────────────────────────────┐
+│  Bird's Eye View                     │
+│                                      │
+│  ↑ Z (Forward)                       │
+│  │                                   │
+│  │    ┏━━━━━━━━┓  ← GT (绿色)       │
+│  │    ┃        ┃  箭头指向前方      │
+│  │    ┃  →     ┃                    │
+│  │    ┗━━━━━━━━┛                    │
+│  │                                   │
+│  │    ╔════════╗  ← Pred (红色)     │
+│  │    ║    ←   ║  箭头指向相反      │
+│  │    ║        ║                    │
+│  │    ╚════════╝                    │
+│  │                                   │
+│  └────────────────→ X (Right)       │
+│                                      │
+│  误差：3.14 rad (180°) - 方向反转   │
+└─────────────────────────────────────┘
+```
+
+**实现要点**：
+- GT用绿色框+绿色箭头
+- Prediction用红色框+红色箭头
+- 标注误差值（弧度和角度）
+- 显示自车位置
+- 标注距离网格
+
+#### B. 图像投影视角 📷
+
+**目的**：在原图上查看目标外观和朝向关系
+
+**可视化内容**：
+- 原始图像
+- 2D边界框（GT绿色，Pred红色）
+- 3D边界框投影（8个顶点+12条边）
+- 朝向指示箭头
+- 信息标注：
+  - 类别、置信度
+  - heading_error值
+  - GT rotation vs Pred rotation
+  - 距离信息
+
+#### C. 角度对比图 📐
+
+**目的**：可视化角度差异的具体模式
+
+**1. 圆形角度图（Polar Plot）**
+```
+        0° (Forward)
+           ↑
+           │
+           │
+  270° ────┼──── 90°
+           │
+           │
+           ↓
+        180° (Backward)
+
+● GT角度位置（绿点）
+● Pred角度位置（红点）
+→ 误差向量（虚线箭头）
+```
+
+**2. 散点图（Scatter Plot）**
+```
+Pred Rotation
+   π ┤         ●●●●●●  ← 反转错误聚集区
+     │         ●
+     │
+     │    ●
+  -π ├────●──────────────
+    -π          0         π
+           GT Rotation
+```
+
+**3. 误差分布直方图**
+```
+Count
+  │  █
+  │  █
+  │  █  █
+  │  █  █  █
+  │  █  █  █  █
+  └──────────────────→
+     Heading Error (rad)
+```
+
+#### D. 多目标对比面板 📊
+
+**目的**：同时展示多个bad cases，发现共性
+
+**布局设计**：
+```
+┌─────────────────────────────────────────────────────┐
+│  Top 10 Worst Heading Errors                        │
+├──────────┬──────────┬──────────┬─────────────────────┤
+│  Case 1  │  Case 2  │  Case 3  │  ...                │
+│  ┌────┐  │  ┌────┐  │  ┌────┐  │                     │
+│  │BEV │  │  │BEV │  │  │BEV │  │  每个case包含：     │
+│  └────┘  │  └────┘  │  └────┘  │  - BEV图            │
+│  ┌────┐  │  ┌────┐  │  ┌────┐  │  - 原图             │
+│  │IMG │  │  │IMG │  │  │IMG │  │  - 关键信息         │
+│  └────┘  │  └────┘  │  └────┘  │                     │
+│  Info    │  Info    │  Info    │                     │
+├──────────┴──────────┴──────────┴─────────────────────┤
+│  共性分析：                                           │
+│  - 80%为反转错误（error ≈ π）                        │
+│  - 90%为vehicle类别                                   │
+│  - 集中在中远距离（30-80m）                           │
+└─────────────────────────────────────────────────────┘
+```
+
+#### E. 交互式分析面板 🖱️
+
+**目的**：支持深入分析和探索
+
+**功能设计**：
+```
+┌─────────────────────────────────────────────────────┐
+│  Heading Error Analysis Dashboard                    │
+├─────────────────────────────────────────────────────┤
+│  [Filters]                                           │
+│  Class:    [✓All ✓Vehicle ✓Pedestrian □Bicycle]    │
+│  Distance: [0-30m] [30-60m] [60-100m] [100m+]       │
+│  Error:    [>2.5] [1.5-2.5] [0.5-1.5] [<0.5]        │
+│  Pattern:  [Reversal (≈π)] [Other]                  │
+├─────────────────────────────────────────────────────┤
+│  ┌──────────────┐  ┌──────────────┐                 │
+│  │ Distribution │  │ Scatter Plot │                 │
+│  │    Chart     │  │  GT vs Pred  │                 │
+│  └──────────────┘  └──────────────┘                 │
+├─────────────────────────────────────────────────────┤
+│  [Selected Case Detail]                             │
+│  ┌────────────┐  ┌────────────┐  ┌────────────┐    │
+│  │   Image    │  │    BEV     │  │   Stats    │    │
+│  │            │  │            │  │            │    │
+│  └────────────┘  └────────────┘  └────────────┘    │
+├─────────────────────────────────────────────────────┤
+│  Navigation: [< Prev]  Case 15/234  [Next >]       │
+└─────────────────────────────────────────────────────┘
+```
+
+### 3. 统计分析图表
+
+#### A. 误差模式分析
+
+**1. 误差类型饼图**
+```
+┌─────────────────────┐
+│  Error Patterns     │
+│                     │
+│   ╱────╲            │
+│  │ 68% │  反转错误  │
+│  │  π  │            │
+│   ╲────╱            │
+│   ╱──╲ 22% 其他     │
+│  │10%│  中等误差    │
+│   ╲──╱              │
+└─────────────────────┘
+```
+
+**2. 各类别误差对比**
+```
+Avg Error (rad)
+   3.0 ┤
+   2.5 ┤        █
+   2.0 ┤     █  █
+   1.5 ┤  █  █  █
+   1.0 ┤  █  █  █  █
+   0.5 ┤  █  █  █  █
+   0.0 └──────────────→
+       Veh Ped Bic Rid
+```
+
+**3. 距离-误差关系图**
+```
+Error (rad)
+   3.0 ┤  ●●●
+   2.5 ┤ ●●●●●
+   2.0 ┤●●●●●●●
+   1.5 ┤●●●●●●●●
+   1.0 ┤ ●●●●●●●
+   0.5 ┤  ●●●●●
+   0.0 └────────────────→
+       0   30   60   90
+          Distance (m)
+```
+
+#### B. GT vs Pred角度关系矩阵
+
+**热力图展示**：
+```
+Pred Angle
+    π ┤ ██░░░░░░  ← 反转区域
+      │ ░░░░░░░░
+      │ ░░░░█░░░  ← 正常区域
+      │ ░░░░░░░░
+   -π └───────────
+     -π    0    π
+        GT Angle
+
+颜色深度 = case数量
+```
+
+### 4. 输出报告
+
+#### A. 自动生成分析报告
+
+**报告结构**：
+```markdown
+# Heading误差分析报告
+
+## 1. 概览
+- 总样本数：596,602
+- 高误差样本（>1.5rad）：5,839 (0.98%)
+- 反转错误（>3.04rad）：4,012 (68.7%)
+
+## 2. 典型Case展示
+[Top 20 worst cases with images]
+
+## 3. 误差模式
+- 方向反转（≈180°）：68.7%
+- 其他大误差：31.3%
+
+## 4. 类别分布
+- Vehicle: 86.8% 反转率
+- Bicycle: 38.3% 反转率
+- Rider: 34.0% 反转率
+- Pedestrian: 11.7% 反转率
+
+## 5. 建议
+...
+```
+
+#### B. 交互式HTML报告
+
+**功能**：
+- 可筛选、排序的表格
+- 点击查看详细可视化
+- 导出功能
+- 打印友好版本
+
+---
+
+## 🛠️ 实现工具
+
+### 工具1：数据提取器
+```bash
+python extract_bad_heading_cases.py \
+    --input detailed_3d_matches.json \
+    --threshold 1.5 \
+    --output bad_heading_cases.json
+```
+
+### 工具2：可视化生成器
+```bash
+python visualize_heading_errors.py \
+    --input bad_heading_cases.json \
+    --gt-dir /path/to/labels \
+    --img-dir /path/to/images \
+    --output-dir runs/heading_analysis
+```
+
+### 工具3：交互式查看器
+```bash
+python heading_error_viewer.py \
+    --input bad_heading_cases.json \
+    --port 8080
+# 在浏览器打开 http://localhost:8080
+```
+
+### 工具4：报告生成器
+```bash
+python generate_heading_report.py \
+    --input bad_heading_cases.json \
+    --output heading_error_report.html
+```
+
+---
+
+## 📂 输出目录结构
+
+```
+runs/heading_error_analysis/
+├── overview/
+│   ├── error_distribution.png
+│   ├── gt_vs_pred_scatter.png
+│   ├── error_by_class.png
+│   ├── error_by_distance.png
+│   └── pattern_analysis.png
+├── bev_visualization/
+│   ├── case_000001_bev.png
+│   ├── case_000002_bev.png
+│   └── ...
+├── image_visualization/
+│   ├── case_000001_img.png
+│   ├── case_000002_img.png
+│   └── ...
+├── combined_view/
+│   ├── case_000001_combined.png  # BEV + IMG + Stats
+│   ├── case_000002_combined.png
+│   └── ...
+├── multi_case_panels/
+│   ├── top10_worst.png
+│   ├── reversal_cases.png
+│   └── by_class.png
+├── bad_heading_cases.json
+├── analysis_report.md
+├── analysis_report.html
+└── statistics.json
+```
+
+---
+
+## 🎯 关键洞察点
+
+### 预期发现
+
+1. **反转错误模式**
+   - GT接近0时，Pred接近π
+   - GT为负值时，Pred为大正值
+   - 特定类别更易发生反转
+
+2. **距离相关性**
+   - 远距离目标误差更大？
+   - 近距离反转错误比例？
+
+3. **类别差异**
+   - Vehicle反转率最高（86.8%）
+   - Pedestrian相对较好（11.7%）
+
+4. **视觉特征**
+   - 反转的目标有何共同特征？
+   - 遮挡、截断的影响？
+   - 角度标注的歧义性？
+
+### 可能的根因
+
+1. **模型架构问题**
+   - Heading输出范围映射错误
+   - 损失函数未考虑角度周期性
+   - 前后方向特征不明显
+
+2. **数据问题**
+   - 训练数据角度分布不均
+   - 标注歧义（前/后定义）
+   - 数据增强影响角度
+
+3. **后处理问题**
+   - 角度归一化错误
+   - 坐标系转换问题
+
+---
+
+## 📝 使用流程
+
+### Step 1: 提取数据
+```bash
+python extract_bad_heading_cases.py \
+    --input eval_results_common_match_comparison/yolov5s-300w/20260203_210259/detailed_3d_matches.json \
+    --threshold 1.5 \
+    --top-k 100 \
+    --output bad_cases.json
+```
+
+### Step 2: 生成可视化
+```bash
+python visualize_heading_errors.py \
+    --input bad_cases.json \
+    --gt-dir /data/labels \
+    --img-dir /data/images \
+    --output-dir runs/heading_viz
+```
+
+### Step 3: 查看分析
+```bash
+# 方式1：静态报告
+open runs/heading_viz/analysis_report.html
+
+# 方式2：交互式查看
+python heading_error_viewer.py --input bad_cases.json --port 8080
+```
+
+### Step 4: 深入分析
+- 根据可视化结果筛选特定模式
+- 对比不同条件下的误差
+- 定位具体问题原因
+
+---
+
+## 🔄 迭代优化
+
+基于分析结果，可以：
+
+1. **模型改进**
+   - 优化heading损失函数
+   - 添加角度歧义解决机制
+   - 增强前后方向特征学习
+
+2. **数据改进**
+   - 检查和修正标注错误
+   - 平衡角度分布
+   - 添加难例样本
+
+3. **评估改进**
+   - 区分反转错误和其他错误
+   - 添加角度置信度评估
+   - 引入人工review机制
+
+---
+
+## 📌 总结
+
+这套方案提供：
+- ✅ 多维度可视化（BEV、图像、统计图）
+- ✅ 自动化工具链
+- ✅ 交互式分析能力
+- ✅ 完整的输出报告
+- ✅ 可扩展的分析框架
+
+通过这套方案，可以：
+1. 快速定位heading误差大的样本
+2. 直观理解误差模式
+3. 发现问题根因
+4. 指导模型改进

eval_tools/docs/QUICKSTART.md

@@ -0,0 +1,149 @@
+# 快速开始：双ROI融合模型测试
+
+## 5分钟快速上手
+
+### 1. 检查依赖
+
+```bash
+# 确保在项目根目录
+cd /path/to/yolov5-3d
+
+# 检查Python包
+python -c "import numpy, cv2, torch; print('基础依赖OK')"
+
+# 如果测试ONNX模型，检查onnxruntime
+python -c "import onnxruntime; print('ONNX Runtime已安装')"
+```
+
+### 2. 准备模型和数据
+
+确保以下文件存在：
+- 模型文件：`release/yolov5s-30w/merged_model.onnx` 或 `.torchscript`
+- 测试数据：视频bin文件或图像目录
+
+### 3. 运行测试
+
+#### 方式一：使用交互式脚本（推荐）
+
+```bash
+./eval_tools/test_merged_model.sh
+# 然后按提示选择测试模式（1-7）
+```
+
+#### 方式二：直接命令
+
+```bash
+# ONNX模型
+python eval_tools/test_val_merged_model.py \
+    --source /data1/dongying/Mono3d/G1M3/eval_dataset/20251210222153/sigmastar.1/camera4.bin \
+    --weights release/yolov5s-30w/merged_model.onnx \
+    --model-type onnx
+
+# TorchScript模型
+python eval_tools/test_val_merged_model.py \
+    --source /data1/dongying/Mono3d/G1M3/eval_dataset/20251210222153/sigmastar.1/camera4.bin \
+    --weights release/yolov5s-30w/merged_model.torchscript \
+    --model-type torchscript \
+    --device cuda
+```
+
+### 4. 查看结果
+
+```bash
+# 结果保存在
+ls runs/val_viz_merged/exp/
+
+# 用图像查看器打开
+eog runs/val_viz_merged/exp/000000_*.jpg
+# 或
+display runs/val_viz_merged/exp/000000_*.jpg
+```
+
+## 常用配置
+
+### 提高检测精度（减少误检）
+
+```bash
+python eval_tools/test_val_merged_model.py \
+    --source your/data/path \
+    --weights your/model.onnx \
+    --model-type onnx \
+    --conf-thres 0.4 \
+    --iou-thres 0.5
+```
+
+### 获取更多检测（增加召回）
+
+```bash
+python eval_tools/test_val_merged_model.py \
+    --source your/data/path \
+    --weights your/model.onnx \
+    --model-type onnx \
+    --conf-thres 0.15 \
+    --iou-thres 0.6
+```
+
+## 文件说明
+
+| 文件 | 说明 |
+|------|------|
+| `test_val_merged_model.py` | 主测试脚本 |
+| `test_merged_model.sh` | 交互式测试脚本（推荐） |
+| `README_MERGED_MODEL.md` | 完整文档（详细说明） |
+| `QUICKSTART_MERGED.md` | 本文件（快速入门） |
+
+## 故障排查
+
+### 问题1：找不到模块
+
+```
+ModuleNotFoundError: No module named 'utils'
+```
+
+**解决**：确保在项目根目录运行
+```bash
+cd /deeplearning_team/ydong/dongying/projects/yolov5-3d
+python eval_tools/test_val_merged_model.py ...
+```
+
+### 问题2：ONNX Runtime错误
+
+```
+ImportError: onnxruntime not installed
+```
+
+**解决**：
+```bash
+pip install onnxruntime-gpu  # GPU版本
+# 或
+pip install onnxruntime      # CPU版本
+```
+
+### 问题3：CUDA内存不足
+
+```
+RuntimeError: CUDA out of memory
+```
+
+**解决**：使用CPU推理
+```bash
+# TorchScript模型
+python eval_tools/test_val_merged_model.py \
+    --model-type torchscript \
+    --device cpu \
+    ...
+```
+
+## 下一步
+
+- 阅读完整文档：`README_MERGED_MODEL.md`
+- 调整检测参数优化效果
+- 修改代码处理更多帧（默认10帧）
+- 集成到自己的推理流程
+
+## 技术支持
+
+遇到问题？请检查：
+1. 完整文档 `README_MERGED_MODEL.md`
+2. 脚本帮助 `python eval_tools/test_val_merged_model.py --help`
+3. 交互式脚本选项7查看详细帮助

eval_tools/docs/README_VAL_MERGED.md

@@ -0,0 +1,303 @@
+# 合并模型评测指南
+
+## 概述
+
+`val_merged_model.py` 是专门用于评测合并双ROI模型的脚本。它基于 `val.py` 的评测逻辑，但针对双ROI模型进行了适配。
+
+## 功能特性
+
+1. **双ROI数据加载**：为ROI0和ROI1分别创建数据加载器
+2. **同步推理**：同时对两个ROI进行推理
+3. **跨ROI结果融合**：使用NmsForScale算法合并两个ROI的检测结果，避免重复检测
+4. **完整指标计算**：计算2D mAP和3D指标（深度误差、方向误差、中心点误差）
+
+## 工作流程
+
+```
+输入数据
+   ↓
+ROI0 Dataloader  →  推理 →  NMS  →  ↘
+                                     跨ROI融合  →  2D/3D指标计算
+ROI1 Dataloader  →  推理 →  NMS  →  ↗
+```
+
+### 详细步骤
+
+1. **加载合并模型**：加载包含 `model_roi0` 和 `model_roi1` 的合并模型
+2. **创建数据加载器**：
+   - ROI0: 裁剪区域 [0, 120, 1920, 1080]
+   - ROI1: 中心区域 [704, 352]，原始图像 [1920, 1080]
+3. **推理**：对两个ROI同时进行前向传播
+4. **NMS**：对每个ROI的检测结果独立进行NMS
+5. **跨ROI融合**：使用NmsForScale算法合并检测结果：
+   - 移除边缘伪影（IoU > 0.9且超出ROI边界）
+   - 抑制重复检测（IoU > 0.3，保留更靠近ROI中心的检测）
+   - 移除高度包含的检测（IoU_in > 0.8）
+6. **指标计算**：对融合后的结果计算评测指标
+
+## 使用方法
+
+### 基本用法
+
+```bash
+python eval_tools/val_merged_model.py \
+    --data data/mono3d.yaml \
+    --weights release/yolov5s-30w/merged_model.pt \
+    --imgsz 704 352 \
+    --batch-size 1
+```
+
+### 参数说明
+
+| 参数 | 默认值 | 说明 |
+|------|--------|------|
+| `--data` | `data/mono3d.yaml` | 数据集配置文件路径 |
+| `--weights` | 必需 | 合并模型权重文件路径（.pt格式）|
+| `--batch-size` | `1` | 批大小（仅支持1）|
+| `--imgsz` | `[704, 352]` | 推理图像尺寸 [宽度, 高度] |
+| `--conf-thres` | `0.001` | NMS置信度阈值（低值用于AP计算）|
+| `--conf-thres-plot` | `0.25` | 可视化和3D指标的置信度阈值 |
+| `--iou-thres` | `0.6` | NMS的IoU阈值 |
+| `--max-det` | `300` | 每张图片最大检测数 |
+| `--device` | `''` | 设备（如'0'或'cpu'）|
+| `--workers` | `8` | 数据加载器工作线程数 |
+| `--project` | `runs/val_merged` | 结果保存目录 |
+| `--name` | `exp` | 实验名称 |
+| `--verbose` | - | 按类别报告mAP |
+| `--half` | - | 使用FP16半精度推理 |
+
+### 完整示例
+
+```bash
+# 基本评测
+python eval_tools/val_merged_model.py \
+    --data data/mono3d.yaml \
+    --weights release/yolov5s-30w/merged_model.pt \
+    --imgsz 704 352
+
+# 使用GPU加速和半精度
+python eval_tools/val_merged_model.py \
+    --data data/mono3d.yaml \
+    --weights release/yolov5s-30w/merged_model.pt \
+    --imgsz 704 352 \
+    --device 0 \
+    --half
+
+# 详细输出（按类别显示mAP）
+python eval_tools/val_merged_model.py \
+    --data data/mono3d.yaml \
+    --weights release/yolov5s-30w/merged_model.pt \
+    --imgsz 704 352 \
+    --verbose
+
+# 自定义输出目录
+python eval_tools/val_merged_model.py \
+    --data data/mono3d.yaml \
+    --weights release/yolov5s-30w/merged_model.pt \
+    --imgsz 704 352 \
+    --project runs/my_eval \
+    --name merged_exp1
+```
+
+## 输出结果
+
+### 控制台输出
+
+评测过程中会实时显示：
+
+```
+                 Class     Images  Instances          P          R      mAP50   mAP50-95    Matched   Depth(m)  Orient(°)  Center(m)
+                   all         50        150      0.856      0.823      0.891      0.672         89      2.345     12.678      0.234
+```
+
+- **Class**: 类别名称
+- **Images**: 处理的图片数量
+- **Instances**: 真实标注数量
+- **P**: 精确率
+- **R**: 召回率
+- **mAP50**: mAP@0.5
+- **mAP50-95**: mAP@0.5:0.95
+- **Matched**: 3D匹配的检测数
+- **Depth(m)**: 平均深度误差（米）
+- **Orient(°)**: 平均方向误差（度）
+- **Center(m)**: 平均中心点误差（米）
+
+### 最终统计
+
+```
+3D Metrics (Fused ROI0+ROI1):
+  Total matched: 89
+  Depth error (abs): 2.345m
+  Orientation error: 12.678°
+  Center error: 0.234m
+```
+
+### 保存的文件
+
+结果保存在 `--project/--name` 目录下：
+
+- `confusion_matrix.png`: 混淆矩阵图
+- `F1_curve.png`: F1曲线
+- `P_curve.png`: 精确率曲线
+- `R_curve.png`: 召回率曲线
+- `PR_curve.png`: PR曲线
+
+## ROI配置
+
+脚本内置了ROI配置，与 `test_val_merged_model.py` 保持一致：
+
+```python
+# ROI0: 裁剪区域
+roi0 = (0, 120, 1920, 1080)
+ori_img_size_roi0 = None
+
+# ROI1: 中心区域
+roi1 = (704, 352)
+ori_img_size_roi1 = (1920, 1080)
+
+# 跨ROI NMS使用的ROI坐标（原始图像坐标系）
+roi_list = [
+    (0, 120, 1920, 1080),  # ROI0
+    (608, 364, 1312, 716)   # ROI1在原始图像中的坐标
+]
+```
+
+## 跨ROI NMS算法
+
+`NmsForScale` 函数实现了跨ROI的检测融合：
+
+1. **边缘伪影过滤**：
+   - 计算检测框与ROI的IoU
+   - 如果IoU > 0.9且检测框超出ROI边界，则移除
+
+2. **重复检测抑制**：
+   - 对于来自不同ROI的检测框
+   - 如果IoU > 0.3（高度重叠）
+   - 保留距离ROI中心更近的检测
+
+3. **包含关系处理**：
+   - 如果一个检测框被另一个检测框高度包含（IoU_in > 0.8）
+   - 移除被包含的检测框
+
+## 注意事项
+
+1. **批大小限制**：合并模型评测仅支持 `batch_size=1`
+2. **模型格式**：仅支持PyTorch .pt格式的合并模型（包含Model_Merged类）
+3. **数据集要求**：数据集必须包含3D标注（48列标签格式）
+4. **内存使用**：同时加载两个ROI的数据会增加内存使用
+5. **速度**：由于需要推理两个ROI和跨ROI融合，速度会比单ROI评测慢
+
+## 与单ROI评测的对比
+
+| 特性 | val.py (单ROI) | val_merged_model.py (双ROI) |
+|------|----------------|------------------------------|
+| 批大小 | 支持任意 | 仅支持1 |
+| 数据加载器 | 单个 | 两个（ROI0+ROI1）|
+| 推理次数 | 1次/图像 | 2次/图像 |
+| NMS | 单次 | 2次独立+跨ROI融合 |
+| 结果 | 单ROI检测 | 融合后的全图检测 |
+| 速度 | 快 | 较慢（约2倍推理时间）|
+| 覆盖范围 | 单个ROI | 全图覆盖 |
+
+## 故障排除
+
+### 问题1：模型加载失败
+
+```
+ValueError: Model at xxx is not a merged dual-ROI model
+```
+
+**解决方法**：确保模型是通过 `merge_models_of_2roi.py` 创建的合并模型，包含 `model_roi0` 和 `model_roi1` 属性。
+
+### 问题2：数据加载器长度不匹配
+
+```
+WARNING Dataloader lengths mismatch: ROI0=100, ROI1=98
+```
+
+**解决方法**：检查数据集配置，确保两个ROI访问相同的图像文件。
+
+### 问题3：路径不匹配
+
+```
+WARNING Path mismatch: xxx vs yyy
+```
+
+**解决方法**：这通常表示两个数据加载器返回的图像顺序不一致，检查数据集配置和路径。
+
+### 问题4：内存不足
+
+**解决方法**：
+- 减少 `--workers` 数量
+- 使用 `--half` 启用FP16推理
+- 在CPU上运行：`--device cpu`
+
+## 相关文件
+
+- `val.py`: 单ROI模型评测脚本
+- `merge_models_of_2roi.py`: 合并两个ROI模型的脚本
+- `test_val_merged_model.py`: 合并模型推理测试脚本（仅推理，不计算指标）
+- `val_merged_model.py`: 本脚本，合并模型完整评测
+
+## 工作流程示例
+
+完整的训练+合并+评测流程：
+
+```bash
+# 1. 训练ROI0模型
+python train_mono3d.py --data data/mono3d.yaml --roi 0 120 1920 1080 --weights yolov5s.pt
+
+# 2. 训练ROI1模型  
+python train_mono3d.py --data data/mono3d.yaml --roi 704 352 --ori-img-size 1920 1080 --weights yolov5s.pt
+
+# 3. 评测单个模型（可选）
+python val.py --weights runs/train/roi0_exp/weights/last.pt --data data/mono3d.yaml --roi 0 120 1920 1080
+python val.py --weights runs/train/roi1_exp/weights/last.pt --data data/mono3d.yaml --roi 704 352 --ori-img-size 1920 1080
+
+# 4. 合并模型
+python eval_tools/merge_models_of_2roi.py \
+    --roi0_model_path runs/train/roi0_exp/weights/last.pt \
+    --roi1_model_path runs/train/roi1_exp/weights/last.pt \
+    --save_dir release/merged \
+    --skip-export  # 仅保存合并模型，不导出ONNX
+
+# 5. 评测合并模型
+python eval_tools/val_merged_model.py \
+    --data data/mono3d.yaml \
+    --weights release/merged/merged_model.pt \
+    --imgsz 704 352
+
+# 6. 推理测试（可选）
+python eval_tools/test_val_merged_model.py \
+    --source /path/to/test/images \
+    --weights release/merged/merged_model.pt \
+    --model-type torchscript
+```
+
+## 扩展说明
+
+如果需要修改ROI配置，编辑脚本中的以下部分：
+
+```python
+# 在 run() 函数中，约第375行
+roi0 = (0, 120, 1920, 1080)  # 修改ROI0坐标
+roi1 = (704, 352)             # 修改ROI1尺寸
+ori_img_size_roi1 = (1920, 1080)  # 修改原始图像尺寸
+
+# 在跨ROI NMS部分，约第431行
+roi_list = [
+    (0, 120, 1920, 1080),    # ROI0坐标
+    (608, 364, 1312, 716)    # ROI1在原始图像中的坐标（需要计算）
+]
+```
+
+计算ROI1在原始图像中的坐标：
+```python
+# 如果ROI1是中心裁剪，计算方式：
+ori_w, ori_h = 1920, 1080
+roi1_w, roi1_h = 704, 352
+x1 = (ori_w - roi1_w) // 2  # 608
+y1 = (ori_h - roi1_h) // 2  # 364
+x2 = x1 + roi1_w            # 1312
+y2 = y1 + roi1_h            # 716
+```

eval_tools/docs/ROI_FIX_GUIDE.md

@@ -0,0 +1,146 @@
+# ROI坐标转换问题解决方案
+
+## 问题描述
+
+评测结果中precision和recall都为0，经过诊断发现是**ROI坐标转换问题**。
+
+## 根本原因
+
+1. **检测结果使用ROI坐标系**：保存在`txt_results`中的检测框坐标是在ROI裁剪区域的坐标系中（尺寸: 704x352）
+2. **Ground Truth使用原图坐标系**：GT标注使用的是归一化的原图坐标（尺寸: 1920x1080）
+3. **坐标不匹配导致IoU=0**：两个坐标系不一致，所有框的IoU都是0，无法匹配
+
+## ROI参数
+
+针对`evalset_roi0`数据集：
+- **ROI区域**: [0, 120, 1920, 1080] (xyxy格式)
+- **ROI输入尺寸**: 704 x 352 (模型输入尺寸)
+- **原图尺寸**: 1920 x 1080
+
+### 坐标转换公式
+
+检测框从ROI坐标转换到原图坐标：
+
+```python
+# 1. 缩放
+scale_x = (1920 - 0) / 704 = 2.7273
+scale_y = (1080 - 120) / 352 = 2.7273
+
+x1_scaled = x1 * scale_x
+x2_scaled = x2 * scale_x  
+y1_scaled = y1 * scale_y
+y2_scaled = y2 * scale_y
+
+# 2. 偏移
+x1_final = x1_scaled + 0
+x2_final = x2_scaled + 0
+y1_final = y1_scaled + 120
+y2_final = y2_scaled + 120
+```
+
+## 解决方案
+
+### 1. 修改DetectionParser
+
+在`parser.py`中添加ROI缩放和偏移支持：
+
+```python
+def __init__(self, roi_offset=None, roi_scale=None):
+    self.roi_offset = roi_offset or (0, 0)
+    self.roi_scale = roi_scale or (1.0, 1.0)
+```
+
+在解析2D框时应用转换：
+
+```python
+# 先缩放
+x1 *= self.roi_scale[0]
+x2 *= self.roi_scale[0]
+y1 *= self.roi_scale[1]
+y2 *= self.roi_scale[1]
+
+# 再偏移
+x1 += self.roi_offset[0]
+y1 += self.roi_offset[1]
+x2 += self.roi_offset[0]
+y2 += self.roi_offset[1]
+```
+
+### 2. 更新Evaluator和eval.py
+
+添加`roi_offset`和`roi_scale`参数传递。
+
+### 3. 更新命令行参数
+
+```bash
+--roi 0 120 1920 1080           # ROI区域
+--roi-input-size 704 352        # ROI输入尺寸
+```
+
+## 验证结果
+
+### 修复前
+```
+GT bbox range: x=[18.0, 1919.0], y=[465.0, 751.0]
+Det bbox range: x=[9.5, 704.6], y=[126.1, 236.8]
+IoU: 0.0000
+Matches: 0
+```
+
+### 修复后
+```
+GT bbox range: x=[18.0, 1919.0], y=[465.0, 751.0]
+Det bbox range (scaled): x=[25.9, 1921.6], y=[463.9, 765.8]
+Matches: 16/20
+First match IoU: 0.8397
+```
+
+### 评测结果（100帧测试）
+```
+2D Metrics:
+  Precision: 0.9141
+  Recall:    0.6105
+  mAP:       0.2066
+
+3D Metrics:
+  pedestrian: Lat=0.795m, Long=1.726m, Head=1.140rad (n=8)
+  vehicle: Lat=0.987m, Long=1.731m, Head=0.247rad (n=995)
+```
+
+## 使用方法
+
+### 完整评测（81443帧）
+```bash
+bash eval_tools/run_evaluation_example.sh
+```
+
+### 快速测试（100帧）
+```bash
+bash eval_tools/run_evaluation_quick_test.sh
+```
+
+### 手动运行
+```bash
+python eval_tools/eval.py \
+    --det-path /path/to/detections \
+    --gt-path /path/to/gt \
+    --output-dir results \
+    --img-width 1920 \
+    --img-height 1080 \
+    --roi 0 120 1920 1080 \
+    --roi-input-size 704 352
+```
+
+## 注意事项
+
+1. **ROI1的参数不同**：如果评测ROI1的结果，需要修改ROI参数
+2. **不同模型的输入尺寸可能不同**：根据实际模型调整`--roi-input-size`
+3. **大数据集评测耗时**：81443帧约需10-20分钟，建议先用quick test验证
+
+## 相关文件
+
+- `eval_tools/evaluator/parser.py` - 添加ROI转换逻辑
+- `eval_tools/evaluator/evaluator.py` - 传递ROI参数
+- `eval_tools/eval.py` - 命令行参数处理
+- `eval_tools/run_evaluation_example.sh` - 完整评测脚本
+- `eval_tools/run_evaluation_quick_test.sh` - 快速测试脚本

eval_tools/docs/ROI_GT_PROCESSING_GUIDE.md

@@ -0,0 +1,437 @@
+# Ground Truth ROI Processing for Evaluation
+
+## 概述
+
+为了确保评测指标的准确性，评测时需要对Ground Truth（真值标签）进行与训练时相同的ROI过滤和截断处理。这确保了评测时GT和检测结果在同一坐标系下进行比较。
+
+## 问题背景
+
+### 训练时的处理
+
+在模型训练时，数据经过以下ROI处理（参考 `utils/dataloaders3d.py`中的`post_process_labels_to_roi`）：
+
+1. **ROI计算**：基于标定参数计算灭点（vanishing point），以灭点为中心裁剪ROI区域
+2. **标签过滤**：移除完全在ROI外的目标
+3. **边界截断**：将部分在ROI内的目标边界框clip到ROI边界
+4. **坐标转换**：将标签坐标从原图坐标系转换到ROI相对坐标系
+5. **特殊处理**：对cut-in/cut-out目标进行特殊标记
+
+### 评测时的问题
+
+之前的评测代码中：
+- **检测结果**：在ROI坐标系中（已经过ROI裁剪和resize）
+- **GT标签**：在原图坐标系中（未经ROI处理）
+- **结果**：坐标系不匹配，导致IoU为0，评测指标不准确
+
+## 解决方案
+
+新增 `ROIProcessor` 模块，在评测时对GT标签进行与训练时相同的ROI处理。
+
+### 核心模块
+
+#### 1. ROIProcessor 类
+
+位置：`eval_tools/evaluator/roi_processor.py`
+
+**主要功能**：
+- 加载标定参数
+- 计算ROI区域（基于灭点）
+- 对GT标签进行ROI过滤和截断
+
+**关键方法**：
+```python
+class ROIProcessor:
+    def __init__(self, calib_root, roi_config, ori_img_size):
+        """初始化ROI处理器
+        
+        Args:
+            calib_root: 标定文件根目录
+            roi_config: ROI配置（[width, height]或[x1, y1, x2, y2]）
+            ori_img_size: 原图尺寸 (width, height)
+        """
+        
+    def compute_roi(self, calib_params):
+        """计算ROI区域
+        
+        基于灭点计算规则：
+        - crop_center_x = oriW // 2
+        - crop_center_y = cy - fy * tan(pitch)
+        - ROI以crop_center为中心裁剪
+        """
+        
+    def process_annotations_with_roi(self, annotations, roi_bounds):
+        """对标注进行ROI过滤和截断
+        
+        处理逻辑：
+        1. 移除完全在ROI外的目标
+        2. clip边界框到ROI范围
+        3. 转换坐标到ROI相对坐标系
+        """
+```
+
+#### 2. Evaluator 更新
+
+位置：`eval_tools/evaluator/evaluator.py`
+
+**主要更新**：
+- 集成 `ROIProcessor`
+- 在worker函数中对GT进行ROI处理
+- 支持多进程评测时的ROI处理
+
+## 使用方法
+
+### 1. 配置文件方式（推荐）
+
+创建配置文件（参考 `eval_tools/configs/eval_config_with_roi_gt.yaml`）：
+
+```yaml
+# Dataset paths
+dataset:
+  det_path: "/path/to/detection_results"
+  gt_path: "/path/to/ground_truth"
+
+# Image properties
+image:
+  width: 1920
+  height: 1080
+
+# ROI Ground Truth Processing
+roi_gt:
+  enabled: true                         # 启用GT的ROI处理
+  calib_root: "/path/to/calibrations"   # 标定文件根目录（包含各case的camera4.json）
+  roi_config: [1920, 960]               # ROI配置，必须与训练配置一致！
+  
+# 其他配置...
+```
+
+运行评测：
+```bash
+python eval_tools/core/eval.py --config eval_tools/configs/eval_config_with_roi_gt.yaml
+```
+
+### 2. ROI配置说明
+
+#### 方式1：尺寸模式（常用）
+```yaml
+roi_config: [1920, 960]  # [width, height]
+```
+- ROI以灭点为中心裁剪
+- width和height指定ROI的宽度和高度
+- **必须与训练时yaml中的roi配置一致**
+
+训练配置对应（`data/mono3d.yaml`）：
+```yaml
+# For ROI0
+roi: [1920, 960]  # 训练时配置
+
+# For ROI1
+# roi: [704, 352]  # 训练时配置
+```
+
+#### 方式2：边界模式
+```yaml
+roi_config: [0, 120, 1920, 1080]  # [x1, y1, x2, y2]
+```
+- 固定的ROI边界
+- 适用于已知固定ROI的情况
+
+### 3. 标定文件要求
+
+ROI处理需要标定参数来计算灭点。标定文件结构：
+
+```
+calib_root/
+├── case1/
+│   └── camera4.json    # 标定文件
+├── case2/
+│   └── camera4.json
+└── ...
+```
+
+标定文件格式（`camera4.json`）：
+```json
+{
+  "focal_u": 1450.0,
+  "focal_v": 1450.0,
+  "cu": 960.0,
+  "cv": 540.0,
+  "yaw": 0.0,
+  "pitch": -5.0,
+  "distort_coeffs": [0.1, 0.05, 0.02, 0.01]
+}
+```
+
+**关键参数**：
+- `focal_u`, `focal_v`: 焦距
+- `cu`, `cv`: 主点坐标
+- `pitch`: 俯仰角（用于计算灭点）
+
+### 4. 完整评测示例
+
+#### ROI0模型评测
+```bash
+python eval_tools/core/eval.py \
+  --config eval_tools/configs/eval_config_with_roi_gt.yaml \
+  --det-path /data/inference_results/roi0 \
+  --gt-path /data/ground_truth
+```
+
+配置文件中：
+```yaml
+roi_gt:
+  enabled: true
+  calib_root: "/data/ground_truth"
+  roi_config: [1920, 960]  # ROI0配置
+```
+
+#### ROI1模型评测
+配置文件中修改：
+```yaml
+roi_gt:
+  enabled: true
+  calib_root: "/data/ground_truth"
+  roi_config: [704, 352]   # ROI1配置
+```
+
+## ROI处理流程
+
+### 1. GT标签ROI处理流程
+
+```
+原始GT标签（原图坐标系）
+    ↓
+加载标定参数
+    ↓
+计算灭点位置
+vanish_y = cy - fy * tan(pitch)
+crop_center = (oriW//2, vanish_y)
+    ↓
+计算ROI区域
+roi_x1 = crop_center_x - roi_width/2
+roi_y1 = crop_center_y - roi_height/2
+roi_x2 = roi_x1 + roi_width
+roi_y2 = roi_y1 + roi_height
+    ↓
+过滤GT标签
+1. 移除完全在ROI外的目标
+2. 保留完全在ROI内的目标
+3. 保留部分在ROI内的目标
+    ↓
+截断边界框
+clip bbox到ROI边界
+    ↓
+坐标转换
+转换到ROI相对坐标系
+    ↓
+处理后的GT标签（ROI坐标系）
+    ↓
+与检测结果匹配和评测
+```
+
+### 2. 核心计算公式
+
+#### 灭点计算
+```python
+vanish_y = cy - fy * tan(pitch * π/180)
+crop_center_x = image_width // 2
+crop_center_y = vanish_y
+```
+
+#### ROI边界计算
+```python
+roi_x1 = crop_center_x - roi_width / 2
+roi_y1 = crop_center_y - roi_height / 2
+roi_x2 = roi_x1 + roi_width
+roi_y2 = roi_y1 + roi_height
+```
+
+#### 坐标转换
+```python
+# 原图坐标 -> ROI相对坐标
+new_x1 = x1 - roi_x1
+new_y1 = y1 - roi_y1
+new_x2 = x2 - roi_x1
+new_y2 = y2 - roi_y1
+
+# Clip到ROI边界
+new_x1 = clip(new_x1, 0, roi_width-1)
+new_y1 = clip(new_y1, 0, roi_height-1)
+new_x2 = clip(new_x2, 0, roi_width-1)
+new_y2 = clip(new_y2, 0, roi_height-1)
+```
+
+## 实现细节
+
+### 1. 多进程支持
+
+ROI处理支持多进程评测，每个worker进程独立创建ROI处理器：
+
+```python
+@staticmethod
+def _process_frame_3d(pair, ...):
+    from .roi_processor import ROIProcessor
+    
+    # 在worker中创建ROI处理器
+    if 'roi_processor_config' in pair:
+        roi_processor = ROIProcessor(...)
+        gts, _ = roi_processor.process_case_frame(...)
+```
+
+### 2. 标定缓存
+
+ROI处理器会缓存已加载的标定参数，避免重复读取：
+
+```python
+self.calib_cache = {}  # case_name -> calib_params
+```
+
+### 3. 处理标记
+
+处理后的标注会添加额外标记：
+
+```python
+{
+    'bbox_2d': [x1, y1, x2, y2],  # ROI相对坐标
+    'roi_relative': True,          # 标记已转换
+    'roi_bounds': (x1, y1, x2, y2), # 记录ROI边界
+    'was_clipped': False,          # 是否被clip
+    '3d_info': {
+        'partially_visible': True  # 3D目标是否部分可见
+    }
+}
+```
+
+## 注意事项
+
+### 1. ROI配置一致性
+
+**关键**：评测时的ROI配置必须与训练时完全一致！
+
+- 训练配置：`data/mono3d.yaml` 中的 `roi: [width, height]`
+- 评测配置：`eval_config_with_roi_gt.yaml` 中的 `roi_config: [width, height]`
+
+不一致会导致评测结果不准确。
+
+### 2. 标定文件路径
+
+确保 `calib_root` 指向正确的标定文件目录：
+```python
+calib_root/
+├── case_name1/
+│   └── camera4.json
+├── case_name2/
+│   └── camera4.json
+```
+
+### 3. 坐标系说明
+
+- **原图坐标系**：(0, 0)在左上角，范围[0, 1920]×[0, 1080]
+- **ROI坐标系**：(0, 0)在ROI左上角，范围[0, roi_width]×[0, roi_height]
+- **归一化坐标**：YOLO格式，范围[0, 1]
+
+### 4. 边界情况处理
+
+- 完全在ROI外的目标：直接过滤
+- 部分在ROI内的目标：保留并clip到边界
+- 3D信息处理：部分可见目标标记为 `partially_visible`
+
+## 验证方法
+
+### 1. 检查ROI处理是否生效
+
+运行评测时查看日志：
+```
+ROI processor enabled for GT filtering with config: [1920, 960]
+```
+
+### 2. 对比有无ROI处理的结果
+
+```bash
+# 不使用ROI处理
+python eval_tools/core/eval.py --config eval_config_no_roi.yaml
+
+# 使用ROI处理
+python eval_tools/core/eval.py --config eval_config_with_roi_gt.yaml
+```
+
+预期：使用ROI处理后，评测指标应该更准确（Precision和Recall不会异常低）。
+
+### 3. 验证GT数量
+
+- ROI处理前：所有原图中的GT
+- ROI处理后：仅ROI内的GT（数量应该减少）
+
+可以在代码中添加调试输出：
+```python
+print(f"GT before ROI: {len(gts_before)}")
+print(f"GT after ROI: {len(gts_after)}")
+```
+
+## 性能考虑
+
+1. **标定缓存**：每个case的标定参数只加载一次
+2. **多进程支持**：支持多进程并行评测
+3. **内存优化**：按需加载和处理
+
+## 故障排查
+
+### 问题1：找不到标定文件
+```
+Warning: Calibration file not found for case xxx
+```
+**解决**：检查 `calib_root` 路径和标定文件名（camera4.json）
+
+### 问题2：评测结果仍然异常
+**检查项**：
+1. ROI配置是否与训练一致
+2. 标定文件是否正确
+3. 图像尺寸配置是否正确
+
+### 问题3：GT数量为0
+**原因**：所有GT都在ROI外被过滤
+**检查**：ROI配置是否过小或位置偏移
+
+## 示例脚本
+
+### Python API使用
+```python
+from eval_tools.evaluator import Evaluator, ROIProcessor
+
+# 创建ROI处理器
+roi_processor = ROIProcessor(
+    calib_root="/path/to/calibrations",
+    roi_config=[1920, 960],
+    ori_img_size=(1920, 1080)
+)
+
+# 处理GT标签
+annotations, roi_bounds = roi_processor.process_case_frame(
+    case_name="case1",
+    frame_name="frame001",
+    annotations=gt_annotations
+)
+
+# 创建评测器（配置版）
+config = {
+    'roi_gt': {
+        'enabled': True,
+        'calib_root': '/path/to/calibrations',
+        'roi_config': [1920, 960]
+    }
+}
+evaluator = Evaluator(config=config)
+```
+
+## 总结
+
+通过引入 `ROIProcessor` 和更新评测流程，现在评测时的GT标签会经过与训练时相同的ROI处理，确保了：
+
+1. ✅ GT和检测结果在同一坐标系
+2. ✅ 完全在ROI外的目标被正确过滤
+3. ✅ 部分在ROI内的目标被正确截断
+4. ✅ 评测指标准确反映模型性能
+
+**关键要点**：
+- 评测配置必须与训练配置一致
+- 需要提供正确的标定文件
+- 支持多进程和不同ROI配置

eval_tools/docs/TWO_LEVEL_PATH_EXAMPLE.md

@@ -0,0 +1,379 @@
+# Example: Using 2-Level Path Structure for Model Evaluation
+
+This example demonstrates how to evaluate models with 2-level directory structures.
+
+## Scenario
+
+You have organized your test data with an additional level of hierarchy:
+
+```
+/data/detections/
+    ├── G1M3_AFS1616/          # Level 1: Dataset/configuration name
+    │   ├── case_001/          # Level 2: Individual test cases
+    │   │   └── txt_results/
+    │   │       ├── frame001.txt
+    │   │       └── frame002.txt
+    │   └── case_002/
+    │       └── txt_results/
+    │           └── frame001.txt
+    └── G1M3_AFS1920/
+        └── case_003/
+            └── txt_results/
+                └── frame001.txt
+
+/data/ground_truth/
+    ├── G1M3_AFS1616/
+    │   ├── case_001/
+    │   │   └── labels/
+    │   │       ├── frame001.txt
+    │   │       └── frame002.txt
+    │   └── case_002/
+    │       └── labels/
+    │           └── frame001.txt
+    └── G1M3_AFS1920/
+        └── case_003/
+            └── labels/
+                └── frame001.txt
+```
+
+## Step 1: Create Configuration File
+
+Create `eval_tools/configs/eval_config_2level_example.yaml`:
+
+```yaml
+# Evaluation Configuration for 2-Level Path Structure
+dataset:
+  det_path: "/data/detections"
+  gt_path: "/data/ground_truth"
+  path_depth: 2  # Enable 2-level structure
+
+image:
+  width: 1920
+  height: 1080
+
+model:
+  input_size: 704
+  min_box_size_at_input_scale: 8
+
+performance:
+  num_workers: 32
+
+roi_gt:
+  enabled: true
+  calib_root: "/data/ground_truth"
+  roi_config: [1920, 960]
+
+roi:
+  enabled: false
+  region: [0, 120, 1920, 1080]
+  input_size: [704, 352]
+
+classes:
+  3d_classes: [0, 1, 2, 3]
+  2d_classes: [4, 5, 6, 7, 8, 9, 10, 11, 12, 13]
+  class_names:
+    0: "vehicle"
+    1: "pedestrian"
+    2: "bicycle"
+    3: "rider"
+    4: "roadblock"
+    5: "head"
+    6: "tsr"
+    7: "guideboard"
+    8: "plate"
+    9: "wheel"
+    10: "tl_border"
+    11: "tl_wick"
+    12: "tl_num"
+    13: "tricycle"
+
+matching:
+  iou_threshold: 0.5
+
+metrics_2d:
+  enabled: true
+  conf_threshold: 0.3
+  ap_method: "voc2010"
+
+metrics_3d:
+  enabled: true
+  heading_tolerance: "both"
+  distance_ranges:
+    - [0, 20]
+    - [20, 40]
+    - [40, 60]
+    - [60, 80]
+    - [80, 100]
+    - [100, 999]
+  lateral_distance_ranges:
+    - [-50, -40]
+    - [-40, -30]
+    - [-30, -20]
+    - [-20, -10]
+    - [-10, 0]
+    - [0, 10]
+    - [10, 20]
+    - [20, 30]
+    - [30, 40]
+    - [40, 50]
+
+output:
+  save_path: "evaluation_results/2level_example/{timestamp}"
+  formats: ["json", "txt"]
+  print_details: true
+  per_case_reports: true
+```
+
+## Step 2: Run Evaluation
+
+### Method 1: Using Config File
+
+```bash
+python eval_tools/core/eval.py \
+  --config eval_tools/configs/eval_config_2level_example.yaml \
+  --save-detailed-matches
+```
+
+### Method 2: Command Line Override
+
+```bash
+python eval_tools/core/eval.py \
+  --config eval_tools/configs/eval_config_2level_example.yaml \
+  --path-depth 2 \
+  --det-path /data/detections \
+  --gt-path /data/ground_truth \
+  --output-dir evaluation_results/custom_output
+```
+
+### Method 3: Without Config File
+
+```bash
+python eval_tools/core/eval.py \
+  --det-path /data/detections \
+  --gt-path /data/ground_truth \
+  --path-depth 2 \
+  --img-width 1920 \
+  --img-height 1080 \
+  --iou-threshold 0.5 \
+  --conf-threshold 0.3 \
+  --heading-tolerance both \
+  --output-dir evaluation_results/2level_test
+```
+
+## Step 3: Compare Two Models with 2-Level Paths
+
+### Create Config for Model 1
+
+`eval_tools/configs/eval_config_model1_2level.yaml`:
+
+```yaml
+dataset:
+  det_path: "/data/model1/detections"
+  gt_path: "/data/ground_truth"
+  path_depth: 2
+
+# ... other settings ...
+```
+
+### Create Config for Model 2
+
+`eval_tools/configs/eval_config_model2_2level.yaml`:
+
+```yaml
+dataset:
+  det_path: "/data/model2/detections"
+  gt_path: "/data/ground_truth"
+  path_depth: 2
+
+# ... other settings ...
+```
+
+### Update Comparison Script
+
+Edit `eval_tools/model_comparison/compare_models_with_common_matches.sh`:
+
+```bash
+#!/bin/bash
+
+set -e
+
+# Configuration
+MODEL1_CONFIG="eval_tools/configs/eval_config_model1_2level.yaml"
+MODEL2_CONFIG="eval_tools/configs/eval_config_model2_2level.yaml"
+OUTPUT_BASE="evaluation_results/2level_comparison"
+TIMESTAMP=$(date +%Y%m%d_%H%M%S)
+
+MODEL1_NAME="model1"
+MODEL2_NAME="model2"
+
+# Step 1: Evaluate Model 1
+echo "Step 1: Evaluating Model 1..."
+MODEL1_OUTPUT="${OUTPUT_BASE}/${MODEL1_NAME}/${TIMESTAMP}"
+python eval_tools/core/eval.py \
+  --config ${MODEL1_CONFIG} \
+  --output-dir ${MODEL1_OUTPUT} \
+  --save-detailed-matches
+
+# Step 2: Evaluate Model 2
+echo "Step 2: Evaluating Model 2..."
+MODEL2_OUTPUT="${OUTPUT_BASE}/${MODEL2_NAME}/${TIMESTAMP}"
+python eval_tools/core/eval.py \
+  --config ${MODEL2_CONFIG} \
+  --output-dir ${MODEL2_OUTPUT} \
+  --save-detailed-matches
+
+# Step 3: Find common matches
+echo "Step 3: Finding common matches..."
+COMMON_MATCHES_DIR="${OUTPUT_BASE}/common_matches_${TIMESTAMP}"
+mkdir -p ${COMMON_MATCHES_DIR}
+
+python eval_tools/model_comparison/find_common_matches.py \
+  --model1-matches ${MODEL1_OUTPUT}/detailed_3d_matches.json \
+  --model2-matches ${MODEL2_OUTPUT}/detailed_3d_matches.json \
+  --output ${COMMON_MATCHES_DIR}/common_matches.json \
+  --model1-name "${MODEL1_NAME}" \
+  --model2-name "${MODEL2_NAME}"
+
+# Step 4: Compare models
+echo "Step 4: Comparing models..."
+COMPARISON_DIR="${OUTPUT_BASE}/comparison_${TIMESTAMP}"
+
+python eval_tools/model_comparison/compare_models.py \
+  --model1 ${MODEL1_OUTPUT}/evaluation_report.json \
+  --model2 ${MODEL2_OUTPUT}/evaluation_report.json \
+  --model1-name "${MODEL1_NAME}" \
+  --model2-name "${MODEL2_NAME}" \
+  --common-matches ${COMMON_MATCHES_DIR}/common_matches.json \
+  --output-dir ${COMPARISON_DIR}
+
+echo "✓ Comparison complete!"
+echo "Results: ${COMPARISON_DIR}/comparison_report.txt"
+```
+
+### Run Comparison
+
+```bash
+bash eval_tools/model_comparison/compare_models_with_common_matches.sh
+```
+
+## Expected Output
+
+```
+================================================================================
+YOLOv5-3D Model Evaluation
+================================================================================
+Detection path: /data/detections
+Ground truth path: /data/ground_truth
+Path depth: 2
+Output directory: evaluation_results/2level_example/20260211_143022
+Image size: 1920x1080
+IoU threshold: 0.5
+Confidence threshold: 0.3
+AP method: voc2010
+Heading tolerance: both
+Number of workers: 32
+Evaluate 2D: True
+Evaluate 3D: True
+Save detailed matches: Yes
+================================================================================
+
+Loading data...
+Found 3 case(s) in detection root: /data/detections (path_depth=2)
+Loaded 5 image pairs for evaluation
+
+==================================================
+Evaluating 2D Detection Metrics
+==================================================
+
+Processing case [1/3]: case_001 (2 frames)
+  case_001: 100%|████████████████████| 2/2 [00:00<00:00, 45.23it/s]
+
+Processing case [2/3]: case_002 (1 frames)
+  case_002: 100%|████████████████████| 1/1 [00:00<00:00, 48.12it/s]
+
+Processing case [3/3]: case_003 (2 frames)
+  case_003: 100%|████████████████████| 2/2 [00:00<00:00, 46.87it/s]
+
+==================================================
+Evaluating 3D Detection Metrics
+==================================================
+
+Processing case [1/3]: case_001 (2 frames)
+  case_001: 100%|████████████████████| 2/2 [00:00<00:00, 42.15it/s]
+
+Processing case [2/3]: case_002 (1 frames)
+  case_002: 100%|████████████████████| 1/1 [00:00<00:00, 43.89it/s]
+
+Processing case [3/3]: case_003 (2 frames)
+  case_003: 100%|████████████████████| 2/2 [00:00<00:00, 41.76it/s]
+
+Detailed 3D matches saved to: evaluation_results/2level_example/20260211_143022/detailed_3d_matches.json
+JSON report saved to: evaluation_results/2level_example/20260211_143022/evaluation_report.json
+Text report saved to: evaluation_results/2level_example/20260211_143022/evaluation_report.txt
+Per-case reports saved to: evaluation_results/2level_example/20260211_143022/per_case_reports/ (3 cases)
+
+================================================================================
+EVALUATION SUMMARY - OVERALL
+================================================================================
+
+2D Metrics:
+  Precision: 0.8542
+  Recall:    0.8123
+  mAP:       0.8234
+
+3D Metrics:
+  vehicle [overall]: Lat=0.234m, Long=0.456m, Head=0.123rad (relaxed=0.098rad, rev=12) (n=145)
+  pedestrian [overall]: Lat=0.189m, Long=0.312m, Head=0.234rad (relaxed=0.187rad, rev=8) (n=67)
+================================================================================
+
+✓ Evaluation completed successfully!
+```
+
+## Troubleshooting
+
+### Issue: No cases found
+
+**Check:**
+1. Verify `path_depth` is set correctly (1 or 2)
+2. Ensure directory structure matches the expected format
+3. Check that level1 directory names match between det_path and gt_path
+
+### Issue: Some cases are skipped
+
+**Check:**
+1. Each case must have `txt_results/` subdirectory (for detections)
+2. Each case must have `labels/` subdirectory (for ground truth)
+3. Frame names must match between detections and labels
+
+### Issue: GT case directory not found
+
+**Check:**
+1. Level1 directory names must be identical in both paths
+2. Case names must be identical in both paths
+3. Path structure must be consistent
+
+## Benefits of 2-Level Structure
+
+1. **Better Organization**: Group related test cases by dataset or configuration
+2. **Flexible Comparison**: Compare models across different datasets easily
+3. **Scalability**: Handle large numbers of test cases more efficiently
+4. **Backward Compatible**: Existing 1-level structures continue to work
+
+## Migration from 1-Level to 2-Level
+
+If you have existing 1-level structure and want to migrate:
+
+```bash
+# Original structure
+/data/detections/case1/txt_results/
+/data/detections/case2/txt_results/
+
+# Create 2-level structure
+mkdir -p /data/detections_2level/dataset_A
+mv /data/detections/case1 /data/detections_2level/dataset_A/
+mv /data/detections/case2 /data/detections_2level/dataset_A/
+
+# Update config
+# path_depth: 1 → path_depth: 2
+# det_path: /data/detections → /data/detections_2level
+```

eval_tools/docs/TWO_LEVEL_PATH_SUPPORT.md

@@ -0,0 +1,228 @@
+# Two-Level Path Support for Model Evaluation
+
+## Overview
+
+The evaluation system now supports both 1-level and 2-level directory structures for detection results and ground truth labels. This allows for more flexible organization of test data.
+
+## Directory Structures
+
+### 1-Level Structure (Default)
+
+```
+det_root/
+    case1/
+        txt_results/
+            frame001.txt
+            frame002.txt
+    case2/
+        txt_results/
+            frame001.txt
+
+gt_root/
+    case1/
+        labels/
+            frame001.txt
+            frame002.txt
+    case2/
+        labels/
+            frame001.txt
+```
+
+### 2-Level Structure
+
+```
+det_root/
+    level1_dir/
+        case1/
+            txt_results/
+                frame001.txt
+                frame002.txt
+        case2/
+            txt_results/
+                frame001.txt
+    level2_dir/
+        case3/
+            txt_results/
+                frame001.txt
+
+gt_root/
+    level1_dir/
+        case1/
+            labels/
+                frame001.txt
+                frame002.txt
+        case2/
+            labels/
+                frame001.txt
+    level2_dir/
+        case3/
+            labels/
+                frame001.txt
+```
+
+## Configuration
+
+### YAML Configuration File
+
+Add the `path_depth` parameter to your config file:
+
+```yaml
+dataset:
+  det_path: "/path/to/detection/results"
+  gt_path: "/path/to/ground/truth"
+  path_depth: 2  # Set to 1 for 1-level, 2 for 2-level structure
+```
+
+**Example for 1-level structure:**
+```yaml
+dataset:
+  det_path: "/data1/dongying/Mono3d/G1M3/CNCAP_results/mono3d/evalset_roi0"
+  gt_path: "/mnt/mono3d/xdzhu_data/Mono3d/Testdata"
+  path_depth: 1  # Default
+```
+
+**Example for 2-level structure:**
+```yaml
+dataset:
+  det_path: "/data1/dongying/Mono3d/G1M3/CNCAP_results/mono3d"
+  gt_path: "/mnt/mono3d/xdzhu_data/Mono3d/Mono3d_4face_2m_g1m3/driving_png"
+  path_depth: 2
+```
+
+### Command-Line Arguments
+
+You can also specify the path depth via command line:
+
+```bash
+# 1-level structure (default)
+python eval_tools/core/eval.py \
+  --config eval_tools/configs/eval_config_mono3d.yaml
+
+# 2-level structure
+python eval_tools/core/eval.py \
+  --config eval_tools/configs/eval_config_mono3d.yaml \
+  --path-depth 2
+
+# Or without config file
+python eval_tools/core/eval.py \
+  --det-path /path/to/detections \
+  --gt-path /path/to/labels \
+  --path-depth 2 \
+  --output-dir results
+```
+
+## How It Works
+
+### 1-Level Mode (`path_depth=1`)
+
+1. Scans `det_root` for all subdirectories (cases)
+2. For each case, looks for `txt_results/` subdirectory
+3. Matches with corresponding `gt_root/case/labels/` directory
+
+### 2-Level Mode (`path_depth=2`)
+
+1. Scans `det_root` for all subdirectories (level1 directories)
+2. For each level1 directory, scans for case subdirectories
+3. For each case, looks for `txt_results/` subdirectory
+4. Matches with corresponding `gt_root/level1/case/labels/` directory
+
+**Important:** The level1 directory names must match between detection and ground truth paths.
+
+## Model Comparison Script
+
+The comparison script automatically inherits the `path_depth` setting from the config files:
+
+```bash
+bash eval_tools/model_comparison/compare_models_with_common_matches.sh
+```
+
+The script will:
+1. Read `path_depth` from `eval_config_mono3d.yaml` for Model 1
+2. Read `path_depth` from `eval_config_yolov5s.yaml` for Model 2
+3. Evaluate both models with their respective path structures
+4. Compare results using common matches
+
+## Examples
+
+### Example 1: Evaluating with 2-level structure
+
+```bash
+# Update your config file
+cat > eval_tools/configs/eval_config_2level.yaml << EOF
+dataset:
+  det_path: "/data/results/all_models"
+  gt_path: "/data/ground_truth/all_datasets"
+  path_depth: 2
+
+image:
+  width: 1920
+  height: 1080
+
+# ... other settings ...
+EOF
+
+# Run evaluation
+python eval_tools/core/eval.py --config eval_tools/configs/eval_config_2level.yaml
+```
+
+### Example 2: Comparing models with different path structures
+
+```yaml
+# Model 1 config (1-level)
+dataset:
+  det_path: "/data/model1/results"
+  gt_path: "/data/gt"
+  path_depth: 1
+
+# Model 2 config (2-level)
+dataset:
+  det_path: "/data/model2/results"
+  gt_path: "/data/gt_organized"
+  path_depth: 2
+```
+
+Both models can be compared even with different directory structures.
+
+## Backward Compatibility
+
+- If `path_depth` is not specified, it defaults to `1` (1-level structure)
+- All existing config files and scripts continue to work without modification
+- The system automatically detects and handles both structures
+
+## Troubleshooting
+
+### Issue: "GT case directory not found"
+
+**Cause:** Level1 directory names don't match between detection and ground truth paths.
+
+**Solution:** Ensure that the intermediate directory names are identical:
+```
+det_root/dataset_A/case1/  ← "dataset_A" must match
+gt_root/dataset_A/case1/   ← "dataset_A" must match
+```
+
+### Issue: "No image pairs found"
+
+**Cause:** Incorrect `path_depth` setting.
+
+**Solution:**
+- Check your actual directory structure
+- Set `path_depth: 1` for `root/case/txt_results`
+- Set `path_depth: 2` for `root/level1/case/txt_results`
+
+### Issue: Cases are being skipped
+
+**Cause:** Missing `txt_results/` or `labels/` subdirectories.
+
+**Solution:** Verify that each case directory contains:
+- Detection: `case/txt_results/*.txt`
+- Ground truth: `case/labels/*.txt`
+
+## Implementation Details
+
+The changes are implemented in:
+- `eval_tools/evaluator/evaluator.py`: Modified `load_data_from_paths()` method
+- `eval_tools/core/eval.py`: Added `--path-depth` argument and config support
+- `eval_tools/configs/*.yaml`: Added `path_depth` parameter
+
+The implementation maintains full backward compatibility while adding support for 2-level structures.

eval_tools/docs/VEHICLE_3D_METRICS_BUG_ANALYSIS.md

@@ -0,0 +1,194 @@
+# 车辆 3D 指标异常分析与修复
+
+**问题发现日期**：2026-03-09  
+**涉及文件**：`eval_tools/evaluator/metrics_3d.py`、`eval_tools/evaluator/evaluator.py`
+
+---
+
+## 一、问题现象
+
+在 deeplearning vs deploy 模型对比评测（CNCAP 数据集，ROI1）中发现以下异常：
+
+| 指标 | deeplearning | deploy | 相对差异 |
+|---|---|---|---|
+| **车辆纵向相对误差（Overall）** | 0.1209 | 0.0711 | -41.20% |
+
+然而，将 overall 分解为各纵向分段后，两模型指标几乎相同（差异 < 1%）：
+
+| 区间 | LongRel (dl) | LongRel (dep) | 差异 |
+|---|---|---|---|
+| 0-10m | 0.2883 | 0.2842 | -1.41% |
+| 10-20m | 0.0328 | 0.0328 | -0.07% |
+| 20-30m | 0.0408 | 0.0410 | +0.27% |
+| … | … | … | … |
+
+**典型特征**：overall 与各分段差异巨大，但各分段两模型差距极小。
+
+---
+
+## 二、问题根因：三个 Bug
+
+### Bug 1（主因）：面中心 z 坐标用于相对误差计算和分段路由
+
+**位置**：`metrics_3d.py::add_sample()` 和 `evaluator.py::_process_frame_3d()`
+
+**问题**：对于车辆，`gt_center` 使用的是 GT 面中心坐标（front/back/left/right face）。在计算纵向相对误差和分段路由时直接用该 z 值：
+
+```python
+# 修复前
+longitudinal_distance = gt_center[2]          # 面中心 z（可能趋近 0 或为负）
+gt_depth = max(abs(gt_center[2]), 1e-6)       # 面中心 z 作为分母
+longitudinal_relative_error = longitudinal_error / gt_depth
+```
+
+**后果**：
+- 近距离侧面/后面标注的面中心 z 可能 ≤ 0（面中心在相机平面附近甚至后方）
+- 当 `face_z ≈ 0` 时，`gt_depth = 1e-6`，`longitudinal_relative_error` 爆炸（可达数万）
+- 当 `face_z ≤ 0` 时，`_get_longitudinal_distance_range()` 返回 `None`，样本**不进任何 `long_*` 分段**，但仍留在 `overall`
+
+**量化影响**（修复前）：
+
+| 模型 | overall 样本数 | long_* 总样本数 | 未分段数 | overall 均值 | long_* 加权均值 |
+|---|---|---|---|---|---|
+| deeplearning | 5744 | 5737 | 7 | 0.0711 | 0.0574 |
+| deploy | 5755 | 5731 | **24** | **0.1207** | 0.0577 |
+
+deploy 命中更多面中心 z 异常样本（24 vs 7），被误判为性能更差。
+
+---
+
+### Bug 2：evaluator.py 与 metrics_3d.py 参考点不一致
+
+**位置**：`evaluator.py::_process_frame_3d()` 的 `save_detailed_matches` 代码块
+
+**问题**：`detailed_3d_matches.json` 中存储的 `gt_center_3d` 和 `distance` 字段使用了整体中心而非面中心，与 `metrics_3d.py` 不一致。
+
+**后果**：`compare_models_with_common_matches.sh` 整条流水线基于 `detailed_3d_matches.json`，修复 `metrics_3d.py` 对这条路径完全无效。
+
+---
+
+### Bug 3（次要）：面中心哨兵值 [-1.0, -1.0, -1.0] 未过滤
+
+**位置**：`metrics_3d.py::_get_vehicle_face_center()` 和 `evaluator.py` 中相应逻辑
+
+**问题背景**：GT 标注中，当某面不可见时（`is_visible_from_camera = 0`），该面的三维坐标被设为哨兵值 -1.0：
+
+```
+"3d_right": [-1.0, -1.0, -1.0, -1.0, -1.0, -1.0, 0.0, 0.0]
+              x3d   y3d   z3d  alpha   xc    yc  score  is_visible=0
+```
+
+**代码只检查 `face_data is not None`，不检查可见性标志**，导致：
+- `gt_center = [-1.0, -1.0, -1.0]` 被当成真实坐标
+- `longitudinal_error = |det_z - (-1.0)| ≈ 8~18m`（虚假巨大误差）
+- `lateral_error = |det_x - (-1.0)| ≈ 2~5m`（虚假巨大误差）
+
+**现象**：修复 Bug 1 后，0-10m 分段出现新的异常——deeplearning 比 deploy **差 24%**（纵向误差），但该差异仅来自哨兵值命中数量不同（dl: 15个，deploy: 5个）。
+
+---
+
+## 三、修复方案
+
+### 修复 Bug 1 + Bug 2：使用整体中心 z 进行分段路由和相对误差计算
+
+**`metrics_3d.py`**（`add_sample()` 方法）：
+
+```python
+# 修复后：使用整体中心 z 作为分段路由和相对误差分母
+gt_whole_z = gt['3d_info']['center'][2]
+gt_depth = max(abs(gt_whole_z), 1e-6)
+longitudinal_relative_error = longitudinal_error / gt_depth
+
+longitudinal_distance = gt['3d_info']['center'][2]  # 整体中心 z
+```
+
+**`evaluator.py`**（`_process_frame_3d()` 方法）：
+
+```python
+# 修复后
+gt_whole_z = gt['3d_info']['center'][2]
+gt_depth = max(abs(gt_whole_z), 1e-6)
+longitudinal_relative_error = longitudinal_error / gt_depth
+
+'distance': {
+    'longitudinal': float(gt_whole_z),   # 用整体中心 z 做分段路由
+    'lateral': float(gt_center[0])
+}
+```
+
+---
+
+### 修复 Bug 3：过滤哨兵值面数据
+
+**`metrics_3d.py`**（`_get_vehicle_face_center()` 方法）：
+
+```python
+face_data = gt_3d_info['faces'].get(normalized_face_type)
+# 检查 is_visible_from_camera (face_data[7]) 和哨兵值
+if face_data is None or (len(face_data) >= 8 and face_data[7] == 0) or face_data[2] <= 0:
+    return gt_3d_info['center']  # fallback 到整体中心
+return face_data[:3]
+```
+
+**`evaluator.py`**（`_process_frame_3d()` 方法）：
+
+```python
+face_valid = (
+    face_data is not None and
+    face_data[2] > 0 and
+    not (len(face_data) >= 8 and face_data[7] == 0)
+)
+gt_center = face_data[:3] if face_valid else gt['3d_info']['center']
+```
+
+---
+
+## 四、修复效果
+
+### Overall 指标对比
+
+| 版本 | deeplearning | deploy | 差异 | 说明 |
+|---|---|---|---|---|
+| 原始（有 Bug） | 0.1209 | 0.0711 | -41.20% | 假性差异 |
+| 修复 Bug1+Bug2 | 0.0594 | 0.0535 | -9.91% | 改善但仍偏 |
+| 修复全部 Bug | **0.0547** | **0.0521** | **-4.72%** | 合理差异 |
+
+### 0-10m 分段指标对比
+
+| 版本 | Long 差异 | Lat 差异 | 说明 |
+|---|---|---|---|
+| 修复 Bug1+Bug2 后 | -25.14% | -24.42% | 哨兵值污染 |
+| 修复全部 Bug 后 | **-14.0%** | **-3.85%** | 合理（Lat 持平） |
+
+---
+
+## 五、设计建议
+
+1. **参考点一致性原则**：`metrics_3d.py` 与 `evaluator.py` 生成的 `detailed_3d_matches.json` 必须使用相同的参考点逻辑，否则两条评测路径产生不同结果。
+
+2. **分段路由 vs 误差计算解耦**：分段路由应始终使用整体目标中心 z（稳定、不受面可见性影响），误差计算可使用面中心（业务语义正确）。
+
+3. **哨兵值防御**：任何使用 GT 面数据的地方，必须先检查 `is_visible_from_camera`（第 8 维）和 `face_z > 0`，防止哨兵值 `-1.0` 污染计算。
+
+4. **`compare_models_with_common_matches.sh` 的数据流**：该脚本 **不经过** `metrics_3d.py`，而是完全基于 `detailed_3d_matches.json` 重新聚合。因此 `evaluator.py` 中写入 JSON 的字段计算逻辑必须与 `metrics_3d.py` 保持严格一致。
+
+---
+
+## 六、GT 面数据格式说明
+
+```
+face_data = [x3d, y3d, z3d, alpha, xc, yc, score, is_visible_from_camera]
+索引:          0    1    2     3    4    5    6            7
+
+is_visible_from_camera:
+  1 = 该面在相机视野中可见，(x3d, y3d, z3d) 为有效坐标
+  0 = 该面不可见，(x3d, y3d, z3d) 填充哨兵值 -1.0
+```
+
+GT 面数据来源参考 CLAUDE.md → "3D Label Format (47 dimensions per line)"：
+- dim15~22: front face
+- dim23~30: rear face  
+- dim31~38: left face
+- dim39~46: right face
+
+每组最后一维 `is_visible_from_camera` 即为可见性标志。

eval_tools/docs/det_format.json

@@ -0,0 +1,698 @@
+{
+  "0": {
+    "type": "0",
+    "type_name": "vehicle",
+    "score": "0.9326733946800232",
+    "roi_id": "0",
+    "box2d": [
+      "331.8140869140625",
+      "561.717529296875",
+      "586.11767578125",
+      "717.2656860351562"
+    ],
+    "xyzlhwyaw": [
+      "-4.009787559509277",
+      "0.926640510559082",
+      "11.942784309387207",
+      "4.302463054656982",
+      "1.562669038772583",
+      "1.921964168548584",
+      "0.4575636684894562"
+    ],
+    "face_cls": "front",
+    "cut_cls": "0",
+    "cut_cls_name": "nocut"
+  },
+  "1": {
+    "type": "0",
+    "type_name": "vehicle",
+    "score": "0.9156705737113953",
+    "roi_id": "0",
+    "box2d": [
+      "100.69953918457031",
+      "564.6636352539062",
+      "242.32601928710938",
+      "715.1654052734375"
+    ],
+    "xyzlhwyaw": [
+      "-6.425538063049316",
+      "0.8240152597427368",
+      "7.582361698150635",
+      "4.374484062194824",
+      "1.5695128440856934",
+      "1.9310325384140015",
+      "0.7510586380958557"
+    ],
+    "face_cls": "front",
+    "cut_cls": "0",
+    "cut_cls_name": "nocut"
+  },
+  "2": {
+    "type": "0",
+    "type_name": "vehicle",
+    "score": "0.9065446853637695",
+    "roi_id": "0",
+    "box2d": [
+      "472.00469970703125",
+      "549.619140625",
+      "716.2294311523438",
+      "703.78857421875"
+    ],
+    "xyzlhwyaw": [
+      "-2.9637675285339355",
+      "0.8307619094848633",
+      "14.574830055236816",
+      "4.347611904144287",
+      "1.6718649864196777",
+      "1.915984869003296",
+      "0.5434975624084473"
+    ],
+    "face_cls": "front",
+    "cut_cls": "0",
+    "cut_cls_name": "nocut"
+  },
+  "3": {
+    "type": "0",
+    "type_name": "vehicle",
+    "score": "0.9034286141395569",
+    "roi_id": "0",
+    "box2d": [
+      "24.04340934753418",
+      "543.9300537109375",
+      "103.02033233642578",
+      "706.6632690429688"
+    ],
+    "xyzlhwyaw": [
+      "-10.928886413574219",
+      "0.8560043573379517",
+      "8.215371131896973",
+      "4.361935615539551",
+      "1.6075832843780518",
+      "1.9334548711776733",
+      "0.9796556830406189"
+    ],
+    "face_cls": "tail",
+    "cut_cls": "2",
+    "cut_cls_name": "cutout"
+  },
+  "4": {
+    "type": "2",
+    "type_name": "bicycle",
+    "score": "0.8614475131034851",
+    "roi_id": "0",
+    "box2d": [
+      "1036.5008544921875",
+      "620.2924194335938",
+      "1273.9559326171875",
+      "938.4365234375"
+    ],
+    "xyzlhwyaw": [
+      "0.6904441118240356",
+      "0.7800710797309875",
+      "5.036780834197998",
+      "1.619232416152954",
+      "1.1915533542633057",
+      "0.6803989410400391",
+      "-1.1483073234558105"
+    ],
+    "face_cls": "whole",
+    "cut_cls": "-1",
+    "cut_cls_name": "none"
+  },
+  "5": {
+    "type": "2",
+    "type_name": "bicycle",
+    "score": "0.8564509153366089",
+    "roi_id": "0",
+    "box2d": [
+      "781.7294311523438",
+      "616.6049194335938",
+      "1012.9938354492188",
+      "873.3040161132812"
+    ],
+    "xyzlhwyaw": [
+      "-0.3779735267162323",
+      "0.9046416282653809",
+      "6.767831325531006",
+      "1.599935531616211",
+      "1.189906120300293",
+      "0.6568350791931152",
+      "-0.9946095943450928"
+    ],
+    "face_cls": "whole",
+    "cut_cls": "-1",
+    "cut_cls_name": "none"
+  },
+  "6": {
+    "type": "2",
+    "type_name": "bicycle",
+    "score": "0.844305694103241",
+    "roi_id": "0",
+    "box2d": [
+      "664.0404052734375",
+      "626.8478393554688",
+      "819.7224731445312",
+      "821.651123046875"
+    ],
+    "xyzlhwyaw": [
+      "-1.293802261352539",
+      "0.9600279331207275",
+      "8.032365798950195",
+      "1.6238980293273926",
+      "1.1767773628234863",
+      "0.6766071319580078",
+      "-1.2385268211364746"
+    ],
+    "face_cls": "whole",
+    "cut_cls": "-1",
+    "cut_cls_name": "none"
+  },
+  "7": {
+    "type": "0",
+    "type_name": "vehicle",
+    "score": "0.8426833748817444",
+    "roi_id": "0",
+    "box2d": [
+      "211.8482208251953",
+      "565.5682983398438",
+      "396.64324951171875",
+      "717.7545776367188"
+    ],
+    "xyzlhwyaw": [
+      "-5.606781482696533",
+      "0.8764306902885437",
+      "9.388605117797852",
+      "4.418740749359131",
+      "1.546727180480957",
+      "1.9304075241088867",
+      "0.8216855525970459"
+    ],
+    "face_cls": "front",
+    "cut_cls": "0",
+    "cut_cls_name": "nocut"
+  },
+  "8": {
+    "type": "4",
+    "type_name": "roadblock",
+    "score": "0.8416956067085266",
+    "roi_id": "0",
+    "box2d": [
+      "1598.6632080078125",
+      "656.9686279296875",
+      "1647.850341796875",
+      "769.2286987304688"
+    ],
+    "xyzlhwyaw": [
+      "-1",
+      "-1",
+      "-1",
+      "-1",
+      "-1",
+      "-1",
+      "-1"
+    ],
+    "face_cls": "none",
+    "cut_cls": "-1",
+    "cut_cls_name": "none"
+  },
+  "9": {
+    "type": "0",
+    "type_name": "vehicle",
+    "score": "0.8231446146965027",
+    "roi_id": "0",
+    "box2d": [
+      "1834.783203125",
+      "528.106689453125",
+      "1919.451904296875",
+      "648.326416015625"
+    ],
+    "xyzlhwyaw": [
+      "11.375432968139648",
+      "0.6048312187194824",
+      "10.385001182556152",
+      "4.3299241065979",
+      "1.6334497928619385",
+      "1.9212439060211182",
+      "-2.5305209159851074"
+    ],
+    "face_cls": "front",
+    "cut_cls": "1",
+    "cut_cls_name": "cutin"
+  },
+  "10": {
+    "type": "0",
+    "type_name": "vehicle",
+    "score": "0.8213036060333252",
+    "roi_id": "0",
+    "box2d": [
+      "1158.3846435546875",
+      "546.5782470703125",
+      "1567.0697021484375",
+      "746.7042846679688"
+    ],
+    "xyzlhwyaw": [
+      "2.3088035583496094",
+      "0.7716936469078064",
+      "9.184979438781738",
+      "4.231507778167725",
+      "1.5341267585754395",
+      "1.8410688638687134",
+      "-2.5499801635742188"
+    ],
+    "face_cls": "left",
+    "cut_cls": "0",
+    "cut_cls_name": "nocut"
+  },
+  "11": {
+    "type": "0",
+    "type_name": "vehicle",
+    "score": "0.7833787202835083",
+    "roi_id": "0",
+    "box2d": [
+      "1763.1807861328125",
+      "545.0386352539062",
+      "1840.17529296875",
+      "604.3870849609375"
+    ],
+    "xyzlhwyaw": [
+      "22.13104820251465",
+      "0.8831890821456909",
+      "22.32408905029297",
+      "4.351836204528809",
+      "1.5954947471618652",
+      "1.9008938074111938",
+      "-2.7008376121520996"
+    ],
+    "face_cls": "left",
+    "cut_cls": "0",
+    "cut_cls_name": "nocut"
+  },
+  "12": {
+    "type": "0",
+    "type_name": "vehicle",
+    "score": "0.7422913312911987",
+    "roi_id": "0",
+    "box2d": [
+      "1657.364501953125",
+      "542.6961669921875",
+      "1714.572265625",
+      "611.0717163085938"
+    ],
+    "xyzlhwyaw": [
+      "14.675742149353027",
+      "0.7690245509147644",
+      "21.358781814575195",
+      "4.2856221199035645",
+      "1.6538622379302979",
+      "1.9027550220489502",
+      "1.6300535202026367"
+    ],
+    "face_cls": "front",
+    "cut_cls": "0",
+    "cut_cls_name": "nocut"
+  },
+  "13": {
+    "type": "0",
+    "type_name": "vehicle",
+    "score": "0.7374464273452759",
+    "roi_id": "0",
+    "box2d": [
+      "1338.939697265625",
+      "475.69091796875",
+      "1645.552734375",
+      "641.1358032226562"
+    ],
+    "xyzlhwyaw": [
+      "4.492400646209717",
+      "0.47527050971984863",
+      "12.608644485473633",
+      "4.7827959060668945",
+      "2.110551357269287",
+      "2.027017116546631",
+      "-2.5330119132995605"
+    ],
+    "face_cls": "left",
+    "cut_cls": "0",
+    "cut_cls_name": "nocut"
+  },
+  "14": {
+    "type": "0",
+    "type_name": "vehicle",
+    "score": "0.7201020121574402",
+    "roi_id": "0",
+    "box2d": [
+      "1792.420166015625",
+      "539.8333740234375",
+      "1877.2738037109375",
+      "620.86767578125"
+    ],
+    "xyzlhwyaw": [
+      "14.51260757446289",
+      "0.7203327417373657",
+      "15.318050384521484",
+      "4.332417964935303",
+      "1.5717623233795166",
+      "1.922214150428772",
+      "-2.566664934158325"
+    ],
+    "face_cls": "front",
+    "cut_cls": "1",
+    "cut_cls_name": "cutin"
+  },
+  "15": {
+    "type": "2",
+    "type_name": "bicycle",
+    "score": "0.7104725241661072",
+    "roi_id": "0",
+    "box2d": [
+      "1308.8502197265625",
+      "642.7114868164062",
+      "1452.3756103515625",
+      "927.2647705078125"
+    ],
+    "xyzlhwyaw": [
+      "1.6400929689407349",
+      "0.7745605111122131",
+      "4.959465980529785",
+      "1.5869669914245605",
+      "1.1547799110412598",
+      "0.6512892246246338",
+      "-0.8955580592155457"
+    ],
+    "face_cls": "whole",
+    "cut_cls": "-1",
+    "cut_cls_name": "none"
+  },
+  "16": {
+    "type": "8",
+    "type_name": "plate",
+    "score": "0.6805570125579834",
+    "roi_id": "0",
+    "box2d": [
+      "1316.4764404296875",
+      "792.7472534179688",
+      "1388.9088134765625",
+      "833.673828125"
+    ],
+    "xyzlhwyaw": [
+      "-1",
+      "-1",
+      "-1",
+      "-1",
+      "-1",
+      "-1",
+      "-1"
+    ],
+    "face_cls": "none",
+    "cut_cls": "-1",
+    "cut_cls_name": "none"
+  },
+  "17": {
+    "type": "2",
+    "type_name": "bicycle",
+    "score": "0.6681439876556396",
+    "roi_id": "0",
+    "box2d": [
+      "1400.7972412109375",
+      "617.092529296875",
+      "1517.8599853515625",
+      "818.230712890625"
+    ],
+    "xyzlhwyaw": [
+      "2.5161986351013184",
+      "0.8316869735717773",
+      "6.546628952026367",
+      "1.6171674728393555",
+      "1.1810991764068604",
+      "0.6666929721832275",
+      "-0.9673029184341431"
+    ],
+    "face_cls": "whole",
+    "cut_cls": "-1",
+    "cut_cls_name": "none"
+  },
+  "18": {
+    "type": "2",
+    "type_name": "bicycle",
+    "score": "0.6465133428573608",
+    "roi_id": "0",
+    "box2d": [
+      "961.9541015625",
+      "608.6699829101562",
+      "1143.5264892578125",
+      "833.4479370117188"
+    ],
+    "xyzlhwyaw": [
+      "0.39372718334198",
+      "0.8915371298789978",
+      "7.160281181335449",
+      "1.6071021556854248",
+      "1.1925380229949951",
+      "0.6709716320037842",
+      "-1.0681982040405273"
+    ],
+    "face_cls": "whole",
+    "cut_cls": "-1",
+    "cut_cls_name": "none"
+  },
+  "19": {
+    "type": "9",
+    "type_name": "wheel",
+    "score": "0.6345217823982239",
+    "roi_id": "0",
+    "box2d": [
+      "1873.107666015625",
+      "599.4207763671875",
+      "1907.8341064453125",
+      "647.875"
+    ],
+    "xyzlhwyaw": [
+      "-1",
+      "-1",
+      "-1",
+      "-1",
+      "-1",
+      "-1",
+      "-1"
+    ],
+    "face_cls": "none",
+    "cut_cls": "-1",
+    "cut_cls_name": "none"
+  },
+  "20": {
+    "type": "0",
+    "type_name": "vehicle",
+    "score": "0.6081775426864624",
+    "roi_id": "0",
+    "box2d": [
+      "891.8013916015625",
+      "519.721923828125",
+      "1173.1517333984375",
+      "624.479248046875"
+    ],
+    "xyzlhwyaw": [
+      "0.34501999616622925",
+      "0.6041234731674194",
+      "23.56036949157715",
+      "5.08701229095459",
+      "2.1216015815734863",
+      "2.116706371307373",
+      "0.39338308572769165"
+    ],
+    "face_cls": "right",
+    "cut_cls": "0",
+    "cut_cls_name": "nocut"
+  },
+  "21": {
+    "type": "9",
+    "type_name": "wheel",
+    "score": "0.5973491668701172",
+    "roi_id": "0",
+    "box2d": [
+      "575.4360961914062",
+      "641.5838623046875",
+      "602.0896606445312",
+      "705.3988037109375"
+    ],
+    "xyzlhwyaw": [
+      "-1",
+      "-1",
+      "-1",
+      "-1",
+      "-1",
+      "-1",
+      "-1"
+    ],
+    "face_cls": "none",
+    "cut_cls": "-1",
+    "cut_cls_name": "none"
+  },
+  "22": {
+    "type": "2",
+    "type_name": "bicycle",
+    "score": "0.5623676776885986",
+    "roi_id": "0",
+    "box2d": [
+      "1163.057861328125",
+      "550.9451904296875",
+      "1555.3499755859375",
+      "754.7815551757812"
+    ],
+    "xyzlhwyaw": [
+      "2.7768852710723877",
+      "0.8014246225357056",
+      "9.575440406799316",
+      "4.110759258270264",
+      "1.5350842475891113",
+      "1.747300148010254",
+      "-2.9602367877960205"
+    ],
+    "face_cls": "whole",
+    "cut_cls": "-1",
+    "cut_cls_name": "none"
+  },
+  "23": {
+    "type": "0",
+    "type_name": "vehicle",
+    "score": "0.5153718590736389",
+    "roi_id": "0",
+    "box2d": [
+      "893.3780517578125",
+      "469.33447265625",
+      "1190.5335693359375",
+      "600.2640991210938"
+    ],
+    "xyzlhwyaw": [
+      "0.46041613817214966",
+      "0.2556573748588562",
+      "24.425113677978516",
+      "5.909259796142578",
+      "2.6501197814941406",
+      "2.329179286956787",
+      "0.4991776645183563"
+    ],
+    "face_cls": "right",
+    "cut_cls": "0",
+    "cut_cls_name": "nocut"
+  },
+  "24": {
+    "type": "4",
+    "type_name": "roadblock",
+    "score": "0.4548017382621765",
+    "roi_id": "0",
+    "box2d": [
+      "1379.612548828125",
+      "734.7178344726562",
+      "1431.492919921875",
+      "880.1078491210938"
+    ],
+    "xyzlhwyaw": [
+      "-1",
+      "-1",
+      "-1",
+      "-1",
+      "-1",
+      "-1",
+      "-1"
+    ],
+    "face_cls": "none",
+    "cut_cls": "-1",
+    "cut_cls_name": "none"
+  },
+  "25": {
+    "type": "9",
+    "type_name": "wheel",
+    "score": "0.4520576000213623",
+    "roi_id": "0",
+    "box2d": [
+      "473.79718017578125",
+      "549.5919189453125",
+      "717.8395385742188",
+      "704.925537109375"
+    ],
+    "xyzlhwyaw": [
+      "-1",
+      "-1",
+      "-1",
+      "-1",
+      "-1",
+      "-1",
+      "-1"
+    ],
+    "face_cls": "none",
+    "cut_cls": "-1",
+    "cut_cls_name": "none"
+  },
+  "26": {
+    "type": "0",
+    "type_name": "vehicle",
+    "score": "0.4148864448070526",
+    "roi_id": "0",
+    "box2d": [
+      "1341.7711181640625",
+      "535.1429443359375",
+      "1590.530517578125",
+      "630.9032592773438"
+    ],
+    "xyzlhwyaw": [
+      "3.6830179691314697",
+      "0.7231490612030029",
+      "11.925402641296387",
+      "4.3222455978393555",
+      "1.6602001190185547",
+      "1.889768362045288",
+      "-2.6474227905273438"
+    ],
+    "face_cls": "left",
+    "cut_cls": "0",
+    "cut_cls_name": "nocut"
+  },
+  "27": {
+    "type": "0",
+    "type_name": "vehicle",
+    "score": "0.3970497250556946",
+    "roi_id": "0",
+    "box2d": [
+      "526.6994018554688",
+      "503.987060546875",
+      "750.5567016601562",
+      "635.5084838867188"
+    ],
+    "xyzlhwyaw": [
+      "-3.048304796218872",
+      "0.5863527655601501",
+      "17.065431594848633",
+      "4.724325656890869",
+      "2.1064600944519043",
+      "2.043461322784424",
+      "0.5458307862281799"
+    ],
+    "face_cls": "front",
+    "cut_cls": "0",
+    "cut_cls_name": "nocut"
+  },
+  "28": {
+    "type": "9",
+    "type_name": "wheel",
+    "score": "0.3437325358390808",
+    "roi_id": "0",
+    "box2d": [
+      "417.23626708984375",
+      "658.1126708984375",
+      "444.47637939453125",
+      "715.48876953125"
+    ],
+    "xyzlhwyaw": [
+      "-1",
+      "-1",
+      "-1",
+      "-1",
+      "-1",
+      "-1",
+      "-1"
+    ],
+    "face_cls": "none",
+    "cut_cls": "-1",
+    "cut_cls_name": "none"
+  }
+}

eval_tools/docs/eval_instruction.md

@@ -0,0 +1,93 @@
+- 真值说明
+
+  - 3D真值类别：{0: vehicle, 1: pedestrian, 2: bike, 3: rider}
+
+    - 车辆3D目标真值示例
+
+    - ```
+        label: 1,
+        x: 2, y: 3, w: 4, h: 5,
+        x3d_ori: 6, y3d_ori: 7, z3d_ori: 8,
+        13d: 9, h3d: 10, w3d: 11,
+        rot_y: 12,
+        xc_ori: 13, yc_ori: 14,
+        xc_ori_d: 15, yc_ori_d: 16,
+        alpha_ori: 17,
+        0: 18,
+        # the first face
+        x3d: 19, y3d: 20, z3d: 21, alpha: 22, xc: 23, yc: 24, score: 25, is occ: 26,
+        # the second face
+        x3d: 27, y3d: 28, z3d: 29, alpha: 30, xc: 31, yc: 32, score: 33, is_occ: 34,
+        # the third face
+        x3d: 35, y3d: 36, z3d: 37, alpha: 38, xc: 39, yc: 40, score: 41, is_occ: 42,
+        # the forth face
+        x3d: 43, y3d: 44, z3d: 45, alpha: 46, xc: 47, yc: 48, score: 49, is_occ: 50
+        ```
+
+    - 说明：如上面所示，车辆目标的3D真值中共50个数值，其中，第一个数值为label（标签格式为数字），2-5是2D框信息（归一化后的中心点和宽高），6-8是中心点坐标，9-11是车辆3D尺寸信息，12为heading信息；19-50是四个面的3D信息，包含面的中心点坐标等，四个面的顺序为：前、后、左、右。
+
+    - 注意：有的车辆目标只有2D框信息，此时，真值为6个数值，最后一位为-1，具体格式为：
+
+        ```
+        label:1, x: 2, y: 3, w: 4, h: 5, -1: 6
+        ```
+
+    - 非车辆类别3D目标真值示例
+
+    - ```
+        label: 1,
+        x: 2, y: 3, w: 4, h: 5,
+        x3d_ori: 6, y3d_ori: 7, z3d_ori: 8,
+        13d: 9, h3d: 10, w3d: 11,
+        rot_y: 12,
+        xc_ori: 13, yc_ori: 14,
+        xc_ori_d: 15, yc_ori_d: 16,
+        alpha_ori: 17,
+        0: 18,
+        ```
+        
+    - 说明：如上面所示，非车辆类别目标的3D真值中共18个数值，其中，第一个数值为label（标签格式为数字），2-5是2D框信息（归一化后的中心点和宽高），6-8是中心点坐标，9-11是3D尺寸信息，12为heading信息；
+
+    - 注意：有的非车辆目标只有2D框信息，此时，真值为6个数值，最后一位为-1，具体格式为：
+
+        ```
+        label:1, x: 2, y: 3, w: 4, h: 5, -1: 6
+        ```
+
+  - 纯2D真值类别：{4: roadblock, 5: head, 6: tsr, 7: guideboard, 8: plate, 9: wheel, 10: tl_border, 11: tl_wick, 12: tl_num, 13: tricycle}
+
+    真值格式为：
+
+    ```
+    label:1, x: 2, y: 3, w: 4, h: 5, -1: 6
+    ```
+
+- 检测结果说明
+  - 3D目标类别
+    - 车辆目标的检测结果格式为：
+
+    - ```
+        vehicle 0.95 368.08 574.17 437.89 617.20 cam -30.14 1.43 68.55 5.52 2.50 2.31 2.70 left
+        ```
+
+    - 说明：如上面所示，车辆目标的检测结果中共15个数值，其中，第一个数值为label（格式为字符串），第二个为置信度，3-6为2D框信息（左上和右下角点的像素坐标），8-10为预测的最近面的中心点坐标，11-13为车辆的3D尺寸信息，14为heading信息，15为预测的最近面类别。
+
+    - 非车辆目标的检测结果格式为：
+
+    - ```
+        pedestrian 0.95 368.08 574.17 437.89 617.20 cam -30.14 1.43 68.55 5.52 2.50 2.31 2.70 whole
+        ```
+
+    - 说明：如上面所示，行人目标的检测结果中共15个数值，其中，第一个数值为label（格式为字符串），第二个为置信度，3-6为2D框信息（左上和右下角点的像素坐标），8-10为预测的3d box中心点坐标，11-13为3d box的3D尺寸信息，14为heading信息，15为填充位（非车辆类别都为whole）。
+
+  - 纯2D类别
+  
+    输出格式：
+
+    plate 0.94246 532.12 203.26 558.73 214.86
+
+评测内容：
+
+2D: 每个类别的召回率，准确率和AP(iou=0.5)，以及总的召回率，准确率和mAP。
+
+3D: 评测每个3D类别的横向测距指标、纵向测距指标和heading偏差。

eval_tools/docs/results_txt_format.md

@@ -0,0 +1,13 @@
+2D classes: 
+class_name confidence x1 y1 x2 y2
+
+example:
+tl_border 0.22863 988.54 498.59 997.36 506.38
+wheel 0.92207 501.83 737.12 557.60 890.73
+
+3D classes:
+class_name confidence x1 y1 x2 y2 cam x3d_face y3d_face z3d_face l h w yaw face_type
+
+example:
+vehicle 0.96654 206.40 516.75 698.04 887.78 cam -2.65 0.62 3.79 4.60 1.56 1.98 -1.54 tail
+pedestrian 0.12859 1103.98 549.13 1110.26 564.57 cam 9.41 1.55 110.09 0.89 1.67 0.68 -1.58 whole

eval_tools/evaluator/__init__.py

@@ -0,0 +1,20 @@
+"""
+Evaluation toolkit for YOLOv5-3D model outputs.
+"""
+
+from .parser import GroundTruthParser, DetectionParser
+from .matcher import Matcher2D
+from .metrics_2d import Metrics2D
+from .metrics_3d import Metrics3D
+from .evaluator import Evaluator
+from .roi_processor import ROIProcessor
+
+__all__ = [
+    'GroundTruthParser',
+    'DetectionParser',
+    'Matcher2D',
+    'Metrics2D',
+    'Metrics3D',
+    'Evaluator',
+    'ROIProcessor'
+]

eval_tools/evaluator/matcher.py

@@ -0,0 +1,155 @@
+"""
+2D matching module for ground truth and detection boxes.
+"""
+import numpy as np
+
+
+class Matcher2D:
+    """Match 2D bounding boxes between ground truth and detections."""
+    
+    def __init__(self, iou_threshold=0.5):
+        """
+        Initialize matcher.
+        
+        Args:
+            iou_threshold: float, IoU threshold for matching
+        """
+        self.iou_threshold = iou_threshold
+    
+    def compute_iou(self, box1, box2):
+        """
+        Compute IoU between two boxes.
+        
+        Args:
+            box1: [x1, y1, x2, y2]
+            box2: [x1, y1, x2, y2]
+            
+        Returns:
+            float, IoU value
+        """
+        x1 = max(box1[0], box2[0])
+        y1 = max(box1[1], box2[1])
+        x2 = min(box1[2], box2[2])
+        y2 = min(box1[3], box2[3])
+        
+        if x2 < x1 or y2 < y1:
+            return 0.0
+        
+        intersection = (x2 - x1) * (y2 - y1)
+        area1 = (box1[2] - box1[0]) * (box1[3] - box1[1])
+        area2 = (box2[2] - box2[0]) * (box2[3] - box2[1])
+        union = area1 + area2 - intersection
+        
+        return intersection / union if union > 0 else 0.0
+
+    @staticmethod
+    def _normalize_roi_id(roi_id):
+        """Normalize ROI identifiers like 'roi0'/'0' to plain numeric strings."""
+        if roi_id is None:
+            return None
+
+        roi_id_str = str(roi_id).strip().lower()
+        if roi_id_str.startswith('roi'):
+            roi_id_str = roi_id_str[3:]
+        return roi_id_str or None
+
+    def compute_pair_iou(self, gt, det):
+        """Compute IoU for one GT/detection pair, honoring ROI-specific GT boxes when present."""
+        gt_boxes_by_roi = gt.get('bbox_2d_by_roi')
+        det_box = det.get('bbox_2d')
+        if det_box is None:
+            return 0.0
+
+        if not gt_boxes_by_roi:
+            gt_box = gt.get('bbox_2d')
+            return self.compute_iou(gt_box, det_box) if gt_box is not None else 0.0
+
+        det_roi_id = self._normalize_roi_id(det.get('roi_id'))
+        if det_roi_id is not None:
+            gt_box = gt_boxes_by_roi.get(det_roi_id)
+            return self.compute_iou(gt_box, det_box) if gt_box is not None else 0.0
+
+        return max((self.compute_iou(gt_box, det_box) for gt_box in gt_boxes_by_roi.values()), default=0.0)
+    
+    def compute_iou_matrix(self, gts, dets):
+        """
+        Compute IoU matrix between all GT and detection pairs.
+        
+        Args:
+            gts: list of ground truth dicts
+            dets: list of detection dicts
+            
+        Returns:
+            numpy array of shape (len(gts), len(dets))
+        """
+        if len(gts) == 0 or len(dets) == 0:
+            return np.zeros((len(gts), len(dets)))
+        
+        iou_matrix = np.zeros((len(gts), len(dets)))
+        
+        for i, gt in enumerate(gts):
+            for j, det in enumerate(dets):
+                iou_matrix[i, j] = self.compute_pair_iou(gt, det)
+        
+        return iou_matrix
+    
+    def match(self, gts, dets, class_id):
+        """
+        Match detections to ground truths using greedy algorithm.
+        
+        Args:
+            gts: list of ground truth dicts for a specific class
+            dets: list of detection dicts for a specific class
+            class_id: int, class ID to match
+            
+        Returns:
+            dict with keys:
+                - matches: list of (gt_idx, det_idx, iou) tuples
+                - unmatched_gts: list of unmatched GT indices
+                - unmatched_dets: list of unmatched detection indices
+        """
+        # Filter by class
+        gts_filtered = [gt for gt in gts if gt['label'] == class_id]
+        dets_filtered = [det for det in dets if det['label'] == class_id]
+        
+        # Sort detections by confidence (highest first)
+        det_indices = np.argsort([-det['confidence'] for det in dets_filtered])
+        dets_sorted = [dets_filtered[i] for i in det_indices]
+        
+        # Compute IoU matrix
+        iou_matrix = self.compute_iou_matrix(gts_filtered, dets_sorted)
+        
+        matches = []
+        matched_gt_indices = set()
+        matched_det_indices = set()
+        
+        # Greedy matching: for each detection (sorted by confidence), find best GT
+        for det_idx in range(len(dets_sorted)):
+            best_iou = 0.0
+            best_gt_idx = -1
+            
+            for gt_idx in range(len(gts_filtered)):
+                if gt_idx in matched_gt_indices:
+                    continue
+                
+                iou = iou_matrix[gt_idx, det_idx]
+                if iou >= self.iou_threshold and iou > best_iou:
+                    best_iou = iou
+                    best_gt_idx = gt_idx
+            
+            if best_gt_idx >= 0:
+                matches.append((best_gt_idx, det_idx, best_iou))
+                matched_gt_indices.add(best_gt_idx)
+                matched_det_indices.add(det_idx)
+        
+        # Find unmatched GTs and detections
+        unmatched_gts = [i for i in range(len(gts_filtered)) if i not in matched_gt_indices]
+        unmatched_dets = [i for i in range(len(dets_sorted)) if i not in matched_det_indices]
+        
+        return {
+            'matches': matches,
+            'unmatched_gts': unmatched_gts,
+            'unmatched_dets': unmatched_dets,
+            'gts_filtered': gts_filtered,
+            'dets_sorted': dets_sorted
+        }

eval_tools/evaluator/metrics_2d.py

@@ -0,0 +1,348 @@
+"""2D metrics calculation module.
+
+Supports optional per-distance-range evaluation for 3D-capable classes
+(vehicle, pedestrian, bicycle, rider) that carry z3d / x3d coordinates.
+
+Storage per detection:  (confidence, is_tp, z, x)
+  z = GT z3d for TPs;  detection's own predicted z3d for FPs.
+  x = GT x3d for TPs;  detection's own predicted x3d for FPs.
+  z, x = None when no 3D output (2D-only classes).
+
+Two evaluation views are produced when ``distance_ranges`` is configured:
+  per_class_by_distance          - all lateral positions + longitudinal bins
+  per_class_by_distance_lat_roi  - lateral pre-filter (``lateral_roi``) + longitudinal bins
+                                   only produced when ``lateral_roi`` is also set.
+"""
+import numpy as np
+from collections import defaultdict
+
+
+class Metrics2D:
+    """Calculate 2D detection metrics (Precision, Recall, AP, mAP).
+
+    Parameters
+    ----------
+    num_classes : int
+    distance_ranges : list of [z_min, z_max] pairs (metres), optional
+        Longitudinal distance bins for per-range evaluation.
+    lateral_roi : [x_min, x_max] pair (metres), optional
+        Lateral region-of-interest filter applied on top of ``distance_ranges``
+        to produce a second "lat-filtered" section in the summary.
+        e.g. [-15, 15] keeps only targets within 15 m of the vehicle centre line.
+    """
+
+    def __init__(self, num_classes=14, distance_ranges=None, lateral_roi=None, coord_system='camera'):
+        self.num_classes = num_classes
+        self.distance_ranges = distance_ranges   # [[z0,z1], [z1,z2], ...]
+        self.lateral_roi = lateral_roi           # [x_min, x_max] or None
+        if coord_system not in ('camera', 'ego'):
+            raise ValueError(f"Unsupported coord_system: {coord_system}")
+        self.coord_system = coord_system
+
+        # Per detection: (confidence, is_tp, z, x)
+        self.all_detections = defaultdict(list)
+
+        # Per GT: (z, x) -- both None for 2D-only classes
+        self.all_gt_coords = defaultdict(list)
+
+    def _get_lateral_axis(self):
+        return 0 if self.coord_system == 'camera' else 1
+
+    def _get_longitudinal_axis(self):
+        return 2 if self.coord_system == 'camera' else 0
+
+    # ------------------------------------------------------------------
+    # Internal helpers
+    # ------------------------------------------------------------------
+
+    def _gt_count(self, class_id, z_range=None, x_range=None):
+        """Count GTs matching optional z_range and/or x_range filters."""
+        count = 0
+        for z, x in self.all_gt_coords[class_id]:
+            if z_range is not None:
+                if z is None or not (z_range[0] <= z < z_range[1]):
+                    continue
+            if x_range is not None:
+                if x is None or not (x_range[0] <= x < x_range[1]):
+                    continue
+            count += 1
+        return count
+
+    def _filter_dets(self, class_id, z_range=None, x_range=None):
+        """Return detections matching optional z_range and/or x_range filters.
+
+        TPs use matched GT (z, x); FPs use detection's own predicted (z, x).
+        Detections without coordinates are excluded from any range query.
+        """
+        dets = self.all_detections[class_id]
+        if z_range is None and x_range is None:
+            return dets
+        result = []
+        for conf, is_tp, z, x in dets:
+            if z_range is not None:
+                if z is None or not (z_range[0] <= z < z_range[1]):
+                    continue
+            if x_range is not None:
+                if x is None or not (x_range[0] <= x < x_range[1]):
+                    continue
+            result.append((conf, is_tp, z, x))
+        return result
+
+    # ------------------------------------------------------------------
+    # Data ingestion
+    # ------------------------------------------------------------------
+
+    def add_image_results(self, match_result, all_gts, all_dets, class_id):
+        """Add matching results from one image.
+
+        Args:
+            match_result : dict from Matcher2D.match()
+            all_gts      : unused (kept for API compatibility)
+            all_dets     : unused
+            class_id     : int
+        """
+        gts_filtered = match_result['gts_filtered']
+        dets_sorted  = match_result['dets_sorted']
+        matches      = match_result['matches']
+
+        # TP: map det_idx -> (z, x) from matched GT
+        det_to_gt_coords = {}
+        for gt_idx, det_idx, _iou in matches:
+            gt  = gts_filtered[gt_idx]
+            d3d = gt.get('3d_info')
+            if d3d is not None:
+                long_axis = self._get_longitudinal_axis()
+                lat_axis = self._get_lateral_axis()
+                z, x = d3d['center'][long_axis], d3d['center'][lat_axis]
+            else:
+                z, x = None, None
+            det_to_gt_coords[det_idx] = (z, x)
+
+        # Record all GT coordinates (for GT-count denominator)
+        for gt in gts_filtered:
+            d3d = gt.get('3d_info')
+            if d3d is not None:
+                long_axis = self._get_longitudinal_axis()
+                lat_axis = self._get_lateral_axis()
+                self.all_gt_coords[class_id].append((d3d['center'][long_axis], d3d['center'][lat_axis]))
+            else:
+                self.all_gt_coords[class_id].append((None, None))
+
+        # Record each detection
+        for det_idx, det in enumerate(dets_sorted):
+            is_tp = det_idx in det_to_gt_coords
+            if is_tp:
+                z, x = det_to_gt_coords[det_idx]
+            else:
+                # FP: use detection's own predicted 3D coordinates
+                d3d = det.get('3d_info')
+                if d3d is not None:
+                    long_axis = self._get_longitudinal_axis()
+                    lat_axis = self._get_lateral_axis()
+                    z, x = d3d['center'][long_axis], d3d['center'][lat_axis]
+                else:
+                    z, x = None, None
+            self.all_detections[class_id].append((det['confidence'], is_tp, z, x))
+
+    # ------------------------------------------------------------------
+    # Core metric computation (all accept optional spatial filters)
+    # ------------------------------------------------------------------
+
+    def compute_precision_recall(self, class_id, z_range=None, x_range=None):
+        """Compute precision-recall curve for a class.
+
+        Returns (precisions, recalls, confidences) as numpy arrays.
+        """
+        if class_id not in self.all_detections or not self.all_detections[class_id]:
+            return np.array([]), np.array([]), np.array([])
+
+        detections = self._filter_dets(class_id, z_range=z_range, x_range=x_range)
+        gt_count   = self._gt_count(class_id, z_range=z_range, x_range=x_range)
+
+        if not detections:
+            return np.array([]), np.array([]), np.array([])
+
+        detections  = sorted(detections, key=lambda d: d[0], reverse=True)
+        confidences = np.array([d[0] for d in detections])
+        is_tp       = np.array([d[1] for d in detections])
+
+        tp_cumsum = np.cumsum(is_tp)
+        fp_cumsum = np.cumsum(~is_tp)
+        precisions = tp_cumsum / (tp_cumsum + fp_cumsum)
+        recalls    = tp_cumsum / max(gt_count, 1)
+
+        return precisions, recalls, confidences
+
+    def compute_ap(self, class_id, method='voc2010', z_range=None, x_range=None):
+        """Compute Average Precision for a class."""
+        precisions, recalls, _ = self.compute_precision_recall(
+            class_id, z_range=z_range, x_range=x_range)
+
+        if len(precisions) == 0:
+            return 0.0
+
+        if method == 'voc2010':
+            ap = 0.0
+            for t in np.linspace(0, 1, 11):
+                mask = recalls >= t
+                ap += (float(np.max(precisions[mask])) if mask.any() else 0.0) / 11.0
+            return ap
+        elif method == 'coco':
+            mrec = np.concatenate(([0.], recalls, [1.]))
+            mpre = np.concatenate(([0.], precisions, [0.]))
+            for i in range(mpre.size - 1, 0, -1):
+                mpre[i - 1] = np.maximum(mpre[i - 1], mpre[i])
+            i = np.where(mrec[1:] != mrec[:-1])[0]
+            return float(np.sum((mrec[i + 1] - mrec[i]) * mpre[i + 1]))
+        else:
+            raise ValueError(f"Unknown AP method: {method}")
+
+    def compute_map(self, method='voc2010', z_range=None, x_range=None):
+        """Compute mAP across all classes."""
+        return float(np.mean([
+            self.compute_ap(c, method, z_range=z_range, x_range=x_range)
+            for c in range(self.num_classes)
+        ]))
+
+    def get_class_metrics(self, class_id, conf_threshold=0.5, z_range=None, x_range=None):
+        """Get Precision/Recall/F1/TP/FP/FN at conf_threshold for one class."""
+        gt_count = self._gt_count(class_id, z_range=z_range, x_range=x_range)
+
+        if class_id not in self.all_detections:
+            return dict(precision=0., recall=0., f1_score=0., tp=0, fp=0, fn=gt_count)
+
+        dets     = self._filter_dets(class_id, z_range=z_range, x_range=x_range)
+        filtered = [(conf, is_tp) for conf, is_tp, *_ in dets if conf >= conf_threshold]
+
+        if not filtered:
+            return dict(precision=0., recall=0., f1_score=0., tp=0, fp=0, fn=gt_count)
+
+        tp = sum(1 for _, is_tp in filtered if is_tp)
+        fp = len(filtered) - tp
+        fn = gt_count - tp
+
+        p  = tp / (tp + fp) if (tp + fp) > 0 else 0.
+        r  = tp / (tp + fn) if (tp + fn) > 0 else 0.
+        f1 = 2 * p * r / (p + r) if (p + r) > 0 else 0.
+        return dict(precision=p, recall=r, f1_score=f1, tp=tp, fp=fp, fn=fn)
+
+    def get_overall_metrics(self, conf_threshold=0.5, z_range=None, x_range=None):
+        """Aggregate Precision/Recall/F1 across all classes."""
+        total_tp = total_fp = total_fn = 0
+        for c in range(self.num_classes):
+            m = self.get_class_metrics(c, conf_threshold, z_range=z_range, x_range=x_range)
+            total_tp += m['tp']
+            total_fp += m['fp']
+            total_fn += m['fn']
+
+        p  = total_tp / (total_tp + total_fp) if (total_tp + total_fp) > 0 else 0.
+        r  = total_tp / (total_tp + total_fn) if (total_tp + total_fn) > 0 else 0.
+        f1 = 2 * p * r / (p + r) if (p + r) > 0 else 0.
+        return dict(precision=p, recall=r, f1_score=f1,
+                    tp=total_tp, fp=total_fp, fn=total_fn)
+
+    # ------------------------------------------------------------------
+    # Summary builder
+    # ------------------------------------------------------------------
+
+    def _build_dist_table(self, conf_threshold, ap_method, x_range=None):
+        """Build {class_name: {range_key: metrics_dict}} for given x_range filter.
+
+        Args:
+            x_range : None  -> all lateral positions (full-lateral view)
+                      tuple -> restricted lateral ROI
+        """
+        from .parser import GroundTruthParser
+
+        table = {}
+        for class_id in range(self.num_classes):
+            class_name = GroundTruthParser.CLASS_NAMES.get(class_id, f"class_{class_id}")
+
+            # Skip classes with no 3D coordinates
+            if not any(z is not None for z, _ in self.all_gt_coords.get(class_id, [])):
+                continue
+
+            rows = {}
+            for r in self.distance_ranges:
+                z_range   = (r[0], r[1])
+                range_key = f"{r[0]}-{r[1]}m"
+
+                gt_count = self._gt_count(class_id, z_range=z_range, x_range=x_range)
+                metrics  = self.get_class_metrics(
+                    class_id, conf_threshold, z_range=z_range, x_range=x_range)
+                ap = self.compute_ap(
+                    class_id, ap_method, z_range=z_range, x_range=x_range)
+
+                rows[range_key] = dict(
+                    precision=metrics['precision'],
+                    recall=metrics['recall'],
+                    f1_score=metrics['f1_score'],
+                    ap=ap,
+                    num_gt=gt_count,
+                    tp=metrics['tp'],
+                    fp=metrics['fp'],
+                    fn=metrics['fn'],
+                )
+            table[class_name] = rows
+        return table
+
+    def get_summary(self, conf_threshold=0.5, ap_method='voc2010'):
+        """Return the complete evaluation summary dict.
+
+        Keys always present:
+          per_class    - overall per-class metrics (all objects, no spatial filter)
+          overall      - micro-aggregated overall metrics
+
+        Keys present when ``distance_ranges`` is configured:
+          per_class_by_distance           - longitudinal bins, all lateral positions
+          per_class_by_distance_lat_roi   - longitudinal bins inside lateral_roi
+                                            (only when ``lateral_roi`` is also set)
+          lateral_roi                     - the configured [x_min, x_max] value
+        """
+        from .parser import GroundTruthParser
+
+        summary = {'per_class': {}, 'overall': {}}
+
+        # Per-class overall (no spatial filter)
+        for class_id in range(self.num_classes):
+            class_name = GroundTruthParser.CLASS_NAMES.get(class_id, f"class_{class_id}")
+            metrics = self.get_class_metrics(class_id, conf_threshold)
+            ap      = self.compute_ap(class_id, ap_method)
+            summary['per_class'][class_name] = dict(
+                precision=metrics['precision'],
+                recall=metrics['recall'],
+                f1_score=metrics['f1_score'],
+                ap=ap,
+                num_gt=self._gt_count(class_id),
+                num_det=len(self.all_detections.get(class_id, [])),
+                tp=metrics['tp'],
+                fp=metrics['fp'],
+                fn=metrics['fn'],
+            )
+
+        # Overall aggregated
+        overall = self.get_overall_metrics(conf_threshold)
+        summary['overall'] = dict(
+            precision=overall['precision'],
+            recall=overall['recall'],
+            f1_score=overall['f1_score'],
+            map=self.compute_map(ap_method),
+            num_classes=self.num_classes,
+            tp=overall['tp'],
+            fp=overall['fp'],
+            fn=overall['fn'],
+        )
+
+        if self.distance_ranges:
+            # View 1: all lateral positions + longitudinal bins
+            summary['per_class_by_distance'] = self._build_dist_table(
+                conf_threshold, ap_method, x_range=None)
+
+            # View 2: lateral ROI + longitudinal bins
+            if self.lateral_roi is not None:
+                x_range = (self.lateral_roi[0], self.lateral_roi[1])
+                summary['per_class_by_distance_lat_roi'] = self._build_dist_table(
+                    conf_threshold, ap_method, x_range=x_range)
+                summary['lateral_roi'] = self.lateral_roi
+
+        return summary

eval_tools/evaluator/metrics_3d.py

@@ -0,0 +1,410 @@
+"""
+3D metrics calculation module.
+"""
+import numpy as np
+from collections import defaultdict
+
+from ..class_config import CLASSES_3D as _CLASSES_3D
+
+
+def normalize_angle(angle):
+    """
+    Normalize angle to [-π, π].
+    
+    Args:
+        angle: float, angle in radians
+        
+    Returns:
+        float, normalized angle
+    """
+    while angle > np.pi:
+        angle -= 2 * np.pi
+    while angle < -np.pi:
+        angle += 2 * np.pi
+    return angle
+
+
+class Metrics3D:
+    """Calculate 3D detection metrics (lateral/longitudinal errors, heading error)."""
+    
+    # 3D class IDs — sourced from eval_tools/class_config.py
+    CLASSES_3D = _CLASSES_3D  # vehicle, bus, truck, tanker, unknown, pedestrian, bicycle, motorcyclist, tricycle
+    VEHICLE_LARGE_CLASS_NAME = 'vehicle_large'
+    VEHICLE_SMALL_CLASS_NAME = 'vehicle_small'
+    VEHICLE_SIZE_CLASS_NAMES = [VEHICLE_LARGE_CLASS_NAME, VEHICLE_SMALL_CLASS_NAME]
+    
+    # Face mapping for vehicle class
+    FACE_MAPPING = {
+        'front': [18, 19, 20],  # Indices in GT values for front face center
+        'back': [26, 27, 28],
+        'rear': [26, 27, 28],   # Alias for back (some models use 'rear')
+        'tail': [26, 27, 28],   # Alias for back (some models use 'tail')
+        'left': [34, 35, 36],
+        'right': [42, 43, 44]
+    }
+    
+    def __init__(self, distance_ranges=None, lateral_distance_ranges=None, heading_tolerance='strict',
+                 coord_system='camera', vehicle_size_split=None):
+        """Initialize 3D metrics calculator.
+        
+        Args:
+            distance_ranges: list of [min, max] longitudinal distance ranges in meters (z-axis).
+                           e.g., [[0, 30], [30, 60], [60, 100]]
+                           If None, only overall statistics are computed.
+            lateral_distance_ranges: list of [min, max] lateral distance ranges in meters (x-axis).
+                           e.g., [[-10, -5], [-5, 0], [0, 5], [5, 10]]
+                           If None, no lateral distance grouping is used.
+            heading_tolerance: str, heading error calculation mode
+                           'strict': Standard calculation (default)
+                           'relaxed': Consider 180° symmetry (for symmetric objects)
+                           'both': Calculate both strict and relaxed metrics
+            vehicle_size_split: dict, optional vehicle GT size split config:
+                           {
+                               'enabled': True,
+                               'large_length_threshold': 6.0,
+                               'large_height_threshold': 3.0,
+                           }
+        """
+        if coord_system not in ('camera', 'ego'):
+            raise ValueError(f"Unsupported coord_system: {coord_system}")
+        self.coord_system = coord_system
+        vehicle_size_split = vehicle_size_split or {}
+        self.vehicle_size_split_enabled = vehicle_size_split.get('enabled', True)
+        self.vehicle_large_length_threshold = float(
+            vehicle_size_split.get('large_length_threshold', 6.0)
+        )
+        self.vehicle_large_height_threshold = float(
+            vehicle_size_split.get('large_height_threshold', 3.0)
+        )
+
+        # Store errors for each class and distance range
+        # Format: {class_id: {range_key: {'lateral': [...], 'longitudinal': [...], 'heading': [...], 'heading_relaxed': [...], 'heading_reversal': [...]}}}
+        self.errors = defaultdict(lambda: defaultdict(lambda: {
+            'lateral': [],
+            'longitudinal': [],
+            'longitudinal_relative': [],
+            'heading': [],
+            'heading_relaxed': [],
+            'heading_reversal': []
+        }))
+        
+        # Distance ranges configuration
+        self.distance_ranges = distance_ranges
+        self.lateral_distance_ranges = lateral_distance_ranges
+        
+        # Heading tolerance mode
+        self.heading_tolerance = heading_tolerance
+        
+        # Build range keys
+        self.range_keys = ['overall']
+        if self.distance_ranges:
+            self.range_keys.extend([f"long_{r[0]}-{r[1]}m" for r in self.distance_ranges])
+        if self.lateral_distance_ranges:
+            self.range_keys.extend([f"lat_{r[0]}-{r[1]}m" for r in self.lateral_distance_ranges])
+
+    def _get_lateral_axis(self):
+        return 0 if self.coord_system == 'camera' else 1
+
+    def _get_longitudinal_axis(self):
+        return 2 if self.coord_system == 'camera' else 0
+
+    def _get_vehicle_size_class_name(self, gt):
+        """Classify vehicle GT as large/small by GT 3D box dimensions."""
+        dimensions = (gt.get('3d_info') or {}).get('dimensions')
+        if dimensions is None or len(dimensions) < 2:
+            return None
+
+        length = float(dimensions[0])
+        height = float(dimensions[1])
+        if length > self.vehicle_large_length_threshold and height > self.vehicle_large_height_threshold:
+            return self.VEHICLE_LARGE_CLASS_NAME
+        return self.VEHICLE_SMALL_CLASS_NAME
+
+    def _get_storage_class_keys(self, gt, class_id):
+        """Return all metric buckets that should receive this sample."""
+        class_keys = [class_id]
+        if class_id == 0 and self.vehicle_size_split_enabled:
+            vehicle_size_class = self._get_vehicle_size_class_name(gt)
+            if vehicle_size_class is not None:
+                class_keys.append(vehicle_size_class)
+        return class_keys
+
+    def _append_error_sample(self, class_key, range_key, lateral_error, longitudinal_error,
+                             longitudinal_relative_error, heading_error_strict,
+                             heading_error_relaxed, is_reversal):
+        self.errors[class_key][range_key]['lateral'].append(lateral_error)
+        self.errors[class_key][range_key]['longitudinal'].append(longitudinal_error)
+        self.errors[class_key][range_key]['longitudinal_relative'].append(longitudinal_relative_error)
+        self.errors[class_key][range_key]['heading'].append(heading_error_strict)
+        self.errors[class_key][range_key]['heading_relaxed'].append(heading_error_relaxed)
+        self.errors[class_key][range_key]['heading_reversal'].append(1 if is_reversal else 0)
+    
+    def add_sample(self, gt, det, class_id):
+        """
+        Add a matched GT-detection pair for 3D metric calculation.
+        
+        Args:
+            gt: dict, ground truth with 3d_info
+            det: dict, detection with 3d_info
+            class_id: int, class ID
+        """
+        if class_id not in self.CLASSES_3D:
+            return
+        
+        if gt['3d_info'] is None or det['3d_info'] is None:
+            return
+        
+        # Get GT 3D center based on class type
+        if class_id == 0:  # Vehicle class
+            # Use face-specific center
+            gt_center = self._get_vehicle_face_center(gt['3d_info'], det['3d_info']['face_type'])
+        else:
+            # Non-vehicle: use overall 3D box center
+            gt_center = gt['3d_info']['center']
+        
+        # Get detection 3D center
+        det_center = det['3d_info']['center']
+        
+        lateral_axis = self._get_lateral_axis()
+        longitudinal_axis = self._get_longitudinal_axis()
+
+        # Calculate lateral / longitudinal error in the configured coordinate system.
+        lateral_error = abs(det_center[lateral_axis] - gt_center[lateral_axis])
+        longitudinal_error = abs(det_center[longitudinal_axis] - gt_center[longitudinal_axis])
+        
+        # Calculate longitudinal relative error (relative to GT depth).
+        # Use whole-object center z as the reference depth so that face centers
+        # with near-zero or negative z (e.g. side faces at close range) don't
+        # produce astronomically large relative errors.
+        gt_whole_longitudinal = gt['3d_info']['center'][longitudinal_axis]
+        gt_depth = max(abs(gt_whole_longitudinal), 1e-6)
+        longitudinal_relative_error = longitudinal_error / gt_depth
+        
+        # Calculate heading error
+        gt_rot = gt['3d_info']['rotation']
+        det_rot = det['3d_info']['rotation']
+        heading_error_strict = abs(normalize_angle(det_rot - gt_rot))
+        
+        # Calculate relaxed heading error (considering 180° symmetry)
+        heading_error_relaxed = heading_error_strict
+        is_reversal = False
+        if heading_error_strict > np.pi - 0.2:  # Close to 180° (within ~170°)
+            heading_error_relaxed = np.pi - heading_error_strict
+            is_reversal = True
+        
+        # Get distance for range classification.
+        # Always use whole-object center z so that samples with negative/near-zero
+        # face center z (side faces at close range) still fall into the correct
+        # longitudinal bucket instead of being silently dropped from all segments
+        # while remaining in 'overall' with enormous relative errors.
+        longitudinal_distance = gt['3d_info']['center'][longitudinal_axis]
+        lateral_distance = gt_center[lateral_axis]
+        
+        class_keys = self._get_storage_class_keys(gt, class_id)
+
+        # Store errors in overall
+        for class_key in class_keys:
+            self._append_error_sample(
+                class_key,
+                'overall',
+                lateral_error,
+                longitudinal_error,
+                longitudinal_relative_error,
+                heading_error_strict,
+                heading_error_relaxed,
+                is_reversal,
+            )
+        
+        # Store errors by longitudinal distance range if configured
+        if self.distance_ranges:
+            range_key = self._get_longitudinal_distance_range(longitudinal_distance)
+            if range_key:
+                for class_key in class_keys:
+                    self._append_error_sample(
+                        class_key,
+                        range_key,
+                        lateral_error,
+                        longitudinal_error,
+                        longitudinal_relative_error,
+                        heading_error_strict,
+                        heading_error_relaxed,
+                        is_reversal,
+                    )
+        
+        # Store errors by lateral distance range if configured
+        if self.lateral_distance_ranges:
+            range_key = self._get_lateral_distance_range(lateral_distance)
+            if range_key:
+                for class_key in class_keys:
+                    self._append_error_sample(
+                        class_key,
+                        range_key,
+                        lateral_error,
+                        longitudinal_error,
+                        longitudinal_relative_error,
+                        heading_error_strict,
+                        heading_error_relaxed,
+                        is_reversal,
+                    )
+    
+    def _get_vehicle_face_center(self, gt_3d_info, face_type):
+        """
+        Get vehicle face center from GT based on predicted face type.
+        
+        Args:
+            gt_3d_info: dict, GT 3D info with faces
+            face_type: str, predicted face type (front/back/rear/tail/left/right)
+            
+        Returns:
+            list, [x3d, y3d, z3d] of the specified face center
+        """
+        if gt_3d_info['faces'] is None:
+            # Fallback to overall center if no face info
+            return gt_3d_info['center']
+        
+        # Normalize face_type: rear/tail -> back
+        normalized_face_type = face_type.lower()
+        if normalized_face_type in ['rear', 'tail']:
+            normalized_face_type = 'back'
+        
+        face_data = gt_3d_info['faces'].get(normalized_face_type)
+        # face_data[7] is is_visible_from_camera; when 0 the x3d/y3d/z3d are set to
+        # the sentinel -1.0, which must not be used as a real coordinate.
+        if face_data is None or (len(face_data) >= 8 and face_data[7] == 0) or face_data[2] <= 0:
+            # Fallback to overall center if face not found or not visible
+            return gt_3d_info['center']
+        
+        # Extract x3d, y3d, z3d from face data (first 3 elements)
+        return face_data[:3]
+    
+    def _get_longitudinal_distance_range(self, distance):
+        """Get longitudinal distance range key for a given distance.
+        
+        Args:
+            distance: float, longitudinal distance in meters (z-axis)
+            
+        Returns:
+            str, range key (e.g., 'long_0-30m') or None if not in any range
+        """
+        if not self.distance_ranges:
+            return None
+        
+        for range_min, range_max in self.distance_ranges:
+            if range_min <= distance < range_max:
+                return f"long_{range_min}-{range_max}m"
+        
+        return None
+    
+    def _get_lateral_distance_range(self, distance):
+        """Get lateral distance range key for a given distance.
+        
+        Args:
+            distance: float, lateral distance in meters (x-axis)
+            
+        Returns:
+            str, range key (e.g., 'lat_-10--5m') or None if not in any range
+        """
+        if not self.lateral_distance_ranges:
+            return None
+        
+        for range_min, range_max in self.lateral_distance_ranges:
+            if range_min <= distance < range_max:
+                return f"lat_{range_min}-{range_max}m"
+        
+        return None
+    
+    def compute_statistics(self, class_id, range_key='overall'):
+        """
+        Compute statistical metrics for a class and distance range.
+        
+        Args:
+            class_id: int | str, class ID or synthetic class key
+            range_key: str, distance range key (e.g., 'overall', '0-30m')
+            
+        Returns:
+            dict with mean, median, std, 90th percentile for each error type
+        """
+        if class_id not in self.errors or range_key not in self.errors[class_id] or \
+           len(self.errors[class_id][range_key]['lateral']) == 0:
+            result = {
+                'lateral_error': {'mean': 0, 'median': 0, 'std': 0, 'percentile_90': 0},
+                'longitudinal_error': {'mean': 0, 'median': 0, 'std': 0, 'percentile_90': 0},
+                'longitudinal_relative_error': {'mean': 0, 'median': 0, 'std': 0, 'percentile_90': 0},
+                'heading_error': {'mean': 0, 'median': 0, 'std': 0, 'percentile_90': 0},
+                'num_samples': 0
+            }
+            if self.heading_tolerance in ['relaxed', 'both']:
+                result['heading_error_relaxed'] = {'mean': 0, 'median': 0, 'std': 0, 'percentile_90': 0}
+                result['reversal_count'] = 0
+                result['reversal_percentage'] = 0.0
+            return result
+        
+        lateral = np.array(self.errors[class_id][range_key]['lateral'])
+        longitudinal = np.array(self.errors[class_id][range_key]['longitudinal'])
+        longitudinal_relative = np.array(self.errors[class_id][range_key]['longitudinal_relative'])
+        heading = np.array(self.errors[class_id][range_key]['heading'])
+        heading_relaxed = np.array(self.errors[class_id][range_key]['heading_relaxed'])
+        heading_reversal = np.array(self.errors[class_id][range_key]['heading_reversal'])
+        
+        def stats(arr):
+            return {
+                'mean': float(np.mean(arr)),
+                'median': float(np.median(arr)),
+                'std': float(np.std(arr)),
+                'percentile_90': float(np.percentile(arr, 90))
+            }
+        
+        result = {
+            'lateral_error': stats(lateral),
+            'longitudinal_error': stats(longitudinal),
+            'longitudinal_relative_error': stats(longitudinal_relative),
+            'num_samples': len(lateral)
+        }
+        
+        # Add heading error based on tolerance mode
+        if self.heading_tolerance == 'strict':
+            result['heading_error'] = stats(heading)
+        elif self.heading_tolerance == 'relaxed':
+            result['heading_error'] = stats(heading_relaxed)
+            result['reversal_count'] = int(np.sum(heading_reversal))
+            result['reversal_percentage'] = float(np.sum(heading_reversal) / len(heading_reversal) * 100)
+        elif self.heading_tolerance == 'both':
+            result['heading_error'] = stats(heading)
+            result['heading_error_relaxed'] = stats(heading_relaxed)
+            result['reversal_count'] = int(np.sum(heading_reversal))
+            result['reversal_percentage'] = float(np.sum(heading_reversal) / len(heading_reversal) * 100)
+        
+        return result
+    
+    def get_summary(self):
+        """
+        Get complete 3D evaluation summary.
+        
+        Returns:
+            dict with per-class 3D metrics, including distance ranges if configured
+        """
+        from .parser import GroundTruthParser
+        
+        summary = {}
+
+        summary_class_keys = list(self.CLASSES_3D)
+        if self.vehicle_size_split_enabled:
+            summary_class_keys.extend(self.VEHICLE_SIZE_CLASS_NAMES)
+
+        for class_id in summary_class_keys:
+            if isinstance(class_id, str):
+                class_name = class_id
+            else:
+                class_name = GroundTruthParser.CLASS_NAMES.get(class_id, f"class_{class_id}")
+            
+            # Compute statistics for each distance range
+            class_summary = {}
+            for range_key in self.range_keys:
+                stats = self.compute_statistics(class_id, range_key)
+                if stats['num_samples'] > 0 or range_key == 'overall':
+                    class_summary[range_key] = stats
+            
+            if class_summary:
+                summary[class_name] = class_summary
+        
+        return summary

eval_tools/evaluator/parser.py

@@ -0,0 +1,588 @@
+"""
+Data parser for ground truth and detection results.
+Supports both TXT and JSON formats.
+
+TXT GT format:  normalized xywh bbox + 47-dim 3D labels (47-dim label format per CLAUDE.md)
+JSON GT format: absolute pixel box2d, 3d_ori=[x3d,y3d,z3d,l,h,w,rot_y,xc,yc,...], 3d_front/back/left/right=[x3d,y3d,z3d,alpha,xc,yc,score,is_visible]
+
+TXT Det format: class_name conf x1 y1 x2 y2 coord_sys x3d y3d z3d l h w rot_y face_type
+JSON Det format: type/type_name, score, box2d (absolute pixels), xyzlhwyaw=[x3d,y3d,z3d,l,h,w,rot_y], face_cls
+"""
+import json
+import numpy as np
+
+from ..class_config import CLASS_NAMES, CLASS_NAME_TO_ID, CLASSES_3D, FACE_3D_CLASSES
+
+
+def yaw_to_radians(yaw_value, coord_system):
+    """Convert parsed yaw to radians for downstream evaluation."""
+    yaw = float(yaw_value)
+    if coord_system == 'ego':
+        return float(np.deg2rad(yaw))
+    return yaw
+
+
+class GroundTruthParser:
+    """Parse ground truth annotation files."""
+    
+    # Class ID to name mapping — imported from eval_tools/class_config.py
+    CLASS_NAMES = CLASS_NAMES
+
+    # 3D classes — imported from eval_tools/class_config.py
+    CLASSES_3D = CLASSES_3D
+    VALID_COORD_SYSTEMS = {"camera", "ego"}
+    
+    def __init__(self, min_box_size=8, coord_system='camera'):
+        """
+        Initialize ground truth parser.
+        
+        Args:
+            min_box_size: float, minimum bbox width or height in pixels.
+                          Boxes smaller than this will be filtered out.
+                          Default is 8. Should be calculated based on ROI config:
+                          - ROI0 (1920->704): 8 * 1920 / 704 ≈ 21.8
+                          - ROI1 (704->704): 8 * 704 / 704 = 8
+        """
+        self.min_box_size = min_box_size
+        if coord_system not in self.VALID_COORD_SYSTEMS:
+            raise ValueError(f"Unsupported coord_system: {coord_system}")
+        self.coord_system = coord_system
+    
+    def parse_line(self, line, img_width, img_height):
+        """
+        Parse a single line of ground truth annotation.
+        
+        Args:
+            line: str, annotation line
+            img_width: int, image width
+            img_height: int, image height
+            
+        Returns:
+            dict with keys:
+                - label: int
+                - bbox_2d: [x1, y1, x2, y2] in pixel coordinates
+                - has_3d: bool
+                - 3d_info: dict or None
+        """
+        values = [float(x) for x in line.strip().split()]
+        
+        if len(values) < 6:
+            return None
+        
+        label = int(values[0])
+        
+        # Parse 2D bbox (normalized center + width/height to pixel corners)
+        x_center_norm, y_center_norm = values[1], values[2]
+        w_norm, h_norm = values[3], values[4]
+        
+        x_center_px = x_center_norm * img_width
+        y_center_px = y_center_norm * img_height
+        w_px = w_norm * img_width
+        h_px = h_norm * img_height
+        
+        x1 = x_center_px - w_px / 2
+        y1 = y_center_px - h_px / 2
+        x2 = x_center_px + w_px / 2
+        y2 = y_center_px + h_px / 2
+        
+        bbox_2d = [x1, y1, x2, y2]
+        
+        # Filter out small objects based on configured minimum size
+        bbox_width = x2 - x1
+        bbox_height = y2 - y1
+        if bbox_width < self.min_box_size or bbox_height < self.min_box_size:
+            return None
+        
+        # Check if has 3D annotation
+        has_3d = self.is_3d_annotated(values)
+        
+        result = {
+            'label': label,
+            'bbox_2d': bbox_2d,
+            'has_3d': has_3d,
+            '3d_info': None
+        }
+        
+        if has_3d:
+            result['3d_info'] = self._parse_3d_info(values, label)
+        
+        return result
+    
+    def is_3d_annotated(self, values):
+        """Check if the annotation contains 3D information."""
+        if len(values) == 6 and values[5] == -1:
+            return False
+        if len(values) >= 18:
+            return True
+        return False
+    
+    def _parse_3d_info(self, values, label):
+        """Parse 3D information from annotation values."""
+        info = {
+            'center': [values[5], values[6], values[7]],  # x3d_ori, y3d_ori, z3d_ori
+            'dimensions': [values[8], values[9], values[10]],  # l3d, h3d, w3d
+            'rotation': values[11],  # rot_y
+            'faces': None
+        }
+        
+        # For face_3d_classes, parse face information
+        if label in FACE_3D_CLASSES and len(values) == 50:
+            info['faces'] = {
+                'front': values[18:26],  # x3d, y3d, z3d, alpha, xc, yc, score, is_occ
+                'back': values[26:34],
+                'left': values[34:42],
+                'right': values[42:50]
+            }
+        
+        return info
+    
+    def get_class_name(self, label_id):
+        """Get class name from label ID."""
+        return self.CLASS_NAMES.get(label_id, "unknown")
+
+    def _should_filter_negative_id_gt(self, entry, label):
+        """
+        Filter JSON GT objects that should not participate in 3D-class evaluation.
+
+        Rule:
+        - Only applies to 3D classes: vehicle, pedestrian, bicycle, rider
+        - If GT carries an `id` field and id < 0, drop this GT entirely
+        """
+        if label not in self.CLASSES_3D:
+            return False
+
+        object_id = entry.get('id')
+        if object_id is None:
+            return False
+
+        try:
+            return int(object_id) < 0
+        except (ValueError, TypeError):
+            return False
+
+    def parse_gt_json_entry(self, entry, img_width, img_height):
+        """
+        Parse a single entry from a GT JSON file.
+
+        GT JSON entry format:
+            {
+                "type": "0",           # class id string
+                "type_name": "vehicle",
+                "roi_id": "1",
+                "box2d": ["x1","y1","x2","y2"],   # absolute pixel coords
+                "3d_ori": ["x3d","y3d","z3d","l","h","w","rot_y","xc","yc",...,"alpha","flag"],
+                "3d_front": ["x3d","y3d","z3d","alpha","xc","yc","score","is_visible"],
+                "3d_back":  [...],
+                "3d_left":  [...],
+                "3d_right": [...]
+            }
+
+        Args:
+            entry: dict, single GT JSON entry
+            img_width: int, image width (unused for JSON, bbox already in pixels)
+            img_height: int, image height (unused for JSON, bbox already in pixels)
+
+        Returns:
+            dict or None
+        """
+        raw_type = entry.get('type')
+        if raw_type is None or str(raw_type).strip().lower() in ('', 'none', 'null'):
+            return None
+        try:
+            label = int(raw_type)
+        except (ValueError, TypeError):
+            return None
+
+        if self._should_filter_negative_id_gt(entry, label):
+            return None
+
+        box2d = entry.get('box2d', [])
+        if len(box2d) < 4:
+            return None
+        x1, y1, x2, y2 = float(box2d[0]), float(box2d[1]), float(box2d[2]), float(box2d[3])
+        bbox_2d = [x1, y1, x2, y2]
+
+        # Filter small objects
+        if (x2 - x1) < self.min_box_size or (y2 - y1) < self.min_box_size:
+            return None
+
+        # Check whether 3D annotation is present and valid
+        ori_key = '3d_ori_ego' if self.coord_system == 'ego' else '3d_ori'
+        if self.coord_system == 'ego' and ori_key not in entry and '3d_ori' in entry:
+            has_camera_3d = False
+            try:
+                has_camera_3d = len(entry['3d_ori']) >= 7 and float(entry['3d_ori'][0]) != -1
+            except (ValueError, TypeError):
+                has_camera_3d = False
+            if has_camera_3d:
+                raise ValueError(
+                    "GT JSON is missing ego-coordinate fields (3d_ori_ego). "
+                    "Please regenerate ground truth with ego fields before running ego-coordinate evaluation."
+                )
+        d3_ori = entry.get(ori_key)
+        has_3d = False
+        d3_info = None
+        if d3_ori is not None and len(d3_ori) >= 7:
+            # x3d is d3_ori[0]; -1 indicates no 3D annotation
+            try:
+                has_3d = float(d3_ori[0]) != -1
+            except (ValueError, TypeError):
+                has_3d = False
+
+        if has_3d:
+            d3_info = self._parse_3d_info_from_json(entry, label)
+
+        return {
+            'label': label,
+            'bbox_2d': bbox_2d,
+            'has_3d': has_3d,
+            '3d_info': d3_info,
+            'id': entry.get('id'),
+        }
+
+    def _parse_3d_info_from_json(self, entry, label):
+        """Parse 3D information from a JSON GT entry."""
+        ori_key = '3d_ori_ego' if self.coord_system == 'ego' else '3d_ori'
+        d3_ori = entry[ori_key]
+        info = {
+            'center':     [float(d3_ori[0]), float(d3_ori[1]), float(d3_ori[2])],  # x3d, y3d, z3d
+            'dimensions': [float(d3_ori[3]), float(d3_ori[4]), float(d3_ori[5])],  # l, h, w
+            'rotation':    yaw_to_radians(d3_ori[6], self.coord_system),             # rot_y
+            'faces': None,
+            'coord_system': self.coord_system,
+        }
+
+        # Parse face information for face_3d_classes (vehicle, bus, truck, tanker, unknown)
+        face_keys = {'front': '3d_front', 'back': '3d_back', 'left': '3d_left', 'right': '3d_right'}
+        if self.coord_system == 'ego':
+            face_keys = {name: f"{key}_ego" for name, key in face_keys.items()}
+        if label in FACE_3D_CLASSES and all(k in entry for k in face_keys.values()):
+            info['faces'] = {}
+            for face_name, json_key in face_keys.items():
+                face_data = entry[json_key]
+                if len(face_data) >= 8:
+                    info['faces'][face_name] = [float(v) for v in face_data[:8]]
+                else:
+                    info['faces'][face_name] = [float(v) for v in face_data]
+
+        return info
+
+    def parse_gt_json_file(self, file_path, img_width, img_height):
+        """
+        Parse an entire GT JSON file.
+
+        The JSON file is a dict keyed by object index ("0", "1", ...).
+
+        Returns:
+            list of parsed annotation dicts
+        """
+        try:
+            with open(file_path, 'r') as f:
+                data = json.load(f)
+        except FileNotFoundError:
+            print(f"Warning: File not found: {file_path}")
+            return []
+        except json.JSONDecodeError as e:
+            print(f"Warning: JSON decode error in {file_path}: {e}")
+            return []
+
+        if data is None:
+            return []
+
+        if isinstance(data, dict):
+            items = sorted(data.items(), key=lambda item: int(item[0]) if str(item[0]).isdigit() else str(item[0]))
+        elif isinstance(data, list):
+            items = list(enumerate(data))
+        else:
+            print(f"Warning: unsupported GT JSON root type {type(data).__name__} in {file_path}")
+            return []
+
+        annotations = []
+        for key, entry in items:
+            if not isinstance(entry, dict):
+                print(f"Warning: skipping non-dict GT entry at key={key!r} in {file_path}")
+                continue
+            raw_type = entry.get('type')
+            if raw_type is None or str(raw_type).strip().lower() in ('', 'none', 'null'):
+                print(f"Warning: skipping entry with invalid type={raw_type!r} "
+                      f"(key={key!r}) in {file_path}")
+                continue
+            parsed = self.parse_gt_json_entry(entry, img_width, img_height)
+            if parsed is not None:
+                annotations.append(parsed)
+        return annotations
+
+    def parse_file(self, file_path, img_width, img_height):
+        """
+        Parse entire annotation file. Dispatches to JSON or TXT parser based on extension.
+
+        Args:
+            file_path: str, path to annotation file (.txt or .json)
+            img_width: int, image width
+            img_height: int, image height
+
+        Returns:
+            list of parsed annotations
+        """
+        if str(file_path).endswith('.json'):
+            return self.parse_gt_json_file(file_path, img_width, img_height)
+
+        annotations = []
+        try:
+            with open(file_path, 'r') as f:
+                for line in f:
+                    line = line.strip()
+                    if not line:
+                        continue
+                    parsed = self.parse_line(line, img_width, img_height)
+                    if parsed is not None:
+                        annotations.append(parsed)
+        except FileNotFoundError:
+            print(f"Warning: File not found: {file_path}")
+            return []
+
+        return annotations
+
+
+class DetectionParser:
+    """Parse detection result files."""
+    
+    # Class name to ID mapping — imported from eval_tools/class_config.py
+    CLASS_NAME_TO_ID = CLASS_NAME_TO_ID
+
+    # 3D classes — imported from eval_tools/class_config.py
+    CLASSES_3D = CLASSES_3D
+    VALID_COORD_SYSTEMS = {"camera", "ego"}
+    
+    def __init__(self, min_box_size=0, coord_system='camera'):
+        """
+        Initialize detection parser.
+        
+        Args:
+            min_box_size: float, minimum bbox width or height in pixels.
+                          Detections smaller than this will be filtered out.
+                          Should match the GT min_box_size to ensure
+                          symmetric filtering. Default is 0 (no filtering).
+        """
+        self.min_box_size = min_box_size
+        if coord_system not in self.VALID_COORD_SYSTEMS:
+            raise ValueError(f"Unsupported coord_system: {coord_system}")
+        self.coord_system = coord_system
+    
+    def parse_line(self, line):
+        """
+        Parse a single line of detection result.
+        
+        Args:
+            line: str, detection line
+            
+        Returns:
+            dict with keys:
+                - label: int
+                - confidence: float
+                - bbox_2d: [x1, y1, x2, y2] in pixel coordinates
+                - 3d_info: dict or None
+        """
+        parts = line.strip().split()
+        
+        if len(parts) < 6:
+            return None
+        
+        class_name = parts[0]
+        label = self.map_class_name(class_name)
+        confidence = float(parts[1])
+        
+        # 2D bbox
+        x1, y1, x2, y2 = float(parts[2]), float(parts[3]), float(parts[4]), float(parts[5])
+        bbox_2d = [x1, y1, x2, y2]
+        
+        # Filter small detections
+        if self.min_box_size > 0 and ((x2 - x1) < self.min_box_size or (y2 - y1) < self.min_box_size):
+            return None
+        
+        result = {
+            'label': label,
+            'confidence': confidence,
+            'bbox_2d': bbox_2d,
+            '3d_info': None
+        }
+        
+        # Check if this is a 3D class and has 3D info
+        if label in self.CLASSES_3D and len(parts) >= 15:
+            result['3d_info'] = self._parse_3d_info(parts)
+        
+        return result
+    
+    def _parse_3d_info(self, parts):
+        """Parse 3D information from detection parts."""
+        if self.coord_system == 'ego':
+            raise ValueError("TXT detection format does not support ego-coordinate 3D evaluation.")
+        # Format: label conf x1 y1 x2 y2 coord_sys x3d y3d z3d l3d h3d w3d rot_y face_type
+        # Index:   0     1    2  3  4  5     6       7   8   9   10  11  12   13    14
+        
+        # Get face_type and normalize it
+        face_type = parts[14] if len(parts) > 14 else 'whole'
+        # Normalize rear/tail to back for consistency
+        if face_type.lower() in ['rear', 'tail']:
+            face_type = 'back'
+        
+        info = {
+            'center': [float(parts[7]), float(parts[8]), float(parts[9])],  # x3d, y3d, z3d
+            'dimensions': [float(parts[10]), float(parts[11]), float(parts[12])],  # l3d, h3d, w3d
+            'rotation': float(parts[13]),  # rot_y
+            'face_type': face_type,
+            'coord_system': 'camera',
+        }
+        
+        return info
+    
+    def map_class_name(self, name_str):
+        """Map class name string to class ID."""
+        return self.CLASS_NAME_TO_ID.get(name_str.lower(), -1)
+
+    def parse_det_json_entry(self, entry):
+        """
+        Parse a single entry from a detection JSON file.
+
+        Det JSON entry format:
+            {
+                "type": "0",                # class id string
+                "type_name": "vehicle",
+                "score": "0.93",
+                "roi_id": "0",
+                "box2d": ["x1","y1","x2","y2"],   # absolute pixel coords
+                "xyzlhwyaw": ["x3d","y3d","z3d","l","h","w","rot_y"],
+                "face_cls": "front",        # front/tail/rear/left/right/whole/none
+                "cut_cls": "0",
+                "cut_cls_name": "nocut"
+            }
+
+        Returns:
+            dict or None
+        """
+        try:
+            label = int(entry['type'])
+        except (KeyError, ValueError, TypeError):
+            class_name = entry.get('type_name', '')
+            label = self.map_class_name(class_name)
+
+        try:
+            confidence = float(entry['score'])
+        except (KeyError, ValueError, TypeError):
+            confidence = 0.0
+
+        box2d = entry.get('box2d', [])
+        if len(box2d) < 4:
+            return None
+        x1, y1, x2, y2 = float(box2d[0]), float(box2d[1]), float(box2d[2]), float(box2d[3])
+        bbox_2d = [x1, y1, x2, y2]
+
+        # Filter small detections
+        if self.min_box_size > 0 and ((x2 - x1) < self.min_box_size or (y2 - y1) < self.min_box_size):
+            return None
+
+        result = {
+            'label': label,
+            'confidence': confidence,
+            'bbox_2d': bbox_2d,
+            '3d_info': None,
+            'id': entry.get('track_id', entry.get('id')),
+            'roi_id': self._normalize_roi_id(entry.get('roi_id')),
+        }
+
+        # Parse 3D info for 3D classes
+        if label in self.CLASSES_3D:
+            xyz_key = 'xyzlhwyaw_ego' if self.coord_system == 'ego' else 'xyzlhwyaw'
+            xyzlhwyaw = entry.get(xyz_key, [])
+            using_coord_system = self.coord_system
+            if len(xyzlhwyaw) < 7 and self.coord_system == 'ego':
+                has_camera_3d = False
+                camera_xyz = entry.get('xyzlhwyaw', [])
+                try:
+                    has_camera_3d = len(camera_xyz) >= 7 and str(camera_xyz[0]) != '-1'
+                except (ValueError, TypeError):
+                    has_camera_3d = False
+                if has_camera_3d:
+                    raise ValueError(
+                        "Detection JSON is missing ego-coordinate fields (xyzlhwyaw_ego). "
+                        "Please export ego-coordinate detection results before running ego-coordinate evaluation."
+                    )
+            if len(xyzlhwyaw) >= 7 and str(xyzlhwyaw[0]) != '-1':
+                face_type = entry.get('face_cls', 'whole') or 'whole'
+                if face_type.lower() in ('rear', 'tail'):
+                    face_type = 'back'
+                result['3d_info'] = {
+                    'center':     [float(xyzlhwyaw[0]), float(xyzlhwyaw[1]), float(xyzlhwyaw[2])],
+                    'dimensions': [float(xyzlhwyaw[3]), float(xyzlhwyaw[4]), float(xyzlhwyaw[5])],
+                    'rotation':    yaw_to_radians(xyzlhwyaw[6], using_coord_system),
+                    'face_type':   face_type,
+                    'coord_system': using_coord_system,
+                }
+
+        return result
+
+    @staticmethod
+    def _normalize_roi_id(roi_id):
+        """Normalize ROI identifiers like 'roi0'/'0' to plain numeric strings."""
+        if roi_id is None:
+            return None
+
+        roi_id_str = str(roi_id).strip().lower()
+        if roi_id_str.startswith('roi'):
+            roi_id_str = roi_id_str[3:]
+        return roi_id_str or None
+
+    def parse_det_json_file(self, file_path):
+        """
+        Parse an entire detection JSON file.
+
+        The JSON file is a dict keyed by object index ("0", "1", ...).
+
+        Returns:
+            list of parsed detection dicts
+        """
+        try:
+            with open(file_path, 'r') as f:
+                data = json.load(f)
+        except FileNotFoundError:
+            print(f"Warning: File not found: {file_path}")
+            return []
+        except json.JSONDecodeError as e:
+            print(f"Warning: JSON decode error in {file_path}: {e}")
+            return []
+
+        detections = []
+        for key in sorted(data.keys(), key=lambda k: int(k) if k.isdigit() else k):
+            parsed = self.parse_det_json_entry(data[key])
+            if parsed is not None:
+                detections.append(parsed)
+        return detections
+
+    def parse_file(self, file_path):
+        """
+        Parse entire detection file. Dispatches to JSON or TXT parser based on extension.
+
+        Args:
+            file_path: str, path to detection file (.txt or .json)
+
+        Returns:
+            list of parsed detections
+        """
+        if str(file_path).endswith('.json'):
+            return self.parse_det_json_file(file_path)
+
+        detections = []
+        try:
+            with open(file_path, 'r') as f:
+                for line in f:
+                    line = line.strip()
+                    if not line:
+                        continue
+                    parsed = self.parse_line(line)
+                    if parsed is not None:
+                        detections.append(parsed)
+        except FileNotFoundError:
+            print(f"Warning: File not found: {file_path}")
+            return []
+
+        return detections

eval_tools/evaluator/roi_processor.py

@@ -0,0 +1,343 @@
+"""
+ROI (Region of Interest) processor for ground truth labels.
+
+This module handles ROI computation and ground truth filtering/clipping
+to match the training-time ROI processing logic.
+"""
+import numpy as np
+import json
+from pathlib import Path
+
+
+class ROIProcessor:
+    """Process ground truth labels with ROI filtering and clipping."""
+    
+    def __init__(
+        self,
+        calib_root=None,
+        roi_config=None,
+        ori_img_size=(1920, 1080),
+        roi_bottom_offset=0,
+        roi_right_offset=0,
+        roi_use_true_vp_x=False,
+    ):
+        """
+        Initialize ROI processor.
+        
+        Args:
+            calib_root: str or Path, root directory containing calibration files
+            roi_config: dict or list, ROI configuration
+                - If dict: {'mode': 'size', 'width': 1920, 'height': 960} or 
+                           {'mode': 'bounds', 'x1': 0, 'y1': 120, 'x2': 1920, 'y2': 1080}
+                - If list of 2 values [width, height]: ROI size mode
+                - If list of 4 values [x1, y1, x2, y2]: ROI bounds mode
+            ori_img_size: tuple, original image size (width, height)
+            roi_bottom_offset: int, pixels to trim from the bottom edge of the ROI (shifts y2 upward)
+            roi_right_offset: int, pixels to trim from the right edge of the ROI (shifts x2 leftward)
+            roi_use_true_vp_x: bool, use geometric vanishing point X as crop center for ROI1-style crop
+        """
+        self.calib_root = Path(calib_root) if calib_root else None
+        self.roi_config = self._parse_roi_config(roi_config)
+        self.ori_img_size = ori_img_size
+        self.roi_bottom_offset = roi_bottom_offset
+        self.roi_right_offset = roi_right_offset
+        self.roi_use_true_vp_x = roi_use_true_vp_x
+        self.calib_cache = {}  # Cache calibration parameters
+        
+    def _parse_roi_config(self, roi_config):
+        """Parse ROI configuration into standardized format."""
+        if roi_config is None:
+            return None
+            
+        if isinstance(roi_config, dict):
+            return roi_config
+            
+        if isinstance(roi_config, (list, tuple)):
+            if len(roi_config) == 2:
+                return {'mode': 'size', 'width': roi_config[0], 'height': roi_config[1]}
+            elif len(roi_config) == 4:
+                return {'mode': 'bounds', 'x1': roi_config[0], 'y1': roi_config[1], 
+                        'x2': roi_config[2], 'y2': roi_config[3]}
+        
+        raise ValueError(f"Invalid ROI config: {roi_config}")
+    
+    def load_calibration(self, case_name, frame_name=None, level1_name=None):
+        """
+        Load calibration parameters for a case.
+
+        Args:
+            case_name: str, case identifier
+            frame_name: str, optional frame name (if calibration is per-frame)
+            level1_name: str, optional level1 directory name for 2-level path structure
+
+        Returns:
+            dict with calibration parameters: focal_u, focal_v, cu, cv, yaw, pitch, etc.
+        """
+        if self.calib_root is None:
+            return None
+
+        # Try case-level calibration first
+        cache_key = f"{level1_name}/{case_name}" if level1_name else f"{case_name}"
+        if cache_key in self.calib_cache:
+            return self.calib_cache[cache_key]
+
+        # Look for calibration file.
+        # Supported layouts:
+        # - calib_root/level1/case/calib/L2_calib/camera4.json
+        # - calib_root/level1/case/calib/camera4.json
+        # - calib_root/level1/case/calibration.json
+        # - calib_root/case/calib/L2_calib/camera4.json
+        # - calib_root/case/calib/camera4.json
+        # - calib_root/case/calibration.json
+        case_root = self.calib_root / level1_name / case_name if level1_name else self.calib_root / case_name
+        calib_candidates = [
+            case_root / "calib/L2_calib/camera4.json",
+            case_root / "calib/camera4.json",
+            case_root / "calibration.json",
+        ]
+        case_calib_path = next((path for path in calib_candidates if path.exists()), None)
+        if case_calib_path is None:
+            print(f"Warning: Calibration file not found for case {case_name}")
+            return None
+        
+        try:
+            with open(case_calib_path, 'r') as f:
+                calib_data = json.load(f)
+            
+            # Extract relevant parameters
+            calib_params = {
+                'focal_u': calib_data.get('focal_u', calib_data.get('fx')),
+                'focal_v': calib_data.get('focal_v', calib_data.get('fy')),
+                'cu': calib_data.get('cu', calib_data.get('cx')),
+                'cv': calib_data.get('cv', calib_data.get('cy')),
+                'yaw': calib_data.get('yaw', 0.0),
+                'pitch': calib_data.get('pitch', 0.0),
+                'distort_coeffs': calib_data.get('distort_coeffs', [])
+            }
+            
+            self.calib_cache[cache_key] = calib_params
+            return calib_params
+            
+        except Exception as e:
+            print(f"Error loading calibration for {case_name}: {e}")
+            return None
+    
+    def compute_roi(self, calib_params):
+        """
+        Compute ROI bounds based on calibration and configuration.
+        
+        Matches the logic in LoadImages3D / LoadImagesAndLabels3D.
+        
+        Args:
+            calib_params: dict, calibration parameters
+            
+        Returns:
+            tuple: (roi_x1, roi_y1, roi_x2, roi_y2) or None if ROI disabled
+        """
+        if self.roi_config is None:
+            return None
+        
+        oriW, oriH = self.ori_img_size
+        
+        # Compute vanishing point (crop center)
+        fx = calib_params['focal_u']
+        fy = calib_params['focal_v']
+        cx = calib_params['cu']
+        cy = calib_params['cv']
+        c_pitch = calib_params['pitch']
+        c_yaw = calib_params.get('yaw', 0.0)
+        
+        # Vanishing point coordinates
+        vanish_x = cx + fx * np.tan(c_yaw * np.pi / 180)
+        vanish_y = cy - fy * np.tan(c_pitch * np.pi / 180)
+        
+        # ROI0 uses image center X; ROI1 uses the true geometric vanishing point X.
+        crop_center_x = vanish_x if self.roi_use_true_vp_x else oriW // 2
+        crop_center_y = vanish_y
+        
+        if self.roi_config['mode'] == 'size':
+            # ROI defined by [width, height]
+            roi_width = self.roi_config['width']
+            roi_height = self.roi_config['height']
+            
+            roi_x1 = int(crop_center_x - roi_width / 2.0)
+            roi_y1 = int(crop_center_y - roi_height / 2.0)
+            roi_x2 = roi_x1 + roi_width - self.roi_right_offset
+            roi_y2 = roi_y1 + roi_height - self.roi_bottom_offset
+            
+        elif self.roi_config['mode'] == 'bounds':
+            # ROI defined by [x1, y1, x2, y2]
+            roi_x1 = self.roi_config['x1']
+            roi_y1 = self.roi_config['y1']
+            roi_x2 = self.roi_config['x2']
+            roi_y2 = self.roi_config['y2']
+        else:
+            return None
+        
+        # Clip to image bounds
+        roi_x1 = max(0, roi_x1)
+        roi_y1 = max(0, roi_y1)
+        roi_x2 = min(oriW, roi_x2)
+        roi_y2 = min(oriH, roi_y2)
+        
+        return (roi_x1, roi_y1, roi_x2, roi_y2)
+    
+    def xywhn2xyxy(self, boxes, img_w, img_h):
+        """
+        Convert normalized [x_center, y_center, width, height] to [x1, y1, x2, y2].
+        
+        Args:
+            boxes: np.array of shape (N, 4), normalized boxes
+            img_w: int, image width
+            img_h: int, image height
+            
+        Returns:
+            np.array of shape (N, 4), absolute pixel coordinates
+        """
+        x_center = boxes[:, 0] * img_w
+        y_center = boxes[:, 1] * img_h
+        width = boxes[:, 2] * img_w
+        height = boxes[:, 3] * img_h
+        
+        x1 = x_center - width / 2
+        y1 = y_center - height / 2
+        x2 = x_center + width / 2
+        y2 = y_center + height / 2
+        
+        return np.stack([x1, y1, x2, y2], axis=1)
+    
+    def xyxy2xywhn(self, boxes, img_w, img_h):
+        """
+        Convert [x1, y1, x2, y2] to normalized [x_center, y_center, width, height].
+        
+        Args:
+            boxes: np.array of shape (N, 4), absolute pixel coordinates
+            img_w: int, image width
+            img_h: int, image height
+            
+        Returns:
+            np.array of shape (N, 4), normalized boxes
+        """
+        x1, y1, x2, y2 = boxes[:, 0], boxes[:, 1], boxes[:, 2], boxes[:, 3]
+        
+        x_center = (x1 + x2) / 2 / img_w
+        y_center = (y1 + y2) / 2 / img_h
+        width = (x2 - x1) / img_w
+        height = (y2 - y1) / img_h
+        
+        return np.stack([x_center, y_center, width, height], axis=1)
+    
+    def process_annotations_with_roi(self, annotations, roi_bounds):
+        """
+        Process annotations with ROI filtering and clipping.
+        
+        Matches the logic in post_process_labels_to_roi from dataloaders3d.py.
+        
+        Args:
+            annotations: list of annotation dicts from GroundTruthParser
+            roi_bounds: tuple (roi_x1, roi_y1, roi_x2, roi_y2)
+            
+        Returns:
+            list of processed annotations (some may be filtered out)
+        """
+        if roi_bounds is None or len(annotations) == 0:
+            return annotations
+        
+        roi_x1, roi_y1, roi_x2, roi_y2 = roi_bounds
+        roi_width = roi_x2 - roi_x1
+        roi_height = roi_y2 - roi_y1
+        
+        oriW, oriH = self.ori_img_size
+        
+        processed_annotations = []
+        
+        for ann in annotations:
+            # Get original bbox in pixel coordinates [x1, y1, x2, y2]
+            bbox_orig = ann['bbox_2d']
+            x1, y1, x2, y2 = bbox_orig
+            
+            # Shift to ROI-relative coordinates
+            new_x1 = x1 - roi_x1
+            new_y1 = y1 - roi_y1
+            new_x2 = x2 - roi_x1
+            new_y2 = y2 - roi_y1
+            
+            # Check if box is completely outside ROI
+            if ((new_x1 < 0 and new_x2 < 0) or 
+                (new_x1 >= roi_width and new_x2 >= roi_width) or
+                (new_y1 < 0 and new_y2 < 0) or 
+                (new_y1 >= roi_height and new_y2 >= roi_height)):
+                # Box is completely outside, skip it
+                continue
+            
+            # Check if box is completely inside (before clipping)
+            still_inside = (new_x1 >= 0 and new_y1 >= 0 and 
+                           new_x2 < roi_width and new_y2 < roi_height)
+            
+            # Clip to ROI bounds
+            new_x1 = np.clip(new_x1, 0, roi_width - 1)
+            new_y1 = np.clip(new_y1, 0, roi_height - 1)
+            new_x2 = np.clip(new_x2, 0, roi_width - 1)
+            new_y2 = np.clip(new_y2, 0, roi_height - 1)
+            
+            # Check if box still has valid size after clipping
+            if new_x2 <= new_x1 or new_y2 <= new_y1:
+                continue
+            
+            # Convert back to original image coordinates (to match detection results)
+            # Detection results are saved in original image coordinates after ROI processing
+            final_x1 = new_x1 + roi_x1
+            final_y1 = new_y1 + roi_y1
+            final_x2 = new_x2 + roi_x1
+            final_y2 = new_y2 + roi_y1
+            
+            # Update bbox to original image coordinates (filtered and clipped by ROI)
+            new_ann = ann.copy()
+            new_ann['bbox_2d'] = [final_x1, final_y1, final_x2, final_y2]
+            new_ann['roi_filtered'] = True  # Indicates GT has been filtered by ROI
+            new_ann['roi_bounds'] = roi_bounds
+            new_ann['was_clipped'] = not still_inside
+            
+            # If has 3D info and box was clipped, mark it
+            # (may need special handling for 3D evaluation)
+            if new_ann['has_3d'] and not still_inside:
+                # For partially visible objects, the 3D center may be less reliable
+                # This matches the cut-in/cut-out logic in training
+                if new_ann['3d_info']:
+                    new_ann['3d_info']['partially_visible'] = True
+            
+            processed_annotations.append(new_ann)
+        
+        return processed_annotations
+    
+    def process_case_frame(self, case_name, frame_name, annotations, level1_name=None):
+        """
+        Process annotations for a specific case and frame.
+
+        Args:
+            case_name: str, case identifier
+            frame_name: str, frame identifier
+            annotations: list, annotations from GroundTruthParser
+            level1_name: str, optional level1 directory name for 2-level path structure
+
+        Returns:
+            tuple: (processed_annotations, roi_bounds) or (annotations, None) if no ROI
+        """
+        if self.roi_config is None:
+            return annotations, None
+
+        # Load calibration
+        calib_params = self.load_calibration(case_name, frame_name, level1_name)
+        if calib_params is None:
+            print(f"Warning: Cannot compute ROI without calibration for {case_name}/{frame_name}")
+            return annotations, None
+        
+        # Compute ROI bounds
+        roi_bounds = self.compute_roi(calib_params)
+        if roi_bounds is None:
+            return annotations, None
+        
+        # Process annotations with ROI
+        processed = self.process_annotations_with_roi(annotations, roi_bounds)
+        
+        return processed, roi_bounds

eval_tools/heading_analysis/analyze_heading.sh

@@ -0,0 +1,19 @@
+#!/bin/bash
+#
+# Analyze heading errors between two models
+#
+# Usage: bash eval_tools/heading_analysis/analyze_heading.sh
+
+# Set PYTHONPATH to project root for module imports
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+PROJECT_ROOT="$(cd "${SCRIPT_DIR}/../.." && pwd)"
+export PYTHONPATH="${PROJECT_ROOT}:${PYTHONPATH}"
+
+python eval_tools/heading_analysis/analyze_heading_errors.py \
+    --common-matches eval_results_common_match_comparison_smallset/common_matches_20260203_223240/common_matches.json \
+    --model1-matches eval_results_common_match_comparison_smallset/mono3d/20260203_223240/detailed_3d_matches.json \
+    --model2-matches eval_results_common_match_comparison_smallset/yolov5s-300w/20260203_223240/detailed_3d_matches.json \
+    --model1-name "mono3d" \
+    --model2-name "yolov5s-300w" \
+    --output-dir heading_analysis_results_small \
+    --bad-case-threshold 1.0

eval_tools/heading_analysis/analyze_heading_errors.py

@@ -0,0 +1,423 @@
+#!/usr/bin/env python3
+"""
+Heading Error Analysis Tool
+
+This tool performs comprehensive analysis of heading errors between two models,
+focusing on understanding why one model has larger heading errors.
+
+Usage:
+    python eval_tools/analyze_heading_errors.py \\
+        --common-matches eval_results/common_matches.json \\
+        --model1-matches eval_results/model1/detailed_3d_matches.json \\
+        --model2-matches eval_results/model2/detailed_3d_matches.json \\
+        --model1-name "mono3d" \\
+        --model2-name "yolov5s-300w" \\
+        --output-dir heading_analysis_results
+"""
+
+import argparse
+import json
+import numpy as np
+import matplotlib.pyplot as plt
+from pathlib import Path
+from collections import defaultdict
+import pandas as pd
+from scipy import stats
+
+
+class HeadingErrorAnalyzer:
+    """Analyze heading errors between two models."""
+    
+    def __init__(self, common_matches_data, model1_matches, model2_matches, 
+                 model1_name="Model1", model2_name="Model2"):
+        """
+        Initialize analyzer.
+        
+        Args:
+            common_matches_data: dict, common matches data
+            model1_matches: dict, model1 detailed matches
+            model2_matches: dict, model2 detailed matches
+            model1_name: str, model1 name
+            model2_name: str, model2 name
+        """
+        self.common_matches_data = common_matches_data
+        self.model1_matches = model1_matches
+        self.model2_matches = model2_matches
+        self.model1_name = model1_name
+        self.model2_name = model2_name
+        
+        # Extract heading errors for common matches
+        self.data = self._extract_heading_data()
+    
+    def _extract_heading_data(self):
+        """Extract heading error data for all common matches."""
+        data = {
+            'model1': defaultdict(list),
+            'model2': defaultdict(list),
+            'common': defaultdict(list)
+        }
+        
+        common_matches = self.common_matches_data['common_matches']
+        
+        for case_name, frames in common_matches.items():
+            for frame_name, classes in frames.items():
+                for class_name, match_list in classes.items():
+                    for match_info in match_list:
+                        # Get match indices
+                        m1_idx = match_info['model1_idx']
+                        m2_idx = match_info['model2_idx']
+                        
+                        # Get match data
+                        m1_match = self.model1_matches[case_name][frame_name][class_name][m1_idx]
+                        m2_match = self.model2_matches[case_name][frame_name][class_name][m2_idx]
+                        
+                        # Extract information
+                        item = {
+                            'case': case_name,
+                            'frame': frame_name,
+                            'class': class_name,
+                            'gt_rotation': m1_match.get('gt_rotation', 0),
+                            'model1_rotation': m1_match.get('det_rotation', 0),
+                            'model2_rotation': m2_match.get('det_rotation', 0),
+                            'model1_error': m1_match['errors']['heading'],
+                            'model2_error': m2_match['errors']['heading'],
+                            'lateral_dist': m1_match['distance']['lateral'],
+                            'longitudinal_dist': m1_match['distance']['longitudinal'],
+                            'iou': m1_match['iou'],
+                            'confidence': m1_match['confidence']
+                        }
+                        
+                        data['model1'][class_name].append(m1_match['errors']['heading'])
+                        data['model2'][class_name].append(m2_match['errors']['heading'])
+                        data['common'][class_name].append(item)
+        
+        return data
+    
+    def generate_distribution_analysis(self, output_dir):
+        """Generate distribution analysis and plots."""
+        print("\n" + "="*80)
+        print("Heading Error Distribution Analysis")
+        print("="*80)
+        
+        output_dir = Path(output_dir) / 'distribution'
+        output_dir.mkdir(parents=True, exist_ok=True)
+        
+        results = {}
+        
+        for class_name in self.data['model1'].keys():
+            m1_errors = np.array(self.data['model1'][class_name])
+            m2_errors = np.array(self.data['model2'][class_name])
+            
+            # Calculate statistics
+            stats_data = {
+                'class': class_name,
+                'count': len(m1_errors),
+                'model1': {
+                    'mean': float(np.mean(m1_errors)),
+                    'median': float(np.median(m1_errors)),
+                    'std': float(np.std(m1_errors)),
+                    'p50': float(np.percentile(m1_errors, 50)),
+                    'p75': float(np.percentile(m1_errors, 75)),
+                    'p90': float(np.percentile(m1_errors, 90)),
+                    'p95': float(np.percentile(m1_errors, 95)),
+                    'p99': float(np.percentile(m1_errors, 99))
+                },
+                'model2': {
+                    'mean': float(np.mean(m2_errors)),
+                    'median': float(np.median(m2_errors)),
+                    'std': float(np.std(m2_errors)),
+                    'p50': float(np.percentile(m2_errors, 50)),
+                    'p75': float(np.percentile(m2_errors, 75)),
+                    'p90': float(np.percentile(m2_errors, 90)),
+                    'p95': float(np.percentile(m2_errors, 95)),
+                    'p99': float(np.percentile(m2_errors, 99))
+                }
+            }
+            
+            results[class_name] = stats_data
+            
+            # Print results
+            print(f"\n{class_name.upper()} (n={stats_data['count']:,}):")
+            print(f"  {self.model1_name}:")
+            print(f"    Mean: {stats_data['model1']['mean']:.4f} rad")
+            print(f"    Median: {stats_data['model1']['median']:.4f} rad")
+            print(f"    P90: {stats_data['model1']['p90']:.4f} rad")
+            print(f"    P95: {stats_data['model1']['p95']:.4f} rad")
+            print(f"  {self.model2_name}:")
+            print(f"    Mean: {stats_data['model2']['mean']:.4f} rad")
+            print(f"    Median: {stats_data['model2']['median']:.4f} rad")
+            print(f"    P90: {stats_data['model2']['p90']:.4f} rad")
+            print(f"    P95: {stats_data['model2']['p95']:.4f} rad")
+            print(f"  Change:")
+            print(f"    Mean: +{(stats_data['model2']['mean'] - stats_data['model1']['mean']):.4f} rad "
+                  f"({((stats_data['model2']['mean'] / stats_data['model1']['mean'] - 1) * 100):.1f}%)")
+            
+            # Create plots
+            fig, axes = plt.subplots(2, 2, figsize=(15, 12))
+            fig.suptitle(f'Heading Error Distribution - {class_name}', fontsize=16)
+            
+            # Histogram
+            axes[0, 0].hist(m1_errors, bins=50, alpha=0.5, label=self.model1_name, density=True)
+            axes[0, 0].hist(m2_errors, bins=50, alpha=0.5, label=self.model2_name, density=True)
+            axes[0, 0].set_xlabel('Heading Error (rad)')
+            axes[0, 0].set_ylabel('Density')
+            axes[0, 0].set_title('Histogram')
+            axes[0, 0].legend()
+            axes[0, 0].grid(True, alpha=0.3)
+            
+            # CDF
+            m1_sorted = np.sort(m1_errors)
+            m2_sorted = np.sort(m2_errors)
+            axes[0, 1].plot(m1_sorted, np.arange(len(m1_sorted)) / len(m1_sorted), label=self.model1_name)
+            axes[0, 1].plot(m2_sorted, np.arange(len(m2_sorted)) / len(m2_sorted), label=self.model2_name)
+            axes[0, 1].set_xlabel('Heading Error (rad)')
+            axes[0, 1].set_ylabel('Cumulative Probability')
+            axes[0, 1].set_title('Cumulative Distribution Function')
+            axes[0, 1].legend()
+            axes[0, 1].grid(True, alpha=0.3)
+            
+            # Box plot
+            axes[1, 0].boxplot([m1_errors, m2_errors], labels=[self.model1_name, self.model2_name])
+            axes[1, 0].set_ylabel('Heading Error (rad)')
+            axes[1, 0].set_title('Box Plot')
+            axes[1, 0].grid(True, alpha=0.3)
+            
+            # Q-Q plot
+            stats.probplot(m2_errors - m1_errors, dist="norm", plot=axes[1, 1])
+            axes[1, 1].set_title('Q-Q Plot (Error Difference)')
+            axes[1, 1].grid(True, alpha=0.3)
+            
+            plt.tight_layout()
+            plt.savefig(output_dir / f'{class_name}_distribution.png', dpi=150)
+            plt.close()
+        
+        # Save results
+        with open(output_dir / 'statistics.json', 'w') as f:
+            json.dump(results, f, indent=2)
+        
+        print(f"\n✓ Distribution analysis saved to: {output_dir}")
+        return results
+    
+    def generate_distance_analysis(self, output_dir):
+        """Analyze heading errors by distance ranges."""
+        print("\n" + "="*80)
+        print("Heading Error by Distance Analysis")
+        print("="*80)
+        
+        output_dir = Path(output_dir) / 'distance'
+        output_dir.mkdir(parents=True, exist_ok=True)
+        
+        # Define distance ranges
+        long_ranges = [(0, 20), (20, 40), (40, 60), (60, 80), (80, 100)]
+        lat_ranges = [(-30, -10), (-10, 0), (0, 10), (10, 30)]
+        
+        for class_name, items in self.data['common'].items():
+            # Analyze by longitudinal distance
+            long_stats = {}
+            for range_start, range_end in long_ranges:
+                range_key = f"{range_start}-{range_end}m"
+                m1_errors = []
+                m2_errors = []
+                
+                for item in items:
+                    dist = item['longitudinal_dist']
+                    if range_start <= dist < range_end:
+                        m1_errors.append(item['model1_error'])
+                        m2_errors.append(item['model2_error'])
+                
+                if len(m1_errors) > 0:
+                    long_stats[range_key] = {
+                        'count': len(m1_errors),
+                        'model1_mean': float(np.mean(m1_errors)),
+                        'model2_mean': float(np.mean(m2_errors)),
+                        'diff': float(np.mean(m2_errors) - np.mean(m1_errors))
+                    }
+            
+            # Plot longitudinal distance analysis
+            if long_stats:
+                fig, ax = plt.subplots(figsize=(12, 6))
+                ranges = list(long_stats.keys())
+                m1_means = [long_stats[r]['model1_mean'] for r in ranges]
+                m2_means = [long_stats[r]['model2_mean'] for r in ranges]
+                
+                x = np.arange(len(ranges))
+                width = 0.35
+                
+                ax.bar(x - width/2, m1_means, width, label=self.model1_name)
+                ax.bar(x + width/2, m2_means, width, label=self.model2_name)
+                
+                ax.set_xlabel('Longitudinal Distance Range')
+                ax.set_ylabel('Mean Heading Error (rad)')
+                ax.set_title(f'Heading Error by Longitudinal Distance - {class_name}')
+                ax.set_xticks(x)
+                ax.set_xticklabels(ranges)
+                ax.legend()
+                ax.grid(True, alpha=0.3)
+                
+                plt.tight_layout()
+                plt.savefig(output_dir / f'{class_name}_longitudinal.png', dpi=150)
+                plt.close()
+                
+                print(f"\n{class_name.upper()} - Longitudinal Distance:")
+                for range_key, data in long_stats.items():
+                    print(f"  {range_key}: {self.model1_name}={data['model1_mean']:.4f}, "
+                          f"{self.model2_name}={data['model2_mean']:.4f}, "
+                          f"diff={data['diff']:+.4f} (n={data['count']})")
+        
+        print(f"\n✓ Distance analysis saved to: {output_dir}")
+    
+    def identify_bad_cases(self, output_dir, threshold=1.0):
+        """Identify cases with large heading errors."""
+        print("\n" + "="*80)
+        print(f"Identifying Bad Cases (threshold > {threshold} rad)")
+        print("="*80)
+        
+        output_dir = Path(output_dir) / 'bad_cases'
+        output_dir.mkdir(parents=True, exist_ok=True)
+        
+        bad_cases = []
+        
+        for class_name, items in self.data['common'].items():
+            for item in items:
+                # Check if either model has large error
+                if item['model2_error'] > threshold:
+                    bad_cases.append({
+                        'case': item['case'],
+                        'frame': item['frame'],
+                        'class': class_name,
+                        'gt_rotation': item['gt_rotation'],
+                        'model1_rotation': item['model1_rotation'],
+                        'model2_rotation': item['model2_rotation'],
+                        'model1_error': item['model1_error'],
+                        'model2_error': item['model2_error'],
+                        'error_increase': item['model2_error'] - item['model1_error'],
+                        'longitudinal_dist': item['longitudinal_dist'],
+                        'lateral_dist': item['lateral_dist'],
+                        'iou': item['iou'],
+                        'confidence': item['confidence']
+                    })
+        
+        # Sort by model2 error
+        bad_cases.sort(key=lambda x: x['model2_error'], reverse=True)
+        
+        # Save to CSV
+        if bad_cases:
+            df = pd.DataFrame(bad_cases)
+            csv_path = output_dir / 'bad_cases.csv'
+            df.to_csv(csv_path, index=False)
+            
+            print(f"\nFound {len(bad_cases)} bad cases:")
+            print(f"  Saved to: {csv_path}")
+            print(f"\nTop 10 worst cases:")
+            print(df.head(10).to_string(index=False))
+        else:
+            print(f"\nNo bad cases found with threshold > {threshold} rad")
+        
+        print(f"\n✓ Bad cases analysis saved to: {output_dir}")
+        return bad_cases
+    
+    def generate_summary_report(self, output_dir):
+        """Generate summary text report."""
+        report_path = output_dir / 'heading_analysis_summary.txt'
+        
+        with open(report_path, 'w') as f:
+            f.write("="*80 + "\n")
+            f.write("HEADING ERROR ANALYSIS SUMMARY\n")
+            f.write("="*80 + "\n\n")
+            
+            f.write(f"Model 1: {self.model1_name}\n")
+            f.write(f"Model 2: {self.model2_name}\n\n")
+            
+            f.write("Overall Statistics by Class:\n")
+            f.write("-"*80 + "\n\n")
+            
+            for class_name in self.data['model1'].keys():
+                m1_errors = np.array(self.data['model1'][class_name])
+                m2_errors = np.array(self.data['model2'][class_name])
+                
+                f.write(f"{class_name.upper()} (n={len(m1_errors):,}):\n")
+                f.write(f"  {self.model1_name}:\n")
+                f.write(f"    Mean: {np.mean(m1_errors):.4f} rad ({np.degrees(np.mean(m1_errors)):.2f}°)\n")
+                f.write(f"    Median: {np.median(m1_errors):.4f} rad ({np.degrees(np.median(m1_errors)):.2f}°)\n")
+                f.write(f"    Std: {np.std(m1_errors):.4f} rad\n")
+                f.write(f"  {self.model2_name}:\n")
+                f.write(f"    Mean: {np.mean(m2_errors):.4f} rad ({np.degrees(np.mean(m2_errors)):.2f}°)\n")
+                f.write(f"    Median: {np.median(m2_errors):.4f} rad ({np.degrees(np.median(m2_errors)):.2f}°)\n")
+                f.write(f"    Std: {np.std(m2_errors):.4f} rad\n")
+                
+                diff_mean = np.mean(m2_errors) - np.mean(m1_errors)
+                pct_change = (np.mean(m2_errors) / np.mean(m1_errors) - 1) * 100
+                f.write(f"  Change:\n")
+                f.write(f"    Mean: +{diff_mean:.4f} rad ({pct_change:+.2f}%)\n\n")
+        
+        print(f"\n✓ Summary report saved to: {report_path}")
+
+
+def main():
+    """Main function."""
+    parser = argparse.ArgumentParser(description='Analyze heading errors between two models')
+    parser.add_argument('--common-matches', type=str, required=True,
+                       help='Path to common_matches.json')
+    parser.add_argument('--model1-matches', type=str, required=True,
+                       help='Path to model1 detailed_3d_matches.json')
+    parser.add_argument('--model2-matches', type=str, required=True,
+                       help='Path to model2 detailed_3d_matches.json')
+    parser.add_argument('--model1-name', type=str, default='Model1',
+                       help='Name of model 1')
+    parser.add_argument('--model2-name', type=str, default='Model2',
+                       help='Name of model 2')
+    parser.add_argument('--output-dir', type=str, default='heading_analysis',
+                       help='Output directory for analysis results')
+    parser.add_argument('--bad-case-threshold', type=float, default=1.0,
+                       help='Threshold for identifying bad cases (radians)')
+    
+    args = parser.parse_args()
+    
+    # Load data
+    print("Loading data...")
+    with open(args.common_matches, 'r') as f:
+        common_matches_data = json.load(f)
+    
+    with open(args.model1_matches, 'r') as f:
+        model1_matches = json.load(f)
+    
+    with open(args.model2_matches, 'r') as f:
+        model2_matches = json.load(f)
+    
+    # Create analyzer
+    analyzer = HeadingErrorAnalyzer(
+        common_matches_data,
+        model1_matches,
+        model2_matches,
+        model1_name=args.model1_name,
+        model2_name=args.model2_name
+    )
+    
+    # Create output directory
+    output_dir = Path(args.output_dir)
+    output_dir.mkdir(parents=True, exist_ok=True)
+    
+    # Run analyses
+    print("\n" + "="*80)
+    print("HEADING ERROR ANALYSIS")
+    print("="*80)
+    
+    analyzer.generate_distribution_analysis(output_dir)
+    analyzer.generate_distance_analysis(output_dir)
+    analyzer.identify_bad_cases(output_dir, threshold=args.bad_case_threshold)
+    analyzer.generate_summary_report(output_dir)
+    
+    print("\n" + "="*80)
+    print("ANALYSIS COMPLETE!")
+    print("="*80)
+    print(f"\nResults saved to: {output_dir}/")
+    print("\nGenerated files:")
+    print("  - heading_analysis_summary.txt: Text summary report")
+    print("  - distribution/: Error distribution analysis and plots")
+    print("  - distance/: Distance-based analysis and plots")
+    print("  - bad_cases/bad_cases.csv: List of cases with large errors")
+    print("")
+
+
+if __name__ == '__main__':
+    main()

eval_tools/heading_analysis/extract_bad_heading_cases.py

@@ -0,0 +1,313 @@
+"""
+Extract bad heading error cases from detailed_3d_matches.json
+
+This tool extracts and analyzes cases with large heading errors for visualization.
+
+Usage:
+    python eval_tools/extract_bad_heading_cases.py \
+        --input eval_results_common_match_comparison/yolov5s-300w/20260203_210259/detailed_3d_matches.json \
+        --threshold 1.5 \
+        --top-k 100 \
+        --output bad_heading_cases.json
+"""
+
+import argparse
+import json
+import sys
+from pathlib import Path
+from collections import defaultdict
+import numpy as np
+
+# Allow importing class_config from the eval_tools root
+sys.path.insert(0, str(Path(__file__).parent.parent))
+from class_config import CLASSES_3D as _CLASSES_3D, CLASS_NAMES as _CLASS_NAMES
+_CLASSES_3D_NAMES = set(_CLASS_NAMES[i] for i in _CLASSES_3D)
+
+def parse_args():
+    parser = argparse.ArgumentParser(description='Extract bad heading error cases')
+    parser.add_argument('--input', type=str, required=True,
+                       help='Path to detailed_3d_matches.json')
+    parser.add_argument('--threshold', type=float, default=1.5,
+                       help='Heading error threshold in radians (default: 1.5 ≈ 85°)')
+    parser.add_argument('--top-k', type=int, default=None,
+                       help='Only extract top K worst cases (default: all)')
+    parser.add_argument('--classes', nargs='+', default=None,
+                       help='Filter by classes (e.g., vehicle pedestrian bicycle rider)')
+    parser.add_argument('--min-distance', type=float, default=None,
+                       help='Minimum distance in meters')
+    parser.add_argument('--max-distance', type=float, default=None,
+                       help='Maximum distance in meters')
+    parser.add_argument('--min-confidence', type=float, default=None,
+                       help='Minimum detection confidence')
+    parser.add_argument('--reversal-only', action='store_true',
+                       help='Only extract reversal errors (error > π - 0.1)')
+    parser.add_argument('--output', type=str, default='bad_heading_cases.json',
+                       help='Output JSON file path')
+    parser.add_argument('--stats', action='store_true',
+                       help='Print statistics')
+    return parser.parse_args()
+
+
+def calculate_distance_3d(x, y, z):
+    """Calculate 3D Euclidean distance from ego vehicle."""
+    return np.sqrt(x**2 + y**2 + z**2)
+
+
+def is_reversal_error(error, threshold=0.1):
+    """Check if error is a reversal error (≈ π or 180°)."""
+    return error > (np.pi - threshold)
+
+
+def extract_bad_cases(data, args):
+    """Extract bad heading error cases based on criteria.
+    
+    Args:
+        data: Loaded JSON data from detailed_3d_matches.json
+        args: Command line arguments
+        
+    Returns:
+        List of bad cases with metadata
+    """
+    bad_cases = []
+    stats = defaultdict(lambda: {'count': 0, 'reversal_count': 0, 'total_error': 0})
+    
+    total_processed = 0
+    
+    # Iterate through all matches
+    for case_id, case_data in data.items():
+        for frame_id, frame_data in case_data.items():
+            if not isinstance(frame_data, dict):
+                continue
+                
+            for class_name, class_data in frame_data.items():
+                if class_name not in _CLASSES_3D_NAMES:
+                    continue
+                
+                # Filter by class if specified
+                if args.classes and class_name not in args.classes:
+                    continue
+                
+                # class_data is already the list of matches
+                matches = class_data if isinstance(class_data, list) else []
+                
+                for match in matches:
+                    total_processed += 1
+                    
+                    # Extract key information
+                    # heading_error is in errors['heading'], not directly in match
+                    errors = match.get('errors', {})
+                    heading_error = errors.get('heading', 0) if isinstance(errors, dict) else 0
+                    lateral_error = errors.get('lateral', 0) if isinstance(errors, dict) else 0
+                    longitudinal_error = errors.get('longitudinal', 0) if isinstance(errors, dict) else 0
+                    
+                    gt_rotation = match.get('gt_rotation', 0)
+                    det_rotation = match.get('det_rotation', 0)
+                    
+                    # Get 3D center coordinates
+                    gt_center = match.get('gt_center_3d', [0, 0, 0])
+                    det_center = match.get('det_center_3d', [0, 0, 0])
+                    
+                    # Calculate distance
+                    distance = calculate_distance_3d(*gt_center)
+                    
+                    # Get other metadata
+                    confidence = match.get('confidence', 0)
+                    iou = match.get('iou', 0)
+                    
+                    # Apply filters
+                    if heading_error < args.threshold:
+                        continue
+                    
+                    if args.min_distance and distance < args.min_distance:
+                        continue
+                    
+                    if args.max_distance and distance > args.max_distance:
+                        continue
+                    
+                    if args.min_confidence and confidence < args.min_confidence:
+                        continue
+                    
+                    if args.reversal_only and not is_reversal_error(heading_error):
+                        continue
+                    
+                    # Update statistics
+                    stats[class_name]['count'] += 1
+                    stats[class_name]['total_error'] += heading_error
+                    if is_reversal_error(heading_error):
+                        stats[class_name]['reversal_count'] += 1
+                    
+                    # Create case entry
+                    case_entry = {
+                        'case_id': case_id,
+                        'frame_id': frame_id,
+                        'class': class_name,
+                        'heading_error': float(heading_error),
+                        'heading_error_deg': float(np.degrees(heading_error)),
+                        'gt_rotation': float(gt_rotation),
+                        'gt_rotation_deg': float(np.degrees(gt_rotation)),
+                        'det_rotation': float(det_rotation),
+                        'det_rotation_deg': float(np.degrees(det_rotation)),
+                        'is_reversal': is_reversal_error(heading_error),
+                        'distance': float(distance),
+                        'confidence': float(confidence),
+                        'iou': float(iou),
+                        'gt_center': [float(x) for x in gt_center],
+                        'det_center': [float(x) for x in det_center],
+                        'lateral_error': float(lateral_error),
+                        'longitudinal_error': float(longitudinal_error),
+                        'gt_bbox_2d': match.get('gt_bbox', [0, 0, 0, 0]),
+                        'det_bbox_2d': match.get('det_bbox', [0, 0, 0, 0]),
+                    }
+                    
+                    bad_cases.append(case_entry)
+    
+    # Sort by heading error (descending)
+    bad_cases.sort(key=lambda x: x['heading_error'], reverse=True)
+    
+    # Limit to top-k if specified
+    if args.top_k:
+        bad_cases = bad_cases[:args.top_k]
+    
+    return bad_cases, stats, total_processed
+
+
+def print_statistics(bad_cases, stats, total_processed, args):
+    """Print detailed statistics about extracted cases."""
+    print("\n" + "="*80)
+    print("BAD HEADING ERROR CASES EXTRACTION SUMMARY")
+    print("="*80)
+    
+    print(f"\nInput file: {args.input}")
+    print(f"Threshold: {args.threshold:.2f} rad ({np.degrees(args.threshold):.1f}°)")
+    print(f"Total processed: {total_processed:,}")
+    print(f"Bad cases found: {len(bad_cases):,} ({100*len(bad_cases)/total_processed:.2f}%)")
+    
+    if args.top_k:
+        print(f"Output limited to: Top {args.top_k}")
+    
+    print("\n" + "-"*80)
+    print("STATISTICS BY CLASS")
+    print("-"*80)
+    print(f"{'Class':<15} {'Count':<10} {'Reversal':<12} {'Rev %':<10} {'Avg Error':<12}")
+    print("-"*80)
+    
+    for class_name in sorted(stats.keys()):
+        stat = stats[class_name]
+        count = stat['count']
+        rev_count = stat['reversal_count']
+        avg_error = stat['total_error'] / count if count > 0 else 0
+        rev_pct = 100 * rev_count / count if count > 0 else 0
+        
+        print(f"{class_name:<15} {count:<10,} {rev_count:<12,} {rev_pct:<10.1f} {avg_error:<12.3f}")
+    
+    print("-"*80)
+    
+    # Error distribution
+    if bad_cases:
+        errors = [c['heading_error'] for c in bad_cases]
+        print(f"\nERROR DISTRIBUTION:")
+        print(f"  Min:     {min(errors):.3f} rad ({np.degrees(min(errors)):.1f}°)")
+        print(f"  Max:     {max(errors):.3f} rad ({np.degrees(max(errors)):.1f}°)")
+        print(f"  Mean:    {np.mean(errors):.3f} rad ({np.degrees(np.mean(errors)):.1f}°)")
+        print(f"  Median:  {np.median(errors):.3f} rad ({np.degrees(np.median(errors)):.1f}°)")
+        print(f"  Std:     {np.std(errors):.3f} rad ({np.degrees(np.std(errors)):.1f}°)")
+        
+        # Reversal statistics
+        reversal_count = sum(1 for c in bad_cases if c['is_reversal'])
+        print(f"\n  Reversal errors (>3.04 rad): {reversal_count} ({100*reversal_count/len(bad_cases):.1f}%)")
+        
+        # Distance distribution
+        distances = [c['distance'] for c in bad_cases]
+        print(f"\nDISTANCE DISTRIBUTION:")
+        print(f"  Min:     {min(distances):.1f} m")
+        print(f"  Max:     {max(distances):.1f} m")
+        print(f"  Mean:    {np.mean(distances):.1f} m")
+        print(f"  Median:  {np.median(distances):.1f} m")
+        
+        # Confidence distribution
+        confidences = [c['confidence'] for c in bad_cases]
+        print(f"\nCONFIDENCE DISTRIBUTION:")
+        print(f"  Min:     {min(confidences):.3f}")
+        print(f"  Max:     {max(confidences):.3f}")
+        print(f"  Mean:    {np.mean(confidences):.3f}")
+        print(f"  Median:  {np.median(confidences):.3f}")
+    
+    print("\n" + "="*80)
+
+
+def main():
+    args = parse_args()
+    
+    # Load input JSON
+    input_path = Path(args.input)
+    if not input_path.exists():
+        print(f"Error: Input file not found: {input_path}")
+        sys.exit(1)
+    
+    print(f"Loading data from {input_path}...")
+    with open(input_path, 'r') as f:
+        data = json.load(f)
+    
+    print(f"Loaded {len(data)} cases")
+    
+    # Extract bad cases
+    print(f"\nExtracting bad cases with heading_error > {args.threshold:.2f} rad...")
+    bad_cases, stats, total_processed = extract_bad_cases(data, args)
+    
+    # Print statistics
+    if args.stats or len(bad_cases) > 0:
+        print_statistics(bad_cases, stats, total_processed, args)
+    
+    # Save output
+    output_path = Path(args.output)
+    output_path.parent.mkdir(parents=True, exist_ok=True)
+    
+    output_data = {
+        'metadata': {
+            'source': str(input_path),
+            'threshold': args.threshold,
+            'threshold_degrees': float(np.degrees(args.threshold)),
+            'total_processed': total_processed,
+            'total_extracted': len(bad_cases),
+            'filters': {
+                'classes': args.classes,
+                'min_distance': args.min_distance,
+                'max_distance': args.max_distance,
+                'min_confidence': args.min_confidence,
+                'reversal_only': args.reversal_only,
+                'top_k': args.top_k
+            }
+        },
+        'statistics': {
+            class_name: {
+                'count': stat['count'],
+                'reversal_count': stat['reversal_count'],
+                'reversal_percentage': 100 * stat['reversal_count'] / stat['count'] if stat['count'] > 0 else 0,
+                'avg_error': stat['total_error'] / stat['count'] if stat['count'] > 0 else 0
+            }
+            for class_name, stat in stats.items()
+        },
+        'cases': bad_cases
+    }
+    
+    with open(output_path, 'w') as f:
+        json.dump(output_data, f, indent=2)
+    
+    print(f"\nExtracted {len(bad_cases)} bad cases")
+    print(f"Output saved to: {output_path}")
+    
+    # Show top 5 worst cases
+    if bad_cases:
+        print(f"\nTop 5 Worst Cases:")
+        print("-"*80)
+        print(f"{'No':<5} {'Class':<12} {'Error (rad)':<12} {'Error (°)':<12} {'Distance':<12} {'Reversal':<10}")
+        print("-"*80)
+        for i, case in enumerate(bad_cases[:5], 1):
+            print(f"{i:<5} {case['class']:<12} {case['heading_error']:<12.3f} "
+                  f"{case['heading_error_deg']:<12.1f} {case['distance']:<12.1f} "
+                  f"{'✓' if case['is_reversal'] else '':<10}")
+        print("-"*80)
+
+
+if __name__ == '__main__':
+    main()

eval_tools/heading_analysis/test_extract_bad_cases.sh

@@ -0,0 +1,88 @@
+#!/bin/bash
+# Test script for extracting bad heading cases
+
+
+# Set PYTHONPATH to project root for module imports
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+PROJECT_ROOT="$(cd "${SCRIPT_DIR}/../.." && pwd)"
+export PYTHONPATH="${PROJECT_ROOT}:${PYTHONPATH}"
+echo "========================================="
+
+echo "Bad Heading Cases Extraction Test"
+echo "========================================="
+echo ""
+
+INPUT_FILE="eval_results_common_match_comparison/yolov5s-300w/20260203_210259/detailed_3d_matches.json"
+OUTPUT_DIR="runs/heading_error_analysis"
+
+# Check if input file exists
+if [ ! -f "$INPUT_FILE" ]; then
+    echo "Error: Input file not found: $INPUT_FILE"
+    echo "Please update INPUT_FILE path in this script"
+    exit 1
+fi
+
+mkdir -p "$OUTPUT_DIR"
+
+echo "Test 1: Extract all cases with error > 1.5 rad (≈ 85°)"
+echo "-----------------------------------------------------"
+python eval_tools/heading_analysis/extract_bad_heading_cases.py \
+    --input "$INPUT_FILE" \
+    --threshold 1.5 \
+    --output "$OUTPUT_DIR/all_bad_cases.json" \
+    --stats
+
+echo ""
+echo ""
+echo "Test 2: Extract top 50 worst cases"
+echo "-----------------------------------------------------"
+python eval_tools/heading_analysis/extract_bad_heading_cases.py \
+    --input "$INPUT_FILE" \
+    --threshold 1.5 \
+    --top-k 50 \
+    --output "$OUTPUT_DIR/top50_worst.json" \
+    --stats
+
+echo ""
+echo ""
+echo "Test 3: Extract only reversal errors (≈ 180°)"
+echo "-----------------------------------------------------"
+python eval_tools/heading_analysis/extract_bad_heading_cases.py \
+    --input "$INPUT_FILE" \
+    --threshold 1.5 \
+    --reversal-only \
+    --top-k 100 \
+    --output "$OUTPUT_DIR/reversal_errors.json" \
+    --stats
+
+echo ""
+echo ""
+echo "Test 4: Extract vehicle class only, high confidence"
+echo "-----------------------------------------------------"
+python eval_tools/heading_analysis/extract_bad_heading_cases.py \
+    --input "$INPUT_FILE" \
+    --threshold 1.5 \
+    --classes vehicle \
+    --min-confidence 0.5 \
+    --top-k 50 \
+    --output "$OUTPUT_DIR/vehicle_high_conf.json" \
+    --stats
+
+echo ""
+echo ""
+echo "Test 5: Extract mid-distance cases (30-80m)"
+echo "-----------------------------------------------------"
+python eval_tools/heading_analysis/extract_bad_heading_cases.py \
+    --input "$INPUT_FILE" \
+    --threshold 1.5 \
+    --min-distance 30 \
+    --max-distance 80 \
+    --top-k 50 \
+    --output "$OUTPUT_DIR/mid_distance.json" \
+    --stats
+
+echo ""
+echo "========================================="
+echo "All tests complete!"
+echo "Output directory: $OUTPUT_DIR"
+echo "========================================="

eval_tools/heading_analysis/test_visualize_heading.sh

@@ -0,0 +1,62 @@
+#!/bin/bash
+# Test script for visualizing heading errors
+
+
+# Set PYTHONPATH to project root for module imports
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+PROJECT_ROOT="$(cd "${SCRIPT_DIR}/../.." && pwd)"
+export PYTHONPATH="${PROJECT_ROOT}:${PYTHONPATH}"
+echo "========================================="
+
+echo "Heading Error Visualization Test"
+echo "========================================="
+echo ""
+
+INPUT_FILE="bad_heading_cases_top50.json"
+IMAGE_ROOT="/path/to/your/image/root"  # UPDATE THIS
+OUTPUT_DIR="runs/heading_viz_test"
+
+# Check if input file exists
+if [ ! -f "$INPUT_FILE" ]; then
+    echo "Error: Input file not found: $INPUT_FILE"
+    echo "Please run extract_bad_heading_cases.py first"
+    exit 1
+fi
+
+echo "Test 1: Generate all visualization types (first 10 cases)"
+echo "-----------------------------------------------------"
+python eval_tools/heading_analysis/visualize_heading_errors.py \
+    --input "$INPUT_FILE" \
+    --image-root "$IMAGE_ROOT" \
+    --output "$OUTPUT_DIR/all_viz" \
+    --max-cases 10 \
+    --viz-types bev image angle combined
+
+echo ""
+echo ""
+echo "Test 2: Generate only BEV views (first 20 cases)"
+echo "-----------------------------------------------------"
+python eval_tools/heading_analysis/visualize_heading_errors.py \
+    --input "$INPUT_FILE" \
+    --image-root "$IMAGE_ROOT" \
+    --output "$OUTPUT_DIR/bev_only" \
+    --max-cases 20 \
+    --viz-types bev
+
+echo ""
+echo ""
+echo "Test 3: Generate combined views (first 5 cases, high DPI)"
+echo "-----------------------------------------------------"
+python eval_tools/heading_analysis/visualize_heading_errors.py \
+    --input "$INPUT_FILE" \
+    --image-root "$IMAGE_ROOT" \
+    --output "$OUTPUT_DIR/combined_hq" \
+    --max-cases 5 \
+    --viz-types combined \
+    --dpi 150
+
+echo ""
+echo "========================================="
+echo "All tests complete!"
+echo "Output directory: $OUTPUT_DIR"
+echo "========================================="

eval_tools/heading_analysis/visualize_heading_errors.sh

@@ -0,0 +1,13 @@
+#!/bin/bash
+
+# Set PYTHONPATH to project root for module imports
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+PROJECT_ROOT="$(cd "${SCRIPT_DIR}/../.." && pwd)"
+export PYTHONPATH="${PROJECT_ROOT}:${PYTHONPATH}"
+python eval_tools/heading_analysis/visualize_heading_errors.py \
+    --input bad_heading_cases.json \
+    --image-root /data1/xdzhu/Testdata_0129 \
+    --output runs/heading_viz \
+    --viz-types combined \
+    --merge-same-frame \
+    --workers 8

eval_tools/model_comparison/README_per_case_comparison.md

@@ -0,0 +1,92 @@
+# Per-Case 2D Metrics Comparison Tool
+
+This tool compares `per_case_2d` metrics between two model evaluation reports and identifies cases with significant metric differences.
+
+## Files
+
+- `compare_per_case_2d.py` - Main Python script for comparing per-case metrics
+- `compare_per_case_2d.sh` - Shell script with pre-configured paths for mono3d vs yolov5s-300w-newdata comparison
+
+## Usage
+
+### Quick Start (Using Shell Script)
+
+```bash
+cd /deeplearning_team/ydong/dongying/projects/yolov5-3d
+./eval_tools/model_comparison/compare_per_case_2d.sh
+```
+
+This will compare the two models and save results to `evaluation_results/per_case_2d_comparison.json`.
+
+### Custom Comparison (Using Python Script)
+
+```bash
+python eval_tools/model_comparison/compare_per_case_2d.py \
+    --model1 path/to/model1/evaluation_report.json \
+    --model2 path/to/model2/evaluation_report.json \
+    --model1-name "Model-A" \
+    --model2-name "Model-B" \
+    --threshold 0.1 \
+    --output comparison_results.json \
+    --top-n 30
+```
+
+### Arguments
+
+- `--model1`: Path to first model's evaluation_report.json (required)
+- `--model2`: Path to second model's evaluation_report.json (required)
+- `--model1-name`: Display name for model 1 (default: "Model-1")
+- `--model2-name`: Display name for model 2 (default: "Model-2")
+- `--threshold`: Threshold for significant difference, e.g., 0.1 = 10% (default: 0.1)
+- `--output`: Output JSON file path (default: "per_case_comparison.json")
+- `--top-n`: Number of top different cases to display (default: 20)
+
+## Output
+
+The script generates:
+
+1. **Console Output**:
+   - Summary of total cases and common cases
+   - Top N cases with significant differences
+   - Summary statistics (mean, std, median, range) for each class and metric
+
+2. **JSON File**: Contains detailed comparison data including:
+   - `summary`: Overview statistics
+   - `significant_differences`: List of cases exceeding the threshold
+   - `all_case_comparisons`: Complete per-case comparison data
+   - `summary_statistics`: Statistical analysis by class and metric
+
+## Example Output
+
+```
+Top 30 Cases with Significant Differences
+================================================================================
+
+1. Case: 20251118/seq-53
+   Class: pedestrian, Metric: ap
+   mono3d: 1.0000
+   yolov5s-300w-newdata: 0.0000
+   Difference: -1.0000 (abs: 1.0000)
+
+2. Case: 20251121/seq-30
+   Class: roadblock, Metric: ap
+   mono3d: 1.0000
+   yolov5s-300w-newdata: 0.0000
+   Difference: -1.0000 (abs: 1.0000)
+...
+
+Summary Statistics
+================================================================================
+
+VEHICLE:
+  ap        : mean=-0.0776, std=0.1439, median=-0.0243, range=[-0.7935, +0.0994]
+  precision : mean=+0.1279, std=0.2248, median=+0.0934, range=[-0.9442, +0.6074]
+  recall    : mean=-0.1210, std=0.1579, median=-0.0635, range=[-0.8975, +0.0000]
+```
+
+## Interpretation
+
+- **Positive difference**: Model 2 performs better than Model 1
+- **Negative difference**: Model 1 performs better than Model 2
+- Cases are sorted by absolute difference (largest differences first)
+- Summary statistics show overall trends across all cases

eval_tools/model_comparison/bad_eval_cases.txt

@@ -0,0 +1,3 @@
+019b8ddb-ae8a-70f3-b86f-055894c79724
+019b6bf2-01a6-7029-9232-fce2bbcd2d73
+019b6bf2-0124-7820-91b7-c5eb42150cd2

eval_tools/model_comparison/compare_models_example.sh

@@ -0,0 +1,82 @@
+#!/bin/bash
+# Example script for comparing two model evaluation results
+#
+# Usage: bash eval_tools/compare_models_example.sh
+
+# =============================================================================
+# Configuration
+# =============================================================================
+
+# Model 1 (mono3d)
+
+# Set PYTHONPATH to project root for module imports
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+PROJECT_ROOT="$(cd "${SCRIPT_DIR}/../.." && pwd)"
+export PYTHONPATH="${PROJECT_ROOT}:${PYTHONPATH}"
+MODEL1_REPORT="eval_results_multiprocess/mono3d/20260203_162537/evaluation_report.json"
+
+MODEL1_NAME="mono3d"
+
+# Model 2 (yolov5s-300w)
+MODEL2_REPORT="eval_results_multiprocess/yolov5s/20260203_161644/evaluation_report.json"
+MODEL2_NAME="yolov5s-300w"
+
+# Output directory
+OUTPUT_DIR="comparison_results/$(date +%Y%m%d_%H%M%S)"
+
+# =============================================================================
+# Run Comparison
+# =============================================================================
+
+echo "=========================================="
+echo "Model Evaluation Comparison"
+echo "=========================================="
+echo "Model 1: $MODEL1_NAME"
+echo "  Report: $MODEL1_REPORT"
+echo "Model 2: $MODEL2_NAME"
+echo "  Report: $MODEL2_REPORT"
+echo "Output: $OUTPUT_DIR"
+echo "=========================================="
+
+# Check if reports exist
+if [ ! -f "$MODEL1_REPORT" ]; then
+    echo "Error: Model 1 report not found: $MODEL1_REPORT"
+    exit 1
+fi
+
+if [ ! -f "$MODEL2_REPORT" ]; then
+    echo "Error: Model 2 report not found: $MODEL2_REPORT"
+    exit 1
+fi
+
+# Run comparison with visualization
+python eval_tools/model_comparison/compare_models_visualize.py \
+    --model1 "$MODEL1_REPORT" \
+    --model2 "$MODEL2_REPORT" \
+    --output-dir "$OUTPUT_DIR" \
+    --model1-name "$MODEL1_NAME" \
+    --model2-name "$MODEL2_NAME"
+
+# Check if comparison was successful
+if [ $? -eq 0 ]; then
+    echo ""
+    echo "✓ Comparison completed successfully!"
+    echo ""
+    echo "View results:"
+    echo "  Text report: $OUTPUT_DIR/comparison_report.txt"
+    echo "  JSON report: $OUTPUT_DIR/comparison_report.json"
+    echo "  Plots: $OUTPUT_DIR/comparison_*.png"
+    echo ""
+    
+    # Display summary from text report
+    if [ -f "$OUTPUT_DIR/comparison_report.txt" ]; then
+        echo "=========================================="
+        echo "Quick Summary (from report):"
+        echo "=========================================="
+        grep -A 20 "2D DETECTION METRICS - OVERALL COMPARISON" "$OUTPUT_DIR/comparison_report.txt" | head -25
+    fi
+else
+    echo ""
+    echo "✗ Comparison failed!"
+    exit 1
+fi

eval_tools/model_comparison/compare_models_only.sh

@@ -0,0 +1,48 @@
+#!/bin/bash
+# 
+# Compare Models Only (Skip Evaluation Steps)
+# 
+# This script runs only the comparison steps (Step 4-5), assuming that
+# evaluation and common match finding have already been completed.
+#
+# Usage:
+#   bash eval_tools/compare_models_only.sh <MODEL1_DIR> <MODEL2_DIR> <COMMON_MATCHES_JSON>
+#
+# Example:
+#   bash eval_tools/compare_models_only.sh \
+#     eval_results_common_match_comparison/mono3d/20260203_210259 \
+#     eval_results_common_match_comparison/yolov5s-300w/20260203_210259 \
+#     eval_results_common_match_comparison/common_matches_20260203_210259/common_matches.json
+
+
+set -e  # Exit on error
+
+# Set PYTHONPATH to project root for module imports
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+PROJECT_ROOT="$(cd "${SCRIPT_DIR}/../.." && pwd)"
+export PYTHONPATH="${PROJECT_ROOT}:${PYTHONPATH}"
+
+MODEL1_DIR="evaluation_results/eval_results_common_match_comparison_cncap_yolov5s_20260228_roi0/yolov5s-300w-newdata/20260228_102849"
+MODEL2_DIR="evaluation_results/eval_results_common_match_comparison_cncap_yolov5s_20260228_roi0/yolov5s-300w-newdata-cncap/20260228_102849"
+MODEL1_NAME="yolov5s-300w-newdata"
+MODEL2_NAME="yolov5s-300w-newdata-cncap"
+COMMON_MATCHES_DIR="evaluation_results/eval_results_common_match_comparison_cncap_yolov5s_20260228_roi0/common_matches_20260228_102849"
+COMPARISON_DIR="evaluation_results/eval_results_common_match_comparison_cncap_yolov5s_20260228_roi0/comparison_common_matches_20260228_102849-v2"
+
+python eval_tools/model_comparison/compare_models.py \
+  --model1 ${MODEL1_DIR}/evaluation_report.json \
+  --model2 ${MODEL2_DIR}/evaluation_report.json \
+  --model1-name "${MODEL1_NAME}" \
+  --model2-name "${MODEL2_NAME}" \
+  --common-matches ${COMMON_MATCHES_DIR}/common_matches.json \
+  --output-dir ${COMPARISON_DIR}
+
+echo "✓ Comparison results saved to: ${COMPARISON_DIR}"
+
+
+# python eval_tools/model_comparison/compare_models.py \
+#   --model1 ${MODEL1_DIR}/evaluation_report.json \
+#   --model2 ${MODEL2_DIR}/evaluation_report.json \
+#   --model1-name "${MODEL1_NAME}" \
+#   --model2-name "${MODEL2_NAME}" \
+#   --output-dir ${COMPARISON_TRADITIONAL_DIR}

eval_tools/model_comparison/compare_models_visualize.py

@@ -0,0 +1,374 @@
+#!/usr/bin/env python3
+"""
+Model Evaluation Comparison Tool with Visualization
+
+Extended version with plotting capabilities.
+
+Usage:
+    python eval_tools/compare_models_visualize.py \
+        --model1 eval_results/model1/evaluation_report.json \
+        --model2 eval_results/model2/evaluation_report.json \
+        --output-dir comparison_results \
+        --model1-name "mono3d" \
+        --model2-name "yolov5s-300w"
+"""
+
+import argparse
+import json
+import os
+import sys
+from pathlib import Path
+
+# Add parent directory to path
+sys.path.insert(0, str(Path(__file__).parent.parent))
+
+from eval_tools.compare_models import ModelComparator
+
+try:
+    import matplotlib
+    matplotlib.use('Agg')  # Use non-interactive backend
+    import matplotlib.pyplot as plt
+    import numpy as np
+    MATPLOTLIB_AVAILABLE = True
+except ImportError:
+    MATPLOTLIB_AVAILABLE = False
+    print("Warning: matplotlib not available, visualization will be skipped")
+
+
+class VisualizationComparator(ModelComparator):
+    """Extended comparator with visualization capabilities."""
+    
+    def plot_2d_metrics_comparison(self, output_dir):
+        """Plot 2D metrics comparison."""
+        if not MATPLOTLIB_AVAILABLE:
+            print("Skipping 2D metrics plot (matplotlib not available)")
+            return
+        
+        print("\nGenerating 2D metrics comparison plots...")
+        
+        comparison = self.comparison_results.get('2d_metrics', {})
+        if not comparison:
+            return
+        
+        # Overall metrics bar chart
+        fig, axes = plt.subplots(1, 3, figsize=(15, 5))
+        fig.suptitle('2D Detection Metrics - Overall Comparison', fontsize=16, fontweight='bold')
+        
+        overall = comparison['overall']
+        metrics = ['precision', 'recall', 'map']
+        metric_names = ['Precision', 'Recall', 'mAP']
+        
+        for idx, (metric, metric_name) in enumerate(zip(metrics, metric_names)):
+            if metric not in overall:
+                continue
+            
+            values = overall[metric]
+            model_names = [self.model1_name, self.model2_name]
+            model_values = [values[self.model1_name], values[self.model2_name]]
+            
+            bars = axes[idx].bar(model_names, model_values, color=['#3498db', '#e74c3c'])
+            axes[idx].set_ylabel(metric_name)
+            axes[idx].set_title(f'{metric_name}')
+            axes[idx].set_ylim(0, 1.0)
+            axes[idx].grid(axis='y', alpha=0.3)
+            
+            # Add value labels on bars
+            for bar in bars:
+                height = bar.get_height()
+                axes[idx].text(bar.get_x() + bar.get_width()/2., height,
+                             f'{height:.3f}',
+                             ha='center', va='bottom', fontsize=10)
+        
+        plt.tight_layout()
+        output_file = os.path.join(output_dir, 'comparison_2d_overall.png')
+        plt.savefig(output_file, dpi=150, bbox_inches='tight')
+        plt.close()
+        print(f"  ✓ Saved: {output_file}")
+        
+        # Per-class AP comparison
+        per_class = comparison.get('per_class', {})
+        if per_class:
+            class_names = sorted(per_class.keys())
+            m1_aps = [per_class[c]['ap'][self.model1_name] for c in class_names]
+            m2_aps = [per_class[c]['ap'][self.model2_name] for c in class_names]
+            
+            fig, ax = plt.subplots(figsize=(12, 6))
+            x = np.arange(len(class_names))
+            width = 0.35
+            
+            bars1 = ax.bar(x - width/2, m1_aps, width, label=self.model1_name, color='#3498db')
+            bars2 = ax.bar(x + width/2, m2_aps, width, label=self.model2_name, color='#e74c3c')
+            
+            ax.set_xlabel('Class', fontsize=12)
+            ax.set_ylabel('Average Precision (AP)', fontsize=12)
+            ax.set_title('Per-Class AP Comparison', fontsize=14, fontweight='bold')
+            ax.set_xticks(x)
+            ax.set_xticklabels(class_names, rotation=45, ha='right')
+            ax.legend()
+            ax.grid(axis='y', alpha=0.3)
+            
+            plt.tight_layout()
+            output_file = os.path.join(output_dir, 'comparison_2d_per_class.png')
+            plt.savefig(output_file, dpi=150, bbox_inches='tight')
+            plt.close()
+            print(f"  ✓ Saved: {output_file}")
+    
+    def plot_3d_metrics_comparison(self, output_dir):
+        """Plot 3D metrics comparison."""
+        if not MATPLOTLIB_AVAILABLE:
+            print("Skipping 3D metrics plot (matplotlib not available)")
+            return
+        
+        print("\nGenerating 3D metrics comparison plots...")
+        
+        comparison = self.comparison_results.get('3d_metrics', {})
+        if not comparison:
+            return
+        
+        # Sort distance ranges by starting distance value
+        def get_range_start(range_key):
+            if range_key == 'overall':
+                return -1  # Put 'overall' at the beginning
+            try:
+                # Extract starting distance from format like "0-20m" or "100-999m"
+                return int(range_key.split('-')[0])
+            except (ValueError, IndexError):
+                return float('inf')
+        
+        # For each class with distance ranges
+        for class_name, ranges in comparison.items():
+            if not ranges:
+                continue
+            
+            # Check if we have distance ranges, sorted by distance
+            range_keys = sorted([k for k in ranges.keys() if k != 'overall'], key=get_range_start)
+            if not range_keys:
+                range_keys = ['overall']
+            
+            # Create subplots for lateral, longitudinal, and heading errors
+            fig, axes = plt.subplots(1, 3, figsize=(18, 5))
+            fig.suptitle(f'3D Detection Metrics - {class_name.upper()}', fontsize=16, fontweight='bold')
+            
+            error_types = ['lateral_error', 'longitudinal_error', 'heading_error']
+            error_names = ['Lateral Error (m)', 'Longitudinal Error (m)', 'Heading Error (rad)']
+            
+            for idx, (error_type, error_name) in enumerate(zip(error_types, error_names)):
+                m1_values = []
+                m2_values = []
+                m1_stds = []
+                m2_stds = []
+                labels = []
+                
+                for range_key in range_keys:
+                    if range_key not in ranges:
+                        continue
+                    
+                    metrics = ranges[range_key]
+                    if error_type not in metrics:
+                        continue
+                    
+                    data = metrics[error_type]
+                    m1_values.append(data[self.model1_name]['mean'])
+                    m2_values.append(data[self.model2_name]['mean'])
+                    m1_stds.append(data[self.model1_name]['std'])
+                    m2_stds.append(data[self.model2_name]['std'])
+                    labels.append(range_key)
+                
+                if not m1_values:
+                    continue
+                
+                x = np.arange(len(labels))
+                width = 0.35
+                
+                bars1 = axes[idx].bar(x - width/2, m1_values, width, 
+                                     yerr=m1_stds, label=self.model1_name, 
+                                     color='#3498db', alpha=0.8, capsize=5)
+                bars2 = axes[idx].bar(x + width/2, m2_values, width,
+                                     yerr=m2_stds, label=self.model2_name,
+                                     color='#e74c3c', alpha=0.8, capsize=5)
+                
+                axes[idx].set_xlabel('Distance Range', fontsize=10)
+                axes[idx].set_ylabel(error_name, fontsize=10)
+                axes[idx].set_title(error_name.split('(')[0], fontsize=12)
+                axes[idx].set_xticks(x)
+                axes[idx].set_xticklabels(labels, rotation=45, ha='right')
+                axes[idx].legend()
+                axes[idx].grid(axis='y', alpha=0.3)
+            
+            plt.tight_layout()
+            output_file = os.path.join(output_dir, f'comparison_3d_{class_name}.png')
+            plt.savefig(output_file, dpi=150, bbox_inches='tight')
+            plt.close()
+            print(f"  ✓ Saved: {output_file}")
+    
+    def plot_improvement_heatmap(self, output_dir):
+        """Plot improvement heatmap for 3D metrics."""
+        if not MATPLOTLIB_AVAILABLE:
+            print("Skipping improvement heatmap (matplotlib not available)")
+            return
+        
+        print("\nGenerating improvement heatmap...")
+        
+        comparison = self.comparison_results.get('3d_metrics', {})
+        if not comparison:
+            return
+        
+        # Collect improvement data
+        data_matrix = []
+        row_labels = []
+        col_labels = ['Lateral', 'Longitudinal', 'Heading']
+        
+        # Sort distance ranges by starting distance value
+        def get_range_start(range_key):
+            if range_key == 'overall':
+                return -1  # Put 'overall' at the beginning of each class
+            try:
+                # Extract starting distance from format like "0-20m" or "100-999m"
+                return int(range_key.split('-')[0])
+            except (ValueError, IndexError):
+                return float('inf')
+        
+        for class_name, ranges in sorted(comparison.items()):
+            # Sort ranges: overall first, then by distance
+            sorted_range_keys = sorted(ranges.keys(), key=get_range_start)
+            for range_key in sorted_range_keys:
+                metrics = ranges[range_key]
+                
+                row_data = []
+                for error_type in ['lateral_error', 'longitudinal_error', 'heading_error']:
+                    if error_type in metrics:
+                        # Negative change % means improvement (lower error)
+                        change = -metrics[error_type]['relative_change_%']
+                        row_data.append(change)
+                    else:
+                        row_data.append(0)
+                
+                if any(x != 0 for x in row_data):
+                    data_matrix.append(row_data)
+                    label = f"{class_name}\n{range_key}"
+                    row_labels.append(label)
+        
+        if not data_matrix:
+            return
+        
+        data_matrix = np.array(data_matrix)
+        
+        fig, ax = plt.subplots(figsize=(10, max(6, len(row_labels) * 0.5)))
+        
+        # Create heatmap
+        im = ax.imshow(data_matrix, cmap='RdYlGn', aspect='auto', vmin=-50, vmax=50)
+        
+        # Set ticks
+        ax.set_xticks(np.arange(len(col_labels)))
+        ax.set_yticks(np.arange(len(row_labels)))
+        ax.set_xticklabels(col_labels)
+        ax.set_yticklabels(row_labels, fontsize=8)
+        
+        # Add colorbar
+        cbar = plt.colorbar(im, ax=ax)
+        cbar.set_label(f'Improvement % ({self.model2_name} vs {self.model1_name})', rotation=270, labelpad=20)
+        
+        # Add text annotations
+        for i in range(len(row_labels)):
+            for j in range(len(col_labels)):
+                text = ax.text(j, i, f'{data_matrix[i, j]:.1f}%',
+                             ha="center", va="center", color="black", fontsize=8)
+        
+        ax.set_title(f'3D Metrics Improvement Heatmap\n(Positive = {self.model2_name} Better)', 
+                    fontsize=14, fontweight='bold')
+        
+        plt.tight_layout()
+        output_file = os.path.join(output_dir, 'comparison_3d_improvement_heatmap.png')
+        plt.savefig(output_file, dpi=150, bbox_inches='tight')
+        plt.close()
+        print(f"  ✓ Saved: {output_file}")
+    
+    def generate_visualizations(self, output_dir):
+        """Generate all visualizations."""
+        if not MATPLOTLIB_AVAILABLE:
+            print("\n⚠ Matplotlib not available, skipping visualizations")
+            print("Install with: pip install matplotlib")
+            return
+        
+        print("\n" + "="*80)
+        print("GENERATING VISUALIZATIONS")
+        print("="*80)
+        
+        self.plot_2d_metrics_comparison(output_dir)
+        self.plot_3d_metrics_comparison(output_dir)
+        self.plot_improvement_heatmap(output_dir)
+        
+        print("\n✓ All visualizations generated")
+
+
+def main():
+    """Main function."""
+    parser = argparse.ArgumentParser(
+        description='Compare evaluation results from two models with visualization',
+        formatter_class=argparse.RawDescriptionHelpFormatter
+    )
+    
+    parser.add_argument('--model1', type=str, required=True,
+                       help='Path to model 1 evaluation report JSON')
+    parser.add_argument('--model2', type=str, required=True,
+                       help='Path to model 2 evaluation report JSON')
+    parser.add_argument('--output-dir', type=str, default='comparison_results',
+                       help='Output directory for comparison results')
+    parser.add_argument('--model1-name', type=str, default='Model-1',
+                       help='Display name for model 1')
+    parser.add_argument('--model2-name', type=str, default='Model-2',
+                       help='Display name for model 2')
+    parser.add_argument('--no-plots', action='store_true',
+                       help='Skip visualization generation')
+    
+    args = parser.parse_args()
+    
+    # Load reports
+    print("="*80)
+    print("MODEL COMPARISON TOOL (with Visualization)")
+    print("="*80)
+    print(f"\nLoading model 1: {args.model1}")
+    with open(args.model1, 'r') as f:
+        model1_report = json.load(f)
+    
+    print(f"Loading model 2: {args.model2}")
+    with open(args.model2, 'r') as f:
+        model2_report = json.load(f)
+    
+    # Create output directory
+    os.makedirs(args.output_dir, exist_ok=True)
+    
+    # Compare models
+    comparator = VisualizationComparator(
+        model1_report,
+        model2_report,
+        model1_name=args.model1_name,
+        model2_name=args.model2_name
+    )
+    
+    results = comparator.compare_all()
+    
+    # Generate reports
+    text_output = os.path.join(args.output_dir, 'comparison_report.txt')
+    json_output = os.path.join(args.output_dir, 'comparison_report.json')
+    
+    comparator.generate_text_report(text_output)
+    comparator.generate_json_report(json_output)
+    
+    # Generate visualizations
+    if not args.no_plots:
+        comparator.generate_visualizations(args.output_dir)
+    
+    print("\n" + "="*80)
+    print("COMPARISON COMPLETE")
+    print("="*80)
+    print(f"\nResults saved to: {args.output_dir}/")
+    print(f"  - Text report: comparison_report.txt")
+    print(f"  - JSON report: comparison_report.json")
+    if not args.no_plots and MATPLOTLIB_AVAILABLE:
+        print(f"  - Visualization plots: comparison_*.png")
+    print("")
+
+
+if __name__ == '__main__':
+    main()

eval_tools/model_comparison/compare_models_with_common_matches.sh

@@ -0,0 +1,185 @@
+#!/bin/bash
+# 
+# Common Match Comparison Workflow Example
+# 
+# This script demonstrates the complete workflow for comparing two models
+# using only the GT objects that both models successfully matched.
+#
+# Usage:
+#   bash eval_tools/compare_models_with_common_matches.sh
+
+set -e  # Exit on error
+
+# Set PYTHONPATH to project root for module imports
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+PROJECT_ROOT="$(cd "${SCRIPT_DIR}/../.." && pwd)"
+export PYTHONPATH="${PROJECT_ROOT}:${PYTHONPATH}"
+
+echo "================================================================================"
+echo "Common Match Comparison Workflow"
+echo "================================================================================"
+
+# Configuration
+MODEL1_CONFIG="eval_tools/configs/eval_config_yolov5s_cncap_768-roi1.yaml"
+MODEL2_CONFIG="eval_tools/configs/eval_config_yolov5s_cncap-roi1.yaml"
+OUTPUT_BASE="evaluation_results/eval_results_common_match_comparison_cncap_yolov5s_768_roi1-conf0.4-v2"
+TIMESTAMP=$(date +%Y%m%d_%H%M%S)
+
+MODEL1_NAME="20260317"
+MODEL2_NAME="20260228"
+
+# Heading tolerance mode: strict, relaxed, or both
+HEADING_TOLERANCE="both"
+
+# Step 1: Evaluate Model 1 with detailed match saving
+echo ""
+echo "Step 1: Evaluating Model 1 (${MODEL1_NAME}) with detailed match saving..."
+echo "        Heading tolerance: ${HEADING_TOLERANCE}"
+echo "--------------------------------------------------------------------------------"
+MODEL1_OUTPUT="${OUTPUT_BASE}/${MODEL1_NAME}/${TIMESTAMP}"
+python eval_tools/core/eval.py \
+  --config ${MODEL1_CONFIG} \
+  --output-dir ${MODEL1_OUTPUT} \
+  --heading-tolerance ${HEADING_TOLERANCE} \
+  --save-detailed-matches
+
+# Use the output directory we specified
+MODEL1_DIR=${MODEL1_OUTPUT}
+if [ ! -d "$MODEL1_DIR" ]; then
+    echo "Error: Could not find Model 1 evaluation results at ${MODEL1_DIR}"
+    exit 1
+fi
+echo "✓ Model 1 results: ${MODEL1_DIR}"
+
+# Generate Markdown report for Model 1
+python ${SCRIPT_DIR}/generate_eval_report.py \
+  ${MODEL1_DIR}/evaluation_report.json \
+  --model "${MODEL1_NAME}" \
+  --date $(date +%Y-%m-%d)
+echo "✓ Model 1 Markdown report: ${MODEL1_DIR}/EVALUATION_REPORT.md"
+echo ""
+echo "Step 2: Evaluating Model 2 (${MODEL2_NAME}) with detailed match saving..."
+echo "        Heading tolerance: ${HEADING_TOLERANCE}"
+echo "--------------------------------------------------------------------------------"
+MODEL2_OUTPUT="${OUTPUT_BASE}/${MODEL2_NAME}/${TIMESTAMP}"
+python eval_tools/core/eval.py \
+  --config ${MODEL2_CONFIG} \
+  --output-dir ${MODEL2_OUTPUT} \
+  --heading-tolerance ${HEADING_TOLERANCE} \
+  --save-detailed-matches
+
+# Use the output directory we specified
+MODEL2_DIR=${MODEL2_OUTPUT}
+if [ ! -d "$MODEL2_DIR" ]; then
+    echo "Error: Could not find Model 2 evaluation results at ${MODEL2_DIR}"
+    exit 1
+fi
+echo "✓ Model 2 results: ${MODEL2_DIR}"
+
+# Generate Markdown report for Model 2
+python ${SCRIPT_DIR}/generate_eval_report.py \
+  ${MODEL2_DIR}/evaluation_report.json \
+  --model "${MODEL2_NAME}" \
+  --date $(date +%Y-%m-%d)
+echo "✓ Model 2 Markdown report: ${MODEL2_DIR}/EVALUATION_REPORT.md"
+
+# Step 3: Find common matches
+echo ""
+echo "Step 3: Finding common matches between the two models..."
+echo "--------------------------------------------------------------------------------"
+COMMON_MATCHES_DIR="${OUTPUT_BASE}/common_matches_${TIMESTAMP}"
+mkdir -p ${COMMON_MATCHES_DIR}
+
+python eval_tools/model_comparison/find_common_matches.py \
+  --model1-matches ${MODEL1_DIR}/detailed_3d_matches.json \
+  --model2-matches ${MODEL2_DIR}/detailed_3d_matches.json \
+  --output ${COMMON_MATCHES_DIR}/common_matches.json \
+  --model1-name "${MODEL1_NAME}" \
+  --model2-name "${MODEL2_NAME}"
+
+echo "✓ Common matches saved to: ${COMMON_MATCHES_DIR}/common_matches.json"
+
+# Step 4: Compare models using common matches
+echo ""
+echo "Step 4: Comparing models using common matches only..."
+echo "--------------------------------------------------------------------------------"
+COMPARISON_DIR="${OUTPUT_BASE}/comparison_common_matches_${TIMESTAMP}"
+
+python eval_tools/model_comparison/compare_models.py \
+  --model1 ${MODEL1_DIR}/evaluation_report.json \
+  --model2 ${MODEL2_DIR}/evaluation_report.json \
+  --model1-name "${MODEL1_NAME}" \
+  --model2-name "${MODEL2_NAME}" \
+  --common-matches ${COMMON_MATCHES_DIR}/common_matches.json \
+  --output-dir ${COMPARISON_DIR}
+
+echo "✓ Comparison results saved to: ${COMPARISON_DIR}"
+
+# Generate Markdown report for common-match comparison
+python ${SCRIPT_DIR}/generate_comparison_report.py \
+  ${COMPARISON_DIR}/comparison_report.json \
+  --date $(date +%Y-%m-%d)
+echo "✓ Markdown report: ${COMPARISON_DIR}/COMPARISON_REPORT.md"
+
+# Step 5: Also run traditional comparison (without common match filtering)
+echo ""
+echo "Step 5: Running traditional comparison (all matches) for reference..."
+echo "--------------------------------------------------------------------------------"
+COMPARISON_TRADITIONAL_DIR="${OUTPUT_BASE}/comparison_all_matches_${TIMESTAMP}"
+
+python eval_tools/model_comparison/compare_models.py \
+  --model1 ${MODEL1_DIR}/evaluation_report.json \
+  --model2 ${MODEL2_DIR}/evaluation_report.json \
+  --model1-name "${MODEL1_NAME}" \
+  --model2-name "${MODEL2_NAME}" \
+  --output-dir ${COMPARISON_TRADITIONAL_DIR}
+
+echo "✓ Traditional comparison saved to: ${COMPARISON_TRADITIONAL_DIR}"
+
+# Generate Markdown report for traditional comparison
+python ${SCRIPT_DIR}/generate_comparison_report.py \
+  ${COMPARISON_TRADITIONAL_DIR}/comparison_report.json \
+  --date $(date +%Y-%m-%d)
+echo "✓ Markdown report: ${COMPARISON_TRADITIONAL_DIR}/COMPARISON_REPORT.md"
+
+# Summary
+echo ""
+echo "================================================================================"
+echo "WORKFLOW COMPLETE!"
+echo "================================================================================"
+echo ""
+echo "Results Summary:"
+echo "  Model 1 (${MODEL1_NAME}):"
+echo "    - Evaluation: ${MODEL1_DIR}"
+echo "    - Detailed matches: ${MODEL1_DIR}/detailed_3d_matches.json"
+echo "    - Markdown report: ${MODEL1_DIR}/EVALUATION_REPORT.md"
+echo ""
+echo "  Model 2 (${MODEL2_NAME}):"
+echo "    - Evaluation: ${MODEL2_DIR}"
+echo "    - Detailed matches: ${MODEL2_DIR}/detailed_3d_matches.json"
+echo "    - Markdown report: ${MODEL2_DIR}/EVALUATION_REPORT.md"
+echo ""
+echo "  Common Matches Analysis:"
+echo "    - Common matches data: ${COMMON_MATCHES_DIR}/common_matches.json"
+echo ""
+echo "  Comparison Results:"
+echo "    - Common matches only: ${COMPARISON_DIR}/"
+echo "    - All matches (traditional): ${COMPARISON_TRADITIONAL_DIR}/"
+echo ""
+echo "Key Files to Review:"
+echo "  1. ${COMPARISON_DIR}/comparison_report.txt"
+echo "     (3D comparison based on common matches - fair comparison)"
+echo "  1b. ${COMPARISON_DIR}/COMPARISON_REPORT.md"
+echo "     (Markdown report - common matches)"
+echo ""
+echo "  2. ${COMPARISON_TRADITIONAL_DIR}/comparison_report.txt"
+echo "     (Traditional comparison with all matches - for reference)"
+echo "  2b. ${COMPARISON_TRADITIONAL_DIR}/COMPARISON_REPORT.md"
+echo "     (Markdown report - all matches)"
+echo ""
+echo "  3. ${COMMON_MATCHES_DIR}/common_matches.json"
+echo "     (Detailed statistics about match differences)"
+echo ""
+echo "To view the common-match comparison report:"
+echo "  cat ${COMPARISON_DIR}/comparison_report.txt"
+echo ""

eval_tools/model_comparison/compare_per_case_2d.py

@@ -0,0 +1,285 @@
+#!/usr/bin/env python3
+"""
+Per-Case 2D Metrics Comparison Tool
+
+This script compares per_case_2d metrics from two model evaluation reports
+and identifies cases with significant metric differences.
+
+Usage:
+    python eval_tools/model_comparison/compare_per_case_2d.py \
+        --model1 evaluation_results/.../evaluation_report.json \
+        --model2 evaluation_results/.../evaluation_report.json \
+        --threshold 0.1 \
+        --output comparison_per_case_2d.json
+
+Example:
+    python eval_tools/model_comparison/compare_per_case_2d.py \
+        --model1 evaluation_results/eval_results_common_match_comparison_CNCAP_roi0/mono3d/20260211_113153/evaluation_report.json \
+        --model2 evaluation_results/eval_results_common_match_comparison_CNCAP_roi0/yolov5s-300w-newdata/20260211_113153/evaluation_report.json \
+        --threshold 0.1 \
+        --output per_case_comparison.json
+"""
+
+import argparse
+import json
+import os
+from pathlib import Path
+from collections import defaultdict
+import sys
+import numpy as np
+
+# Allow importing class_config from the eval_tools root
+sys.path.insert(0, str(Path(__file__).parent.parent))
+from class_config import CLASS_NAMES
+
+
+class PerCaseComparator:
+    """Compare per_case_2d metrics between two models."""
+
+    def __init__(self, model1_report, model2_report, model1_name="Model-1", model2_name="Model-2"):
+        """
+        Initialize comparator.
+
+        Args:
+            model1_report: dict, evaluation report for model 1
+            model2_report: dict, evaluation report for model 2
+            model1_name: str, display name for model 1
+            model2_name: str, display name for model 2
+        """
+        self.model1_report = model1_report
+        self.model2_report = model2_report
+        self.model1_name = model1_name
+        self.model2_name = model2_name
+
+    def compare_per_case_metrics(self, threshold=0.1, metric_name='ap'):
+        """
+        Compare per_case_2d metrics and identify cases with significant differences.
+
+        Args:
+            threshold: float, threshold for significant difference (default 0.1 = 10%)
+            metric_name: str, metric to compare ('ap', 'precision', 'recall')
+
+        Returns:
+            dict with comparison results
+        """
+        print(f"\n{'='*80}")
+        print(f"Comparing Per-Case 2D Metrics (threshold={threshold*100}%)")
+        print(f"{'='*80}\n")
+
+        # Get per_case_2d data
+        m1_cases = self.model1_report.get('per_case_2d', {})
+        m2_cases = self.model2_report.get('per_case_2d', {})
+
+        # Find common cases
+        common_cases = set(m1_cases.keys()) & set(m2_cases.keys())
+        print(f"Total cases in {self.model1_name}: {len(m1_cases)}")
+        print(f"Total cases in {self.model2_name}: {len(m2_cases)}")
+        print(f"Common cases: {len(common_cases)}\n")
+
+        # Compare each case
+        case_comparisons = {}
+        significant_diffs = []
+
+        for case_name in sorted(common_cases):
+            m1_case = m1_cases[case_name]
+            m2_case = m2_cases[case_name]
+
+            case_comp = {
+                'case_name': case_name,
+                'per_class': {},
+                'max_diff': 0.0,
+                'max_diff_class': None,
+                'max_diff_metric': None
+            }
+
+            # Compare per-class metrics
+            m1_classes = m1_case.get('per_class', {})
+            m2_classes = m2_case.get('per_class', {})
+
+            for class_name in m1_classes.keys():
+                if class_name not in m2_classes:
+                    continue
+
+                m1_class = m1_classes[class_name]
+                m2_class = m2_classes[class_name]
+
+                class_comp = {}
+                for metric in ['precision', 'recall', 'ap']:
+                    m1_val = m1_class.get(metric, 0.0)
+                    m2_val = m2_class.get(metric, 0.0)
+                    diff = m2_val - m1_val
+
+                    class_comp[metric] = {
+                        self.model1_name: m1_val,
+                        self.model2_name: m2_val,
+                        'diff': diff,
+                        'abs_diff': abs(diff)
+                    }
+
+                    # Track maximum difference
+                    if abs(diff) > case_comp['max_diff']:
+                        case_comp['max_diff'] = abs(diff)
+                        case_comp['max_diff_class'] = class_name
+                        case_comp['max_diff_metric'] = metric
+
+                # Add count metrics for context
+                class_comp['counts'] = {
+                    'num_gt': m1_class.get('num_gt', 0),
+                    'num_det_m1': m1_class.get('num_det', 0),
+                    'num_det_m2': m2_class.get('num_det', 0),
+                }
+
+                case_comp['per_class'][class_name] = class_comp
+
+            case_comparisons[case_name] = case_comp
+
+            # Check if this case has significant differences
+            if case_comp['max_diff'] >= threshold:
+                significant_diffs.append({
+                    'case_name': case_name,
+                    'max_diff': case_comp['max_diff'],
+                    'class': case_comp['max_diff_class'],
+                    'metric': case_comp['max_diff_metric'],
+                    'details': case_comp['per_class'][case_comp['max_diff_class']][case_comp['max_diff_metric']]
+                })
+
+        # Sort by maximum difference
+        significant_diffs.sort(key=lambda x: x['max_diff'], reverse=True)
+
+        results = {
+            'summary': {
+                'total_common_cases': len(common_cases),
+                'cases_with_significant_diff': len(significant_diffs),
+                'threshold': threshold,
+                'model1_name': self.model1_name,
+                'model2_name': self.model2_name
+            },
+            'significant_differences': significant_diffs,
+            'all_case_comparisons': case_comparisons
+        }
+
+        return results
+
+    def print_significant_differences(self, results, top_n=20):
+        """Print top N cases with significant differences."""
+        sig_diffs = results['significant_differences']
+        
+        print(f"\n{'='*80}")
+        print(f"Top {min(top_n, len(sig_diffs))} Cases with Significant Differences")
+        print(f"{'='*80}\n")
+
+        for i, diff in enumerate(sig_diffs[:top_n], 1):
+            details = diff['details']
+            print(f"{i}. Case: {diff['case_name']}")
+            print(f"   Class: {diff['class']}, Metric: {diff['metric']}")
+            print(f"   {self.model1_name}: {details[self.model1_name]:.4f}")
+            print(f"   {self.model2_name}: {details[self.model2_name]:.4f}")
+            print(f"   Difference: {details['diff']:+.4f} (abs: {diff['max_diff']:.4f})")
+            print()
+
+    def generate_summary_stats(self, results):
+        """Generate summary statistics."""
+        all_comps = results['all_case_comparisons']
+        
+        # Collect all differences by class and metric
+        diffs_by_class_metric = defaultdict(list)
+        
+        for case_name, case_comp in all_comps.items():
+            for class_name, class_comp in case_comp['per_class'].items():
+                for metric in ['precision', 'recall', 'ap']:
+                    diff = class_comp[metric]['diff']
+                    diffs_by_class_metric[(class_name, metric)].append(diff)
+        
+        # Calculate statistics
+        stats = {}
+        for (class_name, metric), diffs in diffs_by_class_metric.items():
+            diffs_array = np.array(diffs)
+            stats[f"{class_name}_{metric}"] = {
+                'mean_diff': float(np.mean(diffs_array)),
+                'std_diff': float(np.std(diffs_array)),
+                'median_diff': float(np.median(diffs_array)),
+                'min_diff': float(np.min(diffs_array)),
+                'max_diff': float(np.max(diffs_array)),
+                'num_cases': len(diffs)
+            }
+        
+        return stats
+
+
+def main():
+    parser = argparse.ArgumentParser(
+        description='Compare per_case_2d metrics between two model evaluation reports'
+    )
+    parser.add_argument('--model1', type=str, required=True,
+                        help='Path to model 1 evaluation_report.json')
+    parser.add_argument('--model2', type=str, required=True,
+                        help='Path to model 2 evaluation_report.json')
+    parser.add_argument('--model1-name', type=str, default='Model-1',
+                        help='Display name for model 1')
+    parser.add_argument('--model2-name', type=str, default='Model-2',
+                        help='Display name for model 2')
+    parser.add_argument('--threshold', type=float, default=0.1,
+                        help='Threshold for significant difference (default: 0.1 = 10%%)')
+    parser.add_argument('--output', type=str, default='per_case_comparison.json',
+                        help='Output JSON file path')
+    parser.add_argument('--top-n', type=int, default=20,
+                        help='Number of top different cases to display (default: 20)')
+    
+    args = parser.parse_args()
+
+    # Load evaluation reports
+    print(f"Loading {args.model1}...")
+    with open(args.model1, 'r') as f:
+        model1_report = json.load(f)
+    
+    print(f"Loading {args.model2}...")
+    with open(args.model2, 'r') as f:
+        model2_report = json.load(f)
+
+    # Create comparator
+    comparator = PerCaseComparator(
+        model1_report, model2_report,
+        model1_name=args.model1_name,
+        model2_name=args.model2_name
+    )
+
+    # Compare metrics
+    results = comparator.compare_per_case_metrics(threshold=args.threshold)
+
+    # Print significant differences
+    comparator.print_significant_differences(results, top_n=args.top_n)
+
+    # Generate summary statistics
+    print(f"\n{'='*80}")
+    print("Summary Statistics")
+    print(f"{'='*80}\n")
+    stats = comparator.generate_summary_stats(results)
+    
+    # Print stats for main classes (all 3D classes, skipping vehicle sub-buckets)
+    for class_name in [n for n in CLASS_NAMES.values() if n in stats or f"{n}_ap" in stats]:
+        print(f"\n{class_name.upper()}:")
+        for metric in ['ap', 'precision', 'recall']:
+            key = f"{class_name}_{metric}"
+            if key in stats:
+                s = stats[key]
+                print(f"  {metric:10s}: mean={s['mean_diff']:+.4f}, "
+                      f"std={s['std_diff']:.4f}, median={s['median_diff']:+.4f}, "
+                      f"range=[{s['min_diff']:+.4f}, {s['max_diff']:+.4f}]")
+
+    # Save results
+    output_path = Path(args.output)
+    output_path.parent.mkdir(parents=True, exist_ok=True)
+    
+    # Add summary stats to results
+    results['summary_statistics'] = stats
+    
+    with open(output_path, 'w') as f:
+        json.dump(results, f, indent=2)
+    
+    print(f"\n{'='*80}")
+    print(f"Results saved to: {output_path}")
+    print(f"{'='*80}\n")
+
+
+if __name__ == '__main__':
+    main()

eval_tools/model_comparison/compare_per_case_2d.sh

@@ -0,0 +1,26 @@
+#!/bin/bash
+# Compare per_case_2d metrics between mono3d and yolov5s-300w-newdata models
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+PROJECT_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
+
+MODEL1_PATH="$PROJECT_ROOT/evaluation_results/eval_results_common_match_comparison_CNCAP_roi1/mono3d/20260211_120939/evaluation_report.json"
+MODEL2_PATH="$PROJECT_ROOT/evaluation_results/eval_results_common_match_comparison_CNCAP_roi1/yolov5s-300w-newdata/20260211_120939/evaluation_report.json"
+OUTPUT_PATH="$PROJECT_ROOT/evaluation_results/eval_results_common_match_comparison_CNCAP_roi1/per_case_2d_comparison.json"
+
+echo "Comparing per_case_2d metrics..."
+echo "Model 1: mono3d"
+echo "Model 2: yolov5s-300w-newdata"
+echo ""
+
+python "$SCRIPT_DIR/compare_per_case_2d.py" \
+    --model1 "$MODEL1_PATH" \
+    --model2 "$MODEL2_PATH" \
+    --model1-name "mono3d" \
+    --model2-name "yolov5s-300w-newdata" \
+    --threshold 0.1 \
+    --output "$OUTPUT_PATH" \
+    --top-n 30
+
+echo ""
+echo "Done! Results saved to: $OUTPUT_PATH"

eval_tools/model_comparison/find_common_matches.py

@@ -0,0 +1,439 @@
+#!/usr/bin/env python3
+"""
+Find Common Matches Between Two Models
+
+This tool finds the common GT objects that were successfully matched by both models,
+enabling fair comparison of 3D prediction quality on the same set of targets.
+
+Usage:
+    python eval_tools/find_common_matches.py \
+        --model1-matches eval_results/model1/detailed_3d_matches.json \
+        --model2-matches eval_results/model2/detailed_3d_matches.json \
+        --output common_matches.json
+"""
+
+import argparse
+import json
+import sys
+from pathlib import Path
+from collections import defaultdict
+import numpy as np
+
+
+def find_common_matches(model1_matches, model2_matches):
+    """
+    Find common GT objects matched by both models.
+    
+    Args:
+        model1_matches: dict, detailed matches from model 1
+        model2_matches: dict, detailed matches from model 2
+    
+    Returns:
+        tuple: (common_matches, stats)
+            common_matches: dict with structure {case: {frame: {class: [match_info]}}}
+            stats: dict with match statistics
+    """
+    common_matches = {}
+    stats = {
+        'model1_total': 0,
+        'model2_total': 0,
+        'common': 0,
+        'model1_unique': 0,
+        'model2_unique': 0,
+        'per_class': {}
+    }
+    
+    # Iterate through all cases in model 1
+    for case_name in model1_matches:
+        if case_name not in model2_matches:
+            # Case not in model 2, skip
+            continue
+        
+        common_matches[case_name] = {}
+        
+        # Iterate through frames
+        for frame_name in model1_matches[case_name]:
+            if frame_name not in model2_matches[case_name]:
+                # Frame not in model 2, skip
+                continue
+            
+            common_matches[case_name][frame_name] = {}
+            
+            # Iterate through classes
+            for class_name in model1_matches[case_name][frame_name]:
+                if class_name not in model2_matches[case_name][frame_name]:
+                    # Class not in model 2, skip
+                    continue
+                
+                # Get match lists for this class
+                m1_list = model1_matches[case_name][frame_name][class_name]
+                m2_list = model2_matches[case_name][frame_name][class_name]
+                
+                # Build GT ID to index mappings
+                m1_gt_ids = {m['gt_id']: i for i, m in enumerate(m1_list)}
+                m2_gt_ids = {m['gt_id']: i for i, m in enumerate(m2_list)}
+                
+                # Find common GT IDs
+                common_gt_ids = set(m1_gt_ids.keys()) & set(m2_gt_ids.keys())
+                
+                # Update statistics
+                if class_name not in stats['per_class']:
+                    stats['per_class'][class_name] = {
+                        'model1_total': 0,
+                        'model2_total': 0,
+                        'common': 0,
+                        'model1_unique': 0,
+                        'model2_unique': 0
+                    }
+                
+                stats['model1_total'] += len(m1_list)
+                stats['model2_total'] += len(m2_list)
+                stats['common'] += len(common_gt_ids)
+                stats['model1_unique'] += len(m1_gt_ids) - len(common_gt_ids)
+                stats['model2_unique'] += len(m2_gt_ids) - len(common_gt_ids)
+                
+                stats['per_class'][class_name]['model1_total'] += len(m1_list)
+                stats['per_class'][class_name]['model2_total'] += len(m2_list)
+                stats['per_class'][class_name]['common'] += len(common_gt_ids)
+                stats['per_class'][class_name]['model1_unique'] += len(m1_gt_ids) - len(common_gt_ids)
+                stats['per_class'][class_name]['model2_unique'] += len(m2_gt_ids) - len(common_gt_ids)
+                
+                # Store common match information
+                common_list = []
+                for gt_id in common_gt_ids:
+                    common_list.append({
+                        'gt_id': gt_id,
+                        'model1_idx': m1_gt_ids[gt_id],
+                        'model2_idx': m2_gt_ids[gt_id]
+                    })
+                
+                if common_list:
+                    common_matches[case_name][frame_name][class_name] = common_list
+    
+    # Calculate percentages
+    if stats['model1_total'] > 0:
+        stats['common_percentage_of_model1'] = (stats['common'] / stats['model1_total']) * 100
+    else:
+        stats['common_percentage_of_model1'] = 0
+    
+    if stats['model2_total'] > 0:
+        stats['common_percentage_of_model2'] = (stats['common'] / stats['model2_total']) * 100
+    else:
+        stats['common_percentage_of_model2'] = 0
+    
+    for class_name in stats['per_class']:
+        class_stats = stats['per_class'][class_name]
+        if class_stats['model1_total'] > 0:
+            class_stats['common_percentage_of_model1'] = (class_stats['common'] / class_stats['model1_total']) * 100
+        else:
+            class_stats['common_percentage_of_model1'] = 0
+        
+        if class_stats['model2_total'] > 0:
+            class_stats['common_percentage_of_model2'] = (class_stats['common'] / class_stats['model2_total']) * 100
+        else:
+            class_stats['common_percentage_of_model2'] = 0
+    
+    return common_matches, stats
+
+
+# Default distance ranges matching eval metrics_3d config
+DEFAULT_LONG_RANGES = [
+    (0, 10), (10, 20), (20, 30), (30, 40), (40, 50),
+    (50, 60), (60, 70), (70, 80), (80, 90), (90, 100), (100, 999)
+]
+DEFAULT_LAT_RANGES = [
+    (-50, -40), (-40, -30), (-30, -20), (-20, -10), (-10, 0),
+    (0, 10), (10, 20), (20, 30), (30, 40), (40, 50)
+]
+
+
+def _range_key_long(lo, hi):
+    return f'long_{lo}-{hi}m'
+
+
+def _range_key_lat(lo, hi):
+    return f'lat_{lo}-{hi}m'
+
+
+def _make_stats(data_dict):
+    """Compute mean/std/median/min/max for each list in data_dict."""
+    result = {}
+    for key, values in data_dict.items():
+        if key in ('samples',):
+            result[key] = values
+        elif isinstance(values, list) and len(values) > 0:
+            arr = np.array(values)
+            result[key] = {
+                'mean':   float(np.mean(arr)),
+                'std':    float(np.std(arr)),
+                'median': float(np.median(arr)),
+                'min':    float(np.min(arr)),
+                'max':    float(np.max(arr)),
+            }
+    return result
+
+
+def _empty_bucket():
+    return {
+        'lateral': [], 'longitudinal': [], 'longitudinal_relative': [],
+        'heading': [], 'heading_relaxed': [], 'is_reversal': [], 'samples': 0
+    }
+
+
+def _finalize_class_stats(data):
+    """Convert a bucket dict to stats dict, adding optional fields."""
+    entry = {
+        'num_samples': data['samples'],
+        'lateral_error': _make_stats({'lateral': data['lateral']})['lateral'],
+        'longitudinal_error': _make_stats({'longitudinal': data['longitudinal']})['longitudinal'],
+        'heading_error': _make_stats({'heading': data['heading']})['heading'],
+    }
+    if data['longitudinal_relative']:
+        entry['longitudinal_relative_error'] = _make_stats(
+            {'v': data['longitudinal_relative']})['v']
+    if data['heading_relaxed']:
+        entry['heading_error_relaxed'] = _make_stats(
+            {'v': data['heading_relaxed']})['v']
+    if data['is_reversal']:
+        count = int(sum(data['is_reversal']))
+        entry['reversal_count'] = count
+        entry['reversal_percentage'] = float(count / data['samples'] * 100) if data['samples'] > 0 else 0.0
+    return entry
+
+
+def recompute_3d_stats_from_common_matches(matches_data, common_matches, model_name,
+                                           long_ranges=None, lat_ranges=None):
+    """
+    Recompute 3D statistics based on common matches only.
+
+    Returns a dict with structure:
+      {
+        class_name: {
+          'overall': { ... },
+          'long_0-10m': { ... },
+          ...
+          'lat_-10-0m': { ... },
+          ...
+        }
+      }
+    """
+    if long_ranges is None:
+        long_ranges = DEFAULT_LONG_RANGES
+    if lat_ranges is None:
+        lat_ranges = DEFAULT_LAT_RANGES
+
+    # Bucket structure: class -> range_key -> _empty_bucket()
+    overall = {}   # class -> _empty_bucket()
+    by_long = {}   # class -> range_key -> _empty_bucket()
+    by_lat  = {}   # class -> range_key -> _empty_bucket()
+
+    for case_name, frames in common_matches.items():
+        for frame_name, classes in frames.items():
+            for class_name, common_list in classes.items():
+                if class_name not in overall:
+                    overall[class_name] = _empty_bucket()
+                    by_long[class_name] = {
+                        _range_key_long(lo, hi): _empty_bucket()
+                        for lo, hi in long_ranges
+                    }
+                    by_lat[class_name] = {
+                        _range_key_lat(lo, hi): _empty_bucket()
+                        for lo, hi in lat_ranges
+                    }
+
+                for match_info in common_list:
+                    idx  = match_info[f'{model_name}_idx']
+                    match = matches_data[case_name][frame_name][class_name][idx]
+                    errs  = match['errors']
+                    dist  = match.get('distance', {})
+                    z_val = dist.get('longitudinal', None)
+                    x_val = dist.get('lateral', None)
+
+                    # Helper: fill one bucket
+                    def _fill(bucket):
+                        bucket['lateral'].append(errs['lateral'])
+                        bucket['longitudinal'].append(errs['longitudinal'])
+                        bucket['heading'].append(errs['heading'])
+                        if 'longitudinal_relative' in errs:
+                            bucket['longitudinal_relative'].append(errs['longitudinal_relative'])
+                        if 'heading_relaxed' in errs:
+                            bucket['heading_relaxed'].append(errs['heading_relaxed'])
+                        if 'is_reversal' in errs:
+                            bucket['is_reversal'].append(errs['is_reversal'])
+                        bucket['samples'] += 1
+
+                    _fill(overall[class_name])
+
+                    # Longitudinal range bucket
+                    if z_val is not None:
+                        for lo, hi in long_ranges:
+                            if lo <= z_val < hi:
+                                _fill(by_long[class_name][_range_key_long(lo, hi)])
+                                break
+
+                    # Lateral range bucket
+                    if x_val is not None:
+                        for lo, hi in lat_ranges:
+                            if lo <= x_val < hi:
+                                _fill(by_lat[class_name][_range_key_lat(lo, hi)])
+                                break
+
+    # Build result
+    result = {}
+    for class_name in overall:
+        result[class_name] = {}
+
+        # overall
+        if overall[class_name]['samples'] > 0:
+            result[class_name]['overall'] = _finalize_class_stats(overall[class_name])
+        else:
+            result[class_name]['overall'] = {'num_samples': 0}
+
+        # per longitudinal range
+        for rk, bucket in by_long[class_name].items():
+            if bucket['samples'] > 0:
+                result[class_name][rk] = _finalize_class_stats(bucket)
+
+        # per lateral range
+        for rk, bucket in by_lat[class_name].items():
+            if bucket['samples'] > 0:
+                result[class_name][rk] = _finalize_class_stats(bucket)
+
+    return result
+
+
+def print_statistics(stats, model1_name='model1', model2_name='model2'):
+    """Print match statistics in a readable format."""
+    print("\n" + "="*80)
+    print("COMMON MATCH STATISTICS")
+    print("="*80)
+    
+    print(f"\nOverall:")
+    print(f"  {model1_name} Total Matches: {stats['model1_total']:,}")
+    print(f"  {model2_name} Total Matches: {stats['model2_total']:,}")
+    print(f"  Common Matches: {stats['common']:,} ({stats['common_percentage_of_model1']:.1f}% of {model1_name})")
+    print(f"  {model1_name} Unique: {stats['model1_unique']:,} ({100 - stats['common_percentage_of_model1']:.1f}%)")
+    print(f"  {model2_name} Unique: {stats['model2_unique']:,} ({100 - stats['common_percentage_of_model2']:.1f}%)")
+    
+    print(f"\nPer-Class Statistics:")
+    # Truncate model names if too long for column headers
+    m1_short = model1_name[:10]
+    m2_short = model2_name[:10]
+    print(f"{'Class':<15} {m1_short:>10} {m2_short:>10} {'Common':>10} {'Common%':>10} {m1_short+' Uniq':>12} {m2_short+' Uniq':>12}")
+    print("-" * 80)
+    
+    for class_name, class_stats in sorted(stats['per_class'].items()):
+        print(f"{class_name:<15} {class_stats['model1_total']:>10,} {class_stats['model2_total']:>10,} "
+              f"{class_stats['common']:>10,} {class_stats['common_percentage_of_model1']:>9.1f}% "
+              f"{class_stats['model1_unique']:>12,} {class_stats['model2_unique']:>12,}")
+
+
+def main():
+    """Main function."""
+    parser = argparse.ArgumentParser(
+        description='Find common matches between two model evaluation results',
+        formatter_class=argparse.RawDescriptionHelpFormatter
+    )
+    
+    parser.add_argument('--model1-matches', type=str, required=True,
+                        help='Path to model 1 detailed_3d_matches.json file')
+    parser.add_argument('--model2-matches', type=str, required=True,
+                        help='Path to model 2 detailed_3d_matches.json file')
+    parser.add_argument('--output', type=str, default='common_matches.json',
+                        help='Output path for common matches JSON file')
+    parser.add_argument('--model1-name', type=str, default='model1',
+                        help='Name for model 1 (default: model1)')
+    parser.add_argument('--model2-name', type=str, default='model2',
+                        help='Name for model 2 (default: model2)')
+    
+    args = parser.parse_args()
+    
+    # Load detailed matches
+    print(f"Loading model 1 matches from: {args.model1_matches}")
+    with open(args.model1_matches, 'r') as f:
+        model1_matches = json.load(f)
+    
+    print(f"Loading model 2 matches from: {args.model2_matches}")
+    with open(args.model2_matches, 'r') as f:
+        model2_matches = json.load(f)
+    
+    # Find common matches
+    print("\nFinding common matches...")
+    common_matches, stats = find_common_matches(model1_matches, model2_matches)
+    
+    # Print statistics
+    print_statistics(stats, args.model1_name, args.model2_name)
+    
+    # Recompute 3D stats for common matches
+    print("\nRecomputing 3D statistics for common matches...")
+    model1_stats = recompute_3d_stats_from_common_matches(model1_matches, common_matches, 'model1')
+    model2_stats = recompute_3d_stats_from_common_matches(model2_matches, common_matches, 'model2')
+    
+    # Prepare output
+    output_data = {
+        'match_statistics': stats,
+        'common_matches': common_matches,
+        'model1_3d_stats': model1_stats,
+        'model2_3d_stats': model2_stats,
+        'model_names': {
+            'model1': args.model1_name,
+            'model2': args.model2_name
+        }
+    }
+    
+    # Save output
+    output_path = Path(args.output)
+    output_path.parent.mkdir(parents=True, exist_ok=True)
+    
+    with open(output_path, 'w') as f:
+        json.dump(output_data, f, indent=2)
+    
+    print(f"\n✓ Common matches saved to: {output_path}")
+    
+    # Print 3D stats comparison
+    print("\n" + "="*80)
+    print("3D STATISTICS COMPARISON (COMMON MATCHES ONLY)")
+    print("="*80)
+    
+    for class_name in sorted(model1_stats.keys()):
+        if class_name not in model2_stats:
+            continue
+
+        # New format: class stats are nested under 'overall'
+        m1 = model1_stats[class_name].get('overall', model1_stats[class_name])
+        m2 = model2_stats[class_name].get('overall', model2_stats[class_name])
+
+        print(f"\n{class_name.upper()} (n={m1.get('num_samples', 0):,}):")
+        print(f"{'Metric':<20} {args.model1_name:>15} {args.model2_name:>15} {'Diff':>12} {'Change %':>10}")
+        print("-" * 80)
+
+        for error_type in ['lateral_error', 'longitudinal_error', 'heading_error']:
+            if error_type not in m1 or error_type not in m2:
+                continue
+            m1_mean = m1[error_type]['mean']
+            m2_mean = m2[error_type]['mean']
+            diff = m2_mean - m1_mean
+            change_pct = (diff / m1_mean * 100) if m1_mean > 0 else 0
+
+            error_name = error_type.replace('_', ' ').title()
+            print(f"{error_name:<20} {m1_mean:>15.4f} {m2_mean:>15.4f} {diff:>+12.4f} {change_pct:>+9.2f}%")
+
+        # Print relaxed heading error if available
+        if 'heading_error_relaxed' in m1 and 'heading_error_relaxed' in m2:
+            m1_mean = m1['heading_error_relaxed']['mean']
+            m2_mean = m2['heading_error_relaxed']['mean']
+            diff = m2_mean - m1_mean
+            change_pct = (diff / m1_mean * 100) if m1_mean > 0 else 0
+            print(f"{'Heading Error (Rlx)':<20} {m1_mean:>15.4f} {m2_mean:>15.4f} {diff:>+12.4f} {change_pct:>+9.2f}%")
+
+        # Print reversal statistics if available
+        if 'reversal_count' in m1 and 'reversal_count' in m2:
+            m1_count = m1['reversal_count']
+            m1_pct = m1.get('reversal_percentage', 0)
+            m2_count = m2['reversal_count']
+            m2_pct = m2.get('reversal_percentage', 0)
+            print(f"{'Reversals':<20} {m1_count:>11} ({m1_pct:>5.1f}%) {m2_count:>11} ({m2_pct:>5.1f}%)")
+
+
+if __name__ == '__main__':
+    main()

eval_tools/model_comparison/gen_eval_report.sh

@@ -0,0 +1,34 @@
+#!/bin/bash
+#
+# 根据 evaluation_report.json 自动生成单模型 Markdown 评测报告
+#
+# 用法:
+#   bash eval_tools/model_comparison/gen_eval_report.sh <evaluation_report.json>
+#   bash eval_tools/model_comparison/gen_eval_report.sh <evaluation_report.json> [--output <输出路径>] [--model "模型名"] [--date YYYY-MM-DD]
+#
+# 示例:
+#   bash eval_tools/model_comparison/gen_eval_report.sh \
+#       evaluation_results/.../evaluation_report.json
+#
+#   bash eval_tools/model_comparison/gen_eval_report.sh \
+#       evaluation_results/.../evaluation_report.json \
+#       --model yolov5s-300w-newdata-cncap \
+#       --date 2026-02-28
+
+set -e
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+PROJECT_ROOT="$(cd "${SCRIPT_DIR}/../.." && pwd)"
+export PYTHONPATH="${PROJECT_ROOT}:${PYTHONPATH}"
+
+if [ $# -eq 0 ]; then
+    echo "用法: bash $0 <evaluation_report.json> [选项...]"
+    echo ""
+    echo "选项:"
+    echo "  --output/-o <路径>      输出 Markdown 文件路径（默认：JSON 同目录下的 EVALUATION_REPORT.md）"
+    echo "  --model <名称>          模型名称（默认从目录名推断）"
+    echo "  --date <YYYY-MM-DD>     评测日期（默认：今天）"
+    exit 1
+fi
+
+python "${SCRIPT_DIR}/generate_eval_report.py" "$@"

eval_tools/model_comparison/gen_report.sh

@@ -0,0 +1,35 @@
+#!/bin/bash
+#
+# 根据 comparison_report.json 自动生成 Markdown 评测报告
+#
+# 用法:
+#   bash eval_tools/model_comparison/gen_report.sh <comparison_report.json>
+#   bash eval_tools/model_comparison/gen_report.sh <comparison_report.json> [--output <输出路径>] [--title "标题"] [--background "背景说明"] [--date YYYY-MM-DD]
+#
+# 示例:
+#   bash eval_tools/model_comparison/gen_report.sh \
+#       evaluation_results/eval_results_.../comparison_common_matches_.../comparison_report.json
+#
+#   bash eval_tools/model_comparison/gen_report.sh \
+#       evaluation_results/.../comparison_report.json \
+#       --output my_report.md \
+#       --title "ROI0 模型对比报告"
+
+set -e
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+PROJECT_ROOT="$(cd "${SCRIPT_DIR}/../.." && pwd)"
+export PYTHONPATH="${PROJECT_ROOT}:${PYTHONPATH}"
+
+if [ $# -eq 0 ]; then
+    echo "用法: bash $0 <comparison_report.json> [选项...]"
+    echo ""
+    echo "选项:"
+    echo "  --output/-o <路径>      输出 Markdown 文件路径（默认：JSON 同目录下的 COMPARISON_REPORT.md）"
+    echo "  --title <标题>          自定义报告标题"
+    echo "  --background <说明>     背景说明文字"
+    echo "  --date <YYYY-MM-DD>     评测日期（默认：今天）"
+    exit 1
+fi
+
+python "${SCRIPT_DIR}/generate_comparison_report.py" "$@"

eval_tools/model_comparison/generate_comparison_report.py

@@ -0,0 +1,496 @@
+#!/usr/bin/env python3
+"""
+自动将 comparison_report.json 转换为中文 Markdown 评测报告。
+
+用法:
+    python generate_comparison_report.py <comparison_report.json 路径>
+    python generate_comparison_report.py <comparison_report.json 路径> --output <输出文件路径>
+    python generate_comparison_report.py <comparison_report.json 路径> --title "自定义标题"
+    python generate_comparison_report.py <comparison_report.json 路径> --background "背景说明文字"
+    python generate_comparison_report.py <comparison_report.json 路径> --date 2026-03-01
+
+示例:
+    python generate_comparison_report.py \
+        evaluation_results/eval_results_common_match_comparison_cncap_yolov5s_20260228_roi0/comparison_common_matches_20260228_102849/comparison_report.json
+
+    python generate_comparison_report.py \
+        evaluation_results/.../comparison_report.json \
+        --output my_report.md \
+        --title "ROI1 模型对比报告"
+"""
+
+import json
+import re
+import argparse
+import sys
+from datetime import date
+from pathlib import Path
+
+# Allow importing class_config from the eval_tools root
+sys.path.insert(0, str(Path(__file__).parent.parent))
+from class_config import REPORT_3D_CLASS_LABELS
+
+# ── 阈值设置 ─────────────────────────────────────────────────────────────────
+# AP 差异超过此阈值才标记为"优"，否则标记为"持平"
+AP_TIE_THRESHOLD = 0.005       # 0.5%
+METRIC_TIE_THRESHOLD = 0.005   # 用于 precision/recall/f1 的判断阈值（绝对值）
+ERROR_TIE_THRESHOLD_REL = 2.0  # 3D 误差相对变化（%）小于此值视为持平
+
+
+def fmt(v: float, decimals: int = 4) -> str:
+    return f"{v:.{decimals}f}"
+
+
+def fmt_pct(v: float) -> str:
+    sign = "+" if v >= 0 else ""
+    return f"{sign}{v:.2f}%"
+
+
+def fmt_diff(v: float) -> str:
+    sign = "+" if v >= 0 else ""
+    return f"{sign}{v:.4f}"
+
+
+def judge(diff: float, rel: float, higher_is_better: bool = True,
+          abs_thr: float = AP_TIE_THRESHOLD, rel_thr: float = None,
+          model1_name: str = "model1", model2_name: str = "model2") -> str:
+    """
+    根据 diff (model2 - model1) 判断哪个模型更好。
+    higher_is_better=True  → diff>0 代表 model2 更好
+    higher_is_better=False → diff<0 代表 model2 更好（即误差更小）
+    """
+    if rel_thr is not None:
+        tie = abs(rel) < rel_thr
+    else:
+        tie = abs(diff) < abs_thr
+
+    if tie:
+        return "⚖️ 持平"
+
+    m2_better = (diff > 0) if higher_is_better else (diff < 0)
+    m2_short = model2_name.split("-")[-1]   # e.g. "cncap"
+    m1_short = model1_name.split("-")[-1]   # e.g. "newdata", "mono3d"
+    if m2_better:
+        return f"✅ {m2_short}优"
+    else:
+        return f"✅ {m1_short}优"
+
+
+def build_report(data: dict, model1: str, model2: str,
+                 report_date: str, title: str = None, background: str = None) -> str:
+    """生成完整 Markdown 报告字符串。"""
+
+    m2d = data["2d_metrics"]
+    m3d = data.get("3d_metrics", {})
+    stats = data.get("match_statistics", {})
+    summary = data.get("summary", {})
+
+    # ── 名称简写 ──────────────────────────────────────────────────────────────
+    m1_short = model1
+    m2_short = model2
+    # 取最后一段作为简称用于表格
+    m1_tag = model1.split("-")[-1]    # e.g. "newdata"
+    m2_tag = model2.split("-")[-1]    # e.g. "cncap"
+
+    lines = []
+
+    # ── 标题 ─────────────────────────────────────────────────────────────────
+    auto_title = title or f"模型对比Overall指标总结 ({model1} vs {model2} - 通用数据集评测)"
+    lines.append(f"# {auto_title}")
+    lines.append("")
+    lines.append(f"**对比模型**: {model1} vs {model2}  ")
+    lines.append(f"**评测日期**: {report_date}  ")
+    lines.append(f"**数据集**: 通用数据集 (Common Match Cases)  ")
+
+    total_common = stats.get("common", None)
+    m1_total = stats.get("model1_total", None)
+    m2_total = stats.get("model2_total", None)
+    if total_common is not None and m1_total is not None and m2_total is not None:
+        lines.append(f"**匹配样本**: {total_common:,} ({model1}: {m1_total:,} | {model2}: {m2_total:,})")
+    else:
+        lines.append(f"**匹配样本**: N/A")
+    lines.append("")
+
+    if background:
+        lines.append(f"> **背景说明**: {background}")
+    else:
+        lines.append(f"> **背景说明**: 本次评测对比了 {model1} 与 {model2}，评估两者在通用数据集上的2D/3D检测性能差异。")
+    lines.append("")
+    lines.append("---")
+    lines.append("")
+
+    # ── 2D Overall ────────────────────────────────────────────────────────────
+    ov = m2d["overall"]
+    lines.append("## 📊 2D检测指标 (Overall)")
+    lines.append("")
+    lines.append("### 总体性能对比")
+    lines.append("")
+    lines.append(f"| 指标 | {model1} | {model2} | 差异 | 相对变化 | 结果 |")
+    lines.append("|------|" + "---|" * 5)
+
+    def ov_row(key, label, higher_is_better=True):
+        v1 = ov[key][model1]
+        v2 = ov[key][model2]
+        diff = ov[key]["diff"]
+        rel = ov[key]["relative_change_%"]
+        j = judge(diff, rel, higher_is_better, abs_thr=METRIC_TIE_THRESHOLD,
+                  model1_name=model1, model2_name=model2)
+        return f"| **{label}** | {fmt(v1)} | {fmt(v2)} | {fmt_diff(diff)} | {fmt_pct(rel)} | {j} |"
+
+    lines.append(ov_row("precision", "PRECISION"))
+    lines.append(ov_row("recall", "RECALL"))
+    lines.append(ov_row("f1_score", "F1-Score"))
+    lines.append(ov_row("map", "mAP"))
+    lines.append("")
+
+    # 关键发现
+    prec_diff = ov["precision"]["relative_change_%"]
+    rec_diff = ov["recall"]["relative_change_%"]
+    map_diff = ov["map"]["relative_change_%"]
+    f1_diff = ov["f1_score"]["relative_change_%"]
+
+    ap_wins = summary.get("2d", {}).get("ap", {}).get("wins", "?")
+    ap_losses = summary.get("2d", {}).get("ap", {}).get("losses", "?")
+    ap_ties = summary.get("2d", {}).get("ap", {}).get("ties", "?")
+
+    lines.append("### 关键发现")
+    lines.append("")
+    lines.append(f"- 📊 **Precision**: {model2}{'领先' if prec_diff > 0 else '落后'}{fmt_pct(abs(prec_diff))}，{'误检率略低' if prec_diff > 0 else '误检率略高'}")
+    lines.append(f"- 📊 **Recall**: {model1 if rec_diff < 0 else model2}领先{fmt_pct(abs(rec_diff))}，检出率{'更高' if rec_diff < 0 else '更低'}")
+    lines.append(f"- 📊 **mAP**: {model2 if map_diff > 0 else model1}领先{fmt_pct(abs(map_diff))}（{'极小差异，基本持平' if abs(map_diff) < 2 else '有一定差距'}）")
+    lines.append(f"- 📊 **F1-Score**: {'两模型基本持平' if abs(f1_diff) < 1 else (model2 + '更优' if f1_diff > 0 else model1 + '更优')}（差距{fmt_pct(abs(f1_diff))}）")
+    lines.append(f"- ⚖️ **类别赢负统计 (AP)**: {m2_tag}赢{ap_wins}类, {m1_tag}赢{ap_losses}类, 平局{ap_ties}类")
+    lines.append("")
+
+    lines.append("---")
+    lines.append("")
+
+    # ── 2D Per-Class ──────────────────────────────────────────────────────────
+    pc = m2d.get("per_class", {})
+    lines.append("## 📋 2D检测指标 (Per Class)")
+    lines.append("")
+    lines.append("### 各类别性能对比")
+    lines.append("")
+    lines.append(f"| 类别 | Precision ({m1_tag}) | Precision ({m2_tag}) | Recall ({m1_tag}) | Recall ({m2_tag}) | F1 ({m1_tag}) | F1 ({m2_tag}) | AP ({m1_tag}) | AP ({m2_tag}) | AP差异 | 结果 |")
+    lines.append("|------|" + "---|" * 10)
+
+    adv_m2 = []   # model2 明显更好的类别
+    adv_m1 = []   # model1 明显更好的类别
+
+    for cls, cd in pc.items():
+        prec1 = cd["precision"][model1]
+        prec2 = cd["precision"][model2]
+        rec1  = cd["recall"][model1]
+        rec2  = cd["recall"][model2]
+        f1_1  = cd["f1_score"][model1]
+        f1_2  = cd["f1_score"][model2]
+        ap1   = cd["ap"][model1]
+        ap2   = cd["ap"][model2]
+        ap_d  = cd["ap"]["diff"]
+        ap_r  = cd["ap"]["relative_change_%"]
+        j = judge(ap_d, ap_r, True, abs_thr=AP_TIE_THRESHOLD,
+                  model1_name=model1, model2_name=model2)
+        lines.append(
+            f"| **{cls}** | {fmt(prec1)} | {fmt(prec2)} | {fmt(rec1)} | {fmt(rec2)} "
+            f"| {fmt(f1_1)} | {fmt(f1_2)} | {fmt(ap1)} | {fmt(ap2)} | {fmt_diff(ap_d)} | {j} |"
+        )
+        if abs(ap_r) >= 2.0:   # 相对变化>=2%才算显著
+            if ap_d > 0:
+                adv_m2.append((cls, ap1, ap2, ap_r))
+            elif ap_d < 0:
+                adv_m1.append((cls, ap1, ap2, ap_r))
+
+    lines.append("")
+    lines.append("### 类别分析")
+    lines.append("")
+
+    if adv_m2:
+        lines.append(f"**{model2} 优势类别** (AP更高):")
+        for cls, ap1, ap2, rel in sorted(adv_m2, key=lambda x: -x[3]):
+            mark = "**大幅领先**" if rel > 8 else "领先"
+            lines.append(f"- {cls}: {m2_tag} {fmt(ap2)} > {m1_tag} {fmt(ap1)}（{mark}{fmt_pct(rel)}）")
+        lines.append("")
+
+    if adv_m1:
+        lines.append(f"**{model1} 优势类别** (AP更高):")
+        for cls, ap1, ap2, rel in sorted(adv_m1, key=lambda x: x[3]):
+            mark = "**大幅领先**" if abs(rel) > 8 else "领先"
+            lines.append(f"- {cls}: {m1_tag} {fmt(ap1)} > {m2_tag} {fmt(ap2)}（{mark}{fmt_pct(abs(rel))}）")
+        lines.append("")
+
+    lines.append("---")
+    lines.append("")
+
+    # ── 3D Metrics ────────────────────────────────────────────────────────────
+    if m3d:
+        lines.append("## 🎯 3D检测指标")
+        lines.append("")
+
+        cls_labels = REPORT_3D_CLASS_LABELS
+
+        for cls_key, cls_label in cls_labels.items():
+            if cls_key not in m3d:
+                continue
+            cd = m3d[cls_key]
+            ov3 = cd.get("overall", {})
+            if not ov3:
+                continue
+            n = cd.get("common_samples")
+            n_str = f"{n:,} 个样本" if n is not None else "N/A 个样本"
+
+            lines.append(f"### {cls_label} - {n_str}")
+            lines.append("")
+            lines.append(f"| 指标 | {model1} | {model2} | 差异 | 相对变化 | 结果 |")
+            lines.append("|------|" + "---|" * 5)
+
+            def row3d(key, label, higher_is_better=False):
+                if key not in ov3:
+                    return None
+                v1 = ov3[key][model1]["mean"]
+                v2 = ov3[key][model2]["mean"]
+                diff = ov3[key]["diff"]
+                rel = ov3[key]["relative_change_%"]
+                j = judge(diff, rel, higher_is_better,
+                          rel_thr=ERROR_TIE_THRESHOLD_REL,
+                          model1_name=model1, model2_name=model2)
+                return f"| **{label}** | {fmt(v1)} | {fmt(v2)} | {fmt_diff(diff)} | {fmt_pct(rel)} | {j} |"
+
+            for row in [
+                row3d("lateral_error",               "Lateral Error"),
+                row3d("longitudinal_error",           "Longitudinal Error"),
+                row3d("longitudinal_relative_error",  "Longitudinal Relative Error"),
+                row3d("heading_error",                "Heading Error"),
+                row3d("heading_error_relaxed",        "Heading Error Relaxed"),
+            ]:
+                if row is not None:
+                    lines.append(row)
+
+            if "reversal_info" in ov3:
+                rev1 = ov3["reversal_info"][model1]
+                rev2 = ov3["reversal_info"][model2]
+                rev_j = "✅ " + (m2_tag if rev2["percentage"] < rev1["percentage"] else m1_tag) + "优"
+                if abs(rev1["percentage"] - rev2["percentage"]) < 0.5:
+                    rev_j = "⚖️ 持平"
+                lines.append(
+                    f"| **Reversal Cases** | {rev1['count']:,} ({rev1['percentage']:.2f}%) "
+                    f"| {rev2['count']:,} ({rev2['percentage']:.2f}%) | - | - | {rev_j} |"
+                )
+            lines.append("")
+
+            # ── 纵向区间对比 ──────────────────────────────────────────────
+            def _long_sort_key(k):
+                stripped = k[len("long_"):].replace("m", "")
+                m = re.search(r'(?<=\d)-', stripped)
+                if m:
+                    try:
+                        return float(stripped[:m.start()])
+                    except ValueError:
+                        pass
+                return float('inf')
+
+            long_keys = sorted(
+                [k for k in cd.keys() if k.startswith("long_")],
+                key=_long_sort_key
+            )
+            if long_keys:
+                lines.append(f"#### 纵向区间对比")
+                lines.append("")
+                lines.append(
+                    f"| 区间 | 样本数 "
+                    f"| Lat ({m1_tag}) | Lat ({m2_tag}) | Lat Δ% "
+                    f"| Long ({m1_tag}) | Long ({m2_tag}) | Long Δ% "
+                    f"| LongRel ({m1_tag}) | LongRel ({m2_tag}) | LongRel Δ% "
+                    f"| Head ({m1_tag}) | Head ({m2_tag}) | Head Δ% |"
+                )
+                lines.append("|------|" + "---|" * 13)
+                for rk in long_keys:
+                    rb = cd[rk]
+                    if not rb:
+                        continue
+
+                    def _rv(metric, model):
+                        d = rb.get(metric, {})
+                        if model in d:
+                            return fmt(d[model]["mean"])
+                        return "-"
+
+                    def _rd(metric):
+                        d = rb.get(metric, {})
+                        rel = d.get("relative_change_%")
+                        if rel is None:
+                            return "-"
+                        return fmt_pct(rel)
+
+                    # sample count from any available metric
+                    n_range = "-"
+                    for _mk in ("lateral_error", "longitudinal_error", "heading_error"):
+                        _md = rb.get(_mk, {})
+                        if model1 in _md and "samples" in _md[model1]:
+                            n_range = f"{_md[model1]['samples']:,}"
+                            break
+
+                    # range label: strip prefix and trailing 'm'
+                    rl = rk[len("long_"):]
+
+                    lines.append(
+                        f"| **{rl}** | {n_range} "
+                        f"| {_rv('lateral_error', model1)} | {_rv('lateral_error', model2)} | {_rd('lateral_error')} "
+                        f"| {_rv('longitudinal_error', model1)} | {_rv('longitudinal_error', model2)} | {_rd('longitudinal_error')} "
+                        f"| {_rv('longitudinal_relative_error', model1)} | {_rv('longitudinal_relative_error', model2)} | {_rd('longitudinal_relative_error')} "
+                        f"| {_rv('heading_error', model1)} | {_rv('heading_error', model2)} | {_rd('heading_error')} |"
+                    )
+                lines.append("")
+
+        lines.append("---")
+        lines.append("")
+
+    # ── Match Statistics ──────────────────────────────────────────────────────
+    if stats:
+        lines.append("## 📊 样本匹配统计")
+        lines.append("")
+        lines.append("### 整体匹配情况")
+        lines.append("")
+        lines.append("| 模型 | 总样本数 | 公共样本 | 独有样本 | 公共占比 |")
+        lines.append("|------|----------|----------|----------|---------|")
+        m1_pct = stats.get("common_percentage_of_model1", 0)
+        m2_pct = stats.get("common_percentage_of_model2", 0)
+        m1_uniq = stats.get("model1_unique", 0)
+        m2_uniq = stats.get("model2_unique", 0)
+        lines.append(f"| **{model1}** | {m1_total:,} | {total_common:,} | {m1_uniq:,} | {m1_pct:.2f}% |")
+        lines.append(f"| **{model2}** | {m2_total:,} | {total_common:,} | {m2_uniq:,} | {m2_pct:.2f}% |")
+        lines.append("")
+
+        per_cls_stats = stats.get("per_class", {})
+        if per_cls_stats:
+            lines.append("### 各类别匹配情况 (3D)")
+            lines.append("")
+            lines.append(f"| 类别 | {m1_tag}总数 | {m2_tag}总数 | 公共样本 | {m1_tag}占比 | {m2_tag}占比 |")
+            lines.append("|------|" + "---|" * 5)
+            for cls, cs in per_cls_stats.items():
+                lines.append(
+                    f"| **{cls}** | {cs['model1_total']:,} | {cs['model2_total']:,} "
+                    f"| {cs['common']:,} | {cs['common_percentage_of_model1']:.2f}% "
+                    f"| {cs['common_percentage_of_model2']:.2f}% |"
+                )
+            lines.append("")
+
+        lines.append("---")
+        lines.append("")
+
+    # ── Summary / Conclusions ─────────────────────────────────────────────────
+    lines.append("## 🎯 结论与建议")
+    lines.append("")
+    lines.append("### 2D检测汇总")
+    lines.append("")
+
+    sum2d = summary.get("2d", {})
+    ap_w = sum2d.get("ap", {}).get("wins", 0)
+    ap_l = sum2d.get("ap", {}).get("losses", 0)
+    ap_t = sum2d.get("ap", {}).get("ties", 0)
+    f1_w = sum2d.get("f1_score", {}).get("wins", 0)
+    f1_l = sum2d.get("f1_score", {}).get("losses", 0)
+    f1_t = sum2d.get("f1_score", {}).get("ties", 0)
+
+    lines.append(f"- **AP 类别统计**: {m2_tag}赢{ap_w}类 / {m1_tag}赢{ap_l}类 / 平局{ap_t}类")
+    lines.append(f"- **F1 类别统计**: {m2_tag}赢{f1_w}类 / {m1_tag}赢{f1_l}类 / 平局{f1_t}类")
+    lines.append(f"- **整体mAP**: {model1}={fmt(ov['map'][model1])} vs {model2}={fmt(ov['map'][model2])} ({fmt_pct(ov['map']['relative_change_%'])})")
+    lines.append("")
+
+    if m3d:
+        sum3d = summary.get("3d", {})
+        lat_w = sum3d.get("lateral", {}).get("wins", 0)
+        lat_l = sum3d.get("lateral", {}).get("losses", 0)
+        lon_w = sum3d.get("longitudinal", {}).get("wins", 0)
+        lon_l = sum3d.get("longitudinal", {}).get("losses", 0)
+        hd_w  = sum3d.get("heading", {}).get("wins", 0)
+        hd_l  = sum3d.get("heading", {}).get("losses", 0)
+
+        lines.append("### 3D检测汇总")
+        lines.append("")
+        lines.append(f"- **横向误差 (Lateral)**: {m2_tag}优{lat_w}类 / {m1_tag}优{lat_l}类")
+        lines.append(f"- **纵向误差 (Longitudinal)**: {m2_tag}优{lon_w}类 / {m1_tag}优{lon_l}类")
+        lines.append(f"- **航向误差 (Heading)**: {m2_tag}优{hd_w}类 / {m1_tag}优{hd_l}类")
+        lines.append("")
+
+    lines.append("### 综合建议")
+    lines.append("")
+    # 自动判断整体赢家
+    map_rel = ov["map"]["relative_change_%"]
+    if map_rel > 2:
+        overall_winner = model2
+    elif map_rel < -2:
+        overall_winner = model1
+    else:
+        overall_winner = None
+
+    if overall_winner:
+        lines.append(f"- 🏆 **综合mAP**: {overall_winner} 整体占优（{fmt_pct(abs(map_rel))}）")
+    else:
+        lines.append(f"- ⚖️ **综合mAP**: 两模型基本持平（差距{fmt_pct(abs(map_rel))}）")
+
+    adv_summary_m2 = [(c, r) for c, *_, r in adv_m2]
+    adv_summary_m1 = [(c, r) for c, *_, r in adv_m1]
+    if adv_summary_m2:
+        cls_str = "、".join(c for c, _ in adv_summary_m2)
+        lines.append(f"- ✅ **{model2} 改善**: {cls_str} 类别AP有所提升")
+    if adv_summary_m1:
+        cls_str = "、".join(c for c, _ in adv_summary_m1)
+        lines.append(f"- ⚠️ **{model2} 退化**: {cls_str} 类别AP有所下降")
+
+    lines.append("")
+
+    return "\n".join(lines)
+
+
+def main():
+    parser = argparse.ArgumentParser(
+        description="将 comparison_report.json 转换为中文 Markdown 评测报告"
+    )
+    parser.add_argument("json_path", help="comparison_report.json 的路径")
+    parser.add_argument("--output", "-o", default=None,
+                        help="输出 Markdown 文件路径（默认与 JSON 同目录，文件名 COMPARISON_REPORT.md）")
+    parser.add_argument("--title", default=None,
+                        help="自定义报告标题")
+    parser.add_argument("--background", default=None,
+                        help="背景说明文字")
+    parser.add_argument("--date", default=str(date.today()),
+                        help="评测日期 (默认今天，格式 YYYY-MM-DD)")
+    args = parser.parse_args()
+
+    json_path = Path(args.json_path)
+    if not json_path.exists():
+        print(f"错误: 文件不存在: {json_path}", file=sys.stderr)
+        sys.exit(1)
+
+    with open(json_path, "r", encoding="utf-8") as f:
+        data = json.load(f)
+
+    # 自动从 JSON 中读取模型名称
+    models = list(data["2d_metrics"]["overall"]["precision"].keys())
+    # 过滤掉 diff / relative_change_% 等非模型 key
+    skip = {"diff", "relative_change_%"}
+    models = [m for m in models if m not in skip]
+    if len(models) < 2:
+        print("错误: 无法从 JSON 中自动识别模型名称，请检查文件格式。", file=sys.stderr)
+        sys.exit(1)
+    model1, model2 = models[0], models[1]
+    print(f"模型1: {model1}")
+    print(f"模型2: {model2}")
+
+    report = build_report(data, model1, model2,
+                          report_date=args.date,
+                          title=args.title,
+                          background=args.background)
+
+    # 输出路径
+    if args.output:
+        out_path = Path(args.output)
+    else:
+        out_path = json_path.parent / "COMPARISON_REPORT.md"
+
+    out_path.write_text(report, encoding="utf-8")
+    print(f"报告已生成: {out_path}")
+
+
+if __name__ == "__main__":
+    main()

eval_tools/model_comparison/generate_eval_report.py

@@ -0,0 +1,207 @@
+#!/usr/bin/env python3
+"""
+自动将单模型 evaluation_report.json 转换为中文 Markdown 评测报告。
+
+用法:
+    python generate_eval_report.py <evaluation_report.json 路径>
+    python generate_eval_report.py <evaluation_report.json 路径> --output <输出路径>
+    python generate_eval_report.py <evaluation_report.json 路径> --model "模型名称" --date 2026-03-01
+
+示例:
+    python eval_tools/model_comparison/generate_eval_report.py \
+        evaluation_results/.../evaluation_report.json \
+        --model yolov5s-300w-newdata-cncap
+"""
+
+import json
+import argparse
+import sys
+from datetime import date
+from pathlib import Path
+
+# Allow importing class_config from the eval_tools root
+sys.path.insert(0, str(Path(__file__).parent.parent))
+from class_config import REPORT_3D_CLASS_LABELS
+
+
+def fmt(v: float, decimals: int = 4) -> str:
+    return f"{v:.{decimals}f}"
+
+
+def fmt2(v: float) -> str:
+    return f"{v:.2f}"
+
+
+def pct(v: float) -> str:
+    return f"{v * 100:.1f}%"
+
+
+def build_report(data: dict, model_name: str, report_date: str) -> str:
+    lines = []
+
+    # ── 标题 ─────────────────────────────────────────────────────────────────
+    lines.append(f"# 模型评测报告: {model_name}")
+    lines.append("")
+    lines.append(f"**模型**: {model_name}  ")
+    lines.append(f"**评测日期**: {report_date}  ")
+
+    eval_cfg = data.get("evaluation_config", {})
+    if eval_cfg:
+        lines.append(f"**置信度阈值 (P/R/F1)**: {eval_cfg.get('conf_threshold', '-')}  ")
+        lines.append(f"**IoU 阈值**: {eval_cfg.get('iou_threshold', '-')}  ")
+        lines.append(f"**AP 计算方法**: {eval_cfg.get('ap_method', '-')}  ")
+
+    lines.append("")
+    lines.append("---")
+    lines.append("")
+
+    # ── 2D Overall ────────────────────────────────────────────────────────────
+    ov2d = data["2d_evaluation"]["overall"]
+    lines.append("## 📊 2D检测指标 (Overall)")
+    lines.append("")
+    lines.append("| 指标 | 数值 |")
+    lines.append("|------|------|")
+    lines.append(f"| **Precision** | {fmt(ov2d['precision'])} |")
+    lines.append(f"| **Recall** | {fmt(ov2d['recall'])} |")
+    lines.append(f"| **F1-Score** | {fmt(ov2d['f1_score'])} |")
+    lines.append(f"| **mAP** | {fmt(ov2d['map'])} |")
+    lines.append(f"| **TP** | {ov2d['tp']:,} |")
+    lines.append(f"| **FP** | {ov2d['fp']:,} |")
+    lines.append(f"| **FN** | {ov2d['fn']:,} |")
+    lines.append("")
+
+    # ── 2D Per-Class ──────────────────────────────────────────────────────────
+    lines.append("## 📋 2D检测指标 (Per Class)")
+    lines.append("")
+    lines.append("| 类别 | Precision | Recall | F1 | AP | GT | TP | FP | FN |")
+    lines.append("|------|-----------|--------|----|----|-----|-----|-----|-----|")
+
+    pc2d = data["2d_evaluation"]["per_class"]
+    for cls, cd in pc2d.items():
+        lines.append(
+            f"| **{cls}** | {fmt(cd['precision'])} | {fmt(cd['recall'])} "
+            f"| {fmt(cd['f1_score'])} | {fmt(cd['ap'])} "
+            f"| {cd['num_gt']:,} | {cd['tp']:,} | {cd['fp']:,} | {cd['fn']:,} |"
+        )
+    lines.append("")
+    lines.append("---")
+    lines.append("")
+
+    # ── 3D Overall per class ──────────────────────────────────────────────────
+    m3d = data.get("3d_evaluation", {})
+    if not m3d:
+        return "\n".join(lines)
+
+    lines.append("## 🎯 3D检测指标")
+    lines.append("")
+
+    CLS_LABELS = REPORT_3D_CLASS_LABELS
+
+    LONG_RANGES = ["long_0-10m", "long_10-20m", "long_20-30m", "long_30-40m",
+                   "long_40-50m", "long_50-60m", "long_60-70m", "long_70-80m",
+                   "long_80-90m", "long_90-100m", "long_100-999m"]
+
+    for cls_key, cls_label in CLS_LABELS.items():
+        if cls_key not in m3d:
+            continue
+        cd = m3d[cls_key]
+        ov = cd["overall"]
+        n = ov["num_samples"]
+
+        lines.append(f"### {cls_label} ({n:,} 样本)")
+        lines.append("")
+
+        # Overall stats
+        lines.append("#### 总体指标")
+        lines.append("")
+        lines.append("| 指标 | Mean | Median | Std | P90 |")
+        lines.append("|------|------|--------|-----|-----|")
+
+        def stat_row(label, key):
+            s = ov[key]
+            return (f"| **{label}** | {fmt2(s['mean'])} | {fmt2(s['median'])} "
+                    f"| {fmt2(s['std'])} | {fmt2(s['percentile_90'])} |")
+
+        lines.append(stat_row("Lateral Error (m)",         "lateral_error"))
+        lines.append(stat_row("Longitudinal Error (m)",    "longitudinal_error"))
+        lines.append(stat_row("Long. Relative Error",      "longitudinal_relative_error"))
+        lines.append(stat_row("Heading Error (rad)",       "heading_error"))
+        lines.append(stat_row("Heading Error Relaxed",     "heading_error_relaxed"))
+        rev_pct = ov['reversal_percentage']
+        lines.append(f"| **Reversal** | {ov['reversal_count']:,} ({rev_pct:.2f}%) | - | - | - |")
+        lines.append("")
+
+        # Distance range breakdown
+        avail_ranges = [r for r in LONG_RANGES if r in cd]
+        if avail_ranges:
+            lines.append("#### 按距离分段 (纵向误差 Mean / Lateral Mean / Samples)")
+            lines.append("")
+            lines.append("| 距离段 | 样本数 | Lateral (m) | Longitudinal (m) | Long.Rel | Heading | Reversal% |")
+            lines.append("|--------|--------|-------------|------------------|----------|---------|-----------|")
+            for rng in avail_ranges:
+                r = cd[rng]
+                rov = r
+                rn = rov["num_samples"]
+                if rn == 0:
+                    continue
+                lat  = rov["lateral_error"]["mean"]
+                lon  = rov["longitudinal_error"]["mean"]
+                lrel = rov["longitudinal_relative_error"]["mean"]
+                hd   = rov["heading_error"]["mean"]
+                rev  = rov["reversal_percentage"]
+                lines.append(
+                    f"| {rng.replace('long_', '')} | {rn:,} | {fmt2(lat)} | {fmt2(lon)} "
+                    f"| {lrel:.3f} | {fmt2(hd)} | {rev:.1f}% |"
+                )
+            lines.append("")
+
+    lines.append("---")
+    lines.append("")
+
+    return "\n".join(lines)
+
+
+def main():
+    parser = argparse.ArgumentParser(
+        description="将单模型 evaluation_report.json 转换为中文 Markdown 评测报告"
+    )
+    parser.add_argument("json_path", help="evaluation_report.json 的路径")
+    parser.add_argument("--output", "-o", default=None,
+                        help="输出 Markdown 文件路径（默认与 JSON 同目录，文件名 EVALUATION_REPORT.md）")
+    parser.add_argument("--model", default=None,
+                        help="模型名称（默认从目录名推断）")
+    parser.add_argument("--date", default=str(date.today()),
+                        help="评测日期 (默认今天，格式 YYYY-MM-DD)")
+    args = parser.parse_args()
+
+    json_path = Path(args.json_path).resolve()
+    if not json_path.exists():
+        print(f"错误: 文件不存在: {json_path}", file=sys.stderr)
+        sys.exit(1)
+
+    with open(json_path, "r", encoding="utf-8") as f:
+        data = json.load(f)
+
+    # 从路径推断模型名称：取 json 所在目录的上级目录名
+    if args.model:
+        model_name = args.model
+    else:
+        # e.g. .../yolov5s-300w-newdata-cncap/20260228_102849/evaluation_report.json
+        model_name = json_path.parent.parent.name
+        if not model_name or model_name == ".":
+            model_name = json_path.parent.name
+    print(f"模型: {model_name}")
+
+    report = build_report(data, model_name, args.date)
+
+    if args.output:
+        out_path = Path(args.output)
+    else:
+        out_path = json_path.parent / "EVALUATION_REPORT.md"
+
+    out_path.write_text(report, encoding="utf-8")
+    print(f"报告已生成: {out_path}")
+
+
+if __name__ == "__main__":
+    main()