ADAS/HSAP
2
2
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: initial HSAP platform

Browse Source 
Huaxu Sentinel Active Safety Platform with embedded algorithm code,
Docker Compose setup, and vendored dataset scaffolds for clone-and-run.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Chengfang Lu
2026-05-25 16:59:59 +08:00
commit 7c43b44c57
1619 changed files with 373355 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

138
algorithms/lane_ufld/code/UFLD/README.md Normal file
View File

@@ -0,0 +1,138 @@
UFLD 使用说明（lane0_copy 数据）
代码目录：/home/chengfanglu/DATA/BK2/UFLD
数据目录：/home/chengfanglu/DATA/lane0_copy/
一、环境
source ~/miniconda3/etc/profile.d/conda.sh
conda activate lane_light
cd /home/chengfanglu/DATA/BK2/UFLD
二、数据目录
lane0_copy/
DATASET/（基线包，勿改 list/train_gt.txt）
images/
annotations/segmentation_masks/
list/train_gt.txt（训练：图 + mask 两列）
list/val_gt.txt
list/test_gt.txt
list/test.txt（仅图片）
DATASET-AddBy-zhangsan-20260615/（增量包，结构同上）
lists_merged/（多包合并列表，训练时自动生成）
datasets_registry.json（短名别名）
增量包命名：DATASET-AddBy-姓名-YYYYMMDD（日期 8 位，如 20260615）
目录规范详见：/home/chengfanglu/DATA/lane0_copy/DATASETS_LAYOUT.md
新建增量包命令：
python /home/chengfanglu/DATA/lane0_copy/scripts/build_ufld_pack.py --src /path/to/archive --parent /home/chengfanglu/DATA/lane0_copy --engineer zhangsan --date 20260615
别名（config 里可写 DATASET-A）：编辑 lane0_copy/datasets_registry.json，例如：
{"aliases": {"DATASET-A": "DATASET-AddBy-zhangsan-20260615"}}
三、配置文件
configs/mufld_lane_multi_pack.py — 推荐，多包训练，用 train_packs 控制合并
configs/mufld_lane_culane.py — 单包，data_root 指向 DATASET 目录本身
configs/mufld_lane_smoke.py — 冒烟（少量样本）
configs/tusimple_res18_4lane_v1.py — 对接旧权重 best.pth（griding_num=100）
多包训练请改 configs/mufld_lane_multi_pack.py：
data_root = '/home/chengfanglu/DATA/lane0_copy'
train_packs = ['DATASET']
（多包示例：train_packs = ['DATASET', 'DATASET-A']）
pack_list_name = 'list/train_gt.txt'
remerge_train_list = False（增删包后改为 True，强制重建 lists_merged）
训练时自动合并列表到：lane0_copy/lists_merged/train__DATASET__....txt
四、训练
conda activate lane_light
cd /home/chengfanglu/DATA/BK2/UFLD
冒烟：
UFLD_NUM_WORKERS=0 python train.py configs/mufld_lane_smoke.py
正式（多包）：
python train.py configs/mufld_lane_multi_pack.py
断点续训：
python train.py configs/mufld_lane_multi_pack.py --resume log/你的实验目录/best.pth
日志与权重在：log/时间_lr_.../best.pth
无 GPU 时可设 UFLD_NUM_WORKERS=0
常用 config 项：
batch_size = 16
learning_rate = 0.1
use_aux = True（False 与旧 best.pth 一致，更省显存）
griding_num = 200（旧权重用 100）
num_lanes = 4
五、推理与测试
【5.1 可视化 demo】
先准备 test3.txt（示例取 3 张）：
awk '{print $1}' /home/chengfanglu/DATA/lane0_copy/DATASET/list/test_gt.txt | head -3 > /home/chengfanglu/DATA/lane0_copy/DATASET/test3.txt
python demo.py configs/tusimple_res18_4lane_v1.py --test_model log/20250702_165153_lr_1e-05_b_32_ufld_2lanes_res18/best.pth --data_root /home/chengfanglu/DATA/lane0_copy/DATASET
【5.2 批量测试】
python test.py configs/tusimple_res18_4lane_v1.py --test_model log/20250702_165153_lr_1e-05_b_32_ufld_2lanes_res18/best.pth --data_root /home/chengfanglu/DATA/lane0_copy/DATASET --test_list list/test_gt.txt
多包时 data_root 用 lane0_copy，例如：
python test.py configs/mufld_lane_multi_pack.py --test_model log/xxx/best.pth --data_root /home/chengfanglu/DATA/lane0_copy --test_list lists_merged/train__DATASET.txt
无 test_label.json 时只出预测，不算 TuSimple 官方指标。
【5.3 预测画到图上】
python vis_tusimple_pred.py --pred tmp/tusimple_eval_tmp.0.txt --data_root /home/chengfanglu/DATA/lane0_copy/DATASET --out_dir tmp/vis_pred
六、导出 ONNX
python pth_to_onnx.py --model_path log/20250702_165153_lr_1e-05_b_32_ufld_2lanes_res18/best.pth --output log/20250702_165153_lr_1e-05_b_32_ufld_2lanes_res18/best.onnx
需与训练时 backbone、griding_num、num_lanes 一致。
【6.1 VoVNet backbone】
已从 `BK2/archive/vovnet-detectron2-master` 移植 OSA+eSE 结构（无 detectron2 依赖），与 ResNet 相同接口。
| config `backbone` | 说明 |
|-------------------|------|
| `vov19slim` | V-19-slim-eSE，约 52.7M 参数（288×800） |
| `vov19slim_dw` | slim + depthwise |
| `vov19` / `vov39` / `vov57` / `vov99` | 更大变体 |
示例配置：`configs/tusimple_vov19slim_4lane_v1.py`
```bash
python train.py configs/tusimple_vov19slim_4lane_v1.py
python profile_model.py --backbone vov19slim --griding_num 100 --num_lanes 4
```
VoVNet **无** torchvision 预训练权重，需从头训或自行转换 detectron2 权重。旧 ResNet 的 `best.pth` **不能**直接用于 VoVNet。
七、路径速查
代码：/home/chengfanglu/DATA/BK2/UFLD
数据父目录：/home/chengfanglu/DATA/lane0_copy
基线数据：/home/chengfanglu/DATA/lane0_copy/DATASET
多包配置：configs/mufld_lane_multi_pack.py
已有权重：log/20250702_165153_lr_1e-05_b_32_ufld_2lanes_res18/best.pth

46
algorithms/lane_ufld/code/UFLD/TRAIN_ENV_CPU.md Normal file
View File

@@ -0,0 +1,46 @@
## lane_light + CPU PyTorch（UFLD 训练）
### 1. 激活环境
```bash
source /home/chengfanglu/miniconda3/etc/profile.d/conda.sh
conda activate lane_light
```
### 2. 已安装依赖（`lane_light`）
- `torch` / `torchvision`（**CPU 轮子**，来自 `https://download.pytorch.org/whl/cpu`）
- `opencv-python`, `tqdm`, `tensorboard`, `addict`, `scikit-learn`, `pathspec`（与 `requirements.txt` 对齐；`sklearn` 包名在 pip 中为 `scikit-learn`）
自检：
```bash
python -c "import torch; print('torch', torch.__version__, 'cuda=', torch.cuda.is_available())"
```
### 3. 数据与配置
- 默认数据根仍指向 `lane0_reorganized/lane_training_pack`（见 `configs/mufld_lane_culane.py`）。
- **CPU 建议**使用 `configs/mufld_lane_culane_cpu.py`（`batch_size=4`，学习率与 warmup 已按 batch 相对 16 做了粗略缩放）。内存不够可改配置或命令行覆盖：
```bash
cd /home/chengfanglu/DATA/BK2/UFLD
python train.py configs/mufld_lane_culane_cpu.py --batch_size 2
```
### 4. 运行训练
```bash
cd /home/chengfanglu/DATA/BK2/UFLD
python train.py configs/mufld_lane_culane_cpu.py
```
说明：
- `train.py` 已改为在 **无 CUDA** 时使用 `cpu`；原仓库中写死的 `CUDA_VISIBLE_DEVICES=1,2` 与 `.cuda()` 已去掉，避免 CPU 机直接报错。
- 首次 `pretrained=True` 会下载 ResNet 骨干权重，需联网。
- CPU 训练很慢，建议先用小 `epoch` / 小 `batch_size` 做通路测试。
### 5. 可选：DataLoader `num_workers`
当前 `data/dataloader.py` 里 `num_workers=8`。若 CPU 内存紧张或不想多进程读盘，可自行把该值改小（例如 `0` 或 `2`）。

45
algorithms/lane_ufld/code/UFLD/configs/culane.py Executable file
View File

@@ -0,0 +1,45 @@
# DATA
dataset = 'CULane'
data_root = 'C:\\data\\Tusimple\\test_set'
# TRAIN
epoch = 50
batch_size = 32
optimizer = 'SGD' #['SGD','Adam']
learning_rate = 0.1
weight_decay = 1e-4
momentum = 0.9
scheduler = 'multi' #['multi', 'cos']
steps = [25,38]
gamma = 0.1
warmup = 'linear'
warmup_iters = 695
# NETWORK
use_aux = True
griding_num = 200
backbone = '18'
# LOSS
sim_loss_w = 0.0
shp_loss_w = 0.0
# EXP
note = ''
log_path = None
# FINETUNE or RESUME MODEL PATH
finetune = None
resume = None
# TEST
test_model = './model/culane_18.pth'
test_work_dir = './tmp'
num_lanes = 4

41
algorithms/lane_ufld/code/UFLD/configs/mufld_lane_culane.py Normal file
View File

@@ -0,0 +1,41 @@
# MUFLD lane pack in CULane-style layout for UFLD training.
# Data root layout:
# <data_root>/images/...
# <data_root>/annotations/segmentation_masks/...
# <data_root>/list/train_gt.txt (two columns: training split only)
# <data_root>/list/val_gt.txt (validation pairs, optional custom loop)
# <data_root>/list/test.txt (held-out test images, one per line)
dataset = 'CULane'
data_root = '/home/chengfanglu/DATA/lane0_copy/DATASET'
epoch = 50
batch_size = 16
optimizer = 'SGD'
learning_rate = 0.1
weight_decay = 1e-4
momentum = 0.9
scheduler = 'multi'
steps = [25, 38]
gamma = 0.1
warmup = 'linear'
warmup_iters = 695
use_aux = True
griding_num = 200
backbone = '18'
sim_loss_w = 0.0
shp_loss_w = 0.0
note = 'lane_training_pack_v1'
log_path = './log'
finetune = None
resume = None
test_model = './model/culane_18.pth'
test_work_dir = './tmp'
num_lanes = 4

40
algorithms/lane_ufld/code/UFLD/configs/mufld_lane_culane_cpu.py Normal file
View File

@@ -0,0 +1,40 @@
# CPU 单机训练示例：batch 需显著减小；学习率可按 batch 相对 16 做线性缩放（可选）。
#
# layout 同 configs/mufld_lane_culane.py；
# lane_light 环境与安装说明见 TRAIN_ENV_CPU.md
dataset = "CULane"
data_root = "/home/chengfanglu/DATA/lane0_copy/DATASET"
epoch = 50
batch_size = 4
optimizer = "SGD"
# 若在 CPU 上不收敛可先试更小 lr，例如 batch=4 时约 0.1 * (4 / 16) = 0.025
learning_rate = 0.025
weight_decay = 1e-4
momentum = 0.9
scheduler = "multi"
steps = [25, 38]
gamma = 0.1
warmup = "linear"
# warmup 与原配置按 batch 比例对齐（原为 695 @ bs=16）
warmup_iters = 174
use_aux = True
griding_num = 200
backbone = "18"
sim_loss_w = 0.0
shp_loss_w = 0.0
note = "lane_training_pack_cpu_bs4"
log_path = None
finetune = None
resume = None
test_model = "./model/culane_18.pth"
test_work_dir = "./tmp"
num_lanes = 4

54
algorithms/lane_ufld/code/UFLD/configs/mufld_lane_multi_pack.py Normal file
View File

@@ -0,0 +1,54 @@
# Multi-pack training — control merged packs in this config.
# data_root = parent of DATASET / DATASET-AddBy-* / DATASET-A (alias).
from pathlib import Path
dataset = 'CULane'
data_root = str(Path(__file__).resolve().parents[5] / "datasets" / "lane")
# Pack names: directory under data_root, or alias from datasets_registry.json
train_packs = [
'lane_v1',
]
# Default list inside each pack (relative to pack root)
pack_list_name = 'list/train_gt.txt'
# Cached merged list (auto filename from pack names if merged_train_list is None)
merged_list_dir = 'lists_merged'
merged_train_list = None # e.g. 'lists_merged/train_all_v2.txt'
remerge_train_list = False # True to rebuild merged list every run
# Single-pack fallback (ignored when train_packs is set)
train_list = 'list/train_gt.txt'
epoch = 50
batch_size = 16
optimizer = 'SGD'
learning_rate = 0.1
weight_decay = 1e-4
momentum = 0.9
scheduler = 'multi'
steps = [25, 38]
gamma = 0.1
warmup = 'linear'
warmup_iters = 695
use_aux = True
griding_num = 200
backbone = '18'
sim_loss_w = 0.0
shp_loss_w = 0.0
note = 'multi_pack_v2'
log_path = './log'
finetune = None
resume = None
test_model = './model/culane_18.pth'
test_work_dir = './tmp'
num_lanes = 4

35
algorithms/lane_ufld/code/UFLD/configs/mufld_lane_smoke.py Normal file
View File

@@ -0,0 +1,35 @@
# Smoke test: few samples, 1 epoch, small batch (CPU or GPU).
dataset = "CULane"
data_root = "/home/chengfanglu/DATA/lane0_copy/DATASET"
train_list = "list/train_gt_smoke.txt"
epoch = 1
batch_size = 2
optimizer = "SGD"
learning_rate = 0.025
weight_decay = 1e-4
momentum = 0.9
scheduler = "multi"
steps = [1]
gamma = 0.1
warmup = "linear"
warmup_iters = 10
use_aux = True
griding_num = 200
backbone = "18"
sim_loss_w = 0.0
shp_loss_w = 0.0
note = "dataset_smoke_test"
log_path = "./log"
finetune = None
resume = None
test_model = "./model/culane_18.pth"
test_work_dir = "./tmp"
num_lanes = 4

41
algorithms/lane_ufld/code/UFLD/configs/tusimple_res18.py Executable file
View File

@@ -0,0 +1,41 @@
# DATA
dataset = 'Tusimple'
data_root = '/mnt/HDisk2T/liuxy51/ganxian/data_luojk'
# TRAIN
epoch = 500 # 10
batch_size = 32 # 4
optimizer = 'Adam' #['SGD','Adam']
learning_rate = 1e-5
weight_decay = 1e-4
momentum = 0.9
scheduler = 'cos' #['multi', 'cos']
# steps = [50,75]
gamma = 0.1
warmup = 'linear'
warmup_iters = 100
# NETWORK
backbone = '18'
griding_num = 100
use_aux = False
# LOSS
sim_loss_w = 1.0
shp_loss_w = 0.0
# EXP
note = '_ufld_2lanes_res18'
log_path = './log'
# FINETUNE or RESUME MODEL PATH
finetune = None
resume = None
# TESTNone
test_model = './model/lane_m599_all.pth'
test_work_dir = './tmp'
num_lanes = 2

41
algorithms/lane_ufld/code/UFLD/configs/tusimple_res18_4lane_v1.py Executable file
View File

@@ -0,0 +1,41 @@
# DATA
dataset = 'Tusimple'
data_root = '/mnt/HDisk2T/liuxy51/ganxian/train_2025_03_13_mufld' # 针对clrnet数据集8.1w帧多车道数据
# TRAIN
epoch = 500 # 10
batch_size = 32 # 4
optimizer = 'Adam' #['SGD','Adam']
learning_rate = 1e-5
weight_decay = 1e-4
momentum = 0.9
scheduler = 'cos' #['multi', 'cos']
# steps = [50,75]
gamma = 0.1
warmup = 'linear'
warmup_iters = 100
# NETWORK
backbone = '18'
griding_num = 100
use_aux = False
# LOSS
sim_loss_w = 1.0
shp_loss_w = 0.0
# EXP
note = '_ufld_2lanes_res18'
log_path = './log'
# FINETUNE or RESUME MODEL PATH
finetune = None
resume = None
# TESTNone
test_model = './model/lane_m599_all.pth'
test_work_dir = './tmp'
num_lanes = 4

41
algorithms/lane_ufld/code/UFLD/configs/tusimple_res18_nbg.py Executable file
View File

@@ -0,0 +1,41 @@
# DATA
dataset = 'Tusimple'
data_root = '/mnt/HDisk2T/liuxy51/ganxian/train_2024_03_06'
# TRAIN
epoch = 500 # 10
batch_size = 32 # 4
optimizer = 'Adam' #['SGD','Adam']
learning_rate = 1e-5
weight_decay = 1e-4
momentum = 0.9
scheduler = 'cos' #['multi', 'cos']
# steps = [50,75]
gamma = 0.1
warmup = 'linear'
warmup_iters = 100
# NETWORK
backbone = '18'
griding_num = 100
use_aux = False
# LOSS
sim_loss_w = 1.0
shp_loss_w = 0.0
# EXP
note = '_ufld_2lanes_res18'
log_path = './log'
# FINETUNE or RESUME MODEL PATH
finetune = None
resume = '/mnt/HDisk2T/liuxy51/ganxian/UFLD/log/20240607_162111_lr_1e-05_b_32_ufld_2lanes_res18/ep304.pth' # None
# TESTNone
test_model = './model/lane_m599_all.pth'
test_work_dir = './tmp'
num_lanes = 2

41
algorithms/lane_ufld/code/UFLD/configs/tusimple_res18_nbg2.py Executable file
View File

@@ -0,0 +1,41 @@
# DATA
dataset = 'Tusimple'
data_root = '/mnt/HDisk2T/liuxy51/ganxian/train_2024_03_06_1'
# TRAIN
epoch = 500 # 10
batch_size = 32 # 4
optimizer = 'Adam' #['SGD','Adam']
learning_rate = 1e-5
weight_decay = 1e-4
momentum = 0.9
scheduler = 'cos' #['multi', 'cos']
# steps = [50,75]
gamma = 0.1
warmup = 'linear'
warmup_iters = 100
# NETWORK
backbone = '18'
griding_num = 100
use_aux = False
# LOSS
sim_loss_w = 1.0
shp_loss_w = 0.0
# EXP
note = '_ufld_2lanes_res18'
log_path = './log'
# FINETUNE or RESUME MODEL PATH
finetune = None
resume = None
# TESTNone
test_model = './model/lane_m599_all.pth'
test_work_dir = './tmp'
num_lanes = 2

41
algorithms/lane_ufld/code/UFLD/configs/tusimple_res18_nbg3.py Executable file
View File

@@ -0,0 +1,41 @@
# DATA
dataset = 'Tusimple'
data_root = '/mnt/HDisk2T/liuxy51/ganxian/train_2024_03_06_2'
# TRAIN
epoch = 500 # 10
batch_size = 32 # 4
optimizer = 'Adam' #['SGD','Adam']
learning_rate = 1e-5
weight_decay = 1e-4
momentum = 0.9
scheduler = 'cos' #['multi', 'cos']
# steps = [50,75]
gamma = 0.1
warmup = 'linear'
warmup_iters = 100
# NETWORK
backbone = '18'
griding_num = 100
use_aux = False
# LOSS
sim_loss_w = 1.0
shp_loss_w = 0.0
# EXP
note = '_ufld_2lanes_res18'
log_path = './log'
# FINETUNE or RESUME MODEL PATH
finetune = None
resume = None
# TESTNone
test_model = './model/lane_m599_all.pth'
test_work_dir = './tmp'
num_lanes = 2

44
algorithms/lane_ufld/code/UFLD/configs/tusimple_res34.py Executable file
View File

@@ -0,0 +1,44 @@
# DATA
dataset = 'Tusimple'
# data_root = 'C:\\data\\Tusimple\\test_set'
# data_root = 'C:\\data\\anno\\324lane'
# data_root = 'C:\\data\\Tusimple\\train_set'
data_root = '/data/panh28/yk_syj/data/train_0306'
# TRAIN
epoch = 600
batch_size = 64
optimizer = 'Adam' #['SGD','Adam']
# learning_rate = 0.1
learning_rate = 1e-5
weight_decay = 1e-4
momentum = 0.9
scheduler = 'cos' #['multi', 'cos']
# steps = [50,75]
gamma = 0.1
warmup = 'linear'
warmup_iters = 100
# NETWORK
backbone = '34'
griding_num = 100
use_aux = False
# LOSS
sim_loss_w = 1.0
shp_loss_w = 0.0
# EXP
note = 'lane_res34_2ch_syj_0906_minilearn'
log_path = './log'
# FINETUNE or RESUME MODEL PATH
finetune = None
resume = "/data/panh28/yk_syj/code/UFLD/log/20230906_161808_lr_1e-04_b_64lane_res34_2ch_syj_0906/ep068.pth"
# TESTNone
test_model = './model/lane_m599_all.pth'
# test_model = './model/tusimple_18.pth'
test_work_dir = './tmp'
num_lanes = 2

6
algorithms/lane_ufld/code/UFLD/configs/tusimple_vov19slim_4lane_v1.py Normal file
View File

@@ -0,0 +1,6 @@
# UFLD + VoVNet-19-slim-eSE backbone (train from scratch; no torchvision weights).
from configs.tusimple_res18_4lane_v1 import *
backbone = 'vov19slim'
note = '_ufld_4lanes_vov19slim'

45
algorithms/lane_ufld/code/UFLD/curve_fit.py Executable file
View File

@@ -0,0 +1,45 @@
import numpy as np
import matplotlib.pyplot as plt
from scipy.optimize import curve_fit
# 自定义函数 e指数形式
def func(x, a, b, c):
return a * np.sqrt(x) * (b * np.square(x) + c)
# 定义x、y散点坐标
x = [20, 30, 40, 50, 60, 70]
x = np.array(x)
num = [453, 482, 503, 508, 498, 479]
y = np.array(num)
def get_curve_fit(x, y):
# 非线性最小二乘法拟合
popt, pcov = curve_fit(func, x, y)
# 获取popt里面是拟合系数
# print(popt)
a = popt[0]
b = popt[1]
c = popt[2]
yvals = func(x, a, b, c) # 拟合y值
# print('popt:', popt)
# print('系数a:', a)
# print('系数b:', b)
# print('系数c:', c)
# print('系数pcov:', pcov)
# print('系数yvals:', yvals)
return yvals
yvals = get_curve_fit(x, y)
print(yvals)
# 绘图
plot1 = plt.plot(x, y, 's', label='original values')
plot2 = plt.plot(x, yvals, 'r', label='polyfit values')
plt.xlabel('x')
plt.ylabel('y')
plt.legend(loc=4) # 指定legend的位置右下角
plt.title('curve_fit')
plt.show()

11
algorithms/lane_ufld/code/UFLD/data/constant.py Executable file
View File

@@ -0,0 +1,11 @@
# row anchors are a series of pre-defined coordinates in image height to detect lanes
# the row anchors are defined according to the evaluation protocol of CULane and Tusimple
# since our method will resize the image to 288x800 for training, the row anchors are defined with the height of 288
# you can modify these row anchors according to your training image resolution
tusimple_row_anchor = [ 64, 68, 72, 76, 80, 84, 88, 92, 96, 100, 104, 108, 112,
116, 120, 124, 128, 132, 136, 140, 144, 148, 152, 156, 160, 164,
168, 172, 176, 180, 184, 188, 192, 196, 200, 204, 208, 212, 216,
220, 224, 228, 232, 236, 240, 244, 248, 252, 256, 260, 264, 268,
272, 276, 280, 284]
culane_row_anchor = [121, 131, 141, 150, 160, 170, 180, 189, 199, 209, 219, 228, 238, 248, 258, 267, 277, 287]

118
algorithms/lane_ufld/code/UFLD/data/dataloader.py Executable file
View File

@@ -0,0 +1,118 @@
import torch, os
import numpy as np
import torchvision.transforms as transforms
import data.mytransforms as mytransforms
from data.constant import tusimple_row_anchor, culane_row_anchor
from data.dataset import LaneClsDataset, LaneTestDataset
def get_train_loader(batch_size, data_root, griding_num, dataset, use_aux, distributed, num_lanes,
train_list='list/train_gt.txt', num_workers=8):
target_transform = transforms.Compose([
mytransforms.FreeScaleMask((288, 800)),
mytransforms.MaskToTensor(),
])
segment_transform = transforms.Compose([
mytransforms.FreeScaleMask((36, 100)),
mytransforms.MaskToTensor(),
])
img_transform = transforms.Compose([
transforms.Resize((288, 800)),
transforms.ToTensor(),
# transforms.Normalize((0.723, 0.704, 0.726), (0.191, 0.178, 0.186)),
transforms.Normalize((0.485, 0.456, 0.406), (0.229, 0.224, 0.225)),
])
simu_transform = mytransforms.Compose2([
mytransforms.RandomRotate(6),
mytransforms.RandomUDoffsetLABEL(100),
mytransforms.RandomLROffsetLABEL(200)
])
if dataset == 'CULane':
train_dataset = LaneClsDataset(data_root,
os.path.join(data_root, train_list),
img_transform=img_transform, target_transform=target_transform,
simu_transform =simu_transform,
segment_transform=segment_transform,
row_anchor=culane_row_anchor,
griding_num=griding_num, use_aux=use_aux, num_lanes=num_lanes)
cls_num_per_lane = 18
elif dataset == 'Tusimple':
train_dataset = LaneClsDataset(data_root,
os.path.join(data_root, 'train_val_gt.txt'),
img_transform=img_transform, target_transform=target_transform,
simu_transform =simu_transform,
# simu_transform=None,
griding_num=griding_num,
row_anchor =tusimple_row_anchor,
segment_transform=segment_transform, use_aux=use_aux, num_lanes=num_lanes)
cls_num_per_lane = 56
else:
raise NotImplementedError
if distributed:
sampler = torch.utils.data.distributed.DistributedSampler(train_dataset)
else:
sampler = torch.utils.data.RandomSampler(train_dataset)
train_loader = torch.utils.data.DataLoader(
train_dataset, batch_size=batch_size, sampler=sampler, num_workers=num_workers,
)
return train_loader, cls_num_per_lane
def get_test_loader(batch_size, data_root, dataset, distributed, test_list=None):
img_transforms = transforms.Compose([
transforms.Resize((288, 800)),
transforms.ToTensor(),
transforms.Normalize((0.485, 0.456, 0.406), (0.229, 0.224, 0.225)),
])
if dataset == 'CULane':
if test_list is None:
test_list = 'list/test.txt'
test_dataset = LaneTestDataset(data_root, os.path.join(data_root, test_list), img_transform=img_transforms)
cls_num_per_lane = 18
elif dataset == 'Tusimple':
if test_list is None:
test_list = 'list/test_gt.txt'
test_dataset = LaneTestDataset(data_root, os.path.join(data_root, test_list), img_transform=img_transforms)
cls_num_per_lane = 56
if distributed:
sampler = SeqDistributedSampler(test_dataset, shuffle=False)
else:
sampler = torch.utils.data.SequentialSampler(test_dataset)
loader = torch.utils.data.DataLoader(test_dataset, batch_size=batch_size, sampler=sampler, num_workers=8)
return loader
class SeqDistributedSampler(torch.utils.data.distributed.DistributedSampler):
'''
Change the behavior of DistributedSampler to sequential distributed sampling.
The sequential sampling helps the stability of multi-thread testing, which needs multi-thread file io.
Without sequentially sampling, the file io on thread may interfere other threads.
'''
def __init__(self, dataset, num_replicas=None, rank=None, shuffle=False):
super().__init__(dataset, num_replicas, rank, shuffle)
def __iter__(self):
g = torch.Generator()
g.manual_seed(self.epoch)
if self.shuffle:
indices = torch.randperm(len(self.dataset), generator=g).tolist()
else:
indices = list(range(len(self.dataset)))
# add extra samples to make it evenly divisible
indices += indices[:(self.total_size - len(indices))]
assert len(indices) == self.total_size
num_per_rank = int(self.total_size // self.num_replicas)
# sequential sampling
indices = indices[num_per_rank * self.rank : num_per_rank * (self.rank + 1)]
assert len(indices) == self.num_samples
return iter(indices)

179
algorithms/lane_ufld/code/UFLD/data/dataset.py Executable file
View File

@@ -0,0 +1,179 @@
import torch
from PIL import Image
import os
import pdb
import numpy as np
import cv2
from data.mytransforms import find_start_pos
def loader_func(path):
return Image.open(path)
class LaneTestDataset(torch.utils.data.Dataset):
def __init__(self, path, list_path, img_transform=None):
super(LaneTestDataset, self).__init__()
self.path = path
self.img_transform = img_transform
with open(list_path, 'r') as f:
self.list = f.readlines()
self.list = [l[1:] if l[0] == '/' else l for l in self.list] # exclude the incorrect path prefix '/' of CULane
def __getitem__(self, index):
name = self.list[index].split()[0]
img_path = os.path.join(self.path, name)
img = loader_func(img_path).convert('RGB')
if self.img_transform is not None:
img = self.img_transform(img)
return img, name
def __len__(self):
return len(self.list)
class LaneClsDataset(torch.utils.data.Dataset):
def __init__(self, path, list_path, img_transform=None, target_transform=None, simu_transform=None, griding_num=50, load_name=False,
row_anchor=None, use_aux=False, segment_transform=None, num_lanes=2):
super(LaneClsDataset, self).__init__()
self.img_transform = img_transform
self.target_transform = target_transform
self.segment_transform = segment_transform
self.simu_transform = simu_transform
self.path = path
self.griding_num = griding_num
self.load_name = load_name
self.use_aux = use_aux
self.num_lanes = num_lanes
with open(list_path, 'r') as f:
self.list = f.readlines()
self.row_anchor = row_anchor
self.row_anchor.sort()
def _normalize_seg_label(self, label):
"""MUFLD masks use 0,2,3,4,5; aux CE needs contiguous 0..num_lanes."""
arr = np.array(label, dtype=np.uint8)
if arr.max() <= self.num_lanes:
return label
out = np.zeros_like(arr, dtype=np.uint8)
for lane_idx in range(1, self.num_lanes + 1):
out[arr == (lane_idx + 1)] = lane_idx
return Image.fromarray(out, mode='L')
def __getitem__(self, index):
l = self.list[index]
l_info = l.split()
img_name, label_name = l_info[0], l_info[1]
# print(img_name, label_name)
if img_name[0] == '/':
img_name = img_name[1:]
label_name = label_name[1:]
label_path = os.path.join(self.path, label_name)
label = loader_func(label_path).convert('L')
img_path = os.path.join(self.path, img_name)
img = loader_func(img_path).convert('RGB')
# print('---------------', img_path, label_path)
if self.simu_transform is not None:
img, label = self.simu_transform(img, label)
# print(',,,,,,,,,,,,,,,,', img.size, label.size)
lane_pts = self._get_index(label)
# get the coordinates of lanes at row anchors
w, h = img.size
cls_label = self._grid_pts(lane_pts, self.griding_num, w)
# make the coordinates to classification label
if self.use_aux:
assert self.segment_transform is not None
seg_label = self.segment_transform(self._normalize_seg_label(label))
if self.img_transform is not None:
img = self.img_transform(img)
if self.use_aux:
return img, cls_label, seg_label
if self.load_name:
return img, cls_label, img_name
return img, cls_label
def __len__(self):
return len(self.list)
def _grid_pts(self, pts, num_cols, w):
# pts : numlane,n,2
num_lane, n, n2 = pts.shape
col_sample = np.linspace(0, w - 1, num_cols)
assert n2 == 2
to_pts = np.zeros((n, num_lane))
for i in range(num_lane):
pti = pts[i, :, 1]
to_pts[:, i] = np.asarray(
[int(pt // (col_sample[1] - col_sample[0])) if pt != -1 else num_cols for pt in pti])
return to_pts.astype(int)
def _get_index(self, label):
w, h = label.size
if h != 288:
scale_f = lambda x : int((x * 1.0/288) * h)
sample_tmp = list(map(scale_f,self.row_anchor))
all_idx = np.zeros((self.num_lanes,len(sample_tmp),2))
for i,r in enumerate(sample_tmp):
label_r = np.asarray(label)[int(round(r))]
for lane_idx in range(1, self.num_lanes + 1):
if np.max(label_r) == 2:
pos = np.where(label_r == lane_idx)[0]
else:
pos = np.where(label_r == (lane_idx + 1))[0]
if len(pos) == 0:
all_idx[lane_idx - 1, i, 0] = r
all_idx[lane_idx - 1, i, 1] = -1
continue
pos = np.mean(pos)
all_idx[lane_idx - 1, i, 0] = r
all_idx[lane_idx - 1, i, 1] = pos
# data augmentation: extend the lane to the boundary of image
all_idx_cp = all_idx.copy()
for i in range(self.num_lanes):
if np.all(all_idx_cp[i,:,1] == -1):
continue
# if there is no lane
valid = all_idx_cp[i,:,1] != -1
# get all valid lane points' index
valid_idx = all_idx_cp[i,valid,:]
# get all valid lane points
if valid_idx[-1,0] == all_idx_cp[0,-1,0]:
# if the last valid lane point's y-coordinate is already the last y-coordinate of all rows
# this means this lane has reached the bottom boundary of the image
# so we skip
continue
if len(valid_idx) < 6:
continue
# if the lane is too short to extend
valid_idx_half = valid_idx[len(valid_idx) // 2:,:]
p = np.polyfit(valid_idx_half[:,0], valid_idx_half[:,1],deg = 1)
start_line = valid_idx_half[-1,0]
pos = find_start_pos(all_idx_cp[i,:,0],start_line) + 1
fitted = np.polyval(p, all_idx_cp[i, pos:, 0])
fitted = np.array([-1 if y < 0 or y > w-1 else y for y in fitted])
assert np.all(all_idx_cp[i,pos:,1] == -1)
all_idx_cp[i,pos:,1] = fitted
if -1 in all_idx[:, :, 0]:
pdb.set_trace()
return all_idx_cp

187
algorithms/lane_ufld/code/UFLD/data/mytransforms.py Executable file
View File

@@ -0,0 +1,187 @@
import numbers
import random
import numpy as np
from PIL import Image, ImageOps, ImageFilter
#from config import cfg
import torch
import pdb
import cv2
# ===============================img tranforms============================
class Compose2(object):
def __init__(self, transforms):
self.transforms = transforms
def __call__(self, img, mask, bbx=None):
if bbx is None:
for t in self.transforms:
# print(t)
# print('\\: ', img.size, mask.size)
img, mask = t(img, mask)
# print('//: ', img.size, mask.size)
return img, mask
for t in self.transforms:
img, mask, bbx = t(img, mask, bbx)
return img, mask, bbx
class FreeScale(object):
def __init__(self, size):
self.size = size # (h, w)
def __call__(self, img, mask):
return img.resize((self.size[1], self.size[0]), Image.BILINEAR), mask.resize((self.size[1], self.size[0]), Image.NEAREST)
class FreeScaleMask(object):
def __init__(self,size):
self.size = size
def __call__(self,mask):
return mask.resize((self.size[1], self.size[0]), Image.NEAREST)
class Scale(object):
def __init__(self, size):
self.size = size
def __call__(self, img, mask):
# if img.size != mask.size:
# print(img.size)
# print(mask.size)
assert img.size == mask.size
w, h = img.size
if (w <= h and w == self.size) or (h <= w and h == self.size):
return img, mask
if w < h:
ow = self.size
oh = int(self.size * h / w)
return img.resize((ow, oh), Image.BILINEAR), mask.resize((ow, oh), Image.NEAREST)
else:
oh = self.size
ow = int(self.size * w / h)
return img.resize((ow, oh), Image.BILINEAR), mask.resize((ow, oh), Image.NEAREST)
class RandomRotate(object):
"""Crops the given PIL.Image at a random location to have a region of
the given size. size can be a tuple (target_height, target_width)
or an integer, in which case the target will be of a square shape (size, size)
"""
def __init__(self, angle):
self.angle = angle
def __call__(self, image, label):
# w, h = image.size
# if w != 1280 or h != 720:
# image = image.resize(image, (1280, 720))
# print(w, h)
assert label is None or image.size == label.size
angle = random.randint(0, self.angle * 2) - self.angle
label = label.rotate(angle, resample=Image.NEAREST)
image = image.rotate(angle, resample=Image.BILINEAR)
return image, label
# ===============================label tranforms============================
class DeNormalize(object):
def __init__(self, mean, std):
self.mean = mean
self.std = std
def __call__(self, tensor):
for t, m, s in zip(tensor, self.mean, self.std):
t.mul_(s).add_(m)
return tensor
class MaskToTensor(object):
def __call__(self, img):
return torch.from_numpy(np.array(img, dtype=np.int32)).long()
def find_start_pos(row_sample,start_line):
# row_sample = row_sample.sort()
# for i,r in enumerate(row_sample):
# if r >= start_line:
# return i
l,r = 0,len(row_sample)-1
while True:
mid = int((l+r)/2)
if r - l == 1:
return r
if row_sample[mid] < start_line:
l = mid
if row_sample[mid] > start_line:
r = mid
if row_sample[mid] == start_line:
return mid
class RandomLROffsetLABEL(object):
def __init__(self,max_offset):
self.max_offset = max_offset
def __call__(self,img,label):
offset = np.random.randint(-self.max_offset,self.max_offset)
w, h = img.size
# print('max_offset:', self.max_offset, 'ro_offset: ', offset)
img = np.array(img)
if offset > 0:
img[:,offset:,:] = img[:,0:w-offset,:]
img[:,:offset,:] = 0
if offset < 0:
real_offset = -offset
img[:,0:w-real_offset,:] = img[:,real_offset:,:]
img[:,w-real_offset:,:] = 0
label = np.array(label)
if offset > 0:
label[:,offset:] = label[:,0:w-offset]
label[:,:offset] = 0
if offset < 0:
offset = -offset
label[:,0:w-offset] = label[:,offset:]
label[:,w-offset:] = 0
return Image.fromarray(img),Image.fromarray(label)
class RandomUDoffsetLABEL(object):
def __init__(self,max_offset):
self.max_offset = max_offset
def __call__(self,img,label):
offset = np.random.randint(-self.max_offset,self.max_offset)
# offset = np.random.randint(0, self.max_offset)
# offset = 17
# print('max_offset:', self.max_offset, 'do_offset: ', offset)
w, h = img.size
# if w != 1280 or h != 720:
# img = img.resize(img, (1280, 720))
# print(w, h)
img = np.array(img)
if offset > 0:
# print('dim > 0:', h - offset, offset)
# print(img[offset:,:,:].shape, img[0:h-offset,:,:].shape)
img[offset:,:,:] = img[0:h-offset,:,:]
img[:offset,:,:] = 0
if offset < 0:
real_offset = -offset
# print('dim < 0:', h - real_offset, real_offset)
img[0:h-real_offset,:,:] = img[real_offset:,:,:]
img[h-real_offset:,:,:] = 0
label = np.array(label)
if offset > 0:
# print('dla > 0:', h - offset, offset)
# # print(label[0:h-offset,:].shape, label[offset:,:].shape)
# if label[0:h-offset,:].shape == label[offset:,:].shape:
label[offset:,:] = label[0:h-offset,:]
label[:offset,:] = 0
# else:
# print(label[0:h - offset, :].shape, label[offset:, :].shape, '/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/')
if offset < 0:
offset = -offset
# print('dla < 0:', h - offset, offset)
label[0:h-offset,:] = label[offset:,:]
label[h-offset:,:] = 0
return Image.fromarray(img),Image.fromarray(label)

149
algorithms/lane_ufld/code/UFLD/demo.py Executable file
View File

@@ -0,0 +1,149 @@
import torch, os, cv2
from model.model import parsingNet
from utils.common import merge_config, checkpoint_state_dict
from utils.dist_utils import dist_print
import torch
import scipy.special, tqdm
import numpy as np
import torchvision.transforms as transforms
from data.dataset import LaneTestDataset
from data.constant import culane_row_anchor, tusimple_row_anchor
from scipy.optimize import curve_fit
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
# 自定义函数 e指数形式
def func(x, a, b, c):
return a * np.sqrt(x) * (b * np.square(x) + c)
def get_curve_fit(x, y):
# 非线性最小二乘法拟合
popt, pcov = curve_fit(func, x, y)
# 获取popt里面是拟合系数
# print(popt)
a = popt[0]
b = popt[1]
c = popt[2]
yvals = func(x, a, b, c) # 拟合y值
# print('popt:', popt)
# print('系数a:', a)
# print('系数b:', b)
# print('系数c:', c)
# print('系数pcov:', pcov)
# print('系数yvals:', yvals)
return yvals
if __name__ == "__main__":
torch.backends.cudnn.benchmark = True
args, cfg = merge_config()
dist_print('start testing...')
from model.backbone import SUPPORTED_BACKBONES
assert cfg.backbone in SUPPORTED_BACKBONES
if cfg.dataset == 'CULane':
cls_num_per_lane = 18
elif cfg.dataset == 'Tusimple':
cls_num_per_lane = 56
else:
raise NotImplementedError
net = parsingNet(pretrained=False, backbone=cfg.backbone, cls_dim=(cfg.griding_num+1, cls_num_per_lane, 4),
use_aux=False).to(device)
# use_aux=False).cuda() # we dont need auxiliary segmentation in testing
net.load_state_dict(checkpoint_state_dict(cfg.test_model, map_location=device), strict=False)
net.eval()
img_transforms = transforms.Compose([
transforms.Resize((288, 800)),
transforms.ToTensor(),
transforms.Normalize((0.485, 0.456, 0.406), (0.229, 0.224, 0.225)),
])
if cfg.dataset == 'CULane':
splits = ['test0_normal.txt']
# splits = ['test0_normal.txt', 'test1_crowd.txt', 'test2_hlight.txt', 'test3_shadow.txt', 'test4_noline.txt', 'test5_arrow.txt', 'test6_curve.txt', 'test7_cross.txt', 'test8_night.txt']
datasets = [LaneTestDataset(cfg.data_root,os.path.join(cfg.data_root, 'list/test_split/'+split),img_transform = img_transforms) for split in splits]
img_w, img_h = 1280, 720
# img_w, img_h = 1640, 590
row_anchor = culane_row_anchor
elif cfg.dataset == 'Tusimple':
splits = ['test3.txt']
datasets = [LaneTestDataset(cfg.data_root,os.path.join(cfg.data_root, split),img_transform = img_transforms) for split in splits]
# img_w, img_h = 998, 560
# img_w, img_h = 960, 546
img_w, img_h = 1280, 720
row_anchor = tusimple_row_anchor
else:
raise NotImplementedError
for split, dataset in zip(splits, datasets):
loader = torch.utils.data.DataLoader(dataset, batch_size=1, shuffle=False, num_workers=1)
fourcc = cv2.VideoWriter_fourcc(*'MJPG')
print(split[:-3]+'avi')
vout = cv2.VideoWriter(split[:-3]+'avi', fourcc, 15.0, (img_w, img_h))
for i, data in enumerate(tqdm.tqdm(loader)):
imgs, names = data
# imgs = imgs.cuda()
imgs = imgs.to(device)
with torch.no_grad():
out = net(imgs)
# print(out)
col_sample = np.linspace(0, 800 - 1, cfg.griding_num)
col_sample_w = col_sample[1] - col_sample[0]
out_j = out[0].data.cpu().numpy()
out_j = out_j[:, ::-1, :]
prob = scipy.special.softmax(out_j[:-1, :, :], axis=0)
idx = np.arange(cfg.griding_num) + 1
idx = idx.reshape(-1, 1, 1)
loc = np.sum(prob * idx, axis=0)
out_j = np.argmax(out_j, axis=0)
loc[out_j == cfg.griding_num] = 0
out_j = loc
x_list = []
y_list = []
# import pdb; pdb.set_trace()
vis = cv2.imread(os.path.join(cfg.data_root, names[0]))
for i in range(out_j.shape[1]):
# print(out_j.shape[1])
if np.sum(out_j[:, i] != 0) > 2:
for k in range(out_j.shape[0]):
# print(out_j.shape[0])
if out_j[k, i] > 0:
ppp = (int(out_j[k, i] * col_sample_w * img_w / 800) - 1, int(img_h * (row_anchor[cls_num_per_lane-1-k]/288)) - 1)
# if len(x_list) >= 1:
# if abs(ppp[0] - x_list[-1]) < 12 or abs(ppp[0] - x_list[-1]) > 30:
# x_list.append(ppp[0])
# y_list.append(ppp[1])
# print('x_list[-1]', x_list[-1])
# else:
# x_list.append(ppp[0])
# y_list.append(ppp[1])
x_list.append(ppp[0])
y_list.append(ppp[1])
print(ppp)
# cv2.circle(vis, ppp, 5, (0, 255, 0), -1)
x = np.array(x_list)
yvals = np.array(y_list)
print(yvals)
# yvals = get_curve_fit(x, yvals)
# print(yvals)
start_p = (x[0], int(yvals[0]))
end_p = (x[0], int(yvals[0]))
# for i in range(len(yvals) - 1):
# if abs(int(yvals[i]) - int(yvals[i + 1])) <= 50:
# end_p = (x[i + 1], int(yvals[i + 1]))
# else:
# cv2.line(vis, start_p, end_p, (0, 0, 255), 3)
# start_p = (x[i + 1], int(yvals[i + 1]))
# if i == len(yvals) - 2:
# cv2.line(vis, start_p, end_p, (0, 0, 255), 3)
# cv2.line(vis, (x[i], int(yvals[i])), (x[i + 1], int(yvals[i + 1])), (0, 0, 255), 3)
cv2.imshow('vis', vis)
cv2.waitKey(1)
vout.write(vis)
vout.release()

157
algorithms/lane_ufld/code/UFLD/demo_new.py Executable file
View File

@@ -0,0 +1,157 @@
import torch, os, cv2
from model.model import parsingNet
from utils.common import merge_config
from utils.dist_utils import dist_print
import torch
import scipy.special, tqdm
import numpy as np
import torchvision.transforms as transforms
from data.dataset import LaneTestDataset
from data.constant import culane_row_anchor, tusimple_row_anchor
from scipy.optimize import curve_fit
from lane_show import is_in_poly, handle_point, poly_fitting, draw_values
import time
# device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
# 自定义函数 e指数形式
def func(x, a, b, c):
return a * np.sqrt(x) * (b * np.square(x) + c)
def get_curve_fit(x, y):
# 非线性最小二乘法拟合
popt, pcov = curve_fit(func, x, y)
# 获取popt里面是拟合系数
# print(popt)
a = popt[0]
b = popt[1]
c = popt[2]
yvals = func(x, a, b, c) # 拟合y值
# print('popt:', popt)
# print('系数a:', a)
# print('系数b:', b)
# print('系数c:', c)
# print('系数pcov:', pcov)
# print('系数yvals:', yvals)
return yvals
if __name__ == "__main__":
torch.backends.cudnn.benchmark = True
args, cfg = merge_config()
dist_print('start testing...')
from model.backbone import SUPPORTED_BACKBONES
assert cfg.backbone in SUPPORTED_BACKBONES
if cfg.dataset == 'CULane':
cls_num_per_lane = 18
elif cfg.dataset == 'Tusimple':
cls_num_per_lane = 56
else:
raise NotImplementedError
net = parsingNet(pretrained=False, backbone=cfg.backbone, cls_dim=(cfg.griding_num + 1, cls_num_per_lane, 4),
# use_aux=False).to(device)
use_aux=False).cuda() # we dont need auxiliary segmentation in testing
state_dict = torch.load(cfg.test_model, map_location='cuda')['model']
compatible_state_dict = {}
for k, v in state_dict.items():
if 'module.' in k:
compatible_state_dict[k[7:]] = v
else:
compatible_state_dict[k] = v
net.load_state_dict(compatible_state_dict, strict=False)
net.eval()
# net = torch.load(cfg.test_model, map_location='cuda')
img_transforms = transforms.Compose([
transforms.Resize((288, 800)),
transforms.ToTensor(),
transforms.Normalize((0.485, 0.456, 0.406), (0.229, 0.224, 0.225)),
])
if cfg.dataset == 'CULane':
splits = ['test0_normal.txt']
# splits = ['test0_normal.txt', 'test1_crowd.txt', 'test2_hlight.txt', 'test3_shadow.txt', 'test4_noline.txt', 'test5_arrow.txt', 'test6_curve.txt', 'test7_cross.txt', 'test8_night.txt']
datasets = [LaneTestDataset(cfg.data_root, os.path.join(cfg.data_root, 'list/test_split/' + split),
img_transform=img_transforms) for split in splits]
img_w, img_h = 1280, 720
# img_w, img_h = 1640, 590
row_anchor = culane_row_anchor
elif cfg.dataset == 'Tusimple':
splits = ['test1.txt']
datasets = [LaneTestDataset(cfg.data_root, os.path.join(cfg.data_root, split), img_transform=img_transforms) for
split in splits]
# img_w, img_h = 998, 560
# img_w, img_h = 960, 546
img_w, img_h = 1280, 720
row_anchor = tusimple_row_anchor
else:
raise NotImplementedError
for split, dataset in zip(splits, datasets):
loader = torch.utils.data.DataLoader(dataset, batch_size=1, shuffle=False, num_workers=1)
fourcc = cv2.VideoWriter_fourcc(*'MJPG')
print(split[:-3] + 'avi')
vout = cv2.VideoWriter(split[:-3] + 'avi', fourcc, 5.0, (img_w, img_h))
s = 0
count = 0
for i, data in enumerate(tqdm.tqdm(loader)):
imgs, names = data
imgs = imgs.cuda()
# imgs = imgs.to(device)
with torch.no_grad():
start_t = time.time()
out = net(imgs)
end_t = time.time()
count_t = end_t - start_t
print('the pre time is : ', count_t)
if count > 0:
s = s + count_t
count = count + 1
col_sample = np.linspace(0, 800 - 1, cfg.griding_num)
col_sample_w = col_sample[1] - col_sample[0]
out_j = out[0].data.cpu().numpy()
out_j = out_j[:, ::-1, :]
prob = scipy.special.softmax(out_j[:-1, :, :], axis=0)
idx = np.arange(cfg.griding_num) + 1
idx = idx.reshape(-1, 1, 1)
loc = np.sum(prob * idx, axis=0)
out_j = np.argmax(out_j, axis=0)
loc[out_j == cfg.griding_num] = 0
out_j = loc
print('out:', len(out_j), out_j.shape)
x_list = []
y_list = []
# import pdb; pdb.set_trace()
vis = cv2.imread(os.path.join(cfg.data_root, names[0]))
for i in range(out_j.shape[1]):
# print(out_j.shape[1])
if np.sum(out_j[:, i] != 0) > 2:
poly = [[50, 50], [50, 719], [1250, 50], [1250, 719]] # ROI区域
lane_x = []
lane_y = []
for k in range(out_j.shape[0]):
# print(out_j.shape[0])
if out_j[k, i] > 0:
ppp = (int(out_j[k, i] * col_sample_w * img_w / 800) - 1,
int(img_h * (row_anchor[cls_num_per_lane - 1 - k] / 288)) - 1)
is_in = is_in_poly(ppp, poly)
if is_in == True:
# 将处理后的点坐标添如一个空列表做拟合用
lane_x.append(ppp[0])
lane_y.append(ppp[1])
cv2.circle(vis, ppp, 5, (0, 255, 0), -1)
lx, ly, rx, ry = handle_point(lane_x, lane_y)
# print(lx, ly, rx, ry)
# curvature, distance_from_center = poly_fitting(lx, ly, rx, ry)
# draw_values(vis, curvature, distance_from_center)
# cv2.imshow('vis', vis)
# cv2.waitKey(1)
vout.write(vis)
print('mean count time: ', s/count)
vout.release()

8
algorithms/lane_ufld/code/UFLD/demo_onnx.py Executable file
View File

@@ -0,0 +1,8 @@
import onnx
# 加载模型
model = onnx.load('./model/tusimple_18.onnx')
# 检查模型格式是否完整及正确
onnx.checker.check_model(model)
# 获取输出层，包含层名称、维度信息
output = model.graph.output
print(output)

79
algorithms/lane_ufld/code/UFLD/evaluation/culane/CMakeLists.txt Executable file
View File

@@ -0,0 +1,79 @@
# Thanks for the contribution of zchrissirhcz imzhuo@foxmail.com
cmake_minimum_required(VERSION 3.1)
project(culane_evaluator)
set(CMAKE_CXX_STANDARD 11)
add_definitions(
-DCPU_ONLY
)
set(SRC_LST
${CMAKE_SOURCE_DIR}/src/counter.cpp
${CMAKE_SOURCE_DIR}/src/evaluate.cpp
${CMAKE_SOURCE_DIR}/src/lane_compare.cpp
${CMAKE_SOURCE_DIR}/src/spline.cpp
)
set(HDR_LST
${CMAKE_SOURCE_DIR}/include/counter.hpp
${CMAKE_SOURCE_DIR}/include/hungarianGraph.hpp
${CMAKE_SOURCE_DIR}/include/lane_compare.hpp
${CMAKE_SOURCE_DIR}/include/spline.hpp
)
if (CMAKE_SYSTEM_NAME MATCHES "Windows")
list(APPEND SRC_LST ${CMAKE_SOURCE_DIR}/getopt/getopt.c)
list(APPEND HDR_LST ${CMAKE_SOURCE_DIR}/getopt/getopt.h)
endif()
add_executable(${PROJECT_NAME}
${SRC_LST}
${HDR_LST}
)
set(dep_libs "")
#--- OpenCV
# You may switch different version of OpenCV like this:
# set(OpenCV_DIR "/usr/local/opencv-4.3.0" CACHE PATH "")
find_package(OpenCV REQUIRED
COMPONENTS core highgui imgproc imgcodecs
)
if(NOT OpenCV_FOUND) # if not OpenCV 4.x/3.x, then imgcodecs are not found
find_package(OpenCV REQUIRED COMPONENTS core highgui imgproc)
endif()
list(APPEND dep_libs
PUBLIC ${OpenCV_LIBS}
)
#--- OpenMP
find_package(OpenMP)
if(NOT TARGET OpenMP::OpenMP_CXX AND (OpenMP_CXX_FOUND OR OPENMP_FOUND))
target_compile_options(${PROJECT_NAME} PRIVATE ${OpenMP_CXX_FLAGS})
endif()
if(OpenMP_CXX_FOUND OR OPENMP_FOUND)
message(STATUS "Building with OpenMP")
if(OpenMP_CXX_FOUND)
list(APPEND dep_libs PUBLIC OpenMP::OpenMP_CXX)
else()
list(APPEND dep_libs PRIVATE "${OpenMP_CXX_FLAGS}")
endif()
endif()
set(dep_incs ${CMAKE_SOURCE_DIR}/include)
if (CMAKE_SYSTEM_NAME MATCHES "Windows")
list(APPEND dep_incs "${CMAKE_SOURCE_DIR}/getopt")
endif()
# --- target config with include dirs / libs
target_link_libraries(${PROJECT_NAME}
${dep_libs}
)
target_include_directories(${PROJECT_NAME}
PUBLIC ${dep_incs}
)

50
algorithms/lane_ufld/code/UFLD/evaluation/culane/Makefile Executable file
View File

@@ -0,0 +1,50 @@
PROJECT_NAME:= evaluate
# config ----------------------------------
INCLUDE_DIRS := include
LIBRARY_DIRS := lib
# You may switch different versions of opencv like this:
# export PKG_CONFIG_PATH=/usr/local/opencv-4.1.1/lib/pkgconfig:$PKG_CONFIG_PATH
# then use `pkg-config opencv4 --cflags --libs` since `opencv4.pc` is found
COMMON_FLAGS := -DCPU_ONLY
CXXFLAGS := -std=c++11 -fopenmp #`pkg-config --cflags opencv`
LDFLAGS := -fopenmp -Wl,-rpath,./lib #`pkg-config --libs opencv`
BUILD_DIR := build
# make rules -------------------------------
CXX ?= g++
BUILD_DIR ?= ./build
LIBRARIES += opencv_core opencv_highgui opencv_imgproc opencv_imgcodecs
CXXFLAGS += $(COMMON_FLAGS) $(foreach includedir,$(INCLUDE_DIRS),-I$(includedir))
LDFLAGS += $(COMMON_FLAGS) $(foreach includedir,$(LIBRARY_DIRS),-L$(includedir)) $(foreach library,$(LIBRARIES),-l$(library))
SRC_DIRS += $(shell find * -type d -exec bash -c "find {} -maxdepth 1 \( -name '*.cpp' -o -name '*.proto' \) | grep -q ." \; -print)
CXX_SRCS += $(shell find src/ -name "*.cpp")
CXX_TARGETS:=$(patsubst %.cpp, $(BUILD_DIR)/%.o, $(CXX_SRCS))
ALL_BUILD_DIRS := $(sort $(BUILD_DIR) $(addprefix $(BUILD_DIR)/, $(SRC_DIRS)))
.PHONY: all
all: $(PROJECT_NAME)
.PHONY: $(ALL_BUILD_DIRS)
$(ALL_BUILD_DIRS):
@mkdir -p $@
$(BUILD_DIR)/%.o: %.cpp | $(ALL_BUILD_DIRS)
@echo "CXX" $<
@$(CXX) $(CXXFLAGS) -c -o $@ $<
$(PROJECT_NAME): $(CXX_TARGETS)
@echo "CXX/LD" $@
@$(CXX) -o $@ $^ $(LDFLAGS)
.PHONY: clean
clean:
@rm -rf $(CXX_TARGETS)
@rm -rf $(PROJECT_NAME)
@rm -rf $(BUILD_DIR)

23
algorithms/lane_ufld/code/UFLD/evaluation/culane/calTotal.m Executable file
View File

@@ -0,0 +1,23 @@
%% Calculate overall Fmeasure from each scenarios
clc; clear; close all;
allFile = 'output/vgg_SCNN_DULR_w9_iou0.5.txt';
all = textread(allFile,'%s');
TP = 0;
FP = 0;
FN = 0;
for i=1:9
tpline = (i-1)*14+4;
tp = str2double(all(tpline));
fp = str2double(all(tpline+2));
fn = str2double(all(tpline+4));
TP = TP + tp;
FP = FP + fp;
FN = FN + fn;
end
P = TP/(TP + FP)
R = TP/(TP + FN)
F = 2*P*R/(P + R)*100

201
algorithms/lane_ufld/code/UFLD/evaluation/culane/getopt/LICENSE.md Executable file
View File

@@ -0,0 +1,201 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

51
algorithms/lane_ufld/code/UFLD/evaluation/culane/getopt/getopt.c Executable file
View File

@@ -0,0 +1,51 @@
/* *****************************************************************
*
* Copyright 2016 Microsoft
*
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
******************************************************************/
#include "getopt.h"
#include <windows.h>
char* optarg = NULL;
int optind = 1;
int getopt(int argc, char *const argv[], const char *optstring)
{
if ((optind >= argc) || (argv[optind][0] != '-') || (argv[optind][0] == 0))
{
return -1;
}
int opt = argv[optind][1];
const char *p = strchr(optstring, opt);
if (p == NULL)
{
return '?';
}
if (p[1] == ':')
{
optind++;
if (optind >= argc)
{
return '?';
}
optarg = argv[optind];
optind++;
}
return opt;
}

36
algorithms/lane_ufld/code/UFLD/evaluation/culane/getopt/getopt.h Executable file
View File

@@ -0,0 +1,36 @@
/* *****************************************************************
*
* Copyright 2016 Microsoft
*
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
******************************************************************/
#ifndef GETOPT_H__
#define GETOPT_H__
#ifdef __cplusplus
extern "C" {
#endif
extern char *optarg;
extern int optind;
int getopt(int argc, char *const argv[], const char *optstring);
#ifdef __cplusplus
}
#endif
#endif

3
algorithms/lane_ufld/code/UFLD/evaluation/culane/getopt/readme.txt Executable file
View File

@@ -0,0 +1,3 @@
For windows build, `getopt.c` and `getopt.h` are required.
They are taken from the [iotivity](https://github.com/iotivity/iotivity) open source project, under Apache LICENSE 2.0.

47
algorithms/lane_ufld/code/UFLD/evaluation/culane/include/counter.hpp Executable file
View File

@@ -0,0 +1,47 @@
#ifndef COUNTER_HPP
#define COUNTER_HPP
#include "lane_compare.hpp"
#include "hungarianGraph.hpp"
#include <iostream>
#include <algorithm>
#include <tuple>
#include <vector>
#include <opencv2/core.hpp>
using namespace std;
using namespace cv;
// before coming to use functions of this class, the lanes should resize to im_width and im_height using resize_lane() in lane_compare.hpp
class Counter
{
public:
Counter(int _im_width, int _im_height, double _iou_threshold=0.4, int _lane_width=10):tp(0),fp(0),fn(0){
im_width = _im_width;
im_height = _im_height;
sim_threshold = _iou_threshold;
lane_compare = new LaneCompare(_im_width, _im_height, _lane_width, LaneCompare::IOU);
};
double get_precision(void);
double get_recall(void);
long getTP(void);
long getFP(void);
long getFN(void);
void setTP(long);
void setFP(long);
void setFN(long);
// direct add tp, fp, tn and fn
// first match with hungarian
tuple<vector<int>, long, long, long, long> count_im_pair(const vector<vector<Point2f> > &anno_lanes, const vector<vector<Point2f> > &detect_lanes);
void makeMatch(const vector<vector<double> > &similarity, vector<int> &match1, vector<int> &match2);
private:
double sim_threshold;
int im_width;
int im_height;
long tp;
long fp;
long fn;
LaneCompare *lane_compare;
};
#endif

71
algorithms/lane_ufld/code/UFLD/evaluation/culane/include/hungarianGraph.hpp Executable file
View File

@@ -0,0 +1,71 @@
#ifndef HUNGARIAN_GRAPH_HPP
#define HUNGARIAN_GRAPH_HPP
#include <vector>
using namespace std;
struct pipartiteGraph {
vector<vector<double> > mat;
vector<bool> leftUsed, rightUsed;
vector<double> leftWeight, rightWeight;
vector<int>rightMatch, leftMatch;
int leftNum, rightNum;
bool matchDfs(int u) {
leftUsed[u] = true;
for (int v = 0; v < rightNum; v++) {
if (!rightUsed[v] && fabs(leftWeight[u] + rightWeight[v] - mat[u][v]) < 1e-2) {
rightUsed[v] = true;
if (rightMatch[v] == -1 || matchDfs(rightMatch[v])) {
rightMatch[v] = u;
leftMatch[u] = v;
return true;
}
}
}
return false;
}
void resize(int leftNum, int rightNum) {
this->leftNum = leftNum;
this->rightNum = rightNum;
leftMatch.resize(leftNum);
rightMatch.resize(rightNum);
leftUsed.resize(leftNum);
rightUsed.resize(rightNum);
leftWeight.resize(leftNum);
rightWeight.resize(rightNum);
mat.resize(leftNum);
for (int i = 0; i < leftNum; i++) mat[i].resize(rightNum);
}
void match() {
for (int i = 0; i < leftNum; i++) leftMatch[i] = -1;
for (int i = 0; i < rightNum; i++) rightMatch[i] = -1;
for (int i = 0; i < rightNum; i++) rightWeight[i] = 0;
for (int i = 0; i < leftNum; i++) {
leftWeight[i] = -1e5;
for (int j = 0; j < rightNum; j++) {
if (leftWeight[i] < mat[i][j]) leftWeight[i] = mat[i][j];
}
}
for (int u = 0; u < leftNum; u++) {
while (1) {
for (int i = 0; i < leftNum; i++) leftUsed[i] = false;
for (int i = 0; i < rightNum; i++) rightUsed[i] = false;
if (matchDfs(u)) break;
double d = 1e10;
for (int i = 0; i < leftNum; i++) {
if (leftUsed[i] ) {
for (int j = 0; j < rightNum; j++) {
if (!rightUsed[j]) d = min(d, leftWeight[i] + rightWeight[j] - mat[i][j]);
}
}
}
if (d == 1e10) return ;
for (int i = 0; i < leftNum; i++) if (leftUsed[i]) leftWeight[i] -= d;
for (int i = 0; i < rightNum; i++) if (rightUsed[i]) rightWeight[i] += d;
}
}
}
};
#endif // HUNGARIAN_GRAPH_HPP

37
algorithms/lane_ufld/code/UFLD/evaluation/culane/include/lane_compare.hpp Executable file
View File

@@ -0,0 +1,37 @@
#ifndef LANE_COMPARE_HPP
#define LANE_COMPARE_HPP
#include "spline.hpp"
#include <vector>
#include <iostream>
#include <opencv2/core.hpp>
#include <opencv2/imgproc.hpp>
using namespace std;
using namespace cv;
class LaneCompare{
public:
enum CompareMode{
IOU,
Caltech
};
LaneCompare(int _im_width, int _im_height, int _lane_width = 10, CompareMode _compare_mode = IOU){
im_width = _im_width;
im_height = _im_height;
compare_mode = _compare_mode;
lane_width = _lane_width;
}
double get_lane_similarity(const vector<Point2f> &lane1, const vector<Point2f> &lane2);
void resize_lane(vector<Point2f> &curr_lane, int curr_width, int curr_height);
private:
CompareMode compare_mode;
int im_width;
int im_height;
int lane_width;
Spline splineSolver;
};
#endif

28
algorithms/lane_ufld/code/UFLD/evaluation/culane/include/spline.hpp Executable file
View File

@@ -0,0 +1,28 @@
#ifndef SPLINE_HPP
#define SPLINE_HPP
#include <vector>
#include <cstdio>
#include <math.h>
#include <opencv2/core.hpp>
using namespace cv;
using namespace std;
struct Func {
double a_x;
double b_x;
double c_x;
double d_x;
double a_y;
double b_y;
double c_y;
double d_y;
double h;
};
class Spline {
public:
vector<Point2f> splineInterpTimes(const vector<Point2f> &tmp_line, int times);
vector<Point2f> splineInterpStep(vector<Point2f> tmp_line, double step);
vector<Func> cal_fun(const vector<Point2f> &point_v);
};
#endif

37
algorithms/lane_ufld/code/UFLD/evaluation/culane/run-full.sh Executable file
View File

@@ -0,0 +1,37 @@
root=../../
data_dir=${root}data/CULane/
exp=vgg_SCNN_DULR_w9
detect_dir=${root}tools/prob2lines/output/${exp}/
w_lane=30;
iou=0.5; # Set iou to 0.3 or 0.5
im_w=1640
im_h=590
frame=1
list0=${data_dir}list/test_split/test0_normal.txt
list1=${data_dir}list/test_split/test1_crowd.txt
list2=${data_dir}list/test_split/test2_hlight.txt
list3=${data_dir}list/test_split/test3_shadow.txt
list4=${data_dir}list/test_split/test4_noline.txt
list5=${data_dir}list/test_split/test5_arrow.txt
list6=${data_dir}list/test_split/test6_curve.txt
list7=${data_dir}list/test_split/test7_cross.txt
list8=${data_dir}list/test_split/test8_night.txt
out0=./output/out0_normal.txt
out1=./output/out1_crowd.txt
out2=./output/out2_hlight.txt
out3=./output/out3_shadow.txt
out4=./output/out4_noline.txt
out5=./output/out5_arrow.txt
out6=./output/out6_curve.txt
out7=./output/out7_cross.txt
out8=./output/out8_night.txt
./evaluate -a $data_dir -d $detect_dir -i $data_dir -l $list0 -w $w_lane -t $iou -c $im_w -r $im_h -f $frame -o $out0
./evaluate -a $data_dir -d $detect_dir -i $data_dir -l $list1 -w $w_lane -t $iou -c $im_w -r $im_h -f $frame -o $out1
./evaluate -a $data_dir -d $detect_dir -i $data_dir -l $list2 -w $w_lane -t $iou -c $im_w -r $im_h -f $frame -o $out2
./evaluate -a $data_dir -d $detect_dir -i $data_dir -l $list3 -w $w_lane -t $iou -c $im_w -r $im_h -f $frame -o $out3
./evaluate -a $data_dir -d $detect_dir -i $data_dir -l $list4 -w $w_lane -t $iou -c $im_w -r $im_h -f $frame -o $out4
./evaluate -a $data_dir -d $detect_dir -i $data_dir -l $list5 -w $w_lane -t $iou -c $im_w -r $im_h -f $frame -o $out5
./evaluate -a $data_dir -d $detect_dir -i $data_dir -l $list6 -w $w_lane -t $iou -c $im_w -r $im_h -f $frame -o $out6
./evaluate -a $data_dir -d $detect_dir -i $data_dir -l $list7 -w $w_lane -t $iou -c $im_w -r $im_h -f $frame -o $out7
./evaluate -a $data_dir -d $detect_dir -i $data_dir -l $list8 -w $w_lane -t $iou -c $im_w -r $im_h -f $frame -o $out8
cat ./output/out*.txt>./output/${exp}_iou${iou}_split.txt

12
algorithms/lane_ufld/code/UFLD/evaluation/culane/run-lite.sh Executable file
View File

@@ -0,0 +1,12 @@
root=../../
data_dir=${root}data/CULane/
exp=vgg_SCNN_DULR_w9
detect_dir=${root}tools/prob2lines/output/${exp}/
w_lane=30;
iou=0.5; # Set iou to 0.3 or 0.5
im_w=1640
im_h=590
frame=1
list=${data_dir}list/test.txt
out=./output/${exp}_iou${iou}.txt
./evaluate -a $data_dir -d $detect_dir -i $data_dir -l $list -w $w_lane -t $iou -c $im_w -r $im_h -f $frame -o $out

134
algorithms/lane_ufld/code/UFLD/evaluation/culane/src/counter.cpp Executable file
View File

@@ -0,0 +1,134 @@
/*************************************************************************
> File Name: counter.cpp
> Author: Xingang Pan, Jun Li
> Mail: px117@ie.cuhk.edu.hk
> Created Time: Thu Jul 14 20:23:08 2016
************************************************************************/
#include "counter.hpp"
double Counter::get_precision(void)
{
cerr<<"tp: "<<tp<<" fp: "<<fp<<" fn: "<<fn<<endl;
if(tp+fp == 0)
{
cerr<<"no positive detection"<<endl;
return -1;
}
return tp/double(tp + fp);
}
double Counter::get_recall(void)
{
if(tp+fn == 0)
{
cerr<<"no ground truth positive"<<endl;
return -1;
}
return tp/double(tp + fn);
}
long Counter::getTP(void)
{
return tp;
}
long Counter::getFP(void)
{
return fp;
}
long Counter::getFN(void)
{
return fn;
}
void Counter::setTP(long value)
{
tp = value;
}
void Counter::setFP(long value)
{
fp = value;
}
void Counter::setFN(long value)
{
fn = value;
}
tuple<vector<int>, long, long, long, long> Counter::count_im_pair(const vector<vector<Point2f> > &anno_lanes, const vector<vector<Point2f> > &detect_lanes)
{
vector<int> anno_match(anno_lanes.size(), -1);
vector<int> detect_match;
if(anno_lanes.empty())
{
return make_tuple(anno_match, 0, detect_lanes.size(), 0, 0);
}
if(detect_lanes.empty())
{
return make_tuple(anno_match, 0, 0, 0, anno_lanes.size());
}
// hungarian match first
// first calc similarity matrix
vector<vector<double> > similarity(anno_lanes.size(), vector<double>(detect_lanes.size(), 0));
for(int i=0; i<anno_lanes.size(); i++)
{
const vector<Point2f> &curr_anno_lane = anno_lanes[i];
for(int j=0; j<detect_lanes.size(); j++)
{
const vector<Point2f> &curr_detect_lane = detect_lanes[j];
similarity[i][j] = lane_compare->get_lane_similarity(curr_anno_lane, curr_detect_lane);
}
}
makeMatch(similarity, anno_match, detect_match);
int curr_tp = 0;
// count and add
for(int i=0; i<anno_lanes.size(); i++)
{
if(anno_match[i]>=0 && similarity[i][anno_match[i]] > sim_threshold)
{
curr_tp++;
}
else
{
anno_match[i] = -1;
}
}
int curr_fn = anno_lanes.size() - curr_tp;
int curr_fp = detect_lanes.size() - curr_tp;
return make_tuple(anno_match, curr_tp, curr_fp, 0, curr_fn);
}
void Counter::makeMatch(const vector<vector<double> > &similarity, vector<int> &match1, vector<int> &match2) {
int m = similarity.size();
int n = similarity[0].size();
pipartiteGraph gra;
bool have_exchange = false;
if (m > n) {
have_exchange = true;
swap(m, n);
}
gra.resize(m, n);
for (int i = 0; i < gra.leftNum; i++) {
for (int j = 0; j < gra.rightNum; j++) {
if(have_exchange)
gra.mat[i][j] = similarity[j][i];
else
gra.mat[i][j] = similarity[i][j];
}
}
gra.match();
match1 = gra.leftMatch;
match2 = gra.rightMatch;
if (have_exchange) swap(match1, match2);
}

302
algorithms/lane_ufld/code/UFLD/evaluation/culane/src/evaluate.cpp Executable file
View File

@@ -0,0 +1,302 @@
/*************************************************************************
> File Name: evaluate.cpp
> Author: Xingang Pan, Jun Li
> Mail: px117@ie.cuhk.edu.hk
> Created Time: 2016年07月14日 星期四 18时28分45秒
************************************************************************/
#include "counter.hpp"
#include "spline.hpp"
#if __linux__
#include <unistd.h>
#elif _MSC_VER
#include "getopt.h"
#endif
#include <iostream>
#include <fstream>
#include <sstream>
#include <cstdlib>
#include <string>
#include <opencv2/core.hpp>
#include <opencv2/highgui.hpp>
using namespace std;
using namespace cv;
void help(void)
{
cout<<"./evaluate [OPTIONS]"<<endl;
cout<<"-h : print usage help"<<endl;
cout<<"-a : directory for annotation files (default: /data/driving/eval_data/anno_label/)"<<endl;
cout<<"-d : directory for detection files (default: /data/driving/eval_data/predict_label/)"<<endl;
cout<<"-i : directory for image files (default: /data/driving/eval_data/img/)"<<endl;
cout<<"-l : list of images used for evaluation (default: /data/driving/eval_data/img/all.txt)"<<endl;
cout<<"-w : width of the lanes (default: 10)"<<endl;
cout<<"-t : threshold of iou (default: 0.4)"<<endl;
cout<<"-c : cols (max image width) (default: 1920)"<<endl;
cout<<"-r : rows (max image height) (default: 1080)"<<endl;
cout<<"-s : show visualization"<<endl;
cout<<"-f : start frame in the test set (default: 1)"<<endl;
}
void read_lane_file(const string &file_name, vector<vector<Point2f> > &lanes);
void visualize(string &full_im_name, vector<vector<Point2f> > &anno_lanes, vector<vector<Point2f> > &detect_lanes, vector<int> anno_match, int width_lane);
int main(int argc, char **argv)
{
// process params
string anno_dir = "/data/driving/eval_data/anno_label/";
string detect_dir = "/data/driving/eval_data/predict_label/";
string im_dir = "/data/driving/eval_data/img/";
string list_im_file = "/data/driving/eval_data/img/all.txt";
string output_file = "./output.txt";
int width_lane = 10;
double iou_threshold = 0.4;
int im_width = 1920;
int im_height = 1080;
int oc;
bool show = false;
int frame = 1;
while((oc = getopt(argc, argv, "ha:d:i:l:w:t:c:r:sf:o:")) != -1)
{
switch(oc)
{
case 'h':
help();
return 0;
case 'a':
anno_dir = optarg;
break;
case 'd':
detect_dir = optarg;
break;
case 'i':
im_dir = optarg;
break;
case 'l':
list_im_file = optarg;
break;
case 'w':
width_lane = atoi(optarg);
break;
case 't':
iou_threshold = atof(optarg);
break;
case 'c':
im_width = atoi(optarg);
break;
case 'r':
im_height = atoi(optarg);
break;
case 's':
show = true;
break;
case 'f':
frame = atoi(optarg);
break;
case 'o':
output_file = optarg;
break;
}
}
cout<<"------------Configuration---------"<<endl;
cout<<"anno_dir: "<<anno_dir<<endl;
cout<<"detect_dir: "<<detect_dir<<endl;
cout<<"im_dir: "<<im_dir<<endl;
cout<<"list_im_file: "<<list_im_file<<endl;
cout<<"width_lane: "<<width_lane<<endl;
cout<<"iou_threshold: "<<iou_threshold<<endl;
cout<<"im_width: "<<im_width<<endl;
cout<<"im_height: "<<im_height<<endl;
cout<<"-----------------------------------"<<endl;
cout<<"Evaluating the results..."<<endl;
// this is the max_width and max_height
if(width_lane<1)
{
cerr<<"width_lane must be positive"<<endl;
help();
return 1;
}
ifstream ifs_im_list(list_im_file, ios::in);
if(ifs_im_list.fail())
{
cerr<<"Error: file "<<list_im_file<<" not exist!"<<endl;
return 1;
}
Counter counter(im_width, im_height, iou_threshold, width_lane);
vector<int> anno_match;
string sub_im_name;
// pre-load filelist
vector<string> filelists;
while (getline(ifs_im_list, sub_im_name)) {
filelists.push_back(sub_im_name);
}
ifs_im_list.close();
vector<tuple<vector<int>, long, long, long, long>> tuple_lists;
tuple_lists.resize(filelists.size());
#pragma omp parallel for
for (int i = 0; i < filelists.size(); i++)
{
auto sub_im_name = filelists[i];
string full_im_name = im_dir + sub_im_name;
string sub_txt_name = sub_im_name.substr(0, sub_im_name.find_last_of(".")) + ".lines.txt";
string anno_file_name = anno_dir + sub_txt_name;
string detect_file_name = detect_dir + sub_txt_name;
vector<vector<Point2f> > anno_lanes;
vector<vector<Point2f> > detect_lanes;
read_lane_file(anno_file_name, anno_lanes);
read_lane_file(detect_file_name, detect_lanes);
//cerr<<count<<": "<<full_im_name<<endl;
tuple_lists[i] = counter.count_im_pair(anno_lanes, detect_lanes);
if (show)
{
auto anno_match = get<0>(tuple_lists[i]);
visualize(full_im_name, anno_lanes, detect_lanes, anno_match, width_lane);
waitKey(0);
}
}
long tp = 0, fp = 0, tn = 0, fn = 0;
for (auto result: tuple_lists) {
tp += get<1>(result);
fp += get<2>(result);
// tn = get<3>(result);
fn += get<4>(result);
}
counter.setTP(tp);
counter.setFP(fp);
counter.setFN(fn);
double precision = counter.get_precision();
double recall = counter.get_recall();
double F = 2 * precision * recall / (precision + recall);
cerr<<"finished process file"<<endl;
cout<<"precision: "<<precision<<endl;
cout<<"recall: "<<recall<<endl;
cout<<"Fmeasure: "<<F<<endl;
cout<<"----------------------------------"<<endl;
ofstream ofs_out_file;
ofs_out_file.open(output_file, ios::out);
ofs_out_file<<"file: "<<output_file<<endl;
ofs_out_file<<"tp: "<<counter.getTP()<<" fp: "<<counter.getFP()<<" fn: "<<counter.getFN()<<endl;
ofs_out_file<<"precision: "<<precision<<endl;
ofs_out_file<<"recall: "<<recall<<endl;
ofs_out_file<<"Fmeasure: "<<F<<endl<<endl;
ofs_out_file.close();
return 0;
}
void read_lane_file(const string &file_name, vector<vector<Point2f> > &lanes)
{
lanes.clear();
ifstream ifs_lane(file_name, ios::in);
if(ifs_lane.fail())
{
return;
}
string str_line;
while(getline(ifs_lane, str_line))
{
vector<Point2f> curr_lane;
stringstream ss;
ss<<str_line;
double x,y;
while(ss>>x>>y)
{
curr_lane.push_back(Point2f(x, y));
}
lanes.push_back(curr_lane);
}
ifs_lane.close();
}
void visualize(string &full_im_name, vector<vector<Point2f> > &anno_lanes, vector<vector<Point2f> > &detect_lanes, vector<int> anno_match, int width_lane)
{
Mat img = imread(full_im_name, 1);
Mat img2 = imread(full_im_name, 1);
vector<Point2f> curr_lane;
vector<Point2f> p_interp;
Spline splineSolver;
Scalar color_B = Scalar(255, 0, 0);
Scalar color_G = Scalar(0, 255, 0);
Scalar color_R = Scalar(0, 0, 255);
Scalar color_P = Scalar(255, 0, 255);
Scalar color;
for (int i=0; i<anno_lanes.size(); i++)
{
curr_lane = anno_lanes[i];
if(curr_lane.size() == 2)
{
p_interp = curr_lane;
}
else
{
p_interp = splineSolver.splineInterpTimes(curr_lane, 50);
}
if (anno_match[i] >= 0)
{
color = color_G;
}
else
{
color = color_G;
}
for (int n=0; n<p_interp.size()-1; n++)
{
line(img, p_interp[n], p_interp[n+1], color, width_lane);
line(img2, p_interp[n], p_interp[n+1], color, 2);
}
}
bool detected;
for (int i=0; i<detect_lanes.size(); i++)
{
detected = false;
curr_lane = detect_lanes[i];
if(curr_lane.size() == 2)
{
p_interp = curr_lane;
}
else
{
p_interp = splineSolver.splineInterpTimes(curr_lane, 50);
}
for (int n=0; n<anno_lanes.size(); n++)
{
if (anno_match[n] == i)
{
detected = true;
break;
}
}
if (detected == true)
{
color = color_B;
}
else
{
color = color_R;
}
for (int n=0; n<p_interp.size()-1; n++)
{
line(img, p_interp[n], p_interp[n+1], color, width_lane);
line(img2, p_interp[n], p_interp[n+1], color, 2);
}
}
namedWindow("visualize", 1);
imshow("visualize", img);
namedWindow("visualize2", 1);
imshow("visualize2", img2);
}

73
algorithms/lane_ufld/code/UFLD/evaluation/culane/src/lane_compare.cpp Executable file
View File

@@ -0,0 +1,73 @@
/*************************************************************************
> File Name: lane_compare.cpp
> Author: Xingang Pan, Jun Li
> Mail: px117@ie.cuhk.edu.hk
> Created Time: Fri Jul 15 10:26:32 2016
************************************************************************/
#include "lane_compare.hpp"
double LaneCompare::get_lane_similarity(const vector<Point2f> &lane1, const vector<Point2f> &lane2)
{
if(lane1.size()<2 || lane2.size()<2)
{
cerr<<"lane size must be greater or equal to 2"<<endl;
return 0;
}
Mat im1 = Mat::zeros(im_height, im_width, CV_8UC1);
Mat im2 = Mat::zeros(im_height, im_width, CV_8UC1);
// draw lines on im1 and im2
vector<Point2f> p_interp1;
vector<Point2f> p_interp2;
if(lane1.size() == 2)
{
p_interp1 = lane1;
}
else
{
p_interp1 = splineSolver.splineInterpTimes(lane1, 50);
}
if(lane2.size() == 2)
{
p_interp2 = lane2;
}
else
{
p_interp2 = splineSolver.splineInterpTimes(lane2, 50);
}
Scalar color_white = Scalar(1);
for(int n=0; n<p_interp1.size()-1; n++)
{
line(im1, p_interp1[n], p_interp1[n+1], color_white, lane_width);
}
for(int n=0; n<p_interp2.size()-1; n++)
{
line(im2, p_interp2[n], p_interp2[n+1], color_white, lane_width);
}
double sum_1 = cv::sum(im1).val[0];
double sum_2 = cv::sum(im2).val[0];
double inter_sum = cv::sum(im1.mul(im2)).val[0];
double union_sum = sum_1 + sum_2 - inter_sum;
double iou = inter_sum / union_sum;
return iou;
}
// resize the lane from Size(curr_width, curr_height) to Size(im_width, im_height)
void LaneCompare::resize_lane(vector<Point2f> &curr_lane, int curr_width, int curr_height)
{
if(curr_width == im_width && curr_height == im_height)
{
return;
}
double x_scale = im_width/(double)curr_width;
double y_scale = im_height/(double)curr_height;
for(int n=0; n<curr_lane.size(); n++)
{
curr_lane[n] = Point2f(curr_lane[n].x*x_scale, curr_lane[n].y*y_scale);
}
}

178
algorithms/lane_ufld/code/UFLD/evaluation/culane/src/spline.cpp Executable file
View File

@@ -0,0 +1,178 @@
#include <vector>
#include <iostream>
#include "spline.hpp"
using namespace std;
using namespace cv;
vector<Point2f> Spline::splineInterpTimes(const vector<Point2f>& tmp_line, int times) {
vector<Point2f> res;
if(tmp_line.size() == 2) {
double x1 = tmp_line[0].x;
double y1 = tmp_line[0].y;
double x2 = tmp_line[1].x;
double y2 = tmp_line[1].y;
for (int k = 0; k <= times; k++) {
double xi = x1 + double((x2 - x1) * k) / times;
double yi = y1 + double((y2 - y1) * k) / times;
res.push_back(Point2f(xi, yi));
}
}
else if(tmp_line.size() > 2)
{
vector<Func> tmp_func;
tmp_func = this->cal_fun(tmp_line);
if (tmp_func.empty()) {
cout << "in splineInterpTimes: cal_fun failed" << endl;
return res;
}
for(int j = 0; j < tmp_func.size(); j++)
{
double delta = tmp_func[j].h / times;
for(int k = 0; k < times; k++)
{
double t1 = delta*k;
double x1 = tmp_func[j].a_x + tmp_func[j].b_x*t1 + tmp_func[j].c_x*pow(t1,2) + tmp_func[j].d_x*pow(t1,3);
double y1 = tmp_func[j].a_y + tmp_func[j].b_y*t1 + tmp_func[j].c_y*pow(t1,2) + tmp_func[j].d_y*pow(t1,3);
res.push_back(Point2f(x1, y1));
}
}
res.push_back(tmp_line[tmp_line.size() - 1]);
}
else {
cerr << "in splineInterpTimes: not enough points" << endl;
}
return res;
}
vector<Point2f> Spline::splineInterpStep(vector<Point2f> tmp_line, double step) {
vector<Point2f> res;
/*
if (tmp_line.size() == 2) {
double x1 = tmp_line[0].x;
double y1 = tmp_line[0].y;
double x2 = tmp_line[1].x;
double y2 = tmp_line[1].y;
for (double yi = std::min(y1, y2); yi < std::max(y1, y2); yi += step) {
double xi;
if (yi == y1) xi = x1;
else xi = (x2 - x1) / (y2 - y1) * (yi - y1) + x1;
res.push_back(Point2f(xi, yi));
}
}*/
if (tmp_line.size() == 2) {
double x1 = tmp_line[0].x;
double y1 = tmp_line[0].y;
double x2 = tmp_line[1].x;
double y2 = tmp_line[1].y;
tmp_line[1].x = (x1 + x2) / 2;
tmp_line[1].y = (y1 + y2) / 2;
tmp_line.push_back(Point2f(x2, y2));
}
if (tmp_line.size() > 2) {
vector<Func> tmp_func;
tmp_func = this->cal_fun(tmp_line);
double ystart = tmp_line[0].y;
double yend = tmp_line[tmp_line.size() - 1].y;
bool down;
if (ystart < yend) down = 1;
else down = 0;
if (tmp_func.empty()) {
cerr << "in splineInterpStep: cal_fun failed" << endl;
}
for(int j = 0; j < tmp_func.size(); j++)
{
for(double t1 = 0; t1 < tmp_func[j].h; t1 += step)
{
double x1 = tmp_func[j].a_x + tmp_func[j].b_x*t1 + tmp_func[j].c_x*pow(t1,2) + tmp_func[j].d_x*pow(t1,3);
double y1 = tmp_func[j].a_y + tmp_func[j].b_y*t1 + tmp_func[j].c_y*pow(t1,2) + tmp_func[j].d_y*pow(t1,3);
res.push_back(Point2f(x1, y1));
}
}
res.push_back(tmp_line[tmp_line.size() - 1]);
}
else {
cerr << "in splineInterpStep: not enough points" << endl;
}
return res;
}
vector<Func> Spline::cal_fun(const vector<Point2f> &point_v)
{
vector<Func> func_v;
int n = point_v.size();
if(n<=2) {
cout << "in cal_fun: point number less than 3" << endl;
return func_v;
}
func_v.resize(point_v.size()-1);
vector<double> Mx(n);
vector<double> My(n);
vector<double> A(n-2);
vector<double> B(n-2);
vector<double> C(n-2);
vector<double> Dx(n-2);
vector<double> Dy(n-2);
vector<double> h(n-1);
//vector<func> func_v(n-1);
for(int i = 0; i < n-1; i++)
{
h[i] = sqrt(pow(point_v[i+1].x - point_v[i].x, 2) + pow(point_v[i+1].y - point_v[i].y, 2));
}
for(int i = 0; i < n-2; i++)
{
A[i] = h[i];
B[i] = 2*(h[i]+h[i+1]);
C[i] = h[i+1];
Dx[i] = 6*( (point_v[i+2].x - point_v[i+1].x)/h[i+1] - (point_v[i+1].x - point_v[i].x)/h[i] );
Dy[i] = 6*( (point_v[i+2].y - point_v[i+1].y)/h[i+1] - (point_v[i+1].y - point_v[i].y)/h[i] );
}
//TDMA
C[0] = C[0] / B[0];
Dx[0] = Dx[0] / B[0];
Dy[0] = Dy[0] / B[0];
for(int i = 1; i < n-2; i++)
{
double tmp = B[i] - A[i]*C[i-1];
C[i] = C[i] / tmp;
Dx[i] = (Dx[i] - A[i]*Dx[i-1]) / tmp;
Dy[i] = (Dy[i] - A[i]*Dy[i-1]) / tmp;
}
Mx[n-2] = Dx[n-3];
My[n-2] = Dy[n-3];
for(int i = n-4; i >= 0; i--)
{
Mx[i+1] = Dx[i] - C[i]*Mx[i+2];
My[i+1] = Dy[i] - C[i]*My[i+2];
}
Mx[0] = 0;
Mx[n-1] = 0;
My[0] = 0;
My[n-1] = 0;
for(int i = 0; i < n-1; i++)
{
func_v[i].a_x = point_v[i].x;
func_v[i].b_x = (point_v[i+1].x - point_v[i].x)/h[i] - (2*h[i]*Mx[i] + h[i]*Mx[i+1]) / 6;
func_v[i].c_x = Mx[i]/2;
func_v[i].d_x = (Mx[i+1] - Mx[i]) / (6*h[i]);
func_v[i].a_y = point_v[i].y;
func_v[i].b_y = (point_v[i+1].y - point_v[i].y)/h[i] - (2*h[i]*My[i] + h[i]*My[i+1]) / 6;
func_v[i].c_y = My[i]/2;
func_v[i].d_y = (My[i+1] - My[i]) / (6*h[i]);
func_v[i].h = h[i];
}
return func_v;
}

244
algorithms/lane_ufld/code/UFLD/evaluation/eval_wrapper.py Executable file
View File

@@ -0,0 +1,244 @@
from data.dataloader import get_test_loader
from evaluation.tusimple.lane import LaneEval
from utils.dist_utils import is_main_process, dist_print, get_rank, get_world_size, dist_tqdm, synchronize
import os, json, torch, scipy
import numpy as np
import platform
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
def generate_lines(out, shape, names, output_path, griding_num, localization_type='abs', flip_updown=False):
col_sample = np.linspace(0, shape[1] - 1, griding_num)
col_sample_w = col_sample[1] - col_sample[0]
for j in range(out.shape[0]):
out_j = out[j].data.cpu().numpy()
if flip_updown:
out_j = out_j[:, ::-1, :]
if localization_type == 'abs':
out_j = np.argmax(out_j, axis=0)
out_j[out_j == griding_num] = -1
out_j = out_j + 1
elif localization_type == 'rel':
prob = scipy.special.softmax(out_j[:-1, :, :], axis=0)
idx = np.arange(griding_num) + 1
idx = idx.reshape(-1, 1, 1)
loc = np.sum(prob * idx, axis=0)
out_j = np.argmax(out_j, axis=0)
loc[out_j == griding_num] = 0
out_j = loc
else:
raise NotImplementedError
name = names[j]
line_save_path = os.path.join(output_path, name[:-3] + 'lines.txt')
save_dir, _ = os.path.split(line_save_path)
if not os.path.exists(save_dir):
os.makedirs(save_dir)
with open(line_save_path, 'w') as fp:
for i in range(out_j.shape[1]):
if np.sum(out_j[:, i] != 0) > 2:
for k in range(out_j.shape[0]):
if out_j[k, i] > 0:
fp.write(
'%d %d ' % (int(out_j[k, i] * col_sample_w * 1640 / 800) - 1, int(590 - k * 20) - 1))
fp.write('\n')
def run_test(net, data_root, exp_name, work_dir, griding_num, use_aux, distributed, batch_size=8, test_list=None):
# torch.backends.cudnn.benchmark = True
output_path = os.path.join(work_dir, exp_name)
if not os.path.exists(output_path) and is_main_process():
os.mkdir(output_path)
synchronize()
loader = get_test_loader(batch_size, data_root, 'CULane', distributed, test_list=test_list)
# import pdb;pdb.set_trace()
for i, data in enumerate(dist_tqdm(loader)):
imgs, names = data
imgs = imgs.cuda()
with torch.no_grad():
out = net(imgs)
if len(out) == 2 and use_aux:
out, seg_out = out
generate_lines(out,imgs[0,0].shape,names,output_path,griding_num,localization_type = 'rel',flip_updown = True)
def generate_tusimple_lines(out,shape,griding_num,localization_type='rel'):
out = out.data.cpu().numpy()
out_loc = np.argmax(out,axis=0)
if localization_type == 'rel':
prob = scipy.special.softmax(out[:-1, :, :], axis=0)
idx = np.arange(griding_num)
idx = idx.reshape(-1, 1, 1)
loc = np.sum(prob * idx, axis=0)
loc[out_loc == griding_num] = griding_num
out_loc = loc
lanes = []
for i in range(out_loc.shape[1]):
out_i = out_loc[:,i]
lane = [int(round((loc + 0.5) * 1280.0 / (griding_num - 1))) if loc != griding_num else -2 for loc in out_i]
lanes.append(lane)
return lanes
def run_test_tusimple(net, data_root, work_dir, exp_name, griding_num, use_aux, distributed, batch_size=1, test_list=None):
output_path = os.path.join(work_dir,exp_name+'.%d.txt'% get_rank())
fp = open(output_path,'w')
loader = get_test_loader(batch_size, data_root, 'Tusimple', distributed, test_list=test_list)
for i,data in enumerate(dist_tqdm(loader)):
imgs,names = data
# imgs = imgs.cuda()
imgs = imgs.to(device)
with torch.no_grad():
out = net(imgs)
if len(out) == 2 and use_aux:
out = out[0]
for i,name in enumerate(names):
tmp_dict = {}
tmp_dict['lanes'] = generate_tusimple_lines(out[i],imgs[0,0].shape,griding_num)
tmp_dict['h_samples'] = [160, 170, 180, 190, 200, 210, 220, 230, 240, 250, 260,
270, 280, 290, 300, 310, 320, 330, 340, 350, 360, 370, 380, 390, 400, 410, 420,
430, 440, 450, 460, 470, 480, 490, 500, 510, 520, 530, 540, 550, 560, 570, 580,
590, 600, 610, 620, 630, 640, 650, 660, 670, 680, 690, 700, 710]
tmp_dict['raw_file'] = name
tmp_dict['run_time'] = 10
json_str = json.dumps(tmp_dict)
fp.write(json_str+'\n')
fp.close()
def combine_tusimple_test(work_dir,exp_name):
size = get_world_size()
all_res = []
for i in range(size):
output_path = os.path.join(work_dir,exp_name+'.%d.txt'% i)
with open(output_path, 'r') as fp:
res = fp.readlines()
all_res.extend(res)
names = set()
all_res_no_dup = []
for i, res in enumerate(all_res):
pos = res.find('clips')
name = res[pos:].split('\"')[0]
if name not in names:
names.add(name)
all_res_no_dup.append(res)
output_path = os.path.join(work_dir,exp_name+'.txt')
with open(output_path, 'w') as fp:
fp.writelines(all_res_no_dup)
def eval_lane(net, dataset, data_root, work_dir, griding_num, use_aux, distributed, test_list=None, skip_eval=False):
net.eval()
if dataset == 'CULane':
run_test(net, data_root, 'culane_eval_tmp', work_dir, griding_num, use_aux, distributed, test_list=test_list)
synchronize() # wait for all results
if is_main_process():
res = call_culane_eval(data_root, 'culane_eval_tmp', work_dir)
TP,FP,FN = 0,0,0
for k, v in res.items():
val = float(v['Fmeasure']) if 'nan' not in v['Fmeasure'] else 0
val_tp,val_fp,val_fn = int(v['tp']),int(v['fp']),int(v['fn'])
TP += val_tp
FP += val_fp
FN += val_fn
dist_print(k,val)
P = TP * 1.0/(TP + FP)
R = TP * 1.0/(TP + FN)
F = 2*P*R/(P + R)
dist_print(F)
synchronize()
elif dataset == 'Tusimple':
exp_name = 'tusimple_eval_tmp'
run_test_tusimple(net, data_root, work_dir, exp_name, griding_num, use_aux, distributed, test_list=test_list)
synchronize() # wait for all results
if is_main_process():
pred_path = os.path.join(work_dir, exp_name + '.0.txt')
label_json = os.path.join(data_root, 'test_label.json')
if skip_eval or not os.path.isfile(label_json):
dist_print('skip TuSimple metrics (no test_label.json); predictions:', pred_path)
else:
res = LaneEval.bench_one_submit(pred_path, label_json)
res = json.loads(res)
for r in res:
dist_print(r['name'], r['value'])
synchronize()
def read_helper(path):
lines = open(path, 'r').readlines()[1:]
lines = ' '.join(lines)
values = lines.split(' ')[1::2]
keys = lines.split(' ')[0::2]
keys = [key[:-1] for key in keys]
res = {k : v for k,v in zip(keys,values)}
return res
def call_culane_eval(data_dir, exp_name,output_path):
if data_dir[-1] != '/':
data_dir = data_dir + '/'
detect_dir=os.path.join(output_path,exp_name)+'/'
w_lane=30
iou=0.5; # Set iou to 0.3 or 0.5
im_w=1640
im_h=590
frame=1
list0 = os.path.join(data_dir,'list/test_split/test0_normal.txt')
list1 = os.path.join(data_dir,'list/test_split/test1_crowd.txt')
list2 = os.path.join(data_dir,'list/test_split/test2_hlight.txt')
list3 = os.path.join(data_dir,'list/test_split/test3_shadow.txt')
list4 = os.path.join(data_dir,'list/test_split/test4_noline.txt')
list5 = os.path.join(data_dir,'list/test_split/test5_arrow.txt')
list6 = os.path.join(data_dir,'list/test_split/test6_curve.txt')
list7 = os.path.join(data_dir,'list/test_split/test7_cross.txt')
list8 = os.path.join(data_dir,'list/test_split/test8_night.txt')
if not os.path.exists(os.path.join(output_path,'txt')):
os.mkdir(os.path.join(output_path,'txt'))
out0 = os.path.join(output_path,'txt','out0_normal.txt')
out1=os.path.join(output_path,'txt','out1_crowd.txt')
out2=os.path.join(output_path,'txt','out2_hlight.txt')
out3=os.path.join(output_path,'txt','out3_shadow.txt')
out4=os.path.join(output_path,'txt','out4_noline.txt')
out5=os.path.join(output_path,'txt','out5_arrow.txt')
out6=os.path.join(output_path,'txt','out6_curve.txt')
out7=os.path.join(output_path,'txt','out7_cross.txt')
out8=os.path.join(output_path,'txt','out8_night.txt')
eval_cmd = './evaluation/culane/evaluate'
if platform.system() == 'Windows':
eval_cmd = eval_cmd.replace('/', os.sep)
# print('./evaluate -a %s -d %s -i %s -l %s -w %s -t %s -c %s -r %s -f %s -o %s'%(data_dir,detect_dir,data_dir,list0,w_lane,iou,im_w,im_h,frame,out0))
os.system('%s -a %s -d %s -i %s -l %s -w %s -t %s -c %s -r %s -f %s -o %s'%(eval_cmd,data_dir,detect_dir,data_dir,list0,w_lane,iou,im_w,im_h,frame,out0))
# print('./evaluate -a %s -d %s -i %s -l %s -w %s -t %s -c %s -r %s -f %s -o %s'%(data_dir,detect_dir,data_dir,list1,w_lane,iou,im_w,im_h,frame,out1))
os.system('%s -a %s -d %s -i %s -l %s -w %s -t %s -c %s -r %s -f %s -o %s'%(eval_cmd,data_dir,detect_dir,data_dir,list1,w_lane,iou,im_w,im_h,frame,out1))
# print('./evaluate -a %s -d %s -i %s -l %s -w %s -t %s -c %s -r %s -f %s -o %s'%(data_dir,detect_dir,data_dir,list2,w_lane,iou,im_w,im_h,frame,out2))
os.system('%s -a %s -d %s -i %s -l %s -w %s -t %s -c %s -r %s -f %s -o %s'%(eval_cmd,data_dir,detect_dir,data_dir,list2,w_lane,iou,im_w,im_h,frame,out2))
# print('./evaluate -a %s -d %s -i %s -l %s -w %s -t %s -c %s -r %s -f %s -o %s'%(data_dir,detect_dir,data_dir,list3,w_lane,iou,im_w,im_h,frame,out3))
os.system('%s -a %s -d %s -i %s -l %s -w %s -t %s -c %s -r %s -f %s -o %s'%(eval_cmd,data_dir,detect_dir,data_dir,list3,w_lane,iou,im_w,im_h,frame,out3))
# print('./evaluate -a %s -d %s -i %s -l %s -w %s -t %s -c %s -r %s -f %s -o %s'%(data_dir,detect_dir,data_dir,list4,w_lane,iou,im_w,im_h,frame,out4))
os.system('%s -a %s -d %s -i %s -l %s -w %s -t %s -c %s -r %s -f %s -o %s'%(eval_cmd,data_dir,detect_dir,data_dir,list4,w_lane,iou,im_w,im_h,frame,out4))
# print('./evaluate -a %s -d %s -i %s -l %s -w %s -t %s -c %s -r %s -f %s -o %s'%(data_dir,detect_dir,data_dir,list5,w_lane,iou,im_w,im_h,frame,out5))
os.system('%s -a %s -d %s -i %s -l %s -w %s -t %s -c %s -r %s -f %s -o %s'%(eval_cmd,data_dir,detect_dir,data_dir,list5,w_lane,iou,im_w,im_h,frame,out5))
# print('./evaluate -a %s -d %s -i %s -l %s -w %s -t %s -c %s -r %s -f %s -o %s'%(data_dir,detect_dir,data_dir,list6,w_lane,iou,im_w,im_h,frame,out6))
os.system('%s -a %s -d %s -i %s -l %s -w %s -t %s -c %s -r %s -f %s -o %s'%(eval_cmd,data_dir,detect_dir,data_dir,list6,w_lane,iou,im_w,im_h,frame,out6))
# print('./evaluate -a %s -d %s -i %s -l %s -w %s -t %s -c %s -r %s -f %s -o %s'%(data_dir,detect_dir,data_dir,list7,w_lane,iou,im_w,im_h,frame,out7))
os.system('%s -a %s -d %s -i %s -l %s -w %s -t %s -c %s -r %s -f %s -o %s'%(eval_cmd,data_dir,detect_dir,data_dir,list7,w_lane,iou,im_w,im_h,frame,out7))
# print('./evaluate -a %s -d %s -i %s -l %s -w %s -t %s -c %s -r %s -f %s -o %s'%(data_dir,detect_dir,data_dir,list8,w_lane,iou,im_w,im_h,frame,out8))
os.system('%s -a %s -d %s -i %s -l %s -w %s -t %s -c %s -r %s -f %s -o %s'%(eval_cmd,data_dir,detect_dir,data_dir,list8,w_lane,iou,im_w,im_h,frame,out8))
res_all = {}
res_all['res_normal'] = read_helper(out0)
res_all['res_crowd']= read_helper(out1)
res_all['res_night']= read_helper(out8)
res_all['res_noline'] = read_helper(out4)
res_all['res_shadow'] = read_helper(out3)
res_all['res_arrow']= read_helper(out5)
res_all['res_hlight'] = read_helper(out2)
res_all['res_curve']= read_helper(out6)
res_all['res_cross']= read_helper(out7)
return res_all

238
algorithms/lane_ufld/code/UFLD/evaluation/eval_wrapper0.py Executable file
View File

@@ -0,0 +1,238 @@
from data.dataloader import get_test_loader
from evaluation.tusimple.lane import LaneEval
from utils.dist_utils import is_main_process, dist_print, get_rank, get_world_size, dist_tqdm, synchronize
import os, json, torch, scipy
import numpy as np
import platform
def generate_lines(out, shape, names, output_path, griding_num, localization_type='abs', flip_updown=False):
col_sample = np.linspace(0, shape[1] - 1, griding_num)
col_sample_w = col_sample[1] - col_sample[0]
for j in range(out.shape[0]):
out_j = out[j].data.cpu().numpy()
if flip_updown:
out_j = out_j[:, ::-1, :]
if localization_type == 'abs':
out_j = np.argmax(out_j, axis=0)
out_j[out_j == griding_num] = -1
out_j = out_j + 1
elif localization_type == 'rel':
prob = scipy.special.softmax(out_j[:-1, :, :], axis=0)
idx = np.arange(griding_num) + 1
idx = idx.reshape(-1, 1, 1)
loc = np.sum(prob * idx, axis=0)
out_j = np.argmax(out_j, axis=0)
loc[out_j == griding_num] = 0
out_j = loc
else:
raise NotImplementedError
name = names[j]
line_save_path = os.path.join(output_path, name[:-3] + 'lines.txt')
save_dir, _ = os.path.split(line_save_path)
if not os.path.exists(save_dir):
os.makedirs(save_dir)
with open(line_save_path, 'w') as fp:
for i in range(out_j.shape[1]):
if np.sum(out_j[:, i] != 0) > 2:
for k in range(out_j.shape[0]):
if out_j[k, i] > 0:
fp.write(
'%d %d ' % (int(out_j[k, i] * col_sample_w * 1640 / 800) - 1, int(590 - k * 20) - 1))
fp.write('\n')
def run_test(net, data_root, exp_name, work_dir, griding_num, use_aux,distributed, batch_size=8):
# torch.backends.cudnn.benchmark = True
output_path = os.path.join(work_dir, exp_name)
if not os.path.exists(output_path) and is_main_process():
os.mkdir(output_path)
synchronize()
loader = get_test_loader(batch_size, data_root, 'CULane', distributed)
# import pdb;pdb.set_trace()
for i, data in enumerate(dist_tqdm(loader)):
imgs, names = data
imgs = imgs.cuda()
with torch.no_grad():
out = net(imgs)
if len(out) == 2 and use_aux:
out, seg_out = out
generate_lines(out,imgs[0,0].shape,names,output_path,griding_num,localization_type = 'rel',flip_updown = True)
def generate_tusimple_lines(out,shape,griding_num,localization_type='rel'):
out = out.data.cpu().numpy()
out_loc = np.argmax(out,axis=0)
if localization_type == 'rel':
prob = scipy.special.softmax(out[:-1, :, :], axis=0)
idx = np.arange(griding_num)
idx = idx.reshape(-1, 1, 1)
loc = np.sum(prob * idx, axis=0)
loc[out_loc == griding_num] = griding_num
out_loc = loc
lanes = []
for i in range(out_loc.shape[1]):
out_i = out_loc[:,i]
lane = [int(round((loc + 0.5) * 1280.0 / (griding_num - 1))) if loc != griding_num else -2 for loc in out_i]
lanes.append(lane)
return lanes
def run_test_tusimple(net,data_root,work_dir,exp_name,griding_num,use_aux, distributed,batch_size = 8):
output_path = os.path.join(work_dir,exp_name+'.%d.txt'% get_rank())
fp = open(output_path,'w')
loader = get_test_loader(batch_size,data_root,'Tusimple', distributed)
for i,data in enumerate(dist_tqdm(loader)):
imgs,names = data
imgs = imgs.cuda()
with torch.no_grad():
out = net(imgs)
if len(out) == 2 and use_aux:
out = out[0]
for i,name in enumerate(names):
tmp_dict = {}
tmp_dict['lanes'] = generate_tusimple_lines(out[i],imgs[0,0].shape,griding_num)
tmp_dict['h_samples'] = [160, 170, 180, 190, 200, 210, 220, 230, 240, 250, 260,
270, 280, 290, 300, 310, 320, 330, 340, 350, 360, 370, 380, 390, 400, 410, 420,
430, 440, 450, 460, 470, 480, 490, 500, 510, 520, 530, 540, 550, 560, 570, 580,
590, 600, 610, 620, 630, 640, 650, 660, 670, 680, 690, 700, 710]
tmp_dict['raw_file'] = name
tmp_dict['run_time'] = 10
json_str = json.dumps(tmp_dict)
fp.write(json_str+'\n')
fp.close()
def combine_tusimple_test(work_dir,exp_name):
size = get_world_size()
all_res = []
for i in range(size):
output_path = os.path.join(work_dir,exp_name+'.%d.txt'% i)
with open(output_path, 'r') as fp:
res = fp.readlines()
all_res.extend(res)
names = set()
all_res_no_dup = []
for i, res in enumerate(all_res):
pos = res.find('clips')
name = res[pos:].split('\"')[0]
if name not in names:
names.add(name)
all_res_no_dup.append(res)
output_path = os.path.join(work_dir,exp_name+'.txt')
with open(output_path, 'w') as fp:
fp.writelines(all_res_no_dup)
def eval_lane(net, dataset, data_root, work_dir, griding_num, use_aux, distributed):
net.eval()
if dataset == 'CULane':
run_test(net,data_root, 'culane_eval_tmp', work_dir, griding_num, use_aux, distributed)
synchronize() # wait for all results
if is_main_process():
res = call_culane_eval(data_root, 'culane_eval_tmp', work_dir)
TP,FP,FN = 0,0,0
for k, v in res.items():
val = float(v['Fmeasure']) if 'nan' not in v['Fmeasure'] else 0
val_tp,val_fp,val_fn = int(v['tp']),int(v['fp']),int(v['fn'])
TP += val_tp
FP += val_fp
FN += val_fn
dist_print(k,val)
P = TP * 1.0/(TP + FP)
R = TP * 1.0/(TP + FN)
F = 2*P*R/(P + R)
dist_print(F)
synchronize()
elif dataset == 'Tusimple':
exp_name = 'tusimple_eval_tmp'
run_test_tusimple(net, data_root, work_dir, exp_name, griding_num, use_aux, distributed)
synchronize() # wait for all results
if is_main_process():
combine_tusimple_test(work_dir,exp_name)
res = LaneEval.bench_one_submit(os.path.join(work_dir,exp_name + '.txt'),os.path.join(data_root,'test_label.json'))
res = json.loads(res)
for r in res:
dist_print(r['name'], r['value'])
synchronize()
def read_helper(path):
lines = open(path, 'r').readlines()[1:]
lines = ' '.join(lines)
values = lines.split(' ')[1::2]
keys = lines.split(' ')[0::2]
keys = [key[:-1] for key in keys]
res = {k : v for k,v in zip(keys,values)}
return res
def call_culane_eval(data_dir, exp_name,output_path):
if data_dir[-1] != '/':
data_dir = data_dir + '/'
detect_dir=os.path.join(output_path,exp_name)+'/'
w_lane=30
iou=0.5; # Set iou to 0.3 or 0.5
im_w=1640
im_h=590
frame=1
list0 = os.path.join(data_dir,'list/test_split/test0_normal.txt')
list1 = os.path.join(data_dir,'list/test_split/test1_crowd.txt')
list2 = os.path.join(data_dir,'list/test_split/test2_hlight.txt')
list3 = os.path.join(data_dir,'list/test_split/test3_shadow.txt')
list4 = os.path.join(data_dir,'list/test_split/test4_noline.txt')
list5 = os.path.join(data_dir,'list/test_split/test5_arrow.txt')
list6 = os.path.join(data_dir,'list/test_split/test6_curve.txt')
list7 = os.path.join(data_dir,'list/test_split/test7_cross.txt')
list8 = os.path.join(data_dir,'list/test_split/test8_night.txt')
if not os.path.exists(os.path.join(output_path,'txt')):
os.mkdir(os.path.join(output_path,'txt'))
out0 = os.path.join(output_path,'txt','out0_normal.txt')
out1=os.path.join(output_path,'txt','out1_crowd.txt')
out2=os.path.join(output_path,'txt','out2_hlight.txt')
out3=os.path.join(output_path,'txt','out3_shadow.txt')
out4=os.path.join(output_path,'txt','out4_noline.txt')
out5=os.path.join(output_path,'txt','out5_arrow.txt')
out6=os.path.join(output_path,'txt','out6_curve.txt')
out7=os.path.join(output_path,'txt','out7_cross.txt')
out8=os.path.join(output_path,'txt','out8_night.txt')
eval_cmd = './evaluation/culane/evaluate'
if platform.system() == 'Windows':
eval_cmd = eval_cmd.replace('/', os.sep)
# print('./evaluate -a %s -d %s -i %s -l %s -w %s -t %s -c %s -r %s -f %s -o %s'%(data_dir,detect_dir,data_dir,list0,w_lane,iou,im_w,im_h,frame,out0))
os.system('%s -a %s -d %s -i %s -l %s -w %s -t %s -c %s -r %s -f %s -o %s'%(eval_cmd,data_dir,detect_dir,data_dir,list0,w_lane,iou,im_w,im_h,frame,out0))
# print('./evaluate -a %s -d %s -i %s -l %s -w %s -t %s -c %s -r %s -f %s -o %s'%(data_dir,detect_dir,data_dir,list1,w_lane,iou,im_w,im_h,frame,out1))
os.system('%s -a %s -d %s -i %s -l %s -w %s -t %s -c %s -r %s -f %s -o %s'%(eval_cmd,data_dir,detect_dir,data_dir,list1,w_lane,iou,im_w,im_h,frame,out1))
# print('./evaluate -a %s -d %s -i %s -l %s -w %s -t %s -c %s -r %s -f %s -o %s'%(data_dir,detect_dir,data_dir,list2,w_lane,iou,im_w,im_h,frame,out2))
os.system('%s -a %s -d %s -i %s -l %s -w %s -t %s -c %s -r %s -f %s -o %s'%(eval_cmd,data_dir,detect_dir,data_dir,list2,w_lane,iou,im_w,im_h,frame,out2))
# print('./evaluate -a %s -d %s -i %s -l %s -w %s -t %s -c %s -r %s -f %s -o %s'%(data_dir,detect_dir,data_dir,list3,w_lane,iou,im_w,im_h,frame,out3))
os.system('%s -a %s -d %s -i %s -l %s -w %s -t %s -c %s -r %s -f %s -o %s'%(eval_cmd,data_dir,detect_dir,data_dir,list3,w_lane,iou,im_w,im_h,frame,out3))
# print('./evaluate -a %s -d %s -i %s -l %s -w %s -t %s -c %s -r %s -f %s -o %s'%(data_dir,detect_dir,data_dir,list4,w_lane,iou,im_w,im_h,frame,out4))
os.system('%s -a %s -d %s -i %s -l %s -w %s -t %s -c %s -r %s -f %s -o %s'%(eval_cmd,data_dir,detect_dir,data_dir,list4,w_lane,iou,im_w,im_h,frame,out4))
# print('./evaluate -a %s -d %s -i %s -l %s -w %s -t %s -c %s -r %s -f %s -o %s'%(data_dir,detect_dir,data_dir,list5,w_lane,iou,im_w,im_h,frame,out5))
os.system('%s -a %s -d %s -i %s -l %s -w %s -t %s -c %s -r %s -f %s -o %s'%(eval_cmd,data_dir,detect_dir,data_dir,list5,w_lane,iou,im_w,im_h,frame,out5))
# print('./evaluate -a %s -d %s -i %s -l %s -w %s -t %s -c %s -r %s -f %s -o %s'%(data_dir,detect_dir,data_dir,list6,w_lane,iou,im_w,im_h,frame,out6))
os.system('%s -a %s -d %s -i %s -l %s -w %s -t %s -c %s -r %s -f %s -o %s'%(eval_cmd,data_dir,detect_dir,data_dir,list6,w_lane,iou,im_w,im_h,frame,out6))
# print('./evaluate -a %s -d %s -i %s -l %s -w %s -t %s -c %s -r %s -f %s -o %s'%(data_dir,detect_dir,data_dir,list7,w_lane,iou,im_w,im_h,frame,out7))
os.system('%s -a %s -d %s -i %s -l %s -w %s -t %s -c %s -r %s -f %s -o %s'%(eval_cmd,data_dir,detect_dir,data_dir,list7,w_lane,iou,im_w,im_h,frame,out7))
# print('./evaluate -a %s -d %s -i %s -l %s -w %s -t %s -c %s -r %s -f %s -o %s'%(data_dir,detect_dir,data_dir,list8,w_lane,iou,im_w,im_h,frame,out8))
os.system('%s -a %s -d %s -i %s -l %s -w %s -t %s -c %s -r %s -f %s -o %s'%(eval_cmd,data_dir,detect_dir,data_dir,list8,w_lane,iou,im_w,im_h,frame,out8))
res_all = {}
res_all['res_normal'] = read_helper(out0)
res_all['res_crowd']= read_helper(out1)
res_all['res_night']= read_helper(out8)
res_all['res_noline'] = read_helper(out4)
res_all['res_shadow'] = read_helper(out3)
res_all['res_arrow']= read_helper(out5)
res_all['res_hlight'] = read_helper(out2)
res_all['res_curve']= read_helper(out6)
res_all['res_cross']= read_helper(out7)
return res_all

111
algorithms/lane_ufld/code/UFLD/evaluation/tusimple/lane.py Executable file
View File

@@ -0,0 +1,111 @@
import os
import numpy as np
from sklearn.linear_model import LinearRegression
import json, shutil
class LaneEval(object):
lr = LinearRegression()
pixel_thresh = 20
pt_thresh = 0.85
@staticmethod
def get_angle(xs, y_samples):
xs, ys = xs[xs >= 0], y_samples[xs >= 0]
if len(xs) > 1:
LaneEval.lr.fit(ys[:, None], xs)
k = LaneEval.lr.coef_[0]
theta = np.arctan(k)
else:
theta = 0
return theta
@staticmethod
def line_accuracy(pred, gt, thresh):
pred = np.array([p if p >= 0 else -100 for p in pred])
gt = np.array([g if g >= 0 else -100 for g in gt])
return np.sum(np.where(np.abs(pred - gt) < thresh, 1., 0.)) / len(gt)
@staticmethod
def bench(pred, gt, y_samples, running_time):
if any(len(p) != len(y_samples) for p in pred):
raise Exception('Format of lanes error.')
if running_time > 200 or len(gt) + 2 < len(pred):
return 0., 0., 1.
angles = [LaneEval.get_angle(np.array(x_gts), np.array(y_samples)) for x_gts in gt]
threshs = [LaneEval.pixel_thresh / np.cos(angle) for angle in angles]
line_accs = []
fp, fn = 0., 0.
matched = 0.
for x_gts, thresh in zip(gt, threshs):
accs = [LaneEval.line_accuracy(np.array(x_preds), np.array(x_gts), thresh) for x_preds in pred]
max_acc = np.max(accs) if len(accs) > 0 else 0.
if max_acc < LaneEval.pt_thresh:
fn += 1
else:
matched += 1
line_accs.append(max_acc)
fp = len(pred) - matched
if len(gt) > 4 and fn > 0:
fn -= 1
s = sum(line_accs)
if len(gt) > 4:
s -= min(line_accs)
# print('s:', s, len(gt))
return s / max(min(4.0, len(gt)), 1.), fp / len(pred) if len(pred) > 0 else 0., fn / max(min(len(gt), 4.), 1.)
@staticmethod
def bench_one_submit(pred_file, gt_file):
print(pred_file,"//////////////////",gt_file)
try:
json_pred = [json.loads(line) for line in open(pred_file).readlines()]
except BaseException as e:
raise Exception('Fail to load json file of the prediction.')
json_gt = [json.loads(line) for line in open(gt_file).readlines()]
print("****************************************",len(json_gt), len(json_pred))
if len(json_gt) != len(json_pred):
# print(len(json_gt), json_pred)
raise Exception('We do not get the predictions of all the test tasks')
gts = {l['raw_file']: l for l in json_gt}
accuracy, fp, fn = 0., 0., 0.
for pred in json_pred:
if 'raw_file' not in pred or 'lanes' not in pred or 'run_time' not in pred:
raise Exception('raw_file or lanes or run_time not in some predictions.')
raw_file = pred['raw_file']
pred_lanes = pred['lanes']
run_time = pred['run_time']
if raw_file not in gts:
raise Exception('Some raw_file from your predictions do not exist in the test tasks.')
gt = gts[raw_file]
gt_lanes = gt['lanes']
y_samples = gt['h_samples']
try:
a, p, n = LaneEval.bench(pred_lanes, gt_lanes, y_samples, run_time)
except BaseException as e:
raise Exception('Format of lanes error.')
# if a <= 0.85:
print(raw_file, 'a: ', a)
# srcfile = "C:/data/Tusimple/test_set/" + raw_file
#fpath, fname = os.path.split(srcfile)
#shutil.copy(srcfile, './out/' + fname)
accuracy += a
fp += p
fn += n
num = len(gts)
# the first return parameter is the default ranking parameter
return json.dumps([
{'name': 'Accuracy', 'value': accuracy / num, 'order': 'desc'},
{'name': 'FP', 'value': fp / num, 'order': 'asc'},
{'name': 'FN', 'value': fn / num, 'order': 'asc'}
])
if __name__ == '__main__':
import sys
try:
if len(sys.argv) != 3:
raise Exception('Invalid input arguments')
print(LaneEval.bench_one_submit(sys.argv[1], sys.argv[2]))
except Exception as e:
print(e.message)
sys.exit(e.message)

102
algorithms/lane_ufld/code/UFLD/evaluation/tusimple/lane0.py Executable file
View File

@@ -0,0 +1,102 @@
import numpy as np
from sklearn.linear_model import LinearRegression
import json
class LaneEval(object):
lr = LinearRegression()
pixel_thresh = 20
pt_thresh = 0.85
@staticmethod
def get_angle(xs, y_samples):
xs, ys = xs[xs >= 0], y_samples[xs >= 0]
if len(xs) > 1:
LaneEval.lr.fit(ys[:, None], xs)
k = LaneEval.lr.coef_[0]
theta = np.arctan(k)
else:
theta = 0
return theta
@staticmethod
def line_accuracy(pred, gt, thresh):
pred = np.array([p if p >= 0 else -100 for p in pred])
gt = np.array([g if g >= 0 else -100 for g in gt])
return np.sum(np.where(np.abs(pred - gt) < thresh, 1., 0.)) / len(gt)
@staticmethod
def bench(pred, gt, y_samples, running_time):
if any(len(p) != len(y_samples) for p in pred):
raise Exception('Format of lanes error.')
if running_time > 200 or len(gt) + 2 < len(pred):
return 0., 0., 1.
angles = [LaneEval.get_angle(np.array(x_gts), np.array(y_samples)) for x_gts in gt]
threshs = [LaneEval.pixel_thresh / np.cos(angle) for angle in angles]
line_accs = []
fp, fn = 0., 0.
matched = 0.
for x_gts, thresh in zip(gt, threshs):
accs = [LaneEval.line_accuracy(np.array(x_preds), np.array(x_gts), thresh) for x_preds in pred]
max_acc = np.max(accs) if len(accs) > 0 else 0.
if max_acc < LaneEval.pt_thresh:
fn += 1
else:
matched += 1
line_accs.append(max_acc)
fp = len(pred) - matched
if len(gt) > 4 and fn > 0:
fn -= 1
s = sum(line_accs)
if len(gt) > 4:
s -= min(line_accs)
return s / max(min(4.0, len(gt)), 1.), fp / len(pred) if len(pred) > 0 else 0., fn / max(min(len(gt), 4.) , 1.)
@staticmethod
def bench_one_submit(pred_file, gt_file):
try:
json_pred = [json.loads(line) for line in open(pred_file).readlines()]
except BaseException as e:
raise Exception('Fail to load json file of the prediction.')
json_gt = [json.loads(line) for line in open(gt_file).readlines()]
print(len(json_gt), len(json_pred))
if len(json_gt) != len(json_pred):
raise Exception('We do not get the predictions of all the test tasks')
gts = {l['raw_file']: l for l in json_gt}
accuracy, fp, fn = 0., 0., 0.
for pred in json_pred:
if 'raw_file' not in pred or 'lanes' not in pred or 'run_time' not in pred:
raise Exception('raw_file or lanes or run_time not in some predictions.')
raw_file = pred['raw_file']
pred_lanes = pred['lanes']
run_time = pred['run_time']
if raw_file not in gts:
raise Exception('Some raw_file from your predictions do not exist in the test tasks.')
gt = gts[raw_file]
gt_lanes = gt['lanes']
y_samples = gt['h_samples']
try:
a, p, n = LaneEval.bench(pred_lanes, gt_lanes, y_samples, run_time)
except BaseException as e:
raise Exception('Format of lanes error.')
accuracy += a
fp += p
fn += n
num = len(gts)
# the first return parameter is the default ranking parameter
return json.dumps([
{'name': 'Accuracy', 'value': accuracy / num, 'order': 'desc'},
{'name': 'FP', 'value': fp / num, 'order': 'asc'},
{'name': 'FN', 'value': fn / num, 'order': 'asc'}
])
if __name__ == '__main__':
import sys
try:
if len(sys.argv) != 3:
raise Exception('Invalid input arguments')
print(LaneEval.bench_one_submit(sys.argv[1], sys.argv[2]))
except Exception as e:
print(e.message)
sys.exit(e.message)

47
algorithms/lane_ufld/code/UFLD/export.py Executable file
View File

@@ -0,0 +1,47 @@
import torch, os, cv2
from model.model import parsingNet
from utils.common import merge_config
from utils.dist_utils import dist_print
import torch
import scipy.special, tqdm
import numpy as np
import torchvision.transforms as transforms
from data.dataset import LaneTestDataset
from data.constant import culane_row_anchor, tusimple_row_anchor
from PIL import Image
# Export to TorchScript that can be used for LibTorch
torch.backends.cudnn.benchmark = True
# From cuLANE, Change this line if you are using TuSimple
cls_num_per_lane = 18
griding_num = 200
backbone =18
net = parsingNet(pretrained = False,backbone='18', cls_dim = (griding_num+1,cls_num_per_lane,4),
use_aux=False)
# Change test_model where your model stored.
test_model = '/data/Models/UltraFastLaneDetection/culane_18.pth'
#state_dict = torch.load(test_model, map_location='cpu')['model'] # CPU
state_dict = torch.load(test_model, map_location='cuda')['model'] # CUDA
compatible_state_dict = {}
for k, v in state_dict.items():
if 'module.' in k:
compatible_state_dict[k[7:]] = v
else:
compatible_state_dict[k] = v
net.load_state_dict(compatible_state_dict, strict=False)
net.eval()
# Test Input Image
img = torch.zeros(1, 3, 288, 800) # image size(1,3,320,192) iDetection
y = net(img) # dry run
ts = torch.jit.trace(net, img)
#ts.save('UFLD.torchscript-cpu.pt') # CPU
ts.save('UFLD.torchscript-cuda.pt') # CUDA

39
algorithms/lane_ufld/code/UFLD/get_frame.py Executable file
View File

@@ -0,0 +1,39 @@
import cv2
import os
from copy import deepcopy
video_full_path = r"C:\data\lane_videos\2022-08-08-14-33-37 Usb Cam Image Raw.mp4"
cap = cv2.VideoCapture(video_full_path)
print(cap.isOpened())
video_name = video_full_path.split('\\')[-1][0:-4]
print(video_name)
frame_count = 1
success = True
while(success):
success, frame = cap.read()
print('Read a new frame: ', success)
if success == False:
break
# params.append(cv.CV_IMWRITE_PXM_BINARY)
# params.append(1)#
# height, weight, banch = frame.shape
# print(weight, height, banch)
print(frame_count)
if frame_count % 10 == 0:
dim = (1280, 720)
resized_img = cv2.resize(frame, dim, interpolation=cv2.INTER_AREA)
save_path = os.path.join("C:\\data\\lane_videos\\frame\\", video_name)
if not os.path.exists(save_path):
os.makedirs(save_path)
cv2.imwrite(save_path + "\\" + video_name + "_%d.png" % frame_count, resized_img)
frame_line = deepcopy(resized_img)
# for i in range(16, 72):
# cv2.line(frame_line, (0, i * 10), (1279, i * 10), (255, 0, 0), 1)
# save_path_l = os.path.join("C:\\Users\\linf1\\Documents\\ZED\\img\\", video_name + '_l')
# if not os.path.exists(save_path_l):
# os.makedirs(save_path_l)
# cv2.imwrite(save_path_l + "\\" + video_name + "_%d.jpg" % frame_count, frame_line)
print('save the %d frame' % frame_count)
frame_count = frame_count + 1
cap.release()

25
algorithms/lane_ufld/code/UFLD/get_picname.py Executable file
View File

@@ -0,0 +1,25 @@
import os
import re
import cv2
rootdir = 'C:\\data\\Tusimple\\test_set\\51'
pattern_number_oeder = '(\d*?).jpg'
f1 = open('C:\\data\\Tusimple\\test_set\\51.txt', 'w')
for root, dirs, files in os.walk(rootdir):
for dir in dirs:
print(os.path.join(root, dir))
files.sort(key=lambda x:int(re.findall(pattern_number_oeder, x)[0]))
for file in files:
img_path = os.path.join(root, file)
# img = cv2.imread(img_path)
# dim = (1280, 720)
# resized_img = cv2.resize(img, dim, interpolation=cv2.INTER_AREA)
# cv2.imwrite(img_path, img)
# save_path = os.path.join(root.split('\\')[-1], file)
save_path = img_path
print(save_path)
# print(os.path.join(root, file))
f1.write(save_path)
f1.write('\n')
f1.close()

139
algorithms/lane_ufld/code/UFLD/lane_show.py Executable file
View File

@@ -0,0 +1,139 @@
iport cv2
import math
import cap
import numpy as np
def is_in_poly(p, poly):
"""
对点进行筛选，选出符合ROI特定区域内的点
:param p: 待判断的点坐标， [x, y]
:param poly: 多边形顶点，[[x1,y1], [x2,y2], [x3,y3], [x4,y4], ...]
return: is_in若为True，则说明点在ROI区域，保留，反之则删除。
"""
px, py = p[0], p[1]
is_in = False
for i, corner in enumerate(poly):
# len(poly) = 4 next_i=(0,1,2,3,0,1,2......)
next_i = i + 1 if i + 1 < len(poly) else 0
x1, y1 = corner
x2, y2 = poly[next_i]
if (x1 == px and y1 == py) or (x2 == px and y2 == py): # if point is on vertex
is_in = True
break
if min(y1, y2) < py <= max(y1, y2): # 判断y是否处于y1与y2之间
x = x1 + (py - y1) * (x2 - x1) / (y2 - y1)
if x == px: # if point is on edge
is_in = True
break
elif x > px: # if point is on left-side of line
is_in = True
return is_in
def handle_point(x, y):
"""
根据x的大小对 x,y 进行排序。再找到最大间隔，并据此把控制点分成两部分。
return: 返回的是左车道线的x,y坐标以及右车道线x,y的坐标
"""
lx = [] # 存储左车道线x坐标
ly = [] # 存储左车道线y坐标
rx = [] # 存储右车道线x坐标
ry = [] # 存储右车道线y坐标
points = zip(x, y)
# 从小到大排序
sorted_points = sorted(points)
x = [point[0] for point in sorted_points]
y = [point[1] for point in sorted_points]
# 分割
Max = 0
k = 0
# 找出x坐标最大间隔，分出左车道和右车道
for i in range(len(x) - 1):
# 计算欧几里得距离
d = np.int(math.hypot(x[i + 1] - x[i], y[i + 1] - y[i]))
if d > Max:
Max = d
k = i
for i in range(len(x)):
# 坐车道点
if i < k + 1:
lx.append(x[i])
ly.append(y[i])
# 右车道点
else:
rx.append(x[i])
ry.append(y[i])
return lx, ly, rx, ry
def poly_fitting(lx, ly, rx, ry):
"""
分别对两部分控制点进行二次多项式拟合
"""
lx = np.array(lx)
ly = np.array(ly)
rx = np.array(rx)
ry = np.array(ry)
fl = np.polyfit(lx, ly, 2) # 用2次多项式拟合
fr = np.polyfit(rx, ry, 2) # 用2次多项式拟合
ploty = np.linspace(0, 719, 720)
leftx = fl[0]*ploty**2 + fl[1]*ploty + fl[2]
rightx = fr[0]*ploty**2 + fr[1]*ploty + fr[2]
# 定义从像素空间到米的x和y转换
ym_per_pix = 30/720 # meters per pixel in y dimension
xm_per_pix = 3.7/700 # meters per pixel in x dimension
y_eval = np.max(ploty) # 719
# 将新多项式拟合到世界空间中的x，y
left_fit_cr = np.polyfit(ploty*ym_per_pix, leftx*xm_per_pix, 2)
right_fit_cr = np.polyfit(ploty*ym_per_pix, rightx*xm_per_pix, 2)
# 计算新的曲率半径
left_curverad = ((1 + (2*left_fit_cr[0]*y_eval*ym_per_pix + left_fit_cr[1])**2)**1.5) / np.absolute(2*left_fit_cr[0])
right_curverad = ((1 + (2*right_fit_cr[0]*y_eval*ym_per_pix + right_fit_cr[1])**2)**1.5) / np.absolute(2*right_fit_cr[0])
curvature = ((left_curverad + right_curverad) / 2) # 曲率
lane_width = np.absolute(leftx[719] - rightx[719])
lane_xm_per_pix = 3.7 / lane_width
# 车辆应该保持偏移的距离
veh_pos = (((leftx[719] + rightx[719]) * lane_xm_per_pix) / 2.)
# 当前车辆偏移的距离
cen_pos = ((1280 * lane_xm_per_pix) / 2.)
# cen_pos = ((cap.get(3) * lane_xm_per_pix) / 2.)
# 计算车辆偏移距离
distance_from_center = cen_pos - veh_pos
return curvature, distance_from_center
def draw_values(img,curvature,distance_from_center):
"""
将曲率和车道偏移距离里显示在图片上
"""
font = cv2.FONT_HERSHEY_SIMPLEX
radius_text = "Radius of Curvature: %sm"%(round(curvature))
if distance_from_center > 0:
pos_flag = 'right'
else:
pos_flag = 'left'
cv2.putText(img, radius_text, (100, 100), font, 1, (255, 255, 255), 2)
center_text = "Vehicle is %.3fm %s of center"%(abs(distance_from_center), pos_flag)
cv2.putText(img, center_text, (100, 150), font, 1, (255, 255, 255), 2)
return img
# if __name__ == "__main__":
# poly = [[0, 0], [0, 719], [1279, 0], [1279, 719]]
# lane_x = []
# lane_y = []
# is_in = is_in_poly(ppp, poly)
# if is_in == True:
# # 将处理后的点坐标添如一个空列表做拟合用
# lane_x.append(ppp[0])
# lane_y.append(ppp[1])
# cv2.circle(frame, ppp, 5, (0, 255, 0), -1)
#
# lx, ly, rx, ry = handle_point(lane_x, lane_y)
# curvature, distance_from_center = poly_fitting(lx, ly, rx, ry)
# draw_values(frame, curvature, distance_from_center)

7
algorithms/lane_ufld/code/UFLD/launch_training.sh Executable file
View File

@@ -0,0 +1,7 @@
export CUDA_VISIBLE_DEVICES=0,1,2,3
export NGPUS=4
export OMP_NUM_THREADS=1 # you can change this value according to your number of cpu cores
python -m torch.distributed.launch --nproc_per_node=$NGPUS train.py configs/culane.py
# python train.py configs/tusimple.py

98
algorithms/lane_ufld/code/UFLD/model/backbone.py Executable file
View File

@@ -0,0 +1,98 @@
import torch
import torch.nn.modules
import torchvision
from model.vovnet import VOVNET_ALIASES, STAGE_SPECS
RESNET_BACKBONES = ['18', '34', '50', '101', '152', '50next', '101next', '50wide', '101wide']
VOVMNET_BACKBONES = list(VOVNET_ALIASES.keys())
SUPPORTED_BACKBONES = RESNET_BACKBONES + VOVMNET_BACKBONES + ['vgg']
def is_vovnet(backbone):
return str(backbone) in VOVNET_ALIASES
def get_backbone_spec(backbone):
"""Channel layout for parsingNet pool / aux heads."""
backbone = str(backbone)
if is_vovnet(backbone):
from model.vovnet import VOVNET_ALIASES as _aliases
variant = _aliases[backbone]
ch = STAGE_SPECS[variant]['stage_out_ch']
return {
'pool_in': ch[3],
'aux': (ch[1], ch[2], ch[3]),
'family': 'vov',
}
if backbone in ['18', '34']:
return {'pool_in': 512, 'aux': (128, 256, 512), 'family': 'resnet'}
if backbone == 'vgg':
return {'pool_in': 512, 'aux': (128, 256, 512), 'family': 'vgg'}
return {'pool_in': 2048, 'aux': (512, 1024, 2048), 'family': 'resnet'}
def build_backbone(backbone, pretrained=False):
backbone = str(backbone)
if is_vovnet(backbone):
from model.vovnet import vovnet
return vovnet(backbone, pretrained=pretrained)
if backbone == 'vgg':
return vgg16bn(pretrained=pretrained)
return resnet(backbone, pretrained=pretrained)
class vgg16bn(torch.nn.Module):
def __init__(self, pretrained=False):
super(vgg16bn, self).__init__()
model = list(torchvision.models.vgg16_bn(pretrained=pretrained).features.children())
model = model[:33] + model[34:43]
self.model = torch.nn.Sequential(*model)
def forward(self, x):
return self.model(x)
class resnet(torch.nn.Module):
def __init__(self, layers, pretrained=False):
super(resnet, self).__init__()
if layers == '18':
model = torchvision.models.resnet18(pretrained=pretrained)
elif layers == '34':
model = torchvision.models.resnet34(pretrained=pretrained)
elif layers == '50':
model = torchvision.models.resnet50(pretrained=pretrained)
elif layers == '101':
model = torchvision.models.resnet101(pretrained=pretrained)
elif layers == '152':
model = torchvision.models.resnet152(pretrained=pretrained)
elif layers == '50next':
model = torchvision.models.resnext50_32x4d(pretrained=pretrained)
elif layers == '101next':
model = torchvision.models.resnext101_32x8d(pretrained=pretrained)
elif layers == '50wide':
model = torchvision.models.wide_resnet50_2(pretrained=pretrained)
elif layers == '101wide':
model = torchvision.models.wide_resnet101_2(pretrained=pretrained)
else:
raise NotImplementedError(f'Unknown ResNet backbone: {layers}')
self.conv1 = model.conv1
self.bn1 = model.bn1
self.relu = model.relu
self.maxpool = model.maxpool
self.layer1 = model.layer1
self.layer2 = model.layer2
self.layer3 = model.layer3
self.layer4 = model.layer4
def forward(self, x):
x = self.conv1(x)
x = self.bn1(x)
x = self.relu(x)
x = self.maxpool(x)
x = self.layer1(x)
x2 = self.layer2(x)
x3 = self.layer3(x2)
x4 = self.layer4(x3)
return x2, x3, x4

126
algorithms/lane_ufld/code/UFLD/model/model.py Executable file
View File

@@ -0,0 +1,126 @@
import torch
from model.backbone import build_backbone, get_backbone_spec
import numpy as np
class conv_bn_relu(torch.nn.Module):
def __init__(self,in_channels, out_channels, kernel_size, stride=1, padding=0, dilation=1,bias=False):
super(conv_bn_relu,self).__init__()
self.conv = torch.nn.Conv2d(in_channels,out_channels, kernel_size,
stride = stride, padding = padding, dilation = dilation,bias = bias)
self.bn = torch.nn.BatchNorm2d(out_channels)
self.relu = torch.nn.ReLU()
def forward(self,x):
x = self.conv(x)
x = self.bn(x)
x = self.relu(x)
return x
class parsingNet(torch.nn.Module):
def __init__(self, size=(288, 800), pretrained=True, backbone='50', cls_dim=(37, 10, 4), use_aux=False):
super(parsingNet, self).__init__()
self.size = size
self.w = size[0]
self.h = size[1]
self.cls_dim = cls_dim # (num_gridding, num_cls_per_lane, num_of_lanes)
# num_cls_per_lane is the number of row anchors
self.use_aux = use_aux
self.total_dim = np.prod(cls_dim)
# input : nchw,
# output: (w+1) * sample_rows * 4
spec = get_backbone_spec(backbone)
c2, c3, c4 = spec['aux']
pool_in = spec['pool_in']
self.model = build_backbone(backbone, pretrained=pretrained)
if self.use_aux:
self.aux_header2 = torch.nn.Sequential(
conv_bn_relu(c2, 128, kernel_size=3, stride=1, padding=1),
conv_bn_relu(128, 128, 3, padding=1),
conv_bn_relu(128, 128, 3, padding=1),
conv_bn_relu(128, 128, 3, padding=1),
)
self.aux_header3 = torch.nn.Sequential(
conv_bn_relu(c3, 128, kernel_size=3, stride=1, padding=1),
conv_bn_relu(128, 128, 3, padding=1),
conv_bn_relu(128, 128, 3, padding=1),
)
self.aux_header4 = torch.nn.Sequential(
conv_bn_relu(c4, 128, kernel_size=3, stride=1, padding=1),
conv_bn_relu(128, 128, 3, padding=1),
)
self.aux_combine = torch.nn.Sequential(
conv_bn_relu(384, 256, 3,padding=2,dilation=2),
conv_bn_relu(256, 128, 3,padding=2,dilation=2),
conv_bn_relu(128, 128, 3,padding=2,dilation=2),
conv_bn_relu(128, 128, 3,padding=4,dilation=4),
torch.nn.Conv2d(128, cls_dim[-1] + 1, 1)
# output : n, num_of_lanes+1, h, w
)
initialize_weights(self.aux_header2,self.aux_header3,self.aux_header4,self.aux_combine)
self.cls = torch.nn.Sequential(
torch.nn.Linear(1800, 2048),
torch.nn.ReLU(),
torch.nn.Linear(2048, self.total_dim),
)
self.pool = torch.nn.Conv2d(pool_in, 8, 1)
# 1/32,2048 channel
# 288,800 -> 9,40,2048
# (w+1) * sample_rows * 4
# 37 * 10 * 4
initialize_weights(self.cls)
def forward(self, x):
# n c h w - > n 2048 sh sw
# -> n 2048
x2,x3,fea = self.model(x)
if self.use_aux:
x2 = self.aux_header2(x2)
x3 = self.aux_header3(x3)
x3 = torch.nn.functional.interpolate(x3,scale_factor = 2,mode='bilinear')
x4 = self.aux_header4(fea)
x4 = torch.nn.functional.interpolate(x4,scale_factor = 4,mode='bilinear')
aux_seg = torch.cat([x2,x3,x4],dim=1)
aux_seg = self.aux_combine(aux_seg)
else:
aux_seg = None
fea = self.pool(fea).view(-1, 1800)
group_cls = self.cls(fea).view(-1, *self.cls_dim)
if self.use_aux:
return group_cls, aux_seg
return group_cls
def initialize_weights(*models):
for model in models:
real_init_weights(model)
def real_init_weights(m):
if isinstance(m, list):
for mini_m in m:
real_init_weights(mini_m)
else:
if isinstance(m, torch.nn.Conv2d):
torch.nn.init.kaiming_normal_(m.weight, nonlinearity='relu')
if m.bias is not None:
torch.nn.init.constant_(m.bias, 0)
elif isinstance(m, torch.nn.Linear):
m.weight.data.normal_(0.0, std=0.01)
elif isinstance(m, torch.nn.BatchNorm2d):
torch.nn.init.constant_(m.weight, 1)
torch.nn.init.constant_(m.bias, 0)
elif isinstance(m,torch.nn.Module):
for mini_m in m.children():
real_init_weights(mini_m)
else:
print('unkonwn module', m)

120
algorithms/lane_ufld/code/UFLD/model/model0.py Executable file
View File

@@ -0,0 +1,120 @@
import torch
from model.backbone import resnet
import numpy as np
class conv_bn_relu(torch.nn.Module):
def __init__(self,in_channels, out_channels, kernel_size, stride=1, padding=0, dilation=1,bias=False):
super(conv_bn_relu,self).__init__()
self.conv = torch.nn.Conv2d(in_channels,out_channels, kernel_size,
stride = stride, padding = padding, dilation = dilation,bias = bias)
self.bn = torch.nn.BatchNorm2d(out_channels)
self.relu = torch.nn.ReLU()
def forward(self,x):
x = self.conv(x)
x = self.bn(x)
x = self.relu(x)
return x
class parsingNet(torch.nn.Module):
def __init__(self, size=(288, 800), pretrained=True, backbone='50', cls_dim=(37, 10, 4), use_aux=False):
super(parsingNet, self).__init__()
self.size = size
self.w = size[0]
self.h = size[1]
self.cls_dim = cls_dim # (num_gridding, num_cls_per_lane, num_of_lanes)
# num_cls_per_lane is the number of row anchors
self.use_aux = use_aux
self.total_dim = np.prod(cls_dim)
# input : nchw,
# output: (w+1) * sample_rows * 4
self.model = resnet(backbone, pretrained=pretrained)
if self.use_aux:
self.aux_header2 = torch.nn.Sequential(
conv_bn_relu(128, 128, kernel_size=3, stride=1, padding=1) if backbone in ['34','18'] else conv_bn_relu(512, 128, kernel_size=3, stride=1, padding=1),
conv_bn_relu(128,128,3,padding=1),
conv_bn_relu(128,128,3,padding=1),
conv_bn_relu(128,128,3,padding=1),
)
self.aux_header3 = torch.nn.Sequential(
conv_bn_relu(256, 128, kernel_size=3, stride=1, padding=1) if backbone in ['34','18'] else conv_bn_relu(1024, 128, kernel_size=3, stride=1, padding=1),
conv_bn_relu(128,128,3,padding=1),
conv_bn_relu(128,128,3,padding=1),
)
self.aux_header4 = torch.nn.Sequential(
conv_bn_relu(512, 128, kernel_size=3, stride=1, padding=1) if backbone in ['34','18'] else conv_bn_relu(2048, 128, kernel_size=3, stride=1, padding=1),
conv_bn_relu(128,128,3,padding=1),
)
self.aux_combine = torch.nn.Sequential(
conv_bn_relu(384, 256, 3,padding=2,dilation=2),
conv_bn_relu(256, 128, 3,padding=2,dilation=2),
conv_bn_relu(128, 128, 3,padding=2,dilation=2),
conv_bn_relu(128, 128, 3,padding=4,dilation=4),
torch.nn.Conv2d(128, cls_dim[-1] + 1,1)
# output : n, num_of_lanes+1, h, w
)
initialize_weights(self.aux_header2,self.aux_header3,self.aux_header4,self.aux_combine)
self.cls = torch.nn.Sequential(
torch.nn.Linear(1800, 2048),
torch.nn.ReLU(),
torch.nn.Linear(2048, self.total_dim),
)
self.pool = torch.nn.Conv2d(512,8,1) if backbone in ['34','18'] else torch.nn.Conv2d(2048,8,1)
# 1/32,2048 channel
# 288,800 -> 9,40,2048
# (w+1) * sample_rows * 4
# 37 * 10 * 4
initialize_weights(self.cls)
def forward(self, x):
# n c h w - > n 2048 sh sw
# -> n 2048
x2,x3,fea = self.model(x)
if self.use_aux:
x2 = self.aux_header2(x2)
x3 = self.aux_header3(x3)
x3 = torch.nn.functional.interpolate(x3,scale_factor = 2,mode='bilinear')
x4 = self.aux_header4(fea)
x4 = torch.nn.functional.interpolate(x4,scale_factor = 4,mode='bilinear')
aux_seg = torch.cat([x2,x3,x4],dim=1)
aux_seg = self.aux_combine(aux_seg)
else:
aux_seg = None
fea = self.pool(fea).view(-1, 1800)
group_cls = self.cls(fea).view(-1, *self.cls_dim)
if self.use_aux:
return group_cls, aux_seg
return group_cls
def initialize_weights(*models):
for model in models:
real_init_weights(model)
def real_init_weights(m):
if isinstance(m, list):
for mini_m in m:
real_init_weights(mini_m)
else:
if isinstance(m, torch.nn.Conv2d):
torch.nn.init.kaiming_normal_(m.weight, nonlinearity='relu')
if m.bias is not None:
torch.nn.init.constant_(m.bias, 0)
elif isinstance(m, torch.nn.Linear):
m.weight.data.normal_(0.0, std=0.01)
elif isinstance(m, torch.nn.BatchNorm2d):
torch.nn.init.constant_(m.weight, 1)
torch.nn.init.constant_(m.bias, 0)
elif isinstance(m,torch.nn.Module):
for mini_m in m.children():
real_init_weights(mini_m)
else:
print('unkonwn module', m)

296
algorithms/lane_ufld/code/UFLD/model/vovnet.py Normal file
View File

@@ -0,0 +1,296 @@
# VoVNet backbone (OSA + eSE), adapted from vovnet-detectron2 (no detectron2 dependency).
# Reference: BK2/archive/vovnet-detectron2-master/vovnet/vovnet.py
from collections import OrderedDict
import torch
import torch.nn as nn
import torch.nn.functional as F
VoVNet19_slim_dw_eSE = {
'stem': [64, 64, 64],
'stage_conv_ch': [64, 80, 96, 112],
'stage_out_ch': [112, 256, 384, 512],
'layer_per_block': 3,
'block_per_stage': [1, 1, 1, 1],
'eSE': True,
'dw': True,
}
VoVNet19_dw_eSE = {
'stem': [64, 64, 64],
'stage_conv_ch': [128, 160, 192, 224],
'stage_out_ch': [256, 512, 768, 1024],
'layer_per_block': 3,
'block_per_stage': [1, 1, 1, 1],
'eSE': True,
'dw': True,
}
VoVNet19_slim_eSE = {
'stem': [64, 64, 128],
'stage_conv_ch': [64, 80, 96, 112],
'stage_out_ch': [112, 256, 384, 512],
'layer_per_block': 3,
'block_per_stage': [1, 1, 1, 1],
'eSE': True,
'dw': False,
}
VoVNet19_eSE = {
'stem': [64, 64, 128],
'stage_conv_ch': [128, 160, 192, 224],
'stage_out_ch': [256, 512, 768, 1024],
'layer_per_block': 3,
'block_per_stage': [1, 1, 1, 1],
'eSE': True,
'dw': False,
}
VoVNet39_eSE = {
'stem': [64, 64, 128],
'stage_conv_ch': [128, 160, 192, 224],
'stage_out_ch': [256, 512, 768, 1024],
'layer_per_block': 5,
'block_per_stage': [1, 1, 2, 2],
'eSE': True,
'dw': False,
}
VoVNet57_eSE = {
'stem': [64, 64, 128],
'stage_conv_ch': [128, 160, 192, 224],
'stage_out_ch': [256, 512, 768, 1024],
'layer_per_block': 5,
'block_per_stage': [1, 1, 4, 3],
'eSE': True,
'dw': False,
}
VoVNet99_eSE = {
'stem': [64, 64, 128],
'stage_conv_ch': [128, 160, 192, 224],
'stage_out_ch': [256, 512, 768, 1024],
'layer_per_block': 5,
'block_per_stage': [1, 3, 9, 3],
'eSE': True,
'dw': False,
}
STAGE_SPECS = {
'V-19-slim-dw-eSE': VoVNet19_slim_dw_eSE,
'V-19-dw-eSE': VoVNet19_dw_eSE,
'V-19-slim-eSE': VoVNet19_slim_eSE,
'V-19-eSE': VoVNet19_eSE,
'V-39-eSE': VoVNet39_eSE,
'V-57-eSE': VoVNet57_eSE,
'V-99-eSE': VoVNet99_eSE,
}
# Short names used in UFLD configs (backbone='vov19slim', ...)
VOVNET_ALIASES = {
'vov19slim_dw': 'V-19-slim-dw-eSE',
'vov19_dw': 'V-19-dw-eSE',
'vov19slim': 'V-19-slim-eSE',
'vov19': 'V-19-eSE',
'vov39': 'V-39-eSE',
'vov57': 'V-57-eSE',
'vov99': 'V-99-eSE',
}
def _bn(ch):
return nn.BatchNorm2d(ch)
def dw_conv3x3(in_channels, out_channels, module_name, postfix, stride=1, kernel_size=3, padding=1):
return [
(f'{module_name}_{postfix}/dw', nn.Conv2d(
in_channels, out_channels, kernel_size=kernel_size, stride=stride,
padding=padding, groups=out_channels, bias=False)),
(f'{module_name}_{postfix}/pw', nn.Conv2d(
in_channels, out_channels, kernel_size=1, stride=1, padding=0, bias=False)),
(f'{module_name}_{postfix}/bn', _bn(out_channels)),
(f'{module_name}_{postfix}/relu', nn.ReLU(inplace=True)),
]
def conv3x3(in_channels, out_channels, module_name, postfix, stride=1, groups=1,
kernel_size=3, padding=1):
return [
(f'{module_name}_{postfix}/conv', nn.Conv2d(
in_channels, out_channels, kernel_size=kernel_size, stride=stride,
padding=padding, groups=groups, bias=False)),
(f'{module_name}_{postfix}/bn', _bn(out_channels)),
(f'{module_name}_{postfix}/relu', nn.ReLU(inplace=True)),
]
def conv1x1(in_channels, out_channels, module_name, postfix, stride=1, groups=1,
kernel_size=1, padding=0):
return [
(f'{module_name}_{postfix}/conv', nn.Conv2d(
in_channels, out_channels, kernel_size=kernel_size, stride=stride,
padding=padding, groups=groups, bias=False)),
(f'{module_name}_{postfix}/bn', _bn(out_channels)),
(f'{module_name}_{postfix}/relu', nn.ReLU(inplace=True)),
]
class Hsigmoid(nn.Module):
def __init__(self, inplace=True):
super().__init__()
self.inplace = inplace
def forward(self, x):
return F.relu6(x + 3.0, inplace=self.inplace) / 6.0
class eSEModule(nn.Module):
def __init__(self, channel):
super().__init__()
self.avg_pool = nn.AdaptiveAvgPool2d(1)
self.fc = nn.Conv2d(channel, channel, kernel_size=1, padding=0)
self.hsigmoid = Hsigmoid()
def forward(self, x):
return x * self.hsigmoid(self.fc(self.avg_pool(x)))
class _OSA_module(nn.Module):
def __init__(self, in_ch, stage_ch, concat_ch, layer_per_block, module_name,
identity=False, depthwise=False):
super().__init__()
self.identity = identity
self.depthwise = depthwise
self.is_reduced = False
self.layers = nn.ModuleList()
in_channel = in_ch
if self.depthwise and in_channel != stage_ch:
self.is_reduced = True
self.conv_reduction = nn.Sequential(
OrderedDict(conv1x1(in_channel, stage_ch, f'{module_name}_reduction', '0')))
for i in range(layer_per_block):
if self.depthwise:
self.layers.append(nn.Sequential(OrderedDict(
dw_conv3x3(stage_ch, stage_ch, module_name, str(i)))))
else:
self.layers.append(nn.Sequential(OrderedDict(
conv3x3(in_channel, stage_ch, module_name, str(i)))))
in_channel = stage_ch
in_channel = in_ch + layer_per_block * stage_ch
self.concat = nn.Sequential(OrderedDict(
conv1x1(in_channel, concat_ch, module_name, 'concat')))
self.ese = eSEModule(concat_ch)
def forward(self, x):
identity_feat = x
output = [x]
if self.depthwise and self.is_reduced:
x = self.conv_reduction(x)
for layer in self.layers:
x = layer(x)
output.append(x)
x = torch.cat(output, dim=1)
xt = self.ese(self.concat(x))
if self.identity:
xt = xt + identity_feat
return xt
class _OSA_stage(nn.Sequential):
def __init__(self, in_ch, stage_ch, concat_ch, block_per_stage, layer_per_block,
stage_num, depthwise=False):
super().__init__()
if stage_num != 2:
self.add_module('Pooling', nn.MaxPool2d(kernel_size=3, stride=2, ceil_mode=True))
module_name = f'OSA{stage_num}_1'
self.add_module(module_name, _OSA_module(
in_ch, stage_ch, concat_ch, layer_per_block, module_name, depthwise=depthwise))
for i in range(block_per_stage - 1):
module_name = f'OSA{stage_num}_{i + 2}'
self.add_module(module_name, _OSA_module(
concat_ch, stage_ch, concat_ch, layer_per_block, module_name,
identity=True, depthwise=depthwise))
class VoVNetBody(nn.Module):
"""VoVNet stages; forward returns (stage3, stage4, stage5) at strides 8/16/32."""
def __init__(self, variant='V-19-slim-eSE', input_ch=3):
super().__init__()
if variant in VOVNET_ALIASES:
variant = VOVNET_ALIASES[variant]
if variant not in STAGE_SPECS:
raise KeyError(f'Unknown VoVNet variant {variant!r}, choose from {list(STAGE_SPECS)}')
self.variant = variant
spec = STAGE_SPECS[variant]
stem_ch = spec['stem']
config_stage_ch = spec['stage_conv_ch']
config_concat_ch = spec['stage_out_ch']
block_per_stage = spec['block_per_stage']
layer_per_block = spec['layer_per_block']
depthwise = spec['dw']
conv_type = dw_conv3x3 if depthwise else conv3x3
stem = conv3x3(input_ch, stem_ch[0], 'stem', '1', stride=2)
stem += conv_type(stem_ch[0], stem_ch[1], 'stem', '2', stride=1)
stem += conv_type(stem_ch[1], stem_ch[2], 'stem', '3', stride=2)
self.stem = nn.Sequential(OrderedDict(stem))
in_ch_list = [stem_ch[2]] + config_concat_ch[:-1]
self.stage2 = _OSA_stage(
in_ch_list[0], config_stage_ch[0], config_concat_ch[0],
block_per_stage[0], layer_per_block, 2, depthwise=depthwise)
self.stage3 = _OSA_stage(
in_ch_list[1], config_stage_ch[1], config_concat_ch[1],
block_per_stage[1], layer_per_block, 3, depthwise=depthwise)
self.stage4 = _OSA_stage(
in_ch_list[2], config_stage_ch[2], config_concat_ch[2],
block_per_stage[2], layer_per_block, 4, depthwise=depthwise)
self.stage5 = _OSA_stage(
in_ch_list[3], config_stage_ch[3], config_concat_ch[3],
block_per_stage[3], layer_per_block, 5, depthwise=depthwise)
self.out_channels = {
'c3': config_concat_ch[1],
'c4': config_concat_ch[2],
'c5': config_concat_ch[3],
}
self._initialize_weights()
def _initialize_weights(self):
for m in self.modules():
if isinstance(m, nn.Conv2d):
nn.init.kaiming_normal_(m.weight)
def forward(self, x):
x = self.stem(x)
x = self.stage2(x)
c3 = self.stage3(x)
c4 = self.stage4(c3)
c5 = self.stage5(c4)
return c3, c4, c5
class vovnet(nn.Module):
"""UFLD-compatible wrapper (same interface as resnet: x2, x3, x4)."""
def __init__(self, variant='vov19slim', pretrained=False):
super().__init__()
if pretrained:
import warnings
warnings.warn(
'VoVNet has no torchvision pretrained weights in UFLD; '
'train from scratch or load a custom checkpoint.',
UserWarning,
stacklevel=2,
)
key = variant if variant in VOVNET_ALIASES else variant
self.body = VoVNetBody(key)
self.variant = self.body.variant
def forward(self, x):
return self.body(x)

139
algorithms/lane_ufld/code/UFLD/predict0729.py Executable file
View File

@@ -0,0 +1,139 @@
import torch, os, cv2, glob
from model.model import parsingNet
from utils.common import merge_config
from utils.dist_utils import dist_print
# import torch
import scipy.special, tqdm
import numpy as np
import torchvision.transforms as transforms
from data.dataset import LaneTestDataset
from data.constant import culane_row_anchor, tusimple_row_anchor
from scipy.optimize import curve_fit
from lane_show import is_in_poly, handle_point, poly_fitting, draw_values
import time
import PIL
import re
class PredictLane:
def __init__(self):
# super(PredictLane, self).__init__()
# self.img =img
self.cls_num_per_lane = 56
self.griding_num = 100
self.backbone = '34'
start_0 = time.time()
self.net = parsingNet(pretrained=False, backbone=self.backbone, cls_dim=(self.griding_num + 1, self.cls_num_per_lane, 4),
# use_aux=False).to(device)
use_aux=False).cuda() # we dont need auxiliary segmentation in testing
state_dict = torch.load('./model/curb_c599.pth', map_location='cuda')['model']
compatible_state_dict = {}
for k, v in state_dict.items():
if 'module.' in k:
compatible_state_dict[k[7:]] = v
else:
compatible_state_dict[k] = v
self.net.load_state_dict(compatible_state_dict, strict=False)
self.net.eval()
# net = torch.load(cfg.test_model, map_location='cuda')
end_0 = time.time()
count_0 = end_0 - start_0
print('the load net time is : ', count_0)
self.img_transforms = transforms.Compose([
transforms.Resize((288, 800)),
transforms.ToTensor(),
transforms.Normalize((0.485, 0.456, 0.406), (0.229, 0.224, 0.225)),
])
self.count = 0
def predict(self, img):
img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
self.count += 1
start_1 = time.time()
img_i = PIL.Image.fromarray(img.astype(np.uint8))
img_t = self.img_transforms(img_i)
img_w, img_h = 1280, 720
row_anchor = tusimple_row_anchor
# print(img_t)
img_t = img_t.reshape(1, 3, 288, 800)
# print(img_t)
# print(img_t.shape)
imgs = img_t.cuda()
end_1 = time.time()
count_1 = end_1 - start_1
print('the predeal time is : ', count_1)
# imgs = imgs.to(device)
with torch.no_grad():
start_t = time.time()
out = self.net(imgs)
# print(out[0].shape)
end_t = time.time()
count_t = end_t - start_t
print('the pre time is : ', count_t)
col_sample = np.linspace(0, 800 - 1, self.griding_num)
col_sample_w = col_sample[1] - col_sample[0]
out_j = out[0].data.cpu().numpy()
# print(out_j.shape, type(out_j))
out_j = out_j[:, ::-1, :]
prob = scipy.special.softmax(out_j[:-1, :, :], axis=0)
idx = np.arange(self.griding_num) + 1
idx = idx.reshape(-1, 1, 1)
loc = np.sum(prob * idx, axis=0)
out_j = np.argmax(out_j, axis=0)
loc[out_j == self.griding_num] = 0
out_j = loc
vis = img
lanes_list = []
for i in range(out_j.shape[1]):
# print(i)
points_list = []
# print(out_j.shape[1])
if np.sum(out_j[:, i] != 0) > 2:
# poly = [[400, 211], [23, 403], [930, 230], [1276, 442]] # ROI区域
poly = [[0, 0], [0, 720], [1280, 0], [1280, 720]] # ROI区域
lane_x = []
lane_y = []
for k in range(out_j.shape[0]):
# print(out_j.shape[0])6
if out_j[k, i] > 0:
ppp = (int(out_j[k, i] * col_sample_w * img_w / 800) - 1,
int(img_h * (row_anchor[self.cls_num_per_lane - 1 - k] / 288)) - 1)
is_in = is_in_poly(ppp, poly)
if is_in == True:
# 将处理后的点坐标添如一个空列表做拟合用
lane_x.append(ppp[0])
lane_y.append(ppp[1])
points_list.append((float(ppp[0]), float(ppp[1])))
cv2.circle(vis, ppp, 5, (0, 255, 0), -1)
lx, ly, rx, ry = handle_point(lane_x, lane_y)
# print('1111111111', lx, ly, rx, ry)
# curvature, distance_from_center = poly_fitting(lx, ly, rx, ry)
# draw_values(vis, curvature, distance_from_center)
# print(points_list)
# lane = np.uint8()
if points_list != []:
lanes_list.append(points_list)
cv2.imshow('vis', vis)
cv2.waitKey(1)
return lanes_list
if __name__ == "__main__":
torch.backends.cudnn.benchmark = True
# args, cfg = merge_config()
data_root = r'C:\data\curb_data\curb_data\2'
dist_print('start testing...')
backbone = '34'
# jpg_file = 'n_9\\frame6000.jpg'
pic_list = glob.glob(data_root + '/*.png')
pattern_number_oeder = '(\d*?).png'
pic_list.sort(key=lambda x: int(re.findall(pattern_number_oeder, x)[0]))
a = PredictLane()
# print(pic_list)
for pic in pic_list:
print(pic)
img = cv2.imread(pic)
lanes_list = a.predict(img)

139
algorithms/lane_ufld/code/UFLD/profile_model.py Normal file
View File

@@ -0,0 +1,139 @@
"""Count parameters and FLOPs for a UFLD parsingNet checkpoint."""
import argparse
import os
import re
import torch
from model.backbone import is_vovnet
from model.model import parsingNet
from utils.common import checkpoint_state_dict
from utils.config import Config
# TuSimple / lane0 UFLD row anchors (fixed in original repo)
CLS_NUM_PER_LANE = 56
INPUT_SIZE = (288, 800)
def parse_cfg_txt(cfg_path):
"""Parse saved cfg.txt from train.py (Config repr line)."""
with open(cfg_path, encoding='utf-8') as f:
text = f.read()
m = re.search(r"\{.*\}", text, re.DOTALL)
if not m:
raise ValueError(f'cannot parse dict from {cfg_path}')
# cfg.txt uses single-quoted python dict repr
return eval(m.group())
def build_net(backbone, griding_num, num_lanes, use_aux, pretrained=False):
return parsingNet(
pretrained=pretrained,
backbone=str(backbone),
cls_dim=(griding_num + 1, CLS_NUM_PER_LANE, num_lanes),
use_aux=use_aux,
)
def count_params(net):
total = sum(p.numel() for p in net.parameters())
trainable = sum(p.numel() for p in net.parameters() if p.requires_grad)
return total, trainable
def count_flops(net, input_size):
x = torch.randn(1, 3, *input_size)
from torch.utils.flop_counter import FlopCounterMode
with FlopCounterMode(display=False) as fc:
with torch.no_grad():
net(x)
total = fc.get_total_flops()
breakdown = fc.get_flop_counts().get('Global', {})
return total, breakdown, tuple(net(x).shape)
def main():
parser = argparse.ArgumentParser(description='UFLD params / FLOPs profiler')
parser.add_argument(
'--run_dir',
default='log/20250702_165153_lr_1e-05_b_32_ufld_2lanes_res18',
help='training log dir containing cfg.txt and optional best.pth',
)
parser.add_argument('--config', default=None, help='config .py (overrides cfg.txt)')
parser.add_argument('--model_path', default=None, help='.pth checkpoint (optional)')
parser.add_argument('--backbone', default=None)
parser.add_argument('--griding_num', type=int, default=None)
parser.add_argument('--num_lanes', type=int, default=None)
parser.add_argument('--use_aux', action='store_true', default=None)
parser.add_argument('--height', type=int, default=INPUT_SIZE[0])
parser.add_argument('--width', type=int, default=INPUT_SIZE[1])
args = parser.parse_args()
backbone = '18'
griding_num = 100
num_lanes = 4
use_aux = False
model_path = args.model_path
if args.config:
cfg = Config.fromfile(args.config)
backbone = cfg.backbone
griding_num = cfg.griding_num
num_lanes = cfg.num_lanes
use_aux = getattr(cfg, 'use_aux', False)
if model_path is None and getattr(cfg, 'test_model', None):
model_path = cfg.test_model
elif args.run_dir and os.path.isfile(os.path.join(args.run_dir, 'cfg.txt')):
cfg = parse_cfg_txt(os.path.join(args.run_dir, 'cfg.txt'))
backbone = cfg.get('backbone', backbone)
griding_num = cfg.get('griding_num', griding_num)
num_lanes = cfg.get('num_lanes', num_lanes)
use_aux = cfg.get('use_aux', use_aux)
if model_path is None:
for name in ('best.pth', 'latest.pth', 'model.pt'):
p = os.path.join(args.run_dir, name)
if os.path.isfile(p):
model_path = p
break
if args.backbone is not None:
backbone = args.backbone
if args.griding_num is not None:
griding_num = args.griding_num
if args.num_lanes is not None:
num_lanes = args.num_lanes
if args.use_aux is not None:
use_aux = args.use_aux
input_size = (args.height, args.width)
net = build_net(backbone, griding_num, num_lanes, use_aux, pretrained=False)
if model_path and os.path.isfile(model_path):
net.load_state_dict(checkpoint_state_dict(model_path, map_location='cpu'), strict=False)
net.eval()
total, trainable = count_params(net)
flops, breakdown, out_shape = count_flops(net, input_size)
print('=== UFLD model profile ===')
if model_path:
print('checkpoint:', os.path.abspath(model_path))
if args.run_dir:
print('run_dir:', os.path.abspath(args.run_dir))
arch = f'VoVNet({backbone})' if is_vovnet(backbone) else f'ResNet{backbone}'
print(
f'arch: {arch}, griding_num={griding_num}, num_lanes={num_lanes}, '
f'use_aux={use_aux}, input=1x3x{input_size[0]}x{input_size[1]}'
)
print(f'output shape: {out_shape}')
print(f'Parameters: {total:,} ({total / 1e6:.4f} M)')
print(f'Trainable: {trainable:,}')
print(f'FLOPs (PyTorch FlopCounterMode, 1 image): {flops:,} ({flops / 1e9:.4f} GFLOPs)')
print('FLOPs breakdown (top ops):')
for op, v in sorted(breakdown.items(), key=lambda kv: -kv[1])[:6]:
print(f' {op}: {v / 1e9:.4f} G')
if __name__ == '__main__':
main()

225
algorithms/lane_ufld/code/UFLD/pth2onnx_and_check.py Normal file
View File

@@ -0,0 +1,225 @@
# encoding: utf-8
import time
import tkinter.filedialog
import cv2
import numpy as np
import onnx
import onnxruntime
import scipy.special
from scipy.optimize import curve_fit
import matplotlib.pyplot as plt
import torch
import torch.backends.cudnn
import torchvision.transforms as transforms
from PIL import Image as imim
from torch import nn
from data.constant import tusimple_row_anchor
from model.model import parsingNet
from utils.common import merge_config
import glob
# 一些预定设置
torch.backends.cudnn.benchmark = True
args, cfg = merge_config()
griding_num = 100
img_w, img_h = 1280, 720
row_anchor = tusimple_row_anchor
cls_num_per_lane = 56
model_path = "/home/ljk/桌面/ClrNet_onnx_modify/UFLD/ep327_ufld.pth"
# model_path = "/home/ljk/桌面/ep015_2ch_res18.pth"
folder_path = "/home/ljk/桌面/pic_1228_lane/pic1_720/*.jpg"
def func(x, a, b, c, d):
return a * pow(x, 3) + b * pow(x, 2) + c * x + d
class ufld_2(nn.Module):
def __init__(self):
super().__init__()
# self.path = "/home/ljk/桌面/semantic_code/1.lane_detect/3.model_weight/1.pth/shuangyujing_model_SpeedUp.pth"
# print(self.path)
# self.net = torch.load(self.path)
backbone_ = '18'
self.net = parsingNet(pretrained=False, backbone=backbone_, cls_dim=(cfg.griding_num + 1, 56, 2),
use_aux=False).cuda()
self.path = model_path
print(self.path)
state_dict = torch.load(self.path, map_location='cpu')['model']
compatible_state_dict = {}
for k, v in state_dict.items():
if 'module.' in k:
compatible_state_dict[k[7:]] = v
else:
compatible_state_dict[k] = v
self.net.load_state_dict(compatible_state_dict, strict=False)
self.net.eval()
# self.onnx_model_name = self.path[:-4] + ".onnx"
self.onnx_model_name = "/home/ljk/桌面/ClrNet_onnx_modify/UFLD/ep327_ufld.onnx"
def export_onnx(self):
x = torch.randn(1, 3, 288, 800).cuda()
# torch.onnx.export 可以检查剪枝模型出了什么错
with torch.no_grad():
torch.onnx.export(self.net, x, self.onnx_model_name, export_params=True, opset_version=11,
input_names=['input'],
output_names=['output'])
print('\n', "start check...")
onnx_model = onnx.load(self.onnx_model_name)
try:
onnx.checker.check_model(onnx_model)
except:
print("Model incorrect")
else:
print("Model correct")
def output_cmp(self, img_path):
start = time.time()
img = imim.open(img_path)
img_transforms = transforms.Compose([
transforms.Resize((288, 800)),
transforms.ToTensor(),
transforms.Normalize((0.485, 0.456, 0.406), (0.229, 0.224, 0.225)),
])
img = img_transforms(img).unsqueeze_(0).cuda()
# 原模型的输出-------------------
with torch.no_grad():
out = self.net(img)
pth_output = to_numpy(out)
end = time.time()
cost_time = end - start
print("pth_cost_time:", cost_time * 1000, " ms")
print("原模型前5*4个向量输出：\n", pth_output[0][0][:5])
start = time.time()
ort_session = onnxruntime.InferenceSession(self.onnx_model_name)
end = time.time()
cost_time = end - start
print("onnx_cost_time:", cost_time * 1000, " ms")
ort_inputs = {'input': to_numpy(img)}
ort_out = ort_session.run(['output'], ort_inputs)[0]
print("onnx的输出大小：", ort_out.shape)
print("onnx前5*4个向量输出：\n", ort_out[0][0][:5])
# np.testing.assert_allclose(pth_output, ort_out, rtol=1e-01, atol=1e-03)
print("*****************************")
print("精度对比结束，满足精度！")
print("*****************************")
# _ = pro_process(pth_output, img_path)
print(img_path)
_ = pro_process(ort_out, img_path)
# cv2.namedWindow('vis_onnx')
# cv2.imshow("vis_onnx", pro_process(ort_out, img_path))
def get_path():
filename = tkinter.filedialog.askopenfilename()
return filename
# 工具函数：tensor->numpy
def to_numpy(tensor):
return tensor.detach().cpu().numpy() if tensor.requires_grad else tensor.cpu().numpy()
# 工具函数：后处理与可视化
def pro_process(out, path):
col_sample = np.linspace(0, 800 - 1, griding_num)
col_sample_w = col_sample[1] - col_sample[0]
out_j = out[0]
out_j = out_j[:, ::-1, :]
prob = scipy.special.softmax(out_j[:-1, :, :], axis=0)
idx = np.arange(griding_num) + 1
idx = idx.reshape(-1, 1, 1)
loc = np.sum(prob * idx, axis=0)
out_j = np.argmax(out_j, axis=0)
loc[out_j == griding_num] = 0
out_j = loc
vis = cv2.imread(path)
left_points = []
right_points = []
for i in range(out_j.shape[1]):
if np.sum(out_j[:, i] != 0) > 2:
if i == 1:
print("Left Line")
else:
print("Right Line")
count = 0
for k in range(out_j.shape[0]):
if out_j[k][i] > 0:
x = int(out_j[k][i] * col_sample_w * img_w / 800) - 1
y = int(img_h * (row_anchor[cls_num_per_lane - 1 - k] / 288)) - 1
ppp = (x, y)
# print(ppp)
if i == 0:
left_points.append(ppp)
else:
right_points.append(ppp)
cv2.circle(vis, ppp, 2, (255, 0, 0), -1)
count += 1
print(count)
print("\n")
# if len(left_points) == 0 or len(right_points) == 0:
# return vis
# left_points_x = np.reshape(left_points, (len(left_points), -1))[:, 1]
# left_points_y = np.reshape(left_points, (len(left_points), -1))[:, 0]
# right_points_x = np.reshape(right_points, (len(right_points), -1))[:, 1]
# right_points_y = np.reshape(right_points, (len(right_points), -1))[:, 0]
# popt_left, _ = curve_fit(func, left_points_x, left_points_y)
# popt_right, _ = curve_fit(func, right_points_x, right_points_y)
# right_points_y_func = []
# left_points_y_func = []
# for i in range(len(left_points_x)):
# ppp = (int(func(left_points_x[i], popt_left[0], popt_left[1], popt_left[2], popt_left[3])),
# left_points_x[i])
# # cv2.circle(vis, ppp, 5, (0, 255, 0), -1)
# left_points_y_func.append(
# int(func(left_points_x[i], popt_left[0], popt_left[1], popt_left[2], popt_left[3])))
# for i in range(len(right_points_x)):
# ppp = (
# int(func(right_points_x[i], popt_right[0], popt_right[1], popt_right[2], popt_right[3])),
# right_points_x[i])
# # cv2.circle(vis, ppp, 5, (0, 255, 0), -1)
# right_points_y_func.append(
# int(func(right_points_x[i], popt_right[0], popt_right[1], popt_right[2], popt_right[3])))
# left_points_y_func = np.reshape(left_points_y_func, (len(left_points_y_func), -1))
# right_points_y_func = np.reshape(right_points_y_func, (len(right_points_y_func), -1))
# left_points_y_func = np.squeeze(left_points_y_func)
# right_points_y_func = np.squeeze(right_points_y_func)
cv2.namedWindow('vis_onnx', 0)
cv2.imshow("vis_onnx", vis)
cv2.waitKey(0)
# cv2.destroyAllWindows()
# cv2.imwrite(path, vis)
return vis
def main():
files = glob.glob(folder_path)
files = sorted(files)
model = ufld_2()
model.export_onnx()
for i in range(len(files)):
model.output_cmp(files[i])
if __name__ == '__main__':
main()

99
algorithms/lane_ufld/code/UFLD/pth_to_onnx.py Executable file
View File

@@ -0,0 +1,99 @@
"""Export UFLD .pth checkpoint to ONNX (matches tusimple_res18_4lane_v1 / best.pth)."""
import argparse
import os
import torch
from model.model import parsingNet
from utils.common import checkpoint_state_dict
from utils.config import Config
def export_onnx(
model_path,
output_path=None,
backbone='18',
griding_num=100,
num_lanes=4,
opset=11,
):
if output_path is None:
base, _ = os.path.splitext(model_path)
output_path = base + '.onnx'
cls_num_per_lane = 56 # TuSimple row anchors
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
net = parsingNet(
pretrained=False,
backbone=backbone,
cls_dim=(griding_num + 1, cls_num_per_lane, num_lanes),
use_aux=False,
).to(device)
net.load_state_dict(checkpoint_state_dict(model_path, map_location=device), strict=False)
net.eval()
dummy = torch.randn(1, 3, 288, 800, device=device)
os.makedirs(os.path.dirname(output_path) or '.', exist_ok=True)
torch.onnx.export(
net,
dummy,
output_path,
opset_version=opset,
input_names=['input'],
output_names=['output'],
dynamic_axes={'input': {0: 'batch'}, 'output': {0: 'batch'}},
dynamo=False, # legacy exporter embeds weights (PyTorch 2.9+ default may omit)
)
print('saved:', os.path.abspath(output_path))
print('input: [1, 3, 288, 800]')
print('output:', tuple(net(dummy).shape))
return output_path
def main():
default_pth = 'log/20250702_165153_lr_1e-05_b_32_ufld_2lanes_res18/best.pth'
parser = argparse.ArgumentParser(description='Export UFLD checkpoint to ONNX')
parser.add_argument('--config', default=None, help='optional py config (overrides defaults)')
parser.add_argument('--model_path', default=default_pth, help='path to .pth')
parser.add_argument('--output', default=None, help='output .onnx path')
parser.add_argument('--backbone', default=None)
parser.add_argument('--griding_num', type=int, default=None)
parser.add_argument('--num_lanes', type=int, default=None)
parser.add_argument('--opset', type=int, default=11)
args = parser.parse_args()
backbone = '18'
griding_num = 100
num_lanes = 4
model_path = args.model_path
output = args.output
if args.config:
cfg = Config.fromfile(args.config)
backbone = cfg.backbone
griding_num = cfg.griding_num
num_lanes = cfg.num_lanes
if getattr(cfg, 'test_model', None):
model_path = cfg.test_model
if args.backbone is not None:
backbone = args.backbone
if args.griding_num is not None:
griding_num = args.griding_num
if args.num_lanes is not None:
num_lanes = args.num_lanes
export_onnx(
model_path,
output_path=output,
backbone=backbone,
griding_num=griding_num,
num_lanes=num_lanes,
opset=args.opset,
)
if __name__ == '__main__':
main()

6
algorithms/lane_ufld/code/UFLD/requirements.txt Executable file
View File

@@ -0,0 +1,6 @@
opencv-python
tqdm
tensorboard
addict
sklearn
pathspec

40
algorithms/lane_ufld/code/UFLD/ros_cap_node.py Executable file
View File

@@ -0,0 +1,40 @@
#!/usr/bin/env python
# coding:utf-8
import cv2
import numpy as np
import rospy
from std_msgs.msg import Header
from sensor_msgs.msg import Image
from cv_bridge import CvBridge, CvBridgeError
import time
if __name__ == "__main__":
capture = cv2.VideoCapture(0) # 定义摄像头
rospy.init_node('camera_node', anonymous=True) # 定义节点
image_pub = rospy.Publisher('/image_view/image_raw', Image, queue_size=1) # 定义话题
while not rospy.is_shutdown(): # Ctrl C正常退出，如果异常退出会报错device busy！
start = time.time()
ret, frame = capture.read()
if ret: # 如果有画面再执行
# frame = cv2.flip(frame,0) #垂直镜像操作
frame = cv2.flip(frame, 1) # 水平镜像操作
ros_frame = Image()
header = Header(stamp=rospy.Time.now())
header.frame_id = "Camera"
ros_frame.header = header
ros_frame.width = 640
ros_frame.height = 480
ros_frame.encoding = "bgr8"
ros_frame.step = 1920
ros_frame.data = np.array(frame).tostring() # 图片格式转换
image_pub.publish(ros_frame) # 发布消息
end = time.time()
print("cost time:", end - start) # 看一下每一帧的执行时间，从而确定合适的rate
rate = rospy.Rate(25) # 10hz
capture.release()
cv2.destroyAllWindows()
print("quit successfully!")

22
algorithms/lane_ufld/code/UFLD/ros_get_pic_deal.py Executable file
View File

@@ -0,0 +1,22 @@
#!/usr/bin/env python
# coding:utf-8
import rospy
import numpy as np
from sensor_msgs.msg import Image
from cv_bridge import CvBridge, CvBridgeError
import cv2
def callback(data):
global bridge
cv_img = bridge.imgmsg_to_cv2(data, "bgr8")
cv2.imshow("frame", cv_img)
cv2.waitKey(1)
if __name__ == '__main__':
rospy.init_node('img_process_node', anonymous=True)
bridge = CvBridge()
rospy.Subscriber('/image_view/image_raw', Image, callback)
rospy.spin()

160
algorithms/lane_ufld/code/UFLD/scripts/convert_tusimple.py Executable file
View File

@@ -0,0 +1,160 @@
import os
import cv2
import tqdm
import numpy as np
import pdb
import json, argparse
def calc_k(line):
'''
Calculate the direction of lanes
'''
line_x = line[::2]
line_y = line[1::2]
length = np.sqrt((line_x[0]-line_x[-1])**2 + (line_y[0]-line_y[-1])**2)
if length < 90:
return -10 # if the lane is too short, it will be skipped
p = np.polyfit(line_x, line_y,deg = 1)
rad = np.arctan(p[0])
return rad
def draw(im, line, idx, show=False):
'''
Generate the segmentation label according to json annotation
'''
line_x = line[::2]
line_y = line[1::2]
pt0 = (int(line_x[0]),int(line_y[0]))
if show:
cv2.putText(im,str(idx),(int(line_x[len(line_x) // 2]),int(line_y[len(line_x) // 2]) - 20),cv2.FONT_HERSHEY_SIMPLEX, 1.0, (255, 255, 255), lineType=cv2.LINE_AA)
idx = idx * 60
for i in range(len(line_x)-1):
cv2.line(im,pt0,(int(line_x[i+1]),int(line_y[i+1])),(idx,),thickness=16)
pt0 = (int(line_x[i+1]),int(line_y[i+1]))
def get_tusimple_list(root, label_list):
'''
Get all the files' names from the json annotation
'''
label_json_all = []
for l in label_list:
l = os.path.join(root,l)
label_json = [json.loads(line) for line in open(l).readlines()]
label_json_all += label_json
names = [l['raw_file'] for l in label_json_all]
h_samples = [np.array(l['h_samples']) for l in label_json_all]
lanes = [np.array(l['lanes']) for l in label_json_all]
line_txt = []
for i in range(len(lanes)):
line_txt_i = []
for j in range(len(lanes[i])):
if np.all(lanes[i][j] == -2):
continue
valid = lanes[i][j] != -2
line_txt_tmp = [None]*(len(h_samples[i][valid])+len(lanes[i][j][valid]))
line_txt_tmp[::2] = list(map(str,lanes[i][j][valid]))
line_txt_tmp[1::2] = list(map(str,h_samples[i][valid]))
line_txt_i.append(line_txt_tmp)
line_txt.append(line_txt_i)
return names,line_txt
def generate_segmentation_and_train_list(root, line_txt, names):
"""
The lane annotations of the Tusimple dataset is not strictly in order, so we need to find out the correct lane order for segmentation.
We use the same definition as CULane, in which the four lanes from left to right are represented as 1,2,3,4 in segentation label respectively.
"""
print(root, line_txt[0], names[0])
train_gt_fp = open(os.path.join(root, 'train_gt.txt'), 'w')
for i in tqdm.tqdm(range(len(line_txt))):
tmp_line = line_txt[i]
lines = []
for j in range(len(tmp_line)):
lines.append(list(map(float,tmp_line[j])))
ks = np.array([calc_k(line) for line in lines]) # get the direction of each lane
k_neg = ks[ks<0].copy()
k_pos = ks[ks>0].copy()
k_neg = k_neg[k_neg != -10] # -10 means the lane is too short and is discarded
k_pos = k_pos[k_pos != -10]
k_neg.sort()
k_pos.sort()
label_path = names[i][:-3]+'png'
label = np.zeros((720,1280),dtype=np.uint8)
bin_label = [0,0,0,0]
if len(k_neg) == 1: # for only one lane in the left
which_lane = np.where(ks == k_neg[0])[0][0]
draw(label,lines[which_lane],2)
bin_label[1] = 1
elif len(k_neg) == 2: # for two lanes in the left
which_lane = np.where(ks == k_neg[1])[0][0]
draw(label,lines[which_lane],1)
which_lane = np.where(ks == k_neg[0])[0][0]
draw(label,lines[which_lane],2)
bin_label[0] = 1
bin_label[1] = 1
elif len(k_neg) > 2: # for more than two lanes in the left,
which_lane = np.where(ks == k_neg[1])[0][0] # we only choose the two lanes that are closest to the center
draw(label,lines[which_lane],1)
which_lane = np.where(ks == k_neg[0])[0][0]
draw(label,lines[which_lane],2)
bin_label[0] = 1
bin_label[1] = 1
if len(k_pos) == 1: # For the lanes in the right, the same logical is adopted.
which_lane = np.where(ks == k_pos[0])[0][0]
draw(label,lines[which_lane],3)
bin_label[2] = 1
elif len(k_pos) == 2:
which_lane = np.where(ks == k_pos[1])[0][0]
draw(label,lines[which_lane],3)
which_lane = np.where(ks == k_pos[0])[0][0]
draw(label,lines[which_lane],4)
bin_label[2] = 1
bin_label[3] = 1
elif len(k_pos) > 2:
which_lane = np.where(ks == k_pos[-1])[0][0]
draw(label,lines[which_lane],3)
which_lane = np.where(ks == k_pos[-2])[0][0]
draw(label,lines[which_lane],4)
bin_label[2] = 1
bin_label[3] = 1
cv2.imwrite(os.path.join(root,label_path),label)
train_gt_fp.write(names[i] + ' ' + label_path + ' '+' '.join(list(map(str,bin_label))) + '\n')
train_gt_fp.close()
def get_args():
parser = argparse.ArgumentParser()
parser.add_argument('--root', required=True, help='The root of the Tusimple dataset')
return parser
if __name__ == "__main__":
args = get_args().parse_args()
# training set
names, line_txt = get_tusimple_list(args.root, ['label_data_0601.json', 'label_data_0531.json', 'label_data_0313.json'])
# generate segmentation and training list for training
generate_segmentation_and_train_list(args.root, line_txt, names)
# testing set
names,line_txt = get_tusimple_list(args.root, ['test_tasks_0627.json'])
# generate testing set for testing
with open(os.path.join(args.root, 'test.txt'), 'w') as fp:
for name in names:
fp.write(name + '\n')

153
algorithms/lane_ufld/code/UFLD/speed_real.py Executable file
View File

@@ -0,0 +1,153 @@
# Thanks for the contribution of KopiSoftware https://github.com/KopiSoftware
import torch
import time
import numpy as np
from model.model import parsingNet
import torchvision.transforms as transforms
import cv2
from matplotlib import pyplot as plt
from PIL import Image
img_transforms = transforms.Compose([
transforms.Resize((288, 800)),
transforms.ToTensor(),
transforms.Normalize((0.485, 0.456, 0.406), (0.229, 0.224, 0.225)),
])
def resize(x, y):
global cap
cap.set(3,x)
cap.set(4,y)
def test_practical_without_readtime():
global cap
for i in range(10):
_,img = cap.read()
img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
img2 = Image.fromarray(img)
x = img_transforms(img2)
x = x.unsqueeze(0).cuda()+1
y = net(x)
print("pracrical image input size:",img.shape)
print("pracrical tensor input size:",x.shape)
t_all = []
for i in range(100):
_,img = cap.read()
img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
img2 = Image.fromarray(img)
x = img_transforms(img2)
x = x.unsqueeze(0).cuda()+1
t1 = time.time()
y = net(x)
t2 = time.time()
t_all.append(t2 - t1)
print("practical with out read time:")
print('\taverage time:', np.mean(t_all) / 1)
print('\taverage fps:',1 / np.mean(t_all))
# print('fastest time:', min(t_all) / 1)
# print('fastest fps:',1 / min(t_all))
# print('slowest time:', max(t_all) / 1)
# print('slowest fps:',1 / max(t_all))
def test_practical():
global cap
for i in range(10):
_,img = cap.read()
img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
img2 = Image.fromarray(img)
x = img_transforms(img2)
x = x.unsqueeze(0).cuda()+1
y = net(x)
print("pracrical image input size:",img.shape)
print("pracrical tensor input size:",x.shape)
t_all = []
t_capture = []
t_preprocessing = []
t_net = []
for i in range(100):
t1 = time.time()
t3 = time.time()
_,img = cap.read()
t4 = time.time()
t5 = time.time()
img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
img2 = Image.fromarray(img)
x = img_transforms(img2)
x = x.unsqueeze(0).cuda()+1
t6 = time.time()
y = net(x)
t2 = time.time()
t_all.append(t2 - t1)
t_capture.append(t4 - t3)
t_preprocessing.append(t6 - t5)
t_net.append(t2 - t6)
print("practical with read time:")
print('\taverage time:', np.mean(t_all) / 1)
print('\taverage fps:',1 / np.mean(t_all))
print('\tcapture time:', np.mean(t_capture) / 1)
print('\tpre-processing time:', np.mean(t_preprocessing) / 1)
print('\tdetect time:', np.mean(t_net) / 1)
# print('fastest time:', min(t_all) / 1)
# print('fastest fps:',1 / min(t_all))
# print('slowest time:', max(t_all) / 1)
# print('slowest fps:',1 / max(t_all))
###x = torch.zeros((1,3,288,800)).cuda() + 1
def test_theoretical():
x = torch.zeros((1,3,288,800)).cuda() + 1
for i in range(10):
y = net(x)
t_all = []
for i in range(100):
t1 = time.time()
y = net(x)
t2 = time.time()
t_all.append(t2 - t1)
print("theortical")
print('\taverage time:', np.mean(t_all) / 1)
print('\taverage fps:',1 / np.mean(t_all))
# print('fastest time:', min(t_all) / 1)
# print('fastest fps:',1 / min(t_all))
# print('slowest time:', max(t_all) / 1)
# print('slowest fps:',1 / max(t_all))
if __name__ == "__main__":
###captrue data from camera or video
#cap = cv2.VideoCapture("video.mp4") #uncommen to activate a video input
cap = cv2.VideoCapture(0) #uncommen to activate a camera imput
#resize(480, 640) #ucommen to change input size
# torch.backends.cudnn.deterministic = False
torch.backends.cudnn.benchmark = True
net = parsingNet(pretrained = False, backbone='18',cls_dim = (100+1,56,4),use_aux=False).cuda()
# net = parsingNet(pretrained = False, backbone='18',cls_dim = (200+1,18,4),use_aux=False).cuda()
net.eval()
test_practical_without_readtime()
test_practical()
cap.release()
test_theoretical()

32
algorithms/lane_ufld/code/UFLD/speed_simple.py Executable file
View File

@@ -0,0 +1,32 @@
import torch
import time
import numpy as np
from model.model import parsingNet
# torch.backends.cudnn.deterministic = False
torch.backends.cudnn.benchmark = True
net = parsingNet(pretrained = False, backbone='18',cls_dim = (100+1,56,4),use_aux=False).cuda()
# net = parsingNet(pretrained = False, backbone='18',cls_dim = (200+1,18,4),use_aux=False).cuda()
net.eval()
x = torch.zeros((1,3,288,800)).cuda() + 1
for i in range(10):
y = net(x)
t_all = []
for i in range(100):
t1 = time.time()
y = net(x)
t2 = time.time()
t_all.append(t2 - t1)
print('average time:', np.mean(t_all) / 1)
print('average fps:',1 / np.mean(t_all))
print('fastest time:', min(t_all) / 1)
print('fastest fps:',1 / min(t_all))
print('slowest time:', max(t_all) / 1)
print('slowest fps:',1 / max(t_all))

48
algorithms/lane_ufld/code/UFLD/test.py Executable file
View File

@@ -0,0 +1,48 @@
import os
import torch
from model.model import parsingNet
from utils.common import merge_config, checkpoint_state_dict
from utils.dist_utils import dist_print
from evaluation.eval_wrapper import eval_lane
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
if __name__ == "__main__":
torch.backends.cudnn.benchmark = True
args, cfg = merge_config()
distributed = False
if 'WORLD_SIZE' in os.environ:
distributed = int(os.environ['WORLD_SIZE']) > 1
if distributed:
torch.cuda.set_device(args.local_rank)
torch.distributed.init_process_group(backend='nccl', init_method='env://')
dist_print('start testing...')
from model.backbone import SUPPORTED_BACKBONES
assert cfg.backbone in SUPPORTED_BACKBONES
if cfg.dataset == 'CULane':
cls_num_per_lane = 18
elif cfg.dataset == 'Tusimple':
cls_num_per_lane = 56
else:
raise NotImplementedError
net = parsingNet(pretrained=False, backbone=cfg.backbone, cls_dim=(cfg.griding_num+1, cls_num_per_lane,
cfg.num_lanes), use_aux=False).to(device)
# cfg.num_lanes), use_aux=False).cuda()
# we don't need auxiliary segmentation in testing
net.load_state_dict(checkpoint_state_dict(cfg.test_model, map_location=device), strict=False)
if distributed:
net = torch.nn.parallel.DistributedDataParallel(net, device_ids=[args.local_rank])
if not os.path.exists(cfg.test_work_dir):
os.mkdir(cfg.test_work_dir)
test_list = getattr(cfg, 'test_list', None)
skip_eval = getattr(cfg, 'skip_eval', False)
eval_lane(net, cfg.dataset, cfg.data_root, cfg.test_work_dir, cfg.griding_num, False, distributed,
test_list=test_list, skip_eval=skip_eval)

54
algorithms/lane_ufld/code/UFLD/test0.py Executable file
View File

@@ -0,0 +1,54 @@
import torch, os
from model.model import parsingNet
from utils.common import merge_config
from utils.dist_utils import dist_print
from evaluation.eval_wrapper import eval_lane
import torch
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
if __name__ == "__main__":
torch.backends.cudnn.benchmark = True
args, cfg = merge_config()
print(cfg)
distributed = False
if 'WORLD_SIZE' in os.environ:
distributed = int(os.environ['WORLD_SIZE']) > 1
if distributed:
torch.cuda.set_device(args.local_rank)
torch.distributed.init_process_group(backend='nccl', init_method='env://')
dist_print('start testing...')
assert cfg.backbone in ['18','34','50','101','152','50next','101next','50wide','101wide']
if cfg.dataset == 'CULane':
cls_num_per_lane = 18
elif cfg.dataset == 'Tusimple':
cls_num_per_lane = 56
else:
raise NotImplementedError
net = parsingNet(pretrained=False, backbone=cfg.backbone, cls_dim=(cfg.griding_num+1, cls_num_per_lane,
cfg.num_lanes), use_aux=False).to(device)
# cfg.num_lanes), use_aux=False).cuda()
# we don't need auxiliary segmentation in testing
state_dict = torch.load(cfg.test_model, map_location='cuda')['model']
compatible_state_dict = {}
for k, v in state_dict.items():
if 'module.' in k:
compatible_state_dict[k[7:]] = v
else:
compatible_state_dict[k] = v
net.load_state_dict(compatible_state_dict, strict=False)
if distributed:
net = torch.nn.parallel.DistributedDataParallel(net, device_ids=[args.local_rank])
if not os.path.exists(cfg.test_work_dir):
os.mkdir(cfg.test_work_dir)
eval_lane(net, cfg.dataset, cfg.data_root, cfg.test_work_dir, cfg.griding_num, False, distributed)

146
algorithms/lane_ufld/code/UFLD/test_onnx.py Executable file
View File

@@ -0,0 +1,146 @@
import torch, os, cv2
from model.model import parsingNet
from utils.common import merge_config
from utils.dist_utils import dist_print
import torch
import scipy.special, tqdm
import numpy as np
import torchvision.transforms as transforms
from data.dataset import LaneTestDataset
from data.constant import culane_row_anchor, tusimple_row_anchor
from scipy.optimize import curve_fit
from lane_show import is_in_poly, handle_point, poly_fitting, draw_values
import time
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
# 自定义函数 e指数形式
def func(x, a, b, c):
return a * np.sqrt(x) * (b * np.square(x) + c)
def get_curve_fit(x, y):
# 非线性最小二乘法拟合
popt, pcov = curve_fit(func, x, y)
# 获取popt里面是拟合系数
# print(popt)
a = popt[0]
b = popt[1]
c = popt[2]
yvals = func(x, a, b, c) # 拟合y值
# print('popt:', popt)
# print('系数a:', a)
# print('系数b:', b)
# print('系数c:', c)
# print('系数pcov:', pcov)
# print('系数yvals:', yvals)
return yvals
if __name__ == "__main__":
torch.backends.cudnn.benchmark = True
args, cfg = merge_config()
dist_print('start testing...')
from model.backbone import SUPPORTED_BACKBONES
assert cfg.backbone in SUPPORTED_BACKBONES
if cfg.dataset == 'CULane':
cls_num_per_lane = 18
elif cfg.dataset == 'Tusimple':
cls_num_per_lane = 56
else:
raise NotImplementedError
# blob = cv.dnn.blobFromImage(img_t, scalefactor=1.0, swapRB=True, crop=False) # 将image转化为 1x3x64x64 格式输入模型中
net = cv2.dnn.readNetFromONNX("./model/tusimple_18.onnx")
# model = onnx.load('./model/tusimple_18.onnx')
# net.setInput(blob)
img_transforms = transforms.Compose([
transforms.Resize((288, 800)),
transforms.ToTensor(),
transforms.Normalize((0.485, 0.456, 0.406), (0.229, 0.224, 0.225)),
])
if cfg.dataset == 'CULane':
splits = ['test0_normal.txt']
# splits = ['test0_normal.txt', 'test1_crowd.txt', 'test2_hlight.txt', 'test3_shadow.txt', 'test4_noline.txt', 'test5_arrow.txt', 'test6_curve.txt', 'test7_cross.txt', 'test8_night.txt']
datasets = [LaneTestDataset(cfg.data_root, os.path.join(cfg.data_root, 'list/test_split/' + split),
img_transform=img_transforms) for split in splits]
img_w, img_h = 1280, 720
# img_w, img_h = 1640, 590
row_anchor = culane_row_anchor
elif cfg.dataset == 'Tusimple':
splits = ['test4.txt']
datasets = [LaneTestDataset(cfg.data_root, os.path.join(cfg.data_root, split), img_transform=img_transforms) for
split in splits]
# img_w, img_h = 998, 560
# img_w, img_h = 960, 546
img_w, img_h = 1280, 720
row_anchor = tusimple_row_anchor
else:
raise NotImplementedError
for split, dataset in zip(splits, datasets):
loader = torch.utils.data.DataLoader(dataset, batch_size=1, shuffle=False, num_workers=1)
fourcc = cv2.VideoWriter_fourcc(*'MJPG')
print(split[:-3] + 'avi')
vout = cv2.VideoWriter(split[:-3] + 'avi', fourcc, 15.0, (img_w, img_h))
for i, data in enumerate(tqdm.tqdm(loader)):
imgs, names = data
print(imgs.shape, names)
imgs = imgs.cuda()
print(type(imgs))
# imgs = imgs.to(device)
with torch.no_grad():
start_t = time.time()
imgs = imgs.numpy()
blob = cv2.dnn.blobFromImage(imgs, scalefactor=1.0, swapRB=False, crop=False)
net.setInput(blob)
out = net(imgs)
end_t = time.time()
count_t = end_t - start_t
print('the pre time is : ', count_t)
col_sample = np.linspace(0, 800 - 1, cfg.griding_num)
col_sample_w = col_sample[1] - col_sample[0]
out_j = out[0].data.cpu().numpy()
out_j = out_j[:, ::-1, :]
prob = scipy.special.softmax(out_j[:-1, :, :], axis=0)
idx = np.arange(cfg.griding_num) + 1
idx = idx.reshape(-1, 1, 1)
loc = np.sum(prob * idx, axis=0)
out_j = np.argmax(out_j, axis=0)
loc[out_j == cfg.griding_num] = 0
out_j = loc
print('out:', len(out_j), out_j.shape)
x_list = []
y_list = []
# import pdb; pdb.set_trace()
vis = cv2.imread(os.path.join(cfg.data_root, names[0]))
for i in range(out_j.shape[1]):
# print(out_j.shape[1])
if np.sum(out_j[:, i] != 0) > 2:
poly = [[50, 50], [50, 719], [1250, 50], [1250, 719]] # ROI区域
lane_x = []
lane_y = []
for k in range(out_j.shape[0]):
# print(out_j.shape[0])
if out_j[k, i] > 0:
ppp = (int(out_j[k, i] * col_sample_w * img_w / 800) - 1,
int(img_h * (row_anchor[cls_num_per_lane - 1 - k] / 288)) - 1)
is_in = is_in_poly(ppp, poly)
if is_in == True:
# 将处理后的点坐标添如一个空列表做拟合用
lane_x.append(ppp[0])
lane_y.append(ppp[1])
cv2.circle(vis, ppp, 5, (0, 255, 0), -1)
lx, ly, rx, ry = handle_point(lane_x, lane_y)
# print(lx, ly, rx, ry)
curvature, distance_from_center = poly_fitting(lx, ly, rx, ry)
draw_values(vis, curvature, distance_from_center)
cv2.imshow('vis', vis)
cv2.waitKey(1)
vout.write(vis)
vout.release()

211
algorithms/lane_ufld/code/UFLD/train.py Executable file
View File

@@ -0,0 +1,211 @@
import os
if "CUDA_VISIBLE_DEVICES" not in os.environ:
os.environ["CUDA_VISIBLE_DEVICES"] = ""
import torch, datetime
import numpy as np
from model.model import parsingNet
from data.dataloader import get_train_loader
from utils.dist_utils import dist_print, dist_tqdm, is_main_process, DistSummaryWriter
from utils.factory import get_metric_dict, get_loss_dict, get_optimizer, get_scheduler
from utils.metrics import MultiLabelAcc, AccTopk, Metric_mIoU, update_metrics, reset_metrics
from utils.common import merge_config, save_model, cp_projects, load_checkpoint, checkpoint_state_dict
from utils.common import get_work_dir, get_logger
from utils.dataset_packs import resolve_train_list
import time
#from prun_model import *
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
top1_list=[]
top1_max=0
torch.cuda.is_available()
print(torch.cuda.is_available())
def inference(net, data_label, use_aux):
if use_aux:
img, cls_label, seg_label = data_label
img = img.to(device)
cls_label = cls_label.long().to(device)
seg_label = seg_label.long().to(device)
cls_out, seg_out = net(img)
return {'cls_out': cls_out, 'cls_label': cls_label, 'seg_out':seg_out, 'seg_label': seg_label}
else:
img, cls_label = data_label
img = img.to(device)
cls_label = cls_label.long().to(device)
cls_out = net(img)
return {'cls_out': cls_out, 'cls_label': cls_label}
def resolve_val_data(results, use_aux):
results['cls_out'] = torch.argmax(results['cls_out'], dim=1)
if use_aux:
results['seg_out'] = torch.argmax(results['seg_out'], dim=1)
return results
def calc_loss(loss_dict, results, logger, global_step):
loss = 0
for i in range(len(loss_dict['name'])):
data_src = loss_dict['data_src'][i]
datas = [results[src] for src in data_src]
loss_cur = loss_dict['op'][i](*datas)
if global_step % 20 == 0:
logger.add_scalar('loss/'+loss_dict['name'][i], loss_cur, global_step)
loss += loss_cur * loss_dict['weight'][i]
return loss
def train(net, data_loader, loss_dict, optimizer, scheduler,logger, epoch, metric_dict, use_aux):
net.train()
progress_bar = dist_tqdm(train_loader)
t_data_0 = time.time()
for b_idx, data_label in enumerate(progress_bar):
# print(b_idx, len(data_label))
t_data_1 = time.time()
reset_metrics(metric_dict)
global_step = epoch * len(data_loader) + b_idx
t_net_0 = time.time()
results = inference(net, data_label, use_aux)
loss = calc_loss(loss_dict, results, logger, global_step)
optimizer.zero_grad()
loss.backward()
optimizer.step()
scheduler.step(global_step)
t_net_1 = time.time()
results = resolve_val_data(results, use_aux)
update_metrics(metric_dict, results)
if global_step % 20 == 0:
for me_name, me_op in zip(metric_dict['name'], metric_dict['op']):
logger.add_scalar('metric/' + me_name, me_op.get(), global_step=global_step)
logger.add_scalar('meta/lr', optimizer.param_groups[0]['lr'], global_step=global_step)
if hasattr(progress_bar, 'set_postfix'):
for me_name, me_op in zip(metric_dict['name'], metric_dict['op']):
if (me_name == 'top1' and b_idx==progress_bar.total-1):
top1_list.append(me_op.get())
kwargs = {me_name: '%.3f' % me_op.get() for me_name, me_op in zip(metric_dict['name'], metric_dict['op'])}
progress_bar.set_postfix(loss = '%.3f' % float(loss),
data_time = '%.3f' % float(t_data_1 - t_data_0),
net_time = '%.3f' % float(t_net_1 - t_net_0),
**kwargs)
t_data_0 = time.time()
if __name__ == "__main__":
torch.backends.cudnn.benchmark = True
args, cfg = merge_config()
work_dir = get_work_dir(cfg)
distributed = False
# distributed = True
if 'WORLD_SIZE' in os.environ:
distributed = int(os.environ['WORLD_SIZE']) > 1
if distributed:
if not torch.cuda.is_available():
raise RuntimeError("Distributed training requires CUDA")
torch.cuda.set_device(args.local_rank)
torch.distributed.init_process_group(backend='nccl', init_method='env://')
dist_print(datetime.datetime.now().strftime('[%Y/%m/%d %H:%M:%S]') + ' start training...')
dist_print(cfg)
from model.backbone import SUPPORTED_BACKBONES
assert cfg.backbone in SUPPORTED_BACKBONES, f'backbone must be one of {SUPPORTED_BACKBONES}'
train_list = resolve_train_list(cfg)
dist_print("train_list:", train_list)
num_workers = int(os.environ.get("UFLD_NUM_WORKERS", "4" if device.type == "cuda" else "0"))
train_loader, cls_num_per_lane = get_train_loader(
cfg.batch_size, cfg.data_root, cfg.griding_num, cfg.dataset, cfg.use_aux,
distributed, cfg.num_lanes, train_list=train_list, num_workers=num_workers,
)
net = parsingNet(
pretrained=True, backbone=cfg.backbone,
cls_dim=(cfg.griding_num + 1, cls_num_per_lane, cfg.num_lanes), use_aux=cfg.use_aux,
).to(device)
if device.type == "cuda" and torch.cuda.device_count() > 1 and not distributed:
print("Let's use", torch.cuda.device_count(), "GPUs!")
net = torch.nn.DataParallel(net)
#
# net = parsingNet(pretrained=True, backbone=cfg.backbone,
# cls_dim=(cfg.griding_num + 1, cls_num_per_lane, cfg.num_lanes), use_aux=cfg.use_aux).to(device)
if distributed:
net = torch.nn.parallel.DistributedDataParallel(net, device_ids=[args.local_rank])
optimizer = get_optimizer(net, cfg)
if cfg.finetune is not None:
dist_print('finetune from ', cfg.finetune)
state_all = checkpoint_state_dict(cfg.finetune)
state_clip = {} # only use backbone parameters
for k, v in state_all.items():
if 'model' in k:
state_clip[k] = v
net.load_state_dict(state_clip, strict=False)
if cfg.resume is not None:
dist_print('==> Resume model from ' + cfg.resume)
resume_dict = load_checkpoint(cfg.resume, map_location='cpu')
model = resume_dict['model']
if isinstance(model, torch.nn.Module):
net.load_state_dict(model.state_dict(), strict=False)
else:
net.load_state_dict(model, strict=False)
if 'optimizer' in resume_dict.keys():
optimizer.load_state_dict(resume_dict['optimizer'])
resume_epoch = int(os.path.split(cfg.resume)[1][2:5]) + 1
else:
resume_epoch = 0
scheduler = get_scheduler(optimizer, cfg, len(train_loader))
dist_print(len(train_loader))
metric_dict = get_metric_dict(cfg)
loss_dict = get_loss_dict(cfg)
logger = get_logger(work_dir, cfg)
cp_projects(args.auto_backup, work_dir)
file = open(work_dir + '/precision_record.txt', 'w')
for epoch in range(resume_epoch, cfg.epoch):
train(net, train_loader, loss_dict, optimizer, scheduler, logger, epoch, metric_dict, cfg.use_aux)
# if epoch % 2==0:
#if epoch > 0:
#print('save the ', epoch, 'model')
#save_model(net, optimizer, epoch, work_dir, distributed)
if(epoch==0):
top1_max=top1_list[-1]
save_model(net, optimizer, epoch, work_dir, distributed)
print('epcoh=0 and save best pth, the epoch: ', epoch, 'done')
if(epoch!=0 and top1_list[-1]>top1_max):
print('top1_list[-1]: ', top1_list[-1])
print('top1_max: ', top1_max)
save_model(net, optimizer, epoch, work_dir, distributed)
print('save best pth, the epoch: ', epoch, 'done')
file.write('top1_list[-1]: '+ str(top1_list[-1])+"\n")
file.write('top1_max: ' + str(top1_max) + "\n")
top1_max = top1_list[-1]
else:
save_model(net, optimizer, epoch, work_dir, distributed)
print('donnot need to save best pth, the epoch: ', epoch, 'done')
file.close()
logger.close()

154
algorithms/lane_ufld/code/UFLD/train_bk.py Executable file
View File

@@ -0,0 +1,154 @@
import os
os.environ["CUDA_VISIBLE_DEVICES"] = '0,1'
import torch, datetime
import numpy as np
from model.model import parsingNet
from data.dataloader import get_train_loader
from utils.dist_utils import dist_print, dist_tqdm, is_main_process, DistSummaryWriter
from utils.factory import get_metric_dict, get_loss_dict, get_optimizer, get_scheduler
from utils.metrics import MultiLabelAcc, AccTopk, Metric_mIoU, update_metrics, reset_metrics
from utils.common import merge_config, save_model, cp_projects
from utils.common import get_work_dir, get_logger
import time
torch.cuda.is_available()
def inference(net, data_label, use_aux):
if use_aux:
img, cls_label, seg_label = data_label
img, cls_label, seg_label = img.cuda(), cls_label.long().cuda(), seg_label.long().cuda()
cls_out, seg_out = net(img)
return {'cls_out': cls_out, 'cls_label': cls_label, 'seg_out':seg_out, 'seg_label': seg_label}
else:
img, cls_label = data_label
img, cls_label = img.cuda(), cls_label.long().cuda()
cls_out = net(img)
return {'cls_out': cls_out, 'cls_label': cls_label}
def resolve_val_data(results, use_aux):
results['cls_out'] = torch.argmax(results['cls_out'], dim=1)
if use_aux:
results['seg_out'] = torch.argmax(results['seg_out'], dim=1)
return results
def calc_loss(loss_dict, results, logger, global_step):
loss = 0
for i in range(len(loss_dict['name'])):
data_src = loss_dict['data_src'][i]
datas = [results[src] for src in data_src]
loss_cur = loss_dict['op'][i](*datas)
if global_step % 20 == 0:
logger.add_scalar('loss/'+loss_dict['name'][i], loss_cur, global_step)
loss += loss_cur * loss_dict['weight'][i]
return loss
def train(net, data_loader, loss_dict, optimizer, scheduler,logger, epoch, metric_dict, use_aux):
net.train()
progress_bar = dist_tqdm(train_loader)
t_data_0 = time.time()
for b_idx, data_label in enumerate(progress_bar):
# print(b_idx, len(data_label))
t_data_1 = time.time()
reset_metrics(metric_dict)
global_step = epoch * len(data_loader) + b_idx
t_net_0 = time.time()
results = inference(net, data_label, use_aux)
loss = calc_loss(loss_dict, results, logger, global_step)
optimizer.zero_grad()
loss.backward()
optimizer.step()
scheduler.step(global_step)
t_net_1 = time.time()
results = resolve_val_data(results, use_aux)
update_metrics(metric_dict, results)
if global_step % 20 == 0:
for me_name, me_op in zip(metric_dict['name'], metric_dict['op']):
logger.add_scalar('metric/' + me_name, me_op.get(), global_step=global_step)
logger.add_scalar('meta/lr', optimizer.param_groups[0]['lr'], global_step=global_step)
if hasattr(progress_bar, 'set_postfix'):
kwargs = {me_name: '%.3f' % me_op.get() for me_name, me_op in zip(metric_dict['name'], metric_dict['op'])}
progress_bar.set_postfix(loss = '%.3f' % float(loss),
data_time = '%.3f' % float(t_data_1 - t_data_0),
net_time = '%.3f' % float(t_net_1 - t_net_0),
**kwargs)
t_data_0 = time.time()
if __name__ == "__main__":
torch.backends.cudnn.benchmark = True
args, cfg = merge_config()
work_dir = get_work_dir(cfg)
distributed = False
# distributed = True
if 'WORLD_SIZE' in os.environ:
distributed = int(os.environ['WORLD_SIZE']) > 1
if distributed:
torch.cuda.set_device(args.local_rank)
torch.distributed.init_process_group(backend='nccl', init_method='env://')
dist_print(datetime.datetime.now().strftime('[%Y/%m/%d %H:%M:%S]') + ' start training...')
dist_print(cfg)
assert cfg.backbone in ['18', '34', '50', '101', '152', '50next', '101next', '50wide', '101wide']
train_loader, cls_num_per_lane = get_train_loader(cfg.batch_size, cfg.data_root, cfg.griding_num, cfg.dataset, cfg.use_aux, distributed, cfg.num_lanes)
net = parsingNet(pretrained=True, backbone=cfg.backbone, cls_dim=(cfg.griding_num+1, cls_num_per_lane, cfg.num_lanes),use_aux=cfg.use_aux).cuda()
if torch.cuda.device_count() > 1:
print("Let's use", torch.cuda.device_count(), "GPUs!")
net = torch.nn.DataParallel(net)
net.cuda()
if distributed:
net = torch.nn.parallel.DistributedDataParallel(net, device_ids=[args.local_rank])
optimizer = get_optimizer(net, cfg)
if cfg.finetune is not None:
dist_print('finetune from ', cfg.finetune)
state_all = torch.load(cfg.finetune)['model']
state_clip = {} # only use backbone parameters
for k, v in state_all.items():
if 'model' in k:
state_clip[k] = v
net.load_state_dict(state_clip, strict=False)
if cfg.resume is not None:
dist_print('==> Resume model from ' + cfg.resume)
resume_dict = torch.load(cfg.resume, map_location='cpu')
net.load_state_dict(resume_dict['model'])
if 'optimizer' in resume_dict.keys():
optimizer.load_state_dict(resume_dict['optimizer'])
resume_epoch = int(os.path.split(cfg.resume)[1][2:5]) + 1
else:
resume_epoch = 0
scheduler = get_scheduler(optimizer, cfg, len(train_loader))
dist_print(len(train_loader))
metric_dict = get_metric_dict(cfg)
loss_dict = get_loss_dict(cfg)
logger = get_logger(work_dir, cfg)
cp_projects(args.auto_backup, work_dir)
for epoch in range(resume_epoch, cfg.epoch):
train(net, train_loader, loss_dict, optimizer, scheduler, logger, epoch, metric_dict, cfg.use_aux)
save_model(net, optimizer, epoch, work_dir, distributed)
print('save the ', epoch, 'model')
logger.close()

172
algorithms/lane_ufld/code/UFLD/utils/common.py Executable file
View File

@@ -0,0 +1,172 @@
import os, argparse
from utils.dist_utils import is_main_process, dist_print, DistSummaryWriter
from utils.config import Config
import torch
import time
def load_checkpoint(path, map_location='cpu'):
"""Load legacy UFLD checkpoints (may contain DataParallel); PyTorch 2.6+ needs weights_only=False."""
return torch.load(path, map_location=map_location, weights_only=False)
def checkpoint_state_dict(path, map_location='cpu'):
"""Return state_dict from a .pth file (handles DataParallel module or raw state_dict)."""
ckpt = load_checkpoint(path, map_location=map_location)
model = ckpt['model'] if isinstance(ckpt, dict) and 'model' in ckpt else ckpt
if isinstance(model, torch.nn.Module):
model = model.state_dict()
compatible = {}
for k, v in model.items():
if 'module.' in k:
compatible[k[7:]] = v
else:
compatible[k] = v
return compatible
def str2bool(v):
if isinstance(v, bool):
return v
if v.lower() in ('yes', 'true', 't', 'y', '1'):
return True
elif v.lower() in ('no', 'false', 'f', 'n', '0'):
return False
else:
raise argparse.ArgumentTypeError('Boolean value expected.')
def get_args():
parser = argparse.ArgumentParser()
parser.add_argument('config', help = 'path to config file')
parser.add_argument('--local_rank', type=int, default=0)
parser.add_argument('--dataset', default = None, type = str)
parser.add_argument('--data_root', default = None, type = str)
parser.add_argument('--epoch', default = None, type = int)
parser.add_argument('--batch_size', default = None, type = int)
parser.add_argument('--optimizer', default = None, type = str)
parser.add_argument('--learning_rate', default = None, type = float)
parser.add_argument('--weight_decay', default = None, type = float)
parser.add_argument('--momentum', default = None, type = float)
parser.add_argument('--scheduler', default = None, type = str)
parser.add_argument('--steps', default = None, type = int, nargs='+')
parser.add_argument('--gamma', default = None, type = float)
parser.add_argument('--warmup', default = None, type = str)
parser.add_argument('--warmup_iters', default = None, type = int)
parser.add_argument('--backbone', default = None, type = str)
parser.add_argument('--griding_num', default = None, type = int)
parser.add_argument('--use_aux', default = None, type = str2bool)
parser.add_argument('--sim_loss_w', default = None, type = float)
parser.add_argument('--shp_loss_w', default = None, type = float)
parser.add_argument('--note', default = None, type = str)
parser.add_argument('--log_path', default = None, type = str)
parser.add_argument('--finetune', default = None, type = str)
parser.add_argument('--resume', default = None, type = str)
parser.add_argument('--test_model', default = None, type = str)
parser.add_argument('--test_work_dir', default = None, type = str)
parser.add_argument('--num_lanes', default = None, type = int)
parser.add_argument('--test_list', default = None, type = str,
help='relative path under data_root, e.g. list/test.txt')
parser.add_argument('--skip_eval', default = None, type = str2bool,
help='only run inference, skip metric evaluation')
parser.add_argument('--auto_backup', action='store_true', help='automatically backup current code in the log path')
return parser
def merge_config():
args = get_args().parse_args()
cfg = Config.fromfile(args.config)
items = ['dataset','data_root','epoch','batch_size','optimizer','learning_rate',
'weight_decay','momentum','scheduler','steps','gamma','warmup','warmup_iters',
'use_aux','griding_num','backbone','sim_loss_w','shp_loss_w','note','log_path',
'finetune','resume', 'test_model','test_work_dir', 'num_lanes', 'test_list', 'skip_eval']
for item in items:
if getattr(args, item) is not None:
dist_print('merge ', item, ' config')
setattr(cfg, item, getattr(args, item))
return args, cfg
def save_model(net, optimizer, epoch, save_path, distributed):
if is_main_process():
# model_state_dict = net.state_dict()
# state = {'model': model_state_dict, 'optimizer': optimizer.state_dict()}
# # state = {'model': net, 'optimizer': optimizer}
# assert os.path.exists(save_path)
# #model_path = os.path.join(save_path, 'ep%03d.pth' % epoch)
# model_path = os.path.join(save_path, 'best.pth')
# torch.save(state, model_path)
model_state_all = net
state = {'model': model_state_all, 'optimizer': optimizer.state_dict(),'epoch':epoch}
assert os.path.exists(save_path)
model_path = os.path.join(save_path, 'best.pth')
torch.save(state, model_path)
def save_model_pruned_model(model, optimizer, epoch, save_path, distributed):
state = {'model': model.state_dict(), 'optimizer': optimizer.state_dict(),'epoch':epoch}
assert os.path.exists(save_path)
model_path = os.path.join(save_path, 'pruned_model.pth')
torch.save(state, model_path)
def save_model_fine_tune(net, optimizer, epoch, save_path, distributed):
if is_main_process():
# model_state_dict = net.state_dict()
# state = {'model': model_state_dict, 'optimizer': optimizer.state_dict()}
# # state = {'model': net, 'optimizer': optimizer}
# assert os.path.exists(save_path)
# #model_path = os.path.join(save_path, 'ep%03d.pth' % epoch)
# model_path = os.path.join(save_path, 'best.pth')
# torch.save(state, model_path)
model_state_dict = net.state_dict() ##注意此处
state = {'model': model_state_dict, 'optimizer': optimizer.state_dict(),'epoch':epoch}
assert os.path.exists(save_path)
model_path = os.path.join(save_path, 'fine_tune_model.pth')
torch.save(state, model_path)
import pathspec
def cp_projects(auto_backup, to_path):
if is_main_process() and auto_backup:
with open('./.gitignore','r') as fp:
ign = fp.read()
ign += '\n.git'
spec = pathspec.PathSpec.from_lines(pathspec.patterns.GitWildMatchPattern, ign.splitlines())
all_files = {os.path.join(root,name) for root,dirs,files in os.walk('./') for name in files}
matches = spec.match_files(all_files)
matches = set(matches)
to_cp_files = all_files - matches
dist_print('Copying projects to '+ to_path + ' for backup')
t0 = time.time()
warning_flag = True
for f in to_cp_files:
dirs = os.path.join(to_path,'code',os.path.split(f[2:])[0])
if not os.path.exists(dirs):
os.makedirs(dirs)
os.system('cp %s %s'%(f,os.path.join(to_path,'code',f[2:])))
elapsed_time = time.time() - t0
if elapsed_time > 5 and warning_flag:
dist_print('If the program is stuck, it might be copying large files in this directory. please don\'t set --auto_backup. Or please make you working directory clean, i.e, don\'t place large files like dataset, log results under this directory.')
warning_flag = False
import datetime,os
def get_work_dir(cfg):
now = datetime.datetime.now().strftime('%Y%m%d_%H%M%S')
hyper_param_str = '_lr_%1.0e_b_%d' % (cfg.learning_rate, cfg.batch_size)
log_path = cfg.log_path or './log'
print(log_path, now + hyper_param_str + cfg.note)
work_dir = os.path.join(log_path, now + hyper_param_str + cfg.note)
return work_dir
def get_logger(work_dir, cfg):
logger = DistSummaryWriter(work_dir)
config_txt = os.path.join(work_dir, 'cfg.txt')
if is_main_process():
with open(config_txt, 'w') as fp:
fp.write(str(cfg))
return logger

121
algorithms/lane_ufld/code/UFLD/utils/common_bk.py Executable file
View File

@@ -0,0 +1,121 @@
import os, argparse
from utils.dist_utils import is_main_process, dist_print, DistSummaryWriter
from utils.config import Config
import torch
import time
def str2bool(v):
if isinstance(v, bool):
return v
if v.lower() in ('yes', 'true', 't', 'y', '1'):
return True
elif v.lower() in ('no', 'false', 'f', 'n', '0'):
return False
else:
raise argparse.ArgumentTypeError('Boolean value expected.')
def get_args():
parser = argparse.ArgumentParser()
parser.add_argument('config', help = 'path to config file')
parser.add_argument('--local_rank', type=int, default=0)
parser.add_argument('--dataset', default = None, type = str)
parser.add_argument('--data_root', default = None, type = str)
parser.add_argument('--epoch', default = None, type = int)
parser.add_argument('--batch_size', default = None, type = int)
parser.add_argument('--optimizer', default = None, type = str)
parser.add_argument('--learning_rate', default = None, type = float)
parser.add_argument('--weight_decay', default = None, type = float)
parser.add_argument('--momentum', default = None, type = float)
parser.add_argument('--scheduler', default = None, type = str)
parser.add_argument('--steps', default = None, type = int, nargs='+')
parser.add_argument('--gamma', default = None, type = float)
parser.add_argument('--warmup', default = None, type = str)
parser.add_argument('--warmup_iters', default = None, type = int)
parser.add_argument('--backbone', default = None, type = str)
parser.add_argument('--griding_num', default = None, type = int)
parser.add_argument('--use_aux', default = None, type = str2bool)
parser.add_argument('--sim_loss_w', default = None, type = float)
parser.add_argument('--shp_loss_w', default = None, type = float)
parser.add_argument('--note', default = None, type = str)
parser.add_argument('--log_path', default = None, type = str)
parser.add_argument('--finetune', default = None, type = str)
parser.add_argument('--resume', default = None, type = str)
parser.add_argument('--test_model', default = None, type = str)
parser.add_argument('--test_work_dir', default = None, type = str)
parser.add_argument('--num_lanes', default = None, type = int)
parser.add_argument('--auto_backup', action='store_true', help='automatically backup current code in the log path')
return parser
def merge_config():
args = get_args().parse_args()
cfg = Config.fromfile(args.config)
items = ['dataset','data_root','epoch','batch_size','optimizer','learning_rate',
'weight_decay','momentum','scheduler','steps','gamma','warmup','warmup_iters',
'use_aux','griding_num','backbone','sim_loss_w','shp_loss_w','note','log_path',
'finetune','resume', 'test_model','test_work_dir', 'num_lanes']
for item in items:
if getattr(args, item) is not None:
dist_print('merge ', item, ' config')
setattr(cfg, item, getattr(args, item))
return args, cfg
def save_model(net, optimizer, epoch, save_path, distributed):
if is_main_process():
model_state_dict = net.state_dict()
state = {'model': model_state_dict, 'optimizer': optimizer.state_dict()}
# state = {'model': net, 'optimizer': optimizer}
assert os.path.exists(save_path)
model_path = os.path.join(save_path, 'ep%03d.pth' % epoch)
torch.save(state, model_path)
import pathspec
def cp_projects(auto_backup, to_path):
if is_main_process() and auto_backup:
with open('./.gitignore','r') as fp:
ign = fp.read()
ign += '\n.git'
spec = pathspec.PathSpec.from_lines(pathspec.patterns.GitWildMatchPattern, ign.splitlines())
all_files = {os.path.join(root,name) for root,dirs,files in os.walk('./') for name in files}
matches = spec.match_files(all_files)
matches = set(matches)
to_cp_files = all_files - matches
dist_print('Copying projects to '+ to_path + ' for backup')
t0 = time.time()
warning_flag = True
for f in to_cp_files:
dirs = os.path.join(to_path,'code',os.path.split(f[2:])[0])
if not os.path.exists(dirs):
os.makedirs(dirs)
os.system('cp %s %s'%(f,os.path.join(to_path,'code',f[2:])))
elapsed_time = time.time() - t0
if elapsed_time > 5 and warning_flag:
dist_print('If the program is stuck, it might be copying large files in this directory. please don\'t set --auto_backup. Or please make you working directory clean, i.e, don\'t place large files like dataset, log results under this directory.')
warning_flag = False
import datetime,os
def get_work_dir(cfg):
now = datetime.datetime.now().strftime('%Y%m%d_%H%M%S')
hyper_param_str = '_lr_%1.0e_b_%d' % (cfg.learning_rate, cfg.batch_size)
print(cfg.log_path, now + hyper_param_str + cfg.note)
work_dir = os.path.join(cfg.log_path, now + hyper_param_str + cfg.note)
return work_dir
def get_logger(work_dir, cfg):
logger = DistSummaryWriter(work_dir)
config_txt = os.path.join(work_dir, 'cfg.txt')
if is_main_process():
with open(config_txt, 'w') as fp:
fp.write(str(cfg))
return logger

352
algorithms/lane_ufld/code/UFLD/utils/config.py Executable file
View File

@@ -0,0 +1,352 @@
import json
import os.path as osp
import shutil
import sys
import tempfile
from argparse import Action, ArgumentParser
from collections import abc
from importlib import import_module
from addict import Dict
BASE_KEY = '_base_'
DELETE_KEY = '_delete_'
class ConfigDict(Dict):
def __missing__(self, name):
raise KeyError(name)
def __getattr__(self, name):
try:
value = super(ConfigDict, self).__getattr__(name)
except KeyError:
ex = AttributeError(f"'{self.__class__.__name__}' object has no "
f"attribute '{name}'")
except Exception as e:
ex = e
else:
return value
raise ex
def add_args(parser, cfg, prefix=''):
for k, v in cfg.items():
if isinstance(v, str):
parser.add_argument('--' + prefix + k)
elif isinstance(v, int):
parser.add_argument('--' + prefix + k, type=int)
elif isinstance(v, float):
parser.add_argument('--' + prefix + k, type=float)
elif isinstance(v, bool):
parser.add_argument('--' + prefix + k, action='store_true')
elif isinstance(v, dict):
add_args(parser, v, prefix + k + '.')
elif isinstance(v, abc.Iterable):
parser.add_argument('--' + prefix + k, type=type(v[0]), nargs='+')
else:
print(f'cannot parse key {prefix + k} of type {type(v)}')
return parser
class Config(object):
"""A facility for config and config files.
It supports common file formats as configs: python/json/yaml. The interface
is the same as a dict object and also allows access config values as
attributes.
Example:
>>> cfg = Config(dict(a=1, b=dict(b1=[0, 1])))
>>> cfg.a
1
>>> cfg.b
{'b1': [0, 1]}
>>> cfg.b.b1
[0, 1]
>>> cfg = Config.fromfile('tests/data/config/a.py')
>>> cfg.filename
"/home/kchen/projects/mmcv/tests/data/config/a.py"
>>> cfg.item4
'test'
>>> cfg
"Config [path: /home/kchen/projects/mmcv/tests/data/config/a.py]: "
"{'item1': [1, 2], 'item2': {'a': 0}, 'item3': True, 'item4': 'test'}"
"""
@staticmethod
def _file2dict(filename):
filename = osp.abspath(osp.expanduser(filename))
if filename.endswith('.py'):
with tempfile.TemporaryDirectory() as temp_config_dir:
temp_config_file = tempfile.NamedTemporaryFile(
dir=temp_config_dir, suffix='.py')
temp_config_name = osp.basename(temp_config_file.name)
# close temp file
temp_config_file.close()
shutil.copyfile(filename,
osp.join(temp_config_dir, temp_config_name))
temp_module_name = osp.splitext(temp_config_name)[0]
sys.path.insert(0, temp_config_dir)
mod = import_module(temp_module_name)
sys.path.pop(0)
cfg_dict = {
name: value
for name, value in mod.__dict__.items()
if not name.startswith('__')
}
# delete imported module
del sys.modules[temp_module_name]
elif filename.endswith(('.yml', '.yaml', '.json')):
import mmcv
cfg_dict = mmcv.load(filename)
else:
raise IOError('Only py/yml/yaml/json type are supported now!')
cfg_text = filename + '\n'
with open(filename, 'r') as f:
cfg_text += f.read()
if BASE_KEY in cfg_dict:
cfg_dir = osp.dirname(filename)
base_filename = cfg_dict.pop(BASE_KEY)
base_filename = base_filename if isinstance(
base_filename, list) else [base_filename]
cfg_dict_list = list()
cfg_text_list = list()
for f in base_filename:
_cfg_dict, _cfg_text = Config._file2dict(osp.join(cfg_dir, f))
cfg_dict_list.append(_cfg_dict)
cfg_text_list.append(_cfg_text)
base_cfg_dict = dict()
for c in cfg_dict_list:
if len(base_cfg_dict.keys() & c.keys()) > 0:
raise KeyError('Duplicate key is not allowed among bases')
base_cfg_dict.update(c)
base_cfg_dict = Config._merge_a_into_b(cfg_dict, base_cfg_dict)
cfg_dict = base_cfg_dict
# merge cfg_text
cfg_text_list.append(cfg_text)
cfg_text = '\n'.join(cfg_text_list)
return cfg_dict, cfg_text
@staticmethod
def _merge_a_into_b(a, b):
# merge dict `a` into dict `b` (non-inplace). values in `a` will
# overwrite `b`.
# copy first to avoid inplace modification
b = b.copy()
for k, v in a.items():
if isinstance(v, dict) and k in b and not v.pop(DELETE_KEY, False):
if not isinstance(b[k], dict):
raise TypeError(
f'{k}={v} in child config cannot inherit from base '
f'because {k} is a dict in the child config but is of '
f'type {type(b[k])} in base config. You may set '
f'`{DELETE_KEY}=True` to ignore the base config')
b[k] = Config._merge_a_into_b(v, b[k])
else:
b[k] = v
return b
@staticmethod
def fromfile(filename):
cfg_dict, cfg_text = Config._file2dict(filename)
return Config(cfg_dict, cfg_text=cfg_text, filename=filename)
@staticmethod
def auto_argparser(description=None):
"""Generate argparser from config file automatically (experimental)
"""
partial_parser = ArgumentParser(description=description)
partial_parser.add_argument('config', help='config file path')
cfg_file = partial_parser.parse_known_args()[0].config
cfg = Config.fromfile(cfg_file)
parser = ArgumentParser(description=description)
parser.add_argument('config', help='config file path')
add_args(parser, cfg)
return parser, cfg
def __init__(self, cfg_dict=None, cfg_text=None, filename=None):
if cfg_dict is None:
cfg_dict = dict()
elif not isinstance(cfg_dict, dict):
raise TypeError('cfg_dict must be a dict, but '
f'got {type(cfg_dict)}')
super(Config, self).__setattr__('_cfg_dict', ConfigDict(cfg_dict))
super(Config, self).__setattr__('_filename', filename)
if cfg_text:
text = cfg_text
elif filename:
with open(filename, 'r') as f:
text = f.read()
else:
text = ''
super(Config, self).__setattr__('_text', text)
@property
def filename(self):
return self._filename
@property
def text(self):
return self._text
@property
def pretty_text(self):
indent = 4
def _indent(s_, num_spaces):
s = s_.split('\n')
if len(s) == 1:
return s_
first = s.pop(0)
s = [(num_spaces * ' ') + line for line in s]
s = '\n'.join(s)
s = first + '\n' + s
return s
def _format_basic_types(k, v):
if isinstance(v, str):
v_str = f"'{v}'"
else:
v_str = str(v)
attr_str = f'{str(k)}={v_str}'
attr_str = _indent(attr_str, indent)
return attr_str
def _format_list(k, v):
# check if all items in the list are dict
if all(isinstance(_, dict) for _ in v):
v_str = '[\n'
v_str += '\n'.join(
f'dict({_indent(_format_dict(v_), indent)}),'
for v_ in v).rstrip(',')
attr_str = f'{str(k)}={v_str}'
attr_str = _indent(attr_str, indent) + ']'
else:
attr_str = _format_basic_types(k, v)
return attr_str
def _format_dict(d, outest_level=False):
r = ''
s = []
for idx, (k, v) in enumerate(d.items()):
is_last = idx >= len(d) - 1
end = '' if outest_level or is_last else ','
if isinstance(v, dict):
v_str = '\n' + _format_dict(v)
attr_str = f'{str(k)}=dict({v_str}'
attr_str = _indent(attr_str, indent) + ')' + end
elif isinstance(v, list):
attr_str = _format_list(k, v) + end
else:
attr_str = _format_basic_types(k, v) + end
s.append(attr_str)
r += '\n'.join(s)
return r
cfg_dict = self._cfg_dict.to_dict()
text = _format_dict(cfg_dict, outest_level=True)
return text
def __repr__(self):
return f'Config (path: {self.filename}): {self._cfg_dict.__repr__()}'
def __len__(self):
return len(self._cfg_dict)
def __getattr__(self, name):
return getattr(self._cfg_dict, name)
def __getitem__(self, name):
return self._cfg_dict.__getitem__(name)
def __setattr__(self, name, value):
if isinstance(value, dict):
value = ConfigDict(value)
self._cfg_dict.__setattr__(name, value)
def __setitem__(self, name, value):
if isinstance(value, dict):
value = ConfigDict(value)
self._cfg_dict.__setitem__(name, value)
def __iter__(self):
return iter(self._cfg_dict)
def dump(self):
cfg_dict = super(Config, self).__getattribute__('_cfg_dict')
format_text = json.dumps(cfg_dict, indent=2)
return format_text
def merge_from_dict(self, options):
"""Merge list into cfg_dict
Merge the dict parsed by MultipleKVAction into this cfg.
Examples:
>>> options = {'model.backbone.depth': 50,
... 'model.backbone.with_cp':True}
>>> cfg = Config(dict(model=dict(backbone=dict(type='ResNet'))))
>>> cfg.merge_from_dict(options)
>>> cfg_dict = super(Config, self).__getattribute__('_cfg_dict')
>>> assert cfg_dict == dict(
... model=dict(backbone=dict(depth=50, with_cp=True)))
Args:
options (dict): dict of configs to merge from.
"""
option_cfg_dict = {}
for full_key, v in options.items():
d = option_cfg_dict
key_list = full_key.split('.')
for subkey in key_list[:-1]:
d.setdefault(subkey, ConfigDict())
d = d[subkey]
subkey = key_list[-1]
d[subkey] = v
cfg_dict = super(Config, self).__getattribute__('_cfg_dict')
super(Config, self).__setattr__(
'_cfg_dict', Config._merge_a_into_b(option_cfg_dict, cfg_dict))
class DictAction(Action):
"""
argparse action to split an argument into KEY=VALUE form
on the first = and append to a dictionary. List options should
be passed as comma separated values, i.e KEY=V1,V2,V3
"""
@staticmethod
def _parse_int_float_bool(val):
try:
return int(val)
except ValueError:
pass
try:
return float(val)
except ValueError:
pass
if val.lower() in ['true', 'false']:
return True if val.lower() == 'true' else False
return val
def __call__(self, parser, namespace, values, option_string=None):
options = {}
for kv in values:
key, val = kv.split('=', maxsplit=1)
val = [self._parse_int_float_bool(v) for v in val.split(',')]
if len(val) == 1:
val = val[0]
options[key] = val
setattr(namespace, self.dest, options)

163
algorithms/lane_ufld/code/UFLD/utils/dataset_packs.py Normal file
View File

@@ -0,0 +1,163 @@
"""Resolve train_list from config train_packs (DATASET, DATASET-A, ...)."""
from __future__ import annotations
import json
import os
from pathlib import Path
from utils.dist_utils import dist_print, is_main_process
def parse_gt_line(line: str) -> tuple[str, str] | None:
parts = line.strip().split()
if len(parts) < 2:
return None
return parts[0].lstrip("/"), parts[1].lstrip("/")
def apply_pack_prefix(img: str, msk: str, prefix: str) -> tuple[str, str]:
if not prefix:
return img, msk
if not img.startswith(prefix):
img = prefix + img
if not msk.startswith(prefix):
msk = prefix + msk
return img, msk
def load_registry(data_root: Path) -> dict:
path = data_root / "datasets_registry.json"
if not path.is_file():
return {}
return json.loads(path.read_text(encoding="utf-8"))
def resolve_pack_dir(pack: str, data_root: Path, registry: dict) -> str:
"""Map config name (e.g. DATASET-A) to directory name under data_root."""
aliases = registry.get("aliases", {})
if pack in aliases:
pack = aliases[pack]
pack_dirs = registry.get("pack_dirs", {})
if pack in pack_dirs:
pack = pack_dirs[pack]
pack_path = data_root / pack
if not pack_path.is_dir():
raise FileNotFoundError(
f"pack directory not found: {pack_path} (config train_packs entry: {pack!r})"
)
return pack
def pack_list_path(data_root: Path, pack_dir: str, list_name: str) -> Path:
p = data_root / pack_dir / list_name
if not p.is_file():
raise FileNotFoundError(f"pack list not found: {p}")
return p
def merge_pack_lists(
data_root: Path,
pack_dirs: list[str],
list_name: str,
out_path: Path,
*,
validate: bool = True,
) -> int:
merged: list[tuple[str, str]] = []
seen: set[str] = set()
for pack_dir in pack_dirs:
prefix = f"{pack_dir}/"
list_path = pack_list_path(data_root, pack_dir, list_name)
for line in list_path.read_text(encoding="utf-8", errors="replace").splitlines():
parsed = parse_gt_line(line)
if not parsed:
continue
img, msk = apply_pack_prefix(parsed[0], parsed[1], prefix)
if img in seen:
continue
seen.add(img)
if validate:
if not (data_root / img).is_file():
raise FileNotFoundError(f"missing image: {data_root / img}")
if not (data_root / msk).is_file():
raise FileNotFoundError(f"missing mask: {data_root / msk}")
merged.append((img, msk))
out_path.parent.mkdir(parents=True, exist_ok=True)
out_path.write_text("\n".join(f"{a} {b}" for a, b in merged) + "\n", encoding="utf-8")
return len(merged)
def merged_list_basename(pack_dirs: list[str]) -> str:
safe = "__".join(p.replace("/", "_") for p in pack_dirs)
if len(safe) > 180:
safe = safe[:180]
return f"train__{safe}.txt"
def resolve_train_list(cfg) -> str:
"""
Return train list path relative to cfg.data_root.
- If cfg.train_packs is set: merge packs and return lists_merged/... path.
- Else: use cfg.train_list (default list/train_gt.txt).
"""
train_packs = getattr(cfg, "train_packs", None)
if not train_packs:
return getattr(cfg, "train_list", "list/train_gt.txt")
per_pack_lists: dict = {}
if isinstance(train_packs, str):
train_packs = [p.strip() for p in train_packs.split(",") if p.strip()]
elif isinstance(train_packs, dict):
per_pack_lists = dict(train_packs)
train_packs = list(per_pack_lists.keys())
else:
train_packs = list(train_packs)
data_root = Path(cfg.data_root).resolve()
registry = load_registry(data_root)
pack_dirs = [resolve_pack_dir(p, data_root, registry) for p in train_packs]
list_name = getattr(cfg, "pack_list_name", "list/train_gt.txt")
if per_pack_lists:
# merge with different list per pack — sequential merge
merged_dir = Path(getattr(cfg, "merged_list_dir", "lists_merged"))
out_name = getattr(cfg, "merged_train_list", None) or merged_list_basename(pack_dirs)
out_path = data_root / merged_dir / out_name
if is_main_process():
merged: list[tuple[str, str]] = []
seen: set[str] = set()
for pack, pack_dir in zip(train_packs, pack_dirs):
prefix = f"{pack_dir}/"
rel_list = per_pack_lists.get(pack, list_name)
list_path = pack_list_path(data_root, pack_dir, rel_list)
for line in list_path.read_text(encoding="utf-8", errors="replace").splitlines():
parsed = parse_gt_line(line)
if not parsed:
continue
img, msk = apply_pack_prefix(parsed[0], parsed[1], prefix)
if img in seen:
continue
seen.add(img)
merged.append((img, msk))
out_path.parent.mkdir(parents=True, exist_ok=True)
out_path.write_text("\n".join(f"{a} {b}" for a, b in merged) + "\n", encoding="utf-8")
dist_print(f"merged {len(merged)} samples -> {out_path}")
return str((merged_dir / out_name).as_posix())
merged_dir = Path(getattr(cfg, "merged_list_dir", "lists_merged"))
out_name = getattr(cfg, "merged_train_list", None) or merged_list_basename(pack_dirs)
out_rel = (merged_dir / out_name).as_posix()
out_path = data_root / merged_dir / out_name
force = getattr(cfg, "remerge_train_list", False)
if is_main_process():
if force or not out_path.is_file():
n = merge_pack_lists(data_root, pack_dirs, list_name, out_path, validate=True)
dist_print(f"train_packs {train_packs} -> {n} samples, list={out_rel}")
else:
dist_print(f"reuse merged list: {out_rel}")
return out_rel

173
algorithms/lane_ufld/code/UFLD/utils/dist_utils.py Executable file
View File

@@ -0,0 +1,173 @@
import torch
import torch.distributed as dist
import pickle
def get_world_size():
if not dist.is_available():
return 1
if not dist.is_initialized():
return 1
return dist.get_world_size()
def to_python_float(t):
if hasattr(t, 'item'):
return t.item()
else:
return t[0]
def get_rank():
if not dist.is_available():
return 0
if not dist.is_initialized():
return 0
return dist.get_rank()
def is_main_process():
return get_rank() == 0
def can_log():
return is_main_process()
def dist_print(*args, **kwargs):
if can_log():
print(*args, **kwargs)
def synchronize():
"""
Helper function to synchronize (barrier) among all processes when
using distributed training
"""
if not dist.is_available():
return
if not dist.is_initialized():
return
world_size = dist.get_world_size()
if world_size == 1:
return
dist.barrier()
def dist_cat_reduce_tensor(tensor):
if not dist.is_available():
return tensor
if not dist.is_initialized():
return tensor
# dist_print(tensor)
rt = tensor.clone()
all_list = [torch.zeros_like(tensor) for _ in range(get_world_size())]
dist.all_gather(all_list,rt)
# dist_print(all_list[0][1],all_list[1][1],all_list[2][1],all_list[3][1])
# dist_print(all_list[0][2],all_list[1][2],all_list[2][2],all_list[3][2])
# dist_print(all_list[0][3],all_list[1][3],all_list[2][3],all_list[3][3])
# dist_print(all_list[0].shape)
return torch.cat(all_list,dim = 0)
def dist_sum_reduce_tensor(tensor):
if not dist.is_available():
return tensor
if not dist.is_initialized():
return tensor
if not isinstance(tensor, torch.Tensor):
return tensor
rt = tensor.clone()
dist.all_reduce(rt, op=dist.reduce_op.SUM)
return rt
def dist_mean_reduce_tensor(tensor):
rt = dist_sum_reduce_tensor(tensor)
rt /= get_world_size()
return rt
def all_gather(data):
"""
Run all_gather on arbitrary picklable data (not necessarily tensors)
Args:
data: any picklable object
Returns:
list[data]: list of data gathered from each rank
"""
world_size = get_world_size()
if world_size == 1:
return [data]
# serialized to a Tensor
buffer = pickle.dumps(data)
storage = torch.ByteStorage.from_buffer(buffer)
tensor = torch.ByteTensor(storage).to("cuda")
# obtain Tensor size of each rank
local_size = torch.LongTensor([tensor.numel()]).to("cuda")
size_list = [torch.LongTensor([0]).to("cuda") for _ in range(world_size)]
dist.all_gather(size_list, local_size)
size_list = [int(size.item()) for size in size_list]
max_size = max(size_list)
# receiving Tensor from all ranks
# we pad the tensor because torch all_gather does not support
# gathering tensors of different shapes
tensor_list = []
for _ in size_list:
tensor_list.append(torch.ByteTensor(size=(max_size,)).to("cuda"))
if local_size != max_size:
padding = torch.ByteTensor(size=(max_size - local_size,)).to("cuda")
tensor = torch.cat((tensor, padding), dim=0)
dist.all_gather(tensor_list, tensor)
data_list = []
for size, tensor in zip(size_list, tensor_list):
buffer = tensor.cpu().numpy().tobytes()[:size]
data_list.append(pickle.loads(buffer))
return data_list
from torch.utils.tensorboard import SummaryWriter
class DistSummaryWriter(SummaryWriter):
def __init__(self, *args, **kwargs):
if can_log():
super(DistSummaryWriter, self).__init__(*args, **kwargs)
def add_scalar(self, *args, **kwargs):
if can_log():
super(DistSummaryWriter, self).add_scalar(*args, **kwargs)
def add_figure(self, *args, **kwargs):
if can_log():
super(DistSummaryWriter, self).add_figure(*args, **kwargs)
def add_graph(self, *args, **kwargs):
if can_log():
super(DistSummaryWriter, self).add_graph(*args, **kwargs)
def add_histogram(self, *args, **kwargs):
if can_log():
super(DistSummaryWriter, self).add_histogram(*args, **kwargs)
def add_image(self, *args, **kwargs):
if can_log():
super(DistSummaryWriter, self).add_image(*args, **kwargs)
def close(self):
if can_log():
super(DistSummaryWriter, self).close()
import tqdm
def dist_tqdm(obj, *args, **kwargs):
if can_log():
return tqdm.tqdm(obj, *args, **kwargs)
else:
return obj

129
algorithms/lane_ufld/code/UFLD/utils/factory.py Executable file
View File

@@ -0,0 +1,129 @@
from utils.loss import SoftmaxFocalLoss, ParsingRelationLoss, ParsingRelationDis
from utils.metrics import MultiLabelAcc, AccTopk, Metric_mIoU
from utils.dist_utils import DistSummaryWriter
import torch
def get_optimizer(net,cfg):
training_params = filter(lambda p: p.requires_grad, net.parameters())
if cfg.optimizer == 'Adam':
optimizer = torch.optim.AdamW(training_params, lr=cfg.learning_rate, weight_decay=cfg.weight_decay)
elif cfg.optimizer == 'SGD':
optimizer = torch.optim.SGD(training_params, lr=cfg.learning_rate, momentum=cfg.momentum,
weight_decay=cfg.weight_decay)
else:
raise NotImplementedError
return optimizer
def get_scheduler(optimizer, cfg, iters_per_epoch):
if cfg.scheduler == 'multi':
scheduler = MultiStepLR(optimizer, cfg.steps, cfg.gamma, iters_per_epoch, cfg.warmup, iters_per_epoch if cfg.warmup_iters is None else cfg.warmup_iters)
elif cfg.scheduler == 'cos':
scheduler = CosineAnnealingLR(optimizer, cfg.epoch * iters_per_epoch, eta_min = 0, warmup = cfg.warmup, warmup_iters = cfg.warmup_iters)
else:
raise NotImplementedError
return scheduler
def get_loss_dict(cfg):
if cfg.use_aux:
loss_dict = {
'name': ['cls_loss', 'relation_loss', 'aux_loss', 'relation_dis'],
'op': [SoftmaxFocalLoss(2), ParsingRelationLoss(), torch.nn.CrossEntropyLoss(), ParsingRelationDis()],
'weight': [1.0, cfg.sim_loss_w, 1.0, cfg.shp_loss_w],
'data_src': [('cls_out', 'cls_label'), ('cls_out',), ('seg_out', 'seg_label'), ('cls_out',)]
}
else:
loss_dict = {
'name': ['cls_loss', 'relation_loss', 'relation_dis'],
'op': [SoftmaxFocalLoss(2), ParsingRelationLoss(), ParsingRelationDis()],
'weight': [1.0, cfg.sim_loss_w, cfg.shp_loss_w],
'data_src': [('cls_out', 'cls_label'), ('cls_out',), ('cls_out',)]
}
return loss_dict
def get_metric_dict(cfg):
if cfg.use_aux:
metric_dict = {
'name': ['top1', 'top2', 'top3', 'iou'],
'op': [MultiLabelAcc(), AccTopk(cfg.griding_num, 2), AccTopk(cfg.griding_num, 3), Metric_mIoU(cfg.num_lanes+1)],
'data_src': [('cls_out', 'cls_label'), ('cls_out', 'cls_label'), ('cls_out', 'cls_label'), ('seg_out', 'seg_label')]
}
else:
metric_dict = {
'name': ['top1', 'top2', 'top3'],
'op': [MultiLabelAcc(), AccTopk(cfg.griding_num, 2), AccTopk(cfg.griding_num, 3)],
'data_src': [('cls_out', 'cls_label'), ('cls_out', 'cls_label'), ('cls_out', 'cls_label')]
}
return metric_dict
class MultiStepLR:
def __init__(self, optimizer, steps, gamma = 0.1, iters_per_epoch = None, warmup = None, warmup_iters = None):
self.warmup = warmup
self.warmup_iters = warmup_iters
self.optimizer = optimizer
self.steps = steps
self.steps.sort()
self.gamma = gamma
self.iters_per_epoch = iters_per_epoch
self.iters = 0
self.base_lr = [group['lr'] for group in optimizer.param_groups]
def step(self, external_iter = None):
self.iters += 1
if external_iter is not None:
self.iters = external_iter
if self.warmup == 'linear' and self.iters < self.warmup_iters:
rate = self.iters / self.warmup_iters
for group, lr in zip(self.optimizer.param_groups, self.base_lr):
group['lr'] = lr * rate
return
# multi policy
if self.iters % self.iters_per_epoch == 0:
epoch = int(self.iters / self.iters_per_epoch)
power = -1
for i, st in enumerate(self.steps):
if epoch < st:
power = i
break
if power == -1:
power = len(self.steps)
# print(self.iters, self.iters_per_epoch, self.steps, power)
for group, lr in zip(self.optimizer.param_groups, self.base_lr):
group['lr'] = lr * (self.gamma ** power)
import math
class CosineAnnealingLR:
def __init__(self, optimizer, T_max , eta_min = 0, warmup = None, warmup_iters = None):
self.warmup = warmup
self.warmup_iters = warmup_iters
self.optimizer = optimizer
self.T_max = T_max
self.eta_min = eta_min
self.iters = 0
self.base_lr = [group['lr'] for group in optimizer.param_groups]
def step(self, external_iter = None):
self.iters += 1
if external_iter is not None:
self.iters = external_iter
if self.warmup == 'linear' and self.iters < self.warmup_iters:
rate = self.iters / self.warmup_iters
for group, lr in zip(self.optimizer.param_groups, self.base_lr):
group['lr'] = lr * rate
return
# cos policy
for group, lr in zip(self.optimizer.param_groups, self.base_lr):
group['lr'] = self.eta_min + (lr - self.eta_min) * (1 + math.cos(math.pi * self.iters / self.T_max)) / 2

74
algorithms/lane_ufld/code/UFLD/utils/loss.py Executable file
View File

@@ -0,0 +1,74 @@
import torch
import torch.nn as nn
import torch.nn.functional as F
import numpy as np
class OhemCELoss(nn.Module):
def __init__(self, thresh, n_min, ignore_lb=255, *args, **kwargs):
super(OhemCELoss, self).__init__()
self.thresh = -torch.log(torch.tensor(thresh, dtype=torch.float)).cuda()
self.n_min = n_min
self.ignore_lb = ignore_lb
self.criteria = nn.CrossEntropyLoss(ignore_index=ignore_lb, reduction='none')
def forward(self, logits, labels):
N, C, H, W = logits.size()
loss = self.criteria(logits, labels).view(-1)
loss, _ = torch.sort(loss, descending=True)
if loss[self.n_min] > self.thresh:
loss = loss[loss>self.thresh]
else:
loss = loss[:self.n_min]
return torch.mean(loss)
class SoftmaxFocalLoss(nn.Module):
def __init__(self, gamma, ignore_lb=255, *args, **kwargs):
super(SoftmaxFocalLoss, self).__init__()
self.gamma = gamma
self.nll = nn.NLLLoss(ignore_index=ignore_lb)
def forward(self, logits, labels):
scores = F.softmax(logits, dim=1)
factor = torch.pow(1.-scores, self.gamma)
log_score = F.log_softmax(logits, dim=1)
log_score = factor * log_score
loss = self.nll(log_score, labels)
return loss
class ParsingRelationLoss(nn.Module):
def __init__(self):
super(ParsingRelationLoss, self).__init__()
def forward(self,logits):
n,c,h,w = logits.shape
loss_all = []
for i in range(0,h-1):
loss_all.append(logits[:,:,i,:] - logits[:,:,i+1,:])
#loss0 : n,c,w
loss = torch.cat(loss_all)
return torch.nn.functional.smooth_l1_loss(loss,torch.zeros_like(loss))
class ParsingRelationDis(nn.Module):
def __init__(self):
super(ParsingRelationDis, self).__init__()
self.l1 = torch.nn.L1Loss()
# self.l1 = torch.nn.MSELoss()
def forward(self, x):
n,dim,num_rows,num_cols = x.shape
x = torch.nn.functional.softmax(x[:,:dim-1,:,:],dim=1)
embedding = torch.Tensor(np.arange(dim-1)).float().to(x.device).view(1,-1,1,1)
pos = torch.sum(x*embedding,dim = 1)
diff_list1 = []
for i in range(0,num_rows // 2):
diff_list1.append(pos[:,i,:] - pos[:,i+1,:])
loss = 0
for i in range(len(diff_list1)-1):
loss += self.l1(diff_list1[i],diff_list1[i+1])
loss /= len(diff_list1) - 1
return loss

103
algorithms/lane_ufld/code/UFLD/utils/metrics.py Executable file
View File

@@ -0,0 +1,103 @@
import numpy as np
import torch
import time,pdb
def converter(data):
if isinstance(data,torch.Tensor):
data = data.cpu().data.numpy().flatten()
return data.flatten()
def fast_hist(label_pred, label_true,num_classes):
#pdb.set_trace()
hist = np.bincount(num_classes * label_true.astype(int) + label_pred, minlength=num_classes ** 2)
hist = hist.reshape(num_classes, num_classes)
return hist
class Metric_mIoU():
def __init__(self,class_num):
self.class_num = class_num
self.hist = np.zeros((self.class_num,self.class_num))
def update(self,predict,target):
predict,target = converter(predict),converter(target)
self.hist += fast_hist(predict,target,self.class_num)
def reset(self):
self.hist = np.zeros((self.class_num,self.class_num))
def get_miou(self):
miou = np.diag(self.hist) / (
np.sum(self.hist, axis=1) + np.sum(self.hist, axis=0) -
np.diag(self.hist))
miou = np.nanmean(miou)
return miou
def get_acc(self):
acc = np.diag(self.hist) / self.hist.sum(axis=1)
acc = np.nanmean(acc)
return acc
def get(self):
return self.get_miou()
class MultiLabelAcc():
def __init__(self):
self.cnt = 0
self.correct = 0
def reset(self):
self.cnt = 0
self.correct = 0
def update(self,predict,target):
predict,target = converter(predict),converter(target)
self.cnt += len(predict)
self.correct += np.sum(predict==target)
def get_acc(self):
return self.correct * 1.0 / self.cnt
def get(self):
return self.get_acc()
class AccTopk():
def __init__(self,background_classes,k):
self.background_classes = background_classes
self.k = k
self.cnt = 0
self.top5_correct = 0
def reset(self):
self.cnt = 0
self.top5_correct = 0
def update(self,predict,target):
predict,target = converter(predict),converter(target)
self.cnt += len(predict)
background_idx = (predict == self.background_classes) + (target == self.background_classes)
self.top5_correct += np.sum(predict[background_idx] == target[background_idx])
not_background_idx = np.logical_not(background_idx)
self.top5_correct += np.sum(np.absolute(predict[not_background_idx]-target[not_background_idx])<self.k)
def get(self):
return self.top5_correct * 1.0 / self.cnt
def update_metrics(metric_dict, pair_data):
for i in range(len(metric_dict['name'])):
metric_op = metric_dict['op'][i]
data_src = metric_dict['data_src'][i]
metric_op.update(pair_data[data_src[0]], pair_data[data_src[1]])
def reset_metrics(metric_dict):
for op in metric_dict['op']:
op.reset()
if __name__ == '__main__':
# p = np.random.randint(5, size=(800, 800))
# t = np.zeros((800, 800))
# me = Metric_mIoU(5)
# me.update(p,p)
# me.update(p,t)
# me.update(p,p)
# me.update(p,t)
# print(me.get_miou())
# print(me.get_acc())
a = np.array([1, 2, 3, 4, 5, 6, 7, 8, 9, 0])
b = np.array([1, 1, 2, 2, 2, 3, 3, 4, 4, 0])
me = AccTopk(0,5)
me.update(b,a)
print(me.get())

130
algorithms/lane_ufld/code/UFLD/vis_pth_onnx_compare.py Normal file
View File

@@ -0,0 +1,130 @@
#!/usr/bin/env python3
"""Compare PyTorch vs ONNX UFLD inference with identical post-process and visualization."""
import argparse
import os
import cv2
import numpy as np
import onnxruntime as ort
import scipy.special
import torch
import torchvision.transforms as transforms
from PIL import Image
from data.constant import tusimple_row_anchor
from model.model import parsingNet
from utils.common import checkpoint_state_dict
def decode_lanes(out, griding_num, cls_num_per_lane, img_w, img_h, row_anchor):
"""Shared post-process (demo_new.py logic). out: (101, 56, 4) numpy."""
col_sample = np.linspace(0, 800 - 1, griding_num)
col_sample_w = col_sample[1] - col_sample[0]
out_j = out[:, ::-1, :]
prob = scipy.special.softmax(out_j[:-1, :, :], axis=0)
idx = np.arange(griding_num) + 1
idx = idx.reshape(-1, 1, 1)
loc = np.sum(prob * idx, axis=0)
out_j = np.argmax(out_j, axis=0)
loc[out_j == griding_num] = 0
out_j = loc
lanes = []
colors = [(0, 255, 0), (255, 0, 0), (0, 0, 255), (0, 255, 255)]
for lane_i in range(out_j.shape[1]):
pts = []
if np.sum(out_j[:, lane_i] != 0) <= 2:
continue
for k in range(out_j.shape[0]):
if out_j[k, lane_i] > 0:
x = int(out_j[k, lane_i] * col_sample_w * img_w / 800) - 1
y = int(img_h * (row_anchor[cls_num_per_lane - 1 - k] / 288)) - 1
pts.append((x, y))
lanes.append((pts, colors[lane_i % len(colors)]))
return lanes
def draw_lanes(vis, lanes, radius=5):
for pts, color in lanes:
for p in pts:
cv2.circle(vis, p, radius, color, -1)
return vis
def main():
ap = argparse.ArgumentParser()
ap.add_argument('--pth', default='log/20250702_165153_lr_1e-05_b_32_ufld_2lanes_res18/best.pth')
ap.add_argument('--onnx', default='log/20250702_165153_lr_1e-05_b_32_ufld_2lanes_res18/best.onnx')
ap.add_argument('--image', required=True)
ap.add_argument('--out_dir', default='log/20250702_165153_lr_1e-05_b_32_ufld_2lanes_res18/vis_compare')
ap.add_argument('--griding_num', type=int, default=100)
ap.add_argument('--num_lanes', type=int, default=4)
ap.add_argument('--backbone', default='18')
ap.add_argument('--img_w', type=int, default=1280)
ap.add_argument('--img_h', type=int, default=720)
args = ap.parse_args()
cls_num_per_lane = 56
row_anchor = tusimple_row_anchor
tf = transforms.Compose([
transforms.Resize((288, 800)),
transforms.ToTensor(),
transforms.Normalize((0.485, 0.456, 0.406), (0.229, 0.224, 0.225)),
])
tensor = tf(Image.open(args.image).convert('RGB')).unsqueeze(0)
net = parsingNet(
pretrained=False,
backbone=args.backbone,
cls_dim=(args.griding_num + 1, cls_num_per_lane, args.num_lanes),
use_aux=False,
)
net.load_state_dict(checkpoint_state_dict(args.pth, map_location='cpu'), strict=False)
net.eval()
with torch.no_grad():
pt_out = net(tensor).numpy()[0]
sess = ort.InferenceSession(args.onnx, providers=['CPUExecutionProvider'])
onnx_out = sess.run(['output'], {'input': tensor.numpy()})[0][0]
diff = np.abs(pt_out - onnx_out)
print(f'output max diff: {diff.max():.6e}, mean diff: {diff.mean():.6e}')
print(f'allclose: {np.allclose(pt_out, onnx_out, rtol=1e-3, atol=1e-4)}')
vis_base = cv2.imread(args.image)
if vis_base is None:
raise FileNotFoundError(args.image)
vis_base = cv2.resize(vis_base, (args.img_w, args.img_h))
lanes_pt = decode_lanes(pt_out, args.griding_num, cls_num_per_lane, args.img_w, args.img_h, row_anchor)
lanes_onnx = decode_lanes(onnx_out, args.griding_num, cls_num_per_lane, args.img_w, args.img_h, row_anchor)
vis_pt = draw_lanes(vis_base.copy(), lanes_pt)
vis_onnx = draw_lanes(vis_base.copy(), lanes_onnx)
panel = np.hstack([vis_pt, vis_onnx])
cv2.putText(panel, 'PyTorch', (20, 40), cv2.FONT_HERSHEY_SIMPLEX, 1.2, (255, 255, 255), 2)
cv2.putText(panel, 'ONNX', (args.img_w + 20, 40), cv2.FONT_HERSHEY_SIMPLEX, 1.2, (255, 255, 255), 2)
# pixel diff on visualization (lane points only)
pt_pts = {p for lane, _ in lanes_pt for p in lane}
onnx_pts = {p for lane, _ in lanes_onnx for p in lane}
print(f'lane points: pytorch={len(pt_pts)}, onnx={len(onnx_pts)}, only_pt={len(pt_pts-onnx_pts)}, only_onnx={len(onnx_pts-pt_pts)}')
os.makedirs(args.out_dir, exist_ok=True)
base = os.path.splitext(os.path.basename(args.image))[0]
out_panel = os.path.join(args.out_dir, f'{base}_pth_vs_onnx.jpg')
out_pt = os.path.join(args.out_dir, f'{base}_pytorch.jpg')
out_onnx = os.path.join(args.out_dir, f'{base}_onnx.jpg')
cv2.imwrite(out_panel, panel)
cv2.imwrite(out_pt, vis_pt)
cv2.imwrite(out_onnx, vis_onnx)
print('saved:', out_panel)
print('saved:', out_pt)
print('saved:', out_onnx)
if __name__ == '__main__':
main()

66
algorithms/lane_ufld/code/UFLD/vis_tusimple_pred.py Normal file
View File

@@ -0,0 +1,66 @@
"""Visualize TuSimple-format predictions from test.py (tusimple_eval_tmp.0.txt)."""
import argparse
import json
import os
import cv2
LANE_COLORS = [(0, 255, 0), (0, 0, 255), (255, 0, 0), (0, 255, 255)]
def draw_lanes(img, lanes, h_samples):
for lane_idx, xs in enumerate(lanes):
color = LANE_COLORS[lane_idx % len(LANE_COLORS)]
pts = [(int(x), int(y)) for x, y in zip(xs, h_samples) if x >= 0]
for i in range(len(pts) - 1):
cv2.line(img, pts[i], pts[i + 1], color, 3)
for p in pts:
cv2.circle(img, p, 4, color, -1)
def main():
parser = argparse.ArgumentParser()
parser.add_argument(
'--pred',
default='tmp/tusimple_eval_tmp.0.txt',
help='prediction jsonl from test.py',
)
parser.add_argument(
'--data_root',
default='/home/chengfanglu/DATA/lane0_copy/DATASET',
help='dataset root (images under data_root/raw_file)',
)
parser.add_argument(
'--out_dir',
default='tmp/vis_pred',
help='directory to save overlay images',
)
args = parser.parse_args()
os.makedirs(args.out_dir, exist_ok=True)
n_ok = 0
with open(args.pred, 'r') as f:
for line in f:
item = json.loads(line)
rel = item['raw_file']
img_path = os.path.join(args.data_root, rel)
if not os.path.isfile(img_path):
print('skip (missing):', img_path)
continue
img = cv2.imread(img_path)
if img is None:
print('skip (read failed):', img_path)
continue
draw_lanes(img, item['lanes'], item['h_samples'])
out_path = os.path.join(args.out_dir, rel)
os.makedirs(os.path.dirname(out_path), exist_ok=True)
cv2.imwrite(out_path, img)
n_ok += 1
print('saved', out_path)
print(f'done: {n_ok} images -> {args.out_dir}')
if __name__ == '__main__':
main()