documentation (final results)

dataset construction pipeline path

root 1: /slurm_home/yslee/DAIR_V2X_Seq_hslee/projects/cctv
root 2: /slurm_home/yslee/DAIR_V2X_Seq_hslee/projects/V2X-Graph_gnode31_ablation_with_map_more_scenario/
run pipeline: {root1}/run_cheonan.sh
- 1️⃣ extract bounding box & segmentation map with (mask r-cnn)
  - python run_cheonan.py extract_tracks …
- 2️⃣ compute homography
  - python run_cheonan.py compute_homographies …
- 3️⃣ extract more precise bounding box (yolov8x-visdrone)
  - python run_cheonan.py extract_yolov8n …
- 4️⃣ visualization
  - python visualize_bev_view.py
- 5️⃣ export with csv file
  - python export_trajectory_csv.py
- 6️⃣ convert compatable for v2x-seq-tfd (not included in run_cheonan.sh)
  - {root2}/run_convert_cctv.sh
- hyperparameter (run_cheonan.sh)
  - cctv: name of cctv
  - num_segment: segment number (.ts) for calculating homography
  - dataset_dir: directory where raw video stream is located
  - frame_step: subsampling value
  - total_segment: .ts file number
utils
- {root1}/extract_first_frame.py
  - extract first frame of the camera video stream
- 7️⃣ {root2}/annotate_lanes_notebook.ipynb
  - annotation tool for making map.json
output
- {root2}/new_cheonan
  - detections, detections_yolov8n, homography., tracks, tracks_yolov8n
    - results of 1️⃣2️⃣3️⃣
  - cctv_{camera_name}pixels_lanes.json, cctv{camera_name}_vector_map.json
    - results of 7️⃣
    - former → pixel space coordinate, latter → world space coordinate
- {root1}/output/cheonan
  - bev_{camera_name}.mp4
    - results of 4️⃣
  - {camera_name}_trajectory.csv
    - results of 5️⃣
- {root2}/cctv_dataset
  - train/data, processed, val/data, processed
    - results of 6️⃣
    - processed (.pt) files workes for input of model

CSV Extraction algorithm

1. world coordinate

box_bottom_center() → extract the bottom center coordinate($x_{px},y_{px}$) (have to select $Z=0$ plane)
applyh(Homography, bbc) → ($x_{world}, y_{world}$) (meter m)
safe margin (min_y) to delete the outlier

2. velocity and theta

compute_motion() → calculate the world cooridnate difference → multiply with fps (m/s)
theta = arctan2(y,x) (rad)
- data refining (backfill & forward-fill) due to the nature of cctv data, a stationary vehicle has a speed vector 0, causing an unknown direction theta.
- stop state processing
  - fill the previous stop state frames with backfill direction information when the vehicle started moving
- stop processing after movement
  - forward-fill the last known direction of driving in the frames after stopping

baseline code path

model	root	train script	eval script	visualization	dataset	raw video & homography & tracked bounding box & map	checkpoint
hivt w/o map	/slurm_home/yslee/DAIR_V2X_Seq_hslee/projects/HiVT_plugin_ablation_without_map_more_scenario	{root}/hivt_train_cctv.sh	{root}/hivt_eval_cctv.sh	-	use graph w/o map’s	use graph w/o map’s	{root}/lightning_logs/version_87/checkpoints/epoch=1059-step=12719.ckpt
hivt	/slurm_home/yslee/DAIR_V2X_Seq_hslee/projects/HiVT_plugin_ablation_with_map_more_scenario	{root}/hivt_train_cctv.sh	{root}/hivt_eval_cctv.sh	-	use graph’s	use graph’s	{root}/lightning_logs/version_83/checkpoints/epoch=2954-step=35459.ckpt
graph w/o map	/slurm_home/yslee/DAIR_V2X_Seq_hslee/projects/V2X-Graph_gnode31_ablation_without_map_more_scenario	{root}/train_cctv.sh	included in train script	-	{root}/cctv_dataset	{root}new_cheonan	{root}/lightning_logs/version_109/checkpoints/epoch=3224-step=38699.ckpt
graph	/slurm_home/yslee/DAIR_V2X_Seq_hslee/projects/V2X-Graph_gnode31_ablation_with_map_more_scenario	{root}/train_cctv.sh	included in train script	{root}/visualize_bev.sh	{root}/cctv_dataset	{root}new_cheonan	{root}/lightning_logs/version_104/checkpoints/epoch=1679-step=20159.ckpt
v2itraj w/o map	/slurm_home/yslee/DAIR_V2X_Seq_hslee/projects/V2ITrajNet_without_map_more_scenario	{root}/hivt_train_cctv.sh	{root}/hivt_eval_cctv.sh	-	use graph w/o map’s	use graph w/o map’s	{root}/my_logs/cctv_v2itrajnet/version_10/checkpoints/epoch=1029-step=24719.ckpt
v2itraj	/slurm_home/yslee/DAIR_V2X_Seq_hslee/projects/V2ITrajNet_with_map_more_scenario	{root}/hivt_train_cctv.sh	{root}/hivt_eval_cctv.sh	-	use graph’s	use graph’s	{root}/my_logs/cctv_v2itrajnet/version_10/checkpoints/epoch=1219-step=29279.ckpt

Parameter setup

preprocess dataset

past, future step: 40
- window-size: 40+40=80
- stride: 25