BEVFusion（北大&阿里）环境搭建教程_编程语言

BEVFusion环境搭建

论文题目：BEVFusion: A Simple and Robust LiDAR-Camera Fusion Framework

论文地址：https://arxiv.org/pdf/2205.13790.pdf

代码地址：ADLab-AutoDrive/BEVFusion: Offical PyTorch implementation of “BEVFusion: A Simple and Robust LiDAR-Camera Fusion Framework” (github.com)

前言：这是今年新发的一篇论文，我在***次阅读时，代码还未开源，前几天发现开源了，而且这个框架做的很清晰，可以基于这个工作熟悉一下融合方案，也能稍稍改进一下。

笔者环境：

sys.platform: linux

Python: 3.8.3 | packaged by conda-forge | (default, Jun 1 2020, 17:43:00) [GCC 7.5.0]

CUDA available: True

GPU 0,1: NVIDIA GeForce RTX 3090

CUDA_HOME: /usr

NVCC: Build cuda_11.5.r11.5/compiler.30672275_0

GCC: gcc (Ubuntu 7.5.0-6ubuntu2) 7.5.0

PyTorch: 1.8.0+cu111

PyTorch compiling details: PyTorch built with:

GCC 7.3
C++ Version: 201402
Intel® Math Kernel Library Version 2020.0.0 Product Build 20191122 for Intel® 64 architecture applications
Intel® MKL-DNN v1.7.0 (Git Hash 7aed236906b1f7a05c0917e5257a1af05e9ff683)
OpenMP 201511 (a.k.a. OpenMP 4.5)
NNPACK is enabled
CPU capability usage: AVX2
CUDA Runtime 11.1
NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86
CuDNN 8.0.5
Magma 2.5.2
Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.1, CUDNN_VERSION=8.0.5, CXX_COMPILER=/opt/rh/devtoolset-7/root/usr/bin/c++, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.8.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON,
TorchVision: 0.9.0+cu111
OpenCV: 4.6.0
MMCV: 1.4.0
MMCV Compiler: GCC 7.3
MMCV CUDA Compiler: 11.1
MMDetection: 2.11.0
MMDetection3D: 0.11.0+9d3e162

说人话：torch1.8.0+cuda111，显卡是3090，其余环境根据根据官方文档配的

一、安装

1、克隆BEVFusion代码
```
git clone https://github.com/ADLab-AutoDrive/BEVFusion.git
```
2、新建虚拟环境
```
conda create -n bevfusion python=3.8.3
conda activate bevfusion #激活
```
3、根据cuda版本安装torch

尽量选择1.7或者1.8，cuda版本要对应
2022年11月22日更新：尽量选1.7吧，1.8的bug导致训练有些问题
2022年12月2日更新：***近太忙，前几天发现1.7的torch不适合在这个项目上用，具体原因和3090显卡有关，缺少相关库（3090***低支持cuda11.1，torch1.7***高支持cuda11.0）
```
pip install torch=1.8.0+cu111 torchvision=0.9.0+cu111 torchaudio==0.8.0 -f https://download.pytorch.org/whl/torch_stable.html
```
4、安装mmcv-full

这里要根据自己的torch和cuda来，另外，BEVFusion官方文档推荐1.4.0版本

从以下网址找到自己合适的版本
```
https://download.openmmlab.com/mmcv/dist/{cu_version}/{torch_version}/index.html
```
其中cu_version代表cuda版本，torch_version代表torch版本，

例如，我的是torch1.8.0+cuda111，所以我的地址是：
```
https://download.openmmlab.com/mmcv/dist/cu111/torch1.8.0/index.html
```
因为创建的环境是python3.8，所以选择cp38，别忘了mmcv-full版本是1.4.0，下载whl文件，并传到服务器上，然后pip whl文件（不再赘述pip安装whl了，百度一下）

5、安装MMDetection

下面是BEVFusion官方推荐的环境

python=3.8.3
mmdet=2.11.0 (please install mmdet in mmdetection-2.11.0)
mmcv=1.4.0
mmdet3d=0.11.0
numpy=1.19.2
torch=1.7.0
torchvision=0.8.0

注意please install mmdet in mmdetection-2.11.0，作者已经在仓库中放了这个文件夹，我们直接进入这个文件夹

然后执行：
```
pip install -r requirements/build.txt
pip install -v -e .  # or "python setup.py develop"(推荐执行后者)
```
6、安装MMDetection3D

退回到BEVFusion根目录下，执行下面指令
```
pip install -v -e .  # or "python setup.py develop"（推荐执行后者）
```
***此配置完毕

以下是我的环境

Package Version Editable project location

absl-py 1.3.0
addict 2.4.0
anyio 3.6.2
argon2-cffi 21.3.0
argon2-cffi-bindings 21.2.0
asttokens 2.1.0
attrs 22.1.0
backcall 0.2.0
beautifulsoup4 4.11.1
black 22.10.0
bleach 5.0.1
cachetools 5.2.0
certifi 2022.9.24
cffi 1.15.1
charset-normalizer 2.1.1
click 8.1.3
contourpy 1.0.6
cycler 0.11.0
Cython 0.29.32
debugpy 1.6.3
decorator 5.1.1
defusedxml 0.7.1
depthwise-conv2d-implicit-gemm 0.0.0
descartes 1.1.0
entrypoints 0.4
exceptiongroup 1.0.4
executing 1.2.0
fastjsonschema 2.16.2
filelock 3.8.0
fire 0.4.0
flake8 5.0.4
fonttools 4.38.0
google-auth 2.14.1
google-auth-oauthlib 0.4.6
grpcio 1.50.0
h5py 3.7.0
huggingface-hub 0.11.0
idna 3.4
imageio 2.22.4
importlib-metadata 5.0.0
importlib-resources 5.10.0
iniconfig 1.1.1
ipykernel 6.17.1
ipython 8.6.0
ipython-genutils 0.2.0
ipywidgets 8.0.2
jedi 0.18.1
Jinja2 3.1.2
joblib 1.2.0
jsonschema 4.17.0
jupyter 1.0.0
jupyter_client 7.4.6
jupyter-console 6.4.4
jupyter_core 5.0.0
jupyter-server 1.23.2
jupyterlab-pygments 0.2.2
jupyterlab-widgets 3.0.3
kiwisolver 1.4.4
llvmlite 0.31.0
loguru 0.6.0
lyft-dataset-sdk 0.0.8
Markdown 3.4.1
MarkupSafe 2.1.1
matplotlib 3.6.2
matplotlib-inline 0.1.6
mccabe 0.7.0
mistune 2.0.4
mmcls 0.24.1
mmcv-full 1.4.0
mmdet 2.11.0 /home/wistful/work/my_bevfusion/mmdetection-2.11.0
mmdet3d 0.11.0 /home/wistful/work/my_bevfusion
mmpycocotools 12.0.3
msgpack 1.0.4
msgpack-numpy 0.4.8
multimethod 1.9
mypy-extensions 0.4.3
nbclassic 0.4.8
nbclient 0.7.0
nbconvert 7.2.5
nbformat 5.7.0
nest-asyncio 1.5.6
networkx 2.2
ninja 1.11.1
notebook 6.5.2
notebook_shim 0.2.2
numba 0.48.0
numpy 1.23.4
nuscenes-devkit 1.1.9
oauthlib 3.2.2
opencv-python 4.6.0.66
packaging 21.3
pandas 1.4.4
pandocfilters 1.5.0
parso 0.8.3
pathspec 0.10.2
pexpect 4.8.0
pickleshare 0.7.5
Pillow 9.3.0
pip 22.3.1
pkgutil_resolve_name 1.3.10
platformdirs 2.5.4
plotly 5.11.0
pluggy 1.0.0
plyfile 0.7.4
prettytable 3.5.0
prometheus-client 0.15.0
prompt-toolkit 3.0.32
protobuf 3.20.3
psutil 5.9.4
ptyprocess 0.7.0
pure-eval 0.2.2
pyasn1 0.4.8
pyasn1-modules 0.2.8
pycocotools 2.0.6
pycodestyle 2.9.1
pycparser 2.21
pyflakes 2.5.0
Pygments 2.13.0
pyparsing 3.0.9
pyquaternion 0.9.9
pyrsistent 0.19.2
pytest 7.2.0
python-dateutil 2.8.2
pytz 2022.6
PyWavelets 1.4.1
PyYAML 6.0
pyzmq 24.0.1
qtconsole 5.4.0
QtPy 2.3.0
requests 2.28.1
requests-oauthlib 1.3.1
rsa 4.9
scikit-image 0.19.3
scikit-learn 1.1.3
scipy 1.9.3
Send2Trash 1.8.0
setuptools 65.5.1
Shapely 1.8.5.post1
six 1.16.0
sniffio 1.3.0
soupsieve 2.3.2.post1
stack-data 0.6.1
tabulate 0.9.0
tenacity 8.1.0
tensorboard 2.11.0
tensorboard-data-server 0.6.1
tensorboard-plugin-wit 1.8.1
tensorpack 0.11
termcolor 2.1.0
terminado 0.17.0
terminaltables 3.1.10
threadpoolctl 3.1.0
tifffile 2022.10.10
timm 0.6.11
tinycss2 1.2.1
toml 0.10.2
tomli 2.0.1
torch 1.8.0+cu111
torchaudio 0.8.0
torchpack 0.3.1
torchvision 0.9.0+cu111
tornado 6.2
tqdm 4.64.1
traitlets 5.5.0
trimesh 2.35.39
typing_extensions 4.4.0
urllib3 1.26.12
wcwidth 0.2.5
webencodings 0.5.1
websocket-client 1.4.2
Werkzeug 2.2.2
wheel 0.38.4
widgetsnbextension 4.0.3
yapf 0.32.0
zipp 3.10.0

sys.platform: linux
Python: 3.8.3 | packaged by conda-forge | (default, Jun 1 2020, 17:43:00) [GCC 7.5.0]
CUDA available: True
GPU 0,1: NVIDIA GeForce RTX 3090
CUDA_HOME: /usr/local/cuda
NVCC: Build cuda_11.1.TC455_06.29069683_0
GCC: gcc (Ubuntu 7.5.0-6ubuntu2) 7.5.0
PyTorch: 1.8.0+cu111
PyTorch compiling details: PyTorch built with:
- GCC 7.3
- C++ Version: 201402
- Intel® Math Kernel Library Version 2020.0.0 Product Build 20191122 for Intel® 64 architecture applications
- Intel® MKL-DNN v1.7.0 (Git Hash 7aed236906b1f7a05c0917e5257a1af05e9ff683)
- OpenMP 201511 (a.k.a. OpenMP 4.5)
- NNPACK is enabled
- CPU capability usage: AVX2
- CUDA Runtime 11.1
- NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86
- CuDNN 8.0.5
- Magma 2.5.2
- Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.1, CUDNN_VERSION=8.0.5, CXX_COMPILER=/opt/rh/devtoolset-7/root/usr/bin/c++, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.8.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON,
  TorchVision: 0.9.0+cu111
  OpenCV: 4.6.0
  MMCV: 1.4.0
  MMCV Compiler: GCC 7.3
  MMCV CUDA Compiler: 11.1
  MMDetection: 2.20.0
  MMDetection3D: 0.11.0+9d3e162
  
  二、组织数据集
  
  注意，一般来说使用mmdetection3d框架的工作，要把数据集组织到mmdetection3d/data下，BEVFusion整个目录就是mmdetection3d的组织格式。所以要把数据集组织到BEVFusion/data下
  
  It is recommended to symlink the dataset root to $MMDETECTION3D/data. If your folder structure is different from the following, you may need to change the corresponding paths in config files.
  
  数据集组织我就不再赘述了，按照官方说明的组织形式组织好
  
  mmdetection3d
  ├── mmdet3d
  ├── tools
  ├── configs
  ├── data
  │ ├── nuscenes
  │ │ ├── maps
  │ │ ├── samples
  │ │ ├── sweeps
  │ │ ├── v1.0-test
  | | ├── v1.0-trainval
  │ ├── kitti
  │ │ ├── ImageSets
  │ │ ├── testing
  │ │ │ ├── calib
  │ │ │ ├── image_2
  │ │ │ ├── velodyne
  │ │ ├── training
  │ │ │ ├── calib
  │ │ │ ├── image_2
  │ │ │ ├── label_2
  │ │ │ ├── velodyne
  │ ├── waymo
  │ │ ├── waymo_format
  │ │ │ ├── training
  │ │ │ ├── validation
  │ │ │ ├── testing
  │ │ │ ├── gt.bin
  │ │ ├── kitti_format
  │ │ │ ├── ImageSets
  │ ├── lyft
  │ │ ├── v1.01-train
  │ │ │ ├── v1.01-train (train_data)
  │ │ │ ├── lidar (train_lidar)
  │ │ │ ├── images (train_images)
  │ │ │ ├── maps (train_maps)
  │ │ ├── v1.01-test
  │ │ │ ├── v1.01-test (test_data)
  │ │ │ ├── lidar (test_lidar)
  │ │ │ ├── images (test_images)
  │ │ │ ├── maps (test_maps)
  │ │ ├── train.txt
  │ │ ├── val.txt
  │ │ ├── test.txt
  │ │ ├── sample_submission.csv
  │ ├── scannet
  │ │ ├── meta_data
  │ │ ├── scans
  │ │ ├── batch_load_scannet_data.py
  │ │ ├── load_scannet_data.py
  │ │ ├── scannet_utils.py
  │ │ ├── README.md
  │ ├── sunrgbd
  │ │ ├── OFFICIAL_SUNRGBD
  │ │ ├── matlab
  │ │ ├── sunrgbd_data.py
  │ │ ├── sunrgbd_utils.py
  │ │ ├── README.md
  
  （以上数据集不用全需要的，自己需要哪个组织哪个）
  
  然后利用BEVFusion/tools/create_data.py脚本组织文件就行了，放一下组织nuScenes数据集成功的截图吧
  
  三、训练与测试
  
  1、分布式训练
```
# first train camera stream
./tools/dist_train.sh configs/bevfusion/cam_stream/bevf_pp_4x8_2x_nusc_cam.py 8
# then train LiDAR stream
./tools/dist_train.sh configs/bevfusion/lidar_stream/hv_pointpillars_secfpn_sbn-all_4x8_2x_nus-3d.py 8
# then train BEVFusion
./tools/dist_train.sh configs/bevfusion/bevf_pp_2x8_1x_nusc.py 8
```
  ./tools/dist_train.sh configs/bevfusion/cam_stream/bevf_pp_4x8_2x_nusc_cam.py 8中，8代表GPU个数
  
  使用sh文件进行训练和测试时，可能会遇到无权限的问题，使用chmod 777 tools/dist_train.sh进行权限更改即可
  
  其中，配置文件我就不再叙述，可以使用官方的简单测试，如果需要使用自己的配置文件，请参考mmdetection3d官方文档
  
  值得注意的是，我在使用分布式和非分布式方法做测试的时候，非分布式运行正常，分布式仍然提示找不到数据集，我搜寻资料得知是因为工作路径的原因导致的，我用print(os.getcwd())打印了一下，发现使用非分布式训练时，工作路径/home/wistful/work/my_bevfusion/mmdetection3d/tools下，即在BEVFusion/mmdetection3d/tools下，而使用分布式训练时，工作路径在/home/wistful/work/my_bevfusion下，即在BEVFusion根目录下。所以我使用os.chdir('/home/wistful/work/my_bevfusion/mmdetection3d/tools')将工作路径改变了一下，问题解决
  
  2、非分布式训练
  
  直接在pycharm中根据各参数设置一下，训练即可，不再叙述。
  
  3、测试
```
./tools/dist_test.sh configs/bevfusion/bevf_pp_2x8_1x_nusc.py ./work_dirs/bevfusion_pp.pth 8 --eval bbox
 # 前者是配置文件，后者是GPU个数
# 如需可视化
./tools/dist_test.sh configs/bevfusion/bevf_pp_2x8_1x_nusc.py ./work_dirs/bevfusion_pp.pth 8 --eval bbox --show-dir YourVisualResultsDir --show --eval bbox #根据test.py的参数看一下
```
  可视化结果会保存在你指定的目录下，如下所示
  导入CloudCampare查看