Towards Accurate Vehicle Behaviour Classification With Multi Relational Graph Convolutional Networks
Link to our paper on arxiv : https://arxiv.org/abs/2002.00786
For any queries mail : firstname.lastname@example.org (or) email@example.com
Our method detects and classifies objects of interest (vehicles) in a video scene into 6 classes moving away, moving towards us, parked, lane change(L->R), lane change(R->L), overtake. Note that our method is not based on classifying the ego-vehicle.
Base-line Implementation details
Apart from comparisons in the paper, we compare with Structural-RNN, a LSTM based graph network. Since the tasks in their paper confine only to driver-anticipation, we use one of their methods similar to our task. Specifically, we use the detection method of activity-anticipation mentioned in the paper due to the similarity in the architecture and task . We use Vehicles as Humans and Lane Markings as Objects in their architecture for our purpose. Similar to the Human-Object, Human-Human and Object-Object interactions, we observe the Vehicle-Lane, Vehicle-Vehicle and Lane-Lane interactions for all time-steps. We give features similar to the baselines in the paper.
|moving towards us||51||35||41||89.5|
We selected 3 main datasets to perform the experiments.
Graphs for all datasets can be downloaded from graphs.
For information on how each graph is stored as a npz file, go through the README file in the same link.
On apollo we have selected sequences from scene-parsing dataset and picked around 70 small sequences(each containing aroud 100 images) manually that include behaviours of our interest. Similarly on Kitti, we use tracking sequences 4,5,10 which are in line with our class requirement.
dgl pytorch == 1.2.0 pandas numpy tqdm
Installing without GPU:
pip3 install requirements.txt
To install and use GPU for dgl, cuda support can be installed from their official website, dgl .
And set use_cuda = 1 in training/testing codes.
Testing and Training on apollo dataset
git clone https://github.com/ma8sa/temporal-MR-GCN.git cd temporal-MR-GCN --testing python3 lstm_rgcn_test_apollo.py --training python3 lstm_rgcn_train_apollo.py
NOTE : Make sure to extract the corresponding graphs(lstm_graphs_apollo) and place it in the same folder where you are running the code from.
In training, lstm_rgcn_train_apollo has all the parameters in main function to tune. main_model.py contains the complete model. rgcn_layer.py contains the dgl format of MR-GCN. graphs_preproc_apollo.py conatains all the data preprocessing methods used.
Testing on Indian/Kitti dataset (Transfer Learning)
for indian, python3 lstm_rgcn_test_ind_kitti.py indian for kitti, python3 lstm_rgcn_test_ind_kitti.py kitti
NOTE : Make sure to extract the corresponding graphs (lstm_graphs_kitti for kitti and lstm_graphs_indian for indian) and place it in the same folder where you are running the code from.
1->Moving towards us
Results on Apollo
| | 0 | 1 | 2 | 3 | 4 | 5 |
| ————- | ————- | ———— | ———— | ———— | ———— | ———— |
| class accuracy(train)| 95 | 98 | 97 | 96 | 96 | 97 |
| class counts(train) | 2673 | 685 | 3574 |424 | 452 | 525 | | class accuracy(val) | 85 | 89 | 94 | 84 | 86 | 72 | | class counts(val) | 814 | 237 | 1415 | 162 | 130 | 73 |
Since the number of cars showing overtake behaviour are less, we augmented and added few synthetic-graphs and augmented data to train data only for apollo. With a little more data, the model can clearly learn overtake too above 80%, as number of synthetic graphs added were too low 76.
Results on Kitti tested with weights trained on Apollo
| | 0 | 1 | 2 | | ————- | ————- | ———— | ———— | | class accuracy| 99 | 98 | 98 | | class counts | 504 | 230 | 674 |
Results on Indian tested with weights trained on Apollo
| | 0 | 1 | 2 | | ————- | ————- | ———— | ———— | | class accuracy| 99 | 92 | 99 | | class counts | 324 | 229 | 2547 |