Waymo motion prediction challenge 2021: 3rd place solution

Overview

Waymo motion prediction challenge 2021: 3rd place solution

header

Team behind this solution:

  1. Artsiom Sanakoyeu [Homepage] [Twitter] [Telegram Channel] [LinkedIn]
  2. Stepan Konev [LinkedIn]
  3. Kirill Brodt [GitHub]

Dataset

Download datasets uncompressed/tf_example/{training,validation,testing}

Prerender

Change paths to input dataset and output folders

python prerender.py \
    --data /home/data/waymo/training \
    --out ./train
    
python prerender.py \
    --data /home/data/waymo/validation \
    --out ./dev \
    --use-vectorize \
    --n-shards 1
    
python prerender.py \
    --data /home/data/waymo/testing \
    --out ./test \
    --use-vectorize \
    --n-shards 1

Training

MODEL_NAME=xception71
python train.py \
    --train-data ./train \
    --dev-data ./dev \
    --save ./${MODEL_NAME} \
    --model ${MODEL_NAME} \
    --img-res 224 \
    --in-channels 25 \
    --time-limit 80 \
    --n-traj 6 \
    --lr 0.001 \
    --batch-size 48 \
    --n-epochs 120

Submit

python submit.py \
    --test-data ./test/ \
    --model-path ${MODEL_PATH_TO_JIT} \
    --save ${SAVE}

Visualize predictions

python visualize.py \
    --model ${MODEL_PATH_TO_JIT} \
    --data ${DATA_PATH} \
    --save ./viz

Citation

If you find our work useful, please cite it as:

@article{konev2021motioncnn,
  title={MotionCNN: A Strong Baseline for Motion Prediction in Autonomous Driving},
  author={Konev, Stepan and Brodt, Kirill and Sanakoyeu, Artsiom},
  year={2021}
}

Related repos

Comments
  • Metrics

    Metrics

    Hi! @kbrodt Thanks for sharing this great code!

    Where are the codes of the evaluation metrics (for example: ADE, FDE, minADE, minFDE and so on)? Or where can I find it?

    Looking forward to your reply!

    opened by chx-Github 9
  • Regarding the training epochs

    Regarding the training epochs

    Thanks for sharing this awesome codes!

    Is it necessary to train the model for 120 epochs? Since there are more than 1M training samples. Can you share some performance during the training progress? Such as the performance with 30epochs, 60 epochs, 90 epochs? Since I trained it for several epochs but the loss is still very large.

    To double check the training process, can you share how many training samples for each epoch?

    Thanks so much!

    opened by FutureOpenAI 8
  • Loss

    Loss

    Hi! I tried your method and I observed that in training, l2loss and log_softmax have so large difference. so my network does not learn multimodal tracks, only one best track is fitted. Do you have any solution?

    opened by zsgj-Xxx 7
  • Angle conversion error at prerender.py

    Angle conversion error at prerender.py

    Thank you for amazing works.

    I found the conversion error at prerender.py

    https://github.com/kbrodt/waymo-motion-prediction-2021/blob/3665d4b3ba39a6b879b663747f93b6e525018c00/prerender.py#L706 https://github.com/kbrodt/waymo-motion-prediction-2021/blob/3665d4b3ba39a6b879b663747f93b6e525018c00/prerender.py#L707

    These two lines has conversion error.

    In order to convert coordinate properly, it should be changed like this tmp[3] = other_v_yaw - ANGLE tmp[4] = other_bbox_yaw - ANGLE

    opened by KyuhwanYeon 3
  • Question About the waymo dataset lane type

    Question About the waymo dataset lane type

    Hi, @kbrodt,

    Thank you for your great work. I am new to this area, so I have a question regarding to the lane type of waymo dataset. In general, the lanes in waymo dataset, can be broadly categorized into 6: lanecenters, roadlines, stopsign, speedbump, roadedge and crosswalks. So, is the lane centers are just the the center of each line in which the vehicles can drive on? Theoretically, a car's center should be on the lanecenters if this car is staying in this lane. Is my interpretation right? Also, is the lanecenters in waymo the same as the centerlines in Argoverse Dataset? Are you familiar with that dataset? I am looking forward to your reply. Thank you in advance.

    opened by SwagJ 2
  • Vehicles driving on the same lane

    Vehicles driving on the same lane

    Hello!

    Thanks for sharing you work!

    I have one questions: from the motion dataset, is that possible to know if two vehicles are driving on the same lane? I notice that there is a column called "roadgraph_samples/id" but I am not sure whether that means the lane ID.

    Thanks.

    opened by 18627242758 2
  • Congratulations

    Congratulations

    Привет Кирилл,

    Я вижу что ты разбираешься в машинном обучении. Я хотел бы сконнектиться и перенять опыт, если возможно. Как с тобой можно связаться?

    Дмитрий

    opened by Rendok 1
  • I can't download the dataset

    I can't download the dataset

    The infomation is showed as below: Additional permissions required to list objects in this bucket. Ask a bucket owner to grant you 'storage.objects.list' permission.

    opened by fengsky401 1
  • socket.gaierror: [Errno -2] Name or service not known

    socket.gaierror: [Errno -2] Name or service not known

    After running the following command :

    (venvpy37cu10) [[email protected] project]$ python train.py --train-data ./train --dev-data ./dev --save ./xception71 --model xception71 --img-res 224 --in-channels 25 --time-limit 80 --n-traj 6 --lr 0.001 --batch-size 48 --n-epochs 120

    Below error has occurred :

    Screen Shot 2022-07-30 at 10 59 36 PM
    opened by rohansd 1
  • magic_const and shift

    magic_const and shift

    Hi,

    Thanks for your open-source work.

    I don't understand the magic_const and shift in rasterize() function prerender.py.

    Would you please give some explanation?

    opened by ShoufaChen 2
  • Preprocessing issue

    Preprocessing issue

    I tried running with all requirements , it keeps on reading the records but when it tries to write down .....here it fails , please help me to proceed further

    [email protected]:/app/waymo-adas-main/waymo-motion-prediction-2021# python3 prerender.py --data /app/waymo-adas-main/waymo-dataset/original/validation/ --out /app/waymo-adas-main/data/train1
    False
    Namespace(data='/app/waymo-adas-main/waymo-dataset/original/validation/', each=0, n_jobs=20, n_shards=8, no_valid=False, out='/app/waymo-adas-main/data/train1', use_vectorize=False)
    1215it [00:28, 42.50it/s]
      0%|                                                                                                                                                                                   | 0/1215 [00:00<?, ?it/s]
    multiprocessing.pool.RemoteTraceback:
    """
    Traceback (most recent call last):
      File "/usr/lib/python3.8/multiprocessing/pool.py", line 125, in worker
        result = (True, func(*args, **kwds))
      File "prerender.py", line 731, in merge
        parsed = tf.io.parse_single_example(data, features_description)
      File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/traceback_utils.py", line 153, in error_handler
        raise e.with_traceback(filtered_tb) from None
      File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/execute.py", line 58, in quick_execute
        tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
    UnicodeDecodeError: 'utf-8' codec can't decode byte 0x96 in position 40: invalid start byte
    """
    
    The above exception was the direct cause of the following exception:
    
    Traceback (most recent call last):
      File "prerender.py", line 836, in <module>
        main()
      File "prerender.py", line 832, in main
        r.get()
      File "/usr/lib/python3.8/multiprocessing/pool.py", line 771, in get
        raise self._value
    UnicodeDecodeError: 'utf-8' codec can't decode byte 0x96 in position 40: invalid start byte
    
    opened by karanveersingh5623 9
  • About preprocessing

    About preprocessing

    Congrats! Actually I have a question which has confused me a few days. When we rasterize the map information, why we need to shift and rotate the local coordinate system that the target agent is located at a specific location? I konw that many papers has used this method like buliding a relative coordinate system and the center is target agent. why? and what is the meaning of "to eliminate the redundant degrees of freedom" in your report? Thank you in advance!

    opened by zyandtom 1
Releases(0.1)
Style-based Neural Drum Synthesis with GAN inversion

Style-based Drum Synthesis with GAN Inversion Demo TensorFlow implementation of a style-based version of the adversarial drum synth (ADS) from the pap

Sound and Music Analysis (SoMA) Group 29 Nov 19, 2022
IEEE-CIS Technical Challenge on Predict+Optimize for Renewable Energy Scheduling

IEEE-CIS Technical Challenge on Predict+Optimize for Renewable Energy Scheduling This is my code, data and approach for the IEEE-CIS Technical Challen

3 Sep 18, 2022
Scenic: A Jax Library for Computer Vision and Beyond

Scenic Scenic is a codebase with a focus on research around attention-based models for computer vision. Scenic has been successfully used to develop c

Google Research 1.6k Dec 27, 2022
Joint parameterization and fitting of stroke clusters

StrokeStrip: Joint Parameterization and Fitting of Stroke Clusters Dave Pagurek van Mossel1, Chenxi Liu1, Nicholas Vining1,2, Mikhail Bessmeltsev3, Al

Dave Pagurek 44 Dec 01, 2022
Code for the paper: Adversarial Machine Learning: Bayesian Perspectives

Code for the paper: Adversarial Machine Learning: Bayesian Perspectives This repository contains code for reproducing the experiments in the ** Advers

Roi Naveiro 2 Nov 11, 2022
SPTAG: A library for fast approximate nearest neighbor search

SPTAG: A library for fast approximate nearest neighbor search SPTAG SPTAG (Space Partition Tree And Graph) is a library for large scale vector approxi

Microsoft 4.3k Jan 01, 2023
An official source code for "Augmentation-Free Self-Supervised Learning on Graphs"

Augmentation-Free Self-Supervised Learning on Graphs An official source code for Augmentation-Free Self-Supervised Learning on Graphs paper, accepted

Namkyeong Lee 59 Dec 01, 2022
Deep Learning Tutorial for Kaggle Ultrasound Nerve Segmentation competition, using Keras

Deep Learning Tutorial for Kaggle Ultrasound Nerve Segmentation competition, using Keras This tutorial shows how to use Keras library to build deep ne

Marko Jocić 922 Dec 19, 2022
PyTorch implementation of our ICCV 2021 paper, Interpretation of Emergent Communication in Heterogeneous Collaborative Embodied Agents.

PyTorch implementation of our ICCV 2021 paper, Interpretation of Emergent Communication in Heterogeneous Collaborative Embodied Agents.

Saim Wani 4 May 08, 2022
VM3000 Microphones

VM3000-Microphones This project was completed by Ricky Leman under the supervision of Dr Ben Travaglione and Professor Melinda Hodkiewicz as part of t

UWA System Health Lab 0 Jun 04, 2021
Code for "Intra-hour Photovoltaic Generation Forecasting based on Multi-source Data and Deep Learning Methods."

pv_predict_unet-lstm Code for "Intra-hour Photovoltaic Generation Forecasting based on Multi-source Data and Deep Learning Methods." IEEE Transactions

FolkScientistInDL 8 Oct 08, 2022
DeepFaceEditing: Deep Face Generation and Editing with Disentangled Geometry and Appearance Control

DeepFaceEditing: Deep Face Generation and Editing with Disentangled Geometry and Appearance Control One version of our system is implemented using the

260 Nov 28, 2022
Supplementary code for SIGGRAPH 2021 paper: Discovering Diverse Athletic Jumping Strategies

SIGGRAPH 2021: Discovering Diverse Athletic Jumping Strategies project page paper demo video Prerequisites Important Notes We suspect there are bugs i

54 Dec 06, 2022
Blind visual quality assessment on 360° Video based on progressive learning

Blind visual quality assessment on omnidirectional or 360 video (ProVQA) Blind VQA for 360° Video via Progressively Learning from Pixels, Frames and V

5 Jan 06, 2023
Code from Daniel Lemire, A Better Alternative to Piecewise Linear Time Series Segmentation

PiecewiseLinearTimeSeriesApproximation code from Daniel Lemire, A Better Alternative to Piecewise Linear Time Series Segmentation, SIAM Data Mining 20

Daniel Lemire 21 Oct 27, 2022
PyTorch implementation of Federated Learning with Non-IID Data, and federated learning algorithms, including FedAvg, FedProx.

Federated Learning with Non-IID Data This is an implementation of the following paper: Yue Zhao, Meng Li, Liangzhen Lai, Naveen Suda, Damon Civin, Vik

Youngjoon Lee 48 Dec 29, 2022
This is the official Pytorch implementation of "Lung Segmentation from Chest X-rays using Variational Data Imputation", Raghavendra Selvan et al. 2020

README This is the official Pytorch implementation of "Lung Segmentation from Chest X-rays using Variational Data Imputation", Raghavendra Selvan et a

Raghav 42 Dec 15, 2022
Joint Gaussian Graphical Model Estimation: A Survey

Joint Gaussian Graphical Model Estimation: A Survey Test Models Fused graphical lasso [1] Group graphical lasso [1] Graphical lasso [1] Doubly joint s

Koyejo Lab 1 Aug 10, 2022
Team Enigma at ArgMining 2021 Shared Task: Leveraging Pretrained Language Models for Key Point Matching

Team Enigma at ArgMining 2021 Shared Task: Leveraging Pretrained Language Models for Key Point Matching This is our attempt of the shared task on Quan

Manav Nitin Kapadnis 12 Jul 08, 2022
TensorFlow CNN for fast style transfer

Fast Style Transfer in TensorFlow Add styles from famous paintings to any photo in a fraction of a second! It takes 100ms on a 2015 Titan X to style t

1 Dec 14, 2021