Sora for hierarchical parallel motion planner: a safe end-to-end method against OOD events

Siyu Teng, Ran Yan, Xiaotong Zhang, Yuchen Li, Xingxia Wang, Yutong Wang, Yonglin Tian, Hui Yu, Lingxi Li, Long Chen, Fei Yue Wang

Research output: Contribution to journalArticlepeer-review

9 Downloads (Pure)


End-to-end motion planners have shown great potential for enabling fully autonomous driving. However, when facing out-of-distribution (OOD) events, these planners might not guarantee the optimal prediction of control commands. To better enhance safety, an end-to-end method that benefits robust and general policy learning from potential OOD events is urgently desirable. In this perspective, Sore4PMP, a hierarchical parallel motion planner, is presented as a suitable solution. Based on raw perception data and descriptive prompts, Sore4PMP can first leverage the advanced generative capabilities of Sora to generate virtual OOD events, and then integrate these events into the decision-making process, thereby enhancing the robustness and generalization of autonomous vehicles (AVs) in emergency scenarios. With a comprehensive outlook, this perspective aims to provide a potential direction for the development of foundation models coupled with autonomous driving and finally promote the safety, efficiency, reliability, and sustainability of AVs.
Original languageEnglish
Pages (from-to)4573-4576
Number of pages4
JournalIEEE Transactions on Intelligent Vehicles
Issue number4
Early online date2 May 2024
Publication statusPublished - 12 Jun 2024


  • Cognition
  • Feature extraction
  • Planning
  • Robustness
  • Task analysis
  • Vectors
  • Visualization

Cite this