Home / Current Issue / Paper 1712020
Review of Smart Auto X: Autonomous Self Driving Car Using IoT and Deep Learning Technologies
Subject area: Science,Engineering and Technology · Area of research: Autonomous Vehicles
Abstract
This study introduces a comprehensive framework for autonomous vehicle control that integrates data-driven modelling, deep learning, and multi-sensor fusion to enhance robustness and adaptability within dynamic driving contexts. Initially, the nonlinear vehicle dynamics are modelled using the Deep Koopman Operator (DK) methodology, wherein deep neural networks extract basis functions to approximate the infinite-dimensional Koopman operator within a lifted linear space. An Extended State Observer (ESO) is utilized to estimate total disturbances in real time and to compensate for model uncertainties, resulting in an ESO-based Deep Koopman Model Predictive Control (ESO-DKMPC) scheme that improves trajectory-tracking precision. To further bolster policy robustness, imitation learning is incorporated via perturbation-based data augmentation, facilitating effective generalization to previously unseen driving scenarios. Concurrently, an end-to-end convolutional neural network (CNN) is employed to directly map raw camera inputs to steering commands, thereby jointly optimizing perception, planning, and control processes. Additionally, sensor fusion techniques that integrate LiDAR, GNSS, IMU, and wheel encoder data are applied to enhance localization accuracy and navigation resilience in complex environments. Simulation and co-simulation experiments conducted on the Car Sim/Simulink platform demonstrate that the proposed hybrid control framework outperforms traditional linear and nonlinear model predictive control approaches, as well as standalone learning-based methods, in terms of tracking performance and generalization capability.
Keywords
Autonomous vehicles, Deep Koopman operator, Model predictive control, Extended state observer, Imitation learning, End-to-end learning, Sensor fusion, Vehicle dynamics, Deep neural networks, Robust control.
References
[1] H. Chen and C. Lv, “Incorporating ESO into Deep Koopman Operator Modelling for Control of Autonomous Vehicles,” IEEE Journal of Intelligent and Fuzzy Systems, 2023. [Online]. Available: 🔗 https://doi.org/10.48550/arXiv.2305.XXXX
[2] M. Bansal, A. Krizhevsky, and A. Ogale, “ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worst,” arXiv preprint, arXiv:1812.03079, 2018. [Online]. Available: 🔗 https://arxiv.org/abs/1812.03079
[3] M. Bojarski et al., “End to End Learning for Self-Driving Cars,” arXiv preprint, arXiv:1604.07316, 2016. [Online]. Available: 🔗 https://arxiv.org/abs/1604.07316
[4] Q. Li, J. Peña Queralta, T. N. Gia, Z. Zou, and T. Westerlund, “Multi-Sensor Fusion for Navigation and Mapping in Autonomous Vehicles: Accurate Localization in Urban Environments,” arXiv preprint, arXiv:2103.13719, 2021. [Online]. Available: 🔗 https://arxiv.org/abs/2103.13719
[5] K. Vinoth and P. Sasikumar, “Multi-Sensor Fusion and Segmentation for Autonomous Vehicle Multi-Object Tracking Using Deep Q Networks,” Scientific Reports, vol. 14, no. 31130, 2024. [Online]. Available: 🔗 https://doi.org/10.1038/s41598-024-82356-0
How to cite this paper
@article{1712020,
author = {Sayee Gosavi, Snehal Pandurang Dimble, Omkar Mukund Bodke, Parth Dilip Kalbhor, Prof. Priti R. Vanmali},
title = {Review of Smart Auto X: Autonomous Self Driving Car Using IoT and Deep Learning Technologies},
journal = {Iconic Research And Engineering Journals},
year = {2025},
volume = {9},
number = {5},
pages = {848-851},
issn = {2456-8880},
url = {https://www.irejournals.com/formatedpaper/1712020.pdf},
abstract = {This study introduces a comprehensive framework for autonomous vehicle control that integrates data-driven modelling, deep learning, and multi-sensor fusion to enhance robustness and adaptability within dynamic driving contexts. Initially, the nonlinear vehicle dynamics are modelled using the Deep Koopman Operator (DK) methodology, wherein deep neural networks extract basis functions to approximate the infinite-dimensional Koopman operator within a lifted linear space. An Extended State Observer (ESO) is utilized to estimate total disturbances in real time and to compensate for model uncertainties, resulting in an ESO-based Deep Koopman Model Predictive Control (ESO-DKMPC) scheme that improves trajectory-tracking precision. To further bolster policy robustness, imitation learning is incorporated via perturbation-based data augmentation, facilitating effective generalization to previously unseen driving scenarios. Concurrently, an end-to-end convolutional neural network (CNN) is employed to directly map raw camera inputs to steering commands, thereby jointly optimizing perception, planning, and control processes. Additionally, sensor fusion techniques that integrate LiDAR, GNSS, IMU, and wheel encoder data are applied to enhance localization accuracy and navigation resilience in complex environments. Simulation and co-simulation experiments conducted on the Car Sim/Simulink platform demonstrate that the proposed hybrid control framework outperforms traditional linear and nonlinear model predictive control approaches, as well as standalone learning-based methods, in terms of tracking performance and generalization capability.},
keywords = {Autonomous vehicles, Deep Koopman operator, Model predictive control, Extended state observer, Imitation learning, End-to-end learning, Sensor fusion, Vehicle dynamics, Deep neural networks, Robust control.},
month = {November},
doi = {https://doi.org/10.64388/IREV9I5-1712020}
}