Robotic perception systems often include approaches that can extract valuable features or information from the studied dataset. These methods often involve the application of deep learning approaches, such as convolutional neural networks (CNNs), for processing of images, as well as the incorporation of 3D data. The notion of image categorization is well delineated via the use of networks that include convolutional networks. However, some network topologies exhibit a substantial scope and need significant amounts of time and memory resources. On the other hand, the neural networks FlowNet3D and PointFlowNet have the capability to accurately predict scene flow. Specifically, these networks are capable of estimating the three-dimensional movements of point clouds (PCs) within a dynamic environment. When using PCs in robotic applications, it is crucial to examine the robustness of accurately recognizing the points that belong to the object. This article examines the use of robotic perception systems inside autonomous vehicles and the inherent difficulties linked to the analysis and processing of information obtained from diverse sensors. The researchers put out a late fusion methodology that integrates the results of many classifiers in order to enhance the accuracy of categorization. Additionally, the authors propose a weighted fusion technique that incorporates the proximity to objects as a significant factor. The findings indicate that the fusion methods described in this study exhibit superior performance compared to both single modality classification and classic fusion strategies.
Keywords
Object Classification, Intelligent Robotic Perception System, Robotic Perception Environments, Late Fusion Strategies, Deep Learning, Convolutional Neural Networks.
A. Elfes, “Using occupancy grids for mobile robot perception and navigation,” IEEE Computer, vol. 22, no. 6, pp. 46–57, Jun. 1989, doi: 10.1109/2.30720.
R. Siegwart, I. Nourbakhsh, and D. Scaramuzza, “Introduction to autonomous mobile robots,” Choice Reviews Online, vol. 49, no. 03, pp. 49–1492, Nov. 2011, doi: 10.5860/choice.49-1492.
I. Kim et al., “Nanophotonics for light detection and ranging technology,” Nature Nanotechnology, vol. 16, no. 5, pp. 508–524, May 2021, doi: 10.1038/s41565-021-00895-3.
B. W. Parkinson and J. J. Spilker, Global positioning system : theory and applications. 1996, p. 114. [Online]. Available: https://arc.aiaa.org/doi/pdf/10.2514/5.9781600866388.0000.0000
M. Sun, Z. Zhao, and X. Ma, “Sensing and Handling Engagement Dynamics in Human-Robot Interaction Involving Peripheral Computing Devices,” 2010 5th ACM/IEEE International Conference on Human-Robot Interaction (HRI), May 2017, doi: 10.1145/3025453.3025469.
N. Zeng, H. Zhang, Y. Chen, B. Chen, and Y. Liu, “Path planning for intelligent robot based on switching local evolutionary PSO algorithm,” Assembly Automation, vol. 36, no. 2, pp. 120–126, Apr. 2016, doi: 10.1108/aa-10-2015-079.
E. Macias-Garcia, D. Galeana-Pérez, J. Medrano-Hermosillo, and E. Bayro–Corrochano, “Multi-stage deep learning perception system for mobile robots,” Integrated Computer-aided Engineering, vol. 28, no. 2, pp. 191–205, Mar. 2021, doi: 10.3233/ica-200640.
C. Dong, “Remote sensing, hydrological modeling and in situ observations in snow cover research: A review,” Journal of Hydrology, vol. 561, pp. 573–583, Jun. 2018, doi: 10.1016/j.jhydrol.2018.04.027.
A. Buosciolo, G. Pesce, and A. Sasso, “New calibration method for position detector for simultaneous measurements of force constants and local viscosity in optical tweezers,” Optics Communications, vol. 230, no. 4–6, pp. 357–368, Feb. 2004, doi: 10.1016/j.optcom.2003.11.062.
I. Nesnas, L. Fesq, and R. Volpe, “Autonomy for space robots: past, present, and future,” Current Robotics Reports, vol. 2, no. 3, pp. 251–263, Jun. 2021, doi: 10.1007/s43154-021-00057-2.
T. Fukuda and N. Kubota, “An intelligent robotic system based on a fuzzy approach,” Proceedings of the IEEE, vol. 87, no. 9, pp. 1448–1470, Jan. 1999, doi: 10.1109/5.784220.
W. Ma, X. Zhang, and G. Yin, “Design on intelligent perception system for lower limb rehabilitation exoskeleton robot,” 2016 13th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI), Aug. 2016, doi: 10.1109/urai.2016.7625785.
S. Cebollada, L. Payá, M. Flores, and L. Payá, “A state-of-the-art review on mobile robotics tasks using artificial intelligence and visual data,” Expert Systems With Applications, vol. 167, p. 114195, Apr. 2021, doi: 10.1016/j.eswa.2020.114195.
F. Semeraro, A. Griffiths, and A. Cangelosi, “Human–robot collaboration and machine learning: A systematic review of recent research,” Robotics and Computer-Integrated Manufacturing, vol. 79, p. 102432, Feb. 2023, doi: 10.1016/j.rcim.2022.102432.
J. R. Mosig, “The Weighted Averages algorithm revisited,” IEEE Transactions on Antennas and Propagation, vol. 60, no. 4, pp. 2011–2018, Apr. 2012, doi: 10.1109/tap.2012.2186244.
A. Behl, D. Paschalidou, S. Donné, and A. Geiger, “PointFlowNet: Learning Representations for Rigid Motion Estimation From Point Clouds,” 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Jun. 2019, doi: 10.1109/CVPR.2019.00815.
X. Liu, C. R. Qi, and L. J. Guibas, “FlowNet3D: Learning Scene Flow in 3D Point Clouds,” 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Jun. 2019, doi: 10.1109/CVPR.2019.00062.
Md. Z. Hussain, M. Ashraf, D. K. Singh, A. Haldorai, D. K. Mishra, and T. N. Shanavas, “Intelligent data post and read data system like to feed for IoT sensors,” International Journal of System Assurance Engineering and Management, Jun. 2022, doi: 10.1007/s13198-022-01683-5.
S. Ji, W. Xu, M. Yang, and K. Yu, “3D convolutional neural networks for human action recognition,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 1, pp. 221–231, Jan. 2013, doi: 10.1109/tpami.2012.59.
R. Furuta, C. Wild, Y. Weng, and C. D. Weiss, “Capture of an early fusion-active conformation of HIV-1 gp41,” Nature Structural & Molecular Biology, vol. 5, no. 4, pp. 276–279, Apr. 1998, doi: 10.1038/nsb0498-276.
C. K. Mohan, N. Dhananjaya, and B. Yegnanarayana, “Video Shot Segmentation Using Late Fusion Technique,” 2008 Seventh International Conference on Machine Learning and Applications, Jan. 2008, doi: 10.1109/icmla.2008.88.
J. Arévalo, T. Solorio, M. Montes-Y-Gómez, and F. A. González, “Gated multimodal units for information fusion,” arXiv (Cornell University), Feb. 2017, [Online]. Available: https://arxiv.org/pdf/1702.01992.pdf
D. Hu, C. Wang, F. Nie, and X. Li, “Dense Multimodal Fusion for Hierarchically Joint Representation,” ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), May 2019, doi: 10.1109/icassp.2019.8683898.
Q. Dai, X. Cheng, Y. Qiao, and Y. Zhang, “Agricultural Pest Super-Resolution and identification with attention enhanced residual and dense fusion generative and adversarial network,” IEEE Access, vol. 8, pp. 81943–81959, Jan. 2020, doi: 10.1109/access.2020.2991552.
V. De Silva, J. Roche, and A. M. Kondoz, “Robust fusion of LiDAR and Wide-Angle camera data for autonomous mobile robots,” Sensors, vol. 18, no. 8, p. 2730, Aug. 2018, doi: 10.3390/s18082730.
Acknowledgements
Authors thank Reviewers for taking the time and effort necessary to review the manuscript.
Funding
No funding was received to assist with the preparation of this manuscript.
Ethics declarations
Conflict of interest
The authors have no conflicts of interest to declare that are relevant to the content of this article.
Availability of data and materials
No data available for above study.
Author information
Contributions
All authors have equal contribution in the paper and all authors have read and agreed to the published version of the manuscript.
Corresponding author
Rodney Adam
Rodney Adam
School of computer science, The University of Sydney, Camperdown NSW 2050, Australia.
Open Access This article is licensed under a Creative Commons Attribution NoDerivs is a more restrictive license. It allows you to redistribute the material commercially or non-commercially but the user cannot make any changes whatsoever to the original, i.e. no derivatives of the original work. To view a copy of this license, visit https://creativecommons.org/licenses/by-nc-nd/4.0/
Cite this article
Rodney Adam and Anandakumar Haldorai, “Optimizing Object Classification in Robotic Perception Environments Exploring Late Fusion Strategies”, Journal of Robotics Spectrum, vol.2, pp. 076-086, 2024. doi: 10.53759/9852/JRS202402008.
