The representation of facial features in three-dimensional space plays a pivotal role in various applications such as facial recognition, virtual reality, and digital entertainment. However, achieving high-fidelity reconstructions from two-dimensional facial images remains a challenging task, particularly in preserving fine texture details. This research addresses this problem by proposing a novel approach that leverages a combination of advanced techniques, including Resnet, Flame model, Bi-FPN, and a differential render architecture. The primary objective of this study is to enhance texture details in reconstructed 3D facial images. The integration of Bi-FPN (Bi-directional Feature Pyramid Network) enhances feature extraction and fusion across multiple scales, facilitating the preservation of texture details across different regions of the face. The objective is to accurately represent facial features from 2D images in three-dimensional space. By combining these methods, the proposed framework achieves significant improvements in preserving fine texture details and overall facial structure. Experimental results demonstrate the effectiveness of the approach, suggesting its potential for various applications such as virtual try-on and facial animation.
Keywords
Bi-FPN, 3D face reconstruction, Flame Model, Resnet.
G. Sanil, K. Prakash, S. Prabhu, V. C. Nayak, and S. Sengupta, “2D-3D Facial Image Analysis for Identification of Facial Features Using Machine Learning Algorithms With Hyper-Parameter Optimization for Forensics Applications,” IEEE Access, vol. 11, pp. 82521–82538, 2023, doi: 10.1109/access.2023.3298443.
D. Zeng, Q. Zhao, S. Long, and J. Li, “Examplar coherent 3D face reconstruction from forensic mugshot database,” Image and Vision Computing, vol. 58, pp. 193–203, Feb. 2017, doi: 10.1016/j.imavis.2016.03.001.
W. N. Widanagamaachchi and A. T. Dharmaratne, “3D Face Reconstruction from 2D Images,” 2008 Digital Image Computing: Techniques and Applications, 2008, doi: 10.1109/dicta.2008.83.
Liu, F., Zeng, D., Li, J., Zhao, Q.: Cascaded regressor based 3D face reconstruction from a single arbitrary view image. arXiv preprint (2015), arXiv:1509.06161.
E. Richardson, M. Sela, and R. Kimmel, “3D Face Reconstruction by Learning from Synthetic Data,” 2016 Fourth International Conference on 3D Vision (3DV), Oct. 2016, doi: 10.1109/3dv.2016.56.
Y. Feng, F. Wu, X. Shao, Y. Wang, and X. Zhou, “Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network,” Lecture Notes in Computer Science, pp. 557–574, 2018, doi: 10.1007/978-3-030-01264-9_33.
Y. Deng, J. Yang, S. Xu, D. Chen, Y. Jia, and X. Tong, “Accurate 3D Face Reconstruction With Weakly-Supervised Learning: From Single Image to Image Set,” 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Jun. 2019, doi: 10.1109/cvprw.2019.00038.
J. McDonagh and G. Tzimiropoulos, “Joint Face Detection and Alignment with a Deformable Hough Transform Model,” Computer Vision – ECCV 2016 Workshops, pp. 569–580, 2016, doi: 10.1007/978-3-319-48881-3_39.
X. Zhu, Z. Lei, X. Liu, H. Shi, and S. Z. Li, “Face Alignment Across Large Poses: A 3D Solution,” 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Jun. 2016, doi: 10.1109/cvpr.2016.23.
C. Bhagavatula, C. Zhu, K. Luu, and M. Savvides, “Faster than Real-Time Facial Alignment: A 3D Spatial Transformer Network Approach in Unconstrained Poses,” 2017 IEEE International Conference on Computer Vision (ICCV), Oct. 2017, doi: 10.1109/iccv.2017.429.
Y. Liu, A. Jourabloo, W. Ren, and X. Liu, “Dense Face Alignment,” 2017 IEEE International Conference on Computer Vision Workshops (ICCVW), Oct. 2017, doi: 10.1109/iccvw.2017.190.
X. Tu et al., “3D Face Reconstruction From A Single Image Assisted by 2D Face Images in the Wild,” IEEE Transactions on Multimedia, vol. 23, pp. 1160–1172, 2021, doi: 10.1109/tmm.2020.2993962.
K. He, X. Zhang, S. Ren, and J. Sun, “Deep Residual Learning for Image Recognition,” 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Jun. 2016, doi: 10.1109/cvpr.2016.90.
P. Dou, S. K. Shah, and I. A. Kakadiaris, “End-to-End 3D Face Reconstruction with Deep Neural Networks,” 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Jul. 2017, doi: 10.1109/cvpr.2017.164.
E. Richardson, M. Sela, R. Or-El, and R. Kimmel, “Learning Detailed Face Reconstruction from a Single Image,” 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Jul. 2017, doi: 10.1109/cvpr.2017.589.
E. Richardson, M. Sela, and R. Kimmel, “3D Face Reconstruction by Learning from Synthetic Data,” 2016 Fourth International Conference on 3D Vision (3DV), Oct. 2016, doi: 10.1109/3dv.2016.56.
T. Li, T. Bolkart, M. J. Black, H. Li, and J. Romero, “Learning a model of facial shape and expression from 4D scans,” ACM Transactions on Graphics, vol. 36, no. 6, pp. 1–17, Nov. 2017, doi: 10.1145/3130800.3130813.
J. Guo, X. Zhu, Y. Yang, F. Yang, Z. Lei, and S. Z. Li, “Towards Fast, Accurate and Stable 3D Dense Face Alignment,” Lecture Notes in Computer Science, pp. 152–168, 2020, doi: 10.1007/978-3-030-58529-7_10.
S. Sanyal, T. Bolkart, H. Feng, and M. J. Black, “Learning to Regress 3D Face Shape and Expression From an Image Without 3D Supervision,” 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Jun. 2019, doi: 10.1109/cvpr.2019.00795.
T. Martyniuk, O. Kupyn, Y. Kurlyak, I. Krashenyi, J. Matas, and V. Sharmanska, “DAD-3DHeads: A Large-scale Dense, Accurate and Diverse Dataset for 3D Head Alignment from a Single Image,” 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Jun. 2022, doi: 10.1109/cvpr52688.2022.02027.
X. Zhu, Z. Lei, X. Liu, H. Shi, and S. Z. Li, “Face Alignment Across Large Poses: A 3D Solution,” 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Jun. 2016, doi: 10.1109/cvpr.2016.23.
Acknowledgements
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Funding
No funding was received to assist with the preparation of this manuscript.
Ethics declarations
Conflict of interest
The authors have no conflicts of interest to declare that are relevant to the content of this article.
Availability of data and materials
Data sharing is not applicable to this article as no new data were created or analysed in this study.
Author information
Contributions
All authors have equal contribution in the paper and all authors have read and agreed to the published version of the manuscript.
Corresponding author
Sincy John
Sincy John
Department of Computer Science and Engineering, Christ University, Bangalore, India.
Open Access This article is licensed under a Creative Commons Attribution NoDerivs is a more restrictive license. It allows you to redistribute the material commercially or non-commercially but the user cannot make any changes whatsoever to the original, i.e. no derivatives of the original work. To view a copy of this license, visit https://creativecommons.org/licenses/by-nc-nd/4.0/
Cite this article
Sincy John and Ajit Danti, “3D Face Reconstruction with Feature Enhancement using Bi-FPN for Forensic Analysis", pp. 392-399, April 2024. doi: 10.53759/7669/jmc202404037.
