2nd International Conference on Materials Science and Sustainable Manufacturing Technology
Implementation of Data Migration and Validation to Azure using Talend
Niranjani V, Logapriya K, Pavani R, Kiruthika M, Ranjith V, Department of Computer Science and Engineering, Sri Eshwar College of Engineering, Coimbatore, India.
This project uses Talend and Azure. Before the source and target systems, data different types, and volume can
be decided, the project's scope must be set the migration process must then be planned, and an Azure storage account must
be created with the proper security settings. Talend is used to extract, transform, and load data from the source system into
the target system utilising Azure components. To ensure that everything is correct, data validation checks are put up, and
Talend is utilised to validate the data in the target system with any problems or errors being fixed as needed. Both user
acceptability testing and post-migration tasks must be finished. In order to assure data's reliability and preciseness,
monitoring is done last. This endeavor necessitates thorough preparation, careful oversight of details, and effective use of
technologies and tools in hopes of successfully completing the transfer and requirements of a specification.
Keywords
ETL, TALEND, Data Migration, Data Validation, Database, Loading.
K. Sharma and V. Attar, “Generalized Big Data Test Framework for ETL migration,” 2016 International Conference on Computing, Analytics and Security Trends (CAST), Dec. 2016, doi: 10.1109/cast.2016.7915025
N. Prasath and J. Sreemathy, “A New Approach for Cloud Data Migration Technique Using Talend ETL Tool,” 2021 7th Conference on Advanced Computing and Communication Systems (ICACCS), Mar. 2021, doi: 10.1109/icaccs51430.2021.9441898.
N. Saranya, R. Brindha, N. Aishwariya, R. Kokila, P. Matheswaran, and P. Poongavi, “Data Migration using ETL Workflow,” 2021 7th International Conference on Advanced Computing and Communication Systems (ICACCS), Mar. 2021, doi:10.1109/icaccs51430.2021.9441840.
J. Sreemathy, S. Priyadharshini, K. Radha, K. Sangeerna, and G. Nivetha, “Data Validation in ETL Using TALEND,” 2019 5th International Conference on Advanced Computing & Communication Systems (ICACCS), Mar. 2019, doi: 10.1109/icaccs.2019.8728420.
C. Shrinivasan, “Data Migration from a Product to a Data Warehouse Using ETL Tool,” 2010 14th European Conference on Software Maintenance and Reengineering, Mar. 2010, doi: 10.1109/csmr.2010.25.
H. Tahir and P. Brezillon, “A shared context approach for supporting experts in data ETL (Extraction, Transformation and Loading) processes,” 2011 11th International Conference on Intelligent Systems Design and Applications, Nov. 2011, doi:10.1109/isda.2011.6121741.
Kamil, M. M. Inggriani, and Y. D. W. Asnar, “Data migration helper using domain information,” 2014 International Conference on Data and Software Engineering (ICODSE), Nov. 2014, doi: 10.1109/icodse.2014.7062492.
P. Pamami, A. Jain, and N. Sharma, “Cloud Migration Metamodel : A framework for legacy to cloud migration,” 2019 9th International Conference on Cloud Computing, Data Science & Engineering (Confluence), Jan. 2019, doi: 10.1109/confluence.2019.8776983.
Y. S. Wijaya and A. A. Arman, “A Framework for Data Migration Between Different Datastore of NoSQL Database,” 2018 International Conference on ICT for Smart Society (ICISS), Oct. 2018, doi: 10.1109/ictss.2018.8549944.
H. Zou, M. Li, Z. Li, and J. Gao, “Design of multi-intelligent data migration strategy based on SDN secondary mode,” 2018 International Conference on Artificial Intelligence and Big Data (ICAIBD), May 2018, doi: 10.1109/icaibd.2018.8396170.
Cite this article
Niranjani V, Logapriya K, Pavani R, Kiruthika M, Ranjith V, “Implementation of Data Migration and Validation to Azure using Talend”, Advances in Computational Intelligence in Materials Science, pp. 073-081, May. 2023. doi:10.53759/acims/978-9914-9946-9-8_13
