The data Engineer is responsible for designing, developing, and maintaining data architecture and pipelines, ensuring the availability, reliability, and efficiency of our data infrastructure.
The data Engineer is using ETL (Extract, Transform, Load) or ELT processes to support data integration and warehousing initiatives, working closely with business stakeholders, data analysts, and other members of the development team to ensure the efficient flow of data across various systems and databases and implement solutions to support our organization's data -driven initiatives.
Requirements: * Design, build, and maintain scalable and reliable data pipelines for ingesting, processing, and transforming large volumes of data from diverse sources.
* Develop and optimize ETL processes to extract data from multiple sources, transform it according to business requirements, and load it into target databases/ data warehouses cleanse, thus validating and enriching raw data for analytics and reporting purposes.
* Collaborate with business analysts and stakeholders to understand data requirements and translate them into technical specifications for ETL development.
* Optimize and tune SSIS packages for performance, scalability, and reliability.
* Troubleshoot and resolve issues related to data integration, ETL processes, and data quality.
* Develop and maintain documentation for ETL processes, data mappings, and data lineage.
* Implement data validation and quality checks to ensure accuracy and completeness of data
* Work with database administrators to ensure proper configuration and optimization of database servers and Storage for ETL processes.
* Keep abreast of industry trends and best practices in ETL development and data integration.
* Develop and maintain monitoring and alerting systems to proactively identify and address data pipeline issues and bottlenecks.
* Enable the core processes of data visualization tools (e.g., Tableau, Power BI).
* Performing automation and scripting tasks. ***With the migration to Clouds solutions, the data engineer is to manage the process and to design and maintain data warehouses, data lakes, and other Storage solutions to proliferate and refine structured and unstructured data efficiently.
This position is open to all candidates.
