Required Data Engineer Team Leader
Your Mission:
As a Data Engineer Team Leader, your mission will be to lead a team of Data engineers in architecting, designing, and implementing robust data infrastructures that empower our analytics, data science, and business teams. You'll play a pivotal role in harnessing the full potential of our data by constructing scalable, efficient, and real-time data pipelines. Your expertise will enable us to process vast amounts of data, ensuring that every byte is accessible, reliable, and actionable.
As a leader, you will drive the next generation of our data capabilities and mentor team members to excel in their roles. Every line of code and every data pipeline optimized will fuel our mission, enabling us to leverage business insights that redefine industry standards and propel us ahead of the competition.
Responsibilities:
Lead the design, construction, installation, and maintenance of large-scale processing systems and other infrastructure.
Build and maintain data pipelines to support the needs of the companys business intelligence, analytics, and data science teams.
Collaborate with other teams to extract, transform, and load (ETL) data from various sources, ensuring data integrity and accuracy.
Implement strategies to improve data reliability, efficiency, and quality, setting high standards for the team to follow.
Ensure architectures support the demands of rapid data growth, while also considering scalability and performance optimization.
Work closely with data architects, modelers, DevOps, and IT team members to align project goals and drive successful outcomes.
Lead the design and implementation of cloud-based solutions, with a focus on Microsoft Azure, and AWS services, while guiding team members in best practices.
Optimize data retrieval processes and oversee the development of dashboards and visualizations for stakeholders.
Manage, Mentor, and guide your team members in developing batch & real-time data processing solutions using tools like Spark and Databricks.
Actively participate in recruiting, hiring, and onboarding new team members, ensuring a cohesive and skilled team.
Lead a culture of innovation, collaboration, and continuous learning within the team.
Requirements: A Bachelor's or Master's degree in computer science, Information Systems, or a related field.
At least 4 years as a Data Engineer with hands-on experience with Spark, Databricks, AWS, Python, and SQL, NoSQL databases.
Proven leadership and management skills, including experience in leading and motivating a team of data engineers, setting clear objectives, providing regular feedback, and fostering professional development.
Extensive experience with cloud architectures and services, specifically AWS (e.g., S3, EC2, RDS, Lambda, Glue, Redshift) or Azures equivalent.
Strong understanding of ETL techniques and frameworks, with proven leadership in implementing them effectively.
Solid understanding of big data technologies and architecture, especially in distributed computing environments.
Familiarity with big data tools like Hadoop, Kafka, and Flink is a plus.
Exceptional analytical and problem-solving skills, with a track record of leading successful projects.
Excellent communication and interpersonal skills, with the ability to convey complex technical information to non-technical stakeholders.
Experience with the following technologies : Apache Airflow , Prefect, Docker and Kubernetes
Demonstrated ability to manage multiple projects simultaneously, prioritize tasks effectively, and drive results in a fast-paced environment.
Experience in collaborating with cross-functional teams and senior management to align data engineering initiatives with business objectives and strategic goals.
Strong decision-making skills, with the ability to make informed judgments and resolve conflicts efficiently.
This position is open to all candidates.