We are looking for a We are looking for a.
As a Big Data Team Lead, you will lead a core team focused on the ingestion of big data and the development of core algorithms. Your role will involve building scalable and robust solutions for processing massive geospatial datasets, including spatially-focused algorithms. You will also be responsible for developing innovative big data solutions to ensure accuracy and efficiency within a fast-paced and evolving technical ecosystem. This position requires a passionate leader with a strong background in big data technologies and a drive to innovate.
RESPONSIBILITIES:
Lead and manage a big data engineering team, focused on processing high-volume spatial-temporal data, while also engaging in hands-on technical tasks.
Collaborate daily with product managers, R&D stakeholders, and data scientists to translate requirements into robust data systems.
Prioritize and scope incoming feature requests and technical improvements to balance innovation with stability.
Maintain and evolve the teams Apache Spark-based data pipelines, including optimization and scaling.
Lead technical planning sessions, foster ownership and accountability within the team.
Drive project-level KPIs and data quality KPIs such as pipeline completion time, job reliability, data freshness metrics and data accuracy.
Manage 36 direct reports.
Requirements: 2 years of experience in leading a data engineering team
Strong expertise in Apache Spark, and big data pipeline design.
Deep understanding of big data architecture in cloud environments, ideally GCP (e.g., Dataproc, BigQuery, GCS, Composer).
Problem-solving mindset with the ability to diagnose complex system behavior and debug issues across layers of the stack.
Experience with geospatial data or spatial analytics (Advantage).
Familiarity with Airflow (Advantage) Proficiency in Scala (Advantage)
Effective communicator with adaptability and prioritization skills to align cross-functional teams and balance short- and long-term goals.
BSc in Computer Science, Engineering or a related technical field (or equivalent experience in the industry).
This position is open to all candidates.