דרושים » תוכנה » Senior Big Data Developer - 2576

We are seeking an experienced Solutions Data Engineer who possess both technical depth and strong interpersonal skills to partner with internal and external teams to develop scalable, flexible, and cutting-edge solutions. Solutions Engineers collaborate with operations and business development to help craft solutions to meet customer business problems.
A Solutions Engineer works to balance various aspects of the project, from safety to design. Additionally, a Solutions Engineer researches advanced technology regarding best practices in the field and seek to find cost-effective solutions.
Job Description:
Were looking for a Solutions Engineer with deep experience in Big Data technologies, real-time data pipelines, and scalable infrastructuresomeone whos been delivering critical systems under pressure, and knows what it takes to bring complex data architectures to life. This isnt just about checking boxes on tech stacksits about solving real-world data problems, collaborating with smart people, and building robust, future-proof solutions.
In this role, youll partner closely with engineering, product, and customers to design and deliver high-impact systems that move, transform, and serve data at scale. Youll help customers architect pipelines that are not only performant and cost-efficient but also easy to operate and evolve.
We want someone whos comfortable switching hats between low-level debugging, high-level architecture, and communicating clearly with stakeholders of all technical levels.
Key Responsibilities:
Build distributed data pipelines using technologies like Kafka, Spark (batch & streaming), Python, Trino, Airflow, and S3-compatible data lakesdesigned for scale, modularity, and seamless integration across real-time and batch workloads.
Design, deploy, and troubleshoot hybrid cloud/on-prem environments using Terraform, Docker, Kubernetes, and CI/CD automation tools.
Implement event-driven and serverless workflows with precise control over latency, throughput, and fault tolerance trade-offs.
Create technical guides, architecture docs, and demo pipelines to support onboarding, evangelize best practices, and accelerate adoption across engineering, product, and customer-facing teams.
Integrate data validation, observability tools, and governance directly into the pipeline lifecycle.
Own end-to-end platform lifecycle: ingestion → transformation → storage (Parquet/ORC on S3) → compute layer (Trino/Spark).
Benchmark and tune storage backends (S3/NFS/SMB) and compute layers for throughput, latency, and scalability using production datasets.
Work cross-functionally with R&D to push performance limits across interactive, streaming, and ML-ready analytics workloads.
Operate and debug object storebacked data lake infrastructure, enabling schema-on-read access, high-throughput ingestion, advanced searching strategies, and performance tuning for large-scale workloads.

Shape the Future of Data - Join our mission to build the foundational pipelines and tools that power measurement, insights, and decision-making across our product, analytics, and leadership teams.
Develop the Platform Infrastructure - Build the core infrastructure that powers our data ecosystem including the Kafka events-system, DDL management with Terraform, internal data APIs on top of Databricks, and custom admin tools (e.g. Django-based interfaces).
Build Real-time Analytical Applications - Develop internal web applications to provide real-time visibility into platform behavior, operational metrics, and business KPIs integrating data engineering with user-facing insights.
Solve Meaningful Problems with the Right Tools - Tackle complex data challenges using modern technologies such as Spark, Kafka, Databricks, AWS, Airflow, and Python. Think creatively to make the hard things simple.
Own It End-to-End - Design, build, and scale our high-quality data platform by developing reliable and efficient data pipelines. Take ownership from concept to production and long-term maintenance.
Collaborate Cross-Functionally - Partner closely with backend engineers, data analysts, and data scientists to drive initiatives from both a platform and business perspective. Help translate ideas into robust data solutions.
Optimize for Analytics and Action - Design and deliver datasets in the right shape, location, and format to maximize usability and impact - whether thats through lakehouse tables, real-time streams, or analytics-optimized storage.
You will report to the Data Engineering Team Lead and help shape a culture of technical excellence, ownership, and impact.

We are seeking a Senior Automation Engineer to join our Big Data team, where we build cutting-edge, AI-driven applications that empower the worlds largest financial institutions in the fight against financial crime.
Youll be part of a dynamic group responsible for the analytics and developer experience foundations, enabling software developers, data engineers, and data scientists to deliver scalable, high-quality analytics solutions quickly. In this role, youll ensure the quality, reliability, and performance of our intelligent cloud-based products through robust automated testing.
How will you make an impact?
Design, develop, and maintain automated tests for backend systems and data pipelines.
Implement and manage a scalable testing framework to support continuous integration and delivery.
Create test plans, and maintain detailed documentation of testing processes, results, and quality metrics.
Collaborate with developers and product managers to understand system architecture and business requirements.
Participate in requirements analysis, technical reviews, and test planning.
Identify, document, and track defects, drive resolution with development teams.
Monitor production environments, investigate anomalies, and push for timely fixes.
Ensure data health and correctness across systems.
Continuously improve QA processes, tools, and best practices.
Advocate for quality across the development lifecycle.

We are seeking an experienced Data Platform Engineer to join our Storage Analytics team. You will design and build data solutions that provide critical insights into storage performance and usage across Apple's entire device ecosystem.
Description:
Working with large-scale telemetry data from millions of Apple devices worldwide, you'll support multiple CoreOS Storage teams, including Software Update, Backup/Restore/Migration, Storage Attribution, and other storage domains.
Responsibilities
Design, build, and maintain scalable data processing infrastructure to handle large-scale telemetry from Apple's global device fleet
Develop highly scalable data pipelines to ingest and process storage performance metrics, usage patterns, and system telemetry with actionable alerting and anomaly detection
Build and maintain robust data platforms and ETL frameworks that enable CoreOS Storage teams to access, process, and derive insights from storage telemetry data
Engineer automated data delivery systems and APIs that serve processed storage metrics to various engineering teams across different storage domains

This role provides the opportunity to lead end-to-end design & implementation of advanced server-side features using a wide range of technologies: Apache Spark, Apache Airflow, Scala, k8s, Node.js (JS/Typescript) with MongoDB, Redis, Kafka, Dockers and Flink (Big Data Stream Processing)

If you are up for the challenge and you have the XM factor, come and join us!

As part of the Data Infrastructure group, youll help build Lemonades data platform for our growing stack of products, customers, and microservices.

We ingest our data from our operational DBs, telematics devices, and more, working with several data types (both structured and unstructured). Our challenge is to provide building tools and infrastructure to empower other teams leveraging data-mesh concepts.

In this role youll:
Help build Lemonades data platform, designing and implementing data solutions for all application requirements in a distributed microservices environment

Build data-platform ingestion layers using streaming ETLs and Change Data Capture

Implement pipelines and scheduling infrastructures

Ensure compliance, data-quality monitoring, and data governance on data platform

Implement large-scale batch and streaming pipelines with data processing frameworks

Collaborate with other Data Engineers, Developers, BI Engineers, ML Engineers, Data Scientists, Analysts and Product managers

Share knowledge with other team members and promote engineering standards

Join our team and lead the design, implementation and maintaining cutting-edge Data Platforms and Big Data Solutions. In this position, you will have the unique opportunity to significantly impact intelligence processes, contribute to the counter terrorism efforts and leveraging advanced Big Data technologies.
Design, install, maintain and upgrade Big Data technologies
Monitor and optimize system performance, identifying areas for improving and implementing solutions
Provide technical support to internal teams and end users, ensuring seamless operations
Design and implement advanced infrastructure solutions using the Hadoop Ecosystem
Analyze user requirements, evaluate and recommend new technologies in the Big Data domain.

We are looking for a talented and dedicated ML & Big Data Analyst who will be a part of the Core Data group.
As part of this role, you will help us process and streamline rich and complicated datasets. Will explore and validate new data sources. Be part of the data science and data engineering force creating new features for our solution.
What does the day-to-day of a ML & Big Data Analyst at our company look like:
Apply your expertise in quantitative analysis and data mining to turn data into insights.
Conduct research and develop tools that will ensure the quality of our companys data and algorithms.
Utilize your deep understanding of our companys data in order to evaluate and improve novel algorithms produced by the data scientists in your team.
Partner with Big Data Engineers to establish an infrastructure to scale solutions.

we are a global pioneer of RADAR systems for active military protection, counter-drone applications, critical infrastructure protection, and border surveillance.
Were seeking a Data Tech Lead to drive technical excellence in data engineering and analytics. As the go-to expert, youll set the technical direction, optimize data pipelines, and tackle programming challengesclosing knowledge gaps, solving data-related questions, and streamlining operations. Youll also design scalable architectures, manage ETL workflows, and enhance data processing efficiency.
Key Responsibilities:
Oversee the technical aspects of data projects by making architectural and design decisions.
Streamline existing operations and implement improvements with the teams collaboration.
Guiding team members in technical matters, and supervising system modifications.
Conducting Code reviews for data analysts, BI Analysts and data engineers.
Bridge technical knowledge gaps within the data team, answering critical product-related questions.

We are seeking an experienced Software Engineer to join a new and groundbreaking R&D team in the filed of cyber security with a focus on internet communication and cryptography.
The team specializes in researching and developing solutions in Cyber Security, Cryptography and Big Data
The team involves in a wide range of solutions for engineering and algorithmic challenges at high scale
A unique integration of Big Data, System Development, Cryptography, Networking and Cyber Security domains
The development process spans from technological ideation to the delivery of the final product .