לוח משרות דרושים איש ביג דאטה Big Data

we are looking for a Data Engineer to join our growing BI team. This role goes beyond building pipelines. You will help shape our data platform as a shared product - supporting analytics, reporting, and decision-making across key company data domains such as Product, Sales, HR, and others. Your work will directly influence how stakeholders interact with data today and how the platform evolves in the years ahead.

What Youll Be Doing

Architect & Own: Lead the design and development of scalable data warehouse and BI solutions. You will make early-stage architectural decisions and own their long-term impact.
Infrastructure as a Product: Build core data infrastructure and developer experiences that others rely on, ensuring high availability and system reliability.
End-to-End ELT/ETL: Solve complex integration problems by sourcing data from structured and unstructured sources using Rivery, Python, and optimal ETL patterns.
Data Quality & Governance: Implement frameworks for schema evolution, anomaly detection, and data freshness. You will determine security models based on privacy requirements and evolve governance processes.
Strategic Collaboration: Partner with Engineers, Product Managers, and Data Analysts to conceptualize data needs and represent key insights in a meaningful way.
Optimization: Assist in owning production processes, optimizing complex code through advanced algorithmic concepts to manage operational cost-benefit tradeoffs.

As part of our Customers Company Data Engineering team, you will not only build scalable data features and pipelines but also directly enable portfolio growth by supporting new business capabilities in the Credit Card and Servicing business units.

Responsibilities

Design and build data solutions that support Credit Card and Servicing business goals.
Develop advanced data pipelines to support the infrastructure, architecture and the product growth initiatives.
Create ETL/ELT processes and SQL queries to bring data to the data warehouse and other data sources.
Own and evolve data lake pipelines, maintenance, schema management, and improvements.
Collaborate with stakeholders across Product, Backend Engineering, and Data Science to align technical work with business outcomes.
Implement new tools and modern development approaches that improve both scalability and business agility.
Ensure adherence to coding best practices and development of reusable code.
Constantly monitor the data platform and make recommendations to enhance architecture, performance, and cost efficiency.

We're in search of an experienced and skilled Senior Data Engineer to join our growing data team. As part of our data team, you'll be at the forefront of crafting a groundbreaking solution that leverages cutting-edge technology to combat fraud. The ideal candidate will have a strong background in designing and implementing large-scale data solutions, with the potential to grow into a leadership role. This position requires a deep understanding of modern data architectures, cloud technologies, and the ability to drive technical initiatives that align with business objectives.

Our ultimate goal is to equip our clients with resilient safeguards against chargebacks, empowering them to safeguard their revenue and optimize their profitability. Join us on this thrilling mission to redefine the battle against fraud.

Your Arena

Design, develop, and maintain scalable, robust data pipelines and ETL processes
Architect and implement complex data models across various storage solutions
Collaborate with R&D teams, data scientists, analysts, and other stakeholders to understand data requirements and deliver high-quality solutions
Ensure data quality, consistency, security, and compliance across all data systems
Play a key role in defining and implementing data strategies that drive business value
Contribute to the continuous improvement of our data architecture and processes
Champion and implement data engineering best practices across the R&D organization, serving as a technical expert and go-to resource for data-related questions and challenges
Participate in and sometimes lead code reviews to maintain high coding standards
Troubleshoot and resolve complex data-related issues in production environments
Evaluate and recommend new technologies and methodologies to improve our data infrastructure

abra professional services is seeking for an senior data engineer. We are seeking a Senior Data Engineer to lead the technical development of our Data Fabric within a military program. This role involves technical leadership in building contextual data pipelines, working with big data lakehouse architectures, and driving data pipeline automation.

As our Senior Data Engineer you will handle Big Data in real time, and become the owner and gatekeeper of all data and data flows within the company. You will bring data ingenuity and technological excellence while gaining a deep business understanding.

This is an amazing opportunity to join a multi-disciplinary A-team while working in a fast-paced, modern cloud, data-oriented environment.



What Youll Do:

Implement robust, reliable and scalable data pipelines and data architecture.
Own and develop DWH (Petabytes of data!).
Own the entire data development process, including business knowledge, methodology, quality assurance, and monitoring.
Collaborate with cross-functional teams to define, design, and ship new features.
Continuously discover, evaluate, and implement new technologies to maximize development efficiency.
Develop tailormade solutions as part of our data pipelines.
Lead complex Big Data projects and build data platforms from scratch.
Work on a high-scale, real-time, real-world business-critical data stores and data endpoints.
Implement data profiling to identify anomalies and maintain data integrity.
Work in a results-driven, high-paced, rewarding environment.

This is a great opportunity to be part of one of the fastest-growing infrastructure companies in history, an organization that is in the center of the hurricane being created by the revolution in artificial intelligence.
We are seeking an experienced Solutions Data Engineer who possess both technical depth and strong interpersonal skills to partner with internal and external teams to develop scalable, flexible, and cutting-edge solutions. Solutions Engineers collaborate with operations and business development to help craft solutions to meet customer business problems.
A Solutions Engineer works to balance various aspects of the project, from safety to design. Additionally, a Solutions Engineer researches advanced technology regarding best practices in the field and seek to find cost-effective solutions.
Job Description:
Were looking for a Solutions Engineer with deep experience in Big Data technologies, real-time data pipelines, and scalable infrastructure-someone whos been delivering critical systems under pressure, and knows what it takes to bring complex data architectures to life. This isnt just about checking boxes on tech stacks-its about solving real-world data problems, collaborating with smart people, and building robust, future-proof solutions.
In this role, youll partner closely with engineering, product, and customers to design and deliver high-impact systems that move, transform, and serve data at scale. Youll help customers architect pipelines that are not only performant and cost-efficient but also easy to operate and evolve.
We want someone whos comfortable switching hats between low-level debugging, high-level architecture, and communicating clearly with stakeholders of all technical levels.
Key Responsibilities:
Build distributed data pipelines using technologies like Kafka, Spark (batch & streaming), Python, Trino, Airflow, and S3-compatible data lakes-designed for scale, modularity, and seamless integration across real-time and batch workloads.
Design, deploy, and troubleshoot hybrid cloud/on-prem environments using Terraform, Docker, Kubernetes, and CI/CD automation tools.
Implement event-driven and serverless workflows with precise control over latency, throughput, and fault tolerance trade-offs.
Create technical guides, architecture docs, and demo pipelines to support onboarding, evangelize best practices, and accelerate adoption across engineering, product, and customer-facing teams.
Integrate data validation, observability tools, and governance directly into the pipeline lifecycle.
Own end-to-end platform lifecycle: ingestion → transformation → storage (Parquet/ORC on S3) → compute layer (Trino/Spark).
Benchmark and tune storage backends (S3/NFS/SMB) and compute layers for throughput, latency, and scalability using production datasets.
Work cross-functionally with R&D to push performance limits across interactive, streaming, and ML-ready analytics workloads.

Were looking for an experienced Data Engineer to join our DEX Platform and help drive the next stage of our growth. In this role, youll design, build, and scale data infrastructure and advanced processing pipelines that power real-time insights and intelligent automation across our platform.

As a senior data engineer, youll own the full service lifecycle-from architecture and development to deployment, monitoring, and scaling-while ensuring data quality, reliability, and compliance. Youll work closely with backend, AI, and data teams to deliver robust, production-ready systems that support our AI-first strategy and our mission to make the workplace run itself.

We are looking for a talented and experienced big data engineer to join our cyber defence group to take over a cutting-edge data solution.

The successful candidate should have hands-on experience with big data processing technologies and architectures.

The candidate will join our growing team of analysts, developers, data scientists, and architects who design and develop innovative solutions to the most critical needs of our customers.

Responsibilities:
Designing, architecting, implementing, and supporting our data pipelines and processing flows.
Collaborate with analytics department to analyze and understand data sources.+
Provide insights and guidance on database technology and data modelling best practices.
Insure the maintenance of data integrity, managing data and analysis flows with attention to detail and high responsibility.
Implementing algorithms with our data scientists.

We are looking for an Experienced Big Data Engineer who loves solving complex problems at an extreme scale. In this role, you arent just building "internal pipelines"-you are developing the actual product our clients interact with. You will write high-performance code, optimize your own Spark processes for client freshness, and own features end-to-end. We need a "problem solver by nature, optimizer in their soul" who brings both technical depth and a great personality to a high-speed, deep technical team.

What youll do
Build Client-Facing Features: Design and implement high-scale product features for premium clients - from architecture to production deployment.
Performance Engineering: Optimize scalable data pipelines and distributed processes to meet strict client SLAs. Every millisecond of freshness matters.
System Integrity: Ensure uncompromising data accuracy and performance across massive, complex datasets.
Strategic Collaboration: Partner with upstream data providers, client-side teams, and Product Managers to turn raw events into production-grade features.

As a Cloud Architect, you will serve as a technical focal point, driving strategic architectural decisions that shape the future of our globally distributed platform.
Platform & Scale Leadership: Lead the architectural efforts to modernize and significantly scale the companys infrastructure across multiple geographies and heterogeneous cloud providers.
Cross-Organizational Influence: Work with other groups in the organization to deeply understand their needs and translate them into concrete, production-ready capabilities and platform offerings.
Multi-Cloud Data Strategy: Define and champion a comprehensive, petabyte-scale multi-cloud data strategy, ensuring optimal data governance, security, and cost-efficient flow for streaming ingestion, data lakes, and advanced analytics workloads across environments.
Microservices Design & Resilience: Provide high-level architectural oversight for the multi-cloud design patterns of our microservices, focusing on cross-cloud resilience, global traffic management, data locality, and failure domain isolation.
AI Transformation Leadership: Lead the AI transformation initiatives aimed at radically enhancing the developer experience, introducing intelligent automation, and leveraging generative AI models to accelerate software delivery, testing, and operational efficiency.
Strategic Alignment: Foster cross-group alignment and build consensus among engineering leaders and business stakeholders for strategic platform projects, translating complex technical roadmap items into clear, measurable business outcomes.
Architecture Definition: Help define the technical vision, architecture, and long-term directions of the Cloud Platform group and the wider R&D organization.
Engineering Excellence: Be a champion for engineering excellence, setting high standards, mentoring engineers across the group to foster solid architectural principles, and supporting their professional growth.
Architect Forum Participation: Be a key member of the companys architect forum, participate in complex technical discussions, and facilitate effective, org-wide decision-making processes.

We're looking for a Data Engineer with 4+ year experience to join our Data Engineering team and help us build and scale our production-grade data platform. You'll work on high-performance systems built on self-hosted ClickHouse, optimize complex data pipelines, and collaborate closely with Product, Analytics, and Infrastructure teams to deliver reliable, fast, and scalable data solutions.

This is a hands-on technical role where you'll have a significant impact on how we ingest, model, store, and serve data that powers our analytics and AI-driven products.
Youll play a key role in shaping the direction of our data platform and have meaningful ownership over critical components of our architecture.

What You'll Do:
Data Modeling & Architecture
Design and evolve data models that reflect business logic and support analytical use cases
Collaborate with the BI and Analytics teams to understand data requirements and translate them into efficient schemas
Performance Optimization
Optimize ClickHouse schemas, partitioning strategies, indexing, and compression
Profile and tune slow queries to improve performance and reduce costs
Implement systems that ensure data quality, consistency, and operational efficiency (e.g., deduplication, validation, anomaly detection)
Monitor pipeline health, data freshness, and query performance with appropriate alerting mechanisms
SQL Compiler Development
Develop and maintain the SQL Compiler layer that translates high-level queries into optimized ClickHouse execution plansImplement query optimization and rewriting strategies to improve performanceDebug and resolve compiler issues to ensure accurate and efficient query translation

Data Pipeline Development & Collaboration
Review and advise the Integration team on pipeline architecture, performance, and best practices.
Provide guidance on data modeling, schema design, and optimization for new data sources.
Troubleshoot and maintain existing pipelines when issues arise or optimization is needed
Ensure data freshness, reliability, and quality across all ingestion pipelines.
Collaboration & Support
Work closely with the Integration team to ensure smooth data ingestion from new sources.
Partner with Infrastructure to support high availability and disaster recovery
Support other teams across the company in accessing and using data effectively.