לוח משרות דרושים מהנדס/ת דאטה/DATA ENGINEER משרה מלאה

We are seeking a skilled and motivated Data Engineer with expertise in Elasticsearch, cloud technologies, and Kafka. As a data engineer, you will be responsible for designing, building and maintaining scalable and efficient data pipelines that will support our organization's data processing needs.

The role will entail:
Design and develop data platforms based on Elasticsearch, Databricks, and Kafka.
Build and maintain data pipelines that are efficient, reliable and scalable.
Collaborate with cross-functional teams to identify data requirements and design solutions that meet those requirements.
Write efficient and optimized code that can handle large volumes of data
Implement data quality checks to ensure accuracy and completeness of the data.
Troubleshoot and resolve data pipeline issues in a timely manner.

Required Senior Data Engineer
In your role as a Data Engineer, you'll be actively engaged in the most fascinating and demanding projects for our clients. Handling high-scale operations, heavy traffic, stringent SLAs, and massive data volumes is just the starting point. Our leadership in the realms of Big Data, Streaming Data, and Complex Event Processing (CEP) stems from our adept use of cutting-edge, optimal technologies.
Responsibilities:
Apply software-engineering methods and best practices (Medallion Architecture) in Data Lake management, using tools like DBT.
Develop Data pipelines to ingest and transform data.
Manage and enhance orchestrated pipelines across company
Data Modeling: Developing data models to ensure efficient storage, retrieval, and processing of data, often involving schema design for NoSQL databases, data lakes, and data warehouses. This will also include catalogs of data.
Scalability: Ensuring that the big data architecture can scale horizontally to handle large volumes of data and high traffic loads.
Data Security: Implementing security measures to protect sensitive data, including encryption, access controls, and data masking.
Data Governance: Establishing data governance policies to ensure data quality, compliance with regulations, and data lineage.

We're looking for an experienced Backend Developer to join our growing data platform team.
As a Backend Developer, you'll work on a massive data processing pipeline, ingesting over a billion daily events from multiple sources. You'll also create the next-generation pipeline and help us scale from a billion events a day to tens of billions of events.
Responsibilities:
Own projects from initial discussions to release, including data exploration, architecture design, benchmarking new technologies, and product feedback.
Work with massive amounts of data from different sources using state-of-the-art technology to make big data accessible in real-time.
Develop and deploy real-time and batch data processing infrastructures.
Manage the development of distributed data pipelines and complex software designs to support high data rates (millions of daily active users) using cloud-based tools.
Work closely with company stakeholders on data-related issues.
Develop unit, integration, end-to-end (e2e), and load tests.

Required Senior Data Engineer
About the Role:
Our Senior Data Engineer will play an essential role by building the underlying infrastructures, collecting, storing, processing and analyzing large sets of data, while collaborating with researchers, architects, and engineers, in order to design and build high-quality data processing for our flows.
In this role, you are responsible for end-to-end development of the data pipeline and data models, working with major data flow that includes structured and unstructured data. You will also hold responsibility for operating parts of our production system. Your focus will be on developing and integrating systems that retrieve and analyzing data that influence people's lives. This role for our Tel Aviv office is a hybrid role working at least two days per week in the office.
Our Technologies stack: Python, Spark, Airflow, DBT, Kafka, AWS, Docker, Kubernetes, MongoDB, Redis, Postgres, Elasticsearch, and more.
The ideal candidate will be:
A technology enthusiast who loves data and get shiver excitement from tech innovations
Desire to know how things work and a greater desire to improve them
Intellectual curiosity to find unusual ways to solve problems
Comfortable taking on challenges and learning new technologies
Comfortable working in a fast-paced dynamic environment.

we are looking for a hands-on Data Engineer Team Lead with a strong technical background in data infrastructure to join our Data Engineering & Data Science Group.
The Data Engineering & Data Science Group is a rapidly expanding team of skilled engineers. By joining them, youll have the opportunity to make a meaningful impact in shaping our data platform and contributing to the groups strategic roadmap.
In this role, youll be responsible for the professional and personal development of your team. Youll lead the technological strategy, build the tech agenda, work closely with the product, data-analysts and data-scientists, and drive your team to be accountable owners of their work as they meet company business goals.
This role is full-time and is based in Herzliya, Israel.
Responsibilities
Build, lead, coach, and mentor the data engineering team
Dedicate 70% of your work to hands-on development
Manage the agile delivery life-cycle and the teams CI/CD pipelines
Ensure coding standards and implementation of best practices by conducting design and code reviews
Lead an innovative roadmap considering cutting edge technologies and open source frameworks
Collaborate with engineers, data architects, data analysts, and data scientists to develop solutions using a use case-driven approach, guiding the design and implementation of technical data flows
Build and expand our data infrastructure, including our data lake, analytics engine, orchestration tools and batch processing frameworks
Enhance and scale our ML platform, and automate the machine learning lifecycle.

Were building the future workforce of Technology organization, while working tightly with domain owners and top management to define what per-role excellence means, leveraging data from development tools to help infer proficiency levels, providing tailored development plans and training & learning material, and harnessing the individuals themselves for their professional and personal development.
Responsibilities:
Design, implement, and maintain the complete data flow, from extraction to visualization using external tools Work within a product team where solutions are collaboratively proposed. You will be expected to translate requirements into technical designs and implement them as part of large-scale data engineering solutions Utilize various Machine Learning models to explore data, collaborating with cutting-edge Big Data tools and techniques Collaborate with various stakeholders to plan, design, develop, TEST and maintain extraordinary features Collaborating with a team of skilled developers to develop high-traffic, cloud-based applications.

We're seeking a seasoned Data Engineer to build the data infrastructure that fuels our groundbreaking intelligent agent. You'll play a crucial role in developing large-scale data-intensive systems that power Apollo's capabilities.
What You'll Do:
- Design and implement massive parallel processing solutions for both real-time and batch scenarios
- Develop real-time stream processing solutions using technologies like Apache Kafka or Amazon Kinesis
- Build infrastructures that bring machine learning capabilities to production
- Orchestrate containerized applications in cloud environments (AWS and GCP)
- Write production-grade Python code and work with various database systems
- Administer and design cloud-based data warehousing solutions
- Work with unstructured data, complex data sets, and perform data modeling
- Collaborate with cross-functional teams to integrate data solutions into our AI systems.

Were looking for an experienced Data Engineer to join our amazing Data department and to help us to take our data platform and data usage to the next level. Our technology stack is based on Googles Cloud Platform and its different components.

What youll be spending your time on: 

Develop end-to-end data models: from gathering data sources to creating ETL processes that enable smarter, data-driven decisions.
Build automated validation processes to ensure data integrity, system stability, and data observability.
Optimize process run-times and provide scalable solutions to complex problems.
Take full ownership of designing, building, deploying, and maintaining data products.

We're looking for an experienced Data Engineer to work on our massive data processing pipeline and our new NextGen BI system to help us deliver more insights and scale from our current billion events a day to tens of billions of events.
Responsibilities:
Pull data from various data sources and integrate into the ETL process
Integrate Machine Learning models in ETL to deliver insights to Marketing, Product and CRM departments
Optimize workloads to reduce cost and improve performance
Work closely with analysts from Product, Marketing and Data Scientists to deliver insights from over a Billion daily events and over 50 Marketing channels.