דרושים » תוכנה » Senior Big Data Developer

We are looking for a Data Engineer Team Lead to join our Data Engineering team in Tel Aviv.

You will be leading one of our data platform teams, that is responsible for processing, storing, and serving data for all our core systems.

What Youll Do:

Lead and mentor a data lake team within our data organization.
Drive high-level projects with company-wide visibility.
Manage the team roadmap, team building, and SW project execution.
Collaborate with stakeholders, including developers, product managers, and business operations.
Take full ownership of end-to-end feature development, from design to production. Contribute to building and designing innovative big data solutions.
Research core technologies and integrate with external APIs and services.

Were looking for an exceptional data engineer to join our journey to create a data-based platform that will transform the shipping industry.
As a senior data engineer, you will take an instrumental part in laying the foundations and building the data infrastructure in the company, plan and implement the relevant data infrastructure and tools, be in charge of pipelining the companys data and be responsible for the integrity of the data as we scale up.
Responsibilities:
Contribute to the ongoing management of the data team platform including platform enhancements, capacity management, performance monitoring, troubleshooting and resolution of technical issues
Designing and optimizing data storage solutions, including data warehouses and data lakes.
Identify, design, and implement process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability.
Developing and maintaining data pipelines for efficient extraction, transformation, and loading (ETL) processes.
Participate in proofs-of-concept on platform innovation, and effectively transition and scale those concepts into production at scale through, engineering, deployment and commercialization.
Part of a cross-disciplinary team, working closely with other data engineers, software engineers, data scientists, data managers and business partners.

You will work in a team of data and software engineers building our Authentic Attention products.

You will process 10s of billions of records a day using mainly Python and Scala, utilizing advanced technologies like Spark, Kafka, Docker, Kubernetes, DataBricks and GCP managed solutions like Google Composer and many more.

You will build the next generation of digital advertising optimization tools used by the biggest advertisers in the world to help them understand and optimize their ROI. This is a true big data product at a huge scale and high impact for our clients and the digital advertising industry as a whole.

You will maintain a very high engineering standard that includes testing, automated deployment, infrastructure as code, high degree of monitoring and availability.

We are looking for a strong software engineer to join our Data and LLM domains, in charge of building and driving forward our LLMOps Platform, Feature and Vector Stores.
We deeply value fast learners and independent thinkers with a proactive mindset, who are able to work independently and as a part of a team.
As a Senior Software Engineer you will:
Collaborate closely with talented individuals to drive innovation within our core business.
Play a significant role in enhancing and expanding our Feature and Vector stores, as well as shaping our newly launched LLM management offering.
Work with the latest technologies and tools in the data space.
Gain professional development and advancement opportunities within our fast-growing organization.
Improve the efficiency, scalability, and stability of various components within our ecosystem.
Advocate for robust testing and code design principles.
Solve complex problems and own major features end-to-end.

We are looking for a Senior Data Engineer.
In your role as a Data Engineer, you'll be actively engaged in the most fascinating and demanding projects for our clients. Handling high-scale operations, heavy traffic, stringent SLAs, and massive data volumes is just the starting point. Our leadership in the realms of Big Data, Streaming Data, and Complex Event Processing (CEP) stems from our adept use of cutting-edge, optimal technologies.
Responsibilities:
Apply software-engineering methods and best practices (Medallion Architecture) in Data Lake management.
Develop Data pipelines to provide ingest and transformation of data.
Manage and enhance orchestrated pipelines across company
Data Modeling: Developing data models to ensure efficient storage, retrieval, and processing of data, often involving schema design for NoSQL databases, data lakes, and data warehouses.
Scalability: Ensuring that the big data architecture can scale horizontally to handle large volumes of data and high traffic loads.
Data Security: Implementing security measures to protect sensitive data, including encryption, access controls, and data masking.
Data Governance: Establishing data governance policies to ensure data quality, compliance with regulations, and data lineage.

We are looking for a brilliant, quick-learner Data Engineer for our data engineering team - an independent logical thinker who understands the importance of data structuring for macro-business decisions.
The position combines high technical skills with business orientation, working closely with the analysts and the R&D team, and affecting the company's cross-departments decisions directly. Our Data Engineer should have the ability to speak in technical and practical terms, and more importantly, lead from one to the other, while dealing with challenges and creating them - to make our team even better than it is.
Roles and Responsibilities:
Creating and structuring end-to-end data pipelines & ETLs: from the source all the way to the analysts hands, enabling them the ideal conditions to make smart and data driven business decisions.
Cracking top industry data challenges while initiating and building creative technical solutions - in house Device Graph, Server-to-Server to multiple systems, Privacy challenges, Online-to-Offline and more.
Deep understanding of the business needs, technical requirements and the companys roadmap - and translating it into custom made data solutions and scalable products.
Craft code following best practices to ensure efficiency, while integrating CI/CD principles.
Writing multi-step scalable processes from more than 50 data sources - Marketing. Operations, CS, Product, CRM and more... tying them up to a valuable & useful source of insights for the analysts.
Understanding data challenges and weaknesses, and managing high standards monitoring and reliability processes.

we are looking for a Performance Solution Architect to work with our globally distributed engineering team on internet-scale data lake analytics platform. This is our flagship product and offers incredible value-add on the already stellar performance of the open-source Trino software. This role provides you with the opportunity to directly contribute to the enhancement of performance. The role entails conducting performance analysis and comparing existing in-market solutions, formulating a comprehensive strategy for performance testing, analyzing the results, and offering appropriate recommendations for enhancing.

As a Data Performance Engineer
Be an expert in architecture and performance approaches
Partner closely with sales engineers, solutions, support, and engineering teams to deliver guidance on performance issues at any phase in the product life cycle.
Maintain a profound understanding of current market solutions and devise instrumentation and approaches for comparing them with the solution, paving the way for future enhancements to .
Collaborate with Product Management and Solution architects to continuously improve performance and eco-system roadmaps.

Were looking for a Senior Software Engineer to join our team. Our team is dealing with high scale big-data processing, complex logic at the core data pipeline of the product. As a Senior Software Engineer, you will be responsible for leading the design, architecture and implementation of new components in the system as well as supporting existing components. Our ideal candidate is someone who has hands-on development experience and leadership skills.

What youll do:

Design logic solutions to complex challenges for our customers who are among the most successful enterprises in the world
Write high-quality code, covered by solid unit and integration tests
Drive technological and architectural decisions
Review the design and code for other team members

The Platform group is responsible for some of the core features and components , like:

AQL(Query Language) - giving our customers the ability to search & analyze their assets in the organization using our query language and wizard. Supporting complex features - query-enhanced capabilities (like data refinement and filtering), assets historical investigation, query optimizations, and much more.

End-to-end ownership of core platform components: Devices management, Users management, Software management, and Vulnerability management. Includes owning the product roadmap of those domains from vision to production, stability and performance roadmap, and more. Owning these core components involves complex Frontend and deep tech backend projects like ETLs, distributed computing, and more.
Responsibilities:


You will be a part of the Axonius R&D team, implementing new and exciting features, end-to-end.

You will work with the best techies and professionals

Our stack includes Python, Vue.js, MongoDB, Redis, RabbitMQ and Docker

Required Experienced Software Engineer, Demand
What are some of the things you do on a day-to-day basis?
Develop one of the largest real time big data operation in the world to support over 40TB of new data every day
Have end to end ownership: Design, build, ship, measure and maintain our frontend and backend services
Collaborate with the brightest software engineers team members
Influence directly on the way billions of people discover the internet
Develop at unimaginable scale, serving hundreds of requests per second
Work on innovative projects that are the next growth engines for us
Here are some of the things software developers in our group did in the last several months:
A unique data pipeline for processing and managing user data signals
Develop a complete user identities graph and find deterministic and probabilistic methods to enrich it
An algorithmic model to predict the performance potential of a specific audience
Infrastructure to create unique audiences for advertisers based on various signals (behavioral, contextual, etc)
An A/B tests system to explore variations and experiments over audiences
Understand and use big data algorithms to estimate key business metrics over billions of impressions
Build user interface using React that serve thousands of users
And many more
Our Tech Stack:
Java, JS, TS, Python, React, Spark, Kafka, Hadoop, Cassandra, Vertica, ES, MySQL, Memcached, HDFS, BigQuery, Kusto, Docker, K8S, Linux, Prometheus, Grafana, Airflow.