דרושים » תוכנה » Senior Big Data Engineer (Data Collection)

משרות על המפה
 
בדיקת קורות חיים
VIP
הפוך ללקוח VIP
רגע, משהו חסר!
נשאר לך להשלים רק עוד פרט אחד:
 
שירות זה פתוח ללקוחות VIP בלבד
AllJObs VIP
30/12/2024
משרה זו סומנה ע"י המעסיק כלא אקטואלית יותר
מיקום המשרה: תל אביב יפו
סוג משרה: משרה מלאה
משרות דומות שיכולות לעניין אותך
סגור
דיווח על תוכן לא הולם או מפלה
מה השם שלך?
תיאור
שליחה
סגור
v נשלח
תודה על שיתוף הפעולה
מודים לך שלקחת חלק בשיפור התוכן שלנו :)
 
נאספה מאתר אינטרנט
14/01/2025
חברה חסויה
Location: Tel Aviv-Yafo
Job Type: Full Time
Required Senior Big Data Engineer
Tel Aviv, Israel
Who we are:
An Israeli-founded big data analytics company. We track and analyze tens of billions of ads every day for the biggest brands in the world.
We operate at a massive scale, handling over 100B events per day and over 1M RPS at peak, we process events in real-time at low latencies (ms) and analyze over 2.5M video years every day. We verify that all ads are fraud free, appear next to appropriate content, appear to people in the right geography and measure the viewability and users engagement throughout the ads lifecycle.
We are global, with HQ in NYC and R&D centers in Tel Aviv, New York, Finland, Berlin, Belgium and San Diego. We work in a fast-paced environment and have a lot of challenges to solve. If you like to work in a huge scale environment and want to help us build products that have a huge impact on the industry, and the web - then your place is with us.
What you will do:
You will work in a team of data and software engineers building our Authentic Attention products.
You will process 10s of billions of records a day using mainly Python and Scala, utilizing advanced technologies like Spark, Kafka, Docker, Kubernetes, DataBricks and GCP managed solutions like Google Composer and many more.
You will build the next generation of digital advertising optimization tools used by the biggest advertisers in the world to help them understand and optimize their ROI. This is a true big data product at a huge scale and high impact for our clients and the digital advertising industry as a whole.
You will maintain a very high engineering standard that includes testing, automated deployment, infrastructure as code, high degree of monitoring and availability.
Requirements:
At least 4 years in Software Development
At least 3 years experience coding in Python\Java\C#\Scala and a like
At least 3 years experience working with relational databases
At least 2 years of experience building ETLs or ELTs
Good analytical skills and well versed with SQL
Good interpersonal and communication skills
Passion for development with high quality and a sharp mind
Experience maintaining business-critical production systems
Nice to have:
Experience with Python
Experience with CI/CD pipeline, Docker & Kubernetes, public cloud providers
Experience with Spark
Experience with non-relational DB
Experience with streaming processing
Experience with online advertising technologies.
This position is open to all candidates.
 
Show more...
הגשת מועמדותהגש מועמדות
עדכון קורות החיים לפני שליחה
עדכון קורות החיים לפני שליחה
8018211
סגור
שירות זה פתוח ללקוחות VIP בלבד
סגור
דיווח על תוכן לא הולם או מפלה
מה השם שלך?
תיאור
שליחה
סגור
v נשלח
תודה על שיתוף הפעולה
מודים לך שלקחת חלק בשיפור התוכן שלנו :)
 
נאספה מאתר אינטרנט
22/01/2025
Location: Tel Aviv-Yafo
Job Type: Full Time and Hybrid work
We're looking for a Senior Big Data Developer to join our Data Engineering team in Tel Aviv.

You will be a part of our Data Management platform, that is responsible for processing, storing, and serving data for all our core systems.

As a Senior Data Engineer, we will encourage you to take a proactive & autonomous approach in research & development while working collaboratively with the team. Our ideal candidate is a fast but thorough problem solver, and a fun person to work with!

What Youll Do:

End-to-end feature development and ownership, from design to production.
Take part in building & designing the next generation of company Identity
solutions.
Implement high scale Big-Data solutions and contribute to our platform
infrastructure and architecture.
Lead and mentor other developers through complex projects that will affect our core business.
Research core technologies and integrations with external APIs and services.
Working with various stakeholders: Product, Engineering, Data providers etc.
Requirements:
Bachelor's Degree in Computer Science/Computer Engineering or equivalent degrees or equivalent experience.
6+ years of software development experience.
Experience in building and optimizing data pipelines, ETLs, and streaming systems using technologies like Hadoop, Spark, and Kafka.
3+ years of experience in Java or Scala.
Experience with SQL & NOSQL DBs.
Experience in dealing with performance and high scale systems.
Experience with Kubernetes - advantage.
Experience with Gitlab, Grafana, InfluxDb, Kibana advantage.
Team player, ego less, strong communication skills.
This position is open to all candidates.
 
Show more...
הגשת מועמדותהגש מועמדות
עדכון קורות החיים לפני שליחה
עדכון קורות החיים לפני שליחה
8030453
סגור
שירות זה פתוח ללקוחות VIP בלבד
סגור
דיווח על תוכן לא הולם או מפלה
מה השם שלך?
תיאור
שליחה
סגור
v נשלח
תודה על שיתוף הפעולה
מודים לך שלקחת חלק בשיפור התוכן שלנו :)
 
נאספה מאתר אינטרנט
לפני 11 שעות
Location: Tel Aviv-Yafo
Job Type: Full Time and Hybrid work
BigPanda is a fast-growing, values-driven, global company that enables Tech Ops teams to keep the digital economy running. our AI-driven IT operations (aka AIOps) platform transforms IT data into insight and action. By eliminating IT noise, automating incident management, and keeping our customers digital services up and running around the clock, we become a mission-critical part of our customers IT operations.
With us, some of the worlds largest enterprises including Hulu, Cisco, United, Abbott, Marriott, Expedia and many others are able to reduce costs and increase efficiencies, accelerate business velocity, and deliver extraordinary customer experiences.
we backed by top-tier investors including Sequoia, Mayfield, Battery, Insight Partners, Advent International, and Greenfield Partners.
We have an awesome team of motivated, knowledgeable, fun-loving, and friendly Pandas. We provide comprehensive health coverage, parental leave, competitive cash and equity compensation, and a supportive, collaborative, and innovative environment to empower you to do the best work of your career.
Our Benefits:
Competitive equity
Hybrid work schedule
Company funded health insurance
6 weeks fully paid Parental Leave
Critical Family Medical Leave
Financial planning services
Employee learning & development budget
Values-based recognition (quarterly and annually)
Social community & ERG programs
FreeFit gym package
Work-life harmony
Dog friendly office
Location requirements:
Youll be required to work at the office 2 days a week.
About the Role:
The Incident Intelligence group drives our core event processing, enrichment, and correlation capabilities, handling millions of daily events across a robust, high-throughput cloud infrastructure.
Our team leverages AI technologies alongside other data processing methods to enrich incidents, delivering deeper insights and smarter responses to complex events.
Were looking for a talented Senior Backend Engineer who will contribute to the team's technical direction, promote best practices and innovation. You will tackle challenging problems, advanced debugging, and diagnosing complex systems. You will utilize Scala, Java, Kafka, and more.
What you will do:
Design and develop highly scalable services and participate in other life-cycle activities such as analysis, testing, performance, and production releases
Utilize state-of-the-art technology to solve complex software system problems
Enhance development productivity by identifying the areas for modification and innovation
Maintain a high standard of coding and introduce engineering best practices
Requirements:
What skills and experience youll bring:
7+ years of experience in backend software engineering
4+ years of Java/Scala programming experience
Experience with stream processing tools and methodologies such as or equivalent to Kafka
Experience in designing and building large-scale distributed cloud-based system
Proven experience in delivering key features from ideation to production
Ability to deep dive into complex systems and troubleshoot challenging issues effectively
A proactive approach to identifying and addressing problems before they become critical
Strong skills in communicating complex ideas and clearly explaining technical concepts and solutions
This position is open to all candidates.
 
Show more...
הגשת מועמדותהגש מועמדות
עדכון קורות החיים לפני שליחה
עדכון קורות החיים לפני שליחה
8065522
סגור
שירות זה פתוח ללקוחות VIP בלבד
סגור
דיווח על תוכן לא הולם או מפלה
מה השם שלך?
תיאור
שליחה
סגור
v נשלח
תודה על שיתוף הפעולה
מודים לך שלקחת חלק בשיפור התוכן שלנו :)
 
נאספה מאתר אינטרנט
22/01/2025
Location: Tel Aviv-Yafo
Job Type: Full Time and Hybrid work
We're looking for a Senior Big Data Developer to join our Data Engineering team in Tel Aviv.

You will be a part of our Data Management platform, that is responsible for processing, storing, and serving data for all our core systems.

As a Senior Big Data Developer, we will encourage you to take a proactive & autonomous approach in research & development while working collaboratively with the team. Our ideal candidate is a fast but thorough problem solver, and a fun person to work with!

What Youll Do:

End-to-end development and ownership, from design to production.
You will lead and mentor other developers through complex projects that will affect our core business.
You will implement high scale Big-Data solutions and contribute to our platform infrastructure and architecture.
Research core technologies and integrations with external APIs and services.
Working with various stakeholders: Product, Engineering, Data providers etc.
Requirements:
Bachelor's degree in computer science/Computer Engineering or equivalent degrees.
6+ years of data engineering experience.
Experience in building and optimizing data pipelines, ETLs, and streaming systems using technologies like Hadoop, Spark, and Kafka.
Experience with SQL & NOSQL DBs.
Experience in dealing with performance and high scale systems.
Experience in Clojure big advantage.
Experience with Kubernetes advantage.
Experience with Gitlab, Grafana, InfluxDb, Kibana advantage.
Versatile team player, ego less, strong communication skills.
This position is open to all candidates.
 
Show more...
הגשת מועמדותהגש מועמדות
עדכון קורות החיים לפני שליחה
עדכון קורות החיים לפני שליחה
8030448
סגור
שירות זה פתוח ללקוחות VIP בלבד
סגור
דיווח על תוכן לא הולם או מפלה
מה השם שלך?
תיאור
שליחה
סגור
v נשלח
תודה על שיתוף הפעולה
מודים לך שלקחת חלק בשיפור התוכן שלנו :)
 
נאספה מאתר אינטרנט
29/01/2025
חברה חסויה
Location: Tel Aviv-Yafo
Job Type: Full Time
We are on the lookout for an outstanding Senior Big Data Developer with strong OOP capabilities, a deep understanding of distributed systems and the ability to deliver in a technically diverse and fast-paced environment.
As part of our team, youll be responsible for our customers data including ingestion, preparation and serving.
To handle around 1 Billion events per day, you will use the latest and strongest frameworks such as Spark, Kafka, Airflow, Flink, Cassandra, Redis, Snowflake, DBT and Elasticsearch, running on k8s.
Role:
- Design, code, and maintain Big Data solutions - both batch and stream processing
- Be fully responsible for the products lifecycle - from design and development to deployment
- Bring a strong opinion to the table and be proactively involved with product planning
- Work in teams and collaborate with others
- Improve application performance
- Troubleshoot and resolve data issues.
Requirements:
- At least 5 years of software development experience
- Experience with Big Data/NoSQL/Stream processing technologies (e.g Spark, Kafka, Flink, Redis)
- Proven experience in leading and delivering complex software projects
- Excellent knowledge of an OO language
- Working experience with Docker and Kubernetes
- Experienced in AWS cloud technologies and administration
- Being a Team player, and fun to work with!
- A passion for clean, robust code and performance tuning.
This position is open to all candidates.
 
Show more...
הגשת מועמדותהגש מועמדות
עדכון קורות החיים לפני שליחה
עדכון קורות החיים לפני שליחה
8040096
סגור
שירות זה פתוח ללקוחות VIP בלבד
סגור
דיווח על תוכן לא הולם או מפלה
מה השם שלך?
תיאור
שליחה
סגור
v נשלח
תודה על שיתוף הפעולה
מודים לך שלקחת חלק בשיפור התוכן שלנו :)
 
נאספה מאתר אינטרנט
14/01/2025
חברה חסויה
Location: Tel Aviv-Yafo
Job Type: Full Time
Required Sr. Software Engineer II
Tel Aviv, Israel
Who we are:
An Israeli-founded big data analytics company. We track and analyze tens of billions of ads every day for the biggest brands in the world.
We operate at a massive scale, handling over 100B events per day and over 1M RPS at peak, we process events in real-time at low latencies (ms) and analyze over 2.5M video years every day. We verify that all ads are fraud free, appear next to appropriate content, appear to people in the right geography and measure the viewability and users engagement throughout the ads lifecycle.
We are global, with HQ in NYC and R&D centers in Tel Aviv, New York, Finland, Berlin, Belgium and San Diego. We work in a fast-paced environment and have a lot of challenges to solve. If you like to work in a huge scale environment and want to help us build products that have a huge impact on the industry, and the web - then your place is with us.
What will you do:
You will join a team of experienced engineers and help them in developing our innovative classification products.
You will lead projects by architecting, designing and implementing solutions that will impact the core components of our system.
Youll develop new and awesome features while leveraging cloud native technology stack, do continuous improvements of our development process by adapting new technologies, and using them to solve product and engineering challenges while raising the bar of code quality and standards.
Requirements:
5+ years of experience coding in an industry-standard language such as Scala, Java, Rust, Closure, Kotlin, Go etc.
Deep understanding of Computer Science fundamentals: object-oriented design, functional programming, data structures, multi-threading and distributed systems.
Experience with in-memory distributed cache such as Aerospike or Redis and messaging systems such as Apache Kafka, etc.
Experience working with Docker, Kubernetes and designing scalable microservices architecture.
Experience in working with SQL (MySQL, PostgreSQL) and Columnar/NoSQL Databases such as (BigQuery, Vertica, Snowflake, Couchbase, Cassandra, etc.).
Experience working in a BigData environment and building scalable distributed systems with stream processing technologies such as Akka Streams, Kafka Streams/Spark/Flink.
Experience working with cloud providers such as GCP or AWS
BSc in Computer Science or equivalent experience.
Experience with Agile development, CI/CD pipelines (Git ,GitLab or Jenkins) and coding for automated testing.
A versatile developer with a getting-things-done attitude.
Nice to have:
Previous experience with online advertising technologies is a big plus.
Familiarity with the cloud-native computing foundation.
This position is open to all candidates.
 
Show more...
הגשת מועמדותהגש מועמדות
עדכון קורות החיים לפני שליחה
עדכון קורות החיים לפני שליחה
8018250
סגור
שירות זה פתוח ללקוחות VIP בלבד
סגור
דיווח על תוכן לא הולם או מפלה
מה השם שלך?
תיאור
שליחה
סגור
v נשלח
תודה על שיתוף הפעולה
מודים לך שלקחת חלק בשיפור התוכן שלנו :)
 
נאספה מאתר אינטרנט
03/02/2025
Location: Tel Aviv-Yafo and Netanya
Job Type: Full Time
We are seeking a highly skilled Senior Data Engineer to join ML Data Group and help drive the development and optimization of our cutting-edge data infrastructure. As a key member of the Platform team, you will play an instrumental role in building and evolving our feature store data pipeline, enabling machine learning teams to efficiently access and work with high-quality, real-time data at scale.
In this dynamic, fast-paced environment, you will collaborate with other data professionals to create robust, scalable data solutions. You will be responsible for architecting, designing, and implementing data pipelines that ensure reliable data ingestion, transformation, and storage, ultimately supporting the production of high-performance ML models.
We are looking for data-driven problem-solvers who thrive in ambiguous, fast-moving environments and are passionate about building data systems that empower teams to innovate and scale. We value independent thinkers with a strong sense of ownership, who can take challenges from concept to production while continuously improving our data infrastructure.
As a Data Engineer you will...
Design and implement large-scale batch & streaming data pipelines infrastructure
Build and optimize data workflows for maximum reliability and performance
Develop solutions for real-time data processing and analytics
Implement data consistency checks and quality assurance processes
Design and maintain state management systems for distributed data processing
Take a crucial role in building the group's engineering culture, tools, and methodologies
Define abstractions, methodologies, and coding standards for the entire Data Engineering pipeline.
Requirements:
5+ years of experience as a Software Engineer with focus on data engineering
Expert knowledge in building and maintaining data pipelines at scale
Strong experience with stream/batch processing frameworks (e.g. Apache Spark, Flink)
Profound understanding of message brokers (e.g. Kafka, RabbitMQ)
Experience with data warehousing and lake technologies
Strong Python programming skills and experience building data engineering tools
Experience with designing and maintaining Python SDKs
Proficiency in Java for data processing applications
Understanding of data modeling and optimization techniques
Bonus Points
Experience with ML model deployment and maintenance in production
Knowledge of data governance and compliance requirements
Experience with real-time analytics and processing
Understanding of distributed systems and cloud architectures
Experience with data visualization and lineage tools/frameworks and techniques.
This position is open to all candidates.
 
Show more...
הגשת מועמדותהגש מועמדות
עדכון קורות החיים לפני שליחה
עדכון קורות החיים לפני שליחה
8046082
סגור
שירות זה פתוח ללקוחות VIP בלבד
סגור
דיווח על תוכן לא הולם או מפלה
מה השם שלך?
תיאור
שליחה
סגור
v נשלח
תודה על שיתוף הפעולה
מודים לך שלקחת חלק בשיפור התוכן שלנו :)
 
נאספה מאתר אינטרנט
14/01/2025
Location: Tel Aviv-Yafo
Job Type: Full Time
As a Sr. Backend Engineer, you'll dive into complex technical challenges, craft innovative solutions, and play a pivotal role in our mission to revolutionize the way companies understand their customer interactions. We're seeking a Java professional who can leverage their deep knowledge to drive our backend development to new heights.

You will be responsible for:

System Design and Development: Lead the design and implementation of backend services and APIs using Java technologies, ensuring scalability, reliability, and performance.
Problem-Solving: Analyze and solve complex technical challenges across the stack, from architecture to implementation.
Collaboration: Work closely with Product Managers, UX designers, and Frontend Engineers to deliver features that meet user needs and business goals.
Ownership: Take ownership of features throughout their lifecycle, from initial concept to deployment and maintenance.
Requirements:
7+ years of hands-on experience in large-scale backend development, with a strong emphasis on Java programming (a must).
Strong analytical and problem-solving skills, with the ability to debug and resolve complex technical issues efficiently.
Experience with cloud platforms (AWS, Azure, or Google Cloud) and building scalable microservices architectures.
Advantageous experience with infrastructure as code, CI/CD pipelines, and building data-oriented applications and APIs.
Excellent communication skills, both verbal and written, with the ability to articulate technical ideas clearly and collaborate effectively within a team environment.
A Bachelors degree in Computer Science, Engineering, or a related field is preferred.
This position is open to all candidates.
 
Show more...
הגשת מועמדותהגש מועמדות
עדכון קורות החיים לפני שליחה
עדכון קורות החיים לפני שליחה
8018357
סגור
שירות זה פתוח ללקוחות VIP בלבד
סגור
דיווח על תוכן לא הולם או מפלה
מה השם שלך?
תיאור
שליחה
סגור
v נשלח
תודה על שיתוף הפעולה
מודים לך שלקחת חלק בשיפור התוכן שלנו :)
 
נאספה מאתר אינטרנט
05/02/2025
Location: Tel Aviv-Yafo
Job Type: Full Time
The worlds largest content recommendation platform, is hiring a Senior Software Engineer for the Infrastructure Engineering R&D group.
What are some of the things you do on a day-to-day basis?
Develop one of the largest real time high scale operation in the world to support over 1.4B users, >500,000 requests / second, 140TB of new data every day
Have end to end ownership: Design, develop, deploy, measure and maintain our services
Solve production issues in the framework code
Collaborate with the brightest software engineers team members
Influence directly on the way billions of people discover the internet
Here are some of the things software devs in our group did in the last several months:
Rewrote part of our recommendation servers to reduce serving time p999 by >40%
Rolled out custom Cassandra compaction strategy
Created and improved our Kafka consumer designed for very high scale
Developed a brand new metrics pipeline, ingesting over 100B metric messages per day
Anomaly detection engine for application metrics
Rolled out a new custom home-built authentication service
Many more things
Our Tech Stack:
Java, Spark, Kafka, Hadoop, Cassandra, Vertica, MySQL, HDFS, BigQuery, Docker, Linux, Kubernetes, Prometheus, Metrictank, Grafana, Airflow, Redis
This is us!
Requirements:
Required:
Experience developing large scale distributed systems
Deep understanding of strong Computer Science fundamentals: object-oriented design, data structures systems, applications programming and multi threading programming
5+ years programming experience in Java or equivalent Object-Oriented language
Production systems understanding
Experience with SQL and No-SQL
Highly Proficient in Linux environment
Proven ability to work effectively across multiple teams
BS.c in computer science or equivalent experience
Preferred:
Experience in Big Data tools
Knowledge in algorithms, data-mining and machine learning.
This position is open to all candidates.
 
Show more...
הגשת מועמדותהגש מועמדות
עדכון קורות החיים לפני שליחה
עדכון קורות החיים לפני שליחה
8049021
סגור
שירות זה פתוח ללקוחות VIP בלבד
סגור
דיווח על תוכן לא הולם או מפלה
מה השם שלך?
תיאור
שליחה
סגור
v נשלח
תודה על שיתוף הפעולה
מודים לך שלקחת חלק בשיפור התוכן שלנו :)
 
פורסם ע"י המעסיק
1 ימים
חברה חסויה
Location: Tel Aviv-Yafo
Job Type: Full Time
We are looking for a passionate and experienced Big Data Team Lead to drive the development of scalable, high-performance distributed systems. If you thrive in a fast-paced, Agile environment and are eager to work with cutting-edge technologies, we want to hear from you! This position is also open to Senior Big Data Engineers with strong leadership potential who are looking to take their first step into a managerial role.

What will you do:
Architect & Develop: Design, build, and maintain robust distributed systems that meet complex scalability and performance requirements. Optimize & Innovate: Implement scalable server architectures using parallel processing, microservices, and security-first development principles Database Management: Utilize SQL, NoSQL, Kafka/Pulsar, ELK, Redis, and column-store databases in system design and optimization. Big Data Technologies: Work with Spark or Flink to enhance system performance (experience is a plus). Cloud & Deployment: Deploy and manage applications with Kubernetes (K8S) and leverage GTP tools (familiarity is a plus). Team Leadership: Lead and mentor a global team of five engineers , fostering collaboration and professional growth. Cross-Team Collaboration: Work closely with cross-functional teams to ensure seamless software integration and performance.
Requirements:
5+ years in software development, with at least 2 years as a Senior Big Data Engineer
* Willingness to work with GoLang. Experience with Scala – advantage. Technical Expertise: Strong understanding of the software development lifecycle (SDLC), Agile methodologies, DevOps, CI/CD, and test automation
* Proficiency in SQL, NoSQL, Kafka/Pulsar, ELK, Redis, and column-store databases
* Experience with Spark or Flink (advantageous).
* Proven experience with Kubernetes (K8S) ; familiarity with GTP tools is a plus. Leadership & Collaboration: Ability to lead, mentor, and inspire a global team.
* Strong interpersonal skills and a team-oriented mindset Open to senior engineers who are looking for their first leadership role.? Passion for exploring and implementing new technologies independently.
* Comfortable in a fast-paced, startup-like environment Education: Bachelor’s or Master’s degree in Computer Science or equivalent experience
This position is open to all candidates.
 
Show more...
הגשת מועמדותהגש מועמדות
עדכון קורות החיים לפני שליחה
עדכון קורות החיים לפני שליחה
8063276
סגור
שירות זה פתוח ללקוחות VIP בלבד