דרושים » ניהול ביניים » data engineer

We are seeking a Senior Backend & Data Engineer to join its SaaS Data Platform team.
This role offers a unique opportunity to design and build large-scale, high-performance data platforms and backend services that power our cloud-based products.
You will own features end to end-from architecture and design through development and production deployment-while working closely with Data Science, Machine Learning, DevOps, and Product teams.
What Youll Do:
Design, develop, and maintain scalable, secure data platforms and backend services on AWS.
Build batch and streaming ETL/ELT pipelines using Spark, Glue, Athena, Iceberg, Lambda, and EKS.
Develop backend components and data-processing workflows in a cloud-native environment.
Optimize performance, reliability, and observability of data pipelines and backend services.
Collaborate with ML, backend, DevOps, and product teams to deliver data-powered solutions.
Drive best practices, code quality, and technical excellence within the team.
Ensure security, compliance, and auditability using AWS best practices (IAM, encryption, auditing).
Tech Stack:
AWS Services: S3, Lambda, Glue, Step Functions, Kinesis, Athena, EMR, Airflow, Iceberg, EKS, SNS/SQS, EventBridge
Languages: Python (Node.js/TypeScript a plus)
Data & Processing: batch & streaming pipelines, distributed computing, serverless architectures, big data workflows
Tooling: CI/CD, GitHub, IaC (Terraform/CDK/SAM), containerized environments, Kubernetes
Observability: CloudWatch, Splunk, Grafana, Datadog
Key Responsibilities:
Design, develop, and maintain scalable, secure backend services and data platforms on AWS
Build and operate batch and streaming ETL/ELT pipelines using Spark, Glue, Athena, Iceberg, Lambda, and EKS
Develop backend components and data processing workflows in a cloud-native environment
Optimize performance, reliability, and observability of data pipelines and backend services
Collaborate with ML, backend, DevOps, and product teams to deliver data-driven solutions
Lead best practices in code quality, architecture, and technical excellence
Ensure security, compliance, and auditability using AWS best practices (IAM, encryption, auditing)

Required senior software engineer - data and ml engineering group
company description
about us
A palo alto networks company, is the global leader in identity security, trusted by organizations around the world to secure human and machine identities in the modern enterprise. Our ai-powered identity security platform applies intelligent privilege controls to every identity with continuous threat prevention, detection and response across the identity lifecycle. with identity security, organizations can reduce operational and security risks by enabling zero trust and least privilege with complete visibility, empowering all users and identities, including workforce, it, developers and machines, to securely access any resource, located anywhere, from everywhere.
job description
w are looking for an experienced and highly motivated senior software engineer with high professional skills to join our ai and data solutions unit. spearhead the development of data and ml frameworks; responsible for our data and ml ci/cd and governance processes.
in this role you will be using aws serverless architecture, aws cdk, and Python to design, develop, TEST, secure & deploy services from planning to production. the team works closely with data scientists, data analytics, data engineers, security architects, product managers and other development teams. together we produce complex and advanced data /ml platforms that meet the highest security standards and best practices to data experts.
we are looking forward to an energetic and versatile person to join our family and help us achieve our goals.

We are looking for an experienced, opinionated, and highly technical Staff Data Engineer to define the long-term direction of our Data Platform.
This role is ideal for a senior data leader who combines deep hands-on expertise with strong architectural judgment, enjoys mentoring others, and takes full ownership of complex, high-scale data systems.
You will drive platform architecture, set engineering standards, and lead the design and delivery of scalable batch and streaming data solutions on AWS.
Key Responsibilities:
Technical Leadership & Architecture:
Own and evolve the overall data platform architecture-scalability, reliability, security, and maintainability.
Lead the long-term strategic direction of the platform, balancing performance, cost, and operational excellence.
Introduce and drive adoption of modern data paradigms: lakehouse (Iceberg), event-driven pipelines, schema-aware processing.
Data Lake & Architecture Ownership:
Design, model, and evolve the Data Lake architecture, including:
Storage layout and data organization
Data formats and table design (e.g., Iceberg)
Batch and streaming ingestion patterns
Schema governance and lifecycle policies
Define and promote best practices for data modeling, partitioning, and data quality.
Ensure the Data Lake supports analytics, ML workloads, and operational systems at scale.
Platform Engineering:
Design and build high-scale ETL/ELT pipelines leveraging Apache Spark (EMR/Glue) and AWS-native services.
Optimize production-grade pipelines using S3, Athena, Kinesis, Lambda, Step Functions, and EKS.
Lead the rollout of modern patterns such as lakehouse architectures and event-driven data pipelines.
Security & Governance:
Ensure alignment with AWS security best practices-IAM, encryption, auditing, and compliance frameworks.
Partner with security and governance teams to support regulated and sensitive data environments.
Mentorship & Collaboration:
Serve as a technical mentor to data engineers; elevate team capabilities.
Lead architecture reviews and cross-team design discussions.
Work closely with Data Science, ML Engineering, Backend, and Product teams to deliver end‑to‑end data solutions.