we are looking for a Senior Data Engineer to join our Data Platform team. We are seeking a hands-on individual with experience in building and scaling modern data infrastructure, passionate about exploring AI technologies and applying them to enhance and evolve our data systems. This role is ideal for someone focused on data systems, performance, automation, and reliability, who will design, build, and maintain core components of our data pipelines, storage, monitoring, and observability stack, with a strong focus on improving workflows and system capabilities, while embracing the Shift4 way.
Responsibilities:
Build and maintain scalable, reliable, and high-performance data infrastructure across cloud and on-prem environments
Partner with engineering, analytics, and observability teams to support data flow from ingestion to consumption
Develop infrastructure as code and automation tools to improve deployment, monitoring, and recovery
Help troubleshoot performance bottlenecks and reliability issues in large-scale data pipelines
Implement data quality processes aligned with security standards to ensure accuracy, consistency, and compliance with financial regulations and PCI requirements.
Continuously improve the performance and efficiency of the data platform with proactive tuning and benchmarking
Proactively explore and apply AI tools to enhance efficiency across the data engineering lifecycle, while improving accessibility and usability of data solutions for teams across the organization
Requirements: 7+ years of experience in data engineering, infrastructure, or DevOps roles
Strong hands-on experience with message queue systems such as
Apache Kafka, Confluent and RabbitMQ
Practical experience with the Elastic Stack (Elasticsearch, Logstash, Kibana), including designing data pipelines, index management, performance and costs optimization.
Hands-on experience with time series databases such as Prometheus and InfluxDB
Experienced with visualization and observability platforms including Grafana, Prometheus, Elastic stack. Designed and maintained interactive dashboards for real-time monitoring of system metrics, logs, and traces, enabling proactive issue detection and improved operational efficiency
Strong knowledge of cloud platforms (AWS, GCP, or Azure) and operating production data infrastructure, ensuring scalability, reliability, and performance of data systems.
Infrastructure as code experience using Terraform or Ansible
Proven proficiency in CI/CD methodologies with the ability to design and implement automated pipelines from scratch
Responsible for organization-wide monitoring systems, with deep familiarity and understanding of the various infrastructure components, services, and devices across the organization
Background in observability and performance tuning at scale
Passion for automation, performance, and clean system design
This position is open to all candidates.