דרושים » הנדסה » Data Engineer

Lead the design, implementation, and ongoing management of our company's data pipeline architecture.
Lead team architectural discussions
Manage a team of 2-3 data engineers
Work as an integral part of the data team and work closely with data Scientists, data Analysts and data Engineers.
Work directly with others (Developers, Product, and Internal stakeholders) to develop a solid understanding of the business application and requirements for which the application is being designed.

Design, build, and launch data infrastructures that support multiple use cases across different products or domains.
Collaborate with engineers, product managers, data scientists, and other stakeholders to understand data needs.
Solve challenging data integration problems, utilizing optimal ETL patterns, frameworks, query techniques, and sourcing from structured and unstructured data sources.
Lead end-to-end data projects from infrastructure design to production monitoring and visualization.

Who We're Looking For - The Dream Maker
We are looking for a hands-on data engineer who will build and develop the entire data engineering capabilities and infrastructure. This role will possess a combination of technical expertise, problem-solving skills, and strong communication abilities.
Your Arena
*  data Standardisation:
*  Make sure that data is standardized, and manage efforts on that initiative (including past data ).
* *Create a live documentation for the existing data schema (main entities + each field)
* Make sure that data schema is consistent and participate in design reviews that involve database design
*  Oversee the adoption of a common data access layer for our main DB entities (disputes, customer data, integrations, billing)
*  Add and maintain a validation processes ayer to prevent divergence from schema and closed set of value* data Correctness & Integrity
* Set up processes to ensure data correctness, validity based on business requirements
*  Set up processes to detect data duplication in our database, and take actions to remediate this with the business stakeholders* data Optimization and Operations
* Monitor and Operate DB Performance, index management, Alerts, scaling on a daily basis* data Engineering technologies and roadmap
*  Analyze and choose the right database and related technologies for our evolving needs including data warehouses, data lakes, graph database, relational and non-relational databases and ETL tools.
* Be a knowledgable source to consult on DB design, optimization and operations for our existing and future DB technologies: Mongodb, dynamodb, rockset, postgres, snowflake.
* Oversee ETL, ELT processes that are used in the company, both inside R&D and in other departments (R&D, Opertations, Marketing), using AWS Glue, HighTouch, Airbyte.
*  Design data tiering practices to find the tradeoffs for cost/performance/flexibility/query* data Security, Access control and regulation
* Support our efforts to map out PII data in our database and limit access to that
* Implement encryption at rest and other practices to protect our databases
* Execute data deletion tasks like removal of personal data, customer(shop/account) data as part of our SOC2/GDPR compliance* business intelligence