Data Engineer

About the role

Master works is seeking a Data Engineer to join our team and play a key role in developing, operating, and maintaining the organization’s data pipelines and data platform infrastructure. The role will be responsible for ensuring reliable data processing, platform availability, security, performance, and compliance with pipeline SLAs.

Key responsibilities

• Design, develop, and maintain ELT/ETL pipelines to produce clean, structured, and reliable datasets.
• Build and maintain data orchestration workflows using Apache Airflow.
• Develop and test data transformations using SQL, Python, and PySpark.
• Monitor pipeline execution, troubleshoot processing issues, and proactively address SLA risks.
• Administer and maintain lakehouse, object storage, Apache Spark, and Trino environments.
• Build and maintain CI/CD pipelines using Bitbucket and Jenkins.
• Manage user access, roles, permissions, and resource quotas across data platforms.
• Monitor platform performance, respond to incidents, and perform system upgrades and patching.
• Optimize storage configurations, including partitioning, data formats, and data retention.
• Collaborate with Data Integration, Data Engineering, BI, and Analytics Engineering teams to support data initiatives.

Required profile

• Bachelor’s degree in Computer Science, Information Systems, Software Engineering, or a related field.
• Minimum 5 years of relevant experience in Data Engineering or a related field.
• Hands‑on experience with lakehouse architecture, object storage, and ELT/ETL development.
• Practical experience with Bitbucket, Jenkins, and CI/CD practices.
• Experience working with enterprise data platforms and supporting production data environments.

Required skills

• SQL
• Python
• PySpark
• Apache Spark
• Trino
• Apache Airflow
• Lakehouse architecture
• Object storage
• Bitbucket
• Jenkins
• CI/CD
• Access management
• Platform monitoring
• Incident response
• Performance optimization