Core Responsibilities:
Platform Development:
Design and implement scalable data platform components, integrating AWS cloud services for data ingestion and processing.
Data Operations:
Build and optimize ETL/ELT pipelines using tools like AWS Glue and Spark, and automate workflows with Airflow.
Governance:
Implement role-based access controls and collaborate with governance teams on data quality and compliance.
Release Management:
Participate in CAB meetings to assess changes and maintain documentation for runbooks and SOPs.
Performance Tuning:
Develop monitoring models and conduct root cause analysis for data ingestion failures.
Leadership:
Provide technical guidance to engineering teams and work closely with product owners to align platform capabilities with business needs.
Required Skills:
Cloud Platforms:
AWS (mandatory), Azure or Ali Cloud (a plus).
Data Technologies:
Spark, Redshift, Glue, Kafka.
Programming:
Python, SQL, Airflow.
DevOps:
Jenkins, Docker/Kubernetes.
Soft Skills:
Strong stakeholder management and communication skills.
Qualifications:
8-10 years in data management or engineering, preferably in financial services.
2-3 years in the insurance industry preferred.
Fluent in English and Cantonese.