Cloud Engineer
We have partnered with a leading FS Client, who are looking for a Cloud Engineer, with hands on experience with Azure cloud platforms.
Responsibilities:
Design \& Management
: Develop, provision, and oversee secure, scalable Azure Databricks platforms to facilitate enterprise-wide data transformation for insurance data workloads.
Collaboration
: Work alongside architects, engineers, and security teams to establish and implement robust infrastructure standards, ensuring reliable connectivity and seamless integration with legacy systems, cloud data sources, and third-party platforms.
Infrastructure as Code
: Utilize tools like Terraform to automate the provisioning and configuration of Azure and Databricks resources, aligning with DevOps best practices.
Automation
: Streamline deployment, monitoring, and incident response workflows using GitHub Actions to enhance consistency, traceability, and operational efficiency.
Monitoring \& Performance
: Regularly assess platform health, resource utilization, and performance; anticipate scaling requirements and conduct tuning to ensure optimal operation of data pipelines and analytics workloads.
Security \& Compliance
: Enforce adherence to enterprise and regulatory standards, including RBAC, managed identities, encryption, PDPO, GDPR, and specific insurance requirements.
Data Governance
: Manage the integration of Informatica tools to support data governance efforts, including cataloguing, data lineage, and compliance checks.
Documentation
: Create and maintain comprehensive documentation of infrastructure architecture, network topology, security configurations, and operational runbooks for governance, audits, and handover processes.
Troubleshooting
: Identify and resolve infrastructure issues, perform root cause analysis, and drive continuous improvement for platform reliability.
Continuous Learning
: Stay updated on the latest Azure, Databricks, and DevOps features, recommending and implementing enhancements to optimize platform capabilities and cost-effectiveness in alignment with Hong Kong Life and General Insurance business priorities.
Requirements:
Education
: Bachelor's degree in Computer Science, Information Technology, or a related field.
Experience
: 5+ years of hands-on experience in designing, deploying, and managing Azure cloud platforms, specifically supporting Azure Databricks for insurance data and analytics workloads.
Technical Expertise
: In-depth knowledge of provisioning, configuring, and managing Azure Databricks clusters and workspaces, including the handling of structured, semi-structured, and unstructured insurance data.
Integration Skills
: Proficient in integrating Azure Data Factory and Azure Data Lake Storage Gen2 with Databricks for automated data workflows.
Infrastructure as Code
: Experienced in using tools like Terraform for automated deployment and configuration of Azure and Databricks services.
Informatica Solutions
: Familiar with deploying and integrating Informatica solutions for effective metadata management, cataloguing, and governance.
Security Knowledge
: Strong understanding of platform security measures (RBAC, NSG, managed identities, Key Vault), monitoring, alerting, and cost optimization in regulated environments.
CI/CD Automation
: Hands-on experience with GitHub Actions for automating CI/CD pipelines related to platform and pipeline deployments.
Incident Response
: Proven ability in incident response, troubleshooting, and performance optimization for critical insurance data workloads.
Communication Skills
: Excellent documentation, collaboration, and communication skills to effectively support technical and business users within the insurance sector.