We are looking for a proficient Data Engineer with strong skills in Spark and SQL to join Unison Group. The successful candidate will be responsible for designing, building, and managing data pipelines that facilitate real-time data processing. This role involves optimizing data flow and enhancing data quality to support the analytical needs of various teams within the organization.
Key Responsibilities:
Develop and maintain robust data pipelines using Apache Spark to process large datasets efficiently
Utilize SQL for data querying, manipulation, and management across various relational database systems
Collaborate with data scientists and analytics teams to understand data requirements and deliver suitable solutions
Implement data transformation and aggregation processes to optimize data for analysis
Monitor and troubleshoot data pipelines, ensuring high availability and reliability
Perform data quality checks and maintain data integrity throughout the data lifecycle
Stay updated on industry trends and best practices in data engineering, particularly in Spark and SQL technologies
Requirements
Bachelor's degree in Computer Science, Information Technology, or a related field
Experience as a Data Engineer, with a focus on Spark and SQL
Good to have experience in Banking/Finance domain
Proficiency in Spark for distributed data processing and ability to write efficient Spark jobs
Strong SQL skills, with experience working with relational databases (e.g., MySQL, PostgreSQL, Oracle)
Experience with data integration, ETL processes, and data warehousing concepts
Knowledge of big data technologies and frameworks is a plus
Strong analytical and problem-solving skills
Excellent communication and teamwork capabilities
Benefits
Salary: SGD 6000 - 8000
Great Work Culture
Includes Standard Annual and Sick Leave