Company Profile - Celebal Technologies a Premier Microsoft Azure partner and have been recently funded by Norwest Venture Capital.
Celebal Technologies is a premier software services company in the field of Data Science, Big Data, Enterprise Cloud \& Automation. Established in 2016, in this short span of time we grew to headcount of 2200+ We help in achieving a competitive advantage with intelligent data solutions, built using cutting-edge technology.
Job Summary: The Senior Data Engineer plays a critical role in designing, developing, and maintaining scalable data infrastructure and pipelines. This position supports data-driven decision-making across the organization by ensuring data quality, accessibility, and performance.
Roles \& Responsibilities:
Data Pipeline Development: Design, build, and maintain robust, scalable data pipelines using tools like Apache Airflow, Kafka, and Spark.
Data Architecture: Develop and optimize data models, data warehouses, and ETL processes to support analytics and machine learning initiatives.
Collaboration: Work closely with data scientists, analysts, and software engineers to understand data requirements and deliver tailored solutions.
Data Quality \& Governance: Ensure data integrity, implement validation checks, and enforce data governance policies.
Performance Optimization: Monitor and improve data processing speed, system uptime, and query performance.
Leadership \& Mentorship: Guide junior engineers, promote best practices, and foster a culture of continuous learning.
Innovation \& Research: Stay updated with emerging technologies in AI, automation, and data streaming; lead initiatives to integrate them into existing systems.
Other duties as required.
Desired Skills \& Experience:
Bachelor's or master's degree in computer science, data science, information technology, or a related field, with 6+ years of experience.
Proficiency in SQL, Python, and ETL/ELT tools
Experience in Microsoft solutions are as follows:
1) Azure Data Factory
2) Microsoft Fabric
3) Azure Synapse Analytics
4) Azure Databricks
5) Power BI
6) Azure SQL Database / SQL Server
7) Azure Purview
Experience with big data technologies (Hadoop, Spark)
Familiarity with cloud platforms (Azure, AWS, and GCP)
Strong understanding of data modelling, data warehousing, and stream processing
Knowledge of data security, compliance, and governance
Knowledge of CI/CD pipelines using Azure DevOps
Understanding of data security and compliance (e.g., GDPR, HIPAA)