JD
Job Overview: We are seeking talented and driven Software and Data Engineers to join our dynamic team. In this role, you will play a key part in maintaining and enhancing our existing data infrastructure, with an immediate focus on managing current priorities. As you grow within the team, you will contribute to building innovative solutions for data warehouses, data governance, AI/ML integration, and more. You will work with cutting-edge open-source tools, modern data engineering practices, and a robust API stack to drive our data initiatives forward.Key Responsibilities: Data Engineering:
Design, develop, and maintain scalable data pipelines and workflows using tools like Apache Spark, Kafka, and Airflow.
Implement data transformations, ETL processes, and manage data storage solutions including Delta Tables and DuckDB.
Optimize data systems for performance, scalability, and reliability.API Development:
Develop and maintain APIs using Flask or similar frameworks to enable seamless data access and integration.
Collaborate with cross-functional teams to integrate APIs with existing and new data platforms.Database Management:
Work with MongoDB and MySQL for efficient data storage and retrieval.
Design and implement data models to support business requirements.
Ensure database performance, reliability, and security.Collaboration and Mentorship:
Collaborate with team members to address current priorities and ensure smooth knowledge transfer.
Provide technical guidance and mentorship to junior engineers when required.Future Projects:
Contribute to the design and development of a modern data warehouse.
Participate in data governance and AI/ML initiatives.
Explore and implement innovative open-source tools to enhance data engineering capabilities.Required Skills and Qualifications:
Bachelor’s degree in Computer Science, Engineering, or a related field.
2-3+ years of experience in data engineering or software engineering roles.
Proficiency in Python and experience with frameworks like Flask.
Strong knowledge of open-source tools such as Apache Kafka, Spark, Airflow, and DuckDB.
Hands-on experience with MongoDB and MySQL.
Familiarity with data modeling, ETL pipelines, and workflow orchestration.
Understanding of Delta Tables and other modern data storage formats.
Knowledge of software engineering best practices, including version control (Git), CI/CD pipelines, and testing.Preferred Qualifications:
Experience with data governance practices and tools.
Familiarity with AI/ML workflows and feature engineering.
Exposure to distributed systems and large-scale data processing.
Strong problem-solving skills and attention to detail.
Excellent communication and teamwork skills.
Regards,