Brighton, MA (Onsite)
Long Term
Key Responsibilities
Design, develop, and implement end-to-end big data solutions across the enterprise.
Develop and maintain big data applications using Apache Spark, Scala, AWS Glue, AWS Lambda, SNS/SQS, and CloudWatch.
Build and optimize data pipelines, ensuring high performance, scalability, and reliability.
Collaborate with cross-functional teams to design and document integration and application technical designs.
Conduct peer reviews of functional and design documentation to maintain high-quality standards.
Develop, configure, and perform unit testing and code reviews to ensure coding best practices.
Troubleshoot and resolve complex issues during testing phases and identify root causes efficiently.
Perform performance testing and optimize system performance.
Manage and maintain SQL-based databases, preferably Amazon Redshift.
Utilize Snowflake for data warehousing (experience in Snowflake is an added advantage).
Implement ETL/ELT processes and ensure data quality across various systems.
Work with Git repositories and manage CI/CD deployment pipelines for continuous integration and delivery.
Provide production support, including troubleshooting and environment tuning.
Ensure adherence to best practices and technical standards throughout the project lifecycle.
Required Skills \& Experience
6+ years of relevant experience in big data design and development.
Proficiency in Scala and/or Python for application development.
Strong expertise in Spark, AWS Glue, Lambda, SNS/SQS, and CloudWatch.
Hands-on experience with ETL/ELT frameworks and data integration.
Advanced knowledge of SQL, with experience in Redshift preferred.
Familiarity with Snowflake and cloud-based data solutions is advantageous.
Experience with CI/CD processes, Git, and production support in large-scale environments.