Duties
Develop and maintain robust ETL (Extract, Transform, Load) processes to integrate data from diverse sources such as AWS cloud services, Hadoop clusters, and on-premises databases like Microsoft SQL Server and Oracle.
Design and implement scalable data warehouses and data lakes using platforms like Azure Data Lake and Hadoop ecosystems to support analytics and reporting needs.
Build and optimize big data processing workflows utilizing Apache Spark, Hive, and other distributed computing tools to handle large datasets efficiently.
Collaborate with cross-functional teams to gather requirements, translate business needs into technical solutions, and ensure seamless data integration via RESTful APIs and other interfaces.
Conduct database design and modeling to ensure efficient storage, retrieval, and analysis of structured and unstructured data.
Utilize programming languages such as Python, Java, Bash (Unix shell), VBA, and Shell Scripting to automate workflows, perform analysis, and develop custom solutions.
Support model training efforts by preparing datasets, tuning algorithms, and validating results for machine learning applications.
Participate in Agile development cycles to deliver iterative improvements rapidly while maintaining high standards of quality and documentation.
Skills
Proven experience with cloud platforms such as AWS and Azure Data Lake for scalable storage solutions.
Strong proficiency in SQL (including Microsoft SQL Server and Oracle) for querying, database design, and optimization.
Expertise in big data technologies including Hadoop ecosystem components like Hive, Spark, and related tools.
Hands-on experience with ETL tools such as Informatica or Talend for data integration tasks.
Knowledge of Looker or similar BI tools for creating dashboards and visual analytics.
Familiarity with Linked Data principles for connecting related datasets across the web or enterprise environments.
Ability to develop RESTful APIs for data access and integration purposes.
Experience with model training processes involving large datasets for predictive analytics or machine learning applications.
Strong analysis skills to interpret complex datasets accurately and derive actionable insights.
Database design expertise ensuring optimized schema structures for data warehouses or lakes.
Programming skills in Python, Java, Bash scripting, VBA or Shell Scripting to automate tasks effectively.
Knowledge of Agile methodologies to facilitate collaborative project development cycles. Join us in shaping the future of data-driven decision-making! We’re committed to fostering an inclusive environment where innovation thrives—empowering you to grow your skills while making a meaningful impact through cutting-edge technology solutions!
Job Type: Full-time
Pay: $67,804.48-$144,605.10 per year