We are seeking a
highly skilled and experienced AI and Gen AI Engineer
with deep expertise in
Databricks Mosaic
to join our data/AI engineering team. You will play a critical role in architecting and evolving our data analytics platform, building intelligent agents, and delivering robust AI governance using the Databricks platform.
Key Responsibilities:
Architect and evolve
the Data Analytics Platform using
Databricks Mosaic, Delta Lake, Unity Catalog, and MLflow
.
Collaborate
with data scientists, analysts, and business stakeholders to understand data requirements and deliver high-quality solutions.
Serve as the
technical authority
on Databricks Mosaic capabilities, including advanced Spark optimization, feature engineering, and real-time analytics.
Optimize performance
of Spark jobs and Databricks notebooks for large-scale data processing.
Manage and monitor
Databricks clusters, jobs, and workflows.
Ensure
data quality, integrity, and governance
across all data engineering processes.
Implement robust CI/CD pipelines
for ML model deployment using Mosaic.
Design and implement
LLM-enabled solutions using Databricks.
Decompose complex requirements
into manageable tasks.
Choose appropriate models, tools, and approaches
from the current generative AI landscape.
Build and deploy performant RAG applications
and LLM chains that leverage the Databricks toolset.
Required Skills \& Qualifications:
10–15 years
of experience in data and AI engineering, with at least
6-12 months hands-on LLM experience
.
Certified
in Databricks Gen AI Engineer Associate and Advanced ML Engineer certifications.
Strong proficiency in
Python
,
AI Models
, and
LLMs
.
Experience utilizing
Databricks-specific tools
: Vector Search, Model Serving, MLflow, Unity Catalog for data governance.
Experience with
CI/CD pipelines
and version control tools (e.g., Git).
Solid understanding of
AI, data governance, security, and compliance standards
.
Excellent problem-solving, communication, and collaboration skills
.