Key Responsibilities:
Design and Develop AI-Powered Solutions: Lead the design, development, and deployment of AI-powered solutions using Azure OpenAI, Azure Databricks, and other components or our technology stack. Data Engineering: Build and maintain scalable data pipelines using Azure Data Factory, Azure Functions, and Kafka. Ingest and transform data from SQL Server, Cosmos DB, and Snowflake. Search & Retrieval: Implement semantic search using Azure Search Index and vector databases (e.g., Azure Cosmos DB with vector search, MongoDB vCore). Apply hybrid retrieval techniques for RAG applications Document Intelligence: Use Azure Document Intelligence to extract structured data from unstructured formats (PDFs, Office docs, JSON). Enable OCR, chunking, and metadata propagation for downstream analytics Technical Leadership: Provide technical leadership and guidance to junior data scientists and engineers on Databricks, and AI agent development. Mentor and coach team members to improve their skills and expertise. AI Agent Development: Design and develop AI agents using Mosaic AI and other relevant technologies. Collaborate with stakeholders to identify opportunities for AI agents to drive business value. Data Science Innovation: Stay abreast of the latest advancements in data science and AI, identifying opportunities to apply new techniques and technologies to drive business innovation. Collaboration and Communication: Collaborate with stakeholders across the organization to identify business problems and develop data-driven solutions. Communicate complex technical concepts to non-technical stakeholders, driving adoption and understanding of data science solutions.
Requirements:
Education/Experience: Degree or advanced degree in data science, physics, mathematics, statistics, computer science or related quantitative field. BS and 10+ years related experience or MS and 7+ years related experience or PhD and less than 2 years’ experience. 1-3 Years Supervisory experience preferred.
Technical Skills:
- Proficiency in Databricks, or a similar platform like AWS SageMaker, Azure Machine Learning, Vertex AI, and others.
- Experience with Azure OpenAI, Azure Search, Vector DBs, and Document Intelligence
- Strong programming skills in languages such as Python, Scala, SQL, etc.
- Familiarity with Kafka, Event Hub, and FiveTran for streaming and ingestion
- Ability to work with multi-modal embeddings and semantic search techniques
- Familiarity with data engineering, data warehousing, and data governance.
Soft Skills:
- Excellent communication and collaboration skills.
- Strong leadership and mentoring skills.
- Ability to drive innovation and stay up to date with the latest advancements in data science and AI.
Nice to Have:
Experience with Cloud Platforms: Experience working with cloud platforms such as AWS, Azure, or Google Cloud.
Experience with AI agents: Experience developing AI agents using Mosaic AI or similar code-first platforms.
Certifications: Databricks or Mosaic AI certifications are a plus. Open-Source Contributions: Contributions to open-source projects related to data science, machine learning, or AI.
