Position: Solution Architect
Experience: 10+ years
Qualification: Bachelor’s or master’s degree in computer science, Software Engineering, or a related field
Location: Gurgaon (Currently Work from Home till further notice)
Role Overview:
We are looking for a highly experienced and technically proficient Solution Architect – Databricks to lead and deliver complex data engineering and cloud migration projects across AWS, Azure, and GCP platforms. The ideal candidate will have deep expertise in Apache Spark, Databricks Lakehouse Platform, cloud-native architecture, and a strong foundation in Python and Java development. This role is client-facing and requires exceptional leadership, architectural decision-making, and hands-on implementation experience for large-scale data migration and modernization initiatives.
Key Responsibilities:
1. Solution Architecture & Delivery
- Design and implement scalable and secure Lakehouse architectures using Databricks across AWS, Azure, and GCP.
- Lead cloud data migration projects from legacy systems (e.g., on-prem, Hadoop, Snowflake, SQL Server) to Databricks.
- Architect solutions for real-time and batch data processing pipelines leveraging Delta Lake and Apache Spark.
- Provide architectural guidance on advanced use cases like streaming, ML pipelines, and data lineage.
2. Technical Leadership
- Mentor and guide cross-functional teams of data engineers and developers.
- Establish and promote best practices for CI/CD, data quality, monitoring, and security within Databricks environments.
- Review and optimize Spark jobs for performance, scalability, and cost-efficiency.
3. Client Engagement & Advisory
- Act as a trusted advisor to enterprise clients, driving strategic roadmaps for cloud adoption and modernization.
- Translate complex technical requirements into scalable solutions aligned with client business goals.
- Conduct technical workshops, PoCs, and architecture reviews with key stakeholders.
4. Platform Expertise
- Hands-on implementation and troubleshooting across cloud platforms: AWS (EMR, Glue, S3, IAM), Azure (ADF, Synapse, ADLS), and GCP (BigQuery, GCS, Dataflow).
- Leverage Unity Catalog, Databricks SQL, MLflow, AutoLoader, and DBFS effectively.
- Evaluate third-party integrations and tools within the Databricks ecosystem for advanced use cases.
Required Qualifications:
Technical Skills:
- Databricks: Advanced knowledge of Databricks architecture, Unity Catalog, SQL Warehousing, and cluster configurations.
- Spark: Deep understanding of Apache Spark internals including RDDs, DAGs, Catalyst, Tungsten, and job tuning.
- Languages: Proficiency in Python (PySpark, pandas) and Java (JVM tuning, Spark APIs).
- Cloud Platforms: Strong experience with AWS, Azure, and GCP services for data processing and storage.
- Tools: Experience with orchestration tools (e.g., Airflow, Azure Data Factory), DevOps (Git, Jenkins, Terraform), and monitoring (Datadog, Prometheus, etc.)
Experience:
- 10+ years in data engineering, big data, or cloud architecture roles.
- At least 3 large-scale data migration or modernization projects delivered successfully.
- Proven client-facing consulting experience with Fortune 500 enterprises.
Job Type: Full-time
Pay: ₹3,000,000.00 - ₹6,000,000.00 per year
Benefits:
- Provident Fund
- Work from home
Schedule:
- Monday to Friday
Work Location: Remote