Job Responsibilties:• Build Data applications with high accuracy and performance across traditional and distributed computing platforms.
• Design, build, and maintain high performance, reusable, and reliable code quality and features being delivered efficiently and on-time. Document everything.
• Develop database processes, gather, and process raw data at scale (including writing scripts, web scraping, calling APIs, write SQL queries in MySQL, handle data cloud etc.).
• Administer data processing workflows associated with tools like MySQL, Oozie, Zookeeper, Sqoop, Hive, Impala for data processing across the distributed platform.
• Work closely with our engineering team to integrate your amazing innovations and algorithms into our production systems.
• Support business decisions with ad hoc analysis as needed and troubleshoot production issues and identify practical solutions.
• Routine check-up, back-up and monitoring of the entire MySQL and Hadoop ecosystem.
• Take end-to-end responsibility of the Traditional Databases (MySQL), Big Data ETL, Analysis and processing life cycle in the organization and manage deployments of bigdata clusters across private and public cloud platforms.
Required Skills:
• 6+ years of experience with SQL (MySQL) a must.
• 3+ years of Hands-on experience working with Cloudera Hadoop Distribution platform and Apache Spark.
• Strong understanding of full dev life cycle, for backend database applications across RDBMS and distributed cloud platforms.
• Experience as a Database developer writing SQL queries, DDL/DML statements, managing databases, writing stored procedures, triggers and functions and knowledge of DB internals.
• Knowledge of database administration, performance tuning, replication, backup, and data restoration.
• Comprehensive knowledge of Hadoop Architecture and HDFS, to design, develop, document and architect Hadoop applications. Working knowledge of SQL, NoSQL, data warehousing & DBA along with Map-Reduce, Hive, Impala, Kafka, HBase, Pig, and Java.
• Experience processing large amounts of structured and unstructured data, extracting, and transforming data from remote data stores, such as relational databases or distributed file systems.
• Working expertise with Apache Spark, Spark streaming, Jupyter Notebook, Python or Scala programming.
• Excellent communication skills, ability to tailor technical information for different audiences. Excellent teamwork skills, ability to self-start, share insights, ask questions, and report progress.
• Working knowledge of the general database architectures, trends, and emerging technologies. Familiarity with caching, partitioning, storage engines, query performance tuning, indexes, and distributed computing frameworks.
• Working knowledge & understanding of data analytics or BI tools - like looker studio, Power BI, or any other BI tools is a must.
Additional Desired Skills:
• Added advantage if you have exposure to advance technology components like – caching techniques, load balancers, distributed logging, distributed queries, queueing engines, containerization, html/CSS optimization, mobile app & web server optimization, cloud services.
• Strong attention to detail on every line of code, every unit test, and every commit message. Comfortable with rapid development cycles and tight schedules.
• Experience with Linux, GitHub, Jira is a plus. Good experience with benchmarking, optimization, and CI/CD pipeline.
• Experience with web paradigms such as REST, Responsive Web Design, Test-driven Development (TDD), Dependency Injection, unit testing frameworks such JUnit, etc.
• Bachelor’s degree or higher in Computer Science with relevant skills in mobile application development and web.