|
Must-Have Skills & Qualifications:
- Minimum 12+ years of IT experience and at least 5+ years of experience in architecting cloud based (preferably MS Azure) data platforms, especially leveraging Databricks and Azure native services.
- At least one project experience in architecting and designing a complex data platform using Datamesh architecture and experience in designing and building data domains and data products.
- Proficient in creating cloud architecture and Technical Design Documents (TDD).
- Experience in building data platforms using Medallion architecture.
- Ability to represent the program in high-level architecture forums and align the solution to organization policies, guidelines and best practices.
- Strong hands-on expertise in designing and building Data engineering solutions using PySpark and Scala.
- Deep expertise in Performance tuning and optimization of Databricks frameworks and fine-tuning of PySpark/Scala data pipelines.
- Hands on experience working with Azure Cloud services like Azure Data Factory, Azure Data Lake Storage, Azure Event hub, Azure SQL, Azure Analytical services, Azure Cosmos DB.
- Should be able to demonstrate expertise in designing and building efficient and scalable data platform solutions using Massively Parallel Processing (MPP) architecture.
- Micro-batch streaming and continuous streaming processes in Databricks for high latency, low latency and ultra-low latency of data accordingly by using inbuilt Apache spark modules
- Data Ingestion to one or more Azure Services - (Azure Data Lake, Azure Storage, Azure SQL, Azure DW) and processing the data in Azure Databricks.
- Ability to design and recommend the ideal infrastructure requirements for optimal extraction, transformation, and loading of data from a wide variety of structured and unstructured data sources.
- Well versed with Relational and Dimensional Modeling techniques like Star, Snowflake Schema, OLTP, OLAP, Normalization, Fact and Dimensional Tables
- Familiar with DevOps principles and experience in CD/CI implementations using Azure native services.
- Very good customer relationship and stakeholder management skills
- Excellent communication and presentation skills.
|