
Premchand
Data Engineer
Habilidades

Conheça meus serviços

Portfólio
Experiência profissional
Azure Data Engineer
Wipro Hydraulics
Jan 2022 - Sep 2026 • 4 yrs 8 mos
Led the implementation of metadata-driven ETL/ELT pipelines using Microsoft to ingest and process data from multiple source systems into Azure Data Lake Storage Gen2, Azure SQL, and Blob Storage. • Developed and optimized PySpark and Spark SQL workflows in Azure Databricks for large-scale structured and unstructured data processing. • Built end-to-end data integration pipelines in ADF using Linked Services, Datasets, Pipelines, Triggers, and parameterized workflows for automated processing. • Implemented incremental load strategies along with SCD Type 1 and Type 2 logic in Microsoft to support historical data tracking and reporting. • Contributed to implementing Medallion Architecture (Bronze, Silver, Gold layers) using Delta Lake and Parquet formats for scalable and analytics-ready data platforms. • Designed and optimized complex SQL queries, Spark transformations, partitioning, and caching techniques to improve ETL and reporting performance. • Implemented monitoring, logging, alerting, and error-handling mechanisms using Azure Monitor and Log Analytics to ensure data reliability and operational efficiency. • Collaborated with business stakeholders, QA teams, and cross-functional teams to gather requirements, resolve production issues, and deliver data-driven insights.