s
siddarthsingh28

Siddarth S

@siddarthsingh28

Senior Data Engineer

Índia
Inglês
Algumas informações são exibidas no idioma inglês.
Sobre mim
I am a Senior Data Engineer with 3+ years of experience building scalable batch and real-time data pipelines and enabling data-driven customer insights. I have strong expertise in Spark, Kafka, Python, SQL, and Azure.... Saiba mais

Habilidades

s
siddarthsingh28
Siddarth S
offline • 
Tempo médio de resposta: 1 hora

Conheça meus serviços

ETLs de dados
I will design and build end to end data pipelines and data warehouse solutions
A partir deUS$ 25 /projetoMais detalhes

Experiência profissional

Infosys

Senior Data Engineer

Infosys • Período integral

Aug 2023 - Present • 2 yrs 9 mos

• Designed and built scalable batch and real-time data pipelines using Spark, Kafka, and Azure, processing 50+ TB/day. • Performed data transformation and integration from multiple data sources, enabling analytics and customer insights. • Developed data models (conceptual, logical, physical) analyzing data flows and dependencies. • Built and maintained robust data pipelines integrating internal and external systems. • Optimized Spark and SQL workloads, reducing pipeline runtimes by 30–35%. • Implemented data governance and quality frameworks including validation and monitoring. • Developed and integrated APIs for data access and downstream consumption. • Automated workflows using Airflow and ADF with scheduling, retries, and dependency management. • Participated in cross-functional projects, translating requirements into technical solutions and ensuring delivery.