Data Engineer

BigData Development Cyberjaya, Selangor, Malaysia

Profile
Contacts

Summary

A Techie IT professional offering 8 years of progressive experience in Azure SQL, Data Factory, Databricks, Azure Analysis Services, Power BI
Leverages expertise in interpreting, building & implementing several applications in Azure and MSBI for Banking, Insurance, Aerospace & Defense and Airlines domains.
Created Data Pipelines with different sources and Destinations and parameterized them.
Implemented Managed Identity connections, For Loop and Data Flows activities in ADF.
Configured the Self-Hosted IR and Azure SSIS IR in Adf with Password free Linked Services.
Configuring Azure data Bricks with multiple clusters and Mounting Data Lakes using Azure app and Databricks Scope credentials.
Transforming data in Notebooks using different pyspark functions and user defined functions.
Creating and scheduling databricks Jobs for Incremental Load of staging tables.
Using multiple Magic and dbutils commands and implementing nested Notebooks to meet the requirements with reusable code.
Handled Data Governance using Unity catalog in data bricks.
Implemented CI/CD by migrating ADF from Dev to higher Env with Azure Devops Pipelines.
Implemented SQL server Analysis migration to Azure analysis services.
Experience in handling the migration from sql server to Azure SQL database.
Adept in interpreting user requirements & designing business reports to meet client requirements and documenting system processes & procedures for future references.
Deft in PowerBI, SQL, Excel for business analysis and preparing & maintaining regular standard Power Pivot reports & Analysis.
Quick learner and an excellent team player with strong ability to learn, adopt new skills and makes efforts to work effectively.
Expertise in ETL from relational data sources i.e. DB2, Oracle and flat files using ADF, SSIS.
Leverages expertise of ITIL framework - Incident Management, Change Management and Problem Management.