Senior/Lead Data engineer,/ Consultant
Currently I am working as Lead Data Engineer, I hold total of 9 years of experience.
I started my career as Database developer/ETL developer building the ETL pipeline using Shell scripting and using Oracle DB to create end-end flow from onboarding the data till generating the reports for users or downstream.
Eventually I started working in Hadoop and cloud environments to process large data. I have 3-4 years of experience in AWS.
I was/am responsible for onboarding different sources and integrating the source with the application.
I use Python for back end coding or creating api using Flask Framework.
Data ingestion/validation/loading is done in AWS using various services offered by Amason, S3 being the distributed storage and Athena and Redshift as the end user/ application consuming services. On daily basis I use EMR/Glue jobs to load the data and query the data in Athena to generate the reports.
I have used different RDMS systems - Oracle, MySQL, Ms SQL Server, PostgreSQL.
I have worked in both SDLC and Agile models.
Major Achievements being integrating open source application like Streamlit for data analysis for Data Scientists/ Product team.
Reducing the execution time on loading the data by 60%. I have also worked on improving application level performance tuning and have cut down the execution times from 30 minutes to 6-7 minutes.
Have Received multiple spot awards and employee of quarter award.
I am looking to working with energetic team that is looking to build revolutionary tech/ product.
Expected Base Salary
*,*00,000 INR / year
Total Professional Experience
Send a connection request to the candidate to get their contact details.Contact Candidate