Data Engineer
Summary
I am a Data Engineer with over four years of experience in managing and optimizing data pipelines for large-scale datasets. My expertise spans across various stages of the data pipeline, including data acquisition, validation, transformation, and visualization. I have successfully orchestrated complex data workflows using Airflow and have extensive hands-on experience with both AWS and Azure cloud platforms.
At JPMorgan Chase, I designed automated data pipelines using Airflow, reducing manual data processing by 20%. I also implemented real-time fraud detection systems with Spark Streaming and Scala, significantly enhancing the ability to identify and prevent fraudulent activities. My work in developing efficient data integration pipelines with Azure Data Factory resulted in a 10% reduction in manual effort for data movement. Furthermore, I optimized data processing in Azure Databricks, reducing processing time by 40% through in-memory processing.
Previously, at Capgemini, I spearheaded the migration of data transformations from AWS to Azure Databricks, achieving an 80% increase in data processing efficiency. I also enhanced Spark jobs and query performance in AWS Redshift, contributing to faster processing times and improved data accessibility. My work on creating a Power BI dashboard enabled more efficient KPI tracking, saving valuable time in manual reporting.
With a strong foundation in both cloud and big data technologies, I have a proven track record of improving data processing efficiencies and building scalable data solutions tailored to business needs.
Expectations
I am seeking opportunities where I can leverage my expertise in data engineering to build and optimize scalable data pipelines, enhance real-time data processing, and contribute to cloud-based big data solutions. I am excited to work in an innovative environment that values automation, efficiency, and collaboration, allowing me to solve complex data challenges while having a meaningful impact on the society
Employment Preferences
Spoken Languages
- English - Fluent
- Hindi - Intermediate
- Telugu - Fluent
Expected Base Salary
**,000 USD
Expected Hourly Rate
** USD/hr
Academic Degree
Experience
Total Professional Experience
Startup Experience
Big-Tech Companies
Enterprise Experience
Skills
- Python3
- JavaScript
- SQL
- C
- C++
- Matlab
- Scala
- R
- Java 8
- Django
- Springboot
- Hibernate
- Microservices
- SpringMVC
- HTML5
- CSS3
- JQuery
- React
- Redux
- Hadoop
- Hive
- Pig
- HDFS
- Spark
- Kafka
- Apache Airflow
- Zookeeper
- Apache Flink
- MySQL
- PostgreSQL
- MongoDB
- Snowflake
- BigQuery
- AWS
- S3
- EMR
- EC2
- Glue
- Lambda
- SDK
- DynamoDB
- Elasticsearch
- QuickSight
- Kinesis
- Athena
- VPC
- Redshift
- Azure
- Data Lake
- Data Factory
- Databricks
- Logic Apps
- HDInsight
- Synapse Analytics
- Stream Analytics
- GCP
- Google Cloud Storage
- Cloud Dataflow
- Cloud Composer
- Pub
- SUB
- Cloud Functions
- Linear Regression
- Logistic Regression
- Decision Tree
- SVM
- K-means
- Random Forest
- TensorFlow
- NumPy
- Matplotlib
- Seaborn
- Plotly
- PySpark
- Agile
- Scrum
- CI
- CD
- Docker
- Jenkins
- GitLab
- Git
- Github
- Terraform
- CodeDeploy
- CodePipeline
- Shell Scripting
- Power BI
- Venv
- Conda
- Jira
- Confluence
- IntelliJ
- PyCharm
- Jupyter Notebook
- VS Code
- Eclipse
Contacts are hidden
Send a connection request to the candidate to get their contact details.
Contact Candidate