Data Engineer
Summary
Experience with Big Data core components and EcoSystem including DataIngestion and Data Processing
(pySpark, Hive, HDFS, Sqoop)
Good understanding of Hadoop architecture and different components of Hadoop ecosystem.
Uploaded and processed large amount of data from various structured and unstructured sources into
HDFS, Azure Cloud.
Importing and exporting data from different databases like MySQL and oracle into HDFS, Hive using
Sqoop.
Developed hive scripts with partitioning and bucketing for the hive tables for better performance.
Designed both Managed and External tables in hive for optimized performance.
Knowledge of Azure cloud services such as Azure Databricks, Storage Account(ADLS Gen2, Azure Blob Storage,Azure SQL Database,ADF.
Knowledge of Amazon Web Services (AWS) such as EC2,IAM,S3,Glue.
In-depth understanding of Spark Architecture including Spark Core, SparkSql, Dataframes.
Developed PySpark programs and created the data frames and worked on transformations.
Expectations
Looking foreword to use my data engineering skills to archive company goal along with my personal goal.
Employment Preferences
Expected Base Salary
*,*00,000 INR
Academic Degree
Security Clearance Level
Experience
Total Professional Experience
Startup Experience
Skills
Contacts are hidden
Send a connection request to the candidate to get their contact details.
Contact Candidate