Data Engineer,
Job Description
Position : Data Engineer
Required (Individual Role):
Bachelors degree in Information Systems, Computer Science, Engineering, or related field
Hands-on experience creating automated data pipelines using modern technology stacks for batch ETL, data streaming, or change data capture, and for data processing to load advanced analytics data repositories
Proficient in designing and implementing data integration processes in a large, distributed environment using cloud services e.g., Azure Event Hub, Data Factory, Data Catalog, Databricks, Stream Analytics, Apache Spark
Experience in designing data lake storage structures, data acquisition, transformation, and distribution processing
Proficient in developing and operationalizing various data distribution patterns such as APIs, event based, publish/subscribe models
Experience migrating data lakes from on-premises to cloud
Experience using the Apache Hadoop technology stack
Advanced experience in SQL programming
Proficient in programming languages (e.g., Python, Java)
Experience with agile delivery using Jira, or Azure DevOps
Experience monitoring performance of data workflows and infrastructure through use of monitoring tools and enabling alerts
Functional knowledge of data visualization tools
Excellent written and oral communication skills.
