Data Engineer

General Software Development Newark, New Jersey, United States

Profile
Contacts

Summary

With over a year of comprehensive expertise, this candidate is a results-oriented professional specializing in system migration, data governance, and analytics. They successfully led the migration of clients from V1 to V2 frameworks, showcasing proficiency in ElasticSearch, Kafka, MongoDB. Adept at end-to-end pipeline management using ADF, they ensured efficient ingestion, ETL, and cube refresh, exhibiting commitment to streamlined processes. Their achievements extend to implementing Data Access Control rules, leveraging APIs to enforce data security, and reducing manual efforts by 60%. With a focus on optimization, they enhanced existing frameworks, increasing stability and reducing bugs by 45%. In the analytics realm, they revolutionized data loading, transitioning from full-load overwrite to ABC incremental framework, cutting refresh cycle load times by 70%. This candidate is distinguished by their impactful contributions to Ship Maintenance Scheduling decisions for the US Office of Naval Research through ML model analysis, EDA, and query time reduction. Their proficiency in automating data extraction, performing ETL transformations, and developing PowerBI dashboards underscores their commitment to data-driven insights and efficiency.

Expectations

As a data engineer, expectations include designing and managing scalable databases, optimizing ETL processes, ensuring data quality and security, and collaborating with cross-functional teams. Stay updated on emerging technologies, automate processes for efficiency, and adapt to evolving business needs. Effective communication, problem-solving, and a commitment to continuous learning are key attributes.