Data Engineer candidate in Chicago, Illinois, United States

Summary

Accomplished Data Engineer with 8+years of experience managing Azure Data Lakes and Analytics, leading migration projects, and designing optimal pipeline architectures on the Azure platform.
Proven expertise in implementing ETL processes using tools such as Informatica PowerCenter, Python, Scala, and AWS services like S3, DynamoDB, and RedShift.
Skilled in building data lakes from scratch using various technologies, including PostgreSQL, MongoDB, MySQL, Hadoop, and Spark, ensuring seamless data integration and processing.
Proficient in creating robust data visualizations using Power BI, Tableau, and QlikView, translating complex datasets into actionable insights for stakeholders.
Demonstrated ability to collaborate with cross-functional teams, enforce Git workflows, and manage tasks efficiently within Jira, fostering transparency and accountability.
In-depth knowledge of SQL Server installations, failover clustering, and database management, ensuring high availability and optimal performance.
Extensive experience in utilizing Python for data analysis, automation, and connecting to relational databases like MySQL and PostgreSQL.
Expertise in optimizing Hadoop cluster performance, configuring resource allocation, and implementing compression techniques for enhanced query speed.
Strong background in data security measures, including authentication, authorization, encryption, and compliance with data privacy regulations.
Skilled in designing and deploying applications using the entire AWS stack, including EC2, S3, Dynamo DB, Lambda functions, and RedShift, with a focus on high availability and fault tolerance.
Hands-on experience in developing Spark applications, using PySpark, Scala, and RDDs for efficient data aggregation and processing.
Proficient in conducting data quality assessments, validation checks, and ensuring data integrity within various tools, including QlikView, Google Analytics, and SAS.
Adept at converting Excel formulas into VBA code, creating compelling data-driven narratives, and implementing version control systems for efficient collaboration.
Experienced in collaborating with Azure DevOps to monitor transitions from development to production environments, ensuring a smooth deployment process.
Demonstrated proficiency in designing and optimizing financial models, revenue and cash flow statements, and conducting client profitability analyses.
Skilled in automating repetitive data tasks using Python scripts, increasing efficiency in data processing and access on AWS cloud processes.
Expert in analyzing product portfolio performance, identifying revenue opportunities, and demand planning for inventory, utilizing SQL, SAS, and Crystal Reports.
Well-versed in creating and maintaining reports and dashboards using various tools, including Excel, Tableau, and Power BI.
Knowledgeable about securing sensitive data in NoSQL databases and adhering to data privacy regulations and industry standards.
Experienced in working with multiple AWS instances, configuring security groups, Elastic Load Balancer, and AMIs for optimal performance.
Proven ability to design and deploy data table structures, reports, and queries in SQL Server, ensuring effective data retrieval and importation.
Proficient in using Google Analytics for in-depth analysis, uncovering insights, and providing data-driven recommendations for business improvement.
Skilled in optimizing Power BI reports for speed and responsiveness, enhancing the overall user experience.
Strong analytical, problem-solving, and communication skills with a commitment to staying updated on the latest technologies and methodologies.