Data Engineer

BigData Development Phoenix, Arizona, United States

Profile
Contacts

Summary

General Motors Chandler, Arizona
Data Engineer Oct 2021- Present
Utilized Scala, Apache Spark, and Hive to effectively implement the Nextgen ETL pipeline programs
for the Quality Information System(QIS) project, enabling Power BI reports on global vehicle quality
metrics, and improving the performance and reusability of the program compared to the legacy system
Rewrote ETL pipelines using Python, Apache Spark, and Hive to support the organization's
retirement initiative for SAS and Oracle processes
Conducted data validations to ensure the correctness of data pipeline output results stored in Hive and
Greenplum DB using SQL and PostgreSQL queries
Created shell scripts and template job configs to enable ETL jobs using Docker images and
orchestrated their execution with Kubernetes pods
Formulated comprehensive unit testing cases to verify code quality and validate program accuracy,
contributing to the robustness and reliability of the software
Implemented batch job definitions within Autosys and Tidal automation platforms, ensuring precise
scheduling of ETL processes to align with critical business requirements
Leveraged Azure DevOps and Git for robust version control, facilitating efficient management of
project repositories, packaging of code, and reliable deployment of code changes into production
environments
Collaborated with other developers and stakeholders using Azure TFS in project planning and daily
standups, delivering timely project updates on user stories
Led a team of 5 people in an organizational ThinkTank Challenge to develop a prototype version of a
data quality check & analysis tool using Apache Spark and Scala in 4 weeks, won the service
recognition award
University of Arizona Information Technology Services Tucson, Arizona
Salesforce Administrator Assistant Mar 2020 - May 2021
Perform unit testing on Salesforce applications to ensure the performance of different functionalities
Participate in Scrum ceremonies with stakeholders to document requirements and explore solutions
Collaborate with 4 person Dev team to develop and revise testing plans for current/new projects
Conduct configuration changes, including Object, User permissions, and Email templates, to align with
the Clients requirements
Tech Core Tucson, Arizona
Tech Intern/ Student Coder Jun 2020 - Aug 2020
Used C# with the .NET framework to develop a web-based platform application to track student
performance for faculty users
Utilized different platforms, including Trello Board, Jira, and Git to track team development status
Collaborated closely with client representatives to identify and address any issues or needs for the
project