Data Engineer

Summary

Established a new company-wide standard data orchestration workflow utilizing Databricks Delta Live
Tables, reducing processing time by 99%.
Shortened deployment time of new metrics by 80% by improving Extract Transform Load (ETL) process.
Created a single parameterized SQL script for generating data warehouses for customer dashboards, reducing
maintenance overhead from 20 static scripts to 1
Identified and removed Pyspark ETL logging bottleneck, reducing processing time by 65% and lowering
AWS resource consumption by over 50%
Discovered excessive S3 resource consumption and developed shell script in Databricks to automate deletion
of excess backups in AWS S3 bucket, reducing S3 storage usage by 71%
Was the teams advanced SQL subject matter expert. Mentored teammate, leading to same-day turnaround
time for customer-facing reports
Led conversations with Data Science, Product, Operations, and Engineering teams to design customer report
and define report metrics
Worked asynchronously in a fully remote environment

Expectations

An asynchronous working environment

Employment Preferences

Expected Base Salary

**0,000 USD

Expected Hourly Rate

** USD/hr

Academic Degree

Experience

Skills

• Python
• SQL
• AWS
• Pyspark
• Hive
• PostgreSQL
• Dbt
• Data Modeling
• Data Warehousing