Data Engineer
6 - 10 years
United States
Contract
Remote
1 month ago
About the role
- Job Type – Long Term Contract
- Location – Remote – EST hours
- Requirements - GC holder or USC required
Overview
We are seeking an experienced Data Engineer to design, develop, and support scalable data solutions that enable analytics, reporting, and operational data services across the organization. This role will focus on building and optimizing ETL/ELT pipelines using Databricks and Apache Spark, developing complex SQL transformations, and enabling secure and scalable data exposure through APIs. The ideal candidate brings strong hands-on technical expertise, a deep understanding of data engineering best practices, and the ability to work collaboratively across technical and business teams in an agile environment.
Responsibilities:
- Design, develop, and maintain ETL/ELT pipelines using Databricks and Apache Spark for batch and incremental data processing
- Implement robust data ingestion patterns from multiple source systems including files, databases, APIs, and streaming sources where applicable
- Optimize Spark jobs and data pipelines for performance, scalability, reliability, and cost efficiency
- Ensure data quality, reconciliation, monitoring, and observability across pipelines and datasets
- Develop advanced SQL transformations for data cleansing, enrichment, aggregation, and reporting
- Design and maintain analytical data models including fact and dimension tables, curated data layers, and reporting views
- Support downstream reporting, analytics, and data science use cases with well-structured and scalable datasets
- Configure and manage data exposure through APIs for internal and external consumers
- Partner with application and integration teams to define API contracts and data payloads
- Ensure secure, scalable, and performant data access patterns
- Support API versioning and backward compatibility for published data services
- Collaborate with cross-functional teams to understand business and technical data requirements
- Participate in agile delivery processes and contribute to continuous improvement initiatives
- Document technical designs, processes, and operational procedures
Requirements:
- 5+ years of experience in data engineering roles
- Strong hands-on experience with Databricks and Apache Spark
- Advanced SQL proficiency including complex joins, window functions, and performance tuning
- Experience building and managing ETL/ELT pipelines
- Experience configuring and supporting data exposure through APIs
- Solid understanding of data warehousing and analytics concepts
- Experience with cloud data platforms such as AWS, Azure, or GCP preferred
- Familiarity with CI/CD practices for data pipelines preferred
- Experience with orchestration tools such as Airflow or Databricks Workflows preferred
- Knowledge of data quality, monitoring, and reconciliation frameworks preferred
- Strong problem-solving and analytical skills
- Ability to work independently on complex data challenges
- Clear communication skills with both technical and non-technical stakeholders
- Comfortable working in an agile and product-oriented environment
Skills
Information Technology