Data Engineer
6 - 10 years
Atlanta Metropolitan Area
Full-time
Hybrid
1 month ago
About the role
We're hiring on behalf of a Haystack partner!
The Role
• Design, build, and maintain complex data systems that facilitate data analysis and reporting.
• Ensure large datasets are efficiently processed and accessible for decision-making with minimal supervision.
• Develop and maintain streaming and batch data pipelines for seamless data ingestion, transformation, and storage.
• Build prototypes to test new concepts and implement data engineering frameworks to enhance processing capabilities.
• Collaborate with cross-functional data and analytics teams to gather requirements and ensure solutions meet partner needs.
• Review and optimize existing data systems and architectures for performance and scalability.
What You'll Need
• Minimum of 4 years of relevant work experience.
• Experience developing data systems on major cloud platforms (AWS, GCP, Azure).
• Hands-on experience building modern data architectures (data lakes, lakehouses) with ingestion, governance, and modeling capabilities.
• Proficiency in data ingestion tools and storage formats (e.g., Kafka, AWS Glue, Iceberg, Parquet).
• Experience developing data pipelines with streaming architectures and tools (e.g., Kafka, Flink).
• Advanced programming skills in Python, Java, Scala, or similar, with expert-level SQL proficiency.
What's On Offer
• Competitive annual salary.
• Opportunities to work with advanced engineering and cloud-based technologies.
• Engage with a global organization focused on sustainable practices.
• Contribute to solutions that impact billions of lives daily.
Apply via Haystack today!
The Role
• Design, build, and maintain complex data systems that facilitate data analysis and reporting.
• Ensure large datasets are efficiently processed and accessible for decision-making with minimal supervision.
• Develop and maintain streaming and batch data pipelines for seamless data ingestion, transformation, and storage.
• Build prototypes to test new concepts and implement data engineering frameworks to enhance processing capabilities.
• Collaborate with cross-functional data and analytics teams to gather requirements and ensure solutions meet partner needs.
• Review and optimize existing data systems and architectures for performance and scalability.
What You'll Need
• Minimum of 4 years of relevant work experience.
• Experience developing data systems on major cloud platforms (AWS, GCP, Azure).
• Hands-on experience building modern data architectures (data lakes, lakehouses) with ingestion, governance, and modeling capabilities.
• Proficiency in data ingestion tools and storage formats (e.g., Kafka, AWS Glue, Iceberg, Parquet).
• Experience developing data pipelines with streaming architectures and tools (e.g., Kafka, Flink).
• Advanced programming skills in Python, Java, Scala, or similar, with expert-level SQL proficiency.
What's On Offer
• Competitive annual salary.
• Opportunities to work with advanced engineering and cloud-based technologies.
• Engage with a global organization focused on sustainable practices.
• Contribute to solutions that impact billions of lives daily.
Apply via Haystack today!
Skills
Information TechnologyTechnology, Information and Internet