Staff Machine Learning Engineer, AI Serving
6 - 10 years
United States
$253.3k - $354.6k/year
Full-time
Remote
1 month ago
About the role
1. About Our Client:The organization operates within the digital community and information sharing industry, managing one of the internet’s largest collections of user-generated content across over 100,000 active communities. It addresses the challenge of facilitating authentic, interest-based conversations by providing a platform where millions of daily active users engage through submitting, voting, and commenting on a variety of topics. The company supports a vast ecosystem that influences content discovery and user engagement at significant scale.
2. About the Opportunity:The Staff Machine Learning Engineer, AI Serving will lead the design and development of a large-scale machine learning inference platform. The role focuses on creating and maintaining a highly available, low-latency GPU-based model serving system that supports search, ranking, and large language models (LLMs) at high query volumes. This position is critical for advancing the platform’s capabilities in ML and Generative AI systems within cloud environments, directly impacting various internal teams by enabling scalable and efficient AI-powered features.
3. Responsibilities:• Lead end-to-end design, implementation, and maintenance of GPU-based ML model serving systems supporting millions of queries per second
• Develop ML and Generative AI systems in cloud production environments using Kubernetes at scale
• Prototype and build high-performance feature hydration and processing components, including routing, caching, and batching
• Lead development of a unified GPU model export framework for optimized inference
• Implement real-time ML observability to monitor feature and model performance
• Work with large language model serving in online, scalable contexts
• Build end-to-end inference performance benchmarking frameworks
• Manage multi-cluster compute environments and network topology relevant to ML inference
4. Requirements:• 7+ years in ML Engineering, AI Platform Engineering, or Cloud AI Deployment roles
• Experience operating orchestration systems such as Kubernetes at scale
• Deep knowledge of cloud technologies for ML platforms, including AWS, Google Cloud Storage, and infrastructure-as-code tools like Terraform
• Proficiency in programming languages common to ML, including Go and Python
• Ability to communicate technical AI concepts effectively to non-technical stakeholders
• Strong emphasis on scalability, reliability, performance, and usability of AI platforms
• Knowledge of model serving, inference pipelines, monitoring, and observability is a plus
• Advanced Python skills and experience with AI/ML frameworks such as Triton, Dynamo, vLLM, and PyTorch
5. Pay Range and Compensation Package:• Base salary range: $253,300—$354,600 USD
• Eligible for equity in the form of restricted stock units and potentially commission depending on the role
• Additional benefits including medical, dental, vision insurance, 401(k) with employer match, vacation, and parental leave
6. Benefits & Perks:• Comprehensive healthcare and income replacement programs
• 401(k) with employer match
• Global benefit programs covering workspace, professional development, and caregiving support
• Family planning support
• Gender-affirming care
• Mental health and coaching benefits
• Flexible vacation and paid volunteer time off
• Generous paid parental leave
Equal Opportunity Statement: Our client is an equal opportunity employer. They celebrate diversity and are committed to creating an inclusive environment for all employees. All qualified applicants will receive consideration for employment without regard to race, color, religion, gender, gender identity or expression, sexual orientation, or national origin.
Note:RemoteHunter is not the Employer of Record (EOR) for this role. Our purpose in this opportunity is to connect exceptional candidates with leading employers. We help job seekers worldwide discover roles that match their goals and guide them to complete their full application directly through the hiring company’s career page or ATS.
2. About the Opportunity:The Staff Machine Learning Engineer, AI Serving will lead the design and development of a large-scale machine learning inference platform. The role focuses on creating and maintaining a highly available, low-latency GPU-based model serving system that supports search, ranking, and large language models (LLMs) at high query volumes. This position is critical for advancing the platform’s capabilities in ML and Generative AI systems within cloud environments, directly impacting various internal teams by enabling scalable and efficient AI-powered features.
3. Responsibilities:• Lead end-to-end design, implementation, and maintenance of GPU-based ML model serving systems supporting millions of queries per second
• Develop ML and Generative AI systems in cloud production environments using Kubernetes at scale
• Prototype and build high-performance feature hydration and processing components, including routing, caching, and batching
• Lead development of a unified GPU model export framework for optimized inference
• Implement real-time ML observability to monitor feature and model performance
• Work with large language model serving in online, scalable contexts
• Build end-to-end inference performance benchmarking frameworks
• Manage multi-cluster compute environments and network topology relevant to ML inference
4. Requirements:• 7+ years in ML Engineering, AI Platform Engineering, or Cloud AI Deployment roles
• Experience operating orchestration systems such as Kubernetes at scale
• Deep knowledge of cloud technologies for ML platforms, including AWS, Google Cloud Storage, and infrastructure-as-code tools like Terraform
• Proficiency in programming languages common to ML, including Go and Python
• Ability to communicate technical AI concepts effectively to non-technical stakeholders
• Strong emphasis on scalability, reliability, performance, and usability of AI platforms
• Knowledge of model serving, inference pipelines, monitoring, and observability is a plus
• Advanced Python skills and experience with AI/ML frameworks such as Triton, Dynamo, vLLM, and PyTorch
5. Pay Range and Compensation Package:• Base salary range: $253,300—$354,600 USD
• Eligible for equity in the form of restricted stock units and potentially commission depending on the role
• Additional benefits including medical, dental, vision insurance, 401(k) with employer match, vacation, and parental leave
6. Benefits & Perks:• Comprehensive healthcare and income replacement programs
• 401(k) with employer match
• Global benefit programs covering workspace, professional development, and caregiving support
• Family planning support
• Gender-affirming care
• Mental health and coaching benefits
• Flexible vacation and paid volunteer time off
• Generous paid parental leave
Equal Opportunity Statement: Our client is an equal opportunity employer. They celebrate diversity and are committed to creating an inclusive environment for all employees. All qualified applicants will receive consideration for employment without regard to race, color, religion, gender, gender identity or expression, sexual orientation, or national origin.
Note:RemoteHunter is not the Employer of Record (EOR) for this role. Our purpose in this opportunity is to connect exceptional candidates with leading employers. We help job seekers worldwide discover roles that match their goals and guide them to complete their full application directly through the hiring company’s career page or ATS.
Skills
Software Development