Research Engineer: Inference / MLOps

Technical responsibilities of the role:

• Deploying various models to production and owning the process from working checkpoint to performant serving infrastructure.
• Optimizing inference quality and performance in the model serving stack to:
• Improve model output quality, e.g. experimenting with various guidance and guardrail schemes, automatic detection of hallucinations or errors.
• Improve the inference performance in terms of latency, throughput, cost; e.g. via custom kernels, cache optimization, GPU scheduling/scaling optimization, quantization.
• Implementing robust monitoring and incident flagging systems for inference management. E.g. flagging and investigating timing anomalies, inference failures, understanding general inference statistics across various models.
• Creating tooling to allow rapid deployment of newly developed models from checkpoint to scaled inference, optimizing GPU availability and choice for new models.
• Making autonomous decisions to holistically improve the inference experience in different models and use cases. E.g. how to approach handling the different kinds of optimization needed for an extreme low-latency streaming model versus a “quality sensitive” slow dubbing model.

Why Join Us?

• Innovative Environment: Be part of a forward-thinking company at the cutting edge of technology and digital transformation.
• Growth Opportunities: Enjoy professional development and career growth in a fast-paced, dynamic industry.
• Competitive Compensation: Receive a competitive salary, performance bonuses, and comprehensive benefits.