Backend Engineer — Model Inference

About the Role

Cohere is hiring a Backend Engineer to optimize model inference systems. You will build low-latency serving infrastructure, implement batching strategies, and develop the backend services that power Cohere's enterprise API.

Requirements

• 5+ years of backend engineering experience
• Strong proficiency in Go, Rust, or C++
• Experience with high-throughput, low-latency systems
• Knowledge of model serving and inference optimization
• Experience with gRPC, REST APIs, and microservices

Nice to Have

• Experience with vLLM, TensorRT-LLM, or similar
• Background in distributed systems
• Familiarity with GPU memory management
• Experience with continuous batching techniques

Benefits

Skills

Related Jobs