Large Language Model Inference System Engineer Graduate (Applied Machine Learning) - 2027 Start

ByteDance is hiring a graduate engineer to develop and optimize the Volcano Ark MaaS inference system. The role focuses on improving large-model inference performance, cost, and stability across large-scale heterogeneous clusters.

What you'll do:

• Participate in the engineering development of the Volcano Ark MaaS inference system, optimizing large model inference performance, cost, and stability in ultra-large-scale heterogeneous inference clusters
• Reduce large model inference costs through system-level approaches such as disaggregated multi-role inference, distributed KV Cache systems, heterogeneous inference, elastic computing, and multi-tenant co-located inference

What they're looking for:

• Individuals who are completing or have recently completed a Bachelor's or Master's degree in Computer Science or a related discipline
• Strong command of algorithms, design patterns, and data structures, with solid knowledge of operating systems and computer architecture
• Proficient in one or more programming languages such as C++ or Python, with good coding style
• Understands GPU hardware architecture, is familiar with high-performance computing software stacks such as CUDA, and has experience in GPU performance analysis
• Strong interest in distributed systems and large-scale heterogeneous inference; enjoys studying low-level principles and performance bottlenecks, and actively follows progress in related fields
• Experience in large model inference system optimization, with practical understanding of PD disaggregation, KV Cache systems, and multi-node inference
• Experience in distributed network communication optimization, with deep understanding of distributed communication operator implementation and RDMA principles
• Familiarity with resource orchestration and scheduling frameworks such as Kubernetes and Ray

Benefits:

• May be eligible for additional discretionary bonuses/incentives and restricted stock units.
• Employees have day one access to medical, dental, and vision insurance, a 401(k) savings plan with company match, paid parental leave, short-term and long-term disability coverage, life insurance, and wellbeing benefits. Benefits may vary depending on the nature of employment and the country work location.
• 10 paid holidays per year.
• 10 paid sick days per year.
• 17 days of Paid Personal Time (prorated upon hire with increasing accruals by tenure).