Portfolio

Dec 29

Released AMRS, an adaptive model routing system for exploration and exploitation.
Nov 14

Released AlphaTrion, an open-source framework to help build GenAI applications, including experiment tracking, adaptive model routing, prompt optimization and performance evaluation.
Jul 22

Joined CNCF TAG Workloads as Tech Lead for a two-year term. CNCF is one of the world’s most successful open-source foundations.
Mar 20

Founded ModelSpec.ai with the mission of creating a model hub and benchmarking platform for AI models.
Feb 22

Published paper AIBrix Towards Scalable, Cost-Effective Large Language Model Inference Infrastructure. See project here vllm-project/aibrix.

Jul 23

Released llmaz, an unified LLM serving platform supporting various inference engines like vLLM, SGLang, TGI, llama.cpp and more.
Mar 13

Released LeaderWorkerSet, a multi-host LLM serving framework. Adopted by companies like AWS, DaoCloud, Google, Nvidia, RedHat and projects like Kubeflow Trainer, llm-d, Nvidia Dynamo, vLLM, SGLang, Volcano and more.

Aug 27

Released Awesome-LLMOps, an awesome & curated list of best LLMOps tools.
Aug 14

Founded InftyAI, a community-driven, open-source initiative for developing scalable AI infrastructure, including model training, model inference, agentic systems, and more. Welcome to join us to explore the ∞ possibilities of AI!

Jun 17

Joined Kubernetes SIG-Scheduling maintainers team. Kubernetes is one of the most popular and influential open-source projects in the world, the foundation of modern cloud-native infrastructure.
Apr 12

Released Kueue, the Kubernetes-native Job queueing system. Adopted by companies like Apple, CERN, DaoCloud, Google, IBM Research, RedHat and more.