[Slides] [Project]

Inference workloads are becoming increasingly prevalent and vital in Cloud Native world. However, it’s not easy, one of the biggest challenges is large foundation model can not fit into a single node, like llama 3.1-405B or DeepSeek R1, which brings out the distributed inference with model parallelism, again, make serving inference workloads more complicated.

Just finished the KubeCon EU 2025 and will end my vocation tomorrow, I’m going to write down something to summarize my KubeCon trip.

Hiking in the South of England.

Stamford Bridge.

[Slides] [Project]

Inference workloads are becoming increasingly prevalent and vital in Cloud Native world. However, it’s not easy, one of the biggest challenges is large foundation model can not fit into a single node, which brings out the distributed inference with model parallelism, again, make serving inference workloads more complicated.

Emirates Stadium.

Anfield Stadium.

Raining in Old Trafford.

Etihad Stadium.

Tottenham Hotspur Stadium.