[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"tag-vllm":3},{"tag":4,"articles":11},{"id":5,"name":6,"slug":7,"article_count":8,"description_zh":9,"description_en":10},"6acb2d1f-934e-4e31-a9d1-8e4392fb099a","vLLM","vllm",6,"vLLM 是面向大型語言模型的高吞吐推理引擎，重點在 PagedAttention、KV cache 管理與連續批次處理，讓 GPU 更有效率地服務聊天、RAG、批次生成與多模型部署。","vLLM is a high-throughput inference engine for large language models, built around PagedAttention, KV cache management, and continuous batching. It matters for chat services, RAG pipelines, batch generation, and multi-model GPU deployment.",[12,21,29],{"id":13,"slug":14,"title":15,"summary":16,"category":17,"image_url":18,"cover_image":18,"language":19,"created_at":20},"670a7f69-911f-41e8-a18b-7d3491253a19","turboquant-vllm-comparison-fp8-kv-cache-en","TurboQuant vs FP8: vLLM’s first broad test","vLLM found FP8 KV-cache quantization beats TurboQuant on speed, while TurboQuant’s strongest variants hurt accuracy.","research","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1778839858405-b5ao.png","en","2026-05-15T10:10:37.219158+00:00",{"id":22,"slug":23,"title":24,"summary":25,"category":26,"image_url":27,"cover_image":27,"language":19,"created_at":28},"6dcd6852-b95a-4f62-853a-cc7eb32fff1a","gemma-4-assistant-models-faster-draft-tokens-en","Gemma 4 assistant models get faster draft tokens","Gemma 4 E2B and E4B assistant models use centroid masking to cut lm_head work about 45x with little quality loss.","tools","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1778278254841-r19z.png","2026-05-08T22:10:34.02358+00:00",{"id":30,"slug":31,"title":32,"summary":33,"category":26,"image_url":34,"cover_image":34,"language":19,"created_at":35},"00a0853d-92b0-45e5-bfcd-97d7f77ec8a0","awesome-open-source-ai-projects-list-en","Awesome Open Source AI: the best projects list","This GitHub list curates battle-tested open-source AI tools, models, and infra, from PyTorch to vLLM, with 2,486 stars.","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1775999039728-fc7m.png","2026-04-12T13:03:36.707391+00:00"]