[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"article-databricks-model-serving-adapts-not-tuned-by-hand-zh":3,"article-related-databricks-model-serving-adapts-not-tuned-by-hand-zh":30,"series-industry-8022a066-f10a-4469-9c38-2e7ebe197f39":73},{"id":4,"slug":5,"title":6,"content":7,"summary":8,"source":9,"source_url":10,"author":11,"image_url":12,"cover_image":12,"category":13,"language":14,"translated_content":11,"related_article_id":15,"keywords":16,"key_takeaways":22,"views":26,"created_at":27,"published_at":28,"topic_cluster_id":29},"8022a066-f10a-4469-9c38-2e7ebe197f39","databricks-model-serving-adapts-not-tuned-by-hand-zh","Databricks 說得對：模型服務應該自適應，不該靠人工調參","\u003Cp data-speakable=\"summary\">生產環境的模型服務應該依模型與流量自動調整，而不是靠工程師手動調參。\u003C\u002Fp>\u003Cp>我站在 Databricks 這邊：模型服務的未來是自適應基礎設施，不是團隊長期盯著 replicas、concurrency 和 autoscaling 旋鈕反覆微調。\u003C\u002Fp>\u003Cp>Databricks 把這件事講得很直接。它的 Custom Model Serving 宣稱可同時支援從 2 MB 的 scikit-learn 分類器到 70B 微調 \u003Ca href=\"\u002Ftag\u002Fllm\">LLM\u003C\u002Fa>，涵蓋單 CPU 核心到 8 張 \u003Ca href=\"\u002Ftag\u002Fgpu\">GPU\u003C\u002Fa> 的部署形態，並能承受 30 萬以上 QPS、p99 額外延遲低於 10ms，對離開自管堆疊的客戶，基礎設施成本最高可降 90%。這不是小修小補，而是把 serving 從內部負擔變成產品能力。\u003C\u002Fp>\u003Ch2>第一個論點：手動調參無法跟上模型多樣性\u003C\u002Fh2>\u003Cp>問題的核心很簡單：不同模型的行為完全不同。排序模型、embedding 模型、反詐欺模型、LLM 對算力、批次處理與並發的\u003Ca href=\"\u002Fnews\u002Fnvidia-latest-news-ai-demand-rivals-zh\">需求\u003C\u002Fa>都不一樣。Databricks 直接指出，xgboost 這類 CPU-heavy 模型可能每核心只能處理 1 個請求，而 \u003Ca href=\"\u002Ftag\u002Fagent\">agent\u003C\u002Fa> 類模型可達每核心數百請求，微調過的 13B LLM 則明顯受益於 batching。單一 serving 模板根本不可能同時適配這些場景。\u003C\u002Fp>\n\u003Cfigure class=\"my-6\">\u003Cimg src=\"https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1781659067537-savu.png\" alt=\"Databricks 說得對：模型服務應該自適應，不該靠人工調參\" class=\"rounded-xl w-full\" loading=\"lazy\" \u002F>\u003C\u002Ffigure>\n\u003Cp>傳統平台把複雜度丟回客戶端，要求團隊自己調整 replica 數、每 replica concurrency、autoscaling 閾值。這不是抽象化，而是把工作延後。每多一個模型，或流量型態一變，就要重新 profile、重新試錯。結果不是更靈活，而是上線延遲、配置脆弱，最後養出一支只負責救火的 serving 團隊。\u003C\u002Fp>\u003Ch2>第二個論點：生產流量要求系統自動判斷，而不是人盯人\u003C\u002Fh2>\u003Cp>Databricks 的架構方向是對的：request-based 與 resource-based \u003Ca href=\"\u002Fnews\u002Fred-hat-risc-v-rhel-preview-signal-not-product-zh\">訊號\u003C\u002Fa>要一起看。前者能快速反應突發流量，後者能告訴系統 CPU 或 GPU 是否真的已經飽和。單看其中一個都不夠。流量暴增時，利用率指標往往還沒追上；只看利用率，又可能在 p99 latency 爆掉之前還誤以為系統很健康。\u003C\u002Fp>\u003Cp>這在真實業務裡很常見。一次促銷開始，反詐欺 endpoint 可能在幾秒內暴增 10 倍；某個區域\u003Ca href=\"\u002Fnews\u002Fllama-cpp-release-kernel-tuning-over-features-zh\">功能\u003C\u002Fa>上線後，白天尖峰、夜間趨近閒置。這種流量不是靠人工巡檢就能穩定處理。能在 runtime 學會模型上限，自動調整 concurrency 與 replica 的 serving 層，才是同時守住延遲、擴展性與成本的唯一務實做法。\u003C\u002Fp>\u003Ch2>第三個論點：真正的收益是組織效率，不只是技術效率\u003C\u002Fh2>\u003Cp>Databricks 把這件事稱為移除「ML Stack Tax」，這個說法很準。這筆稅不只是浪費的算力，還包括每個模型上線後衍生出的會議、儀表板、調參儀式與事故處理。當 serving 是手動的，組織很快就會把目標從「快速交付價值」改成「先撐住不要壞」。\u003C\u002Fp>\n\u003Cfigure class=\"my-6\">\u003Cimg src=\"https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1781659066764-22j5.png\" alt=\"Databricks 說得對：模型服務應該自適應，不該靠人工調參\" class=\"rounded-xl w-full\" loading=\"lazy\" \u002F>\u003C\u002Ffigure>\n\u003Cp>最有說服力的證據，是很多團隊明明已在 dev 驗證模型，卻因為基礎設施還要再調一次而拖了好幾週才進 production。這不是 ops 細節，而是商業成本。如果 serving 平台能自動匹配 runtime 與模型，自動適應流量，並預設提供可觀測性，工程團隊就能把時間花在更好的模型與產品決策上，而不是維持一個脆弱的 serving 堆疊。\u003C\u002Fp>\u003Ch2>反方可能怎麼說\u003C\u002Fh2>\u003Cp>最強的反對意見是：通用自動化會遮蔽重要取捨。有些團隊跑的是極度敏感的工作負載，對延遲、記憶體壓力或成本上限有明確要求，這些情況需要明確控制。黑盒 autoscaler 會讓系統看起來不夠可預測，尤其當同一平台同時服務小型傳統模型與大型 GPU 模型時，操作員自然會想要更多旋鈕，因為旋鈕代表責任邊界。\u003C\u002Fp>\u003Cp>另一個合理擔憂是平台依賴。若 runtime 選擇或 scaling policy 判斷失準，客戶可能失去針對自身邊緣案例做極致優化的能力。對有深厚 infra 能力的團隊來說，這種控制權流失看起來就是成本。\u003C\u002Fp>\u003Cp>但這個反對意見並沒有推翻 Databricks 的論點，只是劃出邊界：平台必須在預設路徑上強勢，在訊號上透明。Databricks 的說法之所以更強，不是因為它承諾魔法，而是因為它主張系統在 runtime 學習每個模型的極限，結合流量與資源訊號，並保持請求路徑短且隔離。這比要求每個客戶重新踩一次同樣的調參坑，更像一份可交付的工程契約。\u003C\u002Fp>\u003Ch2>你能做什麼\u003C\u002Fh2>\u003Cp>如果你是工程師，別再把 serving 當成一次性部署任務，而要把它當成產品表面來管理，明確定義延遲、成本與可觀測性目標；如果你是 PM 或創辦人，優先選擇能把調參工作移出關鍵路徑的平台，因為每花一小時調 serving 旋鈕，就少一小時能拿來提升模型價值。選擇會自適應、決策透明、預設可觀測的系統，讓團隊把精力放在模型品質，而不是基礎設施救火。","我支持 Databricks 的立場：生產環境的模型服務應該依模型與流量自動調整，而不是靠工程師手動調 replicas、concurrency 和 autoscaling 參數。","www.databricks.com","https:\u002F\u002Fwww.databricks.com\u002Fblog\u002Fai-serving-platform-adapts-your-model",null,"https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1781659067537-savu.png","industry","zh","d204283d-bd6c-4113-b603-a604fe071377",[17,18,19,20,21],"Databricks","模型服務","自適應基礎設施","autoscaling","LLM serving",[23,24,25],"模型服務面對多樣化工作負載時，手動調參不具可擴展性。","結合 request-based 與 resource-based 訊號的自適應 serving，才能同時守住延遲與成本。","真正的價值不只在技術效率，而是把團隊從 ML Stack Tax 中解放出來。",0,"2026-06-17T01:17:21.744217+00:00","2026-06-17T01:17:21.739+00:00","5ec48446-5a5a-4f34-82b2-faec57531d69",{"tags":31,"relatedLang":32,"relatedPosts":36},[],{"id":15,"slug":33,"title":34,"language":35},"databricks-model-serving-adapts-not-tuned-by-hand-en","Databricks is right: model serving should adapt, not be tuned by hand","en",[37,43,49,55,61,67],{"id":38,"slug":39,"title":40,"cover_image":41,"image_url":41,"created_at":42,"category":13},"9e33a02c-97e7-4646-9db7-09fc3ee4bd5a","2-billion-nvidia-coherent-ai-plant-huang-warning-zh","2億美元 Nvidia-Coherent AI 廠擴建，黃仁勳再提 AI 規則…","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1781704974176-q77e.png","2026-06-17T14:02:28.980686+00:00",{"id":44,"slug":45,"title":46,"cover_image":47,"image_url":47,"created_at":48,"category":13},"1f259ae1-4769-4de4-980e-429b719bb889","huang-marvell-ai-thesis-hyperscale-infrastructure-zh","黃仁勳一句話，把 Marvell 從題材變論點","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1781703214058-ndp7.png","2026-06-17T13:33:05.013659+00:00",{"id":50,"slug":51,"title":52,"cover_image":53,"image_url":53,"created_at":54,"category":13},"2a151488-09f9-4aa8-a654-3f1d9d7e159c","china-ai-open-source-efficiency-global-sales-zh","中國 AI 轉向：開源、效率、出海","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1781702271555-i3q3.png","2026-06-17T13:17:25.59471+00:00",{"id":56,"slug":57,"title":58,"cover_image":59,"image_url":59,"created_at":60,"category":13},"4a2fbd38-b5c2-4590-9d4b-87f39f95ab9c","ergo-hestia-pricing-time-to-market-databricks-zh","ERGO Hestia 4 招縮短定價上線","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1781697768906-9krk.png","2026-06-17T12:02:22.440161+00:00",{"id":62,"slug":63,"title":64,"cover_image":65,"image_url":65,"created_at":66,"category":13},"0cf56d85-887b-4fb1-8589-046da6513d26","openai-oracle-universal-credits-enterprise-buying-zh","OpenAI 進 Oracle 企業採購圈","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1781696892976-sx90.png","2026-06-17T11:47:35.092555+00:00",{"id":68,"slug":69,"title":70,"cover_image":71,"image_url":71,"created_at":72,"category":13},"dd3d240a-0f53-49a4-90a5-cac17171f3fd","managed-chatgpt-access-policy-layers-zh","4 層規範決定企業版 ChatGPT 可怎麼用","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1781695973066-pbtw.png","2026-06-17T11:32:17.633521+00:00",[74,79,84,89,94,99,104,109,114,119],{"id":75,"slug":76,"title":77,"created_at":78},"ee073da7-28b3-4752-a319-5a501459fb87","ai-in-2026-what-actually-matters-now-zh","2026 AI 真正重要的事","2026-03-26T07:09:12.008134+00:00",{"id":80,"slug":81,"title":82,"created_at":83},"83bd1795-8548-44c9-9a7e-de50a0923f71","trump-ai-framework-power-speech-state-preemption-zh","川普 AI 框架瞄準電力、言論與州權","2026-03-26T07:12:18.695466+00:00",{"id":85,"slug":86,"title":87,"created_at":88},"ea6be18b-c903-4e54-97b7-5f7447a612e0","nvidia-gtc-2026-big-ai-announcements-zh","NVIDIA GTC 2026 重點拆解","2026-03-26T07:14:26.62638+00:00",{"id":90,"slug":91,"title":92,"created_at":93},"4bcec76f-4c36-4daa-909f-54cd702f7c93","claude-users-spreading-out-and-getting-better-zh","Claude 用戶更分散，也更會用","2026-03-26T07:22:52.325888+00:00",{"id":95,"slug":96,"title":97,"created_at":98},"bd903b15-2473-4178-9789-b7557816e535","openclaw-raises-hard-question-for-ai-models-zh","OpenClaw 逼問 AI 模型價值","2026-03-26T07:24:54.707486+00:00",{"id":100,"slug":101,"title":102,"created_at":103},"eeac6b9e-ad9d-4831-8eec-8bba3f9bca6a","gap-google-gemini-checkout-fashion-search-zh","Gap 把結帳搬進 Gemini","2026-03-26T07:28:23.937768+00:00",{"id":105,"slug":106,"title":107,"created_at":108},"0740e53f-605d-4d57-8601-c10beb126f3c","google-pushes-gemini-transition-to-march-2026-zh","Google 把 Gemini 轉換延到 2026 年 3…","2026-03-26T07:30:12.825269+00:00",{"id":110,"slug":111,"title":112,"created_at":113},"e660d801-2421-4529-8fa9-86b82b066990","metas-llama-4-benchmark-scandal-gets-worse-zh","Meta Llama 4 分數風波又擴大","2026-03-26T07:34:21.156421+00:00",{"id":115,"slug":116,"title":117,"created_at":118},"183f9e7c-e143-40bb-a6d5-67ba84a3a8bc","accenture-mistral-ai-sovereign-enterprise-deal-zh","Accenture 攜手 Mistral AI 賣主權 AI","2026-03-26T07:38:14.818906+00:00",{"id":120,"slug":121,"title":122,"created_at":123},"191d9b1b-768a-478c-978c-dd7431a38149","mistral-ai-faces-its-hardest-year-yet-zh","Mistral AI 迎來最硬的一年","2026-03-26T07:40:23.716374+00:00"]