[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"article-videomla-low-rank-kv-cache-video-diffusion-en":3,"article-related-videomla-low-rank-kv-cache-video-diffusion-en":30,"series-research-3a65bf83-79cf-4b24-a099-b102054e1465":76},{"id":4,"slug":5,"title":6,"content":7,"summary":8,"source":9,"source_url":10,"author":11,"image_url":12,"cover_image":12,"category":13,"language":14,"translated_content":11,"related_article_id":15,"keywords":16,"key_takeaways":22,"views":26,"created_at":27,"published_at":28,"topic_cluster_id":29},"3a65bf83-79cf-4b24-a099-b102054e1465","videomla-low-rank-kv-cache-video-diffusion-en","VideoMLA cuts video KV cache memory 92.7%","\u003Cp data-speakable=\"summary\">VideoMLA compresses video diffusion KV caches with a shared low-rank latent and cuts per-\u003Ca href=\"\u002Ftag\u002Ftoken\">token\u003C\u002Fa> memory 92.7%.\u003C\u002Fp>\u003Cul>\u003Cli>\u003Cstrong>Research org\u003C\u002Fstrong>: Unspecified in arXiv abstract\u003C\u002Fli>\u003Cli>\u003Cstrong>Core data\u003C\u002Fstrong>: 92.7% per-token KV memory reduction\u003C\u002Fli>\u003Cli>\u003Cstrong>Breakthrough\u003C\u002Fstrong>: Replaces per-head KV with a shared low-rank latent and decoupled 3D-RoPE key\u003C\u002Fli>\u003C\u002Ful>\u003Cp>Long-rollout video diffusion runs into the same practical wall many streaming models hit: memory and latency grow with the cache you have to keep around. This paper argues that the usual fixed-size sliding-window KV cache is not the only thing worth rethinking. Instead of only changing which tokens sit in the window or how they are positioned, \u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F2605.30351\">VideoMLA: Low-Rank Latent KV Cache for Minute-Scale Autoregressive Video Diffusion\u003C\u002Fa> changes the KV layout itself.\u003C\u002Fp>\u003Cp>That matters for engineers because cache design is not just an implementation detail in long-context video generation. It directly affects how much video you can stream, how fast you can decode, and how expensive the system is to run. The paper’s main claim is simple: if you can shrink the cache without breaking quality, you can push autoregressive video diffusion further into minute-scale generation.\u003C\u002Fp>\u003Ch2>What problem this paper is trying to fix\u003C\u002Fh2>\u003Cp>The authors frame current long-rollout causal video diffusion as centered on a fixed sliding-window \u003Ca href=\"\u002Ftag\u002Fkv-cache\">KV cache\u003C\u002Fa>. Recent work, they say, has mostly optimized within that structure by changing token selection or positional encoding. But the per-head KV layout itself remains a major source of streaming memory use and latency.\u003C\u002Fp>\n\u003Cfigure class=\"my-6\">\u003Cimg src=\"https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1780035485810-fkg5.png\" alt=\"VideoMLA cuts video KV cache memory 92.7%\" class=\"rounded-xl w-full\" loading=\"lazy\" \u002F>\u003C\u002Ffigure>\n\u003Cp>In plain English, the model keeps too much state around for too long. For video generation, that state is especially expensive because you are not just tracking text tokens. You are tracking a sequence of visual tokens across time, and the cache has to support minute-scale rollouts without collapsing quality or speed.\u003C\u002Fp>\u003Cp>So the paper asks a very practical question: can we redesign the cache representation itself, rather than just trimming or reorganizing it?\u003C\u002Fp>\u003Ch2>How VideoMLA works in plain English\u003C\u002Fh2>\u003Cp>VideoMLA introduces Multi-Head Latent Attention, or MLA, to video diffusion. The key move is to replace the usual per-head keys and values with two shared components: a low-rank content latent and a shared decoupled 3D-RoPE positional key.\u003C\u002Fp>\u003Cp>That is a mouthful, but the intuition is straightforward. Instead of storing a separate key and value for every head, the model compresses the content into a shared latent space and handles position with a separate shared key. This reduces the amount of cache state each token needs at every cached layer.\u003C\u002Fp>\u003Cp>The paper reports that this design reduces per-token KV memory by 92.7% at every cached layer. That is the headline engineering win: the cache becomes dramatically smaller without requiring a different rollout paradigm.\u003C\u002Fp>\u003Cp>One important detail is that the authors are not claiming the usual low-rank story from language models carries over cleanly to video. In fact, they explicitly test that assumption and find the opposite of what a simple spectral argument would suggest.\u003C\u002Fp>\u003Ch2>What the paper actually shows\u003C\u002Fh2>\u003Cp>The abstract says pretrained video attention is not low-rank. Its 99%-energy effective rank is far above any practical latent dimension, which means a naive “just approximate the spectrum” approach would predict large reconstruction error. VideoMLA still retains quality at compression ratios where that spectral intuition would expect trouble.\u003C\u002Fp>\n\u003Cfigure class=\"my-6\">\u003Cimg src=\"https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1780035478627-2199.png\" alt=\"VideoMLA cuts video KV cache memory 92.7%\" class=\"rounded-xl w-full\" loading=\"lazy\" \u002F>\u003C\u002Ffigure>\n\u003Cp>That is the interesting part: the method works even though the pretrained spectrum does not look friendly. The authors argue the bottleneck itself, not the pretrained spectrum, determines the effective rank. Their evidence is that both spectral and random initialization occupy nearly the full rank budget from the start, and training preserves that budget while adapting within it.\u003C\u002Fp>\u003Cp>So the paper is not saying “video attention is naturally low-rank.” It is saying the low-rank bottleneck can still be useful because the model learns within the constraint, rather than relying on the original spectrum being compressible in a simple way.\u003C\u002Fp>\u003Cp>On the evaluation side, the abstract gives a few concrete results. On VBench, VideoMLA matches short-horizon streaming video diffusion baselines. At long horizons, it achieves the best overall score among the evaluated methods. The paper also reports 1.23x throughput on a single B200.\u003C\u002Fp>\u003Cp>Those are useful signals, but the abstract does not provide the full \u003Ca href=\"\u002Ftag\u002Fbenchmark\">benchmark\u003C\u002Fa> table, task breakdowns, or the exact quality deltas behind “best overall score.” It also does not spell out the dataset setup beyond VBench, so you should treat the result as promising but still scoped to the reported evaluation.\u003C\u002Fp>\u003Ch2>Why developers should care\u003C\u002Fh2>\u003Cp>If you build video generation systems, this paper points at a lever that is often ignored: the shape of the cache itself. A 92.7% reduction in per-token KV memory is the kind of change that can alter deployment economics, especially when long rollouts are the bottleneck.\u003C\u002Fp>\u003Cp>It also suggests a broader design lesson. For some generative models, the right question is not whether the pretrained representation is already low-rank in the classical sense. The better question may be whether a low-rank bottleneck gives the model enough room to adapt while still saving memory and latency.\u003C\u002Fp>\u003Cp>That said, the paper leaves open a few practical questions. The abstract does not give implementation details for integration into existing stacks, nor does it show how the method behaves across different video domains, resolutions, or rollout lengths beyond the reported long-horizon setting. It also does not provide benchmark numbers in the abstract beyond the relative claims.\u003C\u002Fp>\u003Cp>For practitioners, the take-home is not that every video model should immediately switch to MLA. It is that cache compression can be a first-class systems optimization for video diffusion, and that a shared latent KV design may be a viable path when sliding-window tweaks are not enough.\u003C\u002Fp>\u003Ch2>Bottom line\u003C\u002Fh2>\u003Cp>VideoMLA proposes a concrete cache redesign for autoregressive video diffusion: compress the KV state with a shared low-rank latent, keep position separate, and reclaim a large chunk of memory. The reported gains are strong enough to make the method worth watching, especially if minute-scale generation is part of your roadmap.\u003C\u002Fp>\u003Cp>What it does not prove is equally important: the abstract does not establish broad generality across all video workloads, and it does not replace the need for careful integration work. But for teams fighting streaming memory and latency, this is a very relevant direction.\u003C\u002Fp>\u003Cul>\u003Cli>It attacks the cache layout directly instead of only tuning windowing or position encoding.\u003C\u002Fli>\u003Cli>It reports a 92.7% per-token KV memory reduction and 1.23x throughput on a single B200.\u003C\u002Fli>\u003Cli>It shows low-rank bottlenecks can work even when pretrained video attention is not naturally low-rank.\u003C\u002Fli>\u003C\u002Ful>","VideoMLA compresses video diffusion KV caches with a shared low-rank latent and cuts per-token memory 92.7%.","arxiv.org","https:\u002F\u002Farxiv.org\u002Fabs\u002F2605.30351",null,"https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1780035485810-fkg5.png","research","en","7c996078-6205-4133-b770-261c2c2fb7cb",[17,18,19,20,21],"video diffusion","kv cache","low-rank attention","autoregressive generation","memory optimization",[23,24,25],"VideoMLA compresses video diffusion cache state with a shared low-rank latent and shared positional key.","The paper reports 92.7% lower per-token KV memory and 1.23x throughput on a single B200.","The abstract does not provide full benchmark tables, so the strongest claims are the relative VBench results.",5,"2026-05-29T06:17:31.115044+00:00","2026-05-29T06:17:31.106+00:00","3103988e-c4fe-45e3-98ab-846500c9d507",{"tags":31,"relatedLang":35,"relatedPosts":39},[32],{"name":33,"slug":34},"KV cache","kv-cache",{"id":15,"slug":36,"title":37,"language":38},"videomla-low-rank-kv-cache-video-diffusion-zh","VideoMLA 壓縮影片 KV 快取 92.7%","zh",[40,46,52,58,64,70],{"id":41,"slug":42,"title":43,"cover_image":44,"image_url":44,"created_at":45,"category":13},"154edd47-cb74-4c14-b845-23cd4672b323","vlm-accuracy-visual-cognitive-errors-decade-en","How VLMs Learned Complex Scene Descriptions","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1783926189650-jkdx.png","2026-07-13T07:02:37.258905+00:00",{"id":47,"slug":48,"title":49,"cover_image":50,"image_url":50,"created_at":51,"category":13},"ea29ba1b-1436-4f05-9809-f1108d957877","visual-pretraining-language-models-en","Visual Pretraining Beats Text-Only in Language Models","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1783924380306-981l.png","2026-07-13T06:32:36.107914+00:00",{"id":53,"slug":54,"title":55,"cover_image":56,"image_url":56,"created_at":57,"category":13},"82b70e73-d94e-4a11-9d1c-8bf09e74f798","phinn-eeg-topology-dream-state-eeg-en","PHINN-EEG brings topology to dream-state EEG","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1783922586181-14w4.png","2026-07-13T06:02:35.072233+00:00",{"id":59,"slug":60,"title":61,"cover_image":62,"image_url":62,"created_at":63,"category":13},"abcd6dcf-d3f5-4280-84f7-46439ba1416e","google-android-bench-update-gemini-gap-en","Google’s Android Bench update exposes Gemini’s gap","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1783906370516-crbs.png","2026-07-13T01:32:25.716691+00:00",{"id":65,"slug":66,"title":67,"cover_image":68,"image_url":68,"created_at":69,"category":13},"f15d1c6d-fdb6-4fe0-a671-f0450c038250","llm-benchmarks-not-enough-2026-en","Benchmarks should not pick your LLM in 2026","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1783818163394-1kka.png","2026-07-12T01:02:19.841725+00:00",{"id":71,"slug":72,"title":73,"cover_image":74,"image_url":74,"created_at":75,"category":13},"9386c57c-c41c-4682-afb2-56a661db7f23","rust-breaks-into-tiobe-top-10-en","Rust Breaks Into TIOBE’s Top 10","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1783816366396-apgi.png","2026-07-12T00:32:24.472896+00:00",[77,82,87,92,97,102,107,112,117,122],{"id":78,"slug":79,"title":80,"created_at":81},"a2715e72-1fe8-41b3-abb1-d0cf1f710189","ai-predictions-2026-big-changes-en","AI Predictions for 2026: Brace for Big Changes","2026-03-26T01:25:07.788356+00:00",{"id":83,"slug":84,"title":85,"created_at":86},"8404bd7b-4c2f-4109-9ec4-baf29d88af2b","ml-papers-of-the-week-github-research-desk-en","ML Papers of the Week Turns GitHub Into a Research Desk","2026-03-27T01:11:39.480259+00:00",{"id":88,"slug":89,"title":90,"created_at":91},"87897a94-8065-4464-a016-1f23e89e17cc","ai-ml-conferences-to-watch-in-2026-en","AI\u002FML Conferences to Watch in 2026","2026-03-27T01:51:54.184108+00:00",{"id":93,"slug":94,"title":95,"created_at":96},"6f1987cf-25f3-47a4-b3e6-db0997695be8","openclaw-agents-manipulated-self-sabotage-en","OpenClaw Agents Can Be Manipulated Into Failure","2026-03-28T03:03:18.899465+00:00",{"id":98,"slug":99,"title":100,"created_at":101},"a53571ad-735a-4178-9f93-cb09b699d99c","vega-driving-language-instructions-en","Vega: Driving with Natural Language Instructions","2026-03-28T14:54:04.698882+00:00",{"id":103,"slug":104,"title":105,"created_at":106},"a34581d6-f36e-46da-88bb-582fb3e7425c","personalizing-autonomous-driving-styles-en","Drive My Way: Personalizing Autonomous Driving Styles","2026-03-28T14:54:26.148181+00:00",{"id":108,"slug":109,"title":110,"created_at":111},"2bc1ad7f-26ce-4f02-9885-803b35fd229d","training-knowledge-bases-writeback-rag-en","Training Knowledge Bases with WriteBack-RAG","2026-03-28T14:54:45.643433+00:00",{"id":113,"slug":114,"title":115,"created_at":116},"71adc507-3c54-4605-bbe2-c966acd6187e","packforcing-long-video-generation-en","PackForcing: Efficient Long-Video Generation Method","2026-03-28T14:55:02.646943+00:00",{"id":118,"slug":119,"title":120,"created_at":121},"675942ef-b9ec-4c5f-a997-381250b6eacb","pixelsmile-facial-expression-editing-en","PixelSmile Framework Enhances Facial Expression Editing","2026-03-28T14:55:20.633463+00:00",{"id":123,"slug":124,"title":125,"created_at":126},"6954fa2b-8b66-4839-884b-e46f89fa1bc3","adaptive-block-scaled-data-types-en","IF4: Smarter 4-Bit Quantization That Adapts to Your Data","2026-03-31T06:00:36.65963+00:00"]