Triton

DashAttention makes sparse long-context attention differentiable

DashAttention uses adaptive sparse selection to keep hierarchical attention differentiable and improve long-context efficiency.