Xiaohongshu DoSA sparse attention cuts TTFT by up to 48% via score bounds
Xiaohongshu's Dots team introduced DoSA (Dots Sparse Attention), a sparse-attention method that accelerates the indexer using score bounds while preserving exact Top-K selection results. In tests at ~1M-token context lengths, it reduced first-token latency by about 37%, 46%, and 48% for Dots3-Note-preview, GLM-5.2, and DeepSeek-V3.2 respectively, compared to their SGLang baselines, and filtered roughly 97% of candidates early.
Coverage timeline
机器之心机器之心
稀疏注意力减少了主注意力的计算量，但负责筛选 token 的索引器仍可能成为长上下文推理的瓶颈。小红书 Dots 团队自研的 DoSA 利用分数上下界，在保持原始 Top-K 选择结果的前提下加速索引器，并在测试中提前过滤了约 97% 的候选。在约 1M token 的上下文长度下，Dots3-Note-preview、GLM-5.2 和 DeepSeek-V3.2 的首 token 延迟较各自的 SGLang 基线分别降低约 37%、46% 和 48%。 论文标题：DoSA: Dots Sparse Attention 作者单位：Dots Studio 项目地址：https://github.com/llm-sys/dosa-sglang 01 注意力稀疏了，为什么还是慢？ 大模型从发出请求到收到第一个 token 的时间称为首 token 延迟（TTFT），也就是用户感受到的「首字等待时间」。上下文越长，预填充通常越耗时。 DeepSeek-V3.2 采用的 DSA 减轻了主注意力的计算负担：先由索引器（Indexer）为候选 token 打分，选出分数最高的 K 个，再由主注意力对选中的内容进行计算。然而，索引器本身成为了主要瓶颈。在约 1M token 的上下文长度下，Dots3-Note-preview、GLM-5.2 和 DeepSeek-V3.2 的 Indexer 耗时分别占预填充时间的 81.6%、73.0% 和 90.8%。 图 1｜左：Indexer 耗时占预填充 GPU 时间的比例；右：Indexer 的理论矩阵运算量占比。 02 同样是加速 Indexer，DoSA 有何不同？ 表 1｜不同的基模团队索引器加速路线。 层间复用、减少索引头和按块共享等方法，通过减少独立打分次数或改变选择粒度来节省计算，但不保证与原始逐层、逐查询、使用全部索引头打分的 Top-K 选择结果等价。 DoSA 的关键区别在于：保留原始评分函数，并用数学上下界证明哪些计算可以省略。少量索引头仅用于构造保守上界，不会替代最终评分。只有被证明不可能进入 Top-K 的候选才会被排除，其余候选仍使用全部索引头进行精确打分。 03 先判断能否入选，再决定是否精算 理解 DoSA，只需抓住两个量：Top-K 入选的 阈值下界 ，以及每个候选的 分数上界 。如果某个候选的得分上
Related stories
OpenAI Codex lead Tibo pledges 28-day daily improvement or quota reset; first day boosts GPT-6 inference speed ~50%
OpenAI Codex lead Tibo announced a 28-day commitment to deliver either a meaningful improvement for Codex/Work users or a full quota reset each day, following user complaints about slow models and reduced quotas after DevDay. On the first day, OpenAI boosted default inference speed for GPT-6 Astra and GPT-6.1 Sol by ~50% to 50 TPS across subscriptions and partner products, while also introducing text watermarking. The move comes amid a user trust crisis and a SemiAnalysis report claiming Anthropic subscriptions offer 5x more value than OpenAI's.
4 reports · 2 sources
OpenAI launches GPT-6.1 Sol Ultrafast, 8x faster than standard
OpenAI released GPT-6.1 Sol Ultrafast, a faster version of its frontier model, claiming near-Astra intelligence at up to 8x the speed of the standard Sol. API pricing is $12 per million input tokens and $60 per million output tokens, six times the standard rate. The model is available to all users following the previous day's rollout of GPT-6 to free and paid tiers.
2 reports · 2 sources
OpenAI launches Dots, always-on GPT-6 Astra agents with cloud computers
At DevDay 2026, OpenAI unveiled Dots, always-on AI agents powered by GPT-6 Astra that run on their own cloud computer, accessing a web browser and over 4,000 apps. Available to Pro, Business Premium, and Enterprise users, Dots can be messaged via ChatGPT, Slack, and Microsoft Teams, with a waitlist for iMessage and RCS. OpenAI also announced ChatGPT Space, a collaborative workspace, and expanded Sign in with ChatGPT to 16 launch partners.
10 reports · 4 sources764 points · 641 comments
Tsinghua and Qujing Introduce KTransformers-FineTune for CPU-GPU MoE Fine-Tuning
Tsinghua University's MADSys lab and Qujing Technology, with LlamaFactory, introduced KTransformers-FineTune (KT-FT), a CPU-GPU co-computation method for fine-tuning mixture-of-experts (MoE) models. It claims 2.5-21.2x throughput gains over traditional offloading by having the CPU compute expert weights directly instead of transferring them to the GPU. The approach includes a semantic-aware proactive layout engine and dynamic thread scheduling for routing skew, and is fully open-sourced.
News updates every hour. Subscribe in your RSS reader to get new stories automatically.
Subscribe