DeepSeek open-sources DeepInfer inference stack, reports 82% gross margin
DeepSeek released its internal inference framework DeepInfer under the Apache 2.0 license, along with the token codec component deepseek-recipe. The framework, which supports DeepSeek-V4.1-Flash and targets Nvidia SM100 GPUs and Huawei Ascend 950 NPUs, reportedly achieves a throughput of 155 trillion tokens per day per cluster, with API revenue of $38 per GPU-hour at off-peak pricing, implying a gross margin of 82% after GPU rental costs. The release sparked discussions about the profitability of AI inference services.
Coverage timeline
Hacker Newsggcr
Techmeme
Saritha Rai / Bloomberg : DeepSeek open-sources DeepInfer, its internal production stack designed to accelerate inference, optimize chip performance, and reduce energy and compute costs — DeepSeek is opening up the software it uses to run its artificial intelligence models at scale, potentially making its technology cheaper and easier for businesses to run.

机器之心机器之心
编辑|泽南 没想到这也开源了。 10 月 20 日下午,DeepSeek 开源了内部使用的推理框架 DeepInfer,以及配套的 token 编解码组件 deepseek-recipe。 DeepInfer 是目前支撑 DeepSeek 大规模推理服务的基础,本次以 Apache 2.0 许可证发布,当前支持 DeepSeek-V4.1-Flash。 这次发布迅速引起了关注,昨天大家讨论的都是 DeepSeek 现在让大家都可以快速部署自己的 AI 模型了,今天转念一想,不对,大模型推理怎么这么赚钱啊? DeepSeek 披露,基于这套系统,单个集群每天服务的吞吐量达到 155 万亿 token, 按 API 谷价算,整体收入达到每卡每小时 38 美元 。 这是个什么概念呢?有人算出了毛利率是 82%:假设你卖的 API 使用率够高,并以每张 B300 每小时 7 美元的租金租用显卡,仅扣租金后仍有可观的余量。 Hacker News 上有人直接这么形容:一枚核弹。冷冰冰的数字告诉你 AI 没有泡沫。 DeepInfer 给如今 AI 服务价格的讨论补充了重要的数据,让我们第一次知晓了把模型高效运行起来的工程能力。 对于普通用户来说,调用大模型只是发出请求、等待回答。服务端则需要安排大量请求共享芯片,读取模型参数,调动分布在不同设备上的计算资源,还要控制排队和生成延迟。 大家会发现,模型都是 DeepSeek 的同样型号,但官方 API 和第三方部署的用起来不太一样,这是因为同一个模型,采用不同的推理实现,每张卡能同时服务多少用户、用户需要等多久都可能不同。推理框架解决的正是这类问题,它旨在提高推理效率,让单位算力的有效吞吐,以及用户感受到的响应速度实现最优水平。 这方面应该没有比 DeepSeek 自己做得更好的了。 DeepInfer 主要面向大规模部署,支持英伟达 SM100 GPU 和华为昇腾 950 NPU,整合优化算子、分布式并行、推测解码、异步调度等能力,也支持按 JSON Schema 等规则约束输出。DeepInfer 最初主要基于 vLLM 开发,也参考了 SGLang 和 TensorRT-LLM。 与此同时,deepseek-recipe 负责把不同格式的 API 请求转换成模型使用的提示词和 token 序列,再把输出还原成相应格式的响应

Related stories
DeepSeek opens full-suite app APIs with web search, files, voice, OCR, ASR, TTS
DeepSeek announced the release of a full-suite API covering the capabilities of its consumer app, including web search, deep-thinking search, file input, link reading, safety, voice, OCR, ASR, and TTS. These features are available as standalone interfaces or built-in tools, with V4.1-Flash (deepseek-flash) supporting them across Messages, Responses, and Chat Completions, with server-side tool calls and context assembly. DeepSeek also open-sourced a lightweight chat frontend demo that supports importing conversation history, enabling developers to replicate the full app experience.
2 reports · 2 sources

DSpine parallel speculative decoding achieves up to 4x SGLang throughput, open-sourced
Researchers from Tencent Youtu Lab, Shanghai Jiao Tong University, and Xiamen University released DSpine, a parallel speculative decoding method that injects forming prediction features from predecessor positions into successors at each layer of the draft network, then uses a transition cache for conditional correction. The method achieves up to 4x SGLang throughput and is open-sourced, addressing the suffix decay problem in parallel drafting.
1 reports · 1 sources

DeepSeek agent app Harness hits 1.54M DAUs, consumes ~92T tokens daily
DeepSeek reported that its open-source agent app Harness reached 1.54 million daily active users (DAUs) on November 4, less than three months after its launch. The app consumes approximately 92 trillion tokens per day, translating to about 60 million tokens per user daily, highlighting significant adoption and usage intensity.
1 reports · 1 sources
Llama-modchip runs 284B DeepSeek model on 64GB RAM without GPU
A project called Llama-modchip demonstrates running a 284B-parameter DeepSeek model on a system with 64GB RAM and no GPU, using a modchip-like approach. The technique was discussed on Hacker News, highlighting a potential path for large-model inference on memory-constrained hardware.
1 reports · 1 sources483 points · 207 comments
News updates every hour. Subscribe in your RSS reader to get new stories automatically.
Subscribe