Back to News

Hacker News Questions Validity of Reported DeepSWE Benchmark Score

406 points · 108 comments#deepSWE#benchmark#hacker news#validity

A Hacker News discussion has cast doubt on the validity of a reported DeepSWE benchmark score, questioning whether the result is accurate. The thread highlights skepticism within the AI research community regarding the benchmark's credibility.

Coverage timeline

  1. Hacker Newsantiuniverse

Related stories

  • DeepSeek global outage hits chat and API for nearly 5 hours; token compensation issued

    On October 15, 2026, DeepSeek's web chat and API services experienced a global outage starting at 8:11 AM Beijing time, affecting DeepSeek V4 Pro API and V4.1 Flash API. The issue was resolved around 12:50 PM, lasting nearly five hours. As compensation, DeepSeek credited tokens to developers based on their API usage over the past 7 days, with some users reporting credits of over 300 yuan.

    2 reports · 2 sources
  • DeepSeek releases V4.1-Flash-1017 open-weight model with faster, cheaper agent tasks

    DeepSeek released V4.1-Flash-1017, an open-weight model checkpoint with 763B parameters, now live on its website, app, and API. The update claims stronger benchmark performance, faster task completion, and shorter outputs than its predecessor, achieved via reinforcement learning with self-feedback for end-to-end project development. DeepSeek also opened its Search & Fetch APIs to everyone, priced at $0.75 per 1000 successful requests, and requires DeepSeek Harness v0.2.1-rc.1 or later for optimal use.

    9 reports · 6 sources955 points · 717 comments
  • SemiAnalysis details Huawei-DeepSeek mutual reliance, Ascend 950PR benchmarks

    SemiAnalysis published a deep dive examining the mutual reliance between Huawei and DeepSeek, where Huawei supplies compute hardware and DeepSeek contributes model workloads and kernel optimization. The report includes Ascend 950PR microbenchmarks, an analysis of DeepSeek V4.1 Flash, and details on DeepSeek's support for Ascend hardware, AICPU, and Tilelang Ascend.

    1 reports · 1 sources
  • DeepSeek open-sources DeepInfer inference stack, reports 82% gross margin

    DeepSeek released its internal inference framework DeepInfer under the Apache 2.0 license, along with the token codec component deepseek-recipe. The framework, which supports DeepSeek-V4.1-Flash and targets Nvidia SM100 GPUs and Huawei Ascend 950 NPUs, reportedly achieves a throughput of 155 trillion tokens per day per cluster, with API revenue of $38 per GPU-hour at off-peak pricing, implying a gross margin of 82% after GPU rental costs. The release sparked discussions about the profitability of AI inference services.

    3 reports · 3 sources2335 points · 890 comments

News updates every hour. Subscribe in your RSS reader to get new stories automatically.

Subscribe