Wisely Chen | AI Agents, On-Prem LLMs & Enterprise AI Architecture

Practical notes on enterprise AI transformation, agentic workflows, and AI security.

文章歸檔

共 44 篇文章

Claude First, Grok Seconds Later, ChatGPT an Hour Behind: The September 3 AI Outage and the Assumption Nobody Checked

On September 3 2026, Claude went down at 9:41 AM ET, Grok followed within minutes, and ChatGPT crashed about an hour later — while Gemini stayed up. The tech community's initial hypotheses pointed to Cloudflare or a traffic cascade, but the real story is simpler and more unsettling: Anthropic leased xAI's entire Memphis Colossus 1 facility in May 2026, and SpaceXAI officially acknowledged a Memphis compute center outage that day. Two supposedly independent providers were sharing one physical failure domain. This article traces the evidence, dismantles the competing hypotheses, and asks the question enterprise CTOs should now be asking: do you actually know where your AI providers' inference runs?

Intelligence Index 66, #1: Fable 5.1 May Be the Closest Thing to AGI You Can Buy

Anthropic launched Fable 5.1 / Mythos 5.1 on September 1, 2026. Intelligence Index 66, #1. Terminal-Bench Science doubled to 52.6%. The real excitement: actual science output — Venus terrain maps at 5x resolution, protein design at 50% hit rate (industry average 10-15%), self-written GPU kernels boosting inference 1.4-2.5x. Cache reads dropped 75%, saving agents 25-45%. But per-task cost rose 20% due to 1.7x output tokens. This article breaks down the capability ledger, the cost ledger, and the strategic ledger — guardrails, 30-day data retention, and the IPO-driven competitive landscape.

NUS Trained a Model Whose Only Job Is Writing Harnesses: JIT-Agent Lets DeepSeek-V4-Flash Beat GPT-5.6 by 9 Points on DeepSearchQA at Half the Token Cost

NUS LV-Lab's JIT-Agent paper pushes Harness Engineering from craft to science: they trained a 27B model (based on Qwen3.6) to generate task-adaptive agent harnesses on the fly using a four-module protocol (Memory / Planning / Capability Orchestration / Action). Results: DeepSeek-V4-Flash with a JIT-Agent harness scores 85.1 on DeepSearchQA vs GPT-5.6's 76.0 (+9.1); GLM-5.2 gains up to +20.2 points; across nine benchmarks, JIT-Agent matches mature hand-crafted harnesses like Claude Code and OpenCode while using 50% fewer tokens and 36% less cost. This is the first time someone has treated harness-writing ability as an independent, trainable scaling dimension — orthogonal to model scaling.

Barclays Dissected the AI Profit Chain: For Every $100, Cloud Takes $35-40 and Inference Margins Hit 65%

Barclays' August 28 research note — 'A Primer on AI Lab & AI Hyperscaler Unit Economics' — traces a complete AI profit chain: for every $100 an AI lab earns, $35-40 flows to the Big Three clouds, which pocket $10-20 in operating profit at 34-47% margins. The real surprise is on the model side — paid inference margins jumped from low-teens in 2025 to 48-65% in 2026, with API inference exceeding 80%. But add training costs back, and most labs are barely breaking even or still losing money. Goldman Sachs' July report on Chinese AI labs shows EBIT of -30% to -39%. This piece lays both ledgers side by side.

One URL, Zero Context — Two Hours of Black-Box Red Teaming with an Uncensored AI

A friend's website had been white-box scanned 3-4 times with Claude Code, rescanned with Codex, and a paid security firm was already signed. Before the real audit, he wanted a dress rehearsal with an uncensored AI. Using an RTX 5090 + Qwen3.8-27B huihui (abliterated), given nothing but a URL, I ran a pure black-box red team. Architecture mapped in 3 minutes, first 2 findings in 30 minutes, 10 issues and 2 PoCs in two hours.

Sovereign AI Isn't a Slogan: Sequoia's Four-Tier Ladder, the Open 60 vs Closed 62 Math, and Where You Should Stop

At a Sequoia event in August, Sonya Huang translated the Bitcoin world's 2017 slogan into the AI era: not your weights, not your product. The slogan rests on exactly one hard number — Kimi K3 and GLM-5.3 score 60 on the Artificial Analysis Intelligence Index, Fable 5 scores 62. Starting from that 2-point gap, this piece works out the real cost structure of building your own, and the positional interest a VC has in pushing this thesis.

Fable 5 Is Only 11% of Anthropic's Sales: When the Strongest Model Isn't the Best Seller, It's Time to Calculate Intelligence per Dollar

Ramp's payment data across 70,000 companies shows Fable 5, more than two months after launch, captures only 11.4% of Anthropic model spending and 6% of token usage. This piece breaks down the three forces pushing enterprises away from Fable 5, proposes two new metrics — intelligence per dollar and intelligence per billion parameters — and uses the Artificial Analysis Intelligence Index to compare intelligence density across cloud APIs and open-weight models. The conclusion: the frontier moat is down to two or three points on the Intelligence Index, and the next phase of competition is about delivering sufficient intelligence at the lowest cost.

YouTube Transcript: Two Days with Qwen3.8-27B — RTX 5090 SGLang / vLLM / llama.cpp Benchmarks

Qwen3.8-27B went open weights two days ago with SWE-bench Pro 61.7, beating Opus 4.6 Max's 53.4. Those are Alibaba's numbers — these are mine. I put it on a 32GB RTX 5090 and benchmarked SGLang and vLLM to answer one practical question: how many people can a single consumer card actually serve? Answer: four, at 425 tok/s total throughput, roughly 106 tok/s each. MTP is a free lunch — vLLM with MTP-2 is 73.5% faster single-user — but the same MTP head only gains 20% on the llama.cpp route. Full launch parameters included.

Qwen3.8-27B Open Weights: SWE-bench Pro 61.7 Beats Opus 4.6 Max — 'Just Use On-Prem' Is No Longer a Compromise

On 8/14, Qwen3.8-27B dropped as open weights — Apache 2.0, 27B dense, native multimodal, 262K context. Official benchmarks: SWE-bench Pro 61.7 vs Claude Opus 4.6 Max's 53.4, OSWorld 84.3 vs 72.7, AndroidWorld 81.9 vs 62.0 — wins are all agentic workflow tasks, losses are all knowledge-ceiling tasks. This post breaks down what that win/loss distribution means for enterprise on-prem deployment: the 'capability gap' argument against on-prem just lost its teeth.

Qwen 3.6-27B Local Deployment: Sonnet 4.6-Class AI Agent Running on a DGX Spark / Mac mini

Qwen 3.6-27B, an open-source dense model, hits 136 tokens/sec on the $4,699 NVIDIA DGX Spark — beating Claude Opus 4.5 on benchmarks and edging out Sonnet 4.6 on Terminal-Bench. This post walks IT architects through hardware options for local Qwen 3.6-27B deployment (DGX Spark vs Mac mini M4 Pro 64GB), 12 official benchmarks, the Dflash + DDTree inference stack, a 3-year TCO comparison ($22,500 vs $4,729 per developer), and the architectural rewrites this triggers for on-prem AI Agent setups.

"Opus Is Too Smart, So It Shouldn't Be Doing the Planning" — A Paper That Flips the Agent Ops Paradigm

Columbia's AgentOpt paper ran 9 models in 81 combinations and proved it: Ministral 8B as Planner + Opus as Solver hits 74.27% accuracy, while Opus as Planner sits at 31.71%. Putting the most expensive model in the Planner seat is the worst move — because it's so strong it skips the tools and answers raw. Anthropic's own Advisor Tool is course-correcting in the same direction: cheap models run the main loop, Opus steps back as on-call advisor. The unit of agent pipeline optimization isn't single-model capability — it's how the model combo fits the specific task.

Cracking the Cache: From Gemma4 to Claude Code, Save 80% on Tokens

Open Claude Code in the morning, type one sentence — and 2–10% of your monthly quota is gone. I ran an experiment locally with Gemma4 and watched prompt processing drop from 31 seconds to 0.25 seconds — a 100x speedup. Then I dug into the Claude Code source and unpacked Anthropic's multi-layer cache architecture: DYNAMIC_BOUNDARY splitting, two-tier TTL, cache-break detection. From the KV cache fundamentals in Transformers, to the MLSys 2024 Prompt Cache paper, to the daily money-saving habits — once you understand the mechanism, the same plan can stretch 3–5x further.

Harness Engineering Architecture: AI Can Write Code, But It Can't Ship to Production on Its Own

Amazon let AI fix a bug; AI deleted the entire production environment. DataTalks.Club: AI wiped the whole database. An e-commerce team lost millions of orders to an AI change. Three incidents, one pattern: reset → rebuild → clean state. This post lays out the full picture of Harness Engineering in one architecture diagram — from Amazon's ban to OpenAI's Control Plane, from three-layer defense to the seven-component reference architecture, from five failure modes to three things your team can start tomorrow.

Harness Engineering Fully Decoded: When AI Agents Finish Writing Code, Is Your Repo Ready to Catch It Automatically?

3 OpenAI engineers shipped 1 million lines of code in 5 months using Codex—0 lines written by humans. They call this Harness Engineering—not the engineering of writing code, but the engineering of building constraints and feedback loops. Inspired by this, Ryan Carson published a complete Control-Plane Pattern: risk tier contract, preflight gate, SHA discipline, remediation loop. Last time we talked four layers of defense. This time we look at how a full control plane catches Agent output at speed.

OpenClaw's Five Ways to Browse the Web: From Search API to Taking Over Your Browser

AI agent 'browsing the web' isn't one thing — it's five. Pick the wrong mode and you're either missing capability or handing your accounts to an AI. OpenClaw's five web access architectures — Search API, Web Fetch, Managed Browser, Remote CDP, Extension Relay — each have wildly different capability ranges, security risks, and appropriate use cases. This post breaks down every layer: from the safest search APIs to the most dangerous full browser takeover, including the Accessibility Tree vs screenshot efficiency gap, the manual-login sweet spot for Managed Browser, and WebMCP's future potential.

Token Export: China's AI Is No Longer Selling Products—It's Selling Tokens

China's AI export is undergoing a qualitative shift—from selling products to selling Tokens. In February 2026, Chinese models (MiniMax, Kimi, GLM) overtook the US in production-grade Token call volume for the first time. GLM-5 walked away clean from distillation accusations, beat GPT-5.2 on SWE-bench, was trained entirely on Huawei chips, and its API is 5-8x cheaper. Stack together China's electricity price ($0.08/kWh vs US $0.18/kWh), open-source talent density, and hardware autonomy, and you see a new trade paradigm forming: exporting SOTA-90% reasoning capability to the world in a metered, priceable way. This isn't a story about the tech race. It's a story about cost structure.

The Channel War: OpenClaw, Anthropic, and Who Gets to Decide the Future of AI Agents

OpenClaw v2.19 shipped an Apple Watch MVP. Anthropic blocked OAuth to shut out third-party subscribers. Sam Altman recruited Peter Steinberger and embraced open source. Put these three things together and you see something beyond a technical competition — you see the most brutal reality of the AI industry: whoever controls the channel decides the model's fate.

OpenClaw Week: From the Claude Code 1.5 Era to a Digital Jarvis | Weekly Vlog EP8

A week-long deep dive into OpenClaw — from creator Peter's builder philosophy to the Memory architecture (AGENTS.md, SOUL.md), three token-saving tricks (cut 50%+ easily), and the 'new employee' enterprise security strategy. The AI agent that comes closest to a real digital Jarvis. Worth your time to understand it properly.

OpenClaw Token Optimization Guide: How to Cut AI Agent Operating Cost by 97%

Real intelligence isn’t paying for the most expensive model—it’s careful prompt and system design. This post shares five core optimization strategies—session initialization, model routing, local heartbeats, prompt caching, and rate limiting—shown in practice to reduce OpenClaw cost from ~$1,500/month to under $50.

Moltbot Security Hardening in Practice: A Complete Four-Layer Defense-in-Depth Guide for AI Agents

You don’t need to be a security expert—just be willing to spend an afternoon reading the docs carefully. This post distills Moltbot community battle-tested experience into a four-layer defense-in-depth playbook: Isolation, Quarantine, Rollback, and Transparency. It covers AI Agent Security, Prompt Injection Defense, LLM Agent Security, and an end-to-end Agentic Security framework.

When Unix Philosophy Meets AI: The Command Line Renaissance

When I was a kid I read a book called Unix Power Tools. There was a line I remembered for almost twenty years: ‘Command line pipeline is the best UI interface in the world.’ Back then I had no idea what it meant. But after Claude Code burst onto the scene in April 2025, I finally understood: a brain that understands the world through text plugged into an interface that exposes the world’s state through text. This isn’t retro—it’s structurally the most reasonable choice.

CaMeL: Google DeepMind’s Prompt-Injection Defense Architecture

Simon Willison called this ‘the first credible prompt injection defense’ he’s seen. CaMeL’s core design splits one agent into two: a low-privilege agent that reads external data, and a high-privilege agent that makes decisions—so ‘reading data’ and ‘taking actions’ are always separated.

← 返回首頁