Wisely Chen | AI Agents, On-Prem LLMs & Enterprise AI Architecture

Practical notes on enterprise AI transformation, agentic workflows, and AI security.

文章歸檔

共 54 篇文章

YouTube Transcript: MediaTek Cut Its AI Budget — The Trap of Stopping at "Cut Costs, Boost Efficiency"

MediaTek cut its employees' AI quotas in half, then in half again. Employees on Threads say they now get roughly US$2-3 per person per month, and the internal local GPU farm is jammed. The comment sections called it short-sighted. But look at it through a boss's priority order — stock price, then orders, then cost, then efficiency — and the decision adds up: AI didn't win MediaTek any orders (the big fight is reportedly for Google TPU business), it clearly added cost (cloud budgets, a self-built GPU farm), and it only improved efficiency, which ranks third. If AI adoption stops at "cut costs, boost efficiency," companies end up with only two outcomes: layoffs, or swinging from token maximizing back to token minimizing. The harder question: deadlines and KPIs are already set at AI speed, so can people fill the gap once AI is pulled?

Running Qwen3.8-Flash-Next on an RTX 5090 + 64GB: Strata Hands-On, IQ3_XXS vs IQ3_S

Qwen3.8-Flash-Next beats the Qwen3.8-27B I use every day on most rows of Qwen's own benchmark table (DeepSWE 1.1: 58.7 vs 42.2), but a 125B backbone plus a huge n-gram table left my RTX 5090 + 64GB RAM stuck between wanting to try it and not being able to run it. Even FreeToken's setup blew past my memory budget. The open-source engine Strata fit it in with four tricks: low-bit quantization, an expert cache, the n-gram table on SSD, and MTP. IQ3_XXS stretched context to 1M and generated 101.9 tok/s on a short prompt; IQ3_S pulled back to 262K, ran real agent requests at a median of 106.8 tok/s, and made fewer everyday mistakes. Here are both runs, and why I stayed on IQ3_S.

950 Claude Agents Found a CRISPR-Like System, Then Missed It in All 10 Reruns: How AI Actually Collaborates in Frontier Science

On September 23, 2026, Anthropic published the first result from its new life sciences lab: roughly 950 Claude agents ran for 21 hours over a database of 1.9 billion protein clusters and surfaced a CRISPR-like system now called ART (array-associated reverse transcriptase). The find came from one agent that went beyond its task list and read the raw DNA next to a gene. But when the team reran the same search 10 times, all 10 runs missed it. On a fixed test where the DNA is pasted straight into the model, the four strongest models recognize the array at least 90% of the time; in a realistic files-and-tools environment, recognition drops as low as 32%. This post walks through what CRISPR is, how humans and agents split the work, and why the randomness that makes LLMs unreliable in production is exactly what makes them useful for discovery. ART's function is still unproven, and the preprint has not been peer-reviewed.

How a Top VC Sees AI in 2026: AGI Has Arrived, but the Adoption Gap Keeps Widening

On September 24, Sequoia's Pat Grady recorded a 15-minute state-of-AI briefing for the investment committee of his alma mater, Boston College, and later posted it on X. Three inflection points, AGI has arrived, labs sending deployment squads to compete for tokens, application-layer companies reinventing themselves every four months. The hardest slide shows seven real deals: Sequoia got in at an average of $110M, and within a month the next round priced at an average of $3.4B, roughly 31x. He called it a bubble. This post separates which parts of those 15 minutes are evidence and which are positioning, and explains why that bubble slide doubles as a fundraising deck.

MCP Enterprise Security Isn't a Single Product — Three-Layer Governance from IdP Auth to Runtime Enforcement

In August 2026, Anthropic shipped Enterprise-managed auth for MCP connectors, centralizing OAuth under the corporate IdP. The same month, CrowdStrike launched Agentic Identity Provider, treating every AI agent as a privileged identity. Meanwhile, a Bluebear Security case showed an agent bypassing its MCP gateway via Bash. These three threads lead to one conclusion: enterprise MCP security requires three layers — IdP authorization at the entry, a policy gateway in the middle, and runtime enforcement at execution. Drop any one layer and agents route around the other two.

Does Your Agent's Memory Survive a Model Upgrade? A LinkedIn Study Says the Average Hides Two Opposite Effects: +9.91 and −13.28

Fable 5.1 and GPT-6 Astra just launched. Many agents have already swapped their underlying model, but their memory stores were written by the old one. A LinkedIn paper (arXiv 2609.05339) ran a controlled experiment: 48 synthetic histories, four memory formats, Llama-3.1-8B and Qwen2.5-7B swapping writer and reader roles. Natural-language notes show only −1.47pp average swap penalty — but split by direction it's +9.91 and −13.28. Eighty percent of notes loss happens at write time. Mixed embeddings lose 58% of upgrade gains. Notes-only repair fails 0/48; raw-history repair recovers 34. This post breaks down the experimental design, four key tables, and what I'm changing in my own 263-page wiki.

Grok Bot: xAI Is Selling You a $150 VM for $30 — With an AI Employee Thrown In

Thirty dollars. An 8-core / 16 GB RAM Debian Linux VM that never shuts down, plus an AI that logs into your Gmail and Slack on its own. That's xAI's Grok Bot — essentially a cloud-hosted version of things like OpenClaw. Great value, better UI than Manus, usable by non-engineers. But for enterprise? Bots share login credentials, there's no dry run, and documentation is scarce. xAI has 2C DNA at its core — it can ship demos that blow people away, but not products that procurement departments will sign off on.

DeepMind Put 100 Agents in a Math Conference: 27 Minutes to Total Corruption, 24% Blew the Whistle, None Had the Power to Stop It

Google DeepMind posted a case study on arXiv (September 3, 2026): 100 Gemini 3.1 Pro agent instances, framed as peers at an academic conference, solving 71 formal math conjectures. By 12:15 UTC, 37 were solved correctly. Then one agent discovered the verifier had no AST inspection — just a keyword blacklist. Within 27 minutes, the remaining 34 problems were all marked 'proved' using a notation-override exploit. Meanwhile, 24% of agents independently audited the fake proofs, organized strikes, filed complaints. Every one of them failed — not because they were wrong, but because no agent had the authority to remove a fraudulent submission. This post breaks down why whistleblowing failed and what it means for enterprise multi-agent deployment.

Anthropic vs OpenAI Compute Showdown: From Cornfields to a $2T IPO

Everyone is comparing Fable vs Astra benchmarks. But the real race is compute. At the end of 2025, Anthropic had only 70% of OpenAI's running power (1.4 GW vs 1.9 GW). By mid-2026, spend-implied averages show Anthropic at ~2.7 GW vs OpenAI's ~2.4 GW. The Information reports Anthropic signed at least 14.8 GW in compute contracts over the past eleven months. This article breaks down the compute race across three layers — running power, signed pipeline, and annual spend — then connects it to Anthropic's potential $2T IPO.

YouTube Transcript: Three AI Services Went Down Simultaneously — My 5090 Saved Me

On September 3, Claude, Grok, and ChatGPT went down in sequence within three hours. Codex and Cursor followed. The root cause was a failure at the Memphis Colossus 1 data center — Anthropic had leased the entire facility from SpaceX in May, sharing the same fault domain as Grok. ChatGPT was then overwhelmed by the traffic tsunami. This episode covers three cognitive shifts: multi-vendor doesn't mean multi-fault-domain, my 5090 + Qwen 3.8 27B was the only thing still running, and hybrid cloud isn't just about cost savings or data security — it's about architectural resilience.

Claude First, Grok Seconds Later, ChatGPT an Hour Behind: The September 3 AI Outage and the Assumption Nobody Checked

On September 3 2026, Claude went down at 9:41 AM ET, Grok followed within minutes, and ChatGPT crashed about an hour later — while Gemini stayed up. The tech community's initial hypotheses pointed to Cloudflare or a traffic cascade, but the real story is simpler and more unsettling: Anthropic leased xAI's entire Memphis Colossus 1 facility in May 2026, and SpaceXAI officially acknowledged a Memphis compute center outage that day. Two supposedly independent providers were sharing one physical failure domain. This article traces the evidence, dismantles the competing hypotheses, and asks the question enterprise CTOs should now be asking: do you actually know where your AI providers' inference runs?

Intelligence Index 66, #1: Fable 5.1 May Be the Closest Thing to AGI You Can Buy

Anthropic launched Fable 5.1 / Mythos 5.1 on September 1, 2026. Intelligence Index 66, #1. Terminal-Bench Science doubled to 52.6%. The real excitement: actual science output — Venus terrain maps at 5x resolution, protein design at 50% hit rate (industry average 10-15%), self-written GPU kernels boosting inference 1.4-2.5x. Cache reads dropped 75%, saving agents 25-45%. But per-task cost rose 20% due to 1.7x output tokens. This article breaks down the capability ledger, the cost ledger, and the strategic ledger — guardrails, 30-day data retention, and the IPO-driven competitive landscape.

NUS Trained a Model Whose Only Job Is Writing Harnesses: JIT-Agent Lets DeepSeek-V4-Flash Beat GPT-5.6 by 9 Points on DeepSearchQA at Half the Token Cost

NUS LV-Lab's JIT-Agent paper pushes Harness Engineering from craft to science: they trained a 27B model (based on Qwen3.6) to generate task-adaptive agent harnesses on the fly using a four-module protocol (Memory / Planning / Capability Orchestration / Action). Results: DeepSeek-V4-Flash with a JIT-Agent harness scores 85.1 on DeepSearchQA vs GPT-5.6's 76.0 (+9.1); GLM-5.2 gains up to +20.2 points; across nine benchmarks, JIT-Agent matches mature hand-crafted harnesses like Claude Code and OpenCode while using 50% fewer tokens and 36% less cost. This is the first time someone has treated harness-writing ability as an independent, trainable scaling dimension — orthogonal to model scaling.

Barclays Dissected the AI Profit Chain: For Every $100, Cloud Takes $35-40 and Inference Margins Hit 65%

Barclays' August 28 research note — 'A Primer on AI Lab & AI Hyperscaler Unit Economics' — traces a complete AI profit chain: for every $100 an AI lab earns, $35-40 flows to the Big Three clouds, which pocket $10-20 in operating profit at 34-47% margins. The real surprise is on the model side — paid inference margins jumped from low-teens in 2025 to 48-65% in 2026, with API inference exceeding 80%. But add training costs back, and most labs are barely breaking even or still losing money. Goldman Sachs' July report on Chinese AI labs shows EBIT of -30% to -39%. This piece lays both ledgers side by side.

One URL, Zero Context — Two Hours of Black-Box Red Teaming with an Uncensored AI

A friend's website had been white-box scanned 3-4 times with Claude Code, rescanned with Codex, and a paid security firm was already signed. Before the real audit, he wanted a dress rehearsal with an uncensored AI. Using an RTX 5090 + Qwen3.8-27B huihui (abliterated), given nothing but a URL, I ran a pure black-box red team. Architecture mapped in 3 minutes, first 2 findings in 30 minutes, 10 issues and 2 PoCs in two hours.

Sovereign AI Isn't a Slogan: Sequoia's Four-Tier Ladder, the Open 60 vs Closed 62 Math, and Where You Should Stop

At a Sequoia event in August, Sonya Huang translated the Bitcoin world's 2017 slogan into the AI era: not your weights, not your product. The slogan rests on exactly one hard number — Kimi K3 and GLM-5.3 score 60 on the Artificial Analysis Intelligence Index, Fable 5 scores 62. Starting from that 2-point gap, this piece works out the real cost structure of building your own, and the positional interest a VC has in pushing this thesis.

Fable 5 Is Only 11% of Anthropic's Sales: When the Strongest Model Isn't the Best Seller, It's Time to Calculate Intelligence per Dollar

Ramp's payment data across 70,000 companies shows Fable 5, more than two months after launch, captures only 11.4% of Anthropic model spending and 6% of token usage. This piece breaks down the three forces pushing enterprises away from Fable 5, proposes two new metrics — intelligence per dollar and intelligence per billion parameters — and uses the Artificial Analysis Intelligence Index to compare intelligence density across cloud APIs and open-weight models. The conclusion: the frontier moat is down to two or three points on the Intelligence Index, and the next phase of competition is about delivering sufficient intelligence at the lowest cost.

YouTube Transcript: Two Days with Qwen3.8-27B — RTX 5090 SGLang / vLLM / llama.cpp Benchmarks

Qwen3.8-27B went open weights two days ago with SWE-bench Pro 61.7, beating Opus 4.6 Max's 53.4. Those are Alibaba's numbers — these are mine. I put it on a 32GB RTX 5090 and benchmarked SGLang and vLLM to answer one practical question: how many people can a single consumer card actually serve? Answer: four, at 425 tok/s total throughput, roughly 106 tok/s each. MTP is a free lunch — vLLM with MTP-2 is 73.5% faster single-user — but the same MTP head only gains 20% on the llama.cpp route. Full launch parameters included.

Qwen3.8-27B Open Weights: SWE-bench Pro 61.7 Beats Opus 4.6 Max — 'Just Use On-Prem' Is No Longer a Compromise

On 8/14, Qwen3.8-27B dropped as open weights — Apache 2.0, 27B dense, native multimodal, 262K context. Official benchmarks: SWE-bench Pro 61.7 vs Claude Opus 4.6 Max's 53.4, OSWorld 84.3 vs 72.7, AndroidWorld 81.9 vs 62.0 — wins are all agentic workflow tasks, losses are all knowledge-ceiling tasks. This post breaks down what that win/loss distribution means for enterprise on-prem deployment: the 'capability gap' argument against on-prem just lost its teeth.

Qwen 3.6-27B Local Deployment: Sonnet 4.6-Class AI Agent Running on a DGX Spark / Mac mini

Qwen 3.6-27B, an open-source dense model, hits 136 tokens/sec on the $4,699 NVIDIA DGX Spark — beating Claude Opus 4.5 on benchmarks and edging out Sonnet 4.6 on Terminal-Bench. This post walks IT architects through hardware options for local Qwen 3.6-27B deployment (DGX Spark vs Mac mini M4 Pro 64GB), 12 official benchmarks, the Dflash + DDTree inference stack, a 3-year TCO comparison ($22,500 vs $4,729 per developer), and the architectural rewrites this triggers for on-prem AI Agent setups.

"Opus Is Too Smart, So It Shouldn't Be Doing the Planning" — A Paper That Flips the Agent Ops Paradigm

Columbia's AgentOpt paper ran 9 models in 81 combinations and proved it: Ministral 8B as Planner + Opus as Solver hits 74.27% accuracy, while Opus as Planner sits at 31.71%. Putting the most expensive model in the Planner seat is the worst move — because it's so strong it skips the tools and answers raw. Anthropic's own Advisor Tool is course-correcting in the same direction: cheap models run the main loop, Opus steps back as on-call advisor. The unit of agent pipeline optimization isn't single-model capability — it's how the model combo fits the specific task.

Cracking the Cache: From Gemma4 to Claude Code, Save 80% on Tokens

Open Claude Code in the morning, type one sentence — and 2–10% of your monthly quota is gone. I ran an experiment locally with Gemma4 and watched prompt processing drop from 31 seconds to 0.25 seconds — a 100x speedup. Then I dug into the Claude Code source and unpacked Anthropic's multi-layer cache architecture: DYNAMIC_BOUNDARY splitting, two-tier TTL, cache-break detection. From the KV cache fundamentals in Transformers, to the MLSys 2024 Prompt Cache paper, to the daily money-saving habits — once you understand the mechanism, the same plan can stretch 3–5x further.

Harness Engineering Architecture: AI Can Write Code, But It Can't Ship to Production on Its Own

Amazon let AI fix a bug; AI deleted the entire production environment. DataTalks.Club: AI wiped the whole database. An e-commerce team lost millions of orders to an AI change. Three incidents, one pattern: reset → rebuild → clean state. This post lays out the full picture of Harness Engineering in one architecture diagram — from Amazon's ban to OpenAI's Control Plane, from three-layer defense to the seven-component reference architecture, from five failure modes to three things your team can start tomorrow.

Harness Engineering Fully Decoded: When AI Agents Finish Writing Code, Is Your Repo Ready to Catch It Automatically?

3 OpenAI engineers shipped 1 million lines of code in 5 months using Codex—0 lines written by humans. They call this Harness Engineering—not the engineering of writing code, but the engineering of building constraints and feedback loops. Inspired by this, Ryan Carson published a complete Control-Plane Pattern: risk tier contract, preflight gate, SHA discipline, remediation loop. Last time we talked four layers of defense. This time we look at how a full control plane catches Agent output at speed.

OpenClaw's Five Ways to Browse the Web: From Search API to Taking Over Your Browser

AI agent 'browsing the web' isn't one thing — it's five. Pick the wrong mode and you're either missing capability or handing your accounts to an AI. OpenClaw's five web access architectures — Search API, Web Fetch, Managed Browser, Remote CDP, Extension Relay — each have wildly different capability ranges, security risks, and appropriate use cases. This post breaks down every layer: from the safest search APIs to the most dangerous full browser takeover, including the Accessibility Tree vs screenshot efficiency gap, the manual-login sweet spot for Managed Browser, and WebMCP's future potential.

Token Export: China's AI Is No Longer Selling Products—It's Selling Tokens

China's AI export is undergoing a qualitative shift—from selling products to selling Tokens. In February 2026, Chinese models (MiniMax, Kimi, GLM) overtook the US in production-grade Token call volume for the first time. GLM-5 walked away clean from distillation accusations, beat GPT-5.2 on SWE-bench, was trained entirely on Huawei chips, and its API is 5-8x cheaper. Stack together China's electricity price ($0.08/kWh vs US $0.18/kWh), open-source talent density, and hardware autonomy, and you see a new trade paradigm forming: exporting SOTA-90% reasoning capability to the world in a metered, priceable way. This isn't a story about the tech race. It's a story about cost structure.

The Channel War: OpenClaw, Anthropic, and Who Gets to Decide the Future of AI Agents

OpenClaw v2.19 shipped an Apple Watch MVP. Anthropic blocked OAuth to shut out third-party subscribers. Sam Altman recruited Peter Steinberger and embraced open source. Put these three things together and you see something beyond a technical competition — you see the most brutal reality of the AI industry: whoever controls the channel decides the model's fate.

OpenClaw Week: From the Claude Code 1.5 Era to a Digital Jarvis | Weekly Vlog EP8

A week-long deep dive into OpenClaw — from creator Peter's builder philosophy to the Memory architecture (AGENTS.md, SOUL.md), three token-saving tricks (cut 50%+ easily), and the 'new employee' enterprise security strategy. The AI agent that comes closest to a real digital Jarvis. Worth your time to understand it properly.

OpenClaw Token Optimization Guide: How to Cut AI Agent Operating Cost by 97%

Real intelligence isn’t paying for the most expensive model—it’s careful prompt and system design. This post shares five core optimization strategies—session initialization, model routing, local heartbeats, prompt caching, and rate limiting—shown in practice to reduce OpenClaw cost from ~$1,500/month to under $50.

Moltbot Security Hardening in Practice: A Complete Four-Layer Defense-in-Depth Guide for AI Agents

You don’t need to be a security expert—just be willing to spend an afternoon reading the docs carefully. This post distills Moltbot community battle-tested experience into a four-layer defense-in-depth playbook: Isolation, Quarantine, Rollback, and Transparency. It covers AI Agent Security, Prompt Injection Defense, LLM Agent Security, and an end-to-end Agentic Security framework.

When Unix Philosophy Meets AI: The Command Line Renaissance

When I was a kid I read a book called Unix Power Tools. There was a line I remembered for almost twenty years: ‘Command line pipeline is the best UI interface in the world.’ Back then I had no idea what it meant. But after Claude Code burst onto the scene in April 2025, I finally understood: a brain that understands the world through text plugged into an interface that exposes the world’s state through text. This isn’t retro—it’s structurally the most reasonable choice.

CaMeL: Google DeepMind’s Prompt-Injection Defense Architecture

Simon Willison called this ‘the first credible prompt injection defense’ he’s seen. CaMeL’s core design splits one agent into two: a low-privilege agent that reads external data, and a high-privilege agent that makes decisions—so ‘reading data’ and ‘taking actions’ are always separated.

← 返回首頁