W34

W34 Weekly Readings: The Week AI Raised Homework Scores but Dropped Exam Scores

A study found AI raised homework scores while exam scores fell, AI companies destroy physical books, an agent harness to "run an office of your clones," and why your local LLM feels dumber than it is — this week what hit me most was the education line: AI finishing for you is not you learning

8articles
5+sources
Y

One study stuck with me this week: AI raised students' homework scores, but the same students' exam scores dropped

That one sentence captures the core tension of "AI in education" — AI makes your output look good while potentially hollowing out the learning itself. Homework is for practice; the exam is where it's tested whether you actually learned

AI Models & Products

Claude Opus 5 and Meta's Muse models stayed on the table. No earth-shaking model news, so I put my attention back on the "how to use" end. That's my growing conviction this half-year: model capability is already in surplus; the bottleneck is whether people use it well and correctly, not how much stronger the next version is

"Why your local LLM feels dumber than it is" hit HN. Very practical. It argues that many people find local models bad because quantization, context, and prompt settings aren't dialed in — not the model itself. I relate hard: a tool "feeling bad" is often a settings problem, not a capability one, something I keep confirming across AI tools

AI Dev Tools & Agents

Munder Difflin: a harness that "runs an office of your clones" (HN high score). Tempting and dangerous. Running a swarm of "you-clone" agents in parallel has huge imagined upside. But I just stepped hard into this exact pit — releasing a bunch of agents at once with no coordination and no convergence condition, and they all spun and delivered nothing. My lesson: agent count is not throughput; coordination and closeout are. Before you figure out "who orchestrates, what counts as done," more clones is just more disaster

Dan Shipper wrote "Our AI costs jumped 230%, and I'm not setting token budgets — yet." I half agree, half stay wary. Agree: early on, don't strangle costs while you learn to use AI. Wary: 230% is a steep curve, and "not yet" is not "never." You'll eventually have to do the unit economics

Expert Takes

Jeremy Howard (fast.ai)
Jeremy Howard (fast.ai) —

He wrote "My friends all hate AI; I just joined an AI startup." Someone who co-founded fast.ai and worked in the field for over a decade chose this moment of backlash to return to AI. What I read is an attitude: the more polarized the collective mood, the more you should decide where you stand by your own judgment, not be pushed by the vibe. A good reminder for someone often pulled by the two extremes of opinion on AI

On AI and learning
On AI and learning —

I want to amplify that "homework up, exam down" study. It confirms what I've worried about: when AI is too convenient, the "friction" part of learning is removed — and friction is exactly where learning happens. Building education-related things, I keep reminding myself: a good AI teaching tool doesn't generate the answer for students, it helps them stay in the hard part a little longer

VC & Markets

a16z's "Rise of the Borderless Founder" and "Your Favorite Creator Isn't Real — Does It Matter?" Both describe the same shift: the barriers of geography and authenticity are being lowered by AI. A person can start a business borderlessly; a "creator" need not be real. I read it as both opportunity and warning — the opportunity is a bigger stage for independent workers; the warning is that "authenticity" will increasingly need to be proven

AI companies destroying physical books to obtain training data, prompting calls to save rare books (HN high score). This one made me uncomfortable. Destroying original physical copies to feed a model is a very short-sighted trade. It echoes the earlier "AI eats the web's collective memory" — we're consuming some irreversible things for AI's nourishment

My Take

Stacking the lines, the theme is one word: friction

  • Education: AI removes the friction of learning → homework better, real learning less
  • Data: AI destroys physical originals for nourishment → removes the friction of preservation
  • Agents: releasing a swarm at once → removes the friction of coordination, then everything spins

I felt it firsthand this week — some friction is necessary. Learning needs the friction of getting stuck; coordination needs the friction of convergence. AI's most seductive and most dangerous trait is how well it removes friction, tricking you into thinking the work is done when it merely got faster, not more right

Action Items

  1. When learning with AI, deliberately keep some friction — let AI question and check, don't let it hand you answers, or the homework's pretty and the head's empty
  2. Before spawning multiple agents, define "who orchestrates, what counts as done" — no coordination and convergence, more clones more disaster (I stepped in this exact pit this week)
  3. If a local model feels dumb, check settings before blaming the model — quantization, context, prompt are usually the real culprits
  4. AI cost can be loose early, but know the curve — "not setting a budget yet" is not "no need to do the math"

Sources

RSS Digest: see research/digests/2026-W34.md (284 articles this week, from Hacker News, Anthropic, Meta AI, fast.ai, a16z, and others)

AI educationlearninglocal LLMAI agentAI cost