Annotations
- Highlighted in articleArticleExample User@exampleuserannotated ·
“In deep learning, the transformer is a family of artificial neural network architectures based on the multi-head attention mechanism, in which input data such as text, images, or audio is converted to a sequence of numerical representations called tokens, and each token is converted into a vector through an embedding layer.”
The quiet idea behind almost every AI product today. Attention let models look at the whole context at once instead of word by word, and that one change made scaling work. - Highlighted in articleArticleExample User@exampleuserannotated ·
“Armstrong and Aldrin descended to the surface aboard the LM Eagle, landing in the Sea of Tranquility (Mare Tranquillitatis) on July 20 at 20:17 UTC”
Eight years from Kennedy's speech to boots on the Moon, built with less computing power than a modern calculator. The real lesson is what a clear, public deadline does to an organization. - Clipped from YouTubeYouTubeExample User@exampleuserannotated ·Karpathy's most useful reframe: an LLM is less a chatbot than the kernel process of a new operating system, coordinating memory and tools like browsing, retrieval, calculators and Python to solve problems. Once you see the context window as RAM and tools as peripherals, a lot of agent design stops looking like magic and starts looking like systems engineering.
Transcript not available from this source
- Clipped from YouTubeYouTubeExample User@exampleuserannotated ·The lesson here isn't that things always work out. It's that you can't plan the path ahead of time: a calligraphy class Jobs took after dropping out of Reed looked useless, then showed up ten years later in the Mac's typography. Following curiosity without a guaranteed payoff is how the dots get made at all.
Transcript not available from this source
- Highlighted in articleArticleExample User@exampleuserannotated ·
“On it everyone you love, everyone you know, everyone you ever heard of, every human being who ever was, lived out their lives.”
Earth is about 0.12 of a pixel in this frame, caught in a band of scattered sunlight. Every war and empire in history happened on that speck, which is a useful scale check for any argument that feels world-ending. - Highlighted in articleArticleExample User@exampleuserannotated ·
“It is allegedly easy to get started with training neural nets. Numerous libraries and frameworks take pride in displaying 30-line miracle snippets that solve your data problems, giving the (false) impression that this stuff is plug and play.”
Still the best antidote to tutorial confidence. Neural nets fail silently, so the discipline is to look at your data and overfit a tiny batch before trusting any loss curve. - Highlighted in articleArticleExample User@exampleuserannotated ·
“That means that any 1-in-a-million appeal can find 1,000 true fans. The trick is to practically find those fans, or more accurately, to have them find you.”
The math holds, but the second sentence is the hard part. With discovery running through platform algorithms, being findable increasingly means owning a direct channel to your fans, like email or a membership. - Highlighted in articleArticleExample User@exampleuserannotated ·
“Six decades into the computer revolution, four decades since the invention of the microprocessor, and two decades into the rise of the modern Internet, all of the technology required to transform industries through software finally works and can be widely delivered at global scale.”
The thesis aged well because it was about timing, not just ambition: the stack finally worked. It is worth rereading now with “AI” swapped in and asking which layer of today’s stack does not quite work yet. - Highlighted in articleArticleExample User@exampleuserannotated ·
“Nonetheless, with policy in restrictive territory, the baseline outlook and the shifting balance of risks may warrant adjusting our policy stance.”
Classic Fed language: it opens the door to a cut without committing to one. Note the conditions stacked in front of the verb, restrictive policy plus a shifting balance of risks, which is how a central bank signals without promising. - Highlighted in articleArticleExample User@exampleuserannotated ·
“I’ll refer to this as the “compressed 21st century”: the idea that after powerful AI is developed, we will in a few years make all the progress in biology and medicine that we would have made in the whole 21st century.”
The useful part of this frame is that it names the real bottlenecks: clinical trials, lab throughput, and regulation, not raw intelligence. Whether the compression happens depends as much on those physical loops as on the models. - Highlighted in articleArticleExample User@exampleuserannotated ·
“There's a kind of excited curiosity that's both the engine and the rudder of great work.”
Engine and rudder is the right pairing: curiosity supplies the energy and also picks the direction. If you are only disciplined about a topic and not curious about it, you will usually steer toward safer, smaller problems. - Highlighted in articleArticleExample User@exampleuserannotated ·
“The scaling hypothesis regards the blessings of scale as the secret of AGI: intelligence is ‘just’ simple neural units & learning algorithms applied to diverse experiences at a (currently) unreachable scale.”
Read years later, this is the clearest early statement of the bet the whole industry ended up making. What still stands out is that it treats scale as an empirical claim to be tested, not an article of faith. - Highlighted in articleArticleExample User@exampleuserannotated ·
“This is how the singularity goes: wonders become routine, and then table stakes.”
This line captures how adoption feels from the inside: the bar for 'impressive' keeps resetting, so progress can feel slower to users than it is. - Highlighted in articleArticleExample User@exampleuserannotated ·
The short history of global living conditions and why it matters that we know it
Max Roser·ourworldindata.org“The chart shows that almost 10% of the world's population lives in extreme poverty. It also tells us that two hundred years ago, the same was true for nearly 80% of the world’s population.”
Both numbers matter: the progress is enormous and the remaining 10% is still hundreds of millions of people. Long-run data is the best defense against both complacency and doom. - Highlighted in articleArticleExample User@exampleuserannotated ·
“When building applications with LLMs, we recommend finding the simplest solution possible, and only increasing complexity when needed. This might mean not building agentic systems at all.”
The most underrated advice in AI engineering: start with one well-crafted prompt and justify every added loop. Agent architectures add latency, cost and failure modes, so they have to beat the simple baseline. - Highlighted in articleArticleExample User@exampleuserannotated ·
“That’s because investing is not the study of finance. It’s the study of how people behave with money.”
This explains why a spreadsheet rarely survives a drawdown. Your plan is only as good as the version of you that has to follow it when markets fall. - Highlighted in articleArticleExample User@exampleuserannotated ·
“If you do not work on an important problem, it's unlikely you'll do important work. It's perfectly obvious.”
Obvious, yet rarely done deliberately. Hamming's fix was 'Great Thoughts Time,' a standing Friday lunch where he discussed only the big questions about his field's future. - Highlighted in articleArticleExample User@exampleuserannotated ·
“Again, you can’t connect the dots looking forward. You can only connect them looking backwards, so you have to trust that the dots will somehow connect in your future.”
The calligraphy story makes this concrete: a class with no apparent use ended up shaping the Mac’s typography. It is a good argument for following curiosity even when you cannot yet justify it. - Highlighted in articleArticleExample User@exampleuserannotated ·
“The most common unscalable thing founders have to do at the start is to recruit users manually. Nearly all startups have to.”
Every growth playbook skips this step because it is boring and does not compound on day one. The point is the learning: the first hundred users you recruit by hand tell you what to build next. - Highlighted in articleArticleExample User@exampleuserannotated ·
“The first principle is that you must not fool yourself—and you are the easiest person to fool. So you have to be very careful about that.”
This applies far beyond physics: dashboards, A/B tests, and model evals are all places where we want to believe the result. Building in ways to prove yourself wrong is the whole job. - Highlighted in articleArticleExample User@exampleuserannotated ·
The lethal trifecta for AI agents: private data, untrusted content, and…
Simon Willison·simonwillison.net“If you are a user of LLM systems that use tools (you can call them “AI agents” if you like) it is critically important that you understand the risk of combining tools with the following three characteristics. Failing to understand this can let an attacker steal your data.”
Private data, untrusted content, and a way to send data out: any agent with all three is one prompt injection away from leaking. It is a simple threat model for deciding which tools an agent should never hold at the same time. - Highlighted in articleArticleExample User@exampleuserannotated ·
“Our portfolio shows little change: We continue to make more money when snoring than when active. Inactivity strikes us as intelligent behavior.”
The edge here is temperament, not information. Most investors would do better if trading felt as costly as it actually is after taxes, fees, and mistakes. - Highlighted in articleArticleExample User@exampleuserannotated ·
“The world has a billion users, human users. My sense is that the world is going to have billions of agents, not today.”
If agents outnumber people, the scarce resource stops being attention and becomes trustworthy tooling: identity, permissions, and audit trails built for software that acts on its own. The CPU angle is the tell that agents look less like chatbots and more like users who never log off. - Highlighted in articleArticleExample User@exampleuserannotated ·
“The idea that new code is better than old is patently absurd. Old code has been used.”
Every team that wants a rewrite should read this first. Old code often looks messy because it has absorbed years of bug fixes and edge cases nobody wrote down, and a rewrite throws that knowledge away. - Highlighted in articleArticleExample User@exampleuserannotated ·
“The great enemy of clear language is insincerity. When there is a gap between one’s real and one’s declared aims, one turns as it were instinctively to long words and exhausted idioms, like a cuttlefish spurting out ink.”
Still the best test for corporate and political writing: when the prose gets foggy, ask what the writer would rather not say plainly. Vague prose is often a choice, not a skill gap. - Highlighted in articleArticleExample User@exampleuserannotated ·
“The seeds of their destruction are sown and watered for years, and then they are reaped quickly.”
Frames 2023's bank failures as interest-rate risk, not credit risk: long-duration assets from the low-rate years lost value as rates rose. Runs feel sudden, but the balance-sheet choices behind them were visible for years. - Highlighted in articleArticleExample User@exampleuserannotated ·
“We live in capitalism, its power seems inescapable — but then, so did the divine right of kings. Any human power can be resisted and changed by human beings.”
A short acceptance speech that reframes inevitability as a failure of imagination. It is a useful reminder for tech too, where “this is just how the market works” often ends the conversation too early. - Highlighted in articleArticleExample User@exampleuserannotated ·
“This pattern—human progress moving quicker and quicker as time goes on—is what futurist Ray Kurzweil calls human history’s Law of Accelerating Returns.”
A great intuition pump for why linear forecasts keep underestimating technology. The catch is that exponentials in capability do not automatically mean exponentials in adoption or wellbeing.