On My Mind: Experts in LLM Backrooms
Plus, a framework about thermostats
On My Mind
Finding experts in LLM backrooms.
I’ve had an increasing number of conversations with friends about the ways in which we use AI. It seems like there’s been a dramatic increase since the start of the year (especially among my friends in finance).
So today, I wanted to share one of the methods I’ve found most helpful: having Claude take on the perspective of category experts when trying to learn about a new, emerging, or evolving topic. In particular, I’ve used this technique most frequently in the context of informing macroeconomic views.
Throughout the first few months of this year, I set up several chats oriented around different macro-economic topics – private credit & reinsurers, the Iran conflict, etc. – and instructed each chat to take on the perspectives of a number of widely known, respected investors (e.g. Howard Marks, Ray Dalio, Mohamed El Erian, Stanley Druckenmiller, and more). The goal was to learn as much as possible, as quickly as possible, about the relevant news each day. I didn’t want just any perspective; I wanted that of experts.
Now, the first thing I do most mornings is open each chat and ask Claude to “update the investor perspectives with new and relevant information.” The result is that I keep up to date with current events much better than I used to AND I feel significantly more informed about the second-order consequences of much of what’s going on in the world.
So, why has this worked well? Each selected expert has a large corpus of public material that the LLM can reference and synthesize. Specifying the investors narrows Claude’s context window to the parts of its knowledge base I find most relevant. Instead of asking the macro questions into an abyss – and not knowing the origin of each answer I’d get back – I now have some grounding as to the rest of the “persona” and potential bias behind each viewpoint. Seeing the diversity of perspectives also helps illuminate any consistencies / throughlines. And sometimes, I’ll even ask Claude to simulate a debate between two of the different investors.
It seems to me this pattern will be extended and generalized to all types of work. What I’ve described above is a type of knowledge work. The next iteration seems skills-based, where instead of asking for expert knowledge you’re paying for expert labor. That labor could include software engineers, lawyers, designers, consultants, music producers, and more. In the same way that I ask Claude for someone like Mohamed El Erian’s perspective on macro, a company might only hire a software engineering agent modeled after Andrej Karparthy or a designer that is trained on the work of Jony Ive. The market for licensing one’s likeliness seems like it’s, in aggregate, about to get so much bigger – but for only the top 1% within each category of labor.
There’s a lot to dig into if you keep pulling on the thread: how would highly skilled people make their services legible and replicable? What does economic attribution look like? What happens to the folks who aren’t experts? And many more questions. I’m certainly not the first to write about this, and I definitely won’t be the last.
Frameworking
This week’s framework is borrowed from a passage I read in Andre Agassi’s memoir Open.
The framework: are you a thermometer or a thermostat?
A thermometer reads the temperature of a room. A thermostat can change and control it. Knowing who is what can help in almost any setting.
For instance, think about constructing a team. Thermostats can be great leaders: they set the tone and energy of a project. But placing two thermostats in adjacent roles can create conflicting temperatures and, consequently, breed discomfort. On the other hand, a thermometer adjusts to the temperature in the room. Sometimes that means thermometers are great at going with the flow; other times it might look like significant ups-and-downs if the overall environment is chaotic.
One isn’t more right than the other. Both thermostats and thermometers can do well at work and in social situations. The key is simply to recognize which one you are – and which one your colleagues are – to create the most harmony.
Content I Consumed this Week
The Drama – the new Zendaya and Robert Pattinson movie from a24. I rate it either an A- or B+ – watchable but probably not rewatchable. Would recommend going in knowing as little as possible about the movie. Also best watched in a theater; the collective gasps from the audience definitely elevate the experience.
Currency Wars by James Rickards – about a third of the way through. The weather was exceptionally nice in NYC this weekend, so I stopped by my local Barnes & Noble to pick up a book to read in the park. This one caught my eye. So far, the book has provided a solid history of the ways in which currencies have evolved and how nations have used currency markets as attack vectors.
Redpoint’s Software Market Upside. I LOVE a good deck. Decks might be my favorite form of content. I don’t agree with everything in the deck, but there were certainly a number of great and thought-provoking charts. There’s so much uncertainty around the terminal value of SaaS companies right now. Whoever can figure out what survives (and thrives) on the other end of the AI disruption is sure to make a fortune.
Books on the docket: The Battle of Bretton Woods by Ben Steil