Over the past two years, there have been an abundance of new AI models.
AI models can be trained to complete many types of tasks — from finding information and answering questions to offering customer support, proofreading documents, producing content, and more.
Many of these tasks are objective and have clear optimization functions: find the right answer, identify the most relevant pieces of information, detect any errors or anomalies, etc.
But there is a subset of models with dramatically more subjective outputs — like producing “good” art or developing “entertaining” videos. I call these “models with taste.” Taste-based models are often harder to optimize for because they’re a mix of collective and individual determinations; there’s no obvious answer or output. As a result, frequent feedback is especially valuable for helping the model stay updated on the latest cultural preferences.
Today, there are roughly two ways models can develop “taste”:
Train on user-generated content / data — like Twitter or Reddit feeds — that theoretically reveal the latest in where humans are directing their attention (thereby acting as a proxy for taste).
Leverage a community of human “tastemakers” to help actively train the model around their preferences.
The first presents a number of subpar circumstances. Data might be siloed (e.g. Reddit shutting off its API) or incur bias (e.g. if only a subset of data is shared). A model may also end up overfitting to a specific platform’s algorithm, especially if it has a limited set of options from which to pull data. That might not sound material, until one starts to imagine a significant amount of new media being generated based on Twitter’s trending content. Not ideal.
The latter — a network of humans providing feedback — avoids many of the aforementioned risks. There might still be bias, but only in that it incorporates the preferences of the community members who have opted into helping train the model. Thus, the key becomes ensuring that those community members — i.e. the tastemakers — have a stake in the model actually developing good taste.
Crypto rails can help facilitate this alignment. Providing ownership in the model / giving involved members an economic stake in the model’s output can create an incentive to participate authentically. Crypto also makes it easier for participation to be open and accessible: anyone from anywhere in the world can contribute, so long as they have a wallet and an internet connection.
A notable example of such a model can be seen in the project Botto. Botto is an autonomous artist, where $BOTTO token holders have the ability to help train the model each week. The training is simple: participants upvote or downvote various pieces of images, and Botto learns from the members’ preferences. At the end of the week, the most popular piece is sold at auction and the participants who helped train Botto that week are compensated.
Art is just one category of models with taste. Others might include movies, television, additional forms of storytelling (novels, short stories), comedy, and advertisements / brand campaigns. Even a few years ago, these models with taste wouldn’t have been possible. The tools were less expressive and slower. A model couldn’t be reliably counted on to produce cohesive or (in the case of video) realistic outputs. It’s only today that these are becoming uniquely possible.
Importantly, models with taste have large (and growing) addressable markets. Art is a multi-billion dollar market. Content consumed online constitutes trillions of hours of attention annually. If people are already going to spend time and money on such forms of entertainment, it seems reasonable that giving them a stake in the production would create not only a more engaged user base, but a more satisfied one as well. Imagine a win for Best Picture at the Oscars where the primary humans in the loop were the actual audience members who helped train and develop the storyline. Or an entirely new category for community-created movies. That’d be very cool.
I would think of this as creating a new category of content rather than displacing existing creation. It’s akin to smartphones and Instagram enabling everyone to become a photographer; the existence of those new technologies didn’t eliminate the jobs of actual photographers and, in fact, may have actually helped more people develop an appreciation for the work that photographers do. Models with taste are the same: they expand each of these categories by taking advantage of a new technology — in this case, crypto rails for consumer ownership and economic alignment — to create a new form of engagement.
We’ve seen thousands of new models emerge over the past few years. The next few will probably see additional millions (if not more). At least some should strive to engage stakeholders in new ways — from greater openness and accessibility to novel ownership structures that experiment with incentives. Models with taste are one area especially primed for this innovation, but they’re probably not the last.
Thank you to Simon Hudson, Pooja Najpal, Jesse Walden, Caleb Shough, and Nikhil Raman for feedback on this essay.
Hey Alana, good article again! RLHF is good and is so do-able with crypto incentives, but then... I am not sure if taste is not something that can be incentivized tho. Cud well end up with a community w/o taste amusing themselves.