by Tim Svenonius
What's in a Name? A Renaming Experiment on YouTube
The American painter Charles Willson Peale wanted his sons to grow up to be painters, too, and named them after his favorite artists: Rembrandt, Rubens, Raphael, and Titian. Two sons, Rembrandt Peale and Titian Peale, indeed took up their father’s profession, while Rubens and Raphael chose other vocations. With a success rate of 50 percent, one might wonder how things would have turned out if Peale had named the kids Groucho, Harpo, Chico, and Zeppo. We can probably never know the extent to which our destinies are shaped by the names our parents give us, but at least in the realm of search engine optimization (SEO) we can conduct Peale-like experiments without harming any children.
We performed the following experiment with a selection of videos from our YouTube channel to see if varying the style and tone of a video’s title might affect its popularity. We theorized that for our YouTube audiences, using snappier or wittier titles would result in an increase in views. The results indicate something much stranger than that.
The premise: Over the past two decades SFMOMA’s Interpretive Media team has produced hundreds of short videos featuring artists talking about their ideas, their processes, and their art. These videos have been a staple of the museum’s website (and occasionally in the galleries) and for years formed the basis of its YouTube channel.
When it comes to titles, we’ve taken a straightforward and largely formulaic approach: artist-discusses-concept, or artist-discusses-artwork. Within the context of the museum website these titles are clear enough, because the video is situated within a network of connections: to an artwork, an artist, or an exhibition.
Not so for the YouTube channel: the YouTube user who comes upon one of SFMOMA’s videos will most likely never encounter the museum’s channel or its playlists. In all likelihood they have never physically visited SFMOMA, and perhaps never will. In the typical YouTube use case, all the contextual clues offered by the museum’s website are absent. And with that in mind, the artist-discusses-artwork formula suddenly looks like name-you-don’t-know-discusses-nonsensical-phrase.
Consider the following:
“David Wilson on Frog Woman Rock”
“Rigo 23 on his found ‘lost bird’ posters”
With an eye on the rather modest viewership stats for some videos on our YouTube channel1, we theorized that for our videos to appeal to YouTube users, their titles had to provide more of a hook—to provoke a little. Late in 2014 we tested this hypothesis with a small renaming experiment.
For each video within our test group, the original was taken offline and re-uploaded as if it was a new release. The newly uploaded video was identical to the original, except for its title, description, and tags.
The experiment uses what I’d call an asynchronous A/B test: rather than keeping two identical videos online at the same time, we measured performance over a period of six months after re-upload, and compared that against the historical performance over the first six months for the original upload.
The test group consisted of ten videos on our YouTube channel with very low engagement. Each had been on YouTube at least two years and had received fewer than 250 views in the past year, with some falling well below that.
A few colleagues from the Interpretation and Digital teams got together to dream up snappier titles for the ten selected. The old videos were quietly retired from the channel, and then republished with new names one at a time over a period of four weeks.
The video formerly called “David Wilson on Frog Woman Rock” became “David Wilson’s 16-foot-tall drawing of a rock.” “Rigo 23 on his found ‘lost bird’ posters” became “Have you seen my bird? Rigo 23’s found posters.”
A month later, every video in the test group appeared to be faring better, some surpassing the previous year’s number of views in only a month. But initial appearances are misleading: engagement naturally spikes in the day or two after a video’s release, and tapers off over time. There is no way to predict how long that waning period will be; the leveling off can take days or months. We were therefore cautious about drawing hasty conclusions.
The true outcome of the experiment would have to wait six months—long enough for the initial spike from republishing to flatten out. Six months of stats were then compared to the first six months of the originals.
After six months it was clear that three of the new names were successful, one had failed, and five more fared about the same (+/- 15 percent) as their original counterparts. The average performance overall was 130 percent of the original.
But deriving meaningful lessons from these results isn’t entirely straightforward. In attempting to understand what makes one title “work” and another not, we tried to discern what precisely was different between the successful and the not-so-successful titles.
My best guess is this: the titles that performed best balance something revealed with something withheld. For example, the phrase “Have you seen my bird?” is a mystery element, which acts as bait. The second part, “Rigo 23 on his found posters” is a small morsel of fact that anchors it. Conversely, “David Wilson’s 16-foot-tall drawing” is the factual bit, and “of a rock” piques the curiosity.
Following that logic, we can find a counterexample in a title such as “Solving problems by sipping tea: Daniel Schwartz.” This might have been better curiosity bait if the name Daniel Schwartz meant something to users; otherwise there’s no factual counterbalance to the title’s riddle.
A Control Group
We know that simply republishing impacts a video’s performance, artificially introducing an activity spike. In an attempt to better measure the extent of that spike, we created a control group. Five more videos—again, low performers—were selected for republishing. But in this case we changed nothing. These five videos were simply re-uploaded as though they were new, with the same names.
Now things got really interesting: the net gain from re-uploading the videos without changing anything was greater than the average gain from giving them more thoughtful titles.
What did we learn, in the end? Comparing the gains of renaming with the seemingly arbitrary boost that came from simply rereleasing shows us that the renaming strategy—shedding the straightforward and formulaic in favor of the witty—in a majority of cases (66 percent) did not prove advantageous. Where the renaming resulted in a clear and irrefutable advantage, perhaps what it really told us is that the content of the video lends itself to witty titling. In many videos, the subject matter is too hermetic or academic to easily turn into YouTube-friendly phrasing.
The question of whether and how much a title affects a video’s views is a tricky one to pin down. Countless factors, including release time, coinciding publicity, et cetera can impact view counts, and objectively measuring the relative weights of different variables is close to impossible.
Even though this experiment did not provide us with definite answers, the exercise of asking these questions and closely examining our YouTube content helped us develop our overall strategy for the channel. We began to question not only our video naming conventions, but whether the content forms that generated them were right for YouTube. We began to wonder if intriguingly worded titles alone would be enough to tempt a YouTube viewer to watch a video about an artist they’d never heard of, and we began to look at whether we could shape both our videos and their titles around broader themes.
As you can read in the rest of our series on YouTube, the SFMOMA Lab team has been doing a lot of thinking about how a museum can best make use of YouTube as a platform. We continue to believe that the naming of YouTube videos is important. For information on our further research and findings on the topic, see our article on YouTube naming.
1. We do not, as a rule, promote our videos on YouTube; we generally prefer to direct traffic to our own website. Still, it is likely, in the interconnected ecosystems of the web, that the videos on YouTube are buoyed by the buzz around an event or exhibition, ever so slightly. A rerelease a couple of years later will miss out on all of that, likely resulting in fewer views.