Skip to content

Apples, Oranges, Music, and Meaning

November 27, 2005

Many of my friends know me as the “MusicStrands guy”. They know that I deal with music recommendation, mp3’s, and other cool music technology. Many times, they are interested in the specifics of how I handle music recommendation, and if the recommendation method is truly capable of producing interesting and relevant song recommendations for themselves and others. I figured I should make a blog post about this topic, since I find it extremely interesting. (In fact, I plan on studying this phenomenon for the next couple of years.)

Content vs. Link Characterization
The first thing that you should know about my approach to song recommendation research is that it is largely based on network, or link characterization metrics, rather than content characterization metrics. What this means is that I focus on the links (link characterization) between songs, rather than doing an acoustic analysis (content characterization) of the songs. In other words, I don’t have a computer program that “listens” to the songs and then decide what genre the song belongs to. Instead, I focus on analyzing playlist files. These files are simple arrangements of songs that an individual has strung together. The file doesn’t actually contain the music itself, just references to some music files on a hard drive. If I take enough of these playlist files, I can start characterizing the songs in terms of how they are linked to each other.

The Limitations of Categories
If anyone has tried to organize their music libraries, they find that the task incredibly difficult. Is Linkin Park rap or rock? Is Beck blues, disco, or folk? Should I organize by year? Should I organize by amount of cowbell usage? I would argue that there isn’t a single hierarchical system that works for all (or even the majority) of popular music. Record stores, limited by shelf space, are forced to put albums in preordained slots, such as rock, pop, or jazz. However, as anyone who’s been to Walmart recently can tell you, there’s really only a couple of genre categories that are used frequently. There will be four or five aisles devoted to pop/hip-hop and one devoted to jazz/classical/world music/gospel. The point of all of this is that a “true” categorization of music is impossible, and that the majority of music sold (with Walmart being the biggest retailer) is done so without an effective categorization process.

Apples to Oranges
I’d like to take a step back and talk about “clustering”. Think about how you organize your stuff when you move to a new house or apartment. Most people will have a number of boxes and labels, and they’ll organize and move their belongings according to their own scheme. A box and label scheme may differ slightly from person to person, but in general, you’ll see groups of items that share attributes. For instance, you may see boxes with the label “fragile”, or “bedroom”, or “tools”, or “Justin’s T-Ball trophies”. Each item an individual owns could conceivably be in one or more boxes. Furthermore, certain boxes/categories may not exist for certain individuals, particularly if they suck at T-ball and have never won any trophies.

As intelligent beings, we engage in “apples to oranges” comparisons constantly. We arrange our physical possessions into configurations that optimize their usefulness/efficiency. When an item has the potential for inclusion in two or more categories, we choose an inclusion that optimizes its utility for ourselves (In fact, other animals display this trait, many of them that are arguably “not intelligent.”) This organization behavior is echoed in the digital world. As a windows xp user, I constantly thwart Microsoft’s attempt to categorize my media and documents. I have MS Word documents in many other places besides “My Documents”. Likewise for pictures, videos, etc (and I’m sure Mac users exhibit the same behavior).

This common behavior leads to a conclusion that the categorical labels we use are “ad hoc” cultural artifacts. Common categorical labels achieve prominence through common cultural influences and re-enforcement, but have no intrinsic merit or meaning in and of themselves. However, an agreed upon categorical scheme is important if we want to share our collections of items with someone else…. or is it?

Music and Meaning

Talking about music is like dancing about architecture
-Frank Zappa

We return to the problem of music recommendation. How are we going to share/navigate relevant music recommendations, when no explicit categorization/hierarchy strategy exists? Genre labels can change, artist names can change, lyrics can differ tremendously between identical melodies. In general, words seem to fail as the basis for an effective categorical scheme. In accordance with this issue, my strategy is to not use text/labels at all. I believe that to have any shot of providing relevant recommendations, one should leverage how people choose to arrange items, not by trying to understand the items themselves. After all, a computer is not going to understand that I like to listen to Django Reinhardt and Coldplay when I jog. Nowhere in the music itself is that correlation present. However, others may find this pairing pleasant for jogging as well, and accept it as a recommendation. There are probably many other clusters that a certain Coldplay song belongs to, perhaps with Radiohead, or Franz Ferdinand, and this can seem confusing. However, we can consider the likelihood of a Coldplay song occuring with a Radiohead song (very high), as opposed to a Coldplay song and a Gregorian Chant (extremely low to nil), and stilll come to an understanding of how Coldplay fits into the tapestry of popular music.

State of the InternThis approach is not an attempt to provide a single, all inclusive organizational hierarchy for music, but rather to understand music as a network, and to identify and extract clustering behavior that exists among songs. At the macro level this gives an understanding of the “big picture” of how people are organizing their music at any given point in time (through relevant statistical clustering/dimensionality reduction methods). At the micro level, this approach forms the basis for a machine learning application. The recommendation system must figure out how an individual user is organizing their own music (are Coldplay songs associated more with Radiohead, Django, or perhaps other songs?), and make recommendations in this context. This process may mean a “trial and error” approach towards tuning into exactly what the user likes listening to, and in my opinion, that is why a machine learning approach seems to be a good bet (I’ll leave the strategy for this functionality for another post). MusicStrands already does this to a certain extent, but there is a lot of potential for improvement. I encourage you to try their iTunes plugin (sign up to download it), and tell me what you think!

If you’re interested in learning more about this, you should definitely drop me a line!

From → Uncategorized

  1. Really interesting, Justin. Your explanation is really clear, and this topic is awesome. I think I agree with your theory that organizing by labels isn’t necessary…I wonder though, what types of tasks would you have people do or what would you have the computer “listen” for in order to figure out song order based on a given context. How does the iPod wearer signal that their current listening scheme is meant to be the “in the car mix” vs. the “jogging mix”?

    Anyway, this is really cool stuff, and I know you’ll rock it.

  2. Justin Donaldson permalink


    People are still going to label their own playlists. Words are still needed to locate a playlist on a hard drive, or on an ipod. However, this system is based on the assumptiont that people will have more than one playlist for certain situations. Do you want music to listen to in your car? Put together a list of 6-7 songs that work for you and let the system pick out another 6. Want something romantic? Get some sappy love songs, and the recommender system will give you more of the same. Do you want Post Grunge music from South Korea? (Assuming enough people are putting that on playlists somewhere), the system will eventually be able to figure out that’s what you want, although you may have to be patient, and turn down a couple of its best guess recommendations until it gets it right.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: