Rudolf Mayer: Analysing the Similarity of Album Art with Self-Organizing Maps

Rudolf Mayer from Vienna University of Technology has examined connections between music genres and album cover art using self-organizing maps, SOMs. In traditional record stores, consumers already search music with the help of album covers, and there are a lot of resources used to design album covers for the right target groups. With the growing selection of digital music it is important to create more easier ways to organize and storage music. Computer algorithms that can understand some features of music, and then recommend new artist for users, are already used. There are studies where music is analyzed with not only audio features, but also for example with song lyrics and other texts related to it.

The data used in the research, which included over 900 song sorted by 7 genres, was gathered from music store amazon.com, where you can search by genre and download both the music sample and the album cover. Gathering of data was problematic in this study, for even though there are some music banks for research use, most of them don't include album covers or enough information to search them.

Mayer first trained SOMs sized 22x18 nodes with audio features, Rhythm Patterns, Rhythm Histogram and Statistical Spectrum Description (SSD), and analyzed the best of these to cluster music. According to authors' perception, SSD provided the best arrangement of music. Approximate analyze on this data showed that each genre has very recognizable features on album art. For example, classical music has simple design with photos of people, only few colors and simple background. When maps were trained with image features, Color Histogram, Color Names and Scale Invariant Feature Transform, the idea was that similar music should be located close on the map. But there were basically no continuous areas of similar music with any used image feature. Analytic comparison of SOMs showed also only little percentage of matches between two mappings.

Conclusion was that the used image features are not enough to arrange as complex features as album art has. There is potential in using album art when analyzing music information, but there is a need for more powerful image feature descriptors.