Utilizing User-Generated Documents to Reflect Music Listening Context of Users for Semantic Music Recommendation

Abstract

There are millions of songs available on the Internet. For instance, one of the major online music service Spotify announced that the size of their digital music library contained over 30 million tracks. Such vast amount of data allowed users to easily access music anytime and anywhere. However, the overflow of data also arose a drawback known as the paradox of choice in which users have difficulties in finding the appropriate music that fits the users need. Therefore, the need for novel technologies to guide users search and discover music is arising. There are many technologies that guide users to search and discover music. While it is difficult to strictly separate music search and music discovery, it is possible to distinguish the two based on the requirement of a prior knowledge on the music the user is seeking for. Music search requires a specific knowledge in the music the user is seeking for. For example, if a user intends to buy a specific song from iTunes, the user can query Psy Gangnam Style and the system will retrieve the exact song. On the other hand, music discovery involves retrieving novel music for the users. There is no requirement of prior knowledge in music. For instance, if a user listens to music from Last.fm for a certain period of time, the system analyzes the listening pattern and retrieves novel songs to the user. During the procedure of both music search and music discovery, the information provided by the user and the similarity measure implemented by the system are two important aspects. The information provided by the users includes specific user queries, music listening history, music purchase history, etc. While most of the technologies available nowadays utilize various information provided by the users to identify relevant music, in most cases the users music listening context is neglected. Also, the similarity measures implemented by various systems do not regard the users music listening context. However, since music is consumed in couple of minutes and the preference in music change rapidly based on which context the user is in when seeking for the music, the need for including the users context was upraised in the past years and research in enabling context-based music search and discovery is steadily increasing. In this dissertation, we present a top-down approach in enhancing music description by including users contextual information. Our main idea is to extract users contextual information from user-generated documents and include this information when describing music. Users create these documents, which contain a personal story followed by a song request. In our first proposed method, we perform Latent Semantic Analysis and probabilistic Latent Semantic analysis to find similar documents. We evaluate our system in a quantitative manner using metrics such as precision-recall and reciprocal rank. Additionally, we conduct a user study to evaluate our system in a qualitative manner. The experiment results show that people in similar situation share similar music preference and that there is a strong association between the personal story and the song request. Our second proposed method utilizes the findings from the first approach and attempts to overcome some of the limitations of the first approach. One major limitation of our first approach is the requirement of an input document with sufficient length. This is not feasible for conventional keyword-based search. Additionally, our system suffers the well-known popularity bias, in which popular music are more easily discovered compared to non-popular music. To overcome these limitations, we implement a novel system that utilizes context-relevant keywords as music descriptors. Our system computes normalized term frequency – inverse document frequency on massive collections of user-generated documents to create a generalized Keyword Dictionary. Using the generalized Keyword Dictionary, we describe each song by computing the frequency distribution of the terms within the associated documents. Since there are multiple documents requesting the same song, the music descriptors will capture the consensus of users contextual background in listening to the song. We evaluate our system by comparing the extracted keywords with social tags provided from Last.fm to show that our system produces much more context-relevant terms. Additionally, we perform various experiments in order to identify the correlation between the proposed music descriptors and conventional features. We used acoustic features and lyrics as the conventional features since they are widely used in works related to music retrieval. Finally, we qualitatively evaluated our system by comparing the performance of our proposed music descriptors with other conventional features for music retrieval. The results showed that the performance of the proposed music descriptors was competitive with conventional features, thereby suggesting their potential use for describing music in semantic music search/retrieval. This dissertation provides two major contributions to the research field of music information retrieval. First, it presents a framework to extract users contextual information accurately by utilizing documents written by users. Second, it presents a framework to develop generalized context-relevant music descriptors, which will allow semantic music search and discovery. Lastly, it can be applied to practical applications such as automatically recommending music to services that use text and searching/retrieving music using keywords related to music listening context in a natural language setting.