Music Streaming Session-based Recommendation with Transformer Architectures

Abstract

Recommendation systems have grown in popularity over the last few years, with the rise of big data and development of computing resources. Compared to simple rule based methods or content based filtering methods used for recommendation during the early development stage of recommendation systems, recent methodologies try to implement much more complex models. Latent factor models and collaborative filtering methods were developed to find similarities between users and items without actually knowing their characteristics, and gained popularity. Various item domains, mainly movie and retail, have extensively used these recommendation algorithms. With the development of deep learning architectures, various deep learning based recommendation systems emerged in recent years. While a lot of them were focused on generating the predicted item ratings when given a big data comprised of user ids, item ids, and ratings, there were some efforts to generate next-item recommendations as well. Next-item recommendations receive a session or sequence of actions by some user, and try to predict the next action of a user. NVIDIA recently used Transformers, a deep learning architecture in the field of Natural Language Processing (NLP), to build a session based recommendation system called Transformers4Rec. The system showed state of the art performances for the usual movie and retail domains. In the music domain, unfortunately, advanced models for session-based recommendations have been explored to a small extent. Therefore, this thesis will attempt to apply Transformer based architectures to session-based recommendation for music streaming, by utilizing a dataset from Spotify and framework from NVIDIA. In this thesis, unique characteristics of music data that validates this researchs purpose are explored. The effectiveness of Transformer architectures on music data are shown with next-item prediction performances on actual user streaming session data, and methods for feature engineering and data preprocessing to ensure the best prediction results are investigated. An empirical analysis that compares various Transformer architectures is also provided, with models further analyzed with additional feature information.

최근 트랜스포머 기반 추천시스템들이 다양한 분야에서 높은 성능을 보여왔다. 하지만 음악 스트리밍 분야에는 적용되지 않았었고, 이 논문을 통해 음악 스트리밍 분야에 트랜스포머 기반 세션 추천시스템이 어떤 성능을 보여주는지 탐색해 보았다. 데이터 전처리를 통해 유저들이 음악을 실제로 좋아해서 들었을 법한 세션들만 남기려 노력했고, 세션 기반 추천시스템에 맞게 데이터를 정제했다. 음악과 관련된 다양한 정보들도 모델 훈련에 반영하기 위해 카테고리 형태로 바꿔주었고, 훈련 자체는 세션 기반 추천시스템에서 자주 쓰이는 점진적 훈련법을 활용했다. 최종 실험 결과에서는 데이터의 비정제성과 비밀집성을 극복하고 비슷한 데이터셋과 경쟁력을 갖추는 성과를 보여주었다. 이 연구를 통해 음악 스트리밍 세션 추천시스템에 트랜스포머 기반 모델이라는 새로운 가능성을 보여 주었고, 추후 연구자들이 참고할 수 있는 시작점을 제공하였다.