Music Recommendations Based on User's Mood Using Convolutional Neural Networks
DOI:
https://doi.org/10.24193/subbi.2022.1.04Keywords:
mood, emotion, valence, energy, convolutional neural network,recurrent neural networks, long-short term memory, hybrid, regression, classification.Abstract
This paper proposes a method for music recommendations using emotions, using deep learning techniques. The method is composed of two modules. The emotion detection module, which utilizes a hybrid architecture involving a Convolutional Neural Network (CNN) and a Reccurent Neural Network using Long-Short Term Memory (LSTM) Cells. We compared individual architectures of CNNs and LSTMs against our hybrid approach, outperforming them during experiments. We evaluated the modules on our own data set, created using Spotify’s API and containing 2028 songs from different genres and linguistic families, labeled with valence and arousal values. The model also outperforms other related approaches, however we did not evaluate them on the same data set. The predictions are used by the second module, for which we proposed a simple method of ordering the results based on the similarity to user’s input.
Received by the editors: 10 October 2021.
2010 Mathematics Subject Classification. 68T45.
1998 CR Categories and Descriptors. I.4.8 [Image Processing and Computer Vision]:Scene Analysis – Object recognition; I.2.6 [Artificial Intelligence]: Learning – Connectionism and neural nets; I.2.10 [Artificial Intelligence]: Vision and Scene Understanding – Intensity, color, photometry, and thresholding.
References
Bhattarai, B., and Lee, J. Automatic music mood detection using transfer learning and multilayer perceptron. International Journal of Fuzzy Logic and Intelligent Systems 19, 2 (2019), 88–96.
Clifton, A., Pappu, A., Reddy, S., Yu, Y., Karlgren, J., Carterette, B., and Jones, R. The spotify podcast dataset. arXiv preprint arXiv:2004.04270 (2020), 1–4.
Delbouys, R., Hennequin, R., Piccoli, F., Royo-Letelier, J., and Moussallam, M. Music mood detection based on audio and lyrics with deep neural net. In Proceedings of the 19th International Society for Music Information Retrieval Conference, ISMIR 2018, Paris, France, September 23-27, 2018 (2018), pp. 370–375.
Dey, R., and Salem, F. M. Gate-variants of gated recurrent unit (GRU) neural networks. In IEEE 60th International Midwest Symposium on Circuits and Systems, MWSCAS 2017, Boston, MA, USA, August 6-9, 2017 (2017), IEEE, pp. 1597–1600.
Hevner, K. Experimental studies of the elements of expression in music. The American Journal of Psychology 48, 2 (1936), 246–268.
Kamm, T., Hermansky, H., and Andreou, A. G. Learning the mel-scale and optimal vtn mapping. In Center for Language and Speech Processing, Workshop (1997), pp. 1–8.
Li, T., and Ogihara, M. Detecting emotion in music. CiteSeer (2003), 1–3.
Lidy, T., and Schindler, A. Parallel convolutional neural networks for music genre and mood classification. MIREX2016 (2016), 1–4.
Liu, T., Han, L., Ma, L., and Guo, D. Audio-based deep music emotion recognition. AIP Conference Proceedings 1967, 1 (2018), 040021.
Malik, M., Adavanne, S., Drossos, K., Virtanen, T., Ticha, D., and Jarina, R. Stacked convolutional and recurrent neural networks for music emotion recognition. CoRR abs/1706.02292 (2017).
Peeters, G. A generic training and classification system for mirex08 classification tasks: audio music mood, audio genre, audio artist and audio tag. In Proceedings of the International Symposium on Music Information Retrieval (ISMIR’08) (2008), Citeseer.
Petrescu, A. Spotify dataset. https://github.com/AndreiPetrescu99/SpotifyDataset.git/, 2022.
Raju, A., R.S, D., Gurang, D., Kirthika, R., and Rubeena, S. AI based music recommendation system using deep learning algorithms. IOP Conference Series: Earth and Environmental Science 785 (06 2021), 012013.
Russell, J. A. A circumplex model of affect. Journal of personality and social psychology 39, 6 (1980), 1161.
Tan, K., Villarino, M., and Maderazo, C. Automatic music mood recognition using russell’s twodimensional valence-arousal space from audio and lyrical data as classified using svm and naive bayes. IOP Conference Series: Materials Science and Engineering 482 (03 2019), 012019.
Yang, G. Research on music content recognition and recommendation technology based on deep learning. Security and Communication Networks 2022 (03 2022), Article ID 7696840.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2022 Studia Universitatis Babeș-Bolyai Informatica
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
