Hello everyone!

I am Jun-You Wang (王鈞右), a post-doctoral researcher at the Music and Culture Technology Lab, Institude of Information Science, Academia Sinica.

My research focuses on enhancing human's music experience through the use of computational tools such as Deep Neural Networks (DNNs). As a person who loves music and (perhaps with some exaggeration) was saved by music during my teenage years, it is a great pleasure to contribute to the development of future music culture with my knowledge and experiences in computer science, particularly in machine learning and data science.

My research goal is to make it much easier for people with different backgrounds to understand, appreciate, interact with, and even create, the type of music they like. Take myself as an example. I do not have a background in music/musicology and cannot really appreciate music with in-depth analysis. From my perspective, it would be great to have a computational tool that tells me how does a musical piece make me feel nostalgic, or what makes the performance of a singer distinguished compared with others'. It would also be great to have a computational tool that helps me create my own music in a much easier way, while still allowing me to express my own thought and/or feeling.

These ideas lead to my three research directions!

Music signal analysis

I develop models that analyze music signal automatically. My previous work in this direction includes singing (note) transcription (ICASSP 2021, IEEE/ACM TASLP 2023), automatic lyrics transcription and lyrics(-to-audio) alignment (ASRU 2023, ICASSP 2024). The advancements on these tasks pave the way for future research on music signal analysis of higher-level concepts.

These tasks lead to practical applications such as automatic karaoke content generation, which requires time-aligned lyrics and (possibly) note transcription.

Symbolic music understanding

I also work on developing models that can automatically analyze and understand high-level concepts of symbolic music (e.g., MIDI files, sheet music). My recent works along this direction are now under peer review. Hopefully they will be published.

Music synthesis

I also work on music synthesis tasks such as singing voice synthesis (ASRU 2023) and singing performance style transfer (IEEE SPL 2025). I hope that in the future, we can have a music synthesis system that does not only generate natural music performance, but also generate expressive music performance which can be controlled or specified by human users freely and easily. This means we have to develop a system that can synthesize audio with a wide range of style/expressiveness (sometimes even goes beyond the actual singing/instruments), while also ensuring that such a system is highly controllable. I believe that such a system could change the digital music production in the future.

=======

Working experiences

  • Post-doctoral researcher, Music and Culture Technology Lab, Institude of Information Science, Academia Sinica (Aug. 2024 – )


Education experiences