EMO-Disentanger

🌈 Abstract

The paper proposes a two-stage Transformer-based model for emotion-driven piano performance generation. The first stage focuses on valence modeling via lead sheet (melody + chord) composition, while the second stage addresses arousal modeling by introducing performance-level attributes. The authors also propose a novel functional representation for symbolic music, which takes musical keys into account and encodes both melody and chords with Roman numerals relative to musical keys. Experiments demonstrate the effectiveness of their framework and representation on emotion modeling, and the method enables new capabilities to control the arousal levels of generation under the same lead sheet, leading to more flexible emotion controls.

🙋 Q&A

[01] Introduction

1. What are the key aspects of the proposed two-stage Transformer-based model?

• The first stage focuses on valence modeling via lead sheet (melody + chord) composition.
• The second stage addresses arousal modeling by introducing performance-level attributes such as articulation, tempo, and velocity.

2. What is the novel functional representation for symbolic music proposed in the paper?

• It takes musical keys into account and encodes both melody and chords with Roman numerals relative to musical keys, to consider the interactions among notes, chords and tonalities.
• This is proposed as an alternative to the popular REMI event representation that uses note pitch values and chord names.

3. What are the key findings from the experiments?

• The proposed framework and representation are effective in emotion modeling.
• The method enables new capabilities to control the arousal levels of generation under the same lead sheet, leading to more flexible emotion controls.

[02] Generation Samples

1. What are the three models compared in the generation samples?

• REMI (one): one-stage generation model with REMI representation, baseline
• REMI (two): two-stage generation model with REMI representation, one variant of the proposed framework
• Functional (two): two-stage generation model with functional representation, the main proposal

2. What is the purpose of showing piano performances with different arousal levels (low and high) based on the same lead sheet?

• This demonstrates a new emotion-based music generation application enabled by the two-stage framework, with either REMI or functional representation.

3. What were the findings regarding the REMI representation based on the user study?

• The REMI representation has poor performance in valence modeling.

[03] 4Q generations

1. What are the four quadrants (Q1-Q4) based on valence and arousal levels?

• Q1: High Valence, High Arousal
• Q2: Low Valence, High Arousal
• Q3: Low Valence, Low Arousal
• Q4: High Valence, Low Arousal

2. What are the key findings regarding the performance of the different models in each quadrant?

• The REMI representation has poor performance in valence modeling compared to the functional representation.