Positioning Left-hand Movement in Violin Performance: A System and User Study of Fingering Pattern Generation

Title of the Paper

Positioning Left-hand Movement in Violin Performance: A System and User Study of Fingering Pattern Generation

Bibliographic Information

• Field of Study: Computer-assisted music performance; specifically, violin fingering generation and evaluation
• Keywords:
  • Violin fingering generation
  • Human-computer interaction systems
  • Deep learning
  • Neural networks
  • Dataset
  • Expressive music
  • Music learning
  • User study
  • Artificial intelligence
  • Hand movement generation

Research Background and Problem

• Identified Problems or Challenges:
  • The selection of violin fingering involves complex musical knowledge and is a highly subjective process closely related to the performer's physical conditions, musical preferences, and skill level.
  • Current automated fingering generation methods (e.g., based on Hidden Markov Models and Conditional Random Fields) often rely on predefined hidden states, lack adaptability, and fail to meet diverse styles and needs.
• Why the Problem is Important:
  • Providing performers (especially beginners) with suitable fingering selection tools can enhance teaching efficiency and improve playing comfort and expressiveness.
• Research Motivation and Related Work:
  • The limitations of existing methods (e.g., estimating the "most playable" fingering) restrict the interactivity and diversity of musical fingering generation.
  • There is a need to address the lack of multi-version annotated data and provide more flexible interactive tools.

Solution

• Method or Solution:
  • Propose a system composed of three modular components that utilize Bidirectional Long Short-Term Memory Networks (BLSTM-RNNs) to generate left-hand fingering: ① String Assignment Module, ② Hand Position Module, and ③ Fingering Selection Module.
  • Users can participate in the generation process by specifying input parameters (e.g., strings used or specific patterns), forming a "human-computer collaboration" model.
• Innovations:
  • Provide a multi-path generation framework that allows users to control details, greatly enhancing the flexibility and adaptability of fingering selection.
  • Design and utilize a new dataset with comprehensive fingering annotations, supporting different user habits and styles.
• Implementation Steps and Key Technologies:
  1. Develop a dataset containing 10 pieces of Baroque, Classical, Romantic, and folk music, annotated by 10 professional violinists.
  2. Use BLSTM networks to output probability distributions for string assignment, hand position, and fingering selection.
  3. Flexibly adjust generation schemes through mode selection (basic, minimal, closest) and support manual corrections.
  4. Achieve efficient, customized fingering generation through user evaluation and multi-version path exploration.

Research Results

• Specific Outcomes:
  • Developed an interactive violin fingering generation system capable of generating different fingering arrangements based on performance needs while allowing user adjustments.
  • The dataset includes 10 pieces, totaling 217,690 notes, and the project source code and dataset have been made publicly available.
• Advantages Compared to Existing Solutions:
  • More flexible than traditional HMM and CRF methods, supporting user input and multi-path generation.
  • Objective evaluations show high accuracy in generating correct fingerings, with recommended results ranking highly (e.g., the MRR of the string assignment module reached 91.3%).
• Experimental or Evaluation Results:
  • Objective Testing:
    • The F1 scores for the basic mode were 66.73% for string assignment, 24.07% for hand position, and 41.18% for fingering selection, with better performance under multi-output schemes (e.g., using Mean Reciprocal Rank, MRR).
  • Subjective Testing:
    • Eight performers with an average of 14 years of violin experience participated in the evaluation.
    • Different modes (basic, minimal, closest) showed advantages in transition ease, expressiveness, naturalness, and skill applicability, adapting to various scenarios.
    • Comparisons indicate that the new system performs similarly to the baseline system in reference [15], with potential to surpass it in certain metrics.
• Limitations and Future Directions:
  • Due to the small sample size, statistical significance is limited, requiring further experimental validation.
  • Future work could involve personalized data training to generate specific artists' playing styles or extend to fingering generation for other musical instruments.
  • Aim to develop new methods related to musical expression and animation generation, inspiring more creative applications in the field of computer music.