PiAnnotate: A Web Annotation Tool for Piano Fingering, with a Diagnostic Probe

In one sentence

A web tool for labelling piano fingering after performance, combining piano roll, video and a 3D MANO hand mesh, and storing paired rule-based vs. expert-edited tracks so the annotation history is auditable.

Abstract

Piano fingering shapes how a passage can be played, yet it is difficult to label after a performance. An annotator must decide which finger produced each note while reconciling the score, timing, video, and hand motion. We present PiAnnotate, a web-based pipeline for adding expert fingering annotations to the FurElise performance dataset. The tool brings together a piano-roll view, performance video, and a 3D MANO hand mesh so that reviewers can inspect each assignment in musical and physical context. Rather than storing only the final answer, PiAnnotate keeps paired rule-based and human-edited fingering tracks. These paired tracks make the annotation history auditable by showing where a geometric rule was sufficient, where experts intervened, and how labels changed across review passes. As a final diagnostic, we train a small Transformer probe on the paired tracks. The probe improves on the rule baseline on held-out pieces while remaining conservative about changing labels that were already correct, suggesting that the edited labels contain learnable structure rather than only isolated fixes.

Source: arXiv:2605.23982

Key facts

• Annotates the FurElise performance dataset
• Keeps paired rule-based and human-edited fingering tracks
• A small Transformer probe trained on the pairs beats the rule baseline on held-out pieces

When this paper is relevant

• You are building a piano fingering dataset or need a fingering annotation tool
• You design human-in-the-loop annotation where experts correct rule-based labels
• You need a way to audit annotation quality and revision history
• You look for uses of the MANO hand mesh in music performance analysis

Related

• SKY-Piano: A Multimodal Piano Performance Dataset (2026)
• PianoVAM: A multimodal piano performance dataset (2025)
• Two Web Toolkits for Multimodal Piano Performance Dataset Acquisition and Fingering Annotation (2025)

Cite

Joonhyung Bae, K. Kim, H. Cho, S. Lee, Y.S. Choi, H. Hur, G. Lee, A. Maezawa, J. Park, J. Park, J. Nam (2026). PiAnnotate: A Web Annotation Tool for Piano Fingering, with a Diagnostic Probe. Proceedings of the 2026 Korean Society for Music Informatics (KSMI) Conference. https://arxiv.org/abs/2605.23982

@inproceedings{Bae2026piannotate,
  title = {PiAnnotate: A Web Annotation Tool for Piano Fingering, with a Diagnostic Probe},
  author = {Joonhyung Bae and K. Kim and H. Cho and S. Lee and Y.S. Choi and H. Hur and G. Lee and A. Maezawa and J. Park and J. Park and J. Nam},
  booktitle = {Proceedings of the 2026 Korean Society for Music Informatics (KSMI) Conference},
  year = {2026},
  eprint = {2605.23982},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2605.23982}
}

Machine-readable copy: /papers/piannotate-2026.md