# PiAnnotate: A Web Annotation Tool for Piano Fingering, with a Diagnostic Probe

- Authors: Joonhyung Bae, K. Kim, H. Cho, S. Lee, Y.S. Choi, H. Hur, G. Lee, A. Maezawa, J. Park, J. Park, J. Nam
- Venue: Proceedings of the 2026 Korean Society for Music Informatics (KSMI) Conference
- Year: 2026
- Type: Conference Paper
- arXiv: https://arxiv.org/abs/2605.23982
- Keywords: piano fingering, web annotation, diagnostic probe, annotation tool, music informatics
- Page: https://joonhyungbae.com/publication/piannotate-2026

## In one sentence

A web tool for labelling piano fingering after performance, combining piano roll, video and a 3D MANO hand mesh, and storing paired rule-based vs. expert-edited tracks so the annotation history is auditable.

연주 후 피아노 운지를 라벨링하는 웹 도구. 피아노롤·영상·3D 손 메시를 한 화면에 두고, 규칙 기반 운지와 전문가 수정본을 짝으로 저장해 주석 이력을 감사 가능하게 만든다.

## Abstract

Piano fingering shapes how a passage can be played, yet it is difficult to label after a performance. An annotator must decide which finger produced each note while reconciling the score, timing, video, and hand motion. We present PiAnnotate, a web-based pipeline for adding expert fingering annotations to the FurElise performance dataset. The tool brings together a piano-roll view, performance video, and a 3D MANO hand mesh so that reviewers can inspect each assignment in musical and physical context. Rather than storing only the final answer, PiAnnotate keeps paired rule-based and human-edited fingering tracks. These paired tracks make the annotation history auditable by showing where a geometric rule was sufficient, where experts intervened, and how labels changed across review passes. As a final diagnostic, we train a small Transformer probe on the paired tracks. The probe improves on the rule baseline on held-out pieces while remaining conservative about changing labels that were already correct, suggesting that the edited labels contain learnable structure rather than only isolated fixes.

(Source: arXiv:2605.23982)

## Key facts

- Annotates the FurElise performance dataset
- Keeps paired rule-based and human-edited fingering tracks
- A small Transformer probe trained on the pairs beats the rule baseline on held-out pieces

## When this paper is relevant

- You are building a piano fingering dataset or need a fingering annotation tool / 피아노 운지(fingering) 데이터셋을 만들거나 라벨링 도구가 필요할 때
- You design human-in-the-loop annotation where experts correct rule-based labels / 규칙 기반 자동 라벨 위에 사람 수정을 얹는 human-in-the-loop 주석 설계를 고민할 때
- You need a way to audit annotation quality and revision history / 주석 품질·이력의 감사 가능성(auditability)을 평가하는 방법을 찾을 때
- You look for uses of the MANO hand mesh in music performance analysis / MANO 손 메시를 음악 연주 분석에 쓰는 예를 찾을 때

## Related

- SKY-Piano: A Multimodal Piano Performance Dataset (2026): https://joonhyungbae.com/papers/sky-piano-2026.md
- PianoVAM: A multimodal piano performance dataset (2025): https://joonhyungbae.com/papers/pianovam-2025.md
- Two Web Toolkits for Multimodal Piano Performance Dataset Acquisition and Fingering Annotation (2025): https://joonhyungbae.com/papers/two-web-toolkits-2025.md

## How to cite

Joonhyung Bae, K. Kim, H. Cho, S. Lee, Y.S. Choi, H. Hur, G. Lee, A. Maezawa, J. Park, J. Park, J. Nam (2026). PiAnnotate: A Web Annotation Tool for Piano Fingering, with a Diagnostic Probe. Proceedings of the 2026 Korean Society for Music Informatics (KSMI) Conference. https://arxiv.org/abs/2605.23982

```bibtex
@inproceedings{Bae2026piannotate,
  title = {PiAnnotate: A Web Annotation Tool for Piano Fingering, with a Diagnostic Probe},
  author = {Joonhyung Bae and K. Kim and H. Cho and S. Lee and Y.S. Choi and H. Hur and G. Lee and A. Maezawa and J. Park and J. Park and J. Nam},
  booktitle = {Proceedings of the 2026 Korean Society for Music Informatics (KSMI) Conference},
  year = {2026},
  eprint = {2605.23982},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2605.23982}
}
```