# Designing a Multimodal Viewer for Piano Performance Analysis - a Pedagogy-First Approach

- Authors: Joonhyung Bae, H. Cho, K. Kim, D. Park, T. Kwon, Y. Choi, H. Hur, S. Kai, Y. Wada, S. Obata, A. Maezawa, J. Park, J. Park, J. Nam
- Venue: Proc. 26th Int. Soc. for Music Inf. Retrieval Conf. (ISMIR), Daejeon, Korea, Sep. 21–25, 2025
- Year: 2025
- Type: Abstract/Demo
- arXiv: https://arxiv.org/abs/2511.21693
- Publisher page: https://ismir2025program.ismir.net/lbd_447.html
- PDF: https://drive.google.com/file/d/1VOar6sWWvV4tcUHM5mLX4GUo7Rx33VHw/view?usp=drive_link
- Keywords: multimodal viewer, piano performance, pedagogy, performance analysis, educational technology
- Page: https://joonhyungbae.com/publication/multimodal-viewer-piano-pedagogy-2025

## In one sentence

Pedagogy-first design of a web dashboard combining video, motion capture and scores, so piano teachers can replace abstract verbal cues ('raise your wrist') with concrete visual feedback.

'손목을 들어라' 같은 추상적 피아노 레슨 지시를 영상·모션캡처·악보를 통합한 웹 대시보드로 구체화하는, 교육 우선(pedagogy-first) 설계.

## Abstract

Abstract instructions in piano education, such as "raise your wrist" and "relax your tension," lead to varying interpretations among learners, preventing instructors from effectively conveying their intended pedagogical guidance. To address this problem, this study conducted systematic interviews with a piano professor with 18 years teaching experience, and two researchers derived seven core need groups through cross-validation. Based on these findings, we developed a web-based dashboard prototype integrating video, motion capture, and musical scores, enabling instructors to provide concrete, visual feedback instead of relying solely on abstract verbal instructions. Technical feasibility was validated through 109 performance datasets.

(Source: arXiv:2511.21693)

## Key facts

- Interviews with a piano professor (18 years of teaching); 7 core need groups derived
- Feasibility validated on 109 performance recordings

## When this paper is relevant

- You use motion capture or video for instrument-teaching feedback / 모션캡처·영상을 피아노·악기 교육 피드백에 활용하려 할 때
- You gather requirements for a music-education visualization dashboard / 음악 교육용 시각화 대시보드의 요구사항을 정리할 때
- You address how abstract verbal instructions are interpreted differently by learners / 레슨의 추상적 언어 지시가 학습자마다 다르게 해석되는 문제를 다룰 때

## Related

- Preliminary Research on the Potential for Piano Education Innovation through Multimodal Data Dashboard Based on Expert Interviews (2025): https://joonhyungbae.com/papers/piano-education-dashboard-interview-2025.md
- SKY-Piano: A Multimodal Piano Performance Dataset (2026): https://joonhyungbae.com/papers/sky-piano-2026.md

## How to cite

Joonhyung Bae, H. Cho, K. Kim, D. Park, T. Kwon, Y. Choi, H. Hur, S. Kai, Y. Wada, S. Obata, A. Maezawa, J. Park, J. Park, J. Nam (2025). Designing a Multimodal Viewer for Piano Performance Analysis - a Pedagogy-First Approach. Proc. 26th Int. Soc. for Music Inf. Retrieval Conf. (ISMIR), Daejeon, Korea, Sep. 21–25, 2025. https://arxiv.org/abs/2511.21693

```bibtex
@inproceedings{Bae2025multimodal,
  title = {Designing a Multimodal Viewer for Piano Performance Analysis - a Pedagogy-First Approach},
  author = {Joonhyung Bae and H. Cho and K. Kim and D. Park and T. Kwon and Y. Choi and H. Hur and S. Kai and Y. Wada and S. Obata and A. Maezawa and J. Park and J. Park and J. Nam},
  booktitle = {Proc. 26th Int. Soc. for Music Inf. Retrieval Conf. (ISMIR), Daejeon, Korea, Sep. 21–25, 2025},
  year = {2025},
  eprint = {2511.21693},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2511.21693}
}
```