Virtual Piano
Creative technology
I built Virtual Piano from December 2019 to March 2020 to test the Oculus Quest hand tracking soon after it became available. The experiment lets you play without controllers and align the virtual keyboard with a physical table.
Testing a new input method
Hand tracking was new on the Quest, and I wanted to understand what could be built with it. I designed and created the complete experiment myself. Players could use their hands directly, at a time when most Quest interaction still depended on controllers.
Aligning the piano
The most difficult part was the positioning experience. Virtual Piano has separate play and positioning modes, selected by changing the page in the sheet-music book. Different pinch gestures adjust the height, horizontal position, depth and rotation so the virtual keyboard can be placed over a real table. The interaction worked, but it never became as clear or precise as I wanted. There were few established patterns to follow because both the hardware and its hand-tracking interaction were new.
The physical limitation
The table gives the fingers a surface to meet, but it cannot reproduce the movement and resistance of individual piano keys. The early Quest tracking also introduced a noticeable delay. Hand tracking has improved since 2020, but in my experience the combination is still better suited to a simple song than serious piano playing.
A widely tried experiment
Other people treated Virtual Piano as a novelty and had fun trying an interaction they had not encountered before. By September 2026, SideQuest recorded 100,355 downloads and 144,044 views for the alpha release. The figures show that many people tried the experiment, while the alpha label and the interaction limitations remain an important part of its context.
What I would improve
I would redesign the alignment flow, make the distinction between positioning and playing clearer, and give players better guidance on how the gestures work. Better contemporary tracking would reduce some delay, but the absence of physical keys would still set the limit for what the experience can reproduce.
Designed and built by Edvard Pires Bjørgen from December 2019 to March 2020. Publicly released as VirtualPiano (Alpha) on SideQuest.