SweetSpot
A VR-powered instrument for exploration, creativity and immersive sound
Built in Unity for Oculus Quest 2, this project introduces VIME (VR Interface for Music Expression)—a hands-on, interactive way to create and experience music. Users can experiment, compose, and remix sounds in a virtual environment, redefining how music is played and felt.
Inspired by the idea of an "impossible instrument," the design blends ambient and generative music, encouraging listening, discovery, and improvisation in a fully immersive soundscape.
🏆 Awarded #1 Most Popular Project at the Immersive Media Showcase 2022 among 22 projects at the University of Michigan.
ROLE
Solo Experience Designer - Concept Design, Gameplay Design, Interaction Design
TEAM
Yolanda Yu - Product Designer
David Minnix - Software Engineer
Clayton Izuka - Audio Engineer
Tool
Unity 3D, Resonance Audio, Blender, C#
XR Device
Oculus Quest
TIMELINE
1 month - Nov 2022
DEMO
Turn on your volume at 00:45 🔊
CHALLENGE
Designing an environment as an instrument, new and "impossible".
The interface should be designed for VR, leveraging its interactive, spatial, and sensory capabilities to create an engaging musical experience. It can follow established musical traditions, mental models, and sensorimotor behaviors or introduce a completely novel approach. The scale of the interface should function as a cohesive musical system, seamlessly integrating sound, interaction, and visualization.
CONTRIBUTION
What I am responsible for?
I led the concept development, vision, and UI/UX design, crafting an immersive greenhouse environment with intuitive VR interactions. I designed natural interactions, such as grabbing seed bags to plant crops, pouring seeds to trigger growth, and using objects like trash bins to create ambient sound effects. Additionally, I implemented diegetic UI elements, including wooden notice boards and vegetable crates, to enhance user immersion and familiarity.
Concept & research


How did we generate this concept?
A question rises in my mind "how to define an instrument?" It reminds me that some artists usually make unique sounds out of everyday objects. For example, the Vegetable Orchestra performs on instruments made of fresh vegetables, Bruno Zamborlin transform the whole bicycle into a music interface for interaction, Trashbeatz even created the beat out of trash, etc. These inspire me "wow" might come from an "everyday" surprise.
To make abstract concept more intuitive, I proposed to consolidate all ideas together and set the concept of a rainy garden with a beautiful greenhouse and many seed beds in the open space for the sake of aesthetic and discovery user experience. Players can imagine themselves as a gardener to create their own music by planting vegetables, managing the trash and observing growth changes.
Game mechanics
Players are immersed in a serene rainy garden environment. They have the power to cultivate a harmonious soundscape by strategically planting seeds and placing trash cans throughout the garden.

INTERACTION DESIGN
01
Grab Seed Bag
Hold/Release the grip button on your controller to grab seed bags.
02
Pour Seeds Out
The more you tilt, the more seeds out. The speed will variate depending on how much you tilt.
03
Plant Your Seed in Raise Beds
The seed play its unique sound and turns to the vegetables when it falls on the soil.
04
Rain on Cucumber
It growths and play the unique sound when absorbing raindrop.
05
Drum Beat By Trash Can
When trash bins are moved out of the shed, rain drops on them made the different beats.
Environment Design
Demo1. Rain Effect
Demo2. Rain Effect
Demo3. Greenhouse storing 7 different music seed bags
Reflection
If I had more time, how to improve the user experience?
01.More intuitive onboarding experience for first users
Enhance clarity by highlighting intractable objects and leveraging radio cues for instructions. Future versions could introduce glowing objects or additional interactive props.
02.Add a Screenshot Feature
Users loved the experience and frequently asked to capture their gardens. Adding a screenshot or video feature would enhance engagement and open possibilities for gamified music education.