SweetSpot

A VR-powered instrument for exploration, creativity and immersive sound

Built in Unity for Oculus Quest 2, this project introduces VIME (VR Interface for Music Expression)—a hands-on, interactive way to create and experience music. Users can experiment, compose, and remix sounds in a virtual environment, redefining how music is played and felt.

Inspired by the idea of an "impossible instrument," the design blends ambient and generative music, encouraging listening, discovery, and improvisation in a fully immersive soundscape.

🏆 Awarded #1 Most Popular Project at the Immersive Media Showcase 2022 among 22 projects at the University of Michigan.

ROLE

Solo Experience Designer - Concept Design, Gameplay Design, Interaction Design

TEAM

Yolanda Yu - Product Designer

David Minnix - Software Engineer

Clayton Izuka - Audio Engineer

Tool

Unity 3D, Resonance Audio, Blender, C#

XR Device

Oculus Quest

TIMELINE

1 month - Nov 2022

DEMO

Turn on your volume at 00:45 🔊

CHALLENGE

Designing an environment as an instrument, new and "impossible".

The interface should be designed for VR, leveraging its interactive, spatial, and sensory capabilities to create an engaging musical experience. It can follow established musical traditions, mental models, and sensorimotor behaviors or introduce a completely novel approach. The scale of the interface should function as a cohesive musical system, seamlessly integrating sound, interaction, and visualization.

CONTRIBUTION

What I am responsible for?

I led the concept development, vision, and UI/UX design, crafting an immersive greenhouse environment with intuitive VR interactions. I designed natural interactions, such as grabbing seed bags to plant crops, pouring seeds to trigger growth, and using objects like trash bins to create ambient sound effects. Additionally, I implemented diegetic UI elements, including wooden notice boards and vegetable crates, to enhance user immersion and familiarity.

Concept & research

How did we generate this concept?

A question rises in my mind "how to define an instrument?"  It reminds me that some artists usually make unique sounds out of everyday objects. For example, the Vegetable Orchestra performs on instruments made of fresh vegetables, Bruno Zamborlin transform the whole bicycle into a music interface for interaction, Trashbeatz even created the beat out of trash, etc. These inspire me "wow" might come from an "everyday" surprise.

 

To make abstract concept more intuitive, I proposed to consolidate all ideas together and set the concept of a rainy garden with a beautiful greenhouse and many seed beds in the open space for the sake of aesthetic and discovery user experience.  Players can imagine themselves as a gardener to create their own music by planting vegetables, managing the trash and observing growth changes.

Game mechanics

Players are immersed in a serene rainy garden environment. They have the power to cultivate a harmonious soundscape by strategically planting seeds and placing trash cans throughout the garden.

INTERACTION DESIGN

01

Grab Seed Bag

Hold/Release the grip button on your controller to grab seed bags.

02

Pour Seeds Out

The more you tilt, the more seeds out. The speed will variate depending on how much you tilt.

03

Plant Your Seed in Raise Beds

The seed play its unique sound and turns to the vegetables when it falls on the soil.

04

Rain on Cucumber

It growths and play the unique sound when absorbing raindrop.

05

Drum Beat By Trash Can

When trash bins are moved out of the shed, rain drops on them made the different beats.

Environment Design

Demo1. Rain Effect

Demo2. Rain Effect

Demo3. Greenhouse storing 7 different music seed bags

Reflection

If I had more time, how to improve the user experience?

01.More intuitive onboarding experience for first users

Enhance clarity by highlighting intractable objects and leveraging radio cues for instructions. Future versions could introduce glowing objects or additional interactive props.

02.Add a Screenshot Feature

Users loved the experience and frequently asked to capture their gardens. Adding a screenshot or video feature would enhance engagement and open possibilities for gamified music education.