Catherine Jou, Ann Lin, Elsa Ho
Play With The App
Click here to play with the app.
Note: multiple users can play collaboratively on different browsers.
The source code is available here on GitHub.
- Keys '0', '1', '2', '3', '4', '5', '6', '7', '8', '9', '-': play a note
- Left/Right Arrows: move ship left/right
- KeyDown Event: ship changes to a random color
- Space Bar: switch between instrument modes
- Clicking on Star: turn soundboard recordings on/off
- Microphone Input: cursor emits bubbles whose colors vary based on the volume of input
- Mobile Acceleration Input: click here on your mobile device, enter 'artisan' and shake the device to make the waves rock
process — 4 Stages
Stage 1: Animation
Stage 2: Interaction
The second iteration explored response to user input, which turned the piece into an ocean piano with each number key mapped to a note and a different colored fish that would swim across the screen. The fish are mapped to piano keys in the scale of C major, and they appear at the appropriate height among the lines formed by waves for where that note would appear on a piece of sheet music (in treble clef). Number keys play the fish "notes", the color of the ship changes on mouse-click, and the left and right arrows move the ship left and right. Additionally, each mouse-move event that is underwater creates bubbles that spout from the cursor and get smaller as they rise.
Stage 3: Networking
The third iteration was networked to allow multiple users to interact with the piece by playing in different instrument modes that communicate with each other via a Heroku server. This extended our art piece into an app that allows people to compose a music piece together over the web. To enhance the experience, we added different instrument modes that users can choose his/her own instrument as they play together on the app. Everyone using the app at the same moment can see and hear the same music and move the ship together. Additionally, we the made the ship change color (as if there is a rave going on on-board) on every key-press instead of on mouse-click.
Stage 4: Sensors
The fourth iteration incorporated response to sensors. We added audio and mobile acceleration input so that shaking the phone made the waves choppy and the fish jumble around. We thought utilizing the accelerometer in mobile devices would be a great idea because the action of shaking the phone is like shaking a tambourine, which adds fun and physical element to our music app. The stars also became soundboard buttons that can play backtracks that provide a beat for people to groove to. Bubbles also appear from the mouse position when there is audio input from the mic, and they change color depending on volume of sound picked up by the mic.
With all these changes, this art piece managed to evolve from a standalone animation, to a simple piano that only one person could play with, to a music simulator with which people can randomly have fun co-creating a piece of music.
Some thoughts on how to further develop this piece include: adding more instruments to increase the richness of the music that can be created, re-introducing the sky changing animation but mapping it to the user's real time-of-day, adding a chat feature that lets users communicate verbally (or sing) as they hang out with others on the app, providing a feature that can allow users to record a snippet of a session for playback and layering, and allowing the user to record a session and save it on their own device to share with others.