Project Description

SeaScales is an interactive, animated web art piece made using the p5.js JavaScript library. The project started out as a simple exploration into creating animations for the web, and over time evolved into an ocean-themed electronic keyboard that can be used to co-create melodies online with your friends. 




Catherine Jou, Ann Lin, Elsa Ho

Play With The App


Click here to play with the app.

Note: multiple users can play collaboratively on different browsers.

The source code is available here on GitHub.


  • Keys '0', '1', '2', '3', '4', '5', '6', '7', '8', '9', '-': play a note
  • Left/Right Arrows: move ship left/right
  • KeyDown Event: ship changes to a random color
  • Space Bar: switch between instrument modes
  • Clicking on Star: turn soundboard recordings on/off
  • Microphone Input: cursor emits bubbles whose colors vary based on the volume of input
  • Mobile Acceleration Input: click here on your mobile device, enter 'artisan' and shake the device to make the waves rock

process — 4 Stages

Stage 1: Animation

Looking at the sea can make people feel calm. In Seattle, it is hard for busy people without cars to get to the ocean. Therefore, we first envisioned this art piece to be a little aquarium that calms an audience down with the gentle movement of the waves. The first iteration was a standalone animation piece, with waves animated to a Perlin Noise function, fish swimming across the screen, urchins and sea stars dancing under the waves, and a simulated 24-hour color-changing sky. All animations were done using the p5.js JavaScript library.

Stage 2: Interaction

The second iteration explored response to user input, which turned the piece into an ocean piano with each number key mapped to a note and a different colored fish that would swim across the screen. The fish are mapped to piano keys in the scale of C major, and they appear at the appropriate height among the lines formed by waves for where that note would appear on a piece of sheet music (in treble clef). Number keys play the fish "notes", the color of the ship changes on mouse-click, and the left and right arrows move the ship left and right. Additionally, each mouse-move event that is underwater creates bubbles that spout from the cursor and get smaller as they rise.

Stage 3: Networking

The third iteration was networked to allow multiple users to interact with the piece by playing in different instrument modes that communicate with each other via a Heroku server. This extended our art piece into an app that allows people to compose a music piece together over the web. To enhance the experience, we added different instrument modes that users can choose his/her own instrument as they play together on the app. Everyone using the app at the same moment can see and hear the same music and move the ship together. Additionally, we the made the ship change color (as if there is a rave going on on-board) on every key-press instead of on mouse-click.

Stage 4: Sensors

The fourth iteration incorporated response to sensors. We added audio and mobile acceleration input so that shaking the phone made the waves choppy and the fish jumble around. We thought utilizing the accelerometer in mobile devices would be a great idea because the action of shaking the phone is like shaking a tambourine, which adds fun and physical element to our music app. The stars also became soundboard buttons that can play backtracks that provide a beat for people to groove to. Bubbles also appear from the mouse position when there is audio input from the mic, and they change color depending on volume of sound picked up by the mic. 

With all these changes, this art piece managed to evolve from a standalone animation, to a simple piano that only one person could play with, to a music simulator with which people can randomly have fun co-creating a piece of music.


This was a fun project that allowed me to hone my JavaScript skills as well as learn how to use the p5.js library to create web animations. As the developer for this project, my primary responsibility was to code the piece, however, I was also involved other aspects of the project such as coming up with new ideas, collecting pieces of media used for this project, and editing them via Adobe Photoshop and Audition to suit the needs of the app. It was also interesting to watch how it organically evolved. While the project started out as a simple exploration into creating animations, it eventually evolved into an ocean-themed electronic keyboard with which people can co-create a beautiful piece of music.

Some thoughts on how to further develop this piece include: adding more instruments to increase the richness of the music that can be created, re-introducing the sky changing animation but mapping it to the user's real time-of-day, adding a chat feature that lets users communicate verbally (or sing) as they hang out with others on the app, providing a feature that can allow users to record a snippet of a session for playback and layering, and allowing the user to record a session and save it on their own device to share with others.