by Joohyun Park, Chengchao Zhu

Interactive waterfall invites users to virtuality and reality, or somewhere between.


09. 2017 – 12. 2017



Interaction deign (Javascript, Arduino, Leap motion)
Visual design
Idea development


This project is designed to provide twisted experience partly based on physics and daily behavior. Being inspired by oriental painting and Jean Baudrillard’s simulacre, we tried to bring unexpected moments that will break user’s mental map by creating multiple layers between virtuality and reality.



90% of participants were satisfied with the twisted moment. This result proceeded us to the next question: How to create more interaction to make users stay longer? Through the first brief user test, we drew below insights as additions.

  1. Use of real water for unexpected layer
  2. Additional input source for interaction
  3. Clear tone & manner through abstract visual



  1. Use visual on screen as a trigger point for further interaction along with real water.
  2. Combine all devices in one interface.

Animation: Unity


Unity is the most attractive tool that I’ve learned this semester. I decided to focus on Unity through my academic years. As a beginning, I will study and explore Unity during winter vacation. In this assignment, I succeeded to implement control panel, following camera, and randomly changing color along with object collision. Below is “Joohyun World” and I would like to invite you my bizarre world.


I imagined to use sphere as a main character, and made covered it with earth texture. The background is covered with under sea image so that the whole place could emit the distinctive atmosphere. Earth moves with the direction button, and user can explore the space. Several characters are made in unusual tone & manner, such as dancing soldier and vampire, zombie, and fish.

I made a interaction cube whose color changes randomly whenever it meets Earth. I searched the collide detection, random code is used.





Final Project : Encounter

<The Encounter> : A close encounter with someone is the moment when each universe meets.


The reflection of users on the screen is filled with glittering stars and creates an unexpected moment when they are in close position to each other. As an engaging interactive installation for one person, or as a chance to the close encounter for two people, this project aims to provide users an experience of emotional contact generated by physical touch. Processing and kinect v2 are used.








Demo video (screen ver.)




Ideation Process





Project inspired by contents below.





Question for Users

_ If you have made gravity(change) on the screen, what was your trigger point for that behavior? (What made you play with the screen?)

_ What did you feel when you were standing very close to someone, even a friend or a stranger?

_ What do you feel after experiencing this project with someone else?


Documentation of short video

Title : Contact

Concept : Inspired by the contents of golden records which is loaded up in NASA’s Voyager. Masking and key light function from after effects are mainly used.


NASA’s golden record

Golden record : https://voyager.jpl.nasa.gov/golden-record/whats-on-the-record/



Our intention is to make the twisted ending. So, the film starts with the ECU(extreme close up) size and slowly moves back to capture the scenes out of the frame. Sounds from the real golden record are partly used and all video resources are originated from the contents of golden record.



Final version <Golden Record>




Final Project Concept & Interface

I was supposed to go in team(History of Communication was our presentation subject), but after a long discussion, I decided to go alone. This is because we consist of three people and failed to fine the suitable balance in terms of scale and idea which will suit with the quality from three people. Even we spent a week for idea development, and there were lots of great ideas which were beyond our current ability, I consider those time as a resource for future works.

After that, I did a intense ideation. I tried to bring subject from my surroundings, and came up with this idea. “Touch” or “Contact”. I would like to express the great moment when people hug each other, or touch somebody. I will use kinect v2 to capture people’s motion, and processing.


The shapes of people will be reflected on the screen, covered with universe sketch image. And if people get closer and hug each other, firework particles will appear randomly. I will use the average point of each people by dividing the screen in two areas so that program can recognize the points as different.

For now, I succeed to draw the universe image inside the shadow, and detecting two average points of users.



  1. How can I make users hug each other?
  2. What other interaction points can be added? e.g. sound


I would like get feedback on these two things at the user test.

Week 3: conceptualization & brainstorming

Ideation for short animation film

Greatly inspired by Andrew B Meyers’ work, I would like to variate this concept for my project. What I need is masking skill, and abstract storyline to make stylish feelings.

What I’ve imagined is fixed frame positioned inside the galley, and the lots of diverse variations inside the frame. The core concept comes from “Depaysement” which is the coexistence of objects that cannot be placed together in real life. By putting different objects and environments inside the frame, I would like to create the distinct feelings.


Every transition of content would be led by audience crossing the frame, or other external elements moving one side to another side of the frame.



Through the user test we obtained significant feedback and insight.


// Feedbacks

  • When real water spills out, I want to play with virtual fish which appears on p5 sketch.
  • Splash real to splash virtual / cut real water to cut virtual
  • Fishes say some word when they are touched
  • Make changes in real water movement (e.g. making wave)
  • Change the color of the water (p5 sketch)
  • Flash with quick movement / change on speeds
  • When I swipe animals such as fishes come out
  • Close the valve if real water fall because its spilling! – but fish will make me touch it (or close the valve)
  • I will touch depending on how realistic the fish
  • Depends on my mood. (Don’t want getting wet)
  • Why there is a lever button? You don’t need it.
  • Lights will make me touch the water.
  • Playing with water. expected and want more unexpected moment.


// Keyword

  • fish (as a trigger point to interaction with real water)/ color / splash


// Insight

  • How to let users touch the water? or know that they can touch the water?


Also, we tested to figure out if IR light and film-attached camera will work, but it didn’t work. We tried with different filters which has different thickness, but it didn’t work as well nor it was not stable enough to minutely capture user’s hand movement. We are currently considering using kinect v2 to capture the motion for interaction. The details are on process. Let’s get it.


Pre-production for Final


 – Idea : make users interact with virtual water, and trigger unexpected moment with REAL water as well as further interaction with that.

 – Tone & manner : rough and factory-style appearance / use of normal objects such as valve and water lever as physical input

 – Goal : Create layers of unexpected moment.

1) virtual waterfall when turn on the valve – user expects something real water just like daily life, but the outcome is digital.

2) real water spills when the surface of virtual water meets the certain point.

3) when user touch the real water, some interaction happens in virtual water

Interface sketch



Brainstorming for Final

History of Communication

Our final idea is based on the history of communication and media. We would like to use physical inputs and digital outputs through conceptual context.


How to create suitable visual output. We temporary decided to put human image on the screen with the different dimensions, but I think symbolic expression is better. We will explore further ideas.

Idea development still on process.


Week 2: Character design & storyboard

character design


I would like to make a simple story of alien in a fantasy world(virtual world). What I’ve imagined is oriental character expressed as spiritual object such as ghost, not in horror but through humanistic context. So, it has to be cute and engaging. I referred to the existing character from スタジオジブリ|STUDIO GHIBLI ‘s animation. I would like to combine the shape of Kaonashi, and the face of Korean traditional mask, to create the whole new character.


This character lives in the virtual world that has nothing but vacant and infinite place. I would like to create simple story with these elements. The output of the short film is expected to has no dialogue so story must be proceeded with behavior and desire of the character.


2D character final