by Joohyun Park, Chengchao Zhu

Interactive waterfall invites users to virtuality and reality, or somewhere between.


09. 2017 – 12. 2017



Interaction deign (Javascript, Arduino, Leap motion)
Visual design
Idea development


This project is designed to provide twisted experience partly based on physics and daily behavior. Being inspired by oriental painting and Jean Baudrillard’s simulacre, we tried to bring unexpected moments that will break user’s mental map by creating multiple layers between virtuality and reality.



90% of participants were satisfied with the twisted moment. This result proceeded us to the next question: How to create more interaction to make users stay longer? Through the first brief user test, we drew below insights as additions.

  1. Use of real water for unexpected layer
  2. Additional input source for interaction
  3. Clear tone & manner through abstract visual



  1. Use visual on screen as a trigger point for further interaction along with real water.
  2. Combine all devices in one interface.


Through the user test we obtained significant feedback and insight.


// Feedbacks

  • When real water spills out, I want to play with virtual fish which appears on p5 sketch.
  • Splash real to splash virtual / cut real water to cut virtual
  • Fishes say some word when they are touched
  • Make changes in real water movement (e.g. making wave)
  • Change the color of the water (p5 sketch)
  • Flash with quick movement / change on speeds
  • When I swipe animals such as fishes come out
  • Close the valve if real water fall because its spilling! – but fish will make me touch it (or close the valve)
  • I will touch depending on how realistic the fish
  • Depends on my mood. (Don’t want getting wet)
  • Why there is a lever button? You don’t need it.
  • Lights will make me touch the water.
  • Playing with water. expected and want more unexpected moment.


// Keyword

  • fish (as a trigger point to interaction with real water)/ color / splash


// Insight

  • How to let users touch the water? or know that they can touch the water?


Also, we tested to figure out if IR light and film-attached camera will work, but it didn’t work. We tried with different filters which has different thickness, but it didn’t work as well nor it was not stable enough to minutely capture user’s hand movement. We are currently considering using kinect v2 to capture the motion for interaction. The details are on process. Let’s get it.


Pre-production for Final


 – Idea : make users interact with virtual water, and trigger unexpected moment with REAL water as well as further interaction with that.

 – Tone & manner : rough and factory-style appearance / use of normal objects such as valve and water lever as physical input

 – Goal : Create layers of unexpected moment.

1) virtual waterfall when turn on the valve – user expects something real water just like daily life, but the outcome is digital.

2) real water spills when the surface of virtual water meets the certain point.

3) when user touch the real water, some interaction happens in virtual water

Interface sketch



Final Project Idea

Option 1 ; Develop the mid-term project

+ add sound that is symbolic in terms of water or liquid tone.

+ using real water as the horizon meets the top part of the transparent container

+ updating valve with bigger one to engage users



Option 2 ; new one

1) color marble wall (or table?) with interaction

  • Using random color marble(real oil or color paint that cannot be mixed together) and making interaction with people’s movement
  • Using lots of LED light at the backside to make user’s shape



2) feather interaction

  • fans equipped at the bottom, and making feather flies when people move next to it.
  • feathers are located in each transparent tube so that it looks like a sculpture



My concentration would be developing the mid-term, but I would like to realize other ideas during winter vacation.

Mid-term Project: Macro Part

Mid-term Project (feat. Chenchao)


“Drive digital output with physical input.” We tried to focus on this context and developed idea based on this structure. Through the brainstorming, our conclusion was to utilize the law of nature. Consequently, “GRAVITY” came as core character of our work. And, we used water to express that character. To lap those with our familiar surroundings, we came up with sink. Specifically, virtual waterfall is controlled by the valve to twist the reality.


Trigger point: valve and tube(as familiar objects) attached at not corresponding system(screen? box? computer?)

Users will see physical parts, valve at the bottom and tube on the top, and will expect that something will fall through the tube when they turn the valve.(Because of the daily perception. Human use these kind of machines in this aspect.) However, not the real water will appear, which breaks their expectation. So the Sophisticated virtual water that can satisfy(or attract) users should be designed. This will also make users give imaginary experience.

So, what inputs should we have to make? First one is the valve for controlling water amount, and we wanted one or more inputs for not making users bored. ‘Color control’, which we cannot do in real life, was perfect for that. We can change the color of water in digital screen!

At first, we wanted to realize all color range at p5 sketch, but we failed to match color in sketch with that of RGB LED, so we changed to use only one color range, blue.




  • sketch part : we intended to make water gathered inside the window(horizon going up), but failed since it requires 1) going up function by input value 2) going down function as a default. Those things were more complex than we thought.
  • Light sensor value vibrates every demo, so we had to map the input value both in arduino(map for neutralizing noise) and p5(map for drawing balls)


While we were working with this project, we came up with much refined and developed idea that twists the mental model one more. If we succeed to implement the water horizon change, we would like to use real water to make extra interaction with the screen water. I think this idea is interesting as it breaks the user expectation one more. We will figure out further elements to make this project concise and impactful.


Week 6_Serial Comm

Serial Communication

Input from arduino(physical) to output on p5.js(digital). Codes from arduino and p5 are as follow. With the change of potentiometer, x-position of ellipse is manipulated.

Then, I tried with multiple inputs, one for x-position and another for y-position of controlling ellipse. Split function is used to separate the input values.

Through an hour of figuring out the error, I realized that if(data > 0) should be changed into if(data.length>0) or if(data). This can be considered as multiple data is sent with a string, which means it has a length.

Week 5: Simple lab

Week 5_ catching up week

(Updated Post)

Light is one of my big interest. I did lab project based on ultrasonic sensor and LEDs controlled by distance in pursuit of going ahead one step further. What I intended to make was implementing interactive LED board with IR(infrared) phototransistor and IR LED as input source, but my sensor couldn’t read the input in spite of hours of effort. I’ll figure out the problem with more time.

Ultrasonic sensors reads the distance towards hand, and turn LED with the distance scale. Circuit and codes are below.

same code structure for three sensors. I made functions for each.


While LEDs are bouncing from LOW to HIGH within the off condition(distance > 50), cracking has occurred at each endpoints(LOW and HIGH, value: level=1, difference=5). So I change the level, difference, and the bouncing section until finding the stable condition without cracking. Why this happened?

These are what inspired me for the past days. I would like to develop my final project referring to these conceptual. Installation(or different outputs) that gives audience an interactive experience, and has strong message.

LED cube from google searching

Rain Room, by London and Berlin-based collective Random International

Understanding how sensors work, and how to code on them.

1. servo by distance rate

Servo angle controlled by distance scale. Since the rotation move is not smooth, I tried a couple of things to make it smoother, but couldn’t figure out what was wrong. The things I did were connecting capacitor, changing mapping range.

2. LED with distance sensor and flex sensor

LED brightness separately controlled by distance scale and flex sensor’s scale. The output was smoother than that of servo’s.

In addition, I wanted to make a switch button to make LEDS move between LOW and HIGH while pressing, but it didn’t work. I added codes below. What’s wrong with my codes??

Week 4: Sound


This week I gave weight on implementing sound with arduino. First, I made variable resistor to control the sound pitch.(PWM) An analog input generates the change of sound pitch. The speaker I used was from the junked headphone. I removed the body parts and combined each wire with those of LED for better connection. Therefore, LED doesn’t light up.



Next, I tried to create melodies controlled by switch button. Referring to the pitch library, I reorganized the melody components and delay speeds to sound them like the song “Luv”- Tory Lanez. Switch as digital input is used to start the melody. Since the melody consists of loop, one loop of melody is played per pushing the switch button.

Schematic diagram of the circuit would be like above.



Where is the pitch library? I couldn’t find it so I copied and pasted the whole lists just like below.


FYI (original song)


Week 3: FSR(input) + LED(output)



For this week, I would like to focus on understanding coding and circuits which will eventually become more complicated. I used FSR(Force Sensitive Resistor) as analog input and three LEDs as digital output. Along with the scale of the input force, I aimed to differentiate the number of LEDs lightening.


FSR is connected to analog input A0, with the initial value of 0. Three LEDs are connected in parallel.



Stronger input makes more LEDs to light up. If the force exceeds particular point, all LEDs repeat on-off in a regular sequence.



  • Resistor used for LED: 2.2k ohm’s
  • Resistor used for FSR: 560 ohm’s







This is interactive screen located in ACE platform of W4 subway station. Eidetically, we(and I as well) can predict that this screen is used for searching information such as real-time train schedule or subway maps perhaps.


The main page shows the priority information. And there are map search function as well as arrival schedule. I observed 4 people(3 women, 1 men) using this during 15 minutes of observation, at around 10:00pm. People stayed average of 1~2 minutes, and all have used arrival function only. I can throw two issues from this.

First, in my observation, since most people use their own mobile application for getting subway information, I forecasted that the user group of this screen would mostly be foreigners or tourists who are not familiar with NY-based subway apps or information. However, all people (at least whom I’ve observed through this assignment) seemed U.S. citizens.

Second, referring to the point of reading, this machines can be considered as very week interaction(input-button click/output-data or information). This is because all functions are already being set, and we cannot change the condition nor output of it. With pushing button, user are seemingly sending information to the machine, but I think this process is much closer to just stepping the guideline structure. However, since user’s next step can be changed by using this machine, it can be defined as affecting user. For instance, if user finds train delay, they will likely to change their next way instead of taking subway.


Week 2 _ LED & Switch

Let’s start with things as follows: Arduino, bread board, wires, LED, resister, and pencil for testing switch function.

First, I made a simple circuit to light LED. LED and resister are connected in series.

Then I added two more wires in circuit to use those as a switch. If they join each other, circuit is connected and the LED lights up. This circuit is seemingly ugly in fact, but is shows the principal of the switch function, Connect-disconnect.

Then I tried with pencil. Since pencil is a conductive material, it could be used as a switch as well. LED was less brighter since the resistance occurs between two points within a pencil circuit.