May 10th _ Emoji

  • Final project presentations

0510 NYU IDM _ Physical Interface of Networking | FinalProject_emoji | EmojiDemo | Tool Used_ Arduino (Esp8266) & P5.js & Adafruit.io

For my final project, I create a physical emoji, which allows people to emoji react to jokes by physically squeezing it and making a squeaking sound. A joke will be shown on the screen, and people can squeeze the emoji to drop down and express their emotion to the joke.

In order to avoid the screen being filled up and the joke covered by the emojis, p5 will stop renewing data once the maximum receiving number is reached. Once the maximum is reached after 3 seconds, all emojis will bounce up and fall out from the screen. After all emojis have already left the screen, the joke will be renewed to the next one. The number of emojis will be reset to zero so people will be able to drop down new emojis again.

For connecting p5 and Arduino to make real-time interaction, I used Adafruit.io to send and renew input data.

At first, I was planning to create four different emojis (sad, curious, smile, angry) and each one has their specific way for interaction, such as squeezing for smile face (tilt or force sensor) or wiping tears for crying face (moisture sensor). I end up with only two emojis, angry and smile, and both of them can be triggered by force sensors. I cut down to only two so people will be able to spend less time on deciding which emoji to drop and the interaction between two emoji can be noticed more obvious and easier.

In order to represent how people react to the counterpart in the real world, I made the smiley face scale down and change its color when there is an angry emoji shown around them. The majority always tend to use an aggressive way to change the minority’s opinion, and the minority will possibly start to be afraid and stop speaking out their voice without hesitation or fear.

I decided to use dog squeak balls as my physical emoji’s main component so there will be a goofy squeak sound when people drop down an emoji. And I place them on a box which I laser cut with four holes on it, two for placing emoji and two for LED light.

The difficulties I met through the process are mostly from p5. The most time I spent on this is making emoji automatically refresh after the maximum is reached and switch to the title after every emojis has already fallen out from the screen.

0510 NYU IDM _ Physical Interface of Networking | FinalProject_emoji | emojiMakingProcessTimeLapse | Tool Used_ Arduino (Esp8266) & P5.js & Adafruit.io

If I have more time on this project, I would like to refine the physical part. Probably make the emojis softer and redo the box with wood board or acrylic board instead of cardboard. I’d also like to think about how to make the p5 emoji interaction more visually exciting.

May 3rd _ Final Project Review

  • Final project review

For this week progress, I have done the p5 emoji interacting part, and control the dropping emoji with Arduino. I also decide to scale down to two emojis instead of four, although I have already finished their code. Only smile and angry face will be used because this can make the interacting part more obvious and easier to be noticed. In terms of the Arduino part, I have successfully connected it with my p5 file by using adafruit.io. I haven’t started the physical part yet, so I use two force sensors to represent two different emojis in the demo video, and I will be ending up using force sensors in my physical emojis.

For the next step, I need to finish the topic shown when the emojis are falling and the result page for p5, which will be showing after specific time of no one playing with it or after the screen is filled up. I will also have to refine the interacting part. As for the physical part, I will be starting with prototyping before my components arrive.

Apr. 5th _ Final Project Proposal

  • Assignment :

Homework :

✔️For next week, I expect you to have an idea for your final project. That is, a description of what your thing does, what technologies you will use, and a plan for how to make it happen. Please write this up and post it to the blog. Use photos, illustrations and examples to give as best a description as you can. Try and make a prototype of the basic idea.

✔️Install node.js before the next class meeting

For my final projects, I have several ideas, but one of them is physical emoji reaction. Since Instagram has incorporated emoji reaction for Instagram story, most people start to react by only sending an emoji instead of context. In order to bring back the social media interaction, I’d like to create a physical device. It allows people to send emojis by pressing the button. However, before sending out an emoji, people are required to receive enough emoji from other people. The more emoji they receive, the more they can send out. To indicate the amount you got for each emoji, I will add a LED for each one; the more you got, the brighter the emoji will be.

I haven’t figured out the exact way to link the physical emoji device with Instagram, but for the physical part, I will be doing similar to the Big Enter Pillow. Once the pillow is plugged into the computer by USB, it can be worked as the enter key, such as sending messages. As for mine, I will be doing in a wireless way, so the device can also be worked on phone.

Mar. 29th _ MQTT

  • MQTT

  • online services

Homework :

✔️Using a MQTT broker, create a physical system the responds to an external trigger (for example, send a tweet when a cat is on the mat, or use a SMS message to set the brightness of an LED). Post this to the class blog.

✔️Find an example of an internet/networked artwork or service that you find inspiring. Write up a short note identifying how you believe it works. Use this as inspiration for your final project. Can you improve on this some way? How would you make it differently?

MQTT & Internet/Networked Artwork

For this week assignment, I was trying to create a guest notification. Each time if there is someone stepped on or walked over the specific floor area in front of your room, and the pressure sensor senses it, you will get a SMS message that telling you someone has passed by. I have started testing with a pressure sensor at this point.

An inspired artwork I found these days is the one I saw at Teknopolis 2019, which exhibited several groundbreaking multi-sensory installations. One of the most impressive artwork is the one that people could interact with the elements they drew on the paper. In order to start the interaction, people need to draw their own artwork by using black ink. After you are done with your drawing, you could leave your artwork under the camera for three seconds, and then you could start with playing around with the elements you drew by using your hand. The most intriguing part I found from this installation is actually the combination of visual effect and sound effect and how it successfully caught people attention among several artworks. For my final project idea, I haven’t had a concrete idea yet, but I would like to create a new device which could let people play around with their own work. It could be either combined with generative art or just physical interaction. The main purpose for creating this is that I’d like to make people feel the connection with network/ internet, and let them know they could actually interact with a device, not just feel the coldness from technology.

Mar. 15th _ Midterm Presentations

  • Midterm presentations

Spring Break assignment :

✔️Net Devices Quiz & Midterm follow-up This is due before March 25.

✔️Exercise : Wherever you go, whatever you do, there’s internet infrastructure creeping around. Identify three pieces of network infrastructure in public and sketch it. You can take a photograph and use that as a reference, trace over a picture, or just sit there and sketch it. The sketch must be done on paper with pencil, ink, charcoal, or other writing/drawing implement. That is, no digital drawings or scans. Bring your sketches into class on March 29. Take photos of them and post them to the blog as well. When you post, identify what you think the object is, what it does, and why it exists in this way.

Network Infrastructure

The first network infrastructure I drew is called primary inspection kiosk. I first saw this at Montréal-Pierre Elliott Trudeau International Airport, and I thought this could only be used by returning residents, Canadian. Surprisingly, all travelers, including foreigners, are required to verify their identity and make an on-screen declaration through this machine. During the self-inspection process, I found it easier and faster then I thought. People will need to scan their passport first, take a photo, and answer some questions. After using it, I personally hope that every international airport, especially JFK, could installed a bunch of this kiosk, so people do not have to stand in line for more than two hours waiting for the inspection.

The second one is some cars for transporting check-in luggage. I saw them at Québec City Jean Lesage International Airport when I was checking in for my flight from Quebec to New York. Although I didn’t have any luggage to check in, I keep looking at this system. Passengers will put their luggage on the conveyor belt first, and when the luggage is sent to the end of the belt, the car will come and take the luggage away. In this way, there is no need for a person to stand there and move every passengers’ luggage by hand.

The last one is the self-service check-in machine. People have to select their airline first, and then either enter the booking reference manually or scan the QR code. After confirming your information and selected seat correctly, you will get your boarding pass and finish the check-in process. Although it seems simple, I and my friend didn’t successfully check-in by using this machine. The selected seat of one of us is Gate, which means you are possible to get on this airplane but not guarantee to have a seat, so we have to go to the ground crew for help. 

Mar. 8th _ Network Restrictions | Midterm

  • setting up something to get around network restrictions

    • herbivore

  • packet sniffing

    • wireshark

  • what’s inside the data?

Homework :

✔️Post your midterm documentation and write up. You will be presenting your work in class.

0315 NYU IDM _ Physical Interface of Networking | MidtermProject_Teletubby | TeletubbyDemo | Tool Used_ Arduino (Esp8266) & P5.js

For my midterm project, I decided to switch my topic from temperamental sheep to “Teletubby.” Teletubby is an empathetic elf who cares and understands your inner feeling. At my draft attached below, in order to know how a person feels, I use the force sensor as input data in the beginning. Teletubby will be able to know your mental condition by analyzing the frequency and volume from your answering voice. However, I ended up using a force sensor because I think users could be more interactive with the device.

I began with creating an animation from p5.js, and linked it with arduino by using arest.

p5.js Draft

p5js

To start the interaction, people could input their stress level by pressing the force sensor, and the data will be mapped from 0 to 100, and people could see how much force they pressed by looking to the bar shown on the screen. If the force sensor has been held for a static level, within a range 5, for 3 seconds. In order to control the timing, I wrote a variable, count, and if the difference between data from the last loop and a new loop is smaller than 5, the count will be added one.

arduino

p5.js

After Teletubby got your stress level, he will either smile and wiggle or showing a sad face depends on your condition. If your stress level is lower than 65, this means your at good condition, otherwise, you’re not feeling well, and he will make you feel his empathy by showing a sad face to you.

crying face

smile face

This is actually one of the difficulties I met. In order to have two facial expressions, I was intending to use two different EL wire, red and blue. The EL wire I bought is the one which needs to use two 1.5v battery and control different three lighting mode by clicking the button. In order to connect it with Huzzah and control by it, I solder three wires to make the circuit works. However, once I tried to use two EL wire at the same time, one of it will start blinking and the other one gets really dim, so I gave up using both. I ended up using only one EL wire and make it rotate by using a servo, but I forgot that the servo couldn’t be rotated up to 180 degrees.

0315 NYU IDM _ Physical Interface of Networking | MidtermProject_Teletubby | TeletubbySolderingElWireTimeLapse | Tool Used_ Arduino (Esp8266) & P5.js

If I got more time on this project, the first thing I would do is solving the EL wire problem. I would try to figure out how to make two EL wire work simultaneously. After this, I’d like to add sound for Teletubby, so the user could have a stronger connection and interaction with him. Last but not least, I’d like to refine my prototype and come up with something people can really hang on their wall and play with him every day.

0315 NYU IDM _ Physical Interface of Networking | MidtermProject_Teletubby | TeletubbyMakingProcessTimeLapse | Tool Used_ Arduino (Esp8266) & P5.js

prototype

laserCutting

Mar. 1st _ Remote Control

  • remote control of your microcontroller

  • aREST library introduction

Homework :

✔️Reading : The computer for the 21st Century by Mark Weiser (pdf). This was written in the early 1990s, and set the stage for a lot of what we think of when we talk about connected devices. Think about ubiquitous technologies of today, vs those when you were a child. What has changed, what’s different? What are some technologies that have been around for a hundred year, or a thousand? What does it mean for these to fade into the background and become part of everyday life? 

✔️Midterm assignment: For this project, you are expected to make a physically interactive system that responds to an input and produces an output. You may work in pairs if you like, but each person needs to document what they did. This needs to happen over a WiFi connection with your microcontroller acting as either a client or a server. Some examples :  use the browser as a remote control for a multi-colored lamp, rebuild your love machine as a web app that controls the CSS of your page, make a device that monitors the environment of your home’s temperature that can also control the thermostat. These are some straightforward suggestions, make something unique.

  • The technical implementation does not have to be particularly robust, but the final project should demonstrate a level of polish that goes beyond a regular homework assignment. That is, there should be a clear mapping between the input and the output. What are the affordances of your project? What are the signifiers? How does someone know that there’s something happening in the background when there’s a process going on that’s not immediately apparent?

  • For next week (March 8), you need to write up a description of your project on the blog, and build a prototype of this. The prototype should have the basic  functionality in place. If you struggle with this aspect, then perhaps your idea is too ambitious and you should scale back. Document what is working. Identify what you still have outstanding. Explain what you will do to polish the project, not just in code, but from an interaction and design perspective. 

For my midterm project, I will be creating a temperamental sheep. It will be able to tell users the current weather by changing its color and behavior. 

As the attached photo shown, different weather condition corresponds to a different sheep’s behavior. In order to let the sheep move freely, I will be using two normal wheels and one mecanum wheel. As for the color changes, I will be using RGB LED light and hide them in the sheep’s cotton. However, if only using weather API as an input, it would be hard to see the difference between different weather condition. To demo it more easily, I will be using photoresistor as an input.

WeatherCondition – SheepAction

Draft

IdeaIteration

IdeaIteration

Prototype

Prototype

Feb. 22nd _ Web Server

  • Introduction to the ESP module

  • basics of WiFi and local networking

  • manage a client connections & server things

Homework :

✔️Read Making Things Talk, pp 12-19, 88-106

✔️Using the sensor server example as a starting point, turn the HUZZAH board into a webserver. Use a sensor or switches to change elements on the page. It may be easiest for this exercise if you write the HTML first, then put it into the Huzzah. Remember, the microcontroller can’t hold that much information, so the pages shouldn’t be too large. Post your work to the class blog before 10am Friday.

0301 NYU IDM _ Physical Interface of Networking | HW4_WebServer_Controlled Jumping Fish | ControlledJumpingFishDemo | Tool Used_ Arduino (Esp8266) & Html & Css

For this week homework, I decided to create an interactive animation. Users could control how high the fish jumps by adjusting the potentiometer. 

I wrote the animation with HTML and CSS in CodePen before connecting it with Huzzah. 

CodePen

Due to the limit space of microcontroller, I minified my code before adding the into Arduino code. 

Screen-Shot-2019-03-18-at-4.23.41-PM-1024x557.png

0301 NYU IDM _ Physical Interface of Networking | HW4_WebServer_Controlled Jumping Fish | ControlledJumpingFishScreenRecordingDemo | Tool Used_ Arduino (Esp8266) & Html & Css

The larger of the mapped number of the potentiometer, the higher the sheep jumps. If the mapped number is less than 20, the fish will not start jumping into the air, and if the number is not high enough, the fish will keep rotating on the grass instead of leaving the grass.

Feb. 15th _ Love Machine

Homework :

✔️Read A Brief Rant on the Future of Interaction Design and the responses to that article

✔️Read Ch 6 of The User Illusion by Tor Nørretranders (see class Slack)

✔️Build a love-machine that identifies how compatible two people are with the microcontroller. Think of your input, and how well it maps to your output. Will you use LEDs? a servo? something else? will it be buttons, knobs, sensors? Be inventive. You may work in paris on this project. Document the work on the blog. Use video, shoot it well, use a tripod, keep things in focus. When you write your documentation, be sure to explicitly reference the readings from the last two weeks. How are you making affordances for people? What are the signifiers? How are you using the body in a meaningful way?

0222 NYU IDM _ Physical Interface of Networking | HW3_LoveMachine_He Loves Me... or Not | HeLovesMeOrNotMakingProcessTimeLapse | Tool Used_ Arduino (Esp8266)

Che-Yu and I came up with an idea of using the fortune-telling game “He loves me, he loves me not” or “She loves me, she loves me not” to see whether or not you are compatible with your soulmate. In this game, one person will speak the phrases “He (or she) loves me,” and “He (or she) loves me not,” while picking one petal off from the flower for each phrase. The phrase for the last picked petal represents the truth between the object of their affection returns that affection or not. 

As for our love machine, the player will do the same as the game. While the seven petals are picked off, the color of RGB LED light will change from red to green to blue. In order to solve the issue that there will always be only seven petals and the player might have already known the outcome without picking off the petals, we set a random value for the beginning status. If the LED lights up in the end, it means you are compatible with your affection. If not, you could play the game over and over game until you reach your ideal outcomes!

Make the Flower conductible

Coding Draft

Arduino Code

prototype

prototype

However, we don’t have any testing video at this point, because we have failed to make the entire flower conductive. We use mainly cardboard and copper tape for both prototype and the final version. As we were testing each petal’s conductibility, not every petals work. Instead, only two of them works well. We tried to add more conductive layer to the attached surface to make the rest petals conductible , but it didn’t work as we thought. So our next step will be finding out the conductive problem, solve it, and make this game happen with our love machine.

Conductive petal

Conductive petal

 

Update Feb. 22nd

In order to solve the conductive issue, we folded copper tape into a special shape and sticked it on each petal. As you can see in the attached photo, the attachable surface becomes wider and thicker. 

Updated petal

Updated petal

Feb. 8th _ Observation

Homework

✔️Read Ch 1 and the prologue of the Design of Everyday Things (distributed via Slack)

✔️Read Ch 1 and the prologue of Emotional Design (distributed via Slack)

✔️Observation assignment : Observe someone other than yourself using an interface in public. Document what you believe they are trying to accomplish, what steps they must take to reach their goal, and what sort of barriers they encounter. On the blog, post your observations and reflect on how the information from the readings may have helped streamline this interaction. What were the affordances, and what were the signifiers? Tag these posts with the category ‘observation’

✔️Continue to work with your microcontrollers and switches. Build an interface for your microcontroller that uses the switch you built from the previous week. What about your switch makes it easy, what makes it hard? How can you contextualize this interaction? Make a comment on your original post about how you did, or did not get the switch to work with your microcontroller.

serge synthesizer and oscilloscope

Last Friday, I was observing a student trying to compose sound effect by using serge synthesizer and oscilloscope in IDM audio lab. Serge synthesizer is an analogue module synthesizer and oscilloscope is a laboratory instrument used to display and analyze the waveform of electronic signal. In order to create sound effect, the student has to connect the audio signal and control voltage jack via banana cables. At the beginning, he started with plugging in banana cables to the speaker output. After making some basic sound effect, he started to add more cables and adjust other cables in order to create more variables and achieving the ideal sound. The intriguing part is that there are six different colors for the cables, and each color has a specific length. During the observing process, despite the time he used for listening the sound and wondering about how to adjust it, most of the time he is switching the cable to the one with the suitable length and finding where to plug in. Nonetheless, I noticed that sockets on the synthesizer has four different color, each color for a different function. The red one is for control, purple for sync, etc. This efficiently shorten the student’s time for finding which is the appropriate socket to plug in. After all, this machine seems complicated to me, but actually it possesses a special module arrangement and color differentiation to simplify it’s interface.  

Feb. 1st _ Switch

Homework :

✔️get the materials you need for the class

✔️Make a switch that you can control without using your hands. Document it on the class site with a blog post. When you post, use the category “Switches”

✔️Read the Excerpts from the prologue and Ch1 of “Making Things Talk”

✔️Read “There Will Come Soft Rains” and “The Veldt” by Ray Bradbury

✔️On the blog, write a comment to my post about the Bradbury stories, particularly in light of the fact that we’re talking about building networked devices in this course. This should be at least 300 words long. You may find it easier to pick one of the stories, but if you do, be sure to reference the other one as well.

0208 NYU IDM _ Physical Interface of Networking | HW1_Switch SwitchDemo | Tool Used_ Arduino (Esp8266)

For my switch project, I design a little punching machine. Unfortunately, you still need to use your finger to control it, but the way you trigger it is definitely more intriguing than just press the original switch button. As you might have already noticed, there is a seesaw at the bottom. In order to trigger the switch, all you have to do is try to shoot the disk, rolling in the middle, to the top and be attached by the magnet. For the conductive method, I have use aluminum foil, paper clip, and also some conductible wire.  It might seems difficult to reach the top at once, but practice makes perfect. However, this also makes the switch not convenient enough to be used. So my next step would be thinking about how to make the chance of hitting and attaching at the top more easily. 

IMG_5828.jpg

Process

prototype

Multimeter Testing

Process

Process