3/11 update

Xbee wireless Joystick

enclosure test


I designed a system to limited the rotation. So the cube only rotates along yaw, pitch, roll three axis. Modeled in MAYA, and 3D printed with Makerbot Replicator 2X. The six pillars are made in cooper tubes, springs and marbles.



The sensor I used in this controller is a SparkFun 9 Degrees of Freedom  Sensor Stick. It works well and pretty accurate. However, I also considered about using an optical sensor which are usually used in mouse. It can detect how much distance the cube has rotated if I put  it on one of the pillars. However it was too big to fit in the tracks on my cube. Otherwise it would be a better choice than an IMU board. Because the optical sensor only needs to detect values that have traveled in two axis, while the IMU has to record values of three axis (actually 9 axis from all three sensors on the board and complicated calculations) and update those data very frequently. I will find out a way to shrink down the size of optical sensor if there is one.


I want Unity get sensor data from the IMU board wirelessly without latency. The first try was using Arduino and the serial library of MONO. Arduino can send data through serial port very easily. However, there is a significant latency when Unity tries use this library to read serial data very frequently. My solution is using Teensy. Teensy is a Arduino-compatible micro controller and it can be recolonized as a joystick (a HID device) . The joystick mode makes it much easier have a communication between Unity and Sensor.

The wireless module I used is Xbee. There is one Teensy inside the cube , getting the orientation data and sent it through Xbee to another Teensy that connects to the computer as a HID device. Unity get the data from this HID Teensy. Although the setup is a little bit complicated. The communication is stable and responsive.

Thesis studio2 Presentation


I asked myself a question—what is the relationship between reality and virtuality?Could one of them transform to the other?Are they the opsitie side of each other?

What can be virtual. Something not real, something not actual.


I used to believe that infinity is not real not actual.   Because I could have three candies, four candies, or even a hundred candies, but I could never have infinite candies. Infinity is virtual, because it does nor exist in reality.


I believed so until I saw this. When there are two mirrors facing each other. There is a seemingly endless line of images fading into the distance.

I found infinity–something not real–in reality.

Virtuality could be a reflection in a mirror or a computer simulated landscape. Both are not real, but displays the full qualities of what is real.

And This is what we usually do. We already in reality and we use tools to  experience or create a new reality in virtual world.

And this answers my question. Reality and virtuality are not opposite to each other. We sometimes  can find one of them in the other.


There is a recursion out there when we seeking reality in VR


This answer transforms to my thesis.

It is a Virtual reality game design to express this relationship. It uses an Oculus as a visual output and a cube shaped physical interface as a controller. Players can see the room they are in from the VR device. Whenever players tilt the cube, the scene in the Virtual reality room will also tilts.


So the user is manipulating the environment from outside, from reality, while at the same time observing it from inside the Virtual space. Therefore a recursive experience is created. And there is a paradox of where you are leads the audience to think about where is the border between reality and virtuality.

About the game


the story of that game is that the player is trapped in a recursive looping maze. You moving through rooms that looks similar. And you try to escape from this maze.


The goal of the game is to collect those shiny flags in each room, and when  player collect all the flags , they are able to back to their reality  (or what they believe is reality).

And the game mechanic is to tilt the cube to change gravity or the rotation of the room .


About the style and aesthetic

The flag comes from MC Esher ‘s  painting Mobius strip I. Esher’s work inspires me a lot. The gravity manipulation idea and the scene design is also inspired by his work.  The ambiguation and paradox in his work could be suitable as a form to express my concept.

Prototype 3

The Arduino board is placed outside the cube in the last prototype, which makes it easy to organize wires and pins but not good for user experience. Audience has to avoid touching wires exposed outside. So I put the Arduino board inside the physical device this time and used a longer USB cable. For future prototypes I am considering add a wireless module on the physical part and use a circuit board instead of an Arduino board to scale down the size.

The physical part is built with plastic board to make a more stable structure. I also tried to create more feedbacks on the physical part. For example as the audience solve puzzles two buttons will flip out on the face of the cube and the audience can then do some operations by pressing the buttons.

An Oculus rift is connected to this prototype. Since there are bunches of sensors built in Oculus to detect the audience’s head movement, it is not merely used as an output. The audience can move his/her head to focus on an object in the virtual world to get hints.

All the efforts are made to balance the virtual part and the physical part of the project. They can respond each other and they are also connected with each other. Both of them are input and output at the same time. Each one contains one another. Just like yin and yang in Chinese philosophy—the two opposite or contrary forces are actually complementary.

image02 image38


image04 image03

image05 image06

image08 image07

Prototype 2

More elements and interactions are added in the second iteration. In order to test and assemble quickly, I used cardboard to build the physical puzzle. Audience can spin the bottom part of the cube like a Rubik’s cube.  There is a reed switch built in to detect the rotation of the bottom part. If all the puzzles are solved the top of the cube (which is locked by a servo) will open up. Other puzzle elements like code and locks are also tested in this iteration. Image.6 is a screenshot of the digital output.

image35  image34


Presentation 1


When I was looking for a thesis project. I started from something I am really interested. And one subject is Virtual reality.

In terms of technology, Virtual reality is typically refers to a system usually includes a computer capable of real-time animation, controlled by a set gloves or a position tracker, and using a head-mounted stereoscopic display for visual output.

The core of Virtual reality Is presence.

presence is the sense of being someplace  while in virtual reality; many people feel as if they’ve been teleported. Presence is an incredibly powerful sensation, and it’s unique to VR; there’s no way to create it in any other medium. Most people find it to be kind of magical, and I think that once people have experienced presence, they’ll want it badly.

With this feature, Virtual reality is mostly used for simulations and play experience.

However. If it’s a good thing when people are going to addict to it. When I search for Oculus rift projects. There are bunches of flight simulations and VR dungens or VR shooting games.


Then I also found something creepy . This is a contraption from Japanese company Tenga, which combines an erotic video game, a Fleshlight-like contraption, and the Oculus Rift to create a sex simulation.


This raises people ‘s concerns about virtual reality.

What meaning should we ascribe to virtual sex and violence? What happens when we voluntarily step into an alluring, machine-made, alternate reality — and surrender contact with the real world? (Sexual pleasure is a powerful reinforcer, the big hook. ) How will we deal with users who won’t, or can’t, return to their average lives? Will there be a new class of schizophrenics who are simply jacked-in forever?

And then I came up with an idea. What if I make a virtual reality experience that during the experience, the user has to take off the VR device the head-mounted monitor.


And that leads me to a big idea of my thesis  — to create a connection between virtual experience and physical world.


My solution is make the virtual world as another dimension of physical world that we can observe .

The form of presenting this idea will be a physical installation and a Virtual reality experience powered by Oculus rift. In order to make a connection, the physical installation and the Oculus will present a same thing in different forms. There will also be communications between this two dimensions when player do actions.

For instance,  the stuff we showing in the two dimension is a cube. When player put a hand on the physical device, the player might see a hand-shaped mark on the face of that cube in Oculus.

So Player may put on and take off Oculus repeatedly to observe the changements or interactions between this two world during the experience.

I realize This form provides possibilities of creating puzzles which is another subject that fascinate me .


This just reminds me the game “the room”.

The Room presents the player with a series of strange boxes that have a number of physical mechanisms on them. The player is challenged to figure out how to open each one – typically by undoing a series of locks – to access another puzzle box within it. The game uses a variety of motions enabled by mobile device touchscreens to simulate actions in real life, such as looking around the device, turning keys, and activating switches.


And I also looked for physical puzzle games like Rubik’s Cube.

The left one is a traditional Chinese puzzle made of pieces of woods. It is connected without glues and nails. you try to figure out how to desemble and reassemble those parts when you play it.

I am thinking bring this geometry constructions to my project as a part of the puzzle.

And the player may get stuck or have no action to do in the virtual world and have take of  the oculus and  assemble those kind of pieces in the physical installation to Get cules and hints and goes on put on oculus to solves another puzzle in virtual world.


Prototype 1

Prototype 1- communication between Unity and Arduino

I decided to use Arduino and Unity3D to build a prototype, since they are both powerful and easy to access. This is a simple prototype testing the communication between Arduino and Unity3D. The prototype shows the basic idea of connecting virtual and physical world. Press the button on the physical device (Img. 1), a virtual button will show up in Unity (Img. 2) and the button can control a LED on the physical device to be ON or OFF. I am using an Arduino plug-in Uniduino in this prototype. There is also a Oculus Rift plug-in allows Unity to access Oculus monitor and sensor.

 IMG_20141028_140023 Img . 1                    20141028140524 Img. 2

Thesis Proposal

Is there a way to make a connection between virtual world and real world? Will that make virtual world be another dimension of real world or blur the border of virtual and reality? Thinking about these questions, I came up with a thesis idea of creating a connection between physical world and virtual world.

The form of presenting this idea will be a physical installation and a Virtual reality experience powered by Oculus rift. In order to make a connection, the physical installation and the Oculus will show a same thing in different forms. There will also be communications between this two dimensions when player do actions. For instance,  the stuff we showing in the two dimension is a cube. When player put a hand on the physical device, the player might see a hand-shaped mark on the face of that cube in Oculus. Player may put on and take off Oculus repeatedly during the experience. This form provides possibilities of creating puzzles and narratives.


keep talking and nobody explodes

the room


Hang Ye

Thesis Studio 1

Research Domain 1, Draft #1.

Talking about Virtual Reality, we all immediately come up with an image that a man is looking around with a big equipment covering his eyes. Because of the impressing images of Virtual Reality we describe in movies and fictions, Virtual Reality is typically referred to as a technology or immersive multimedia. It is interesting to see that how people in 1990s define Virtual reality in terms of technology is almost same as the latest VR technology we have in 2014. Frank Biocca described Virtual reality in 1992: Most popular definitions of virtual reality make reference to a particular technological system. This system usually includes a computer capable of real-time animation, controlled by a set of wired gloves and a position tracker, and using a head-mounted stereoscopic display for visual output.⑴ It is true, when the word “Virtual Reality” first came out, it always represented a new technology and it was always together with the word “computer”. In the early 1960s the term “virtual” began to emerge within the computer industry to signify something that appeared to exist without actually existing, e.g., virtual memory.⑵ In 1989, Jaron Lanier (Fig.2 who is widely regarded as the father of Virtual reality) coined the phrase “Virtual reality” to describe the experience made possible by the use of the latest generation of goggles, gloves and related technologies. The terms virtual worlds, virtual cockpits, and virtual workstations were used to describe specific projects. In 1989, Jaron Lanier, CEO of VPL, coined the term virtual reality to bring all of the virtual projects under a single rubric. The term therefore typically refers to three-dimensional realities implemented with stereo viewing goggles and reality gloves.⑶ So, this is the definition of Virtual reality in technology-wise.

001                       002

Fig.1. Cyber Mind Virtual Reality Center        Fig.2. Jaron Lanier


On the other aspect, when we understand “Virtual reality” without this certain history and background, when we see “Virtual reality” in a bigger scale, we may realize so many things could be Virtual reality. When anything new comes along, everyone, like a child discovering the world, thinks that they’ve invented it, but you scratch a little and you find a caveman scratching on a wall is creating virtual reality in a sense. What is new here is that more sophisticated instruments give you the power to do it more easily. Virtual reality is dreams.⑷ To explain it straight forward, Virtual reality simulates the reality and cheats us to make us immersed in this illusion. For example, Oculus rift (a widely used VR equipment today), it has two screens in front of your eyes and present two offset images to simulate the different visions your two eyes see, which create an illusion of depth in your brain because brain detects distance by calculating the difference of what you see in two eyes. It also has an accelerometer on it to detect the movement of your head and move the images based on the head movement, which create an illusion of you looking around in the virtual world. This is a strong and convincible simulation (illusion). It definitely is Virtual reality. However, if the simulation is not so strong, it still could be Virtual reality. 3D movie could be VR, it sometimes cheat us successfully. Everyone has the experience of moving his/her head in the seat, avoiding something come out from the big screen. What about movies without 3D effect? What about black and white movies? My answer is it could also be VR. There is a story says that when the very first short movie L’Arrivée d’un train en gare de La Ciotat (Fig.3) was shown to the public, the audience was so overwhelmed by the moving image of a life-sized train coming directly at them that people screamed and ran to the back of the room. It’s more like an urban legend, but what we could imagine the audience were shocked when they first saw a moving picture. What else? Fictions could be virtual reality. The telephone is virtual reality in that you can meet with someone as if you are together, at least for the auditory sense.⑸ The key to defining virtual reality in terms of human experience rather than technological hardware is the concept of presence. Presence can be thought of as the experience of one’s physical environment; it refers not to one’s surroundings as they exist in the physical world, but to the perception of those surroundings as mediated by both automatic and controlled mental processes.⑹ Presence is the sense of being someplace else while in virtual reality; many people feel as if they’ve been teleported. The term presence is an incredibly powerful sensation, and it will be commonly mentioned when we talk about Virtual reality. What designer and artist create is basically illusions. Paintings are actually creating illusions of three dimensional space on two dimensional canvas. Movies are creating illusions of moving stuffs by showing continues frames constantly. Those illusions create emotions by cheating our brains (mostly visually). Although we have the knowledge of a movie, or a painting is fake, our body is cheated physically, because our body has been processing data this way for thousands years. When we see moving images, our body consider this as alive, in that way, it creates emotions. Because emotions are actually chemical reactions in brain. That explains why people cry for characters that not even exist in real world.



Fig.3. L’Arrivée d’un train en gare de La Ciotat(The Arrival of a Train at La Ciotat Station). It’s a black and white movie.

The history of Virtual reality in terms of technology has been recent and sudden and relatively widely used VR systems have only recently came out. Like most other developing technologies, Virtual reality is also popular in fictions. According to Chesher, “Virtual Reality developed from fiction in 1984 to a rich discourse and a marketed technology by 1992″ (n.p.). The first VR technology is considered to have begun in 1950s. Devised by Morton Heilig, and known as the ‘Sensorama’(Fig.4). This early form of virtual reality was invented in 1957 but was not patented until 1962.

The Sensorama consisted on the following elements:

  • A viewing screen within an enclosed booth which displayed stereoscopic images.
  • Oscillating fans
  • Audio output (speakers)
  • Devices which emitted smells

The viewer would sit on a rotating chair which enabled them to face this screen. They would be shown these stereoscopic images which gave the illusion of depth and the ability to view something from different angles.⑺ It actually works quite similar to Oculus rift, to what we have today. The difference is we put the huge monitor on our heads instead of putting our heads inside the huge monitor.



Fig.4. Sensorama

Providing a strong presence, Virtual reality technology is also considered to be used in different areas. One of the very first thing is military use–flight simulator. By the 1970s, computer-generated graphics had replaced videos and models. These flight simulations were operating in real time, though the graphics were primitive. In 1979, the military experimented with head-mounted displays. These innovations were driven by the greater dangers associated with training on and flying the jet flighters that were being built in the 1970s. By the early 1980s, better software, hardware, and motion-control platforms enabled pilots to navigate through highly detailed virtual worlds.⑻ VR also plays an important role in combat training for the military. It allows the recruits to train under a controlled environment where they are to respond to different types of combat situations. A fully immersive virtual reality that uses Head-mounted display (HMD), data suits, data glove, and VR weapon are used to train for combat. This setup allows the training’s reset time to be cut down, and allows more repetition in a shorter amount of time. The fully immersive training environment allows the soldiers to train through a wide variety of terrains, situations and scenarios.⑼ In game companies’ eyes the simulator is a new possibility of game. Sega’s Master System 3D glasses (Fig.5) appeared in the 80s, a flickering headset that recreated the effect of 3D in a private setting. Though primitive compared to today’s 3D standards, it created a unique effect across different games, including Space Harrier, Maze Hunter, Zaxxon 3D and Missile Defense. In 1995, Nintendo decided to give virtual gaming a try with its 3D red-and-black based headset, the Virtual Boy (Fig.6). Several 3D games were introduced, including Teleroboxer, Red Alert and Mario’s Tennis.


Fig.5. Sega’s Master System 3D glasses


Fig.6.Virtual Boy


However the Virtual Boy was considered as a failure. We can find some common failures of Virtual reality technologies at same time period. The Virtual Boy failed for a number of reasons, among them its high price, the discomfort caused by play and what was widely judged to have been a poorly handled marketing campaign.”⑽ Despite the marketing, the immature of VR technology is a crucial factor of the failure. Many reviewers complained of painful and frustrating physiological symptoms when playing the Virtual Boy (Fig.7). Bill Frischling, writing for The Washington Post, experienced “dizziness, nausea and headaches.”⑾ Reviewers attributed the problems to both the monochromatic display and uncomfortable ergonomics. Nintendo, in the years after Virtual Boy’s demise, has been frank about its failure. Same technology problems still happen nowadays. Players still complain about getting sickness while using Oculus rift. “In motion sickness there’s all this motion but you don’t visually perceive the walls and ceilings are moving. This is what creates the conflict that makes you dizzy. With simulator sickness it’s basically the inverse. These are all the things you want to avoid as  game developers.”⑿ Nate Mitchell, Vice President of Product at Oculus VR revealed that simulator sickness is a top priority for Oculus developers.


Fig.7.The red and dark display of Virtual Boy

There are also other concerns about Virtual reality technology. What do terms like “presence” really mean? Would people become the VR equivalent of couch potatoes, sedated by a potent new medium? The Wall Street Journal got in on the action in 1990 with a piece on VR pioneer Jaron Lanier titled “Computer simulations one day may provide surreal experiences.” The predictable tagline, set a couple of paragraphs above a picture of a heavily dreadlocked Lanier: “A kind of electronic LSD?”⒀ With this strong enough technology, People have already done a sex simulation(Fig.8). Someone rises a  question: What meaning should we ascribe to virtual sex and violence? What happens when we voluntarily step into an alluring, machine-made, alternate reality — and surrender contact with the real world? Sexual pleasure is a powerful reinforcer, the big hook. How will we deal with users who won’t, or can’t, return to their average lives? Will there be a new class of schizophrenics who are simply jacked-in forever?⒁


Fig.8.A sex simulation demo⒂

The history of Virtual reality in terms of human experience could be traced back to ancient philosophical discussions of reality. Plato holds out ideal forms as the “really real” while he denigrates the raw physical forces studied by his Greek predecessors. Aristotle soon demotes Plato’s ideas to a secondary reality, to the flimsy shapes we abstract from the really real-which, for Aristotle, are the individual substances we touch and feel around us. In the medieval period, real things are those that shimmer with symbolic significance. The biblical-religious symbols add super real messages to realities, giving them permanence and meaning, while the merely material aspects of things are less real, merely terrestrial, defective rubbish. In the Renaissance, things counted as real that could be counted and observed repeatedly by the senses. The human mind infers a solid material substrate underlying sense data but the substrate proves less real because it is less quantifiable and observable. Finally, the modern period attributed reality to atomic matter that has internal dynamics or energy, but soon the reality question was doomed by the analytical drive of the sciences toward complexity and by the plurality of artistic styles.⒃ Here is another interesting paragraph I found in the game “the Floor is Jelly”—The universe is before you, too vast to fully see. Your focus at present is taken by a description of yourself reading. You imagine yourself reading it. You briefly wonder if the self you imagine also imagines you.⒄




  1. Biocca, F. (1992). Virtual reality technology: A tutorial. Journal of Communication, 42 (4), 23-72
  2. Samuel Ebersole(1997) A brief history of Virtual reality and its social applications
  3. W.Krueger(1991)Artificial Reality,2nd ed
  4. Morton Heilig  1993 Hamit p57
  5. Rat Kurzweil(2013) This is your future
  6. Jonathan Steuer(1993) Defining Virtual Reality: Dimensions Determining Telepresence p5
  7. How did virtual reality begin? http://www.vrs.org.uk/virtualreality/beginning.html
  8. Virtual Reality: History(1995)  http://archive.ncsa.illinois.edu/Cyberia/VETopLevels/VR.History.html
  9. Loren Bymer (2012) Virtual reality used to train Soldiers in new training simulator
  10. King, Geoff; Krzywinska, Tanya (2006). Tomb Raiders and Space Invaders : Videogame Forms and Contexts.
  11. Frischling, Bill. “Sideline Play.” The Washington Post (1974-Current file): 11. ProQuest Historical Newspapers: The Washington Post
  12. THE PROBLEM OF SIMULATOR SICKNESS IN VR http://oculusrift-blog.com/importance-minimizing-simulation-sickness-vr/2383/
  13. Adi Robertson(2014) VIRTUAL REALITY PANIC
  14. Adi Robertson(2014) VIRTUAL REALITY PANIC
  15. Luke Edwards(2013) At last, an Oculus Rift sex simulator is here: Had to happen (NSFW)
  16. Michael Heim (1993) The Metaphysics of Virtual Reality
  17. Ian Snyder (2014) The Floor is Jelly