3/11 update

Xbee wireless Joystick

enclosure test


I designed a system to limited the rotation. So the cube only rotates along yaw, pitch, roll three axis. Modeled in MAYA, and 3D printed with Makerbot Replicator 2X. The six pillars are made in cooper tubes, springs and marbles.



The sensor I used in this controller is a SparkFun 9 Degrees of Freedom  Sensor Stick. It works well and pretty accurate. However, I also considered about using an optical sensor which are usually used in mouse. It can detect how much distance the cube has rotated if I put  it on one of the pillars. However it was too big to fit in the tracks on my cube. Otherwise it would be a better choice than an IMU board. Because the optical sensor only needs to detect values that have traveled in two axis, while the IMU has to record values of three axis (actually 9 axis from all three sensors on the board and complicated calculations) and update those data very frequently. I will find out a way to shrink down the size of optical sensor if there is one.


I want Unity get sensor data from the IMU board wirelessly without latency. The first try was using Arduino and the serial library of MONO. Arduino can send data through serial port very easily. However, there is a significant latency when Unity tries use this library to read serial data very frequently. My solution is using Teensy. Teensy is a Arduino-compatible micro controller and it can be recolonized as a joystick (a HID device) . The joystick mode makes it much easier have a communication between Unity and Sensor.

The wireless module I used is Xbee. There is one Teensy inside the cube , getting the orientation data and sent it through Xbee to another Teensy that connects to the computer as a HID device. Unity get the data from this HID Teensy. Although the setup is a little bit complicated. The communication is stable and responsive.

Thesis studio2 Presentation


I asked myself a question—what is the relationship between reality and virtuality?Could one of them transform to the other?Are they the opsitie side of each other?

What can be virtual. Something not real, something not actual.


I used to believe that infinity is not real not actual.   Because I could have three candies, four candies, or even a hundred candies, but I could never have infinite candies. Infinity is virtual, because it does nor exist in reality.


I believed so until I saw this. When there are two mirrors facing each other. There is a seemingly endless line of images fading into the distance.

I found infinity–something not real–in reality.

Virtuality could be a reflection in a mirror or a computer simulated landscape. Both are not real, but displays the full qualities of what is real.

And This is what we usually do. We already in reality and we use tools to  experience or create a new reality in virtual world.

And this answers my question. Reality and virtuality are not opposite to each other. We sometimes  can find one of them in the other.


There is a recursion out there when we seeking reality in VR


This answer transforms to my thesis.

It is a Virtual reality game design to express this relationship. It uses an Oculus as a visual output and a cube shaped physical interface as a controller. Players can see the room they are in from the VR device. Whenever players tilt the cube, the scene in the Virtual reality room will also tilts.


So the user is manipulating the environment from outside, from reality, while at the same time observing it from inside the Virtual space. Therefore a recursive experience is created. And there is a paradox of where you are leads the audience to think about where is the border between reality and virtuality.

About the game


the story of that game is that the player is trapped in a recursive looping maze. You moving through rooms that looks similar. And you try to escape from this maze.


The goal of the game is to collect those shiny flags in each room, and when  player collect all the flags , they are able to back to their reality  (or what they believe is reality).

And the game mechanic is to tilt the cube to change gravity or the rotation of the room .


About the style and aesthetic

The flag comes from MC Esher ‘s  painting Mobius strip I. Esher’s work inspires me a lot. The gravity manipulation idea and the scene design is also inspired by his work.  The ambiguation and paradox in his work could be suitable as a form to express my concept.

Prototype 3

The Arduino board is placed outside the cube in the last prototype, which makes it easy to organize wires and pins but not good for user experience. Audience has to avoid touching wires exposed outside. So I put the Arduino board inside the physical device this time and used a longer USB cable. For future prototypes I am considering add a wireless module on the physical part and use a circuit board instead of an Arduino board to scale down the size.

The physical part is built with plastic board to make a more stable structure. I also tried to create more feedbacks on the physical part. For example as the audience solve puzzles two buttons will flip out on the face of the cube and the audience can then do some operations by pressing the buttons.

An Oculus rift is connected to this prototype. Since there are bunches of sensors built in Oculus to detect the audience’s head movement, it is not merely used as an output. The audience can move his/her head to focus on an object in the virtual world to get hints.

All the efforts are made to balance the virtual part and the physical part of the project. They can respond each other and they are also connected with each other. Both of them are input and output at the same time. Each one contains one another. Just like yin and yang in Chinese philosophy—the two opposite or contrary forces are actually complementary.

image02 image38


image04 image03

image05 image06

image08 image07

Prototype 2

More elements and interactions are added in the second iteration. In order to test and assemble quickly, I used cardboard to build the physical puzzle. Audience can spin the bottom part of the cube like a Rubik’s cube.  There is a reed switch built in to detect the rotation of the bottom part. If all the puzzles are solved the top of the cube (which is locked by a servo) will open up. Other puzzle elements like code and locks are also tested in this iteration. Image.6 is a screenshot of the digital output.

image35  image34


Presentation 1


When I was looking for a thesis project. I started from something I am really interested. And one subject is Virtual reality.

In terms of technology, Virtual reality is typically refers to a system usually includes a computer capable of real-time animation, controlled by a set gloves or a position tracker, and using a head-mounted stereoscopic display for visual output.

The core of Virtual reality Is presence.

presence is the sense of being someplace  while in virtual reality; many people feel as if they’ve been teleported. Presence is an incredibly powerful sensation, and it’s unique to VR; there’s no way to create it in any other medium. Most people find it to be kind of magical, and I think that once people have experienced presence, they’ll want it badly.

With this feature, Virtual reality is mostly used for simulations and play experience.

However. If it’s a good thing when people are going to addict to it. When I search for Oculus rift projects. There are bunches of flight simulations and VR dungens or VR shooting games.


Then I also found something creepy . This is a contraption from Japanese company Tenga, which combines an erotic video game, a Fleshlight-like contraption, and the Oculus Rift to create a sex simulation.


This raises people ‘s concerns about virtual reality.

What meaning should we ascribe to virtual sex and violence? What happens when we voluntarily step into an alluring, machine-made, alternate reality — and surrender contact with the real world? (Sexual pleasure is a powerful reinforcer, the big hook. ) How will we deal with users who won’t, or can’t, return to their average lives? Will there be a new class of schizophrenics who are simply jacked-in forever?

And then I came up with an idea. What if I make a virtual reality experience that during the experience, the user has to take off the VR device the head-mounted monitor.


And that leads me to a big idea of my thesis  — to create a connection between virtual experience and physical world.


My solution is make the virtual world as another dimension of physical world that we can observe .

The form of presenting this idea will be a physical installation and a Virtual reality experience powered by Oculus rift. In order to make a connection, the physical installation and the Oculus will present a same thing in different forms. There will also be communications between this two dimensions when player do actions.

For instance,  the stuff we showing in the two dimension is a cube. When player put a hand on the physical device, the player might see a hand-shaped mark on the face of that cube in Oculus.

So Player may put on and take off Oculus repeatedly to observe the changements or interactions between this two world during the experience.

I realize This form provides possibilities of creating puzzles which is another subject that fascinate me .


This just reminds me the game “the room”.

The Room presents the player with a series of strange boxes that have a number of physical mechanisms on them. The player is challenged to figure out how to open each one – typically by undoing a series of locks – to access another puzzle box within it. The game uses a variety of motions enabled by mobile device touchscreens to simulate actions in real life, such as looking around the device, turning keys, and activating switches.


And I also looked for physical puzzle games like Rubik’s Cube.

The left one is a traditional Chinese puzzle made of pieces of woods. It is connected without glues and nails. you try to figure out how to desemble and reassemble those parts when you play it.

I am thinking bring this geometry constructions to my project as a part of the puzzle.

And the player may get stuck or have no action to do in the virtual world and have take of  the oculus and  assemble those kind of pieces in the physical installation to Get cules and hints and goes on put on oculus to solves another puzzle in virtual world.


Prototype 1

Prototype 1- communication between Unity and Arduino

I decided to use Arduino and Unity3D to build a prototype, since they are both powerful and easy to access. This is a simple prototype testing the communication between Arduino and Unity3D. The prototype shows the basic idea of connecting virtual and physical world. Press the button on the physical device (Img. 1), a virtual button will show up in Unity (Img. 2) and the button can control a LED on the physical device to be ON or OFF. I am using an Arduino plug-in Uniduino in this prototype. There is also a Oculus Rift plug-in allows Unity to access Oculus monitor and sensor.

 IMG_20141028_140023 Img . 1                    20141028140524 Img. 2