Update: The paper is now available online, here!

I spent my summer as an intern doing research at the MIT Media Lab. I worked in Professor Pattie Maes' group: Fluid Interfaces. Fluid Interfaces' mission is to "radically rethink human-computer interaction with the aim of making the user experience more seamless, natural and integrated in our physical lives." It's the same group that created Sixth Sense a few years ago.

My main focus was on an augmented reality (AR) platform, working with Valentin Heun on his Smarter Objects system. And I will be continuing this research into the fall semester.

The Beginning

At the beginning of the summer, I learned the basics of augmented reality on iOS by creating a simple demo app. The app detects my watch, and then displays the time on top of the watch face. Besides helping anyone who can't read an analog watch, it's only goals were to gain personal experience in AR programming and hint at the possibilities that can be achieved with augmented reality. See the video for a quick demo.

Main Project

My main focus for summer was to build an augmented World Wide Web for the physical world. It allows anyone with basic web programming skills to be able to add content to the augmented system. The system uses a combination of Objective-C, C++, OpenFrameworks, HTML, CSS, JS, (and a few other things - the exact stack has not been published yet). I spent the most time on allowing different layers of the stack to communicate with each other (such as the C++ to JS and Objective-C to OpenFrameworks). The below shows three demos where this system could be used. The first is to see what time tours start in the main lobby at MIT. The second shows a reservation system for a conference room. The final application shows a public message board, where you can draw and share images. The publication that was submitted at UIST 2014 is the culmination of this system, which we call the Third Surface.

Publication

The Third Surface system was published as Third Surface: An Augmented World Wide Web for the Physical World at the ACM's 2014 User Interface Software and Technology Symposium (UIST 2014).