Digital Footprints

I received a projector for my birthday last year, and ever since then I've been wanting to do something related to projection mapping with it. Projection mapping is a process in which a projector is calibrated to map between in-world locations and screen coordinates, and is used for all sorts of art installations and themed attractions. In taking a course in interactive electronic arts with Professor Joshua Rosenstock, I finally had the chance to learn more about the topic and incorporate it into what I like to do with art - create things that elicit emotions and realizations about how the world works.

Design

Digital Footprints is an interactive art installation that consists of a projector and a camera, both pointed at the ground, and a computer hooked up to them to coordinate the images. Viewers are given a QR code to scan which leads to a webpage where images can be uploaded. The computer then retrieves the images and uses the camera to detect where the viewer is standing, projecting a collage of their images as they move around the area. The project is play on the idea of a digital footprint, as the viewer's images stick around behind them just as anything uploaded to the internet sticks around.

I took inspiration from other art displays that make use of similar projection mapping effects, such as the one at the Boston Museum of Science. Those displays typically make use of special motion tracking cameras or LIDAR sensors, so I set my expectations fairly low as I only had a regular camera to work with. The collage aspect of it was somewhat inspired by other collaborative artworks like r/place. I think those kinda of art pieces are really interesting since they form such a unique means of connection people.

Process

I experimented with a few different projection mapping techniques before I came up with one that worked well. At first I was going to use edge detection to find the boundaries of the projection field, and then calculate the coordinate transformation from there, but that proved to be overcomplicated for my needs. Instead I created a calibration system which projects a series of dots and then compares the images with and without the dots, creating a mapping between where the dots appear according to the computer and in the real world.

One of the reasons I spent so long on the calibration process was because I wanted the system to be portable - I wanted to be able to bring the whole thing to class on the last day to be able to showcase it. Once I had that working the rest of the design was fairly straightforward. I set up a simple Node.js web server for the image uploading portion and added a quick preprocessor to ensure that all of the images were in a format that my program could read. The program uses the mapping from the calibration to convert between the viewer's location in the camera's view and in the real world, and periodically places an uploaded image on the canvas behind them, creating a unique collage of the viewer's images.

Conclusion

I was really happy with how this project turned out. While a little finicky, the projection mapping turned out way better than I had expected and it was a lot of fun seeing my peers interact with it at the end of the course. I would have liked the opportunity to try this out with a larger group of people or over a longer period of time and see the generated collage develop further. I don't often have the opportunity to work on art like this and do it for a course credit so having the chance to do this during an otherwise extremely busy term was a lot of fun.