It almost had to happen. Tom Wait’s spooky spoken word song “What’s He Building in There,” is so evocative, so visual that it’s like film that plays in your mind. The challenge, though, is how to actually make a film that does justice to the genius of the original piece.

Ricardo Rivera, visual artist, filmmaker, and founder of Klip Collective, began exploring video projections when he worked as a club VJ in Philadelphia. “In 1998 I was playing around with Photoshop and discovered how to map images to surfaces,” recalls Rivera. “When After Effects added the ability to preview work through a mini DV connection, I discovered that I could easily play content through a digital projector.” Rivera pointed the projector at a wall in his kitchen and used it to canvas the surfaces. “Then I masked all of the elements in the kitchen using Photoshop and created what was, in effect, a multi-channel projection feed through one projector and one feed.”

Once Rivera had figured out the workflow, the possibilities were endless. Today Klip Collective holds two patents on projection mapping, a technique whereby video content is projected onto non-traditional display surfaces such as the sides of buildings, often as site-specific art. Different physical surfaces come to life in unexpected ways in a dance of shapes, color, and imagery, melding the permanence of architecture with the transience of light. These are the kinds of new frontiers for art that digital tools make possible.

“We map with After Effects, tap into a projector with a laptop as a second or third display to get the 1:1 ratio, and then map the resolution,” explains Rivera. “The key is to maintain the 1:1 pixel ratio so that the final result remains visually coherent across the various surfaces and angels. It has to be accurate down to the millimeter for everything to work. It’s a very intense process.”

In addition to working with Photoshop CC and After Effects CC, the team recently switched to Adobe Premiere Pro CC.  “We initially used Final Cut Pro but moving to Premiere Pro makes our workflow so much faster,” says Rivera.” The integration between the Adobe products is fantastic.”

And they’ve had commercial success with the approach as well: the company works with ad agencies or directly with a diverse list of clients, including organizations such as Central Park Conservancy, the New York Public Library, and Temple University to brands like Gillette, NBC Sports, New Balance Nike/Jordan, Target, Urban Outfitters, and more. Visual installations by Klip Collective have been featured in commercials, music videos, and films, as well as a whole range of different corporate, sporting, and social events.

And that’s where Tom Waits comes back into the story. Last year Klip Collective created a brilliant 15-minute projection-mapped installation of What’s He Building in There? as part of the New Frontier exhibit at the 2013 Sundance Film Festival. Projecting onto a suitably scary-looking building, the imagery took the viewer through the walls revealing strange goings-on in different parts of the structure. Then the walls seem to slide down into the ground, exposing the entire interior of the building. A remarkable piece in it’s own right, it also solves the problem of how to bring Tom Waits’ song to the “big screen” without losing it’s strangeness, or diluting the drama of the original piece with conventional storytelling techniques.

This year, Klip Collective is back at Sundance with a clever pre-roll trailer honoring past festival favorites, which is being shown at all film screenings, making it the only film this year that every Sundance attendee will see. The piece is a narrative, incorporating live action panels and 3D animation projected onto the front of the Egyptian Theater in Park City. A 10-minute story plays out and then loops back into abstract geometric forms against the complex façade of the building. The Egyptian Theater projections themselves will also be included in the Sundance New Frontier exhibit as part of a short called What’s He Projecting In There?

“We used Photoshop for compositing and to create the setting,” says Rivera. “Then we brought that into After Effects and composited everything together to build the base map.” The team filmed all of the live action on green screen. “There are four zones and the characters moved between them.” The Klip crew used Premiere Pro for editing and then moved to After Effects for the final compositing and timing. They used two HD projectors side by side to project the video onto the theater.

“I love how Dynamic Link lets me do a little bit of editing in Premiere Pro, easily go into After Effects for the mapping, and then finish in Premiere Pro, utilizing the alpha channel masks for the piece,” enthuses Rivera. “It’s so much more efficient than if we had to render content every time we wanted to move between our tools.”

And Kilp Collective is already developing ideas for new projects that open up the interiors of buildings to tell their stories. “We have a new project in the works called Vacant America,” says Rivera. “I’ll tour America with my team and do site-specific installations on vacant buildings. For example, we may film a Campbell’s soup can factory and tell a story about that location through the use of projection mapping and the experiences of people who worked there. We’re currently bring partners on board and there’s a lot of interest in this project.”

Learn more about Klip Collective 2013  – or check out the Klip Collective Art Reel 2013

Learn more about Adobe Creative Cloud 

Download a free trial of Adobe Creative Cloud