An idea for a photo viewer interaction in VR, you can pull the images towards you and be enveloped by photographs.

Hanging a photo display in its original context and experimenting with cloth simulation in AR. It feels more real when it reacts to dynamic user movement.

Testing interactive softbody dynamics with an optimized photogrammetry model.

---

overview

.

If you've ever light painted before, you know that it's iterative since one can only imagine the path the light takes through the scene. You run around in the dark with a flashlight, and then peer at the camera to see if the shot turned out.

We love doing this, but for VR, we wanted to create an experience that doesn't exist in reality: slightly eerie, otherworldly, and supernatural.

This project is an ongoing labour of love, but we're sharing our process here to give people a sense of where we are headed.

concept

.

Our goal is to makes it feel as if you are painting with light on a foggy beach at night with the stars all around you, as though you're inside a long exposure shot.

I created the video above (Tiltbrush + Cinema4D + Octane) to show a possible tone and visual target, and Michelle worked on creating a corresponding interactive VR experience in Unity.

For reference and inspiration, and to create both the rendered scene and the Unity environment, we went to Tofino, BC and shot a variety of 360 videos, panos, and timelapses, and reconstructed parts of the beach using photogrammetry.

Michelle and I running from the camera

Awe / wonder

Mystery / fog

Fire / silence

Moonlight / water

Celestial / ominous

Beach sphere

photogrammetry

.

I took hundreds of shots of the beach to recreate a large section of Long Beach. Our photogrammetry tool of choice, Reality Capture, produces photorealistic results with an extremely high poly count. We worked with 3D artist Reece Mills to research how to bring down the size of the model and texture, while preserving its perceived quality.

Photogrammetry process

drawing with light in vr

.

Michelle implemented our light-painting to be similar to what you might experience with a very basic Tilt Brush. Once she had a "flashlight/wand" painting a trail, she started exploring both how to make it look smoother/more fluid, and what else the player could do with the light once it was in the scene.

We asked ourselves when you paint, how does it affect the scene? Why do you paint—does it reveal the next element in the scene, or move a story along? Is it simply a fun mechanic that you can play with, and should it have physics? How physical is this paint?

We are currently working on some of the story-based options, but in our tests, we also looked at how it could help create the right ambience (more on that below).

creating ambience with light and fog

.

Long Beach in Pacific Rim National Park (near Tofino, BC) is one of my favourite places, especially at moonset, when the stars come out. The shadows and fog make it feel like you are in another world. We wanted to see if we could use fog and lighting in Unity to create a similar feel in our experience.

Running in Unity. Still a work in progress, but it's exciting to see it come together.

ongoing work

.

Light painting in Tofino is just one instance of a memory we both really enjoy. As we work on this project, we're always trying to understand better how this gets us closer to reliving a memory, or perhaps experiencing someone else's interpretation of that memory.

We are always wondering: What other things can we create that would help us revisit an event from the past, or just take us back to a happy place?

We'll share more as we learn. The road is long, and we look forward discovering it.

---

Let's go for a VR hike

.

Photogrammetry and VR go together like peanut butter and jelly - one of the main reasons I started learning photogrammetry was to bring it into the Virtual Reality. On a hike to Hanging Lake (just outside of Glenwood Springs, CO) I was inspired to capture a selection of scenes along the trail. I'm currently working on compiling those scenes into a single stereoscopic spherical video to publish on Youtube, Vimeo & Facebook's 360 platforms.

Boat Life VR

.

One of my favorite ongoing projects is to create an interactive VR scene using the photogrammetry scans of boat I lived on last year. The aim of the project is to attach photos and videos to relevant points of the model and create a sort of holistic documentary experience. You'll be able to walk through the various cabins as well as interact with multimedia content in that space. For now, I've compiled a 3D Spherical video as a preview.

---

Merging a Tilt Brush painting with the real world by tracing a photogrammetry model in VR and compositing in C4D and After Effects.

Prototyping what an AR experience with perfect surface reconstruction could look like. Created with photogrammetry & C4D / Octane.

Tracing a photogrammetric stump in Tiltbrush

---