Recording Spatial Video and Viewing it in Apple Vision Pro
During our third demo with the headset the nuances of spatial video come to life
In iOS 17.2, Apple enabled Spatial Video recording on iPhone models 15 Pro and Pro Max. These iPhones capture spatial content using the phone’s main and ultrawide cameras simultaneously, and the format utilizes the stereoscopic difference between the two camera positions and focal lengths to render three dimensional video when viewed using the Apple Vision Pro headset. My first Spatial Video demo got me thinking about use cases, and I’ve been recording a range of content to try out. Yesterday I had the opportunity to see my videos come to life on a Vision Pro headset.
Recording Spatial Video on an iPhone requires a bit of imagination—what you see on-screen is the same flat, familiar preview you see in the regular video recording mode. I was curious to experiment with light, reflection, distance from the subject and movement in my recordings and had both correct and incorrect guesses about how they’d feel in the Vision Pro. Here are my initial findings and impressions.
Available Light
On a few occasions while recording the iPhone warned “More light recommended.” I was pleasantly surprised when viewing that content in the headset: the movies still had plenty of depth and the image quality remained crisp and not grainy. One clip, certainly a moment from the future, was from a night drive in an autonomous Waymo on the streets of San Francisco. In another clip there was a strong, focused spotlight on peonies sitting on a table in a dark bar. This contrast made for a nice dramatic moment and even the objects in the shadowy background had depth.
Distance from subject
Keeping the subject between two and eight feet from the camera resulted in plenty of depth between foreground and background. The best scenes had multiple focal points within that two to eight feet range, and gave the most significant and satisfying sense of immersion. A slow walk through Ugo Rondinone’s stained glass clocks at Art Basel Miami Beach last week played back just like my experience in the original visit.
Movement
Movement is where it gets a little more complicated. Knowing the viewer will be static and the content immersive, it’s best to avoid creating too much difference between what the viewer is seeing and what they’re feeling because it can feel disorienting and create nausea. Shots with the camera stationary (such as on a tripod) are ideal, but camera movement is often necessary or desired to make a satisfying story. I found that recording linear and gently curved movements, especially in open spaces, created videos that were easy to view. By contrast, sharp turns in close quarters created more jarring videos. Given that a stationary camera is ideal, we found that optimizing for movement of the subject is even more important for dynamic, engaging experiences. In multiple videos with Rory, our jovial canine mascot, his presence was felt as he ran, played and chomped on his toys.
Sound
The audio captured in my video recordings completed the spatial experience on the Vision Pro, and I had an incredibly accurate sense of where the sounds were coming from. Closer sounds were especially satisfying—Rory’s chomping on a chew toy had ASMR-level satisfaction.
I also had a chance to look at some static images I captured with a DSLR and my iPhone, non-spatial (2D) videos, and panoramas. Similar to my experience at the previous demo I was keen to expand the image viewer as large as possible and placed it on the wall across the room. I immediately noticed how delightfully exacting the color rendering of photos was in Vision Pro compared to the iPhone 15 Pro Max screen—the gentle sunset pinks behind thick rain clouds over a dark ocean looked the same to me as they do on the iPhone’s HDR Retina screen.
Looking at panoramas in immersive view was also a treat, and surprisingly it wasn’t the distant landscape shots that were the most impressive: the panoramas from tighter, closer spaces—like at the Karnak Temple in Luxor, Egypt or JR’s epic collage, “The Chronicles of Miami,” rendered so perfectly in the headset that they almost felt 3D, despite looking distorted here in their flat view.
Apple has not yet announced a release date for Vision Pro beyond “early next year” and after this third demo of it that timeline feels promising. Every nuance of the experience has tightened and improved with each subsequent demo and we’re eagerly awaiting its release. Simply viewing photos and videos in the headset is already a luxury of focused attention combined with incredible color and resolution quality, and the prospect of creating content for this new spatial world is intriguing.
What are your thoughts?