Experiments in 3D Panoramic video

For the purpose of creating a multicamera panoramic video rig I purchased several small inexpensive Mobius cameras. Before building a full spherical panoramic video rig I have done some experimenting with what can be done with many little cameras. This is one of those experiments.

Sample Panoramic stereo videos

For a stereo image it is necessary to record the same location from two points of view at the same time, similar to the human eye. I experimented with two methods of aligning the cameras for capturing panoramic stereo.

1 - Evenly spacing the cameras pointing outward in a circle. Use the left side of each camera for left pano; use the right side of each camera for the right pano. This allows the cameras to be packed as tight as possible to lessen the effects of parallax in the panorama. This type of layout results in many seams, and the seams for the left and right panorama are in different places.

2 - Evenly space pairs of cameras. The camera pairs are aligned parallel so they capture the same scene at the horizon. Requires a few more cameras or wider lenses because there is a loss in horizontal FoV by putting the cameras parallel but the 3D results are far superior. This type of layout results in half as many seams, and the seams for left and right panorama are in the same location, so they are less obvious.

I used inexpensive Mobius Action Cameras that I purchased off of eBay with the wide angle lens option for about $69 each. The cameras are mainly used by RC aircraft owners for recording flights and providing real-time video for navigation. The cameras capture very good video at 1080p at 30fps. You can learn more about he camera on the RC Groups forum.

The pairs of cameras are mounted vertically to capture the most vertical FoV as possible. The pairs of cameras are placed overlapping so the panorama stitching will have the least amount of parallax error. Then the overlapping pairs are spaced evenly in a circle facing out cover the full 360° panorama in stereoscopic 3D.

With the wide angle lens that comes with the Mobius camera it required 7 pairs of cameras to get enough overlap between images to blend the panoramas together; placing the pairs about 51° apart. This 7 camera pair rig has an Inter-Camera Distance (ICD) of 80mm and a distance between the No Parallax Points (NPP) of 63mm.

 

3D Camera setup

All 14 cameras laid out with the lenses set up in 7 pairs. The cables go through holes to connect to the camera brains, controls, and memory card.

small holes in two pieces of plywood snugly hold the sensor and lens in place.

3D camera setup

The two pieces of plywood are held together with three wood screws. All the cameras are numbered or lettered and color coded.

3D camera setup

The rig is mounted on the center pole for a tripod. The camera brains are currently being held in place with elastics. The elastics make it easy to manipulate the cameras to get at the USB port and memory card. The plan is to have two circles of 7 cameras spaced far enough apart that it is still possible to access the memory card and USB port.

3D camera setup

Close up of the cameras to show how closely they are packed.

  3D camera setup

This is the current rig.

I attached the Mobius mounts to a piece of PVC pipe. I can now easily remove individual cameras. This eliminates the elastics making it easier to access the buttons, memory card and USB connection. Looks much more professional.

  3D camera setup Seven mounts spaced evenly out around the pipe fit perfectly. The camera lenses align with their bodies. Now the mic points into the same direction as their lens. I get 14 directional recordings that can be combined for surround sound.
  3D camera setup

At first I had the lower row close to the top and half way between the columns of the top row but this made it hard to access the USB port of the top row. Now they are only slightly offset to allow a gap between the cameras to align with the USB connection.

I attached an end cap to my tripod to mount the S3D panoramic camera rig.

An earlier experiment with the cameras laid out in 6 pairs. Shown here with only 8 of the 12 cameras. Unfortunately there was not enough overlap between images to blend the panoramas together. I later arranged the 8 cameras into what is now my 14 camera layout. With just 8 cameras I was able to capture 236°.

Eight cameras are enough to show what is possible with a multicamera stereo panoramic rig and justified getting the additional cameras.

 

The cameras recorded 1920 pixel on the long side at about 116, resulting in 360 video that is 6k wide per eye. The videos are downsampled to fit the H.264's 9 MP limit when combined into an over/under format. Resulting full spherical stereoscopic panorama are 360° X 180° and 3072 X 3072 pixels.

I used PTGui and VideoStitch to process the footage.

One of the important things for stereo video or panoramic video is to synchronize the cameras so they all fire their shutter at the same time. The Mobius cameras can not be synced and it is often difficult to get all the cameras to start to record within a few seconds of each other.

I do the synchronizing in VideoStitch. It can use audio, motion, or flash to do the synchronizing. For sound a couple loud claps usually works well. For motion quickly lift the whole rig. All the cameras go from still, to showing massive amounts of movement. I set each camera to the frame that has the first sign of movement. But it is still possible that some of the frames are most of the way though to the next frame by the time the last frame starts to shows movement, so further fine tuning may be needed.

Because it can sometime take a while to get all the cameras up and running I use a dark bag to put over all the cameras until they are all recording. Then I remove the bag. That way I can quickly fast forward to the synchronization spot.

The Mobius cameras have several niceties but also has few limitations.

Pro:

  1. Inexpensive. No paying for things like screen and flash.
  2. Can be ordered with a 116° HFoV lens "B" or even slightly wider lens "C".
  3. The lens and sensor can be detached; moved away from the camera body with an extension cable. Allowing the lenses to be packed even closer together, lessening the amount of parallax error in the final panorama.
  4. Easy to disassemble and modify.
  5. Small and Light weight.
  6. Takes a standard 12mm lens allowing even wider FoV lenses.
  7. The cameras white balance can be locked to a specific temperature.
  8. The cameras can be set to auto white balance and locked to maintain one white balance for the entire shoot.
  9. The cameras can be set to auto exposure and locked to maintain one exposure for the entire shoot.
  10. The cameras are still having their firmware being developed.

Con:

  1. Extream cold of -20C would result in some of the cameras producing weird color.
  2. Camera sometime stops responding to button presses and need to insert a wire in the reset switch.
  3. The cameras cannot be started all at the same time. Even setting them to start with power on and plugging them in does not work. I got as much as 8 frames different with 8 cameras. Nor having them first on standby then add power. Likely if they were all on standby and the trigger buttons were all wired together then they would start a bit closer together. The connectors are very small and need optical-isolation switch to trigger them. skills and hardware I don't have. So I have not tested.
  4. The camera does not have a video mode that takes advantage of the full size of the image sensor. The sensor is 3:2 but the video is cropped down to 16:9 using only 84% of the possible video height requiring that many more cameras to cover the full 360°.
  5. Rolling shutter resulting in a jello effect in some video. Global shutter is necessary to freeze each frame as it happens.
  6. Can not record at 60fps at full FoV.
  7. Part of the camera options are extension cables to move the lens and CCD away from the camera guts. This allows to tightly pack lenses lessening the parallax errors, but as a trade off leaves a mass of equipment in the nadir.

Unfortunately to get 1080P at 60 fps using global shutter synchronized over several cameras is not going to be a $80 per camera solution anytime soon.

The limitations of 30fps, a rolling shutter that causes jello, and the inability to genlock the cameras does not make this a professional solution. It does make a very inexpensive solution that is fun to experiment with.

Options for Spherical Mono Panoramic camera rig

Using the wide angle "C" lens option that is available in Jan 2015, 6 around in portrait pointing slightly down to cover the horizon and 2 more in landscape to cover the zenith. This will capture almost 95% of the sphere.

In the future, the developer should add more recording formats, including an uncropped one using the full CCD but at a reduced image size. Then will be able to arrange the cameras into a cube format with every other camera rotated 90. Timelapse and photo mode use the full CCD. But to be useful requires an external control to trigger all the cameras at once.

Swapping out the stock lens for something wider should allow using only 4 cameras in a tilt up, tilt down, tilt up, tilt down config similar to the Sphericam.

Swapping out the stock lens for something wider should allow using 6 cameras in a tilt up, down, up, down, up, down configuration.

Print your own 360 S3D Panoramic camera rig

Bartosz Barlowski created a 3D printable model based on my design that you can download from Thingiverse.

Page last modified February 5, 2014
All content is 1991-2014 All Rights Reserved.
All trademarks are property of their respective owners.