Volumetric Vol. 3: Sony’s Portable Volumetric Capture System Release

Sony PCL “Kiyosumi Shirakawa BASE” Difference from VOLUMETRIC CAPTURE STUDIO

Speaking of Sony’s Volumetric Capture, the VOLUMETRIC CAPTURE STUDIO, which was developed by the head office and has started operation at Sony’s PCL “Kiyosumi Shirakawa BASE”, will come to mind first. In the Kiyosumi Shirakawa BASE studio, over 100 cameras are installed in a space covered with a green screen. The studio boasts high image quality, which is used for filming commercial videos and music videos, but because high image quality is the goal, which is why it takes a lot of time and money.

The portable type this time has less than 1/10 the number of cameras, and does not require special backgrounds such as green screens. It’s just standing there. The quality of the volumetric video is not as good as that of Kiyosumi Shirakawa BASE, but the simplification of the system not only makes it possible to use volumetric video for more purposes, but also provides the data at a later date. It can be understood that it is aiming for the democratization of volumetric video, such as supporting live streaming as well.

In conventional volumetric capture, it was common to send the subject to a dedicated studio with a fixed camera and to deliver the data at a later date, as it takes time to process the data. On the other hand, with portable volumetric capture, as introduced in the above YouTube video, the shooting team can bring a set of systems and build a temporary shooting system for business trip shooting. Just as TV stations use a combination of studio and location (location) shooting, volumetric capture can also be used for location shooting, so it can be said that it has a complementary relationship with existing studios. deaf.

Reason for reducing the number of cameras

Then, why was Sony able to reduce the number of cameras so much? While most conventional volumetric studios perform 3D reconstruction using the Multi-View Stereo method using multiple RGB cameras, portable volumetric capture systems combine ordinary RGB cameras and depth sensors. Therefore, the data is created using the depth data as well.

With this method, the number of cameras can be reduced, and by reducing the number of cameras, the amount of filming data can be reduced, and the number of computers required can also be reduced, along with data storage. This also contributes to real-time performance, which is necessary for the realization of live streaming. Since the depth sensor is used, the measurable distance means that the depth range is also the range that can be volumetric captured.

When to Use Portable Volumetric Capture

On November 30, 2021, the Sony Group signed an “Official Virtual Fan Engagement Partnership Agreement” with Manchester City Football Club. This time, they actually brought the portable volumetric system to the Manchester City stadium and filmed demo content of professional soccer players.

Using the captured data, they created virtual fan engagement demonstration content in which players appear in 3D on the Etihad Stadium, the home stadium of the team, which is realistically reproduced in virtual space. This is a demonstration that proposes a new use of the Metaverse, in which virtual spectators get excited while watching athletes move in a virtual space.

At CES 2023, Sony also announced a 27-inch version of its Spatial Reality Display, and volumetric video is also suitable for content for such 3D displays. In other words, Sony not only provides a volumetric video shooting system, but also offers live streaming transmission, a system that can display it on the metaverse, and proposals for displaying live-action 3D content at once. We have a system in place to deliver it to users on a consistent basis.

As the use of VR head-mounted displays and metaverses increases in the future, it is expected that the need for not only CG but also live-action content will increase. Under such circumstances, we expect that the portable volumetric capture will be used all over the world and the content will increase.