- Next-generation video content production from volumetric to virtual production
- “POLYMOTION STAGE” Volumetric video recording system
- Introducing a volumetric video recording system consisting of over 100 cameras
- Expectations for cooperation with virtual production and motion control systems
- Interview with Nikon Creates CEO Yasuhiro Miyahara
Next-generation video content production from volumetric to virtual production
In April 2022, Nikon launched Nikon Creates for a new business that plans, shoots, and produces next-generation video content. After that, in October 2022, they will start providing services using the free-viewpoint 3D video volumetric video shooting system “POLYMOTION STAGE” in Heiwajima, Tokyo, and in January 2023, they began services using virtual production. I was able to visit the studio and interview at this timing, so I would like to report it.
Nikon Creates has named this Heiwajima base the Heiwajima Stage, a complex photography facility, which means that it has equipment for volumetric video, virtual production, a motion control system, and a high-speed camera. This is because it is a base where multiple cutting-edge shooting technologies are gathered.
In the virtual production studio, in addition to the main LED of about 10.4m x 4.1m, LEDs are also arranged on the ceiling and on the left and right sides, the camera is ARRI AMIRA, the camera tracking system is RedSpy, and the media server rendering. It has a full-fledged configuration, such as using disguise for the server.
A motion control system has been installed in the studio space for still and video shooting. The Phantom VEO4K capable of shooting 4K video at 1000fps was adopted for the BOLT and high-speed cameras of Mark Roberts Motion Control (MRMC), a British robot control solution provider that became a Nikon group company in 2016.
“POLYMOTION STAGE” Volumetric video recording system
As for the volumetric imaging system, Nikon Creates’ Heiwajima Stage is Microsoft Mixed Reality Capture Studios (MRCS)’s only base in Japan. In addition to hardware such as depth cameras such as the Azure Kinect that have been commercialized, and head-mounted displays that recognize the real space and display holographic images in space such as HoloLens 2, Microsoft has also been working on the volume since around 2018. We are developing a metric video shooting system service.
There are five official partners around the world that use the MRCS system, and the Heiwajima stage of Nikon Creates is one of them. The other four locations are Los Angeles, Washington DC, London, and Seoul.
Rendering processing of MRCS official partner studios is performed by Microsoft’s cloud platform “Azure”. On-premises high-end servers are no longer needed, and it is possible to reduce the large-scale equipment costs that are an issue in opening a volumetric photography studio.
It also features compatibility between official partner studios. For example, the music video for Coldplay x BTS’s song “My Universe” (2021) was filmed at Dimension in the UK and Jump Studio in Seoul. Seamless synthesis is possible. Nikon Creates may also develop cooperation between partners in each country in the future.
Introducing a volumetric video recording system consisting of over 100 cameras
As for the configuration of the shooting system, MRCS bases around the world seem to have the same configuration, and more than 100 4K 30fps cameras are used in synchronization at the Nikon Creates studio. According to the official MRCS web page, you can fine-tune the settings according to the content of the shoot, such as bringing the camera system closer to the subject to shoot at higher resolution.
The captured data is provided as serial number files such as obj and png, and can be read and edited with general-purpose 3DCG software. There was also a space where you could monitor the shooting in live and 3D.
Expectations for cooperation with virtual production and motion control systems
At the CES exhibition held in the United States in January 2023, Nikon demonstrated a system called “Unreal Ride” that links virtual production and motion control systems. Visitors straddle futuristic motorcycles, and the background LEDs show images as if they are speeding through the city. Z 9″ was the configuration to shoot. (Nikon CES 2023 – Virtual Production)
At the Heiwajima stage, it was said that there was no demonstration content shot in conjunction with volumetric, virtual production, or a motion control system, but in the future there will be examples of using cutting-edge technology similar to “Unreal Ride”. I hope that Only then will the Heiwajima Stage, a complex photography facility, be able to demonstrate its true value.
Interview with Nikon Creates CEO Yasuhiro Miyahara
We asked Mr. Yasuhiro Miyahara, CEO of Nikon Creates, about the background behind the launch of the volumetric photography studio and the aims of the service.
――I was surprised at the announcement of the opening of a volumetric photography studio under the Nikon brand. Please tell us about the process leading up to the establishment
Mr. Miyahara:
In 2018, Nikon subsidiary MRMC collaborated with MRCS partner dimension to launch the POLYMOTION STAGE volumetric video capture studio service in the UK and Europe, and the high-quality data received a great response. Around the same time, Nikon’s head office was also considering introducing a volumetric video system, so we started sales with a similar system this time.
――Was the Heiwajima stage a complex of virtual production, still video shooting stage, and volumetric from the concept stage?
Mr. Miyahara:
From the planning stage, it was a concept of a complex photography facility. We also made it a condition to have as large a space as possible and to open near Haneda Airport in anticipation of demand from overseas.
――I had imagined that the studio would focus on Nikon products, but when I toured the studio, I was surprised to see a difference.
Mr. Miyahara:
Nikon Creates is a content production company. As a production company, we give top priority to what is necessary to realize cutting-edge video production.
――Is it possible to link virtual production and BOLT?
Mr. Miyahara:
Each stage has a slope, and it is possible to move BOLT with an electric pallet. We have already worked with the virtual production and the motion control system. We plan to verify the linkage between Volumetric and BOLT in the future.
――Lastly, still images, videos, and 3D have different targets.
Mr. Miyahara:
I expect that future creators will combine new technologies to create unexpected content. At that time, I think that the conventional separation of still images / moving images / 3D will disappear. We aim to use this facility with new ideas and become a place where creatives beyond imagination are born.
Takayuki Aoki｜Profile CEO
of Kadinche Co., Ltd. In 2009, received a Ph.D. (policy and media) from Keio University. After working for Sony Corporation, he established Kadinche Corporation. Engaged in software development related to XR at Kadinche. In 2018, he established Miecle Co., Ltd., a joint venture with Shochiku Co., Ltd., and opened Daikanyama Metaverse Studio in January 2022 to work on content production using virtual production methods.
