What is the Sony Ocellus?
The Sony Ocellus is designed to simplify and facilitate augmented reality (AR) and virtual production applications in broadcast and cinema by providing marker-free camera tracking through multiple sensors. OCELLUS is camera agnostic and can be used with both cinema and broadcast cameras, according to the Sony press release.
There are several methods for camera trackers to detect the position and orientation of the camera. There is the outside-in method, which monitors the direction of the camera from the outside, the inside-out method, which recognizes external markers using a sensor attached to the camera, and the inside-out method (marker-less), which recognizes position information by image analysis without using markers. Each of these methods has its own characteristics and challenges.
The outside-in method tends to require large-scale systems and high implementation costs. In addition, because external monitoring is required, it is primarily used indoors. The inside-out method (marker method) requires the installation of markers, which limits the shooting space. This method is also primarily used indoors. With the current product lineup, the accuracy and stability of location information remains an issue for the inside-out method (markerless).
To use Unreal Engine, for example, other systems require the use of markers, making it unsuitable for free space shooting. These markers also require preparation work, which is time-consuming. In addition, current marker-less camera trackers have issues with the accuracy and stability of position information.
In light of these issues, Sony developed OCELLUS based on its own image processing technology.
Highly accurate detection of position information from multiple directions
The name OCELLUS comes from the term for the eye-like pattern on the wings of butterflies and peacocks, or the simple eye that insects have to detect light.
OCELLUS is equipped with numerous sensors, each of which acquires information independently, so the name is derived from these words. These sensors function as if they were eyes or a single eye, detecting location information with a high degree of accuracy.
The features of OCELLUS include the fact that it can be used without a marker and supports tracking in various spaces. It also has a compact and easy-to-use design.
OCELLUS consists of three main components: the sensor, the processing box, and the lens encoder.
The most important part is the sensor section. The sensor section is equipped with five Sony image sensors. These sensors face in various directions and capture images from each direction. By analyzing the images from the sensors, feature points are extracted and the camera’s position and orientation are estimated. This mechanism makes tracking possible without using markers.
In addition, the sensor is arranged on multiple sides, so tracking can continue even if part of it is blocked. When the sensor sensitivity decreases in dark places, the system is designed to enable stable position measurement by providing assistance with infrared light. An infrared LED is installed, enabling stable tracking even in dark places. The IR LED is located next to the sensor.
OCELLUS tracking data for real-time and post-production use
There are two ways to use the tracking data acquired by OCELLUS: in real time or to record and use later. For real time use, tracking data is output in free-d format from the LAN terminal for in-camera VFX in virtual production. For recording and use later, tracking data is recorded in FBX format on the SD card slot in the processing box and used in post-production. The OCELLUS supports both of these usage methods.
OCELLUS is equipped with an SDI video input terminal, and has the ability to obtain lens data from here. Recent cameras and lenses have communication functions, and the camera recognizes lens focus and iris information. This information is output to the outside via SDI. If you are using a Sony camera with a compatible lens, you can input the SDI signal from the camera to the OCELLUS’s SDI input terminal to obtain lens focus and iris information on the OCELLUS side. This makes it possible to obtain lens data without using a conventional lens encoder.
If you are using a lens with communication functions such as a “/i” terminal or Sony E-mount, you can obtain lens information by connecting it to the SDI input terminal of the OCELLUS.
However, not all lenses support the above features, and in some cases, such as when using older lenses, it may be necessary to obtain lens data using a conventional lens encoder, which is why the OCELLUS also comes with a lens encoder.
Lockable USB Type-C and Fischer Connector – OCELLUS Connectivity
Like the VENICE, the OCELLUS has a robust design, ensuring reliability and durability for use in harsh shooting environments. The sensor and processing unit are connected via a USB Type-C cable. This connection is equipped with a locking mechanism to prevent the cable from accidentally coming loose. A highly reliable Fischer connector is used for the power input.
OCELLUS also takes thermal design into consideration, and is equipped with a cooling fan to ensure stable operation under certain conditions. The product design incorporates measures to enhance reliability, such as the use of industry-standard Fischer connectors and USB terminals with locking mechanisms.
Compatible with cinema cameras, system cameras, and cameras made by other manufacturers
OCELLUS is designed to be used with a variety of cameras, not just Sony products. However, the function to obtain lens data using the SDI input terminal can only be used in combination with compatible Sony cameras. When using cameras from other manufacturers, obtain lens data using the included lens encoder. Sony cameras are also compatible with HDC series system cameras, as well as cinema cameras such as VENICE.
Hopes for use in situations where tracking is difficult, such as outdoors
Sony OCELLUS is intended to be used in a wide range of video production scenarios, including in-camera VFX in virtual production, green screen compositing, AR (Augmented Reality), and post-production, where camera tracking technology will be used for real-time CG compositing and post-production.
It is particularly promising for use in environments where conventional tracking has been difficult. Because OCELLUS uses marker-less tracking, it is considered suitable for use in vast spaces such as outdoors.
