RealSense D400-Series Depth Cameras Are For Devs, Not Consumers (Hands On)

Intel launched a new generation of RealSense depth cameras this week, and they feature stereo cameras, updated depth modules, and Intel RealSense D4 vision processors. Intel’s RealSense D400 Series Depth Cameras offer advanced computer vision capabilities in a convenient form factor with USB connectivity. But these cameras aren’t built for consumers; you need to be a developer to get any meaningful use out of them.

Intel said it designed the new RealSense D400-series cameras for makers, educators, and developers. The D400-series cameras come in a compact USB-powered form factor for easy deployment in hardware prototyping projects. The USB connectivity also makes the new RealSense D400 cameras convenient options for prototyping software that takes advantage of the new depth modules and the integrated vision processor.

“Many of today’s machines and devices use 2D image recognition-based computer vision, but with Intel RealSense’s best-in-class depth technology, we are redefining future technologies to ‘see’ like a human, so devices and machines can truly enrich people’s lives,” said Sagi Ben Moshe, vice president and general manager of the RealSense Group at Intel. “With its compact, ready-to-use form, the Intel RealSense D400 Depth Camera series not only makes it easy for developers to build 3D depth sensing into any design, but they are also ready to be embedded into high-volume products.”

Intel’s RealSense D400 series includes two cameras that serve somewhat different purposes. The RealSense D415 camera features rolling shutter sensors with small lenses that have a narrow field of view. It also includes an Intel RealSense module D410 depth sensor, which processes depth information from the stereo cameras in real time. Intel said the narrow field of view of the D415 provides high depth resolution, which is “ideal for precise measurement.” The combination of smaller lenses, rolling shutter sensors, and the D410 module allow Intel to keep the price of the RealSense D415 under $150.

The RealSense D400-series also includes the RealSense D435 depth camera, which offers global shutter sensors and larger lenses for better low-light performance than the cheaper D415 camera. The D435 also features a more powerful RealSense module D430. Intel said that the D435 camera is the “preferred solution for applications such as robotic navigation and object recognition.” The RealSense D435 with its larger lenses and global shutter sensor is somewhat more expensive than the RealSense D415, but it's still under $200.

Both RealSense D400-series cameras support depth output 1280 x 720 active stereo depth resolution at up to 90 fps. The D400-series cameras can also capture RGB video at up to 1080p, but we experienced significant frame drops at that resolution. They can capture 720p video at a smooth 30 fps.

The Intel RealSense D400-series cameras feature a strong aluminum outer shell, which should protect the components from the elements and from abuse. The aluminum body of both cameras include slots on the top and bottom to allow air to pass through and keep the components cool. The bottom of each camera includes a standard ¼” thread for a tripod mount. Intel includes a miniature tripod in the box, but we’d suggest getting something a little sturdier for long-term use. The back of both cameras also includes two smaller thread points that you can use to mount the camera to the device you’re building.

So, What Can You Do With These Things?

Intel sent us one of each RealSense D400 cameras ahead of the launch so that we could give them a test run. Unfortunately, we haven’t been able to do a whole lot with them. Unless you have a development background, which we do not have, the RealSense D415 and D435 cameras aren’t much use right now. Currently, there are no applications available that support the D400-series--you must create your own.

Image 1 of 7

Intel’s RealSense depth camera platform isn’t new; the company has several devices on the market already. But the new cameras aren’t compatible with the existing RealSense drivers. Intel released a new RealSense software development kit alongside the new cameras, and the old software isn’t compatible with the new version. We tried a variety of RealSense apps, but not one of them detected our cameras.

We were, however, able to play around with the tools in the software developer kit. The RealSense SKD 2.0 includes the Intel RealSense Viewer application, which allows you to see what the camera sees and adjust the camera’s settings. From here, you can configure the two infrared cameras, the depth camera, and the RGB camera to suit your needs.

The RealSense Viewer application is split into three panes. The left pane is the source window, which allows you to select the camera that you wish to adjust. On the right, you’ll find the Preview pane, which allows you to see one or more camera feeds. The Output pane, found below the Preview pane, give you information about the camera’s behavior.

The RealSense Viewer software can address multiple cameras, and each RealSense D400-series camera has its own individually addressable stereo modules and RGB camera. You must enable the module that you wish to view in the preview pane, and you have to make your resolution and frame rate configuration changes before enabling the camera. The software doesn’t allow for on-the-fly adjustment to those values.

The RGB camera offers a few basic adjustments such as the resolution, frame rate, and color profiles. The camera supports 1080p video, but you can scale the resolution down as low as 320 x 180 to save bandwidth if you don’t need the crisp video. The RGB camera also supports 60, 30, 15, and 6 fps video feeds. You can also toggle between auto exposure or manual control, which allows you to adjust brightness, contrast, exposure, gain, gamma, hue, saturation, sharpness, and white balance independently. Exposure changes can be made while the camera is streaming.

The options for the Stereo Module are somewhat more complex. Like the RGB camera, you can adjust the resolution of the Stereo module. It supports 424 x 240 up to 1280 x 800 in several different aspect ratios. It also supports 90, 60, 30, 25, 15 and 6 fps modes. The Stereo Module options panel handles the configuration for the depth camera and the two infrared cameras. You can choose to enable one, two, or all three modules at once. These options must be selected before enabling the camera.

Image 1 of 12

The Stereo Module allows you to make several adjustments to the video feed while the cameras are enabled. You can change the Visual Preset of the depth camera, which adjusts the accuracy of the depth perception for different use cases. You can select the Short Range option, which ignores the background beyond roughly 2-feet, or the High Accuracy mode if you need a precise depth profile of your surroundings. Intel provides five pre-configured profiles, and you can also create a custom profile if the presets don’t work for you.

You can change the exposure and the gain, and you can raise or lower the laser power. These adjustments can be made with the live preview open, so you can watch how the sliders change the image as you adjust them. The settings don’t allow you to disable the infrared emitter, but you can turn the power down low enough that the camera no longer detects the pattern of infrared dots from the emitter. We found it fascinating to watch how the IR lights affected the accuracy of the depth camera. When the lights are not visible, the depth accuracy drops off considerably. However, we found it interesting that the intensity of the IR pattern has little to no effect on the depth accuracy after a certain point, which tells us that developers can fine tune this setting to maximize accuracy while conserving power.

The Stereo Module configuration pane also offers depth visualization options, including a variety of visual presets. You can choose from Dynamic, Fixed, Near, or Far, which changes how the depth pane highlights the surroundings. You can also change the color scheme for different needs. You can select Jet, which assigns colors to different depth values, like a topographical map, but with the colors of a typical heat map. The Classic preset uses blue to represent objects that are near, red to define distant objects, pink to highlight medium distance objects. The Bio, Cold, and Warm options are a lot like Classic, but with different contrasting colors. The software also offers Black to White and White to Black options; a Quantized view, which is like Black to White that’s brighter; and a Pattern option, which looks like a back and while kaleidoscope. The software also allows you to define the minimum and maximum depth perception.

The Stereo Module also offers an advanced mode with copious options, such as Depth Control, Color Correction, Depth Table options, and nearly a dozen other advanced controls that we’re not familiar with and would be best left to visualization professionals. You can also make changes to the post-processing settings by adjusting the Decimation Filter, Depth Disparity, Spatial Filter, and Temporal filter, which each offer multiple adjustment settings.

When you adjust the camera settings, you’ll see the changes in the preview pane. Here you can see the cameras that you have enabled. If only one is running, you’ll see it in the full screen view. When you have more than one video feed enabled, they all show up at the same time. Two live previews would be side-by-side, and three or more cameras show up in a quadrant view.

The preview pane also offers a 3D view, which allows you to visualize the depth profile in real-time. In the upper right corner of the RealSense Viewer, you’ll find a button that toggles between the 2D preview and the 3D preview. Like a 3D modeling application, you can move the camera around to see the depth of your space. This view works with one camera, but you would need multiple RealSense cameras in opposing corners to create a comprehensive 3D map of the space in real time.

The RealSense SDK 2.0 also includes another application called the Depth Quality Tool. This tool looks just like the RealSense Viewer app, but it’s designed for diagnostics. This application allows you to adjust the resolution and frame rate in real time and it records live metrics which you can dump into a report to examine later. To use the Depth Quality Tool, you must point the RealSense camera towards a flat surface.

Promising Device, Now We Need Software

Intel isn’t marketing the RealSense D400-series to consumers, but with sub-$200 price tags and advanced depth sensing capabilities, we imagine there could be several reasons for people to want one of these cameras at home. Older models of RealSense device enable things like automatic background removal without a green screen, or 3D scanning objects and people for use in modeling software or product previews online. The new cameras should provide better accuracy for both those use cases if the software developers embrace the new devices.

For the most part, it appears that Intel is more focused on making these devices accessible and adaptable for hardware designers. We can see the D400-series cameras becoming a standard addition to every high-school and university robotic lab. Computer vision is an important advancement in the tech sector, and to see this technology available at such an affordable price leads us to believe we’ll see a wide variety of devices with computer vision capabilities sooner than later. It’s up to hardware and software designers to figure out what those devices will be.

The RealSense Depth Camera D415 is available through Intel’s website for $149, and the D435 model is available for $179. Intel began shipping the cameras on January 18, but “due to overwhelming demand” the cameras are on back-order, and the company is unable to provide accurate ETAs for new orders at the moment.