RealSense D400-Series Depth Cameras Are For Devs, Not Consumers (Hands On)

Intel launched a new generation of RealSense depth cameras this week, and they feature stereo cameras, updated depth modules, and Intel RealSense D4 vision processors. Intel’s RealSense D400 Series Depth Cameras offer advanced computer vision capabilities in a convenient form factor with USB connectivity. But these cameras aren’t built for consumers; you need to be a developer to get any meaningful use out of them.

Intel said it designed the new RealSense D400-series cameras for makers, educators, and developers. The D400-series cameras come in a compact USB-powered form factor for easy deployment in hardware prototyping projects. The USB connectivity also makes the new RealSense D400 cameras convenient options for prototyping software that takes advantage of the new depth modules and the integrated vision processor.

“Many of today’s machines and devices use 2D image recognition-based computer vision, but with Intel RealSense’s best-in-class depth technology, we are redefining future technologies to ‘see’ like a human, so devices and machines can truly enrich people’s lives,” said Sagi Ben Moshe, vice president and general manager of the RealSense Group at Intel. “With its compact, ready-to-use form, the Intel RealSense D400 Depth Camera series not only makes it easy for developers to build 3D depth sensing into any design, but they are also ready to be embedded into high-volume products.”

Intel’s RealSense D400 series includes two cameras that serve somewhat different purposes. The RealSense D415 camera features rolling shutter sensors with small lenses that have a narrow field of view. It also includes an Intel RealSense module D410 depth sensor, which processes depth information from the stereo cameras in real time. Intel said the narrow field of view of the D415 provides high depth resolution, which is “ideal for precise measurement.” The combination of smaller lenses, rolling shutter sensors, and the D410 module allow Intel to keep the price of the RealSense D415 under $150.

The RealSense D400-series also includes the RealSense D435 depth camera, which offers global shutter sensors and larger lenses for better low-light performance than the cheaper D415 camera. The D435 also features a more powerful RealSense module D430. Intel said that the D435 camera is the “preferred solution for applications such as robotic navigation and object recognition.” The RealSense D435 with its larger lenses and global shutter sensor is somewhat more expensive than the RealSense D415, but it's still under $200.

Both RealSense D400-series cameras support depth output 1280 x 720 active stereo depth resolution at up to 90 fps. The D400-series cameras can also capture RGB video at up to 1080p, but we experienced significant frame drops at that resolution. They can capture 720p video at a smooth 30 fps.

The Intel RealSense D400-series cameras feature a strong aluminum outer shell, which should protect the components from the elements and from abuse. The aluminum body of both cameras include slots on the top and bottom to allow air to pass through and keep the components cool. The bottom of each camera includes a standard ¼” thread for a tripod mount. Intel includes a miniature tripod in the box, but we’d suggest getting something a little sturdier for long-term use. The back of both cameras also includes two smaller thread points that you can use to mount the camera to the device you’re building.

Intel RealSense Depth Camera D415Intel RealSense Depth Camera D435
Use Environment: Indoor / OutdoorIndoor / Outdoor
Depth Technology:Active infrared (IR) stereoActive infrared (IR) stereo
Image Sensor Technology:Rolling Shutter; 1.4um x 1.4um pixel sizeGlobal Shutter; 3um x 3um pixel size
Main Intel® RealSense™ Products:Intel RealSense Vision Processor D4
Intel RealSense module D430
Intel RealSense Vision Processor D4
Intel RealSense module D430
Depth Field of View (FOV)—(Horizontal × Vertical) for HD 16:9:
63.4° x 40.4° (+/- 3°)85.2° x 58° (+/- 3°)
Depth Stream Output Resolution:
Up to 1280 x 720Up to 1280 x 720
Depth Stream Output Frame Rate:Up to 90 fpsUp to 90 fps
Minimum Depth Distance (Min-Z):0.16m0.11m
Maximum Range:Approx. 10mApprox. 10m
RGB Sensor Resolution & Frame Rate:1920 x 1080 at 30 fps1920 x 1080 at 30 fps
RGB Sensor FOV (Horizontal x Vertical):69.4° x 42.5° (+/- 3°)69.4° x 42.5° (+/- 3°)
Camera Dimension (L x D x H):99 x 20 x 23mm90 x 25 x 25mm
Connectors:USB 3.0 Type-CUSB 3.0 Type-C
Mounting Mechanism:One 1/4-20 UNC thread mounting point
Two M3 thread mounting points
One 1/4-20 UNC thread mounting point
Two M3 thread mounting points

So, What Can You Do With These Things?

Intel sent us one of each RealSense D400 cameras ahead of the launch so that we could give them a test run. Unfortunately, we haven’t been able to do a whole lot with them. Unless you have a development background, which we do not have, the RealSense D415 and D435 cameras aren’t much use right now. Currently, there are no applications available that support the D400-series--you must create your own.

Intel’s RealSense depth camera platform isn’t new; the company has several devices on the market already. But the new cameras aren’t compatible with the existing RealSense drivers. Intel released a new RealSense software development kit alongside the new cameras, and the old software isn’t compatible with the new version. We tried a variety of RealSense apps, but not one of them detected our cameras.

We were, however, able to play around with the tools in the software developer kit. The RealSense SKD 2.0 includes the Intel RealSense Viewer application, which allows you to see what the camera sees and adjust the camera’s settings. From here, you can configure the two infrared cameras, the depth camera, and the RGB camera to suit your needs.

The RealSense Viewer application is split into three panes. The left pane is the source window, which allows you to select the camera that you wish to adjust. On the right, you’ll find the Preview pane, which allows you to see one or more camera feeds. The Output pane, found below the Preview pane, give you information about the camera’s behavior.

The RealSense Viewer software can address multiple cameras, and each RealSense D400-series camera has its own individually addressable stereo modules and RGB camera. You must enable the module that you wish to view in the preview pane, and you have to make your resolution and frame rate configuration changes before enabling the camera. The software doesn’t allow for on-the-fly adjustment to those values.

The RGB camera offers a few basic adjustments such as the resolution, frame rate, and color profiles. The camera supports 1080p video, but you can scale the resolution down as low as 320 x 180 to save bandwidth if you don’t need the crisp video. The RGB camera also supports 60, 30, 15, and 6 fps video feeds. You can also toggle between auto exposure or manual control, which allows you to adjust brightness, contrast, exposure, gain, gamma, hue, saturation, sharpness, and white balance independently. Exposure changes can be made while the camera is streaming.

The options for the Stereo Module are somewhat more complex. Like the RGB camera, you can adjust the resolution of the Stereo module. It supports 424 x 240 up to 1280 x 800 in several different aspect ratios. It also supports 90, 60, 30, 25, 15 and 6 fps modes. The Stereo Module options panel handles the configuration for the depth camera and the two infrared cameras. You can choose to enable one, two, or all three modules at once. These options must be selected before enabling the camera.

The Stereo Module allows you to make several adjustments to the video feed while the cameras are enabled. You can change the Visual Preset of the depth camera, which adjusts the accuracy of the depth perception for different use cases. You can select the Short Range option, which ignores the background beyond roughly 2-feet, or the High Accuracy mode if you need a precise depth profile of your surroundings. Intel provides five pre-configured profiles, and you can also create a custom profile if the presets don’t work for you.

You can change the exposure and the gain, and you can raise or lower the laser power. These adjustments can be made with the live preview open, so you can watch how the sliders change the image as you adjust them. The settings don’t allow you to disable the infrared emitter, but you can turn the power down low enough that the camera no longer detects the pattern of infrared dots from the emitter. We found it fascinating to watch how the IR lights affected the accuracy of the depth camera. When the lights are not visible, the depth accuracy drops off considerably. However, we found it interesting that the intensity of the IR pattern has little to no effect on the depth accuracy after a certain point, which tells us that developers can fine tune this setting to maximize accuracy while conserving power.

The Stereo Module configuration pane also offers depth visualization options, including a variety of visual presets. You can choose from Dynamic, Fixed, Near, or Far, which changes how the depth pane highlights the surroundings. You can also change the color scheme for different needs. You can select Jet, which assigns colors to different depth values, like a topographical map, but with the colors of a typical heat map. The Classic preset uses blue to represent objects that are near, red to define distant objects, pink to highlight medium distance objects. The Bio, Cold, and Warm options are a lot like Classic, but with different contrasting colors. The software also offers Black to White and White to Black options; a Quantized view, which is like Black to White that’s brighter; and a Pattern option, which looks like a back and while kaleidoscope. The software also allows you to define the minimum and maximum depth perception.

The Stereo Module also offers an advanced mode with copious options, such as Depth Control, Color Correction, Depth Table options, and nearly a dozen other advanced controls that we’re not familiar with and would be best left to visualization professionals. You can also make changes to the post-processing settings by adjusting the Decimation Filter, Depth Disparity, Spatial Filter, and Temporal filter, which each offer multiple adjustment settings.

When you adjust the camera settings, you’ll see the changes in the preview pane. Here you can see the cameras that you have enabled. If only one is running, you’ll see it in the full screen view. When you have more than one video feed enabled, they all show up at the same time. Two live previews would be side-by-side, and three or more cameras show up in a quadrant view.

The preview pane also offers a 3D view, which allows you to visualize the depth profile in real-time. In the upper right corner of the RealSense Viewer, you’ll find a button that toggles between the 2D preview and the 3D preview. Like a 3D modeling application, you can move the camera around to see the depth of your space. This view works with one camera, but you would need multiple RealSense cameras in opposing corners to create a comprehensive 3D map of the space in real time.

The RealSense SDK 2.0 also includes another application called the Depth Quality Tool. This tool looks just like the RealSense Viewer app, but it’s designed for diagnostics. This application allows you to adjust the resolution and frame rate in real time and it records live metrics which you can dump into a report to examine later. To use the Depth Quality Tool, you must point the RealSense camera towards a flat surface.

Promising Device, Now We Need Software

Intel isn’t marketing the RealSense D400-series to consumers, but with sub-$200 price tags and advanced depth sensing capabilities, we imagine there could be several reasons for people to want one of these cameras at home. Older models of RealSense device enable things like automatic background removal without a green screen, or 3D scanning objects and people for use in modeling software or product previews online. The new cameras should provide better accuracy for both those use cases if the software developers embrace the new devices.

For the most part, it appears that Intel is more focused on making these devices accessible and adaptable for hardware designers. We can see the D400-series cameras becoming a standard addition to every high-school and university robotic lab. Computer vision is an important advancement in the tech sector, and to see this technology available at such an affordable price leads us to believe we’ll see a wide variety of devices with computer vision capabilities sooner than later. It’s up to hardware and software designers to figure out what those devices will be.

The RealSense Depth Camera D415 is available through Intel’s website for $149, and the D435 model is available for $179. Intel began shipping the cameras on January 18, but “due to overwhelming demand” the cameras are on back-order, and the company is unable to provide accurate ETAs for new orders at the moment.

Create a new thread in the News comments forum about this subject
This thread is closed for comments
Comment from the forums
    Your comment
  • bit_user
    Thanks for the review. My only disappointment is that range is limited to 10 M. It's good enough for most purposes, but I wouldn't mind even coarse depth information (via stereo) at greater distances.
  • d0x360
    Funny how far ahead Microsoft was with the Kinect, especially version 2 on the Xbox one. It had everything this does plus a microphone system that could locate someone by their voice and filter out tons of loud noise to hear them and accept commands from them. It was also quite nice for Skype calls because it would follow you around the room no matter where you went and recorded you in 1080p.

    Sure high end devices can do what the Kinect did better because they aren't under cost constraints but when you consider how powerful the $100 Kinect v2 was it's impressive. Its sensors we're high enough resolution that it could do skeletal tracking on your fingers.

    I only bring it up because really we are just starting to see devices from other companies hit the same level of quality. The only thing that detracted from the Kinect was the fact that so little processing resources could be devoted to it's use. If they had included a built in ARM CPU it would have been quite the device...still a horrible idea for games unless of course it was being used for VR tracking. It would have been fantastic for that especially if they had a VR edition that did have an arm cpu.

    I wonder how many patents Intel has to license from Microsoft to build these cameras
  • bit_user
    Anonymous said:
    If they had included a built in ARM CPU it would have been quite the device...

    It's far more efficient to use a portion of the console's GPU resources for this. Those sorts of algorithms tend to be very GPU-friendly. ARM CPUs have compute power in the single to low double-digits of GFLOPS - the XBox One's GPU is 2-3 orders of magnitude faster, depending which we're talking about. Plus, if you want a respectable amount of processing power, you need reasonably fast memory - all of which adds yet more cost to the device.

    MS approach to their HMD design follows the same philosophy as Kinect - make the accessory cheap (i.e. no external cameras) and make up for it with more processing in the GPU. Even so, their GPU requirements for VR are the most modest of any PC VR solution.

    Anonymous said:
    unless of course it was being used for VR tracking. It would have been fantastic for that

    According to the father of Hololens, the Kinect came from the same overall roadmap that featured Hololens. That means they're very much considered in the same universe. MS could do a lot with it, but it largely depends on the world embracing their vision of mixed reality computing.

    Anonymous said:
    I wonder how many patents Intel has to license from Microsoft to build these cameras

    FWIW, MS didn't build the original kinect from the ground-up. They integrated some 3rd-party sensors already on the market.

    Anyway, I suspect Intel's camera is mostly aimed at robotics applications. People have used Kinect for that, as well.