Tackling The High-Res Display Needs Of VR: Kopin's Lightning OLED Microdisplay

Kopin has a few bones to pick with most VR displays. The company believes that LCD displays are inferior to OLED, that the resolutions of VR displays are too low, that the refresh rates are mostly too slow, that the power requirements of displays are too high, and that their physical sizes are too large.

None of the above is really in dispute--by which we mean that VR HMD makers are already working on it or are ahead of the curve. For example, the Rift and Vive both offer AMOLED displays, not LCD; even on upcoming mainstream HMDs, only lower-end models will have LCD displays. And it’s generally agreed that you need at least sustained 90Hz for acceptable VR.

Further, although the Rift and Vive are tethered at this time, there’s a huge industry push towards untethered VR HMDs that either pack in all the computing components on board or rely on a PC with a wireless transmission protocol to pump the data to an untethered headset. In both cases, lower power requirements are a must.

To those ends, Kopin believes its new Lightning microdisplay is the answer. In a press release earlier this year, it stated in part:

Kopin’s Lightning OLED microdisplay addresses the most challenging technical hurdles with VR, including screen door effect due to insufficient display resolution, bulky goggles, and nausea or dizziness due to motion-to-photon latency, as well as heat-build-up caused by high power consumption.

OLED microdisplay on silicon

The Lightning Microdisplay And Accompanying Optics

A Kopin representative laid it out for us at a CES meeting: In VR, hardware is a huge problem. (He suggested that it’s the “biggest,” although one could argue that content is currently a larger issue.) VR HMDs need to get small, and to do that, they need a microdisplay; and that microdisplay needs to handle both VR and AR applications.

The Lightning microdisplay is just one inch square, and it’s directly on silicon, not glass--so, “OLED-on-Silicon.” Kopin told us that the glass other display makers use is powered by “cheap” silicon, which is fine for larger screens but has issues on these tinier ones. Further, the company stated, its solution uses single-crystal polysilicon that has better transistors. A representative told us that its strategy here includes having the headroom to improve the product down the line.

Kopin also asserted that it’s unconcerned with brightness; instead, a representative suggested, it’s more important for the display to have better colors and deeper blacks--which OLED enables.

This first generation of the Lightning microdisplay can offer 2,048 x 2,048 resolution at 120Hz, with “no” latency; the next generation should be able to boast 3,000 x 3,000 resolution, or even 4,000 x 4,000. Within five years, a Kopin representative told me, the company expects to help push HMDs down to the size of eyeglasses. (Even so, there's the key issue of what sort of graphics hardware can push that many pixels.)

Regarding the claim of “no latency,” Kopin said the Lightning microdisplay offers just 10 microseconds of latency--this is not to be confused with milliseconds, which you see abbreviated as “ms.” Ten microseconds is equal to 0.01 milliseconds.  

Aiding and abetting the Lightning microdisplay is Kopin’s optics. Kopin is working on two versions of its proprietary optics: One version uses layers of Fresnel lenses that results in a pattern that looks like the terra cotta tiles you see on some roofs--hence the term “Pantile” that Kopin has applied to this technology. The other type uses “polarization optics,” so although it still uses stacked lenses, the appearance is smooth--according to Kopin, like a pancake shape.

Left: Pancake; Right: Pantile

The Pantile optics are less than 30mm thick, which Kopin said enables those smaller and thinner eyeglasses-type mobile VR designs.

Kopin told us that the optics are still in development.

We were supposed to see a demo of the Lightning microdisplay on a pair of spectacles during our CES meeting, but the device was not working. We did, however, see the display functioning with a colorful test pattern:

"Mobile" VR--And AR And MR

After CES, we had lingering questions about the microdisplay and what it might enable in the XR market, so we reached out to Kopin post-CES for answers. Dr. Hong Choi, CTO of Kopin, responded.

Assuming that all of Kopin’s claims about performance, size, and power are true, we wondered what sort of cost they might add to an HMD. Dr. Choi stated that Kopin has not nailed down pricing for the Lightning microdisplay yet, but he said that “it may be competitive or somewhat higher than the direct view OLED on glass panels used in VR headsets such as Oculus or HTC Vive.” In other words, the cost increase should be nil or negligible, and he believes that the improved user experience would be worth it, regardless.

Dr. Choi also clarified what the company meant when it described the Lightning microdisplay as ideal for "mobile VR." "'Mobile' VR in this case means not tethered to a 'stationary' PC, allowing freedom of movement, and potentially can be connected to a smartphone," he said. "The Lightning OLED display can certainly be used for PC-tethered HMD systems as well, though the power and to some extent smaller form factor are not as critical in this use case."

What he’s saying is that the Lightning microdisplay could work in high-end HMDs like the Rift and Vive, but it’s really aimed at the rapidly developing world of untethered HMDs where weight, power, size, and efficiency are at a premium. This would include devices like Intel’s upcoming Project Alloy and some of the promised mainstream HMDs from PC makers, as well as the experimental wireless HMDs from Intel and Oculus (Project Santa Cruz).

One intriguing tidbit from Dr. Choi's statement above stood out to us: "...potentially can be connected to a smartphone." The smartphone-based HMDs on the market today, such as Google's Daydream and Samsung's Gear VR, use the smartphone and its display, paired with optics in the HMD, to produce the immersive images. Therefore, although Kopin’s Pantile optics could potentially enhance the viewing experience on such products, the Lightning microdisplay has no place. So what gives?

It is possible that Dr. Choi is referring to a different type of device--an HMD that is tethered to a smartphone via a cable, but relies on a display inside of the HMD to produce images. (LG attempted this very thing with its LG 360 VR HMD a year ago at Mobile World Congress, although we found that the execution left much to be desired.) The paradigm, then, would be the same as PC-connected HMDs like the Rift and Vive, which are technically PC peripherals. This new type of device would be essentially a smartphone peripheral. Just as you connect a Rift or Vive to a PC with a cable, you would connect this type of HMD to a smartphone via a cable.

However, because a smartphone is by nature untethered, the HMD would effectively be “untethered.” (Being tethered to an untethered thing is essentially the same as being untethered.) In this case, light and power-efficient microdisplays would be a prerequisite for the HMD to offer any sort of usability.

Just look at the mockups of Kopin optics-enabled glasses: There's essentially no room for computing components on them, but they could be fitted with microdisplays and connected to a computing device--such as a smartphone--with a cable. (Suddenly, the immense capabilities of USB Type-C ports on smartphones look pretty tasty, no?)

Note well that all of the above refers only to occluded VR devices, not AR or MR HMDs. Kopin claims that the Lightning microdisplay could be used for AR or MR, as well. On the surface, this makes no sense, because the microdisplay is on silicon and is therefore opaque. While acknowledging that indeed, the Lightning microdisplay is ideally suited for occluded VR (in particular because of the lower brightness requirements), Dr. Choi elaborated on what Kopin means: "Conceptually, the Lightning OLED display can work for VR, AR or MR if combined with appropriate optics (i.e., see-through optics are possible)," he said. In fact, he noted, opaque displays are fairly standard on AR/MR devices.

"Typically the see-through AR or MR is achieved by see-through optics and non-transparent displays," he explained. "See-through optics [have] a form of a thin waveguide with holographic structures (Hololens using Nokia technology), multiple mirrors imbedded [sic] in a waveguide (Lumus), a partial reflecting/transmitting lens (Seiko Epson), etc. These are all optical AR which is not easy to implement in practice since it requires the alignment of the real image with the computer generated image. On the other hand, most of real world AR is done by video AR with the digital information overlaid on a camera-captured video image. This is much easier to implement with much less computation requirements."

He said that in an AR/MR HMD, you can therefore use an opaque microdisplay that uses "compact, see-around optics." Even so, he allowed that "For AR / MR use, high-brightness transmissive LCoS or reflective LCoS (with appropriate optics) are better suited today [than OLED displays]," but he noted that if OLED displays gain more brightness, they could find a happy home in AR and MR HMDs and glasses.

Theoretically, you could also rely on Lightning microdisplays within an occluded HMD that provides mixed reality via camera passthrough. Intel’s Project Alloy uses the camera passthrough method to an extent, and the Vrvana Totem relies on it heavily.

The validity of Kopin’s claims and its success in the XR market remain to be seen, of course, but it’s important to note that the company is not some scrappy startup. According to the company website, Kopin has been around for almost thirty years, during which time it has provided "
"wearable computing assets" for the military, consumers, and industry in products from companies such as Vuzix, Motorola, and Recon Instruments. "Kopin’s technology portfolio of ultra-small displays, innovative optics, speech enhancement technology, head-gesture command, low-power ASICs, packaging, and system reference designs enable customers to accelerate the development of products for consumers, industry, and the military," reads the site. The company said it has "more than 200 global patents and patents pending in these areas."

Kopin also just formed a joint venture with the Lenovo Capital and Incubator Group (LCIG) called "Lenovo New Vision" that will focus on AR headsets for enterprises. In other words, it has the resources for significant R&D, not to mention a big IP portfolio and longstanding relationships with manufacturers.

  • Valantar
    This looks good in terms of bringing VR hardware forward, but the square form factor is rather baffling. The human field of vision, even per eye, is nowhere close to square. 2:1 would be more accurate. As such, you'd either be limiting horizontal field of view severely (which hurts immersion and realism) or wasting processing power on invisible vertical pixels.

    Also, I'd argue that neither hardware nor content are the biggest hurdles facing VR. Rather, I would say interaction is by far the biggest hurdle - while motion controllers with accurate tracking allow for a certain level of realistic interaction, they're clunky and odd, today require external trackers, and we are still nowhere near actually achieving any kind of passable movement in VR. Without (non-vomit inducing) movement, you'll never have more immersive content than the "experiences" of today, and requiring huge empty spaces or some sort of treadmill for this is simply not realistic in terms of adoption.

    So, what do we need for great VR? Small goggles? Sure. Why not. Good content? Absolutely. But the former won't help unless you have something to do, and the latter won't happen until we can actually meaningfully interact with the content. Inside-out 6dof tracking is a start, but ultimately we need controller-less walking controls without actually walking physically, while avoiding nausea. Otherwise, VR will never live up to its potential.
    Reply
  • Hellcatm
    Valantar I disagree. Shape of the lenses won't matter because your eyes after a while won't notice it.

    Also movement in VR I disagree will be an issue. Look at most of the books and movies that have to do with VR. Ready Player One people are sitting and not walking around in real space. I think the big hurdle is the controls. Using joysticks or pads in your hands will be the issue. We need something tactile like VR gloves that will have some tactile feel when picking up and manipulating things but I think those are many, many years off. You have to trick the brain into thinking your touching everything from rough, to smooth, to jelly like, and many other feelings and sensations.

    AR that's where you want to be able to move around because in AR you're working with the world around you. For instance AR could be for real world, hands on training while VR would be for virtual training. AR you can work on your car with someone helping, VR you can learn to fly, or be in class or work without leaving your home. For gaming AR would be the technology you'd use when running around, and shooting zombies while VR you'd use to fly around or drive. Sure there would be zombie games but you'd set down or you'd be in a contraption to move in place. Since you can't see the word around you in VR I don't think people are going to care about walking or running around while AR you'd want this.
    Reply
  • caustin582
    19310902 said:
    This looks good in terms of bringing VR hardware forward, but the square form factor is rather baffling. The human field of vision, even per eye, is nowhere close to square. 2:1 would be more accurate. As such, you'd either be limiting horizontal field of view severely (which hurts immersion and realism) or wasting processing power on invisible vertical pixels.

    Light enters your eye through the pupil, which is circular. If we can only use a rectangular screen, a square 1:1 ratio for each eye would be the most appropriate shape. And no, your eye lids don't have much of an effect on your vertical field of view unless you're squinting.
    Reply
  • mrmez
    I think every VR company would agree.

    The problem is not making the best screen with, but POWERING them.

    Not everyone has 2x1080's in addition to what will probably be a $1,500 HMD.
    Reply
  • Mildewman
    I returned my psvr because of the low horizontal resolution, so this tech is just what im looking for - full 1080p in each eye.
    Reply
  • bit_user
    19312898 said:
    19310902 said:
    This looks good in terms of bringing VR hardware forward, but the square form factor is rather baffling. The human field of vision, even per eye, is nowhere close to square. 2:1 would be more accurate. As such, you'd either be limiting horizontal field of view severely (which hurts immersion and realism) or wasting processing power on invisible vertical pixels.
    Light enters your eye through the pupil, which is circular. If we can only use a rectangular screen, a square 1:1 ratio for each eye would be the most appropriate shape.
    Unless this thing is in some sort of VR contact lens, then you need to take into account eye movement. I can't quite tell, but it seems like my left/right movement might be greater than top/bottom.

    Anyway, you should be able to use an anamorphic lens, if the aspect ratio of the display panel is a problem. But, they probably knew what they were doing, when they decided to make it square. This should be an improvement over most VR HMDs, which have more vertical resolution than horizontal.

    19314016 said:
    I returned my psvr because of the low horizontal resolution, so this tech is just what im looking for - full 1080p in each eye.
    *sigh* PSVR is 1080p, in each eye. This is 2k x 2k, in each eye.
    Reply
  • redeye
    Quote :
    MILDEWMAN Feb 17, 2017, 10:31 PM
    I returned my psvr because of the low horizontal resolution, so this tech is just what im looking for - full 1080p in each eye.

    how did you return it ?...
    for example, Walmart will not except returns on the playstation VR...
    hygiene concerns, just like some stores won't except headphone returns...

    Though I suppose you could get headlice from it, due the lice surviving 1 to 2 days off of a human head...
    Reply
  • Valantar
    19311063 said:
    Valantar I disagree. Shape of the lenses won't matter because your eyes after a while won't notice it.

    Also movement in VR I disagree will be an issue. Look at most of the books and movies that have to do with VR. Ready Player One people are sitting and not walking around in real space. I think the big hurdle is the controls. Using joysticks or pads in your hands will be the issue. We need something tactile like VR gloves that will have some tactile feel when picking up and manipulating things but I think those are many, many years off. You have to trick the brain into thinking your touching everything from rough, to smooth, to jelly like, and many other feelings and sensations.

    AR that's where you want to be able to move around because in AR you're working with the world around you. For instance AR could be for real world, hands on training while VR would be for virtual training. AR you can work on your car with someone helping, VR you can learn to fly, or be in class or work without leaving your home. For gaming AR would be the technology you'd use when running around, and shooting zombies while VR you'd use to fly around or drive. Sure there would be zombie games but you'd set down or you'd be in a contraption to move in place. Since you can't see the word around you in VR I don't think people are going to care about walking or running around while AR you'd want this.

    Either you didn't read my post, or you didn't understand it. Your first paragraph is exactly what I'm talking about - making "sit-down" VR feasible without puking (due to the impracticality of actual physical movement). Reread my post, then we can talk.

    19312898 said:
    19310902 said:
    This looks good in terms of bringing VR hardware forward, but the square form factor is rather baffling. The human field of vision, even per eye, is nowhere close to square. 2:1 would be more accurate. As such, you'd either be limiting horizontal field of view severely (which hurts immersion and realism) or wasting processing power on invisible vertical pixels.

    Light enters your eye through the pupil, which is circular. If we can only use a rectangular screen, a square 1:1 ratio for each eye would be the most appropriate shape. And no, your eye lids don't have much of an effect on your vertical field of view unless you're squinting.

    Sure, the pupil is round. Your eye, however, isn't. It's oblong, and can move around 2x as much horizontally than vertically, not to mention that vertical visibility is blocked by your brow and cheekbones. Humans have a horizontal field of view of 180-200 degrees (including eye movement). Each eye sees roughly 160 degrees vertically, with the majority of the fields overlapping. Our vertical field of view is far less than that, especially since it doesn't have any overlap or the advantage of eyes tilted to each side.. In VR goggles, you need portions of the displayed images to overlap (aka binocular overlap) to avoid issues with depth perception and the like. Using two square displays means the non-overlapping part (i.e. peripheral vision) has to be prioritized away (again, to avoid people puking or other unforeseen issues), which again limits our field of view, and thus limits immersion.


    19314393 said:
    19312898 said:
    19310902 said:
    This looks good in terms of bringing VR hardware forward, but the square form factor is rather baffling. The human field of vision, even per eye, is nowhere close to square. 2:1 would be more accurate. As such, you'd either be limiting horizontal field of view severely (which hurts immersion and realism) or wasting processing power on invisible vertical pixels.
    Light enters your eye through the pupil, which is circular. If we can only use a rectangular screen, a square 1:1 ratio for each eye would be the most appropriate shape.
    Unless this thing is in some sort of VR contact lens, then you need to take into account eye movement. I can't quite tell, but it seems like my left/right movement might be greater than top/bottom.

    Anyway, you should be able to use an anamorphic lens, if the aspect ratio of the display panel is a problem. But, they probably knew what they were doing, when they decided to make it square. This should be an improvement over most VR HMDs, which have more vertical resolution than horizontal.

    So you're saying they shound compensate for using a dumb aspect ratio by stretching it, effectively lowering resolution (and introducing all kinds of colour shift and artifacts)? Yeah, that sounds brilliant.
    Reply
  • bit_user
    19315973 said:
    19314393 said:
    19312898 said:
    19310902 said:
    This looks good in terms of bringing VR hardware forward, but the square form factor is rather baffling. The human field of vision, even per eye, is nowhere close to square. 2:1 would be more accurate. As such, you'd either be limiting horizontal field of view severely (which hurts immersion and realism) or wasting processing power on invisible vertical pixels.
    Light enters your eye through the pupil, which is circular. If we can only use a rectangular screen, a square 1:1 ratio for each eye would be the most appropriate shape.
    Unless this thing is in some sort of VR contact lens, then you need to take into account eye movement. I can't quite tell, but it seems like my left/right movement might be greater than top/bottom.

    Anyway, you should be able to use an anamorphic lens, if the aspect ratio of the display panel is a problem. But, they probably knew what they were doing, when they decided to make it square. This should be an improvement over most VR HMDs, which have more vertical resolution than horizontal.
    So you're saying they shound compensate for using a dumb aspect ratio by stretching it, effectively lowering resolution (and introducing all kinds of colour shift and artifacts)? Yeah, that sounds brilliant.
    I'm agreeing with you more than anyone else, so far. I don't know exactly why they went with square, but you ignored my point that all the other big VR HMDs on the market are rectangular in the opposite direction as you're saying.

    I wouldn't get too exercised about this point. It sounds like promising tech, and seems like a big step in the right direction.

    I don't see why you consider yourself more expert on the subject than the people building the displays. As the article says, they've been in this industry for a long time, and they're surely talking to all their potential customers. None of whom are exactly stupid. And when you invest the millions it takes to make such a product, you do your homework and try to make sure it's right for the market.

    So, when you see something that doesn't make sense to you, why is your default assumption that the industry is filled with bumbling idiots who don't know what they're doing? Why not try to get Seth to reach out and ask why they went with that particular aspect ratio? You might learn something.

    BTW, anamorphic lenses have a long history, in the film industry. And Prismatic separation can be digitally compensated, and can happen with normal lenses. If it's an issue, then perhaps existing HMDs are already doing that? I know digital cameras have been doing it for ages.
    Reply