Facebook Is Using Computer Vision To Bring AR To The Masses, Dev Platform Available Now

By Kevin Carbotte
published

Facebook showed the world it was serious about virtual reality technology when it dropped more than $2 billion to acquire Oculus VR. The company isn’t done with VR by a long shot, but it’s now setting its sights on augmented reality, as well. During the F8 Developer Conference opening keynote, Mark Zuckerberg announced a development platform that brings AR technology to the palm of your hand through the technology in your smartphone.

Facebook’s AR Studio development platform takes advantage of the cameras that you’ll find on every smartphone and leverages cutting-edge computer vision and machine learning advancements to transform your everyday world into augmented experiences.

Facebook sees AR Studio as the beginning of a new platform that will pave the way for mainstream adoption of augmented reality technology. Zuckerberg said that he originally thought that it would take 5-10 years for AR to catch on, but he thought you would need AR glasses for the technology to work. But recent advancements in computer vision and smartphone technology have proven otherwise. Augmented reality technology is accessible to almost anyone now.

Facebook’s AR Studio platform provides tools for 3D artists and developers to create unique augmented experiences that blend in with the real world. The platform allows developers to create 3D objects and animations that interact with live video feeds. Facebook uses the power of computer vision and Simultaneous Localization And Mapping (SLAM) technology to create realistic interactions between virtual and real-world objects. For example, if you place a virtual object on a table, it will appear as though it’s sitting on the table from any angle at which you look at it, including realistic occlusion from objects around it.

The AR Studio platform supports face masks, styles transfers, and realistic 3D objects, but Facebook sees more than just silly image effects in AR Studios’ future. Facebook expects that developers will create games and other apps with AR Studio, and it’s already experimented with creating an urban art project for augmented reality.

Zuckerberg said that it would take time for the AR Studio platform to take shape. Over time, the company will release more features that allow developers to create more ambitious applications and experiences, but Zuckerberg believes that developers will be slow to adopt the platform at first.

Facebook is currently running a closed beta test of the AR Studio platform. If you have an APP idea, you can apply for access from the Facebook Camera Effects website.   

Kevin Carbotte

 Kevin Carbotte is a contributing writer for Tom's Hardware who primarily covers VR and AR hardware. He has been writing for us for more than four years. 

5 Comments Comment from the forums
  • bit_user
    I wish them luck. I'm skeptical that it's going to work well enough on the range of real-world devices, in real-world conditions. One thing is certain: it'll definitely chew through your battery's charge, making me wonder how willing people will even be to enable the feature.

    I do hope this creates a strong market for purpose-built AR devices, like Lenovo's Phab 2 Pro and Asus' Zenfone AR. Not to mention Hololens and Magic Leap (if they don't fall short ;-)
    Reply
  • dikbozo
    This looks to me like Facebook does face plant. Or face palm, as in what were we thinking? This is somewhat like Philip K. Dick saw for the future of advertising back in the 60's with tailored ads streamed into the eyes of people passing through public spaces. The ad platform is ultimately Zuckerberg's raison d'etre for this 'technology'.
    Reply
  • bit_user
    19589938 said:
    This is somewhat like Philip K. Dick saw for the future of advertising back in the 60's with tailored ads streamed into the eyes of people passing through public spaces. The ad platform is ultimately Zuckerberg's raison d'etre for this 'technology'.
    I'll play devil's advocate.

    I think his main objective is to make Facebook more immersive, pervasive, and engaging. What if he doesn't like the way that FB currently isolates people and confines them to their homes or wherever they can sit down and get sucked into their phone screen? What if he genuinely wants people to get out more?

    Better & more ways to deliver ads are merely a side-benefit. And that's not the only one. AR also provides new ways to gather information about people, and new ways to market FB's services to local businesses, tourism, etc.

    In any case, we should keep in perspective that he spent over $2B of investors' money on buying Oculus. Eventually, he's got to try to make this stuff pay off.
    Reply
  • dikbozo
    19592019 said:
    19589938 said:
    This is somewhat like Philip K. Dick saw for the future of advertising back in the 60's with tailored ads streamed into the eyes of people passing through public spaces. The ad platform is ultimately Zuckerberg's raison d'etre for this 'technology'.
    I'll play devil's advocate.

    I think his main objective is to make Facebook more immersive, pervasive, and engaging. What if he doesn't like the way that FB currently isolates people and confines them to their homes or wherever they can sit down and get sucked into their phone screen? What if he genuinely wants people to get out more?

    Better & more ways to deliver ads are merely a side-benefit. And that's not the only one. AR also provides new ways to gather information about people, and new ways to market FB's services to local businesses, tourism, etc.

    In any case, we should keep in perspective that he spent over $2B of investors' money on buying Oculus. Eventually, he's got to try to make this stuff pay off.

    As tech history is littered with the corpses of numerous and many now forgotten 'whiz kid' ideas, Oculus is not a good example of wisely spent money. FB has a somewhat checkered history in money making terms, and here I do NOT include stock price, and is a questionable staying power such as Google has shown is a moveable feast. TSo many sinkholes for money exist that sooner or later one of these will be one too many for the shareholders and they will vote with their wallets.

    As to Zuck being altruistic, well the jury is very much out on that. He seems to me to be a somewhat ruthless capitalist which kinda precludes that. Just remember his 'free internet' thing, IIRC internet.org? Turned out to be a form of walled garden pushing eyeballs back to FB.

    From a strictly personal view point, more invasive advertising such as AR does not strike me as any sort of good thing. The real benefits of this accrue to FB. I like to explore where I live and discover the new on my own not as the result of some advertising gimmick.
    Reply
  • bit_user
    19593163 said:
    As to Zuck being altruistic, well the jury is very much out on that.
    If you read my post carefully, I never said he was being altruistic. I suggested that maybe he wants to give FB addicts a way to get out more, without having to forego using FB. And create new opportunities for others to use FB in more ways and places.

    19593163 said:
    From a strictly personal view point, more invasive advertising such as AR does not strike me as any sort of good thing.
    And I agree. I also don't like the idea of FB watching more of my activities and integrating with more of my daily and economic life.

    I think we can all agree that Oculus was overvalued. But FB has to make some investments of this sort. The surest way to be left in the dust is not to invest in the future. AR is eventually going to happen, and FB, Google, and MS all want to be there when it does.

    I hope it's clear that I'm much more excited about what this means for AR than for FB. My hope is that FB's announcement leads to a flurry of new apps, which will run better on true AR devices, thus creating a more stable market for these to be introduced and improved.

    Anyway, getting back to FB, I'd just point out that the most negative aspects we've been discussing could be addressed through better legal protections of privacy and personal information.
    Reply