Do we ever talk get tired of hearing of Magic Leap?
A 15-second clip of a floating baby elephant has made the Internet lose its mind.
A mysterious Florida-based company called Magic Leap behind the dazzling display, a technology they’re calling cinematic reality. It seems to be a mixture of augmented and virtual reality on steroids. Whatever it is, it’s freaking cool. And apparently worth a lot of money.
The biggest name in virtual reality right now is Oculus VR, which Facebook recently bought for $2 billion.
Now, Magic Leap says their technology could potentially blow Oculus out of the water.
Operating in stealth mode, Magic Leap just had one of the most successful second rounds in history, raising $542 million. The lead investor is none other than Google.
But everyone’s asking: what is it?
Here’s what we do know: Magic Leap uses “digitized light fields” to overlay 3D images onto the real world. It’s not virtual reality, which totally submerses the viewer in a totally artificial environment. Instead, it’s a mixture of real and artificial.
It sounds similar to augmented reality, which is something that already exists on your smartphone. But early users claim it’s way better. One venture capitalists backing the company put it this way: “it’s so badass you can’t believe it.”
Lots of questions remain about Magic Leap – the biggest being: will this ‘stealth’ company actually deliver in reality?
The AR world has been buzzing about a new Florida based startup called Magic Leap that has already attracted heavy investments from the likes of Google and Qualcomm. There technology which they term cinematic reality promises to revolutionize the way we see the world. What is behind this mysterious blackbox technology from Magic Leap?
According to patent and trademark filings unearthed by MIT Technology Review, the company is developing a sophisticated display technology that can project accurate images onto users’ eyes and produce virtual 3D objects that seem real. That stands in contrast to other virtual reality displays that trick users’ brains into perceiving virtual objects as real by showing different 2D images to each eye.
The Magic Leap filings describe displays that can create the same kind of 3D patterns of light rays, known as “light fields,” that human eyes use to observe real objects. The end result is that the technology lets a user’s eyes focus on the depth of an artificial 3D object just as they would on an object in the real world.
The report said that a trademark filing from July describes Magic Leap’s technology as “wearable computer hardware, namely, an optical display system incorporating a dynamic light-field display.”
Further, the report added that one of Magic Leap’s patents describes how such a device, called a WRAP, for “waveguide reflector array projector,” would operate. According to the patent, the display would be composed of an array of many small curved mirrors, light would be delivered via optical fiber, and each mirror would reflect some of that light to create the light field for the point that a person was viewing. The array would also let the user see the real world at the same time.
A Florida startup called Magic Leap announced Tuesday that it had received $542 million in funding from major Silicon Valley investors led by Google to develop hardware for a new kind of augmented reality hardware. The secretive startup has yet to publicly describe or demonstrate its technology, and declined an interview request. But patent and trademark filings reveal the kind of technology that Magic Leap plans to use to create what the company’s CEO and founder Rony Abovitz has called “the most natural and human-friendly wearable computing interface in the world.”
The filings describe sophisticated display technology that can trick the human visual system better than existing virtual reality displays (such as the Oculus Rift) into perceiving virtual objects as real. The display technology used in most devices can show only flat, 2-D images. Headsets like the Oculus Rift trick your brain into perceiving depth by showing different images to each eye, but your eyes are always focused on the flat screen right in front of them.
When you look at a real 3-D scene, the depth at which your eyes are focused changes as you look at objects at different distances away. “If we leave out those focus cues we get an experience that’s not quite realistic,” says Gordon Wetzstein, who leads the Computational Imaging Research Group at Stanford University.
Magic Leap’s patents suggest an alternative approach. They describe displays that can create the same kind of 3-D patterns of light rays, known as “light fields,” that our eyes take in from the real objects around us. Wetzstein and other researchers have shown that this allows your eyes to focus on the depths of an artificial 3-D scene just as they would in the real world—providing a far more realistic illusion of virtual objects merged with the real world.
Earlier this year, Wetzstein and colleagues used that technique to create a display that allows text to be read clearly by people not wearing their usual corrective lenses (see “Prototype Display Lets You Say Goodbye to Reading Glasses”). He previously worked on glasses-free 3-D displays based on similar methods. And last year, researchers at chip company Nvidia demonstrated abasic wearable display based on light fields.
A trademark filing from July describes Magic Leap’s technology as “Wearable computer hardware, namely, an optical display system incorporating a dynamic light-field display.”
One of Magic Leap’s patents describes how such a device, dubbed a WRAP, for “waveguide reflector array projector,” would operate. The display would be made up of an array of many small curved mirrors; light would be delivered to that array via optical fiber, and each of the tiny elements would reflect some of that light to create the light field for a particular point in 3-D space. The array could be semi-transparent to allow a person to see the real world at the same time.
Multiple layers of such tiny mirrors would allow the display to produce the illusion of virtual objects at different distances. However, Magic Leap’s patent also claims that a single layer of the mirrors would be enough if they were formed from “magnetic liquid.” That would allow the mirrors to be reprogrammed using a magnetic field to rapidly display points at different depths fast enough to fool the eye, like the frames of an animation.
Magic Leap’s greatest challenge may be to find a way to seamlessly integrate virtual 3-D objects created by that display with what a person sees in the real world. Doing so would require the system to sense the world in 3-D and understand exactly what a person is looking at and its exact position, says Wetzstein.
One of Magic Leap’s patents covers the use of motion sensors and eye-tracking cameras on a wearable display to figure out at what depth a person’s eyes are focused. But Wetzstein says he isn’t aware of anyone yet demonstrating a wearable system that can track the distance a person is focusing on.
Another of Magic Leap’s patent filings says that cameras, infrared sensors, and ultrasonic sensors could be used to sense the environment around a person, and to recognize gestures. Depth-sensing cameras are now relatively cheap and compact (see “Intel Says Tablets and Laptops with 3-D Vision Are Coming Soon”). But Wetzstein says Magic Leap will likely need to make major breakthroughs in computer vision software for a wearable device to make sense of the world enough for very rich augmented reality. “They will require very powerful 3-D image recognition, running on your head-mounted display,” he says.
The company is recruiting experts in chip design and fabrication, apparently with a view to creating custom chips to process image data. Dedicated chips could make that work more energy-efficient, something important for a wearable device. Magic Leap already employs Gary Bradski, a pioneer of computer vision research and software, notes Wetzstein. Magic Leap is also trying to recruit people skilled in lasers, mobile and wireless electronics, cameras, manufacturing supply-chain management, 3-D sensing, artificial intelligence, and video game development.
Altogether, many of the underlying techniques Magic Leap needs to realize highly realistic augmented reality have been demonstrated, says Wetzstein. But the company will have to refine and combine them in ways no one has yet managed to do. “I think people are starting to realize this is the future of building consumer devices,” he says. “But it involves big challenges at the intersection of optics, electronics, algorithms, and understanding the human visual system.”
Google is leading a huge $542 million round of funding for the secretive startup Magic Leap, which is said to be working on augmented reality glasses that can create digital objects that appear to exist in the world around you. Though little is known about what Magic Leap is working on, Google is placing a big bet on it: in addition to the funding, Android and Chrome leader Sundar Pichai will join Magic Leap’s board, as will Google’s corporate development vice-president Don Harrison. The funding is also coming directly from Google itself — not from an investment arm like Google Ventures — all suggesting this is a strategic move to align the two companies and eventually partner when the tech is more mature down the road.
“YOU’RE IN THE ROOM, AND THERE’S A DRAGON FLYING AROUND, IT’S JAW-DROPPING.”
Magic Leap’s technology currently takes the shape of something like a pair of glasses, according to The Wall Street Journal. Rather than displaying images on the glasses or projecting them out into the world, Magic Leap’s glasses reportedly project their image right onto their wearer’s eyes — and apparently to some stunning effects.
“It was incredibly natural and almost jarring — you’re in the room, and there’s a dragon flying around, it’s jaw-dropping and I couldn’t get the smile off of my face,” Thomas Tull, CEO of Legendary Pictures, tells the Journal. Legendary also took part in this round of investment, alongside Qualcomm, Kleiner Perkins, Andreessen Horowitz, and Obvious Ventures, among others. Qualcomm’s executive chairman, Paul Jacobs, is also joining Magic Leap’s board.
The eclectic mix of companies participating in this investment round speak to how broadly Magic Leap sees its potential. Its founder says that he wants the company to become “a creative hub for gamers, game designers, writers, coders, musicians, filmmakers, and artists.” Legendary, which makes films including Godzilla and The Dark Knight, is interested in its potential for movies. Google likely sees far more ways to put it to use.
The technology sounds like it could be an obvious companion to Google Glass, but for now the Journal reports that they’re not being integrated. Magic Leap declined to commented on what might happen down the road. Nonetheless, the investment in Magic Leap appears to be Google betting on augmented reality as the future of computing, pitting it in a fight against virtual reality competitors. Eventually, it’ll likely be facing off against Facebook’s Oculus Rift — the biggest name in VR right now, and one that Facebook was willing to pay $2 billion for.
Magic Leap Cinematic Realilty
Magic Leap is run and was founded by Rony Abovitz, who previously founded the medical robotics company Mako Surgical, which was sold for $1.65 billion last year. TheJournal reports that Abovitz has a biomedical engineering degree from the University of Miami. He previously made a bizarre, psychedelic TEDx talk involving 2001, green and purple apes, and a punk band. His new company, which has been around since 2011, is headquartered in Florida, so it isn’t exactly the typical tech startup out out of Silicon Valley. Aboitz says the location allows Magic Leap to recruit globally. It currently has over 100 employees.
“WHEN YOU SEE THIS, YOU WILL SEE THAT THIS IS COMPUTING FOR THE NEXT 30 OR 40 YEARS.”
Though Magic Leap’s product sounds like a pair of augmented reality glasses, Abovitz and his company dislike the term. Magic Leap brands its effect as “Cinematic Reality,” which sounds a bit cooler but doesn’t really mean anything just yet. “Those are old terms – virtual reality, augmented reality. They have legacy behind them,” Abovitz told the South Florida Business Journal back in February, after closing an initial round of funding. “They are associated with things that didn’t necessarily deliver on a promise or live up to expectations. We have the term cinematic reality because we are disassociated with those things. … When you see this, you will see that this is computing for the next 30 or 40 years. To go farther and deeper than we’re going, you would be changing what it means to be human.”
This is all something that Google is eager to view the results of. “We are looking forward to Magic Leap’s next stage of growth, and to seeing how it will shape the future of visual computing,” Pichai says in a statement. What exactly Google will do with augmented reality is still unknown, but, much like how Google has managed to control a great deal of mobile computing through Android, it’s been looking ahead to ensure that it doesn’t miss out on the next leap either. It declined to provide further comment on the investment.
Talking to TechCrunch, Abovitz says that Magic Leap should be launching a product for consumers “relatively soon.” There’s no stated target date for now, though, and it sounds like it still has some development to do.
PhillyAR 