Another piece of Magic Leap's mysterious story has been uncovered thanks to a new patent application revealed on Thursday, March 15.
The new patent application descriptions and drawings outline a method of using a head-mounted device to detect and translate sign language. Yet another part of the new patent information details the ability to use the head-mounted device to recognize text on signs and storefronts and have that text translated into information on the headset. Magic Leap is calling this approach to sign language and text translation "sensory eyewear."
Specifically, the patent application describes the "a mixed reality system for recognizing sign language and text in an environment … these embodiments advantageously may permit greater interaction among differently-abled persons … the sensory eyewear system can recognize and interpret a sign language, and present the translated information to a user of the mixed reality device."
- Don't Miss: Magic Leap's Neal Stephenson Reveals What It's Like to Create Content for the Secretive Startup
Although Magic Leap hasn't promised such functionality in the first iteration of its device, the Magic Leap One: Creator Edition, the simple fact that the company is working on such functionality is exciting and could open up a new world of communication for the deaf and hearing impaired community.
Drawings included in the patent filing show a person wearing a head-mounted device and interacting with a person using sign language. From the device wearer's perspective, we see a menu that offers a possible set of responses to the person using sign language.
In addition to allowing a person using sign language to seamlessly communicate with someone who doesn't understand sign language, the application would also allow sign language users from different parts of the world to communicate effectively.
"Many people do not know any sign languages. A speech- or hearing-challenged person and a conversation partner may not be familiar with the same sign language," reads the patent application description.
"This can impede conversation with the hearing-challenged or the speech-challenged persons. Accordingly, the wearable system that can image signs (e.g., gestures) being made by a conversation partner, convert the signs to text or graphics (e.g., graphics of sign language gestures in a sign language the system user understands), and then display information associated with the sign (e.g., a translation of the signs into a language understood by the user) would greatly help improve the communication between the user and the conversation partner."
The other language in the patent application goes into detail regarding text translation, stating that, "The wearable system can also recognize text in the users environment, modify the text (e.g. by changing the content or display characteristics of the text) and render the modified text to occlude the original text."
This could be just a groundbreaking as the sign language function, as it could immediately erase any language barriers for a person traveling in a foreign country by, for example, instantly changing Japanese kanji on a road sign to English for the person wearing the headset. This kind of tool is already available via Google's Translate app for smartphones, and it works quite well. But embedding a similar functionality into a pair of smart goggles would make the process even more seamless.
A drawing included in the patent application filing shows a person standing on a street with the Chinese characters for a bank displayed on one side, while on the other side the English translation is displayed, indicating that a translation has occurred.
And beyond translation, the "sensory eyewear" could also help users to read text that may be difficult to see.
"The wearable system can cause a head-mounted display to display the text in a font size that is different from a font size associated with the letters or characters associated with the original image," reads another part of the patent application filing. "The wearable system can also improve the display quality of the text. For example, various environmental factors, such as fog, haze, rain, bright light, low light, low light or color contrast between the letters and the surrounding image, etc., can impede a user's clear view of text in the environment without the wearable system. The wearable system may present a sign (e.g., with increased contrast ratio or larger font) that will increase the clarity of the text."
These latest patent revelations are a big departure from the gaming and entertainment experiences being touted by Magic Leap, but they're just a important, if not more so.
Yes, Magic Leap will need to capture the imagination of the public with gaming, cinematic, and sports experiences to get the revenue ball rolling. But basic assistive features like real-time sign language and text translations could transform the Magic Leap One (or future iterations of the device) from a luxury purchase into an absolute necessity.
- Follow Next Reality on Facebook, Twitter, Instagram, YouTube, and Flipboard
- Sign up for our new Next Reality newsletter
- Follow WonderHowTo on Facebook, Twitter, Pinterest, and Flipboard
Cover image via Magic Leap/USPTO
Comments
No Comments Exist
Be the first, drop a comment!