Select users of Meta's mixed-reality Ray-Ban glasses can now experiment with several new AI-powered features, including the ability to translate between languages in real-time, and label music playing in the vicinity via a Shazam integration.
Members of Meta's early access program in the U.S. and Canada can now download firmware v11, which brings "live AI" to their Ray-Ban glasses, allowing them to have continuous conversations with Meta AI, the company's Llama-powered AI assistant.
According to the tech giant, during live AI sessions, Meta AI can now see continuously through users' glasses, providing them with the option to ask real-time questions regarding their surroundings, including "meal prep, gardening, or exploring a new neighborhood."
Per live AI, Meta AI is now able to understand references a user discussed earlier in a session, allowing users to forgo the need to say "Hey Meta" and instead asking follow-up questions in a more natural conversational way.
Live AI works off of real-time AI video, a major focus of this year's Meta Connect conference. Competing with similar technologies being developed by OpenAI and Google, Meta's Ray-Ban glasses might be the first product to hit the market that utilizes real-time AI video on smart glasses.
"Eventually live AI will, at the right moment, give useful suggestions even before you ask," the company says.
Users of Meta's Ray-Ban glasses product will also be able to translate speech in real-time between English and Spanish, French, or Italian, which could become a very useful tool when traveling to foreign countries or working with people who speak a different language.
"When you're talking to someone speaking one of those three languages, you'll hear what they say in English through the glasses' open-ear speakers or viewed as transcripts on your phone, and vice versa," the company explains.
Finally, the launch of live AI will make it easier for users to use the song-identifier app Shazam by saying, "Hey Meta, what is this song?"
In its announcement, Meta says that these AI features "may not always get it right" and that the company is looking for feedback before rolling out the features to more users.
This is Meta's most substantial AI update to its Ray-Ban product since November, when the company began to roll out the ability for users in Europe to ask questions with their voice. Meta first began testing multimodal AI - the tech behind the product's real-time AI features -- last December.
In terms of sales, Meta's Ray-Ban glasses were the top-selling glasses brand in 60% of all Ray-Ban stores across Europe, the Middle East, and Africa in Q4, according to Ray-Ban owner EssilorLuxottica.