Alexa or Siri, they have been useful assistants to a certain degree but lately, they haven’t been the next-gen help that we were hoping them to be. Google has the technology to change the way we look at digital assistants with Project Astra.

The company revealed a new AI assistant in progress under the working name Project Astra. It’s a multi-modal feature that combines the smarts of Gemini with the kind of image recognition abilities you get in Google Lens, besides taking into account powerful natural language responses.

Let’s break it down. At the moment, we interact with digital assistants using voice. Astra is multi-modal, that is, it can use sight and sound in addition to text/speech. On paper, Project Astra can work and behave more like a real assistant or agent compared to a robotic voice that responds to spoken commands.

Here’s an example. Place a few objects in front of the camera, like a dinosaur and a baguette. Astra can potentially tell a story about a dinosaur and its baguette. This can be entertaining to a child in need of a bedtime story.

Demis Hassabis, the head of Google DeepMind and the leader of Google’s AI efforts, showcased an early version of Project Astra and he hopes it will become the universal assistant that we need.

It can “see” the world and knows what things are and where you left them. For example, it can identify a missing part in, say, a car engine or find your missing glasses (in case it has seen it before) and review codes. All of it works in real-time and the process is conversational.

Google hopes to add Astra’s capabilities to Gemini and across its product this year, Alphabet CEO Sundar Pichai said. But he has also said that it will be rolled out cautiously and the project will be quality driven.

“Getting response time down to something conversational is a difficult engineering challenge,” said Hassabis. “It is amazing to see how far AI has come, especially when it comes to spatial understanding, video processing and memory.”

Google demoed Astra with experimental glasses in a video but there is no further information about the product. Google co-founder Sergey Brin, who famously led the introduction to Google Glass in 2012, was there at this year’s I/O conference. He told reporters: “It’s funny, because it’s like the perfect hardware (glasses). It’s like the killer app now, 10 years later.” He said he uses AI for coding tasks. Brin believes coding is the “number one” use of AI today. “It wasn’t completely flawless. But [there were] a couple of minor issues that I’d cleaned up right away. That was pretty amazing.”

He told journalists: “I can’t imagine a better time to be a computer scientist, and I feel super lucky to be able to see the details of how these things work.”

The Astra feature that may set it apart from other products is its built-in “memory”. During a demo, Astra scanned a number of objects and it could “remember” where specific items were placed. At the moment, Astra’s memory is limited to a relatively short window of time but that will improve. It may open up more possibilities for the tech. Instead of asking Astra where you left your glasses a minute ago, it would be helpful to know where you left them last night. If Astra can answer that then “hello sci-fi”.