Google Lens, the company’s visual search tool, has taken a leap forward by allowing users to search the web with video footage. The new video search feature, powered by Google’s advanced Gemini AI model, enables users to capture videos of their surroundings and ask questions about what they see, with instant, AI-generated results.
This feature is accessible through the Google app, where both Android and iOS users can record videos directly within the Lens tool. By holding down the shutter button, users activate the video capture mode, after which they can verbally inquire about the objects or scenes in view. Google’s AI analyzes the video frames to deliver relevant information about what’s being recorded.
How It Works
Lou Wang, director of product management at Google, explains how the feature simplifies the way users interact with their environment.
“If you want to learn more about, say, a school of fish swimming, Lens will analyze the video and provide insights about their behavior, such as why they swim in a circle,” says Wang. “It also links to additional resources for deeper understanding.”
The technology behind this relies heavily on Google’s Gemini AI, which utilizes computer vision to break down video frames into valuable data points. The AI then sifts through those frames, selecting the most informative ones and generating a summary or linking to detailed answers. This marks a significant improvement over previous iterations of Google Lens, which primarily focused on still image searches.
Users can gain access to this feature by signing up for Google’s Search Labs program. This program allows users to opt into experimental features, including the new AI Overviews, which generate concise summaries at the top of search results.
A New Era for Visual Search
The expansion to video search is a key development in Google’s larger effort to make search more interactive and intuitive. Google Lens is already used for billions of image searches each month, and the integration of video is expected to increase these numbers. The company has been positioning Lens as a tool for everyday exploration, allowing users to satisfy their curiosity with just the tap of a button.
This feature is not only about answering everyday questions but has practical applications in other areas like e-commerce. As Wang explains, “If you see a backpack you like in a video, Lens can instantly identify the product and pull up relevant details such as price, availability, and reviews.” This makes Google Lens a powerful tool for businesses and consumers alike, especially in sectors like fashion, electronics, and beauty, where product identification is crucial.
The feature is currently available in select regions and for specific product categories. As the technology matures, Google plans to expand its capabilities and refine its accuracy in different fields of application.
Voice and Video Integration
In addition to video capture, the new Google Lens update allows for voice queries during the video recording process. This multimodal approach—combining both voice and video—makes search more natural and user-friendly. Users can speak directly to the app, asking questions like, “What kind of fish is this?” or “How do I cook this vegetable?” and receive immediate answers based on the video footage.
Wang emphasizes that the driving force behind this update was the realization that people often find themselves needing more contextual information about what they’re seeing. “If you lower the barrier to asking questions and satisfy users’ curiosity faster, we believe this will make search a more integral part of people’s lives,” he said.
Google’s competitors have also been developing visual search tools, but this latest move sets Google Lens apart, especially in the realm of real-time video analysis. Apple’s Visual Look Up feature, for instance, focuses on image recognition and doesn’t yet support the level of interactivity that Google Lens now offers.
By integrating AI-driven insights into video, Google is opening new doors for applications in education, shopping, tourism, and everyday problem-solving. Future updates are expected to include even more precise AI responses and a broader range of supported topics.
Google Lens continues to evolve as a pivotal part of the search ecosystem, and this latest video search feature represents a major step forward in the way users interact with the digital and physical worlds.
