Multimodal AI models can include visual tools that enable them to manipulate images or retrieve external information. A zoom tool can be used to focus on a specific section of a painting. A reverse image search tool can find similar images across the Web. This visual search can retrieve metadata that improves the recognition and interpretation of visual information. We will begin with existing visual tools and consider which additional tools could aid research.

Image: Elise Racine & Digit / Woven Dialogues / Licenced by CC-BY 4.0