📄 ai-sdk/cookbook/node/call-tools-with-image-prompt
File: call-tools-with-image-prompt.md | Updated: 11/15/2025
Source: https://ai-sdk.dev/cookbook/node/call-tools-with-image-prompt
Menu
Google Gemini Image Generation
Get started with Claude 3.7 Sonnet
Get started with OpenAI o3-mini
Generate Text with Chat Prompt
Generate Image with Chat Prompt
streamText Multi-Step Cookbook
Markdown Chatbot with Memoization
Generate Object with File Prompt through Form Submission
Model Context Protocol (MCP) Tools
Share useChat State Across Components
Human-in-the-Loop Agent with Next.js
Render Visual Interface in Chat
Generate Text with Chat Prompt
Generate Text with Image Prompt
Generate Object with a Reasoning Model
Stream Object with Image Prompt
Record Token Usage After Streaming Object
Record Final Object after Streaming Object
Model Context Protocol (MCP) Tools
Retrieval Augmented Generation
Copy markdown
===========================================================================================================================
Some language models that support vision capabilities accept images as part of the prompt. Here are some of the different formats you can use to include images as input.
import { generateText, tool } from 'ai';import { openai } from '@ai-sdk/openai';import { z } from 'zod';
const result = await generateText({ model: openai('gpt-4.1'), messages: [ { role: 'user', content: [ { type: 'text', text: 'can you log this meal for me?' }, { type: 'image', image: new URL( 'https://upload.wikimedia.org/wikipedia/commons/thumb/e/e4/Cheeseburger_%2817237580619%29.jpg/640px-Cheeseburger_%2817237580619%29.jpg', ), }, ], }, ], tools: { logFood: tool({ description: 'Log a food item', inputSchema: z.object({ name: z.string(), calories: z.number(), }), execute({ name, calories }) { storeInDatabase({ name, calories }); // your implementation here }, }), },});
On this page
Deploy and Scale AI Apps with Vercel.
Vercel delivers the infrastructure and developer experience you need to ship reliable AI-powered applications at scale.
Trusted by industry leaders:
- OpenAI
- Photoroom
