Multimodal Prompting (Images & Vision)
Reviewed & published by Brayan K
Multimodal prompting means giving the AI more than text, most often images (and sometimes audio or video). You upload a picture and pair it with a clear instruction: analyze, extract, transcribe, or compare.
Learn Multimodal Prompting (Images & Vision) in our free Prompt Engineering course — a beginner-friendly interactive lesson with worked examples, a practice…
Part of the free Prompt Engineering course at LearnCodingFast — hands-on lessons with examples you run in your browser, plus practice exercises and a quick quiz.
The skill is the same as always: be specific. This lesson shows how to combine images with text instructions and covers common use cases and limitations.
What You'll Learn in This Lesson
1 Image plus a specific instruction
An image alone is not a task. Pair it with a clear instruction about what to do. Same picture, two prompts:
2 Common image use cases
Use case
Example instruction
OCR / transcribe
“Transcribe the handwritten note in this photo.”
Analyze a diagram
“Explain what this flowchart does, step by step.”
Review a screenshot
“Identify the error message and suggest a fix.”
Compare images
“Compare these two charts and list the differences.”
In each case you combine the picture with a precise instruction about what to identify, extract, or compare.
3 Limitations to keep in mind
Image models are powerful but not perfect. They can misread blurry, low-resolution, or ambiguous pictures and miss small text or fine detail.
- Use clear, high-quality images — better input, better output.
- Verify important numbers — treat receipts, IDs, and legal or medical text as drafts to check.
- Say what to do if unreadable — ask the model to flag anything it cannot read.
📋 Copy-paste image prompt template
Try it for real
There is deliberately no model output in this lesson, because an image reply is not deterministic and any "expected output" here would be invented. What you can check is the request. Run this on both prompts: the bare one leaves four decisions to the model — where to look, what to extract, how to shape it, and what to do when the answer is not there.
Common Mistakes (and the fix)
- Sending an image with no instruction. — Always pair it with a specific task.
- Vague requests like 'look at this'. — Name what to analyze, extract, or compare.
- Using a blurry image. — Provide a clear, high-resolution picture.
- Trusting fine details blindly. — Verify important numbers against the original.
Frequently Asked Questions
⏱ Test Yourself — Timed Quiz
10 quick questions, 12 seconds each. Instant feedback — beat the clock!
🎉 Lesson Complete
- ✅ Multimodal prompting adds images (and audio/video) to text
- ✅ Pair every image with a specific text instruction
- ✅ Common uses: OCR, diagrams, screenshots, comparisons
- ✅ Be specific about the task and the output format
- ✅ Verify important details; image reading is not perfect
Practice quiz
What is 'multimodal' prompting?
- Prompting with more than just text, such as images, audio, or video
- Using multiple keyboards
- A faster typing mode
- A type of password
Answer: Prompting with more than just text, such as images, audio, or video. Multimodal means combining text with other inputs like images.
When you send an image, you should also…
- Send it with no instructions
- Delete the image first
- Add a clear text instruction about what to do with it
- Only send the file name
Answer: Add a clear text instruction about what to do with it. Pair the image with a specific text instruction so the model knows the task.
Which instruction is most specific for an image task?
- nice picture
- Extract every line item and total from this receipt as a list
- look at this
- do something
Answer: Extract every line item and total from this receipt as a list. A specific task, extract line items and total, beats a vague request.
OCR with an image model means…
- Deleting the image
- Resizing the file
- Drawing a picture
- Reading text out of an image
Answer: Reading text out of an image. OCR is extracting written text from an image.
A good use case for image prompting is…
- Transcribing a handwritten note
- Sorting your email
- Compressing a video only
- Turning off the screen
Answer: Transcribing a handwritten note. Reading or transcribing text in an image is a classic multimodal task.
Combining an image with text instructions lets you…
- Hide the image
- Tell the model exactly what to analyze, extract, or compare
- Avoid all instructions
- Only see the image
Answer: Tell the model exactly what to analyze, extract, or compare. Text guides what the model should do with the image.
Asking the model to 'compare these two charts' is…
- A drawing task
- Impossible
- Only for text
- A valid multimodal task if both images are provided with a clear instruction
Answer: A valid multimodal task if both images are provided with a clear instruction. Comparison across provided images is a common multimodal use.
A limitation to keep in mind with image prompting is…
- Models never make mistakes on images
- It only works at night
- It can misread blurry, low-resolution, or ambiguous images
- It deletes the image
Answer: It can misread blurry, low-resolution, or ambiguous images. Poor image quality or ambiguity can lead to errors, so verify important results.
For analyzing a diagram, the best prompt…
- asks for a poem
- describes specifically what to identify or explain in the diagram
- just says 'diagram'
- sends no text
Answer: describes specifically what to identify or explain in the diagram. Specific instructions about what to identify produce better diagram analysis.
Which is a strong multimodal prompt?
- Here is a screenshot of an error. Identify the error message and suggest a fix.
- img
- ?
- this
Answer: Here is a screenshot of an error. Identify the error message and suggest a fix.. It names the input, the task, and the desired output.
Continue this course
- Previous: Prompting AI Agents
- Next: Prompt Injection & Safety