Live Recognition - have your iPhone or iPad describe your surroundings 

What is Live Recognition?

Live Recognition is a set of visual intelligence tools built in to iOS and iPadOS that helps you explore your surroundings using the camera on your iPhone or iPad. The tools available will depend on your device model and can include:

Ask,
Scenes,
Doors,
Furniture,
Point & Speak,
Text and
Document Capture.

Ask, which is the most flexible tool, allows you to ask about what the camera sees. This requires a device that supports Apple Intelligence. If you have Apple Intelligence then the Ask tool is probably your most useful choice. It will let you get directly to the information you want rather than listening to a ful description. For example you might say "read this" or  "what is the use by date on this pack?" or "what baguettes are on the menu?".The Doors and Furniture tools require LIDAR which is available only on iPhone Pro and Pro Max models starting with iPhone 12. It is also available on recent iPad Pro models. I find that Ask is usually a good alternative to these two for finding doors and furniture and is an excellent general purpose tool.
Scenes gives a description of what the camera sees but without Apple Intelligence this is a very basic tool.
Document Capture assists with positioning a document to be spoken.

Live Recognition settings

The default settings for Live Recognition will probably suit most but take a look at Settings / Accessibility / Live Recognition if you'd like to fine tune or perhaps set a default question. 

How is Live Recognition used?

Live Recognition can be started in several ways.

Live Recognition or its Ask or Scenes tools can be assigned to a long press of the Action Button which is present on iPhone 15 Pro and Pro Max, all iPhone 16 and later as well as iPhone Air. iPhone Duo does not have an Action Button.
VoiceOver users can start Live Recognition with a four finger triple tap.
Live Recognition or its Ask or Scenes tools  can be added to the Accessibility Shortcut or set as a Back Tap action.

When Live Recognition is started by a VoiceOver user it displays all of the available tools for you to select from. Please experiment with the tools available on your iPhone or iPad. If you aren't using VoiceOver then Magnifier will be opened and you can use the live Recognition tools from that rather cumbersome app - but don't despair, there is a better solution on offer for non-VoiceOver users.

Both Back Tap and the Accessibility Shortcut can be set to Ask a Question or Describe Scene which takes both VoiceOver users and non-VoiceOver users directly to speaking their question or hearing a description of the scene. You'll find Back Tap at Settings / Accessibility / Touch / Back Tap. Use the Accessibility button to display the list containing Ask a Question and Describe Scene. The Accessibility Shortcut setting is at Settings / Accessibility / Accessibility Shortcut.

Alternatively, phones with an Action Button can be set to have a long press of the action button take you directly to speaking a question about what the camera is pointing at or hearing the scene described. Here's how a VoiceOver user can set the Action Button to Ask a Question. A similar process works for Describe Scene.

First click the Action Button  which is above the volume buttons on the left of your iPhone. If you hear nothing then the Action Button is unassigned. If you hear the name of an action then the Action Button is already in use. You can change it to Ask if you wish but you may prefer to use Back Tap or the Accessibility Shortcut.

To set or change the setting of the Action Button go to Settings/ Action Button.

You will probably hear "No Action" or the name of your currently assigned action. Swipe left and you should hear something like "No Action, adjustable". "Adjustable" tells you that you can move through the available choices by swiping up or down with one finger. Swipe vertically with one finger until you locate Accessibility. If you started with No Action then Accessibility is just one swipe down away. Now swipe right to locate the Choose a Feature button and double tap. Swipe right to locate Ask a Question and double tap to select it. All done.

Now, all you need to do to start speaking a question is to long press the Action Button. Release the button when you hear a ping. You may also hear VoiceOver speaking the label of a button but you can ignore that. The ping indicates that the phone is waiting for you to speak so point the camera towards your subject and speak your question. Following the answer you will hear another ping, indicating that you can ask more questions.

Back to home page

last updated 31 October 2026