Apple’s Live Text will use AI – iOS 15

Have you ever wondered if ios AI technology recognizes your handwritten receipts by looking at an image?. Yes, now your ios15 has that handy AI technology. One of the easiest and coolest features that come with iOS 15 is the ability to quickly identify, select, copy, paste, view, and much more text on both the camera and photo app.

Let’s take a look at how the iPhone Live Text OCR works, from photos to screenshots and handwriting.

This is certainly not a new feature for smartphones and we have seen companies like Samsung and Google offer similar tools in the past. But Apple updated usually seems smooth. 

Until now, you needed to download a third-party app to get this functionality on the iPhone and iPad. But with iOS 15  the Apple Camera app includes Optical Character Recognition (OCR) as a seamless alternative, as well as photos and screenshots you’ve already taken in the Photo app.

Apple says this feature is enabled using “deep neural networks” and “device intelligence” and is the second most preferred phrase for machine learning.

Live text works on iPhones, iPads, and Macs. It supports seven languages: English, Chinese, French, Italian, German, Spanish, and Portuguese. It integrates with Apple’s Spotlight search feature on iOS, allowing you to search your camera roll based on the footprints in the pictures.

In addition to extracting text from photos, iOS 15 will allow users to visually search – a feature similar to Google Lens, which enhances Apple’s visual appearance.

Apple did not go into detail about the feature during its WWDC presentation, but it did say the new tool will detect “art, books, wildlife, pets, and landmarks” in photographs. We’ll have to try it ourselves to see how well it works, but it appears that Apple is using AI to make users’ images and other data more helpful.

 

How to use iPhone Live Text OCR in iOS 15

I will give you a very simple example before I go into more detail. When you find a recipe you like in a book or magazine. 

Take out your iPhone or iPad, open the built-in camera app, and select the text you want to take. Wait a second or two and a small icon will appear in the lower right corner. Tap on it and a popup window will appear with the text in it. You can select the words, sentences, etc. that you want to copy. 

Now you have a brief idea of how it works. So let’s look in detail at how it works.

Live text – In the camera app

• In the Camera app with iOS 15, find the new live text button (three-line view square)
• When you move your camera around, look for the yellow viewfinder rectangle to emerge as the camera recognizes text.
• When you have a yellow display machine around the text you want, tap the live text button to select, copy, view, translate and share the identified text.
• Text such as phone numbers, emails, dates, addresses, etc. Live links that can be activated for a long-press
• If the live text button and the yellow display do not show up, try to get closer to the text

In the photo app

 

• Live Text works with previously captured photographs and screenshots.
• Open Apple’s Photo app in iOS 15 and search for the same live text icon(Above I mentioned)
• Tap the icon to recognize all the text in an image
• Or you can swipe or tap, copy, paste, tap. First your text without tapping the live text icon

 

Live text and handwriting

 

During the WWDC keynote address, Apple recognized the handwriting and displayed live text. We tested it too – at least currently in beta – it works, but the accuracy depends on the handwriting being a little clearer.

From personal to work and much more, there are many uses for iPhone Live Text OCR. What are you interested in using the new feature?