Lidar, a technology originally used by meteorologists and aerospace engineers, was later introduced into the field of self-driving cars and has gradually penetrated into the consumer electronics market over the past five years. If you own a Pro model of iPhone or iPad, it's likely it's equipped with a lidar sensor, and you may have used this technology without knowing it.

Apple has been adding lidar (light detection and ranging) technology to its products since 2020, first on the 11-inch second-generation iPad Pro, 12.9-inch fourth-generation iPad Pro, iPhone 12 Pro and iPhone 12 Pro Max. Since then, lidar sensors have appeared on every iPad Pro and Pro model iPhone. If your Pro model device is released in 2020 or later, then it already has a lidar sensor integrated into its rear camera system.

• Don't miss: What happens to your iPhone every time you turn on low battery mode?

LiDAR works by emitting laser light to an object and then detecting the time it takes for light to return to the receiver. Because of the extremely fast speed of light, even slight differences in time can allow the device software to build a three-dimensional point cloud map of the region. This 3D map can be used to measure objects, create 3D models of physical objects, and enable other cool features such as placing virtual objects in your room or navigating in augmented reality (AR).

In AR, applications can better understand the geometry of their surroundings, thereby placing digital objects in the real world more accurately, ensuring they conform to the physical properties of the space. Devices equipped with lidar sensors provide a more realistic AR experience compared to non-Pro models of iPhones and iPads, as they more accurately map the surroundings captured by the camera and provide better motion tracking and depth perception.

Apple debuted the lidar scanner API in ARKit for the first time in iOS and iPadOS 13.4 versions, allowing third-party developers to access the polygonal model mesh of their surroundings and operate on it using RealityKit to create an enhanced experience. Starting from iOS and iPadOS 15.4 versions, Apple has further enhanced the rights of developers and added the lidar API in AV Foundation, providing high-quality and high-precision in-depth information for video recording and photography.

Below, we will explore some of the built-in LiDAR uses of iPhone or iPad, and how third-party applications can expand the scope of this technology.

1. Faster autofocus

Camera equipped with lidar sensors have more advanced autofocus capabilities in Apple's Camera applications than devices without lidar. The camera's ISP (image signal processor) uses lidar to determine the distance between objects and people for more accurate automatic focus – even in low light conditions, while reducing shooting time. This is because lidar measures depth data, rather than processing the image itself.

This faster autofocus is not only suitable for still photos, but also for video shooting, allowing you to focus on shooting moments without worrying about camera focus. However, it should be noted that lidar autofocus is not very friendly to transparent objects such as windows, glass, water, etc., because it will treat it as opaque.

LiDAR autofocus priority is also integrated into the AVFoundation framework, a framework for accessing and capturing still images and videos from the camera, so third-party applications can also take advantage of the lidar. With this feature, photography applications can take advantage of faster and more accurate autofocus. Applications like Focus Puller can even allow lidar autofocus to apply to other hardware, such as BlackMagic cameras.

2. Better portrait shots

When Portrait Mode was first launched in 2016, it relied on machine learning to identify the subject of the photo and appropriately apply background blur. This is still the case with regular iPhone and iPad models. But on Pro-level devices, lidar greatly improves the quality of portraits taken by rear cameras because it can help capture more accurate depth data.

Even third-party apps can take better portraits, such as ProTake, which uses lidar to allow you to record portrait videos with real-time blurred backgrounds. There is also DSLR Camera, which allows you to create AR portraits using lidar.

3. Night mode portrait

LiDAR also enables iPhone 12 Pro and newer Pro models to achieve night-mode portrait shooting. Therefore, when shooting portraits with iPhone 12 Pro, 13 Pro, 14 Pro or 15 Pro models in dark light, the lidar scanning function works more frequently for better scene depth reading, allowing the camera to quickly adapt to subject depth and adjust exposure settings more accurately. This allows for clear portrait photos in low-light environments. On iPhone models without lidar, night mode will not start at all.

4. Improved refocusing accuracy

While you can change the focus of captured portrait photos on any iPhone or iPad running iOS or iPadOS 17 or later, the Pro model devices work better because they use data captured by the lidar scanner. On iPhone 15 Pro models, you can even refocus normal photos after enabling the Portrait option in Photo Mode.

• Complete Guide: Transform your iPhone portrait in photos with this must-know refocus tool

5. Easier and more accurate measurements

Apple's Measure app uses AR technology to map your environment and measure objects within the framework, essentially turning your iPhone or iPad into a digital tape measure. Although this app has been available to any iPhone or iPad user since iOS 12, it is easier and more accurate to use with a lidar scanner on a Pro model device because it better understands your spatial relationship with objects in the picture.

Lidar makes measurements more accurate than devices that rely solely on camera data and motion sensor processing. LiDAR also provides vertical and horizontal edge guidance for Apple's Measure applications. When you place the circle close to the edge, Measure automatically detects the edges and displays yellow guide lines along the side so that you can see and follow these lines more easily. On devices without lidar, marking the first point will only show a dotted line without sticking.

LiDAR also provides more refined measurements for Ruler View, allowing you to measure objects more accurately in incremental ways. Simply draw a line and get close to it to show the ruler.

Pro models equipped with lidar can also benefit from the history-proven functionality (click on the list button) from the Measure application. While the lidar itself is not responsible for this feature, it does provide you with additional measurement details. When you measure an object, click the measurement result and expand the window to view information such as height, distance, angle, etc. For example, when looking at the area, you will see examples of shapes and additional units of measurement.

There are also many third-party measurement applications that also use lidar, such as LiDAR Measuring, which gives you the measurement of the distance between you and any object. It even allows you to set a specific distance, and when you approach a fixed point, it gives an approximate distance prompt. Laser Rangefinder - LiDAR also uses lidar to provide "millisecond-level accuracy" ranging.

6. Anthropometric measurement

A cool feature of Apple Measure app is that it can calculate the height of people sitting or standing. This feature is only available for iPhones and iPads equipped with lidar. Whenever the Measure app detects a person sitting or standing in the screen, it will automatically display measurements on the top of its head, hair, or hat. Press the shutter button to take a photo to share photos with measurements.

• More information: How to quickly measure the height of a person standing or sitting with your iPhone

7. Accessibility

Accessibility tools are very important to Apple, and its continuous efforts are aimed at making technology accessible to everyone, which led to the launch of the Magnifier app, which first appeared in iOS 10, helping users easily amplify and identify items in the camera's field of view. In iOS 16 and iPadOS 16, Apple added Door Detection feature, specifically targeting Pro models of devices, using lidar to map and identify doors and their distances.

Text descriptions about the door will appear on the screen, such as distance, whether the door is open, the type of door, how to open and close, whether there are signs on the door, etc. VoiceOver can also provide descriptions for blind and low-eye users.

• Don't miss: 17 new accessibility features coming to iOS 18 and iPadOS 18

There are other applications that can help users with vision impaired, such as Microsoft's Seeing AI, which combines lidar and augmented reality technology to help users explore their surroundings and provide voice descriptions of objects through spatial audio. In a simpler implementation, there is LiDAR Sense. With this app, you can hold your phone in front of you and walk in front of the rear camera. When approaching objects in the path, the app vibrates more intensely and emits

The above is the detailed content of You're Using Lidar on Your iPhone and iPad — And You Don't Even Know It . For more information, please follow other related articles on the PHP Chinese website!