When Mobile ToF Meets Micro-Vibration:
Multi-Point kHz-Frequency Sensing using Laser Speckle

Sensing micro-vibration plays a critical role in diverse industrial applications, including structural health monitoring and machinery diagnostics. However, most existing vibration sensing technologies rely on physical contact with the target object or demand complex installation procedures, making them intrusive and costly. This paper presents PocketVib, the first cost-effective, portable smartphone-based laser speckle vibrometry system for multi-point kHz-frequency micro-vibration sensing. PocketVib integrates built-in smartphone sensors, i.e., Time-of-Flight (ToF) depth cameras and rolling shutter cameras, to achieve non-contact micron-level vibration sensing. The design features three key innovations: 1) emitter-imager synchronization, which ensures stable speckle pattern acquisition, 2) a dual-lens enhanced imager that stretches speckle patterns to improve their limited coverage on the rolling shutter camera, and 3) a reference-free algorithm that leverages temporal correlations across rolling shutter image rows for precise vibration extraction. We implement PocketVib on commodity smartphones with a custom hardware kit. Extensive experimental results demonstrate that PocketVib enables fast and accurate vibration frequency estimation on mobile devices, achieving frequency measurements up to 1 kHz with an average error of only 0.6 Hz at just 30 fps, while supporting simultaneous measurements at seven points.

Recently, Laser Speckle Vibrometry (LSV) has emerged as a promising technology for micro-vibration sensing through the coherent light interference phenomena. Its standard implementation comprises two core components: 1) a coherent laser source that generates stable speckle patterns, and 2) a defocused high-speed imaging system that captures speckle dynamics.

Principles of LSV: When coherent light illuminates a rough surface, scattered waves with varying optical paths create the speckle patterns. The defocused configuration enlarges individual speckles across multiple sensor pixels, amplifying motion-induced intensity variations. The high sampling rates ensure accurate capture of high-frequency vibrations.

Coherent Laser Source: Current smartphones integrate coherent light sources via their built-in ToF depth cameras, particularly those implementing spot illumination architectures. Unlike conventional flood ToF systems that homogenize the illumination energy via a diffuser, spot ToF preserves beam coherence using diffractive optical elements (DOEs) to generate structured light patterns (i.g., the LiDAR sensor in iPhone Pro devices).

High-Sampling-Rate Defocused Imager: Modern smartphone cameras feature rolling shutter mechanisms and auto-focus systems, making them particularly well-suited as high-sampling-rate defocused imagers for LSV systems.

PocketVib consists of three key components, each addressing specific challenges in adapting LSV to mobile devices:

• Emitter-Imager Synchronization: We conduct a comprehensive profiling of the operational mechanism of the ToF laser emission and design a precise synchronization mechanism between the ToF camera and the rolling shutter camera. This ensures stable speckle patterns, enabling continuous and reliable acquisition of vibration data. (Challenge 1: Unstable Speckle Patterns)
• Dual-Lens Enhanced Imager: We develop the PocketVib Kit equipped with a cylindrical lens to address the limited coverage of defocused speckle patterns on the rolling shutter camera sensor. The lens stretches the speckle patterns along the rolling shutter direction, allowing more rows of the speckle image to be captured and enhancing the accuracy of vibration analysis. (Challenge 2: Limited Coverage of Rolling Shutter)
• Reference-free Micro-vibration Extraction: We leverage the temporal relationship between adjacent rows of the rolling shutter camera to extract micro-vibration information, eliminating the need for an external reference frame. (Challenge 3: Lacking Reference Frame)

We demonstrate PocketVib with an example measurement of a tuning fork's vibration. The corresponding frequency is 964.90 Hz. The left figure shows the experimental setup, while the right figure shows the captured speckle pattern and the corresponding vibration analysis results.

What you'll need:

1. Apple's iPhone Pro series (12-17 generations)
2. A short-focal-length cylindrical lens
3. An IR-pass filter
4. A 3D-printed slit and mounting bracket

Step 1: Remove the IR-cut filter of smartphone cameras

To capture infrared speckle patterns, you need a camera module with its IR-cut filter removed. This modification is a well-known operation for infrared photography, used in both professional and consumer-grade imaging systems. Detailed online guides are available to help you safely perform this modification without damaging your device, or you can opt to purchase a pre-modified camera module. Helpful resources are listed below:

1. https://zh.ifixit.com/Guide/iPhone+15+Rear+Cameras+Replacement/165721
2. https://zh.ifixit.com/Guide/iPhone+DIY+Night+Vision+Camera+and+Massive+External+Battery+Pack/147886
3. https://hackaday.io/project/185259-multispectral-imaging-smartphone-camera

Step 2: Assemble the PocketVib

An example assembly of the PocketVib system on iPhone 12 Pro is illustrated below:

1. Attach the cylindrical lens in front of the modified camera using the 3D-printed mounting bracket. Ensure that the lens is aligned correctly to stretch the speckle patterns along the rolling shutter direction.
2. Place the IR-pass filter in front of the lens to block visible light, allowing only infrared light to reach the camera sensor.
3. Place a small slit on the LiDAR, allowing only a single row of speckles to exist within the camera's field of view (FOV)

Step 3: Install PocketVib App

Detailed instructions are available on our GitHub repository. Now use the PocketVib! Take a short video of a vibrating object!