React Native Vision Camera Frame Processors: Real-Time ML Kit OCR and Face Detection in Expo (2026)

Updated: July 4, 2026

React Native Vision Camera frame processors are worklet-based JavaScript functions that run once per camera frame on a dedicated JS runtime, giving you a 33 ms budget at 30 FPS (16 ms at 60 FPS) to analyse the raw GPU-backed buffer before the next frame arrives. In this guide I profile a real Expo build running react-native-vision-camera v5 with ML Kit face detection and OCR, show the trace where a naive processor burns 47 ms per frame, and walk through the exact runAsync plus runAtTargetFps pattern that pulls the average frame time back under 9 ms.

• A Vision Camera frame processor runs synchronously on the camera JS runtime; if it overruns the frame budget (33 ms at 30 FPS, 16 ms at 60 FPS), the next frame is dropped.
• VisionCamera v5 auto-detects react-native-worklets via Nitro Modules, so you no longer install react-native-worklets-core separately.
• runAsync offloads heavy work (ML Kit face detection, OCR) to a parallel thread; only one runAsync call runs at a time, so it throttles rather than parallelises.
• runAtTargetFps(rate, cb) drops frames on the camera thread itself and is the modern replacement for the deprecated frameProcessInterval prop.
• Vision Camera does not work in Expo Go; you need a development build (eas build --profile development) or a bare workflow.
• On iOS the camera sensor is fixed landscape, so ML Kit needs an explicit outputOrientation hint to align text and face boxes with the UI.

What is a frame processor in React Native Vision Camera?

A frame processor is a JavaScript function marked with the 'worklet' directive that Vision Camera hands each camera frame as a GPU-backed buffer. It runs on the VisionCamera JS Runtime, a separate runtime from the React Native main thread, and is invoked synchronously via JSI without ever going over the bridge. That is the whole reason frame processors exist: the bridge round-trip alone would blow past a 30 FPS budget before your code even started.

In my traces on a 2024 Pixel 9 running Expo SDK 55, a bare frame processor that only reads frame.width and frame.height costs about 0.4 ms per frame. That is the floor. Anything you add (colour space conversion, cropping, ML Kit inference) comes out of the remaining budget. At 30 FPS that budget is 33 ms; at 60 FPS it is 16 ms; at 240 FPS slow-motion it is a brutal 4.2 ms and, honestly, you should not run inference at that rate.

Frame processors were rewritten on top of react-native-worklets in v5 and integrated through Nitro Modules. If you have used Reanimated 4 worklets, the mental model is identical: workletised JS runs off the UI thread, captured variables are serialised across runtime boundaries, and you cannot call regular React state setters from inside the worklet. The official Frame Processors guide is the authoritative reference for the API surface.

Installing Vision Camera v5 in an Expo development build

Vision Camera relies on native code (AVFoundation on iOS, CameraX on Android) plus a Nitro Modules build step, so it does not run inside Expo Go. You need a development build. Here is the exact sequence I use on a fresh Expo SDK 55 app:

# Expo SDK 55, RN 0.81, Xcode 16, Android Studio Ladybug
npx create-expo-app@latest camera-lab --template blank-typescript
cd camera-lab

npx expo install react-native-vision-camera react-native-worklets react-native-nitro-modules
npx expo install react-native-vision-camera-mlkit  # ML Kit plugin

# Add the config plugin so permissions strings and Info.plist entries are injected

Then edit app.json:

{
  "expo": {
    "name": "camera-lab",
    "plugins": [
      [
        "react-native-vision-camera",
        {
          "cameraPermissionText": "$(PRODUCT_NAME) needs access to your camera to scan text and faces.",
          "enableMicrophonePermission": false,
          "enableCodeScanner": false
        }
      ]
    ],
    "ios": { "bundleIdentifier": "com.example.cameralab" },
    "android": { "package": "com.example.cameralab" }
  }
}

Now build the development client and run it on a real device. The simulator has no rear camera and its front camera output is a static image, which will happily give you misleading benchmarks.

eas build --profile development --platform ios
eas build --profile development --platform android
npx expo start --dev-client

Worklets and the 33 ms frame budget

Every frame processor call has a hard deadline. Vision Camera's pipeline keeps a small buffer pool; if your worklet does not return before the next frame is captured, the pipeline stalls, the buffer pool fills, and the camera driver drops subsequent frames. In my traces you can literally see it: a straight-line drop from 30 FPS to 15 FPS the instant the processor mean time crosses 33 ms.

Here is a minimal worklet with an instrumentation counter I use to sanity-check the trace before adding any ML work:

import { Camera, useCameraDevice, useFrameProcessor } from 'react-native-vision-camera';
import { useSharedValue } from 'react-native-worklets';

export function CameraLab() {
  const device = useCameraDevice('back');
  const frameCount = useSharedValue(0);

  const frameProcessor = useFrameProcessor((frame) => {
    'worklet';
    frameCount.value += 1;
    // Baseline: reading a couple of frame properties is ~0.4 ms on a Pixel 9.
    const w = frame.width;
    const h = frame.height;
  }, []);

  if (!device) return null;
  return (
    <Camera
      style={{ flex: 1 }}
      device={device}
      isActive={true}
      fps={30}
      frameProcessor={frameProcessor}
    />
  );
}

Numbers I measured on this baseline: Pixel 9 back camera at 30 FPS held a steady 29.8 FPS with a per-frame mean of 0.42 ms and a p99 of 1.1 ms. On an iPhone 15 Pro at 60 FPS I got 59.6 FPS with a 0.31 ms mean. That is the ceiling of the pipeline itself, and every ML feature you add lives inside the remaining budget.

Reading captured variables inside a worklet

Worklets serialise variables across the JS runtime boundary at declaration time. That means the JavaScript object you close over is a snapshot: mutating it in React-land will not update the worklet. Use useSharedValue (from react-native-worklets, not from Reanimated in v5) for values you need to write from the worklet and read from React, and use the dependency array of useFrameProcessor for values that change and must trigger a new worklet capture.

runAsync vs runAtTargetFps: what to pick when

These two helpers get confused constantly and they are not interchangeable. Here is the comparison I keep pinned to my monitor.

Behavior	runAtTargetFps	runAsync
Thread	Camera JS Runtime (same as processor)	Separate async runtime
Blocks next frame?	Yes, until callback returns	No, returns immediately
Concurrency	1 at a time	1 at a time (queued)
Dropped frames	Every frame except the sampled rate	None dropped by the helper itself
Frame lifetime	Callback receives live frame	Frame is retained until callback finishes
Use for	Throttling cheap work (brightness, exposure)	Offloading heavy work (ML, OCR, image analysis)
Replaces	Deprecated frameProcessInterval prop	Manual thread juggling

The right pattern for ML Kit inference is almost always both at once: throttle the invocation rate to something the model can keep up with, and then push the actual inference off the camera thread so it does not stall the pipeline while it runs. In production I aim for 8–12 FPS on face detection and 4–6 FPS on OCR; that is more than enough for a responsive UI and it leaves the sensor free to keep streaming.

const frameProcessor = useFrameProcessor((frame) => {
  'worklet';
  runAtTargetFps(10, () => {
    'worklet';
    runAsync(frame, () => {
      'worklet';
      const faces = detectFaces(frame);
      facesShared.value = faces;
    });
  });
}, [detectFaces]);

Real-time ML Kit face detection with Vision Camera

ML Kit runs entirely on-device, which fits the same design goals I laid out in the React Native on-device AI guide: no network round-trip, no privacy exposure, and predictable latency you can profile. The react-native-vision-camera-mlkit plugin wraps Google's native ML Kit SDKs (both iOS and Android) and exposes each detector as a worklet-callable function.

import { useFaceDetector } from 'react-native-vision-camera-mlkit';
import { useSharedValue } from 'react-native-worklets';
import { runOnJS } from 'react-native-worklets';

export function LivenessCamera() {
  const device = useCameraDevice('front');
  const { detectFaces } = useFaceDetector({
    performanceMode: 'fast',       // 'accurate' costs ~2.5x more per frame
    landmarkMode: 'none',
    contourMode: 'none',
    classificationMode: 'all',     // needed for eyeOpenProbability + smilingProbability
    minFaceSize: 0.15,
    trackingEnabled: true,          // stable IDs across frames
  });

  const faces = useSharedValue([]);

  const frameProcessor = useFrameProcessor((frame) => {
    'worklet';
    runAtTargetFps(12, () => {
      'worklet';
      runAsync(frame, () => {
        'worklet';
        const result = detectFaces(frame);
        faces.value = result;
      });
    });
  }, [detectFaces]);

  return (
    <Camera
      style={StyleSheet.absoluteFill}
      device={device}
      isActive
      fps={30}
      frameProcessor={frameProcessor}
      pixelFormat="yuv"
    />
  );
}

Numbers from my Pixel 9 profile with this configuration: 12.0 detections/second (matching the target), 21.4 ms mean detection time on the async runtime, camera thread stayed under 1 ms per frame, sustained 29.8 FPS on the camera output. Compare that with the naive version (same code without runAsync), which burnt 22 ms on the camera thread every frame and dropped the sensor to 24 FPS with occasional 12 FPS stalls.

Building a liveness challenge from the signals

The classification fields ML Kit returns are the raw material for active liveness, the kind used to reject a spoofing attack with a photograph. The signals I actually use are leftEyeOpenProbability, rightEyeOpenProbability, smilingProbability, and the head pose Euler angles. Combine a blink challenge (both eye probabilities drop below 0.2 and rise above 0.9 within 800 ms) with a head turn (yaw crosses ±25°) and you have a two-step check that resists most photograph and video attacks without any server round-trip.

ML Kit OCR: text recognition from the frame buffer

OCR is where the "profile before you optimise" advice pays off the most. Text recognition inference in the Latin script model averages 42 ms per frame on my Pixel 9 (significantly heavier than face detection), and the frame buffer orientation trap will silently produce garbage results on iOS if you skip the fix.

import { useTextRecognizer } from 'react-native-vision-camera-mlkit';

export function OcrCamera({ onText }: { onText: (t: string) => void }) {
  const device = useCameraDevice('back');
  const { textRecognition } = useTextRecognizer({ script: 'latin' });

  const frameProcessor = useFrameProcessor((frame) => {
    'worklet';
    runAtTargetFps(5, () => {
      'worklet';
      runAsync(frame, () => {
        'worklet';
        const result = textRecognition(frame, {
          // iOS sensor is fixed-landscape; without this hint text arrives sideways.
          outputOrientation: 'portrait',
        });
        if (result.text.length > 0) {
          runOnJS(onText)(result.text);
        }
      });
    });
  }, [textRecognition, onText]);

  return <Camera style={StyleSheet.absoluteFill} device={device} isActive fps={30} frameProcessor={frameProcessor} />;
}

If you only need barcode scanning (QR, EAN, Code 128), the built-in codeScanner prop is significantly cheaper than ML Kit's barcodeScanning because it uses AVFoundation on iOS and the CameraX MLKit vision analyzer on Android, both of which are hardware-accelerated at the OS level. I cover the code scanner path in the React Native camera and barcode scanner guide; use ML Kit's barcode detector only if you specifically need pose or angle information out of the scan.

Writing a Nitro frame processor plugin

If a community plugin doesn't cover your model, you write your own. Vision Camera v5 dropped the old VisionCameraProxy.initFrameProcessorPlugin path in favour of Nitro Modules. That is the same tooling I described in the native modules comparison guide. You define a HybridObject spec in TypeScript and Nitrogen codegens the native binding. The Vision Camera GitHub repo keeps a working example under example/ that I recommend copying line-for-line the first time.

// specs/MyFrameProcessor.nitro.ts
import type { HybridObject } from 'react-native-nitro-modules';
import type { Frame } from 'react-native-vision-camera';

export interface MyPlugin extends HybridObject<{ ios: 'swift'; android: 'kotlin' }> {
  analyze(frame: Frame): { brightness: number; avgLuma: number };
}

Then in JS you wire it up with NitroModules.createHybridObject and call it from a worklet. The Nitro layer takes care of moving the frame reference across the runtime boundary without copying pixel data, which is critical when you are dealing with 4K frames at 60 FPS. My rule of thumb: if a community plugin exists (ML Kit, TensorFlow Lite, OpenCV bindings), use it. Writing your own plugin is a real week of native work, and I've written enough of them to know that.

Profiling and fixing dropped frames

The single most useful signal for a Vision Camera build is the ratio between camera capture rate and processed-frame rate. Vision Camera exposes this natively via onFrameProcessorPerformanceSuggestionAvailable, but I prefer to keep my own counters because the built-in suggestion only fires when the average is already in trouble. Software Mansion's deep dive on Vision Camera v5 multithreading is worth reading if you want to understand exactly what those counters are measuring.

const captured = useSharedValue(0);
const processed = useSharedValue(0);

const frameProcessor = useFrameProcessor((frame) => {
  'worklet';
  captured.value += 1;
  runAtTargetFps(12, () => {
    'worklet';
    runAsync(frame, () => {
      'worklet';
      detectFaces(frame);
      processed.value += 1;
    });
  });
}, [detectFaces]);

// On the JS side, sample every second:
useEffect(() => {
  const t = setInterval(() => {
    console.log('captured/s:', captured.value, 'processed/s:', processed.value);
    captured.value = 0; processed.value = 0;
  }, 1000);
  return () => clearInterval(t);
}, []);

If captured/s is below your fps prop, the pipeline is stalling, usually because a synchronous processor step is too heavy. If processed/s is far below your runAtTargetFps target, the async runtime is falling behind, which typically means the model is heavier than the throttle rate and you need to either drop the target FPS or move to performanceMode: 'fast'. This same discipline (measure before you tune) is the backbone of the broader React Native performance optimization playbook I maintain for the team.

Common causes of dropped frames I've hit

• Logging. A console.log from inside a worklet crosses runtimes; it costs ~2 ms per call. Strip them in production builds or wrap in a debug flag.
• Pixel format mismatch. Requesting rgb forces a colour space conversion. Use yuv (the sensor's native format) whenever the downstream plugin accepts it. Every ML Kit detector does.
• Wrong fps. Requesting 60 FPS on a device that can only sustain 30 with the current format causes the pipeline to renegotiate every few seconds. Check device.formats and pin an explicit format.
• Missing runAsync. The most common one. Any inference longer than 20 ms belongs on the async runtime.

Expo config plugin, permissions, and EAS

The Vision Camera config plugin injects NSCameraUsageDescription into Info.plist and android.permission.CAMERA into AndroidManifest.xml. If you also want the ML Kit models bundled at install time rather than downloaded on first use (which triggers a Play Services fetch and can fail on locked-down devices), add the Google Play Services metadata block via the plugin's enableGooglePlayServicesModels option.

{
  "plugins": [
    [
      "react-native-vision-camera",
      {
        "cameraPermissionText": "$(PRODUCT_NAME) needs the camera for scanning.",
        "enableCodeScanner": true
      }
    ],
    [
      "react-native-vision-camera-mlkit",
      { "enableGooglePlayServicesModels": ["face", "text"] }
    ]
  ]
}

Then EAS handles the rest. If you are shipping OTA updates alongside binary builds, note that Vision Camera version bumps are almost always native-code changes, so they need a new EAS build; you cannot ship them over EAS Update. That constraint is worth flagging in your release plan.

Frequently Asked Questions

Does React Native Vision Camera work with Expo Go?

No. Vision Camera includes native modules (AVFoundation, CameraX, and a Nitro Modules build step) that Expo Go cannot load. You need a development build via eas build --profile development or a bare React Native workflow. Once the dev client is installed on the device, npx expo start --dev-client gives you the same live-reload experience as Expo Go.

What's the difference between runAsync and runAtTargetFps?

runAtTargetFps(n, cb) throttles a callback on the camera thread itself, running it at most n times per second and dropping every other frame. runAsync(frame, cb) moves work off the camera thread to a separate async runtime, so the camera keeps streaming while heavy work runs in parallel. For ML inference you usually want both: throttle the invocation rate, then push the inference itself off-thread.

How fast can ML Kit face detection run in React Native?

On a 2024-class device (Pixel 9, iPhone 15 Pro) I measure 18–24 ms per frame for ML Kit face detection in performanceMode: 'fast' with landmarkMode: 'none' and contourMode: 'none'. That is fast enough to run every frame at 30 FPS if you push it through runAsync. Switching to performanceMode: 'accurate' costs roughly 2.5× more per frame, so throttle to 8–12 FPS with runAtTargetFps if you need it.

Why is my Vision Camera dropping frames on iOS but not Android?

Most often it is the pixel format: iOS defaults to yuv only when you ask for it explicitly, and Android's CameraX is more forgiving of colour space conversions. Set pixelFormat="yuv" on the <Camera> component, pin an explicit format from device.formats, and check that no frame processor is doing a synchronous per-pixel loop over the buffer. If OCR text is coming back rotated, add the outputOrientation hint on the iOS side.

Do I still need react-native-worklets-core with Vision Camera v5?

No. VisionCamera v5 uses the newer react-native-worklets package (auto-detected through its Nitro Modules integration). Install react-native-worklets and react-native-nitro-modules; you can remove react-native-worklets-core from your dependencies if you were using it under v4.