1. React Native
  2. Headless OCR Prediction

React Native

Headless OCR Prediction

The Vision SDK provides camera-independent OCR prediction methods through the VisionCore module that allow you to process existing images without requiring the camera view. These methods are ideal for workflows where you already have images and need to extract structured data from them.

INFO

For complete VisionCore API reference including setEnvironment() and model management methods, see the VisionCore Module Documentation and Model Management API.

Available Prediction Methods

1. On-Device Prediction

Use the Model Management API for on-device predictions with explicit model selection.

        import { VisionCore } from 'react-native-vision-sdk'
import { useEffect, useState } from 'react'

const HeadlessOCRExample = () => {
    const [isModelLoaded, setIsModelLoaded] = useState(false);

    useEffect(() => {
        // Set the environment for VisionCore
        VisionCore.setEnvironment('prod');
    }, []);

    const handleOnDevicePrediction = async () => {
        try {
            const module = { type: 'shipping_label', size: 'large' };

            // Initialize model manager (Android only, iOS is no-op)
            VisionCore.initializeModelManager({ maxConcurrentDownloads: 2 });

            // Download and load the model
            await VisionCore.downloadModel(
                module,
                'your_api_key',
                null,
                (progress) => {
                    console.log(`Download: ${(progress.progress * 100).toFixed(1)}%`);
                }
            );

            await VisionCore.loadOCRModel(module, 'your_api_key', null);
            setIsModelLoaded(true);

            // Process an existing image with the specific model
            const imageUri = 'file:///path/to/your/image.jpg';
            const barcodes = []; // Optional: include any detected barcodes

            const result = await VisionCore.predictWithModule(module, imageUri, barcodes);
            console.log('On-device prediction result:', result);
        } catch (error) {
            console.error('Prediction failed:', error);
        }
    };

    return (
        // Your UI components here
        <TouchableOpacity onPress={handleOnDevicePrediction}>
            <Text>Predict with On-Device Model</Text>
        </TouchableOpacity>
    );
};

2. Cloud Prediction Methods

Cloud predictions don't require downloading models - they process images on PackageX servers for higher accuracy.

VisionCore.predictShippingLabelCloud(imagePath, barcodes, options)

Processes shipping label images using cloud-based OCR with optional barcode data.

        const handleShippingLabelPrediction = async () => {
    try {
        const imageUri = 'file:///path/to/shipping-label.jpg';
        const barcodes = ['1Z999AA1234567890']; // Optional

        const options = {
            token: null,
            apiKey: 'your_api_key',
            locationId: 'warehouse_001',
            metadata: { orderId: 12345, operator: 'john.doe' },
            recipient: { name: 'John Doe', address: '123 Main St' },
            sender: { name: 'Jane Smith', address: '456 Oak St' },
            shouldResizeImage: true
        };

        const result = await VisionCore.predictShippingLabelCloud(imageUri, barcodes, options);
        console.log('Shipping label result:', result);
    } catch (error) {
        console.error('Shipping label prediction failed:', error);
    }
};

VisionCore.predictItemLabelCloud(imagePath, options)

Processes item label images for SKU, price, and product information extraction.

        const handleItemLabelPrediction = async () => {
    try {
        const imageUri = 'file:///path/to/item-label.jpg';

        const options = {
            token: null,
            apiKey: 'your_api_key',
            shouldResizeImage: true
        };

        const result = await VisionCore.predictItemLabelCloud(imageUri, options);
        console.log('Item label result:', result);
    } catch (error) {
        console.error('Item label prediction failed:', error);
    }
};

VisionCore.predictBillOfLadingCloud(imagePath, barcodes, options)

Processes bill of lading documents for carrier and shipment information.

        const handleBOLPrediction = async () => {
    try {
        const imageUri = 'file:///path/to/bol.jpg';
        const barcodes = ['BOL123456789']; // Optional

        const options = {
            token: null,
            apiKey: 'your_api_key',
            locationId: 'dock_A',
            shouldResizeImage: true
        };

        const result = await VisionCore.predictBillOfLadingCloud(imageUri, barcodes, options);
        console.log('BOL result:', result);
    } catch (error) {
        console.error('BOL prediction failed:', error);
    }
};

VisionCore.predictDocumentClassificationCloud(imagePath, options)

Classifies and extracts data from various document types.

        const handleDocumentClassification = async () => {
    try {
        const imageUri = 'file:///path/to/document.jpg';

        const options = {
            token: null,
            apiKey: 'your_api_key',
            shouldResizeImage: true
        };

        const result = await VisionCore.predictDocumentClassificationCloud(imageUri, options);
        console.log('Document classification result:', result);
    } catch (error) {
        console.error('Document classification failed:', error);
    }
};

VisionCore.predictWithCloudTransformations(imagePath, barcodes, options)

Combines on-device processing with cloud-based transformations for enhanced accuracy.

        const handleHybridPrediction = async () => {
    try {
        const imageUri = 'file:///path/to/document.jpg';
        const barcodes = ['123456789']; // Optional

        const options = {
            token: null,
            apiKey: 'your_api_key',
            locationId: 'facility_B',
            metadata: { processType: 'hybrid', orderId: 12345 },
            recipient: { name: 'John Doe' },
            sender: { name: 'Jane Smith' },
            shouldResizeImage: true
        };

        const result = await VisionCore.predictWithCloudTransformations(imageUri, barcodes, options);
        console.log('Hybrid prediction result:', result);
    } catch (error) {
        console.error('Hybrid prediction failed:', error);
    }
};

Complete Example

Here's a complete example showing how to implement headless OCR prediction using VisionCore with the Model Management API:

        import React, { useState, useEffect } from 'react';
import { View, TouchableOpacity, Text, Alert } from 'react-native';
import { VisionCore } from 'react-native-vision-sdk';
import { launchImageLibrary } from 'react-native-image-picker';

const HeadlessOCRScreen = () => {
    const [isProcessing, setIsProcessing] = useState(false);
    const [isModelReady, setIsModelReady] = useState(false);
    const [downloadProgress, setDownloadProgress] = useState(0);

    const module = { type: 'shipping_label', size: 'large' };

    useEffect(() => {
        // Initialize VisionCore environment
        VisionCore.setEnvironment('prod');

        // Initialize model manager (required on Android)
        VisionCore.initializeModelManager({ maxConcurrentDownloads: 2 });
    }, []);

    const loadModel = async () => {
        try {
            // Download model with progress tracking
            await VisionCore.downloadModel(
                module,
                'your_api_key',
                null,
                (progress) => {
                    setDownloadProgress(progress.progress * 100);
                }
            );

            // Load into memory
            await VisionCore.loadOCRModel(module, 'your_api_key', null);
            setIsModelReady(true);
            Alert.alert('Success', 'Model loaded and ready!');
        } catch (error) {
            console.error('Failed to load model:', error);
            Alert.alert('Error', `Failed to load model: ${error.message}`);
        }
    };

    const selectImageForCloudPrediction = () => {
        launchImageLibrary(
            {
                mediaType: 'photo',
                quality: 0.8,
            },
            async (response) => {
                if (response.assets && response.assets[0]) {
                    const imageUri = response.assets[0].uri;
                    setIsProcessing(true);

                    try {
                        // Use cloud prediction for shipping labels
                        const options = {
                            token: null,
                            apiKey: 'your_api_key',
                            locationId: 'mobile_app',
                            metadata: { source: 'gallery_selection', timestamp: Date.now() },
                            shouldResizeImage: true
                        };

                        const result = await VisionCore.predictShippingLabelCloud(imageUri, [], options);

                        // Display results to user
                        Alert.alert(
                            'Cloud OCR Result',
                            `Extracted data: ${JSON.stringify(result, null, 2)}`
                        );
                    } catch (error) {
                        console.error('OCR Error:', error);
                        Alert.alert('Error', 'Failed to process image');
                    } finally {
                        setIsProcessing(false);
                    }
                }
            }
        );
    };

    const selectImageForOnDevicePrediction = async () => {
        if (!isModelReady) {
            Alert.alert('Model Not Ready', 'Please load the model first');
            return;
        }

        launchImageLibrary(
            {
                mediaType: 'photo',
                quality: 0.8,
            },
            async (response) => {
                if (response.assets && response.assets[0]) {
                    const imageUri = response.assets[0].uri;
                    setIsProcessing(true);

                    try {
                        // Perform on-device prediction with explicit model
                        const result = await VisionCore.predictWithModule(module, imageUri, []);

                        Alert.alert(
                            'On-Device OCR Result',
                            `Extracted data: ${JSON.stringify(result, null, 2)}`
                        );
                    } catch (error) {
                        console.error('On-device OCR Error:', error);
                        Alert.alert('Error', 'Failed to process image on-device');
                    } finally {
                        setIsProcessing(false);
                    }
                }
            }
        );
    };

    const unloadModel = async () => {
        try {
            const unloaded = await VisionCore.unloadModel(module);
            if (unloaded) {
                setIsModelReady(false);
                Alert.alert('Success', 'Model unloaded from memory');
            }
        } catch (error) {
            Alert.alert('Error', `Failed to unload: ${error.message}`);
        }
    };

    return (
        <View style={{ flex: 1, padding: 20 }}>
            <TouchableOpacity
                onPress={loadModel}
                disabled={isModelReady}
                style={{
                    backgroundColor: isModelReady ? '#ccc' : '#28a745',
                    padding: 15,
                    borderRadius: 8,
                    alignItems: 'center',
                    marginBottom: 15
                }}
            >
                <Text style={{ color: 'white', fontSize: 16 }}>
                    {isModelReady ? 'Model Loaded' : 'Load On-Device Model'}
                </Text>
            </TouchableOpacity>

            {downloadProgress > 0 && downloadProgress < 100 && (
                <Text style={{ marginBottom: 15, textAlign: 'center' }}>
                    Download Progress: {downloadProgress.toFixed(0)}%
                </Text>
            )}

            <TouchableOpacity
                onPress={selectImageForCloudPrediction}
                disabled={isProcessing}
                style={{
                    backgroundColor: isProcessing ? '#ccc' : '#007AFF',
                    padding: 15,
                    borderRadius: 8,
                    alignItems: 'center',
                    marginBottom: 15
                }}
            >
                <Text style={{ color: 'white', fontSize: 16 }}>
                    {isProcessing ? 'Processing...' : 'Cloud Prediction'}
                </Text>
            </TouchableOpacity>

            <TouchableOpacity
                onPress={selectImageForOnDevicePrediction}
                disabled={isProcessing || !isModelReady}
                style={{
                    backgroundColor: (isProcessing || !isModelReady) ? '#ccc' : '#17a2b8',
                    padding: 15,
                    borderRadius: 8,
                    alignItems: 'center',
                    marginBottom: 15
                }}
            >
                <Text style={{ color: 'white', fontSize: 16 }}>
                    {isProcessing ? 'Processing...' :
                     !isModelReady ? 'Load Model First' : 'On-Device Prediction'}
                </Text>
            </TouchableOpacity>

            {isModelReady && (
                <TouchableOpacity
                    onPress={unloadModel}
                    style={{
                        backgroundColor: '#dc3545',
                        padding: 15,
                        borderRadius: 8,
                        alignItems: 'center'
                    }}
                >
                    <Text style={{ color: 'white', fontSize: 16 }}>
                        Unload Model
                    </Text>
                </TouchableOpacity>
            )}
        </View>
    );
};

export default HeadlessOCRScreen;

Key Benefits

  1. Camera-Independent: Process existing images without requiring camera access
  2. Flexible Integration: Integrate OCR into any app workflow
  3. Fast Processing: On-device models provide quick results
  4. Cloud Accuracy: Cloud methods offer enhanced accuracy for complex documents
  5. Metadata Support: Pass custom metadata for enhanced processing context

Use Cases

  • Gallery Processing: Allow users to select images from their photo library
  • Batch Processing: Process multiple pre-captured images
  • Server Integration: Process images received from servers or APIs
  • Offline Analysis: Use on-device prediction when network is unavailable
  • Quality Control: Re-process images that failed initial camera capture

API Reference

On-Device Methods

Method Description
VisionCore.predictWithModule(module, imagePath, barcodes) On-device prediction with explicit model selection

Cloud Methods

Method Description
VisionCore.predictShippingLabelCloud(imagePath, barcodes, options) Cloud shipping label OCR
VisionCore.predictItemLabelCloud(imagePath, options) Cloud item label OCR
VisionCore.predictBillOfLadingCloud(imagePath, barcodes, options) Cloud bill of lading OCR
VisionCore.predictDocumentClassificationCloud(imagePath, options) Cloud document classification
VisionCore.predictWithCloudTransformations(imagePath, barcodes, options) Hybrid on-device + cloud OCR

Further Reading

Previous <- Camera Switching
Next Logging APIs ->