1. React Native
  2. Headless OCR Prediction

React Native

Headless OCR Prediction

The Vision SDK provides camera-independent OCR prediction methods through the VisionCore module that allow you to process existing images without requiring the camera view. These methods are ideal for workflows where you already have images and need to extract structured data from them.

INFO

For complete VisionCore API reference including setEnvironment(), loadModel(), and onModelDownloadProgress(), see the VisionCore Module Documentation.

🔧 Available Prediction Methods

1. On-Device Prediction

VisionCore.predict(imagePath, barcodes)

Performs on-device OCR prediction on an existing image using the configured on-device model.

        import { VisionCore } from 'react-native-vision-sdk'
import { useEffect } from 'react'

const HeadlessOCRExample = () => {
    useEffect(() => {
        // Set the environment for VisionCore
        VisionCore.setEnvironment('prod'); // or 'staging', 'dev', etc.
    }, []);

    const handleOnDevicePrediction = async () => {
        try {
            // First load the on-device model
            await VisionCore.loadModel(
                null, // token
                'your_api_key',
                'shipping_label', // model type
                'large' // model size
            );

            // Process an existing image
            const imageUri = 'file:///path/to/your/image.jpg';
            const barcodes = []; // Optional: include any detected barcodes

            const result = await VisionCore.predict(imageUri, barcodes);
            console.log('On-device prediction result:', result);
        } catch (error) {
            console.error('Prediction failed:', error);
        }
    };

    // You can also monitor model download progress
    useEffect(() => {
        const subscription = VisionCore.onModelDownloadProgress((progress, downloadStatus, isReady) => {
            console.log(`Download progress: ${progress * 100}%`);
            console.log('Model ready:', isReady);
        });

        return () => subscription.remove();
    }, []);

    return (
        // Your UI components here
        <TouchableOpacity onPress={handleOnDevicePrediction}>
            <Text>Predict with On-Device Model</Text>
        </TouchableOpacity>
    );
};

2. Cloud Prediction Methods

VisionCore.predictShippingLabelCloud(imagePath, barcodes, options)

Processes shipping label images using cloud-based OCR with optional barcode data.

        const handleShippingLabelPrediction = async () => {
    try {
        const imageUri = 'file:///path/to/shipping-label.jpg';
        const barcodes = ['1Z999AA1234567890']; // Optional

        const options = {
            token: null,
            apiKey: 'your_api_key',
            locationId: 'warehouse_001',
            metadata: { orderId: 12345, operator: 'john.doe' },
            recipient: { name: 'John Doe', address: '123 Main St' },
            sender: { name: 'Jane Smith', address: '456 Oak St' },
            shouldResizeImage: true
        };

        const result = await VisionCore.predictShippingLabelCloud(imageUri, barcodes, options);
        console.log('Shipping label result:', result);
    } catch (error) {
        console.error('Shipping label prediction failed:', error);
    }
};

VisionCore.predictItemLabelCloud(imagePath, options)

Processes item label images for SKU, price, and product information extraction.

        const handleItemLabelPrediction = async () => {
    try {
        const imageUri = 'file:///path/to/item-label.jpg';

        const options = {
            token: null,
            apiKey: 'your_api_key',
            shouldResizeImage: true
        };

        const result = await VisionCore.predictItemLabelCloud(imageUri, options);
        console.log('Item label result:', result);
    } catch (error) {
        console.error('Item label prediction failed:', error);
    }
};

VisionCore.predictBillOfLadingCloud(imagePath, barcodes, options)

Processes bill of lading documents for carrier and shipment information.

        const handleBOLPrediction = async () => {
    try {
        const imageUri = 'file:///path/to/bol.jpg';
        const barcodes = ['BOL123456789']; // Optional

        const options = {
            token: null,
            apiKey: 'your_api_key',
            locationId: 'dock_A',
            shouldResizeImage: true
        };

        const result = await VisionCore.predictBillOfLadingCloud(imageUri, barcodes, options);
        console.log('BOL result:', result);
    } catch (error) {
        console.error('BOL prediction failed:', error);
    }
};

VisionCore.predictDocumentClassificationCloud(imagePath, options)

Classifies and extracts data from various document types.

        const handleDocumentClassification = async () => {
    try {
        const imageUri = 'file:///path/to/document.jpg';

        const options = {
            token: null,
            apiKey: 'your_api_key',
            shouldResizeImage: true
        };

        const result = await VisionCore.predictDocumentClassificationCloud(imageUri, options);
        console.log('Document classification result:', result);
    } catch (error) {
        console.error('Document classification failed:', error);
    }
};

VisionCore.predictWithCloudTransformations(imagePath, barcodes, options)

Combines on-device processing with cloud-based transformations for enhanced accuracy.

        const handleHybridPrediction = async () => {
    try {
        const imageUri = 'file:///path/to/document.jpg';
        const barcodes = ['123456789']; // Optional

        const options = {
            token: null,
            apiKey: 'your_api_key',
            locationId: 'facility_B',
            metadata: { processType: 'hybrid', orderId: 12345 },
            recipient: { name: 'John Doe' },
            sender: { name: 'Jane Smith' },
            shouldResizeImage: true
        };

        const result = await VisionCore.predictWithCloudTransformations(imageUri, barcodes, options);
        console.log('Hybrid prediction result:', result);
    } catch (error) {
        console.error('Hybrid prediction failed:', error);
    }
};

📱 Complete Example

Here's a complete example showing how to implement headless OCR prediction using VisionCore:

        import React, { useState, useEffect } from 'react';
import { View, TouchableOpacity, Text, Alert } from 'react-native';
import { VisionCore } from 'react-native-vision-sdk';
import { launchImageLibrary } from 'react-native-image-picker';

const HeadlessOCRScreen = () => {
    const [isProcessing, setIsProcessing] = useState(false);
    const [isModelReady, setIsModelReady] = useState(false);
    const [modelProgress, setModelProgress] = useState(0);

    useEffect(() => {
        // Initialize VisionCore environment
        VisionCore.setEnvironment('prod');

        // Monitor model download progress
        const subscription = VisionCore.onModelDownloadProgress((progress, downloadStatus, isReady) => {
            setModelProgress(progress * 100);
            setIsModelReady(isReady);
        });

        return () => subscription.remove();
    }, []);

    const selectImageAndPredict = () => {
        launchImageLibrary(
            {
                mediaType: 'photo',
                quality: 0.8,
            },
            async (response) => {
                if (response.assets && response.assets[0]) {
                    const imageUri = response.assets[0].uri;
                    setIsProcessing(true);

                    try {
                        // Use cloud prediction for shipping labels
                        const options = {
                            token: null,
                            apiKey: 'your_api_key',
                            locationId: 'mobile_app',
                            metadata: { source: 'gallery_selection', timestamp: Date.now() },
                            shouldResizeImage: true
                        };

                        const result = await VisionCore.predictShippingLabelCloud(imageUri, [], options);

                        // Display results to user
                        Alert.alert(
                            'OCR Result',
                            `Extracted data: ${JSON.stringify(result, null, 2)}`
                        );
                    } catch (error) {
                        console.error('OCR Error:', error);
                        Alert.alert('Error', 'Failed to process image');
                    } finally {
                        setIsProcessing(false);
                    }
                }
            }
        );
    };

    const selectImageForOnDevicePredict = async () => {
        launchImageLibrary(
            {
                mediaType: 'photo',
                quality: 0.8,
            },
            async (response) => {
                if (response.assets && response.assets[0]) {
                    const imageUri = response.assets[0].uri;
                    setIsProcessing(true);

                    try {
                        // First ensure model is loaded
                        if (!isModelReady) {
                            await VisionCore.loadModel(
                                null,
                                'your_api_key',
                                'shipping_label',
                                'large'
                            );
                        }

                        // Perform on-device prediction
                        const result = await VisionCore.predict(imageUri, []);

                        Alert.alert(
                            'On-Device OCR Result',
                            `Extracted data: ${JSON.stringify(result, null, 2)}`
                        );
                    } catch (error) {
                        console.error('On-device OCR Error:', error);
                        Alert.alert('Error', 'Failed to process image on-device');
                    } finally {
                        setIsProcessing(false);
                    }
                }
            }
        );
    };

    return (
        <View style={{ flex: 1, padding: 20 }}>
            {!isModelReady && modelProgress > 0 && (
                <Text style={{ marginBottom: 20, textAlign: 'center' }}>
                    Model Download: {modelProgress.toFixed(0)}%
                </Text>
            )}

            <TouchableOpacity
                onPress={selectImageAndPredict}
                disabled={isProcessing}
                style={{
                    backgroundColor: isProcessing ? '#ccc' : '#007AFF',
                    padding: 15,
                    borderRadius: 8,
                    alignItems: 'center',
                    marginBottom: 15
                }}
            >
                <Text style={{ color: 'white', fontSize: 16 }}>
                    {isProcessing ? 'Processing...' : 'Cloud Prediction'}
                </Text>
            </TouchableOpacity>

            <TouchableOpacity
                onPress={selectImageForOnDevicePredict}
                disabled={isProcessing || !isModelReady}
                style={{
                    backgroundColor: (isProcessing || !isModelReady) ? '#ccc' : '#28a745',
                    padding: 15,
                    borderRadius: 8,
                    alignItems: 'center'
                }}
            >
                <Text style={{ color: 'white', fontSize: 16 }}>
                    {isProcessing ? 'Processing...' :
                     !isModelReady ? 'Model Loading...' : 'On-Device Prediction'}
                </Text>
            </TouchableOpacity>
        </View>
    );
};

export default HeadlessOCRScreen;

📝 Key Benefits

  1. Camera-Independent: Process existing images without requiring camera access
  2. Flexible Integration: Integrate OCR into any app workflow
  3. Fast Processing: On-device models provide quick results
  4. Cloud Accuracy: Cloud methods offer enhanced accuracy for complex documents
  5. Metadata Support: Pass custom metadata for enhanced processing context

🔍 Use Cases

  • Gallery Processing: Allow users to select images from their photo library
  • Batch Processing: Process multiple pre-captured images
  • Server Integration: Process images received from servers or APIs
  • Offline Analysis: Use on-device prediction when network is unavailable
  • Quality Control: Re-process images that failed initial camera capture
NOTE

Make sure to configure the appropriate ocrMode and ocrType props based on your chosen prediction method. On-device methods require the model to be downloaded first using configureOnDeviceModel.

Previous <- Error Reporting
Next Logging APIs ->