Swift

On-Device AI Scanning (OCR)

The Vision SDK allows for on-device AI scanning (OCR), enabling offline extraction of structured information from documents such as shipping labels. This is ideal when low latency and offline functionality are critical, such as in warehouse and logistics environments.

🛠️ Step 1: Preparing On-Device OCR

Before using the on-device OCR, you must prepare the model using the following methods. This ensures the necessary AI models are downloaded and ready to use.

Option 1: Auto Model Size Configuration

If you want Vision SDK to automatically determine the best model size based on your PackageX subscription:

        OnDeviceOCRManager.shared.prepareOfflineOCR(
    withApiKey: "YOUR_API_KEY", // or use `token`
    andToken: nil,
    forModelClass: .shippingLabel, // Only `.shippingLabel` is supported currently
    withProgressTracking: { currentProgress, totalSize, isModelAlreadyDownloaded in
        print("Download progress: \(currentProgress)/\(totalSize)")
    },
    withCompletion: { error in
        if let error = error {
            print("Model preparation failed: \(error.localizedDescription)")
        } else {
            print("Model prepared successfully.")
        }
    }
)

Option 2: Explicit Model Size Configuration

You can also specify the model size manually using .micro (faster, smaller) or .large (slower, more accurate):

        OnDeviceOCRManager.shared.prepareOfflineOCR(
    withApiKey: "YOUR_API_KEY",
    andToken: nil,
    forModelClass: .shippingLabel,
    withModelSize: .micro, // or `.large`
    withProgressTracking: { currentProgress, totalSize, isModelAlreadyDownloaded in
        print("Download progress: \(currentProgress)/\(totalSize)")
    },
    withCompletion: { error in
        if let error = error {
            print("Model preparation failed: \(error.localizedDescription)")
        } else {
            print("Model prepared successfully.")
        }
    }
)

⚠️ You must wait for the completion block to succeed before proceeding to OCR extraction.

🧩 Selecting the Model Type

When preparing for on-device AI scanning, you can choose which document model you want to use by specifying the forModelClass parameter. This lets the SDK know which type of document you’re scanning so it can load the appropriate AI model.

Here are the supported options:

📦 For Shipping Labels

Use this to scan and extract structured data from shipping labels.

        forModelClass: .shippingLabel

📄 For Bill of Lading (BOL)

Use this to extract data from Bill of Lading documents.

        forModelClass: .billOfLading

🏷️ For Item Labels

Use this to process and extract structured details from product/item labels.

        forModelClass: .itemLabels

🏷️ For Document Classification

Use this to process and extract document type from a document image.

        forModelClass: .documentClassification

📝 Make sure the model is prepared successfully before scanning. Each model may vary in size and complexity depending on the document type.

🧠 Step 2: Extracting Data from Image

Once the model is prepared successfully, use the following method to perform OCR on a given image.

        OnDeviceOCRManager.shared.extractDataFromImage(
    ciImage,
    withBarcodes: barcodesArray
) { data, error in
    if let error = error {
        print("OCR extraction failed: \(error.localizedDescription)")
    } else if let data = data {
        // Handle OCR Response (e.g., parse JSON)
        print("OCR Result: \(String(data: data, encoding: .utf8) ?? "")")
    }
}

Parameters:

ciImage: The CIImage you want to process.
barcodes: A list of String values representing barcodes detected in the image (if any).
completion: A closure returning the extracted data or error.

🔁 The returned data follows the same structure as the PackageX Cloud OCR API response.

✅ Full Sample Flow

        // Step 1: Prepare model
OnDeviceOCRManager.shared.prepareOfflineOCR(
    withApiKey: "YOUR_API_KEY",
    forModelClass: .shippingLabel,
    withCompletion: { error in
        guard error == nil else {
            print("Error preparing model: \(error!.localizedDescription)")
            return
        }

        // Step 2: Perform OCR after successful preparation
        let ciImage = CIImage(image: yourUIImage)!
        let barcodes = ["123456789012"] // Optional if barcodes detected

        OnDeviceOCRManager.shared.extractDataFromImage(ciImage, withBarcodes: barcodes) { data, error in
            if let data = data {
                print("Extracted Data: \(String(data: data, encoding: .utf8) ?? "")")
            } else if let error = error {
                print("OCR failed: \(error.localizedDescription)")
            }
        }
    }
)

On-Device AI Scanning (OCR)

# 🛠️ Step 1: Preparing On-Device OCR

# Option 1: Auto Model Size Configuration

# Option 2: Explicit Model Size Configuration

# 🧩 Selecting the Model Type

# 📦 For Shipping Labels

# 📄 For Bill of Lading (BOL)

# 🏷️ For Item Labels

# 🏷️ For Document Classification

# 🧠 Step 2: Extracting Data from Image

# Parameters:

# ✅ Full Sample Flow

On this page