1. Swift
  2. On-Device AI Scanning (OCR)

Swift

On-Device AI Scanning (OCR)

The Vision SDK allows for on-device AI scanning (OCR), enabling offline extraction of structured information from documents such as shipping labels. This is ideal when low latency and offline functionality are critical, such as in warehouse and logistics environments.

New APIs(OCR) (Recommended)

🛠️ Step 1: Downloading On-Device OCR

Before using the on-device OCR, you must downlaod the model using the following methods. This ensures the necessary AI models are downloaded.

        
OnDeviceOCRManager.shared.downloadModel(
    withApiKey: "YOUR_API_KEY", // or use `token`
    andToken: nil,
    forModelClass: .shippingLabel,
    withModelSize: .large,
    withProgressTracking: { currentProgress, totalSize in
        print("Download progress: \(currentProgress)/\(totalSize)")
    },
    withCompletion: { error in
        if let error = error {
            print("Model download failed: \(error.localizedDescription)")
        } else {
            print("Model downloaded successfully.")
        }
    }
)

⚠️ You must wait for the completion block to succeed before proceeding to model loading.

🛠️ Step 2: Loading On-Device OCR

Before using the on-device OCR, you must load the model using the following methods. This ensures the necessary AI models are loaded and are ready for use.

        
OnDeviceOCRManager.shared.loadModel(
    withApiKey: "YOUR_API_KEY", // or use `token`
    andToken: nil,
    forModelClass: .shippingLabel,
    withModelSize: .large,
    withCompletion: { error in
        if let error = error {
            print("Model loading failed: \(error.localizedDescription)")
        } else {
            print("Model loaded successfully.")
        }
    }
)

⚠️ You must wait for the completion block to succeed before proceeding to data extraction.

🧩 Selecting the Model Type

When preparing for on-device AI scanning, you can choose which document model you want to use by specifying the forModelClass parameter. This lets the SDK know which type of document you’re scanning so it can load the appropriate AI model.

Here are the supported options:

📦 For Shipping Labels

Use this to scan and extract structured data from shipping labels.

        forModelClass: .shippingLabel

📄 For Bill of Lading (BOL)

Use this to extract data from Bill of Lading documents.

        forModelClass: .billOfLading

🏷️ For Item Labels

Use this to process and extract structured details from product/item labels.

        forModelClass: .itemLabels

🏷️ For Document Classification

Use this to process and extract document type from a document image.

        forModelClass: .documentClassification

📝 Make sure the model is prepared successfully before scanning. Each model may vary in size and complexity depending on the document type.

🛠️ Model Management Methods

These methods can be use to handle different models.

Cancel Downloading

Use this method to cancel any in progress download.

        
    // modelClass: VSDKModelClass - Select required Model Class. Curr e.g .shippingLabel
    // modelSize: VSDKModelSize - Select the model size for the above selected Model Class. e.g .micro
    // completion callback is called when the cancel operation completes with or without any error.

    public func cancelDownload(_ modelClass: VSDKModelExternalClass, withModelSize modelSize: VSDKModelExternalSize = .large, withCompletion completion:((_ error: NSError?)->())?)

Model Updating

This method can be used to check for updates for a given model.

        
    // This method must be provided with `apiKey` or `token`.
    // modelClass: VSDKModelClass - Select required Model Class. Curr e.g .shippingLabel
    // modelSize: VSDKModelSize - Select the model size for the above selected Model Class. e.g .micro
    // progress callback returns with normalized progress between 0 - 1
    // completion callback is called when the download operation completes with or without any error.

    public func checkModelUpdate(withApiKey apiKey: String? = nil, andToken token: String? = nil, forModelClass modelClass: VSDKModelExternalClass, withModelSize modelSize: VSDKModelExternalSize = .large, withProgressTracking progress: ((_ currentProgress: Float, _ totalSize: Float)->())?, withCompletion completion:((_ error: NSError?)->())?)

Model Unloading

Use this model to unload any pre-loaded models from memory.

        
    // modelClass: VSDKModelClass - Select required Model Class. Curr e.g .shippingLabel
    // modelSize: VSDKModelSize - Select the model size for the above selected Model Class. e.g .micro

    public func unloadModel(_ modelClass: VSDKModelExternalClass, withModelSize modelSize: VSDKModelExternalSize = .large)

Model Deleting

This method deletes the pre-downloaded model from the disk. It automatically unloads the model if already loaded.

        
    // modelClass: VSDKModelClass - Select required Model Class. Curr e.g .shippingLabel
    // modelSize: VSDKModelSize - Select the model size for the above selected Model Class. e.g .micro

    public func deleteModel(_ modelClass: VSDKModelExternalClass, withModelSize modelSize: VSDKModelExternalSize = .large)

Model Loading Status Check

This method checks if the given model is loaded in the memory or not. Returns true or false.

        
    // modelClass: VSDKModelClass - Select required Model Class. Curr e.g .shippingLabel
    // modelSize: VSDKModelSize - Select the model size for the above selected Model Class. e.g .micro

    public func isModelLoaded(_ modelClass: VSDKModelExternalClass, withModelSize modelSize: VSDKModelExternalSize = .large) -> Bool

This method returns all models and their respective sizes loaded in the memory.

        
    public func getLoadedModels() -> [[String: String]]
    
    // returns array of dictionary with below sample
    //  [   
    //      {"class": "shipping_label", "size": "large"}
    //      {"class": "item_label", "size": "large"}
    //  ]

Model Downloaded Status Check

This method checks if the given model is has been downladed or not. Returns model information if existing model found or returns nil.

        
    // modelClass: VSDKModelClass - Select required Model Class. Curr e.g .shippingLabel
    // modelSize: VSDKModelSize - Select the model size for the above selected Model Class. e.g .micro

    public func getDownloadedModel(_ modelClass: VSDKModelExternalClass, withModelSize modelSize: VSDKModelExternalSize = .large) -> DownloadedModelData?
    
    // returns optional object of DownloadedModelData
    //
    //      public final class DownloadedModelData: NSObject, Codable {
    //
    //          @objc public var modelClass: String? = nil
    //          @objc public var modelSize: String? = nil
    //          @objc public var modelVersionId: String? = nil
    //          @objc public var modelVersion: String? = nil
    //      }
    //
}

This method returns all downloaded models and their respective data.

        
    public func getDownloadedModels() -> [DownloadedModelData]
    
    // returns array of DownloadedModelData 
    //
    //      public final class DownloadedModelData: NSObject, Codable {
    //
    //          @objc public var modelClass: String? = nil
    //          @objc public var modelSize: String? = nil
    //          @objc public var modelVersionId: String? = nil
    //          @objc public var modelVersion: String? = nil
    //      }
    //

🧠 Step 3: Extracting Data from Image

Once the model is downloaded and loaded successfully, use the following method to perform OCR on a given image.

            
OnDeviceOCRManager.shared.extractDataFromImageUsing(
    ciImage,
    withBarcodes: barcodesArray,
    checkImageSharpness: false,
    modelClass: .shippingLabel,
    withModelSize: .large,
) { data, error in
    if let error = error {
        print("OCR extraction failed: \(error.localizedDescription)")
    } else if let data = data {
        // Handle OCR Response (e.g., parse JSON)
        print("OCR Result: \(String(data: data, encoding: .utf8) ?? "")")
    }
}

Parameters:

  • ciImage: The CIImage you want to process.
  • barcodes: A list of DetectedCode values representing barcodes detected in the image (if any).
  • checkImageSharpness: A boolean whether VisionSDK should check if the given image is sharp enough.
  • modelClass: Model Class of the model you want to use for extraction. Make sure the model is already loaded
  • withModelSize: Model Size of the model you want to use for extraction. Make sure the model is already loaded
  • completion: A closure returning the extracted data or error.

🔁 The returned data follows the same structure as the PackageX Cloud OCR API response.

✅ Full Sample Flow

        
        VSDKConstants.apiKey = "your_api_key"
        VSDKConstants.apiEnvironment = .staging // or .dev, .qa, .staging, .production, .sandbox
        
        VSDKConstants.platform = .native   // or .native , .flutter , .reactNative
        
        
        OnDeviceOCRManager.shared.downloadModel(withApiKey: VSDKConstants.apiKey, forModelClass: .shippingLabel, withModelSize: .large) { currentProgress, totalSize in
            
            print("\(Int((currentProgress/totalSize) * 100))% Complete")
            
        } withCompletion: { error in
            
            if let error = error {
                // Handle downloading error
            }
            else {
                
                // Start Model Loading
            }
        }
        
        OnDeviceOCRManager.shared.cancelDownload(.shippingLabel, withModelSize: .large) { error in
            if let error = error {
                // Handle error
            }
            else {
                
                // Do your job
            }
        }
        
        OnDeviceOCRManager.shared.checkModelUpdate(withApiKey: VSDKConstants.apiKey, forModelClass: .shippingLabel, withModelSize: .large) { currentProgress, totalSize in
            
            print("\(Int((currentProgress/totalSize) * 100))% Complete")
            
        } withCompletion: { error in
            
            if let error = error {
                // Handle update error
            }
            else {
                
                // Model update complete
            }
        }
        
        OnDeviceOCRManager.shared.loadModel(withApiKey: VSDKConstants.apiKey, forModelClass: .shippingLabel, withModelSize: .large) { error in
            
            if let error = error {
                // Handle loading error
            }
            else {
                
                // You can now start data extraction
            }
            
        }
        
        OnDeviceOCRManager.shared.unloadModel(.shippingLabel, withModelSize: .large)
        
        OnDeviceOCRManager.shared.deleteModel(.shippingLabel, withModelSize: .large)
        
        
        // check if model is loaded or not
        // returns true or false
        
        let isModelLoaded = OnDeviceOCRManager.shared.isModelLoaded(.shippingLabel, withModelSize: .large)
        
        
        // get all loaded models
        // returns following output
        //  [
        //      {"class": "shipping_label", "size": "large"}
        //      {"class": "item_label", "size": "large"}
        //  ]
        
        let allLoadedModels: [[String : String]] = OnDeviceOCRManager.shared.getLoadedModels()
        
        
        
        // Get model details of sepecific downloaded model
        // returns optional object of DownloadedModelData
        //
        //      public final class DownloadedModelData: NSObject, Codable {
        //
        //          @objc public var modelClass: String? = nil
        //          @objc public var modelSize: String? = nil
        //          @objc public var modelVersionId: String? = nil
        //          @objc public var modelVersion: String? = nil
        //      }

        let modelDetails: VisionSDK.DownloadedModelData = OnDeviceOCRManager.shared.getDownloadedModel(.shippingLabel, withModelSize: .large)
        
        
        // Get all downloaded models
        let allDownloadedModels: [VisionSDK.DownloadedModelData] = OnDeviceOCRManager.shared.getDownloadedModels()
        
        
        // ON-Device Data Extraction example
        
        let image = UIImage(named: "shippingLabelSample")
        let ciImage = CIImage(image: image!)!
        
        OnDeviceOCRManager.shared.extractDataFromImageUsing(ciImage, withBarcodes: [], modelClass: .shippingLabel, withModelSize: .large) { data, error in
            
            if let error = error {
                // Handle loading error
            }
            else {
                
                // check for data and parse it
            }
        }

Legacy Support(OCR) (NOT Recommended for new users)

🛠️ Step 1: Preparing On-Device OCR

Before using the on-device OCR, you must prepare the model using the following methods. This ensures the necessary AI models are downloaded and ready to use.

Option 1: Auto Model Size Configuration

If you want Vision SDK to automatically determine the best model size based on your PackageX subscription:

        OnDeviceOCRManager.shared.prepareOfflineOCR(
    withApiKey: "YOUR_API_KEY", // or use `token`
    andToken: nil,
    forModelClass: .shippingLabel, // Only `.shippingLabel` is supported currently
    withProgressTracking: { currentProgress, totalSize, isModelAlreadyDownloaded in
        print("Download progress: \(currentProgress)/\(totalSize)")
    },
    withCompletion: { error in
        if let error = error {
            print("Model preparation failed: \(error.localizedDescription)")
        } else {
            print("Model prepared successfully.")
        }
    }
)

Option 2: Explicit Model Size Configuration

You can also specify the model size manually using .micro (faster, smaller) or .large (slower, more accurate):

        OnDeviceOCRManager.shared.prepareOfflineOCR(
    withApiKey: "YOUR_API_KEY",
    andToken: nil,
    forModelClass: .shippingLabel,
    withModelSize: .micro, // or `.large`
    withProgressTracking: { currentProgress, totalSize, isModelAlreadyDownloaded in
        print("Download progress: \(currentProgress)/\(totalSize)")
    },
    withCompletion: { error in
        if let error = error {
            print("Model preparation failed: \(error.localizedDescription)")
        } else {
            print("Model prepared successfully.")
        }
    }
)

⚠️ You must wait for the completion block to succeed before proceeding to OCR extraction.

🧩 Selecting the Model Type

When preparing for on-device AI scanning, you can choose which document model you want to use by specifying the forModelClass parameter. This lets the SDK know which type of document you’re scanning so it can load the appropriate AI model.

Here are the supported options:

📦 For Shipping Labels

Use this to scan and extract structured data from shipping labels.

        forModelClass: .shippingLabel

📄 For Bill of Lading (BOL)

Use this to extract data from Bill of Lading documents.

        forModelClass: .billOfLading

🏷️ For Item Labels

Use this to process and extract structured details from product/item labels.

        forModelClass: .itemLabels

🏷️ For Document Classification

Use this to process and extract document type from a document image.

        forModelClass: .documentClassification

📝 Make sure the model is prepared successfully before scanning. Each model may vary in size and complexity depending on the document type.

🧠 Step 2: Extracting Data from Image

Once the model is prepared successfully, use the following method to perform OCR on a given image.

        OnDeviceOCRManager.shared.extractDataFromImage(
    ciImage,
    withBarcodes: barcodesArray
) { data, error in
    if let error = error {
        print("OCR extraction failed: \(error.localizedDescription)")
    } else if let data = data {
        // Handle OCR Response (e.g., parse JSON)
        print("OCR Result: \(String(data: data, encoding: .utf8) ?? "")")
    }
}

Parameters:

  • ciImage: The CIImage you want to process.
  • barcodes: A list of String values representing barcodes detected in the image (if any).
  • completion: A closure returning the extracted data or error.

🔁 The returned data follows the same structure as the PackageX Cloud OCR API response.

✅ Full Sample Flow

        // Step 1: Prepare model
OnDeviceOCRManager.shared.prepareOfflineOCR(
    withApiKey: "YOUR_API_KEY",
    forModelClass: .shippingLabel,
    withCompletion: { error in
        guard error == nil else {
            print("Error preparing model: \(error!.localizedDescription)")
            return
        }

        // Step 2: Perform OCR after successful preparation
        let ciImage = CIImage(image: yourUIImage)!
        let barcodes = ["123456789012"] // Optional if barcodes detected

        OnDeviceOCRManager.shared.extractDataFromImage(ciImage, withBarcodes: barcodes) { data, error in
            if let data = data {
                print("Extracted Data: \(String(data: data, encoding: .utf8) ?? "")")
            } else if let error = error {
                print("OCR failed: \(error.localizedDescription)")
            }
        }
    }
)
Previous <- AI Scanning APIs
Next Error Reporting ->