1. Flutter
  2. On-Device AI Scanning (OCR)

Flutter

On-Device AI Scanning (OCR)

The Vision SDK allows for on-device AI scanning (OCR), enabling offline extraction of structured information from documents such as shipping labels. This is ideal when low latency and offline functionality are critical, such as in warehouse and logistics environments.

🛠️ Step 1: Preparing On-Device OCR

Before using the on-device OCR, you must prepare the model using the following methods. This ensures the necessary AI models are downloaded and ready to use.

        class MyOnDeviceOCRManager {

  final OnDeviceOCRManagerCommunicator onDeviceOCRManagerCommunicator;

  MyOnDeviceOCRManager(OnDeviceOCRManagerListener listener):onDeviceOCRManagerCommunicator = OnDeviceOCRManagerCommunicator(listener);

  void configureOnDeviceOCR(
    {String? apiKey,
    String? token,
    required ModelClass modelClass,
    required ModelSize modelSize}) {
      onDeviceOCRManagerCommunicator.configureOnDeviceOCR(
        apiKey: apiKey,
        token: token,
        modelClass: modelClass,
        modelSize: modelSize,
      );
  }
}

class OnDevceListener implements OnDeviceOCRManagerListener {
  @override void onOnDeviceConfigurationComplete() {}
  @override void onOnDeviceConfigureProgress(double progress) {}
}

⚠️ You must wait for the configure process to complete before proceeding to OCR extraction.

🧩 Selecting the Model Type

When preparing for on-device AI scanning, you can choose which document model you want to use by specifying the modelClass parameter. This lets the SDK know which type of document you’re scanning so it can load the appropriate AI model.

Here are the supported options:

📦 For Shipping Labels

Use this to scan and extract structured data from shipping labels.

        modelClass: ModelClass.shippingLabel

📄 For Bill of Lading (BOL)

Use this to extract data from Bill of Lading documents.

        modelClass: ModelClass.billOfLading

🏷️ For Item Labels

Use this to process and extract structured details from product/item labels.

        modelClass: ModelClass.itemLabel

📝 Make sure the model is prepared successfully before scanning. Each model may vary in size and complexity depending on the document type.

📮 Address Parsing for Shipping Labels

Shipping label results can include parsed sender and recipient addresses, split into structured fields. This is off by default. Turn it on by passing ShippingLabelOptions when you configure:

        onDeviceOCRManagerCommunicator.configureOnDeviceOCR(
  apiKey: apiKey,
  modelClass: ModelClass.shippingLabel,
  modelSize: ModelSize.large,
  shippingLabelOptions: const ShippingLabelOptions(
    parseRecipientAddress: true,
    parseSenderAddress: true,
  ),
);

Parameters:

  • parseRecipientAddress: Parse the recipient address. Defaults to false.
  • parseSenderAddress: Parse the sender address. Defaults to false.

⚠️ Enabling either flag adds a network call per prediction, which increases latency and is billable. Leave both off unless you need the parsed fields.

Only ModelClass.shippingLabel uses these options; they are ignored for other model classes.

If you load models through the Model Management API instead of configureOnDeviceOCR(), pass your credentials and options to the prediction call instead:

        onDeviceOCRManagerCommunicator.makePredictionWithModule(
  modelClass: ModelClass.shippingLabel,
  modelSize: ModelSize.large,
  byteArrayImage: byteArrayImage,
  barcodes: barcodes,
  apiKey: apiKey,
  shippingLabelOptions: const ShippingLabelOptions(
    parseRecipientAddress: true,
  ),
);

🧠 Step 2: Extracting Data from Image

Once the model is prepared successfully, use the following method to perform OCR on a given image.

        class MyOnDeviceOCRManager {

  final OnDeviceOCRManagerCommunicator onDeviceOCRManagerCommunicator;

  MyOnDeviceOCRManager(OnDeviceOCRManagerListener listener):onDeviceOCRManagerCommunicator = OnDeviceOCRManagerCommunicator(listener);

  void getPredictions(Uint8List byteArrayImage, List<String> barcodes) {
    onDeviceOCRManagerCommunicator.getPredictions(byteArrayImage, barcodes);
  }
}

class OnDevceListener implements OnDeviceOCRManagerListener {
  @override void onOnDeviceOCRResult(Map<String, dynamic> result) {}
  @override void onError(String error) {}
}

Parameters:

  • byteArrayImage: The Uint8List byte array of image you want to process.
  • barcodes: A list of String values representing barcodes detected in the image (if any).

🔁 The returned data follows the same structure as the PackageX Cloud OCR API response.

♲ Step 3: Release resources

It is important that when you are done with predictions and you are not planning to get more predictions on-device, or you want to configure a different AI model, then you release the memory being held by AI model. You can do that by using following code:

        class MyOnDeviceOCRManager {

  final OnDeviceOCRManagerCommunicator onDeviceOCRManagerCommunicator;

  MyOnDeviceOCRManager(OnDeviceOCRManagerListener listener):onDeviceOCRManagerCommunicator = OnDeviceOCRManagerCommunicator(listener);

  void release() {
    onDeviceOCRManagerCommunicator.release();
  }
}

Available On-Device AI Scanning

Following table explains all the on-device AI scanning options available currently.

Model Class Nano Micro Small Medium Large XLarge
ShippingLabel X ✓ X X ✓ X
BillOfLading X X X X ✓ X
ItemLabel X X X X ✓ X
DocumentClassification X X X X ✓ X

✅ Full Sample Flow

        class MyOnDeviceOCRManager {

  final OnDeviceOCRManagerCommunicator onDeviceOCRManagerCommunicator;

  MyOnDeviceOCRManager(OnDeviceOCRManagerListener listener):onDeviceOCRManagerCommunicator = OnDeviceOCRManagerCommunicator(listener);

  void configureOnDeviceOCR(
    {String? apiKey,
    String? token,
    required ModelClass modelClass,
    required ModelSize modelSize}) {
      onDeviceOCRManagerCommunicator.configureOnDeviceOCR(
        apiKey: apiKey,
        token: token,
        modelClass: modelClass,
        modelSize: modelSize,
      );
  }

  void getPredictions(Uint8List byteArrayImage, List<String> barcodes) {
    onDeviceOCRManagerCommunicator.getPredictions(byteArrayImage, barcodes);
  }

  void release() {
    onDeviceOCRManagerCommunicator.release();
  }
}

class OnDevceListener implements OnDeviceOCRManagerListener {
  @override void onError(String error) {}
  @override void onOnDeviceConfigurationComplete() {}
  @override void onOnDeviceConfigureProgress(double progress) {}
  @override void onOnDeviceOCRResult(Map<String, dynamic> result) {}
}
Previous <- Model Management API Reference
Next Error Reporting ->