In this getting started guide, we will show you how to use the FormX pre-built Receipt Extractor to create an iOS app that can assist users in capturing Receipt images and extract with FormX.ai

Installation

You can install FormX SDK using CocoaPods or Swift Package Manager

CocoaPods

Add the FormX pods to your Podfile:

  pod 'FormX', :git => 'https://github.com/oursky/formx-sdk.git', tag: '0.1.16'

Swift Package Manager

Add the FormX packages to your Package.swift:

    dependencies: [
        .package(url: "https://github.com/oursky/formx-sdk", from: "0.1.16")
    ],

Configure SDK

First, you will need an Access Token and the Form ID of a Pre-Built Receipt Extractor from the FormX portal.

Access Tokens:
1. Open 'Access Token' tab of 'Manage Team' page on FormX Portal.
2. Click 'Create Token' button
3. Enter a name for the token and create it.
4. Copy the access token and proceed with SDK configuration
Form ID:
1. Click 'Create new extractor' button of 'Extractors' page on FormX Portal.
2. Select 'Receipts' in the list of built-in extractors.
3. Enter a name for the extractor and create it.
4. Open 'Extract' tab in your extractor detail page to obtain the Form ID.

You can then configure the FormX SDK using the access token:

import FormX

// Obtain access token from FormX portal
let apiClient = FormXAPIClient(accessToken: "PUT_ACCESS_TOKEN_HERE")

Capture & Extract Documents (Offline Mode)

The FormX can show a camera view to help the user captures a document using offline ML models. In offline mode, the camera view will highlight the detected document, and provide indiciator on the document image quality, such as unsteady camera and blurry image.

// ViewController.swift
import FormX

class ViewController: UIViewController, FormXCameraViewDelegate {
  let cameraView = FormXCameraView()
  
  override func viewDidLoad() {
    // ...
    self.view.addSubview(self.cameraView)
    self.cameraView.topAnchor.constraint(equalTo: self.view.topAnchor).isActive = true
    self.cameraView.bottomAnchor.constraint(equalTo: self.view.bottomAnchor).isActive = true
    self.cameraView.leadingAnchor.constraint(equalTo: self.view.leadingAnchor).isActive = true
    self.cameraView.trailingAnchor.constraint(equalTo: self.view.trailingAnchor).isActive = true
    self.cameraView.delegate = self
  }
  
  override func viewWillAppear(_ animated: Bool) {
    // ...
    self.cameraView.start()
  }
  
  override func viewDidDisappear(_ animated: Bool) {
    // ...
    self.cameraView.stop()
  }
    
  func formXCameraView(didFailed error: Error) {
    // Handle error.
  }
    
  func formXCameraView(didChangeState view: FormXCameraView) {
    // Capture the document image automatically when document is detected.
    if view.state == .ready {
      view.capture()
    }
  }
    
  func formXCameraView(didCapture images: [CGImage]) {
    guard let image = images.first.map({ UIImage(cgImage: $0) }) else {
      // Handle error.
      return
    }
    guard let imageJpeg = image.jpegData(compressionQuality: 0.9) else {
      // Handle error.
      return
    }

    // Extract fields from the captured image data.
    apiClient.extract(formId: "PUT_FORM_ID_HERE", data: imageJpeg) { result, error in
      if let error = error {
        // Handle error.
        return
      }
        
      guard let response = result?.response else {
        return
      }
      self.handleExtractedDocument(response)
    }
  }
  
  func handleExtractedDocument(_ response: FormXAPIExtractResponse) {
    let items = response.autoExtractionItems
    
    var lines: [String] = []
    for item in items {
      // Handle each detected item field.
      switch item {
      case .intValue(let field):
        lines.append("\(field.name): \(field.value)")
      case .stringValue(let field):
        lines.append("\(field.name): \(field.value)")
      case .purchaseInfoValue(let field):
        // Handle each purchased items.
        lines.append("purchase items:")
        for purchase in field.value {
          lines.append("  \(purchase.name): \(purchase.amount)")
        }
      default:
        print(item)
      }
    }
    self.showAlert(title: "Document fields", message: lines.joined(separator: "\n"))
  }
    
  func formXCameraView(didRequestClose view: FormXCameraView) {
    self.dismiss(animated: true)
  }
}

// App.swift
class MyApp {
...
  func applicationDidEnterBackground(_ application: UIApplication) {
    // Note: This is sample code optimized for brevity
    // For production code, you likely need more involved but robust ways to
    // tell each camera view to stop
    viewController.cameraView.stop()
  }
...
}

To configure the camera view UI (e.g. look & feels, UI elements), you may pass a FormXCameraViewConfiguration to the camera view. Please refer to the SDK API reference for details.

Capture & Extract Documents (Online Mode)

Alternatively to using an offline ML model to detect documents, you may use FormX's online document detection APIs for better accuracy on splitting an image into one or multiple documents.

To enable online mode, change the mode of camera view when setting up:

func viewDidLoad() {
  // ...
  self.view.addSubview(self.cameraView)
  self.cameraView.mode = .online(api: apiClient)
  self.cameraView.delegate = self
}

Interpreting Extracted Fields

The extraction result is stored in FormXAPIExtractResponse . Each extracted field has a name and value. For detailed schema, please refer to the form fields selected on FormX portal.

Resource Management

The various FormX SDK camera views needs to keep processing frames from device camera until it is stopped explicitly. Since frame processing is resource-heavy, failing to stop the camera view appropriately might result in memory leak or degraded system responsiveness.

It is very important to call cameraView.stop when your app no longer needs it. In general, it's advised that you call cameraView.stop in view controller's lifecycle hooks. In the above offline mode sample code, you can see cameraView.stop is called in viewWillDisappear.

Background Handling

When your app is put into background, it should stop all FormX SDK camera views:

func applicationDidEnterBackground(_ application: UIApplication) {
  // Note: This is sample code optimized for brevity
  // For production code, you likely need more involved but robust ways to
  // tell each camera view to stop
  myViewController.cameraView.stop()
}

API Reference

https://oursky.github.io/formx-sdk-documentation/ios/