Human Review with Augmented AI (A2I)

No confidence threshold is right for every document. Some extractions are obviously correct, some are obviously wrong, and a meaningful share sit in between - good enough to not discard, not good enough to trust blindly.

Amazon Augmented AI (A2I) is AWS's answer to that middle band: a managed human-review workflow that Textract can route documents into automatically, based on rules you define ahead of time. The review itself happens outside Textract, but triggering it is a single parameter on a call you already make.

Recipe

Quick-reference recipe card - copy-paste ready.

# --- Python (boto3) ---
import boto3
 
textract = boto3.client("textract")
 
with open("invoice.png", "rb") as f:
    response = textract.analyze_document(
        Document={"Bytes": f.read()},
        FeatureTypes=["FORMS"],
        HumanLoopConfig={
            "HumanLoopName": "invoice-review-0001",
            "FlowDefinitionArn": "arn:aws:sagemaker:us-east-1:123456789012:flow-definition/invoice-review",
        },
    )

// --- TypeScript (AWS SDK v3) ---
import { TextractClient, AnalyzeDocumentCommand } from "@aws-sdk/client-textract";
import { readFileSync } from "node:fs";
 
const textract = new TextractClient({});
 
const response = await textract.send(new AnalyzeDocumentCommand({
  Document: { Bytes: readFileSync("invoice.png") },
  FeatureTypes: ["FORMS"],
  HumanLoopConfig: {
    HumanLoopName: "invoice-review-0001",
    FlowDefinitionArn: "arn:aws:sagemaker:us-east-1:123456789012:flow-definition/invoice-review",
  },
}));

When to reach for this:

• Extractions feed a consequential decision - payment, eligibility, compliance - where an outright error is costly.
• You want a defined confidence floor below which a human, not code, makes the final call.
• Volume is high enough that all-manual review isn't practical, but full automation isn't safe either.
• You already have (or are willing to set up) a private, vendor, or Mechanical Turk workforce to review flagged documents.

Working Example

Set up the prerequisite flow definition once via the SageMaker/A2I API, then pass its ARN into every AnalyzeDocument call that should be eligible for human review.

# --- Python (boto3) ---
import uuid
import boto3
 
sagemaker = boto3.client("sagemaker")
textract = boto3.client("textract")
 
# One-time setup: create the flow definition. HumanLoopRequestSource identifies the
# built-in Textract integration; HumanLoopActivationConditions sets the confidence rule.
flow = sagemaker.create_flow_definition(
    FlowDefinitionName="invoice-review",
    HumanLoopConfig={
        "WorkteamArn": "arn:aws:sagemaker:us-east-1:123456789012:workteam/private-crowd/invoice-reviewers",
        "HumanTaskUiArn": "arn:aws:sagemaker:us-east-1:123456789012:human-task-ui/textract-review-ui",
        "TaskCount": 1,
        "TaskDescription": "Review low-confidence invoice form fields",
        "TaskTitle": "Invoice field review",
    },
    HumanLoopRequestSource={"AwsManagedHumanLoopRequestSource": "AWS/Textract/AnalyzeDocument/Forms/V1"},
    HumanLoopActivationConfig={
        "HumanLoopActivationConditionsConfig": {
            "HumanLoopActivationConditions": '{"Conditions": [{"And": [{"ConditionType": "ImportantFormKeyConfidenceCheck", "ConditionParameters": {"ImportantFormKey": "Total", "KeyValueBlockConfidenceLessThan": 90}}]}]}',
        },
    },
    OutputConfig={"S3OutputPath": "s3://my-bucket/a2i-results/"},
    RoleArn="arn:aws:iam::123456789012:role/A2IExecutionRole",
)
flow_definition_arn = flow["FlowDefinitionArn"]
 
# Per-call: pass the flow definition ARN in HumanLoopConfig on AnalyzeDocument.
with open("invoice.png", "rb") as f:
    response = textract.analyze_document(
        Document={"Bytes": f.read()},
        FeatureTypes=["FORMS"],
        HumanLoopConfig={
            "HumanLoopName": f"invoice-review-{uuid.uuid4().hex[:12]}",
            "FlowDefinitionArn": flow_definition_arn,
        },
    )
 
human_loop = response.get("HumanLoopActivationOutput")
if human_loop and human_loop.get("HumanLoopActivationReasons"):
    print("routed to human review:", human_loop["HumanLoopArn"])
else:
    print("confidence met the bar - no human review triggered")

// --- TypeScript (AWS SDK v3) ---
import { randomUUID } from "node:crypto";
import { SageMakerClient, CreateFlowDefinitionCommand } from "@aws-sdk/client-sagemaker";
import { TextractClient, AnalyzeDocumentCommand } from "@aws-sdk/client-textract";
import { readFileSync } from "node:fs";
 
const sagemaker = new SageMakerClient({});
const textract = new TextractClient({});
 
// One-time setup: create the flow definition. HumanLoopRequestSource identifies the
// built-in Textract integration; HumanLoopActivationConditions sets the confidence rule.
const flow = await sagemaker.send(new CreateFlowDefinitionCommand({
  FlowDefinitionName: "invoice-review",
  HumanLoopConfig: {
    WorkteamArn: "arn:aws:sagemaker:us-east-1:123456789012:workteam/private-crowd/invoice-reviewers",
    HumanTaskUiArn: "arn:aws:sagemaker:us-east-1:123456789012:human-task-ui/textract-review-ui",
    TaskCount: 1,
    TaskDescription: "Review low-confidence invoice form fields",
    TaskTitle: "Invoice field review",
  },
  HumanLoopRequestSource: { AwsManagedHumanLoopRequestSource: "AWS/Textract/AnalyzeDocument/Forms/V1" },
  HumanLoopActivationConfig: {
    HumanLoopActivationConditionsConfig: {
      HumanLoopActivationConditions: JSON.stringify({
        Conditions: [{ And: [{ ConditionType: "ImportantFormKeyConfidenceCheck", ConditionParameters: { ImportantFormKey: "Total", KeyValueBlockConfidenceLessThan: 90 } }] }],
      }),
    },
  },
  OutputConfig: { S3OutputPath: "s3://my-bucket/a2i-results/" },
  RoleArn: "arn:aws:iam::123456789012:role/A2IExecutionRole",
}));
const flowDefinitionArn = flow.FlowDefinitionArn!;
 
// Per-call: pass the flow definition ARN in HumanLoopConfig on AnalyzeDocument.
const response = await textract.send(new AnalyzeDocumentCommand({
  Document: { Bytes: readFileSync("invoice.png") },
  FeatureTypes: ["FORMS"],
  HumanLoopConfig: {
    HumanLoopName: `invoice-review-${randomUUID().slice(0, 12)}`,
    FlowDefinitionArn: flowDefinitionArn,
  },
}));
 
const humanLoop = response.HumanLoopActivationOutput;
if (humanLoop?.HumanLoopActivationReasons?.length) {
  console.log("routed to human review:", humanLoop.HumanLoopArn);
} else {
  console.log("confidence met the bar - no human review triggered");
}

What this demonstrates:

• CreateFlowDefinition is a SageMaker/A2I API call, not a Textract one - the work team, review UI, and activation rule are all configured before any AnalyzeDocument call happens.
• HumanLoopRequestSource set to "AWS/Textract/AnalyzeDocument/Forms/V1" tells A2I this flow definition is for Textract's built-in integration specifically, so it understands Textract's condition types.
• HumanLoopActivationConditions is where the actual rule lives - here, route to review whenever the "Total" form field's confidence is under 90.
• On the AnalyzeDocument side, only HumanLoopName (a unique identifier you generate) and FlowDefinitionArn are required - Textract decides whether to actually start the loop based on the flow definition's conditions, and reports back via HumanLoopActivationOutput.

Deep Dive

What Lives Outside Textract

The flow definition - who reviews, what UI they see, and under what conditions a document gets routed to them - is entirely a SageMaker/A2I concern, created and managed via CreateFlowDefinition (and the supporting work team and human task UI resources) before you ever call AnalyzeDocument. Textract's only job is deciding, per call, whether the conditions in that flow definition are met, and if so, starting a human loop against it automatically. You never call StartHumanLoop yourself for this built-in integration path - passing HumanLoopConfig to AnalyzeDocument is what triggers it.

HumanLoopConfig on AnalyzeDocument

HumanLoopConfig takes three fields: HumanLoopName (a unique identifier per call - generate a fresh one for every document, since reusing a name across calls fails), FlowDefinitionArn (the ARN from CreateFlowDefinition), and an optional DataAttributes block for content classifiers relevant to compliance handling of the reviewed data. The response includes HumanLoopActivationOutput, which reports whether the conditions were actually met for this specific document and, if so, the HumanLoopArn you can use to check the review's status later.

Activation Conditions Are Textract-Specific

HumanLoopActivationConditions is a JSON expression evaluated per call, and its available condition types differ by which service's built-in integration you registered via HumanLoopRequestSource. For Textract's forms integration, conditions can check things like a named form key's confidence (ImportantFormKeyConfidenceCheck) or whether an expected key is missing entirely - letting you target review at the specific fields that matter to your workflow instead of an overall document-level score.

The Analysis Still Happens Synchronously

AnalyzeDocument still returns its normal Blocks response in the same call, whether or not a human loop was triggered - A2I doesn't block or delay the Textract response. The human loop, if started, runs asynchronously alongside it; your application typically proceeds with the machine extraction immediately and separately checks on (or gets notified about) the human-reviewed result for any fields flagged, then reconciles the two.

Workforce Options

The WorkteamArn in the flow definition's HumanLoopConfig determines who reviews flagged documents: a private workforce (your own employees or contractors), a vendor workforce you've contracted through the AWS Marketplace, or Amazon Mechanical Turk's public workforce. Sensitive documents should generally stay off Mechanical Turk - choose a private or vetted vendor workforce whenever the content includes personal or confidential information.

Gotchas

• Calling AnalyzeDocument without realizing the flow definition doesn't exist yet. HumanLoopConfig referencing a nonexistent or misconfigured FlowDefinitionArn fails the call. Fix: create and verify the flow definition via CreateFlowDefinition before wiring it into production Textract calls.
• Reusing a HumanLoopName across documents. Each human loop needs a unique name; reusing one fails. Fix: generate a fresh identifier (a UUID, a document ID) per call.
• Assuming HumanLoopConfig always triggers a review. The loop only starts if the flow definition's HumanLoopActivationConditions are actually met for that document. Fix: check HumanLoopActivationOutput on the response rather than assuming review happened.
• Treating the human-reviewed result as available immediately. Review is asynchronous and depends on reviewer throughput - it does not come back in the AnalyzeDocument response. Fix: poll or subscribe to the human loop's output location (OutputConfig.S3OutputPath) separately from the Textract call.
• Sending sensitive documents to a public Mechanical Turk workforce. Confidential content may not be appropriate for an open workforce. Fix: use a private or vetted vendor workforce for anything containing personal or sensitive data.
• Writing activation conditions that route everything to review. An overly broad or missing condition can send every document to a human, defeating the point of automation. Fix: scope HumanLoopActivationConditions to the specific fields and confidence bar that actually warrant review.

Alternatives

FAQs

Related

• Textract's Structured-Extraction Model - the Confidence values activation conditions evaluate.
• Form & Table Extraction - the form key-value pairs A2I's Textract integration reviews.
• Textract via SDK Best Practices - confidence-threshold guidance that informs when to route to human review.

Stack versions: This page was written for boto3 1.43.x (Python 3.10+) and the AWS SDK for JavaScript v3 (Node.js 18+).