jq & Parsing AWS CLI/SDK JSON Output

The AWS CLI's default output is JSON, and jq is the standard command-line tool for slicing, filtering, and reshaping it. This page covers jq filters against real AWS CLI and SDK-produced JSON, and where it overlaps with the CLI's own --query/JMESPath.

Recipe

Pull specific fields out of a JSON response without writing a script.

# Pretty-print (jq with no filter formats and colorizes JSON)
aws ec2 describe-instances --output json | jq .
 
# Pull one field
aws ec2 describe-instances --output json \
  | jq '.Reservations[].Instances[].InstanceId'
 
# -r strips the surrounding quotes, so the value is script-friendly
aws ec2 describe-instances --output json \
  | jq -r '.Reservations[].Instances[].InstanceId'

When to reach for this:

• You already pipe things through jq for non-AWS JSON and want one consistent tool.
• You need a transform --query's JMESPath cannot express (string interpolation, map/select chains, math).
• You are parsing JSON that did not come from the CLI at all, such as a payload your SDK code wrote to a file for debugging.
• You want colorized, pretty-printed output for reading a large response by eye.

Working Example

A common task: list the IDs and states of only the running EC2 instances, then feed the IDs into another command.

# Filter with jq's select(), then project to just the ID
aws ec2 describe-instances --output json \
  | jq -r '.Reservations[].Instances[]
      | select(.State.Name == "running")
      | .InstanceId'

# Reshape into a compact object per instance
aws ec2 describe-instances --output json \
  | jq '.Reservations[].Instances[]
      | {id: .InstanceId, state: .State.Name, type: .InstanceType}'

# Feed jq's output straight into another aws command
for id in $(aws ec2 describe-instances --output json \
  | jq -r '.Reservations[].Instances[] | select(.State.Name=="stopped") | .InstanceId'); do
  echo "Stopped instance: $id"
done

What this demonstrates:

• select() filters elements by a condition, similar to JMESPath's [?...] filter syntax.
• jq -r (raw output) is what you want when piping a value into a shell loop or another command; without -r you get a quoted JSON string like "i-0123...".
• {key: .Path} builds a new object shape, the jq equivalent of a JMESPath multiselect hash.
• The same JSON your SDK code would parse into a dict or a typed object is exactly what jq sees on the wire.

Deep Dive

How It Works

• jq reads JSON from stdin (or a file argument) and evaluates a filter expression against it, printing the result as JSON by default.
• . is the identity filter (print the input); .Field accesses a key; .Array[] iterates an array's elements.
• | inside a jq filter pipes the output of one expression into the next, just like | does between shell commands, but scoped to jq's own data flow.
• select(condition) keeps only elements where condition is true; combine with map() to transform every kept element.
• -r (raw) unwraps a single JSON string result to plain text; without it, jq always prints valid JSON, quotes included.
• -c (compact) prints one JSON object per line instead of pretty-printed, useful for feeding into wc -l or another line-oriented tool.

jq vs. the CLI's --query/JMESPath

The AWS CLI has its own built-in filter language, JMESPath, exposed as --query and covered in depth on CLI Output Formats & --query/JMESPath. jq is a separate, general-purpose tool that happens to be excellent at the same job.

Both run entirely client-side, after the AWS API has already returned the full response - neither reduces what AWS sends over the wire. For that, use the CLI's own --filters or a request-level filter parameter where the API supports one.

A practical rule: reach for --query first since it needs nothing extra installed, and reach for jq when the transform gets complex enough that JMESPath becomes awkward, or when you are working with JSON that never came from the CLI at all.

Useful jq Filters for AWS JSON

# Count how many items came back
aws s3api list-objects-v2 --bucket my-bucket --output json | jq '.Contents | length'
 
# Sort by a field
aws ec2 describe-volumes --output json \
  | jq '.Volumes | sort_by(.Size) | .[-1]'
 
# Build a CSV-like line per row
aws dynamodb scan --table-name Orders --output json \
  | jq -r '.Items[] | [.id.S, .status.S] | @tsv'

Gotchas

• Forgetting --output json - jq cannot parse the CLI's table or text output. Fix: always pass --output json (or set it as your profile default) before piping to jq.
• Quoted strings breaking a shell loop - Without -r, jq prints "i-0123abc" with quotes, which then gets treated as a literal quoted string downstream. Fix: use jq -r whenever the result feeds into another shell command.
• Confusing jq's select with JMESPath's [?...] - The syntax looks similar but is not interchangeable between the two tools. Fix: keep --query expressions and jq filters in separate mental buckets; do not paste one into the other.
• Assuming jq reduces AWS billing or data transfer - Like --query, it runs after the full response has already arrived. Fix: use server-side filtering (a --filters flag or API FilterExpression) to actually cut what AWS returns.
• Missing jq on a fresh machine - Unlike --query, jq is a separate binary that is not bundled with the AWS CLI. Fix: install it explicitly (brew install jq, apt-get install jq), and do not assume CI images have it preinstalled.
• Iterating a null field - .Reservations[].Instances[] throws Cannot iterate over null if a key is missing from the response. Fix: guard with // [], e.g. (.Reservations // [])[].

Alternatives

FAQs

Related

• Linux CLI Basics - the pipe and redirection basics this page builds on.
• CLI Output Formats & --query/JMESPath - the CLI's built-in alternative to jq.
• Managing AWS Env Vars & Credentials Safely in Shell Sessions - keeping the values you extract this way out of shell history.
• Linux CLI Best Practices - habits for scripting these pipelines safely.

Stack versions: This page was written for boto3 1.43.x (Python 3.10+) and the AWS SDK for JavaScript v3 (Node.js 18+).