AWS CLI Query Filter Tutorial with Examples (ft. jmespath)

AWS CLI is good, but a pain if you don’t know how to use the query filter. Let’s learn how to use the query filter in AWS CLI.

Simple query filter

Let’s use the sts command which gives us the details of the user (caller).

aws sts get-caller-identity

We get this JSON output:

{
    "UserId": "AIDAS75KRR6WG6DEMAS5N",
    "Account": "205989462686",
    "Arn": "arn:aws:iam::205989462686:user/my-cli-user"
}

Now with query filter we can choose to just get the account id, instead of getting all the other details. Just pass the param name i.e. Account

❯ aws sts get-caller-identity --query 'Account'              
"205989462686"

Remove quotes from AWS CLI output

You can notice that we are getting quotes (") in the result, because it’s supposed to be JSON. To get rid of the quotes, use --output text option.

❯ aws sts get-caller-identity --query 'Account' --output text
205989462686

Now, we just get the account id — without the quotes.

Processing data with other commands

We can now pipe the output to other CLI programs i.e. wc which counts the words, lines and characters.

❯ aws sts get-caller-identity --query 'Account' --output text | wc -c
13 # we get 13 which is total characters + newline

Remove newline character from AWS CLI output

We can use tr to remove the newline (\n) character from the CLI output.

❯ aws ... | tr -d '\n' | wc -c
12 # we get 12 instead of 13

Reading nested values from array

❯ aws ec2 describe-vpcs --query "Vpcs[0].VpcId"
"vpc-0ed34e62ff8c36330"

We pass the array index in square brackets ([]) followed by a period (.) and property name.

Get all the values inside array

Instead of putting a index number, we can put * to get all the properties.

❯ aws ec2 describe-vpcs --query "Vpcs[*].VpcId"
[
    "vpc-0ed34e62ff8c36330",
    "vpc-b2b3d0cf",
    "vpc-05e7a71655fc6042d"
]

Filtering values from array

If you want to list all the security groups, we could use AWS CLI describe-security-groups command.

aws ec2 describe-security-groups

Security Groups is an array. Inside, there are IP permissions list, and inside that we have a port number.

{
    "SecurityGroups": [
        {
            "GroupId": "sg-0515fddc85ac7423f",
            "IpPermissions": [
                {
                    "IpProtocol": "tcp",
                    "FromPort": 3306,
                    "ToPort": 3306,
                    ...
                },
                {...}
            ]
        },
        {...}
    ]
}

For example: if you want to filter all the security groups which has port 22 open, you could use query filter of 'SecurityGroups[?IpPermissions[?ToPort==22]].GroupId'

❯ aws ec2 describe-security-groups \
--query 'SecurityGroups[?IpPermissions[?ToPort==`22`]].GroupId'
[
    "sg-07e9d856532ce3e05",
    "sg-074e479d5d49ce7b4",
    "sg-0d8e1d41355a0ccb7",
    "sg-0d5fa9574c75653fc",
    "sg-035a1e119fae392af",
    "sg-015825c780a181dff",
    "sg-002a822adba548b86",
    "sg-0b4a7ddc438077278",
    "sg-08857a66141229a9e",
    "sg-098747db42d9cc9c8"
]

AWS CLI and `jq`

With jq you can perform lot of other operations on the JSON as well. For example we can get the array length.

❯ aws ec2 describe-security-groups --query 'SecurityGroups[?IpPermissions[?ToPort==`22`]].GroupId' | jq length
10

You can learn more about jq filtering on the official docs.

More complex query filters

Internally aws cli uses jmespath to filter the output from the command. Check out more examples and tutorial on the official docs.

Example of filter, sort & join at the same time

Sample data:

{
  "locations": [
    {"name": "Seattle", "state": "WA"},
    {"name": "New York", "state": "NY"},
    {"name": "Bellevue", "state": "WA"},
    {"name": "Olympia", "state": "WA"}
  ]
}

For example let’s get all the locations with state WA (Washington).

To sort them in order of codepoint we use the sort function.
@ is variable placeholder
join combines the array using the ,

Query:

locations[?state == 'WA'].name | sort(@) | {WashingtonCities: join(', ', @)}

Output:

{
  "WashingtonCities": "Bellevue, Olympia, Seattle"
}