jq - recursive search for keys containing "string" stripping empty results

jq-filters.sh

# recursive search for keys containing "string" stripping empty results
jq '.. | objects | with_entries(select(.key | contains("ftp"))) | select(. != {})'

# same, but output propper array
jq '[ .. | objects | with_entries(select(.key | contains("ftp"))) | select(. != {}) ]'
# or
jq 'map( .. | objects | with_entries(select(.key | contains("ftp"))) | select(. != {}) )'

# transform input from {type: a, amount: 1} to {a: 1} and sum all values by type
jq '[ .[] | {(.type): .amount} ] | map(to_entries) | add | group_by(.key) | map({key: .[0].key, value: map(.value) | add}) | from_entries'

This is a great start! JQ is pretty powerful but arcane. In my case, this produces a series of objects that are not valid json:

{
  "ImageId": "ami-fce3c696"
}
{
  "ImageId": "ami-0d72bbaa068279155"
}

I know you can get it to fix this somewhere with the correct map

@chb0github

To have a proper array as result output surround everything with [ and ], example:
jq '[ .. | objects | with_entries(select(.key | contains("ftp"))) | select(. != {}) ]'

Wow! It's so amazing, thx guys!
But how use new array correctly?
In this example:

{
  "ImageId": "ami-fce3c696"
}
{
  "ImageId": "ami-0d72bbaa068279155"
}

How I can get every ImageId value? Or get plain text like that for use that in bash-scripting?

ami-fce3c696
ami-0d72bbaa068279155

I need use another jq in pipe or we have another way?

@ksemele to get raw text use -r modifier. And to get a value just filter them by their key name, example:

❯ echo '{
  "ImageId": "ami-fce3c696"
}
{
  "ImageId": "ami-0d72bbaa068279155"
}' | jq '.ImageId' -r
ami-fce3c696
ami-0d72bbaa068279155

thanks! This is super useful. When I only care about one key this lets me avoid having to figure out the exact complicated structure of the big object being returned. Cheers!

Great query! Here's some explainations(from copilot), in case there's someone like me doesn't know jq so well to understand it :)

The provided jq query is used to filter and transform JSON objects. Here's a step-by-step explanation of what each part of the query does:

1. ..:

   • This is a recursive descent operator. It searches through all levels of the JSON structure.

2. | objects:

   • This filters the results to include only JSON objects.

3. | with_entries(select(.key | contains("ftp"))):

   • with_entries is a function that processes each key-value pair of an object.
   • select(.key | contains("ftp")) filters the key-value pairs to include only those where the key contains the string "ftp".

4. | select(. != {}):

   • This filters out any empty objects from the results.

Putting it all together, the query recursively searches through the JSON structure, selects objects, and then filters those objects to include only key-value pairs where the key contains "ftp". Finally, it removes any empty objects from the results.