What is jq?
jq can transform JSON in various ways, by selecting, iterating, reducing and otherwise mangling JSON documents.
Basically, jq is your sed for json :D.
Can be found at https://stedolan.github.io/jq/
I randomly discovered jq recently and it seems to be quite the program I hoped it would be, in terms of helping me with json.
Ok, enough of the humanly talks, let’s get do something practical.
[note, I was in a rush so geshi is a bit screwed up, I selected most of these source as python instead of json or bash, bear with me :p]
Suppose i have a json file called output.json:
s3lv3n-mc-burger:jqtest selven$ cat output.json {"total":4,"notifications":[{"severity":"urgent","details":{"indices":69,"index_ranges":68},"type":"index_ranges_recalculation","timestamp":"2015-09-25T07:50:30.357Z","node_id":"testnode"},{"severity":"normal","type":"es_cluster_green","timestamp":"2015-09-25T06:45:01.933Z","node_id":"testnode"},{"severity":"urgent","type":"journal_uncommitted_messages_deleted","timestamp":"2015-09-24T16:28:29.122Z","node_id":"testnode"},{"severity":"urgent","details":{"journal_utilization_percentage":96.0},"type":"journal_utilization_too_high","timestamp":"2015-09-24T16:26:04.121Z","node_id":"testnode"}]}
s3lv3n-mc-burger:jqtest selven$ cat output.json | jq .
{ "total": 4, "notifications": [ { "severity": "urgent", "details": { "indices": 69, "index_ranges": 68 }, "type": "index_ranges_recalculation", "timestamp": "2015-09-25T07:50:30.357Z", "node_id": "testnode" }, { "severity": "normal", "type": "es_cluster_green", "timestamp": "2015-09-25T06:45:01.933Z", "node_id": "testnode" }, { "severity": "urgent", "type": "journal_uncommitted_messages_deleted", "timestamp": "2015-09-24T16:28:29.122Z", "node_id": "testnode" }, { "severity": "urgent", "details": { "journal_utilization_percentage": 96 }, "type": "journal_utilization_too_high", "timestamp": "2015-09-24T16:26:04.121Z", "node_id": "testnode" } ] }
🙂 Yup, you now have a prettified JSON output, but hey, you are not here just for that.
How many notifications do i have in that JSON
s3lv3n-mc-burger:jqtest selven$ cat output.json | jq .total 4
awesome!
How about the contents of notifications?
s3lv3n-mc-burger:jqtest selven$ cat output.json | jq '.notifications ' [ { "severity": "urgent", "details": { "indices": 69, "index_ranges": 68 }, "type": "index_ranges_recalculation", "timestamp": "2015-09-25T07:50:30.357Z", "node_id": "testnode" }, { "severity": "green", "type": "es_cluster_green", "timestamp": "2015-09-25T06:45:01.933Z", "node_id": "testnode" }, { "severity": "urgent", "type": "journal_uncommitted_messages_deleted", "timestamp": "2015-09-24T16:28:29.122Z", "node_id": "testnode" }, { "severity": "urgent", "details": { "journal_utilization_percentage": 96 }, "type": "journal_utilization_too_high", "timestamp": "2015-09-24T16:26:04.121Z", "node_id": "testnode" } ]
Now what if we want to get only notifications which are urgent?
s3lv3n-mc-burger:jqtest selven$ cat output.json | jq '.notifications| .[]|select(.severity == "urgent")' { "severity": "urgent", "details": { "indices": 69, "index_ranges": 68 }, "type": "index_ranges_recalculation", "timestamp": "2015-09-25T07:50:30.357Z", "node_id": "testnode" } { "severity": "urgent", "type": "journal_uncommitted_messages_deleted", "timestamp": "2015-09-24T16:28:29.122Z", "node_id": "testnode" } { "severity": "urgent", "details": { "journal_utilization_percentage": 96 }, "type": "journal_utilization_too_high", "timestamp": "2015-09-24T16:26:04.121Z", "node_id": "testnode" }
Let’s access them with indexes
s3lv3n-mc-burger:jqtest selven$ cat output.json | jq '.notifications| .[0]' { "severity": "urgent", "details": { "indices": 69, "index_ranges": 68 }, "type": "index_ranges_recalculation", "timestamp": "2015-09-25T07:50:30.357Z", "node_id": "testnode" }
s3lv3n-mc-burger:jqtest selven$ cat output.json | jq '.notifications| .[1]' { "severity": "green", "type": "es_cluster_green", "timestamp": "2015-09-25T06:45:01.933Z", "node_id": "testnode" }
s3lv3n-mc-burger:jqtest selven$ cat output.json | jq '.notifications| .[2]' { "severity": "urgent", "type": "journal_uncommitted_messages_deleted", "timestamp": "2015-09-24T16:28:29.122Z", "node_id": "testnode" }
s3lv3n-mc-burger:jqtest selven$ cat output.json | jq '.notifications| .[4]' null
s3lv3n-mc-burger:jqtest selven$ cat output.json | jq '.notifications| .[3]' { "severity": "urgent", "details": { "journal_utilization_percentage": 96 }, "type": "journal_utilization_too_high", "timestamp": "2015-09-24T16:26:04.121Z", "node_id": "testnode" }
How about getting the journal utilization percentage from the third notification?
s3lv3n-mc-burger:jqtest selven$ cat output.json | jq '.notifications| .[3].details.journal_utilization_percentage' 96
Note that the above can also be done this way
cat output.json | jq '.notifications[3].details.journal_utilization_percentage'
And here we end it with a beautiful one liner:
s3lv3n-mc-burger:jqtest selven$ all_json=`cat output.json` && notifs=`echo $all_json| jq .total` && urgencies=`echo $all_json | jq '.notifications | .[]| select(.severity == "urgent")| .type'`; if [ $? -eq 0 ]; then clear; echo "Number of notification: $notifs"; echo " With pressing matters as follows: "; echo $urgencies; fi
Output is:
Number of notification: 4
With pressing matters as follows:
“index_ranges_recalculation” “journal_uncommitted_messages_deleted” “journal_utilization_too_high”
If you want to get only the keys
s3lv3n-mc-burger:jqtest selven$ cat output.json| jq 'keys'
Now isn’t that awesome 😀
Note that I have written this in 15 minutes, I just decided to learn jq without the tutorial, and was too much in a hurry to start firing up with jq :p.
Special Characters can fuck your jq filter
s3lv3n-mc-burger:jq selven$ cat test { "SyIf43UmQI6xC-bibitBgVkQ": { "valuea": 12, "valueb": 13 }, "blah": 14, "withoutminus": { "valuec": 15, "valued": 16 } } s3lv3n-mc-burger:jq selven$ cat test|jq '.SyIf43UmQI6xC-bibitBgVkQ' jq: error: eCuBgVkQ/0 is not defined at , line 1: .SyIf43UmQI6xC-bibitBgVkQ jq: 1 compile error s3lv3n-mc-burger:jq selven$ var="SyIf43UmQI6xC-bibitBgVkQ";cat test|jq '.["'$var'"]' { "valuea": 12, "valueb": 13 } s3lv3n-mc-burger:jq selven$ Holy shit it works!
Happy jq discovery. There are several better ways to do the above, I have just :p been brutal above.
One Comments “jq”