Benjamin Steenhoek

Benjamin Steenhoek

PhD student at Iowa State University

Print item keys with jq

less than 1 minute read

Published:

I can use jq to print the keys in an array of JSON objects quickly.

jq '.[0] | keys[]' $filename

I sometimes want to know the names of the keys on the objects in a dataset stored in JSON format, but if the JSON file is very large, it can be slow+consume lots of memory to read the whole with e.g. Python.

For example, when I download a dataset in the following format:

[
	{
		"id": 1,
		"feat1": "foo",
		"feat2": "bar",
		"feat3": "baz",
	},
	{
		"id": 2,
		"feat1": "foo",
		"feat2": "bar",
		"feat3": "baz",
	},
	// 1,000,000,000 more items...
]

The tool will print out:

"id"
"feat1"
"feat2"
"feat3"

This comes in handy when I’m reproducing ML research code, where the implementation is mostly complete but needs some glue code to read the dataset and spit it into a different format. Enjoy!

You May Also Enjoy

How to limit the number of Twitter posts in your timeline using JavaScript BLOG

1 minute read

Published:

I have a bit of a problem scrolling too much of the awesome content on Twitter. There are just too many other people creating cool ideas and software! At first I justified it by the fact that I find academic papers (my feed is curated to mostly academic CS/ML/SE), but now I admit it’s too much! In a bid to waste more time in order to waste less time, I created a simple script that puts an extra UI onto each post in my Twitter feed:

  • For the first five posts, it numbers them to remind me how many I’ve browsed.
  • After that, it blocks them out with a reminder of the limit I set (currently I keep it at five posts).

How to fix docker-compose: command not found error with newer versions of Docker BLOG

less than 1 minute read

Published:

The docker-compose command is missing from recent versions of Docker, replaced by a plugin built into Docker: docker compose. To restore compatibility with scripts which use docker-compose, we can create a wrapper script which forwards its arguments to docker compose. Here’s the script:

# Switch to root
sudo su -
# Write script to file
cat << EOF > /usr/local/bin/docker-compose
#!/bin/bash
docker compose $@
EOF
# Make the script executable so that we can invoke it directly from the shell
chmod +x /usr/local/bin/docker-compose