Stream of Consciousness

This is my stream of consciousness, in which I make available useful notes and thoughts I have which didn't quite make it into blog format.

2023

How to fix docker-compose: command not found error with newer versions of Docker BLOG

less than 1 minute read

Published:

The docker-compose command is missing from recent versions of Docker, replaced by a plugin built into Docker: docker compose. To restore compatibility with scripts which use docker-compose, we can create a wrapper script which forwards its arguments to docker compose. Here’s the script:

# Switch to root
sudo su -
# Write script to file
cat << EOF > /usr/local/bin/docker-compose
#!/bin/bash
docker compose $@
EOF
# Make the script executable so that we can invoke it directly from the shell
chmod +x /usr/local/bin/docker-compose

2022

Siamese network and triplet loss

less than 1 minute read

Published:

Siamese network is an architecture which runs two networks with shared weights (effectively runs the same network twice) on two different inputs simultaneously. It is commonly trained with a contrastive loss such as triplet loss in order to draw together the representations of similar inputs and push apart the representations of contrasting inputs.

Get filepath of Bash activation script

less than 1 minute read

Published:

Use ${BASH_SOURCE[0]} to reference the filepath of a Bash script. Unlike $0, this works if the script is called via bash script.sh or source script.sh.

Beware any vs len

less than 1 minute read

Published:

I fell into the habit of using any() to check if a list is empty. It’s nice because it works for any enumerable, including generators, even if len() is not defined. However, it has a pitfall where if the list is nonempty but contains only falsy values, any() returns False. For this reason, I advise to use len() to check if a list is empty.

Use head -n -0 to get all items in list

less than 1 minute read

Published:

You may know that you can use head -n $n to get the first N lines of a list. But you may not know that you can supply n=-0 to get all items in the list.

Use hard links to replicate log files in a generated directory

1 minute read

Published:

Recently I wanted to write program logs to a file, then copy it to the directory where I store my checkpoints. Because I want to log things before I create the checkpoint directory, I attached my logger to a file in my root directory.

Beware metric auto-reduce with PyTorch Lightning + TorchMetrics

1 minute read

Published:

PyTorch Lightning + TorchMetrics can log metrics per step and per epoch. It also has MetricCollection, which can be used to compute several metrics at once, getting rid of redundant code. Here is how I have it set up:

Print pandas Series as percent

less than 1 minute read

Published:

Use this snippet to print a pd.Series of floats as percentages.

.apply(lambda x: x * 100).apply("{:,.2f}%".format)

Encoder-decoder models

less than 1 minute read

Published:

Encoder decoder is a deep neural network architecture that consists of 2 components:

  • Encoder: input -> encoder memory (real-valued vector),
  • Decoder: encoder memory -> output. Input is variable length, encoder memory is fixed length.

Print item keys with jq

less than 1 minute read

Published:

I can use jq to print the keys in an array of JSON objects quickly.

jq '.[0] | keys[]' $filename

Neural Network Embeddings

less than 1 minute read

Published:

Embeddings are used to map high-dimensional data to low-dimensional floating-point representation. This may improve the performance because it improves the representation of the input given to the model.