How to set up a shared cache for HuggingFace libraries BLOG

3 minute read

Published: January 16, 2024

TL;DR

I set up a shared cache for HuggingFace libraries like transformers and datasets. See the repository: https://github.com/bstee615/shared-hf-cache. To use it, create a shared directory which can be edited by all interested users and set the environment variable export HF_HOME="/huggingface".

How to fix docker-compose: command not found error with newer versions of Docker BLOG

less than 1 minute read

Published: December 20, 2023

The docker-compose command is missing from recent versions of Docker, replaced by a plugin built into Docker: docker compose. To restore compatibility with scripts which use docker-compose, we can create a wrapper script which forwards its arguments to docker compose. Here’s the script:

# Switch to root
sudo su -
# Write script to file
cat << EOF > /usr/local/bin/docker-compose
#!/bin/bash
docker compose $@
EOF
# Make the script executable so that we can invoke it directly from the shell
chmod +x /usr/local/bin/docker-compose

How to use task-spooler as a shared queueing system BLOG

4 minute read

Published: December 16, 2023

TL;DR

I set up a wrapper around task-spooler. See the repository: https://github.com/bstee615/shared-task-spooler. To use it, create a file containing the below script and invoke it using the same arguments as task-spooler.

#!/bin/bash
# Dependency: sudo apt install -y task-spooler
TS_SOCKET=$(dirname -- "$0")/TS_SOCKET exec tsp -L "$USER" $@

friendship ended with earlyoom, now nohang is my best friend

less than 1 minute read

Published: February 13, 2023