devops.cool My 2 cents on technology


Self-Extracting Bash Script

Extract > Update: fixed typo on extract.sh. Sorry about that…

Recently I stumbled upon a problem: How to ship my Puppet manifests - which were hosted on a private GitHub repository - to different servers that not necessarily were part of the same network, and hence, did not share a Puppet Master?

One option was: add a SSH deploy key to each server and clone the repo. Ok, that works, but I wanted to script the whole thing, and creating SSH keys is a bit cumbersome (IMHO).

In the end, my solution was a Self-Extracting shell script hosted on S3! To my complete surprise, it’s really easy to do this. All you need is a bit of shell and an open mind. Here’s how it’s done:

Directory structure

Say you want to self-extract directory files/ from your project. Your directory structure for building the extracting script should look like this:

my_project/
 - files/
 - extract.sh
 - build.sh
 - setup.sh

files extract.sh and build.sh are explained below.

Extract, Build and Setup

The way this whole thing works is this: extract.sh will be the “head” of the final self-extracting script, which will read itself starting at a specific line and pipe the output to TAR, which will extract the files to a temporary directory. Then, it will run setup.sh, which should put all files in place.

build.sh, on the other hand, will compress the folder with the files plus setup.sh and concatenate extract.sh with the tar archive on a new file, your self-extracting script. Got it? Good!

extract.sh

#!/bin/bash
FILE_MARKER=`awk '/^TAR FILE:/ { print NR + 1; exit 0; }' $0`
TMP_DIR=`mktemp -d /tmp/self-extract-bash.XXXXXX`
FILE_DIR=`pwd`

# Extract the file using pipe
tail -n+$FILE_MARKER $0 | tar -zx -C $TMP_DIR

# Run the setup script
$TMPDIR/setup.sh

# Remove the self-extracting script and the temp directory
rm -rf $FILE_DIR/$0 $TMPDIR
exit 0

#TAR File Marker
TAR FILE:

The last line on extract.sh is just a marker. The exit 0 two lines before is there to ensure the script stops executing before the “TAR FILE:” line.

build.sh

#!/bin/bash
tar -zcf files.tar.gz files/ setup.sh
cat extract.sh files.tar.gz > installer.sh # <- Name of your self-extracting script
chmod +x installer.sh

setup.sh

This script should copy the extracted files to wherever you need them, and/or run any other commands you require. For instance:

#!/bin/bash
cp -Rv files/ /opt/my_files
chown -R user.group /opt/my_files

Easy, right?

Once I got my head around this process, it seemed very straightforward. But if you’re having dificulties, just ping me on Twitter (@igorlgentil) and I’ll be happy to help!