Using Github Actions to update a shiny app on a private server

This post will teach you how to set up continuous deployment of shiny apps hosted on your private server (i.e., on an EC2 instance, AWS lightsail, Digital Ocean droplet, etc.). I assume you already have your server running shiny server, and that you are using Github for your development.

If you’re looking for continuous deployment of shiny apps to the shinyapps.io hosting service you can check out this post.

Continuous what…?!

Continuous deployment is the act of deploying your program (a shiny app in our case), every time it is updated, and doing so in an automated way.

More rigorously, let’s assume you already have a shiny app that you are developing, and using Github as your main version control tool. You are deploying your app to a private server (i.e., on an AWS EC2, a lightsail instance, Digital Ocean droplet, or similar). To get your app up to the server, you cloned the app’s repository into the directory /srv/shiny-server/YOUR_APP_NAME/ . Every time your app is updated, you get into the directory to do a git pull , to make sure the updates are applied to your production server.

This can be quite of a headache, ssh-ing into the server, just to do a git pull. Wouldn’t it be nice if the git pull in your server would magically happen every time you push your updates to the main branch automatically? this is what the rest of this post is about.

A small note about shiny app testing

Before I continue with this post, I would like to mention that any continuous integration should be coupled with automated tests for your app. This post doesn’t deal with testing at all (just the final act of deploying it). Rstudio has extensive documentation on shiny app testing here, as well as this chapter here from the “Mastering Shiny” book.

Such tests should be placed before the step I’m about to write about, and passing all tests should be a prerequisite for performing the actual deployment. For example, you can have tests run on every push (to any branch) thus making sure that your app is tested before you finally merge to your main branch (which is then automatically updated in production).

The process

Before diving deeper into the process, a short overview of the steps we’re going to do. We’re going to:

  1. Log in to the server, and create a new user with read/write access to the directory of the app we want to update.
  2. Create an SSH key for this user which will enable Github Actions to log in to the server.
  3. Add secrets to the Github repository (the SSH private key, hostname, user name, port).
  4. Add the SSH key as a repository deployment key.
  5. Create the workflow in the repository.

Creating an additional user

The first step is to create a new user which will be the user Github Actions is using to log in to your server. You can also use an existing user for that, but I prefer to separate my credentials with the credentials I provide external services (such as Github Actions). Log into your server and add a new user (replace github_actions_user with the name you want):

sudo useradd github_actions_user

The next thing is to add write permissions for this newly created user on the directories we want Github Actions to be able to update. Let’s say that the app is called my_shiny_app, and that the current owner of the app is the user ubuntu. We’re going to replace the directory owner with our new user’s group and add ubuntu to this group as well (to retain ubuntu’s write permissions).

cd /srv/shiny-server/
sudo chown github_actions_user:github_actions_user -R my_shiny_app
sudo usermod -a -G github_actions_user ubuntu

The middle line replaces the directory owner and the last line adds ubuntu to the directory ownership (group).

In my setup, users must authenticate with a key to ssh into the server (a password is not enough), so let’s create the keys.

Creating an SSH key for login

First thing is to switch to the new user:

su - github_actions_user

Following the instructions from here to create the new keys (when prompted for file location and for passphrase, just click Enter leaving the default location for the keys and no passphrase).

ssh-keygen -t ed25519 -C "your_email@example.com"
eval "$(ssh-agent -s)"
ssh-add ~/.ssh/id_ed25519

Important note about using a passphrase: after you type the first command you will be prompted for a passphrase. The passphrase encrypts the private key so that it can only be used by the user who knows the passphrase. However, if the repository of the app is private the Github Action will need the key to git pull from the repository. The Github Action we are using (as far as my experience goes) doesn’t work well with private repositories which we access with keys protected by a passphrase, hence I used no passphrase on these keys. I’m pretty sure there’s a better way around it, if you find it - let me know (@SaridResearch on twitter).

Finally, add the key to the authorized list, like this:

cat .ssh/id_ed25519.pub | ssh b@B 'cat >> .ssh/authorized_keys'

Adding secrets to the repository

Now we want to add the required information to the repository. We need the following data: the private key for ssh-ing, the host, port, and user name.

To get the private key while logged in as github_actions_user type:

nano ~/.ssh/id_ed25519

Copy the key.

Go to the repository of the app and click on settings (the top tabs) -> secrets (on the left menu). Click on “New repository secret”. Add a secret named KEY and paste the contents of the key you just copied. Also add the following secrets in a similar manner HOST (the host name), PORT (for the most part it should be 22, unless you changed it), USERNAME (in this example github_actions_user), APPNAME (the location of the app within the shiny-server directory, e.g. my_shiny_app).

Adding a deployment key

You need this step only if your repository is private - the public key should be added to Github, so that Github will allow you access to the repository. For that we need the public key. Get it by typing:

nano ~/.ssh/id_ed25519.pub

Copy the public key, in Github inside the repository of the app go to Deploy keys (on the left menu), add a title to your key, such as “key for pulling to production”, and paste the public key.

Create the workflow

For the final step, we’re going to use the Github Action template, prepared by appleboy and others (from here). Just add the following yml file to your repository under: .github/workflows/main.yml:

# SSH into machine and git pull for updates

name: execute remote ssh to pull updates from master
on:
  # Triggers the workflow on push or pull request events but only for the main branch
  push:
    branches: [ main ]
# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
  # This workflow contains a single job called "build"
  build:
    # The type of runner that the job will run on
    runs-on: ubuntu-latest
    
    # Steps represent a sequence of tasks that will be executed as part of the job
    steps:
      # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
      - uses: appleboy/ssh-action@master
        with:
          host: ${{ secrets.HOST }}
          username: ${{ secrets.USERNAME }}
          key: ${{ secrets.KEY }}
          port: ${{ secrets.PORT }}
          script: |
            cd /srv/shiny-server/${{ secrets.APPNAME }}
            git pull

Conclusion

After completing these steps, every time you push something to the main branch it will magically update the app on your server. Be sure to check it works after a few commits (go to Actions in the top menu and see that it works.

What’s going on behind the scene is that Github Actions spawns up a Docker which logs in to your server via ssh, and then runs cd /srv/shiny-server/my_shiny_app/ and git pull.

For more information on this specific Github Action read https://github.com/appleboy/ssh-action.


Partner and Head of Data Science

Related