6 minutes
Using Github Actions for CI and Automated Releases
Intro
The free CIs are having a difficult time and mostly it’s because of malicious people are abusing to run cryptomining. As a result, TravisCI announced changes in their billing system and GitLab also announced restrictions. So, if we look for other alternatives, Azure Pipelines and Github Actions are the first ones that come to my mind. For the latter, I’ve been playing with it and I’m quite sold on the promise of having everything on Github.
Edit: Just one day after I wrote this down, LayerCI wrote an article about continuous integration and crypto mining. I recommend you to check on it.
Simple CI example for a Rust project
Firstly, I would like to start with a simple example for a Rust project.
name: Build/Test
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
env:
CARGO_TERM_COLOR: always
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Build
run: cargo build --verbose
- name: Run tests
run: cargo test --verbose
Github already guides you towards creating this simple action yaml and since the project doesn’t have external dependencies or systems integrated into it. We can immediately use it and make the building and testing procedure insanely trivial.
I’m aware that there are other action repositories for rust and cargo but the GitHub vms automatically provides rust compiler and cargo which satisfies the simple necessity. But for using the nightly compiler or other versions or rust, probably that would be the way to go.
The YAML explains itself but in a closer look, we can immediately see that this actions run on pushes or pull requests on main and uses an ubuntu vm.
If you want to check it in action, I apply this action to one of my research related repository.
A more complex CI example with some other system integration
I have some other project in python and it has a MySQL/MariaDB integration. Upon my research, I managed to find that jobs can have services that can be set with something like this
services:
mariadb:
image: mariadb:latest
ports:
- 3306
env:
MYSQL_USER: user
MYSQL_PASSWORD: password
MYSQL_DATABASE: test
MYSQL_ROOT_PASSWORD: password
options: --health-cmd="mysqladmin ping" --health-interval=5s --health-timeout=2s --health-retries=3
However, I couldn’t manage to make the DB work. I tried many options, but always getting MySQL service is unhealthy. So in the end, I decided to use an external GitHub action.
Its own documentation is pretty clear so won’t be explaining it much. So my GitHub action ci YAML looked like this.
name: Test
on:
push:
branches: [ master ]
pull_request:
branches: [ master ]
jobs:
test:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: [3.7, 3.8, 3.9]
steps:
- uses: getong/mariadb-action@v1.1
with:
mysql database: 'dpem_test'
mysql root password: 'password'
mysql user: 'root'
mysql password: 'password'
- uses: actions/checkout@v2
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
sudo apt-get install parallel
python -m pip install --upgrade pip
python -m pip install mysql-connector
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
- name: Run tests
run: |
cd tests
./run_tests.sh
As seen above, it’s easy to set a matrix of experiments with the strategy tag and we can use 3 different versions of python without too much of a hassle.
We can also see that we can gather other dependencies from apt-get and/or pip which are necessary for our project.
Again, this GitHub action can be seen in action in a repository
Automated releases with Github Actions
So, I believe this is where the more interesting things began. Using GitHub actions, it’s possible to use the compiled assets to a draft release. And, the best thing is we can do this on 3 different platforms and automatically generate our release binaries for Linux, windows and macos. I know that this is also possible on Azure Pipelines but I only have tested this on GitHub actions and it looks pretty easy to configure.
For the first rust project, the YAML for automated releases looks like this:
name: Create Release Builds
on:
push:
tags:
- "v*" # matches v1.0.1, v1.2, v2, etc
env:
CARGO_TERM_COLOR: always
jobs:
once:
name: Create GitHub release
runs-on: ubuntu-latest
outputs:
upload_url: ${{ steps.create_release.outputs.upload_url }}
steps:
- name: Create a release
id: create_release
uses: actions/create-release@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
tag_name: ${{ github.ref }}
release_name: Release ${{ github.ref }}
draft: true
prerelease: true
build:
name: Create cross-platform release build, tag and upload binaries
needs: once
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-latest, macos-latest, windows-latest]
steps:
- name: Checkout
uses: actions/checkout@v2
- name: Build release version
run: cargo build --release
- name: Make Zip
run:
7z a -tzip ${{ github.workspace }}/target/release/dig-rs-${{ matrix.os }}.zip ${{ github.workspace }}/target/release/dig-rs* "-x!*.d" "-x!*.pdb"
- name: Upload Release Asset
id: upload-release-asset
uses: actions/upload-release-asset@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
upload_url: ${{ needs.once.outputs.upload_url }}
asset_path: ${{ github.workspace }}/target/release/dig-rs-${{ matrix.os }}.zip
asset_name: dig-rs-${{ matrix.os }}.zip
asset_content_type: application/octet-stream
First of all, this only gets triggered with certain tagged commits. A simple “v*” regex allows any commit with a tag with the prefix “v” (like v1.0 v2.5) valid for this action.
The procedure has 2 distinct jobs. The first job creates an empty draft release and sets up an output upload url for any other job to submit assets.
upload_url: ${{ steps.create_release.outputs.upload_url }}
Specifically, this line sets the upload url as seen in its name. It sets up its upload as one of its steps’ (its id is create_release) outputs.
The second job specifies that it needs the first job (named once) and also defines the 3 different os platforms with the strategy tag.
strategy:
matrix:
os: [ubuntu-latest, macos-latest, windows-latest]
Afterwards, we can compile our binary with cargo build --release. Before giving assets to the release, it’s useful to compress the binaries and the nice thing I found while searching online that 7z is available on all 3 platforms. This allows using one command to create one easily selectable zip file depending on the platform.
7z a -tzip ${{ github.workspace }}/target/release/dig-rs-${{ matrix.os }}.zip ${{ github workspace }}/target/release/dig-rs* "-x!*.d" "-x!*.pdb"
The regex dig-rs* allows to collect both dig-rs or dig-rs.exe binary depending on the platform. The necessary exclusion for .d or windows specific .pdb files can be done by "-x!*.d" "-x!*.pdb" on 7z. For more options check 7z documentation.
Afterwards, we can use the release asset action and upload it to the previously defined needed job’s upload path.
That’s all! Again this action is available on one of the myrepositories.
Final words
In my opinion, Github made continuous integrations systems even more convenient to use on projects for multiple purposes. And now with easily set up automated releases, I can automatically compile binaries for my projects for multiple platforms. All I need to do afterwards is to approve the release in the end.
Unfortunately, Github actions is also constantly abused by malicious pull requests for cryptomining and as a result recently, Github restricted automated ci runs for unknown pull requests. It’s not a final solution but it’s a first step towards something.
continuous integration ci travis rust python mysql mariadb mysql cryptomining
1165 Words
2021-04-24 21:14 +0100