Why I’m writing a short post about Github Actions? Because I wanted to use it for my latest Software-project, QR-codengrave. When at some distant point in the future, I have a new computer, don’t remember which IDE I used to build and run, test, and create assets with. Or, say, I have a corrupted virtual environment or
launch.json, I still want to be able to deploy that application, publish bugfixes or bump a release.
Although that’s not the usual argument for using CI/CD - typically the first reason being that collaboration on a piece of software becomes easier - still the appeal is strong enough to try Github’s automation called Github Actions.
I won’t try to run nightlys or have a continuous deployment pipeline as an end in itself, though, and only want an automated build, lint, test run, and deploy stuff when merging back to the production branch.
Because it was such a pain to get it right. I spent hours and hours pushing and hoping that, this time, the goddess of Github Actions would actually have my solution built without errors and at least run some of the tests.
Note: If I had been more diligent, I would have installed yet another tool like this to have my actions run locally, both saving time and the embarassment of tens of failed builds in a row. But I wasn’t.
I had quite some errors that appeared on the way of making QR-codengrave. Some I wasn’t even able to properly resolve so I had to utilize work-arounds. But let’s get to that later.
Github Actions uses
YAML to take orders for its pipelines. It is very well documented and provides many readily-working scripts for most scenarios and programming languages.
The so-called “Workflow file” looks like this:
# This workflow will install Python dependencies, run tests and lint with a single version of Python # For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python name: python_integrate on: push: branches: [ "master" ] pull_request: branches: [ "master" ] permissions: contents: read jobs: build: runs-on: ubuntu-latest steps: - uses: actions/checkout@v3 - name: Set up Python 3.10 uses: actions/setup-python@v3 with: python-version: "3.10" - name: Install dependencies run: | python -m pip install --upgrade pip pip install flake8 pytest pip install qrcodegen # Dependency install of qrcodegen if [ -f requirements.txt ]; then pip install -r requirements.txt; fi - name: Lint with flake8 run: | # stop the build if there are Python syntax errors or undefined names flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics # exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics - name: Test with pytest uses: GabrielBB/xvfb-action@v1 # Diverts tkinter GUI to a virtual frame buffer (VFB) with: run: | pytest
And it does not divert too much from the standard template that I was using first. The only things I added are the run
pytest entry and the usage of a virtual frame buffer to cirumvent issues with my GUI which makes the tests fail when the GUI tries to fire up and there’s no screen to show it on.
The following list of fails describe the errors and resolutions that I faced until I had a stable CI at around action run #50.
/opt/hostedtoolcache/Python/3.10.9/x64/lib/python3.10/importlib/__init__.py:126: in import_module return _bootstrap._gcd_import(name[level:], package, level) test/test_machinify_vector.py:4: in <module> from bin.platform.machinify_vector import MachinifyVector, Tool, EngraveParams E ModuleNotFoundError: No module named 'bin'
I placed my source files under
/bin and made it available to a local instance of PyInstaller (the tool I chose to render my python sources into an executable under Windows). Upon upload, Github Actions couldn’t relate to the paths I chose and had that error prepared for me.
Add an empty file with name
__init__.py into the
/bin folder. This will flag anything in that folder as package, thus making it available to GH actions.
So what looked like a directory mismatch actually was a package-not-found error that my IDE didn’t have as it knew which files I had created.
This error occurs because I was trying to put two workflows into one file. Somehow Github Actions seems to only accept a single one per file.
I had a lot of those and they were painful to resolve. The folder structure that I was targeting looked like:
- assets # images, persistence file, etc. - src # source files - test # pytest files for unit and integration testing - dist # build artifacts - build # build process files
But somehow, it was super hard to make my local IDE build environment, Pyinstaller, and the Github Actions Worker and its trigger of Pyinstaller to cooperate. One of those four parties would always complain that a path is missing or something else was wrong. That’s why I decided to abandon relative paths and instead use python’s
importlib_resources. But in the end, this turned out to be problematic with Pyinstuller on remote Github Actions.
What finally solved the problem was to move
src. This way, the
./ command couldn’t go wrong anywhere and although I don’t like the construct very much, I was tired of putting more time into that for a clean fix (If you know how to handle path pointing in Python for both local and remote, let me know in the discussions).
Later in the development process, I decided to add some integration tests into my solution. This way I wanted to make sure that child windows would actually perform the callbacks to update the main application, and vice versa e.g. that the persisted tool list is cascaded into the tool configuration window.
These tests were running locally but with a disadvantage: When I simulated an error or warning case, the corresponding
messageBox would insist to be closed manually by the user before continuing the tests. Which was just a bit annoying for me in the IDE to do, but would just be impossible to do from within Github Action’s CI pipeline.
So I bit the bullet and added a wrapper class to tkinter’s
messageBox so I could inject a mock that wouldn’t actually trigger the popup:
class MsgBox: """Re-implementation due to testing purposes: With this trick, we are able to mock these windows so we do not have to wait for users to manually close the dialog, unblocking the application again.""" def showinfo(self, title, message): showinfo(title=title, message=message) def error(self, title, message): showerror(title=title, message=message)
Although in theory it should be possible also to invoke the popup’s “OK button” from within the test, I wasn’t able to have that automated reliably.
When this worked, I pushed to Github with high expectations - and got another beautiful error:
_tkinter.TclError: no display name and no $DISPLAY environment variable
At least that was easy to understand: Tkinter just didn’t know where to draw the windows - seems legit when there’s no display connected.
So I searched the internet and found a single line of code that miraculously solved my problem by introducing a virtual frame buffer:
uses: GabrielBB/xvfb-action@v1 # Diverts tkinter GUI to a virtual frame buffer (VFB)
Now, on Push, Github Actions rained green ticks down on me which felt really good for a change.
engrave and cut
the struggle to successful builds
Faster Engrave Algo
for CNC: Decision aid
Decision aid, Pros & Cons
3D signs - tutorial
Job preparation and quick demo
of an audio amplifier
From investigation to problem solution
From Pixel to Vector graphics
Symptoms of insufficient spindle power
Beginner CNC issue: Speeds/Feeds
Automatic Z-referencing, Tool length, Tool changes
EdingCNC Settings & Variables
Software/Hardware setup & kinematics
Assembling Sorotec’s Basicline 0607
Switch box setup
Entry-point and questions
Circuit & Application examples
USB-PD power supply explained
Can I use a Metabo/CAS battery for my system?
How to set protective voltage & current limits
Current inrush limiters explained
Electronics knowledge: TVS diodes
Or how to kill your circuits
How to properly talk to VideoLan’s VLC player
Prototype build, issues, improvement ideas for Revision2
How To: First steps with FreeCAD
Professionalizing circuits for AnywhereAmps
AnywhereAmp Alpha’s simple single-supply preamp
Design, folding, stability, amplifiers
Evolving the mobile, foldable instrument combo
Fix ‘ERROR: favicon not found’
Export the instructions from Markdown to PDF! but how?
Why some images display OK locally but won’t on Github
How to make ToC sticky
How to solve issues with disappearing posts
Configure: landing page’s header image, Navigation