Super fast npm install on Github Actions

The folks at Voorhoede share how they integrated the actions/[email protected] build step into their GitHub Workflow so that it caches npm install results.

Super fast npm install on Github Actions →

Speed up build times with this little Git trick

When building applications on build pipelines like GitHub Actions, Google Cloud Build, CircleCI, etc. every second counts. Here’s this small trick I use to speed up build times: when cloning the repo from its Git source, I instruct Git to do a shallow clone of the single branch it is building.

💡 If you’re running a prebuilt “git clone” step on your Build Pipeline it most likely already uses this trick. Be sure to double check though.

~

Shallow Cloning

When doing a git clone you, by default, get the entire history of the project along with that. Take this clone of Laravel for example:

$ git clone [email protected]:laravel/laravel.git
Cloning into 'laravel'...
remote: Enumerating objects: 19, done.
remote: Counting objects: 100% (19/19), done.
remote: Compressing objects: 100% (15/15), done.
remote: Total 32004 (delta 5), reused 11 (delta 3), pack-reused 31985
Receiving objects: 100% (32004/32004), 9.94 MiB | 6.98 MiB/s, done.
Resolving deltas: 100% (18934/18934), done.

That’s a whopping 32004 objects totalling ±10MiB that have been downloaded, even though the default Laravel branch only counts 66 files spread across 36 directories.

The objects contained in this ±10MiB make up the entire history of every file and folder the project. To build the project we don’t really need all that, as we’re only interested in the latest version of each file and folder. By leveraging the --depth argument of our git clone command, we can enforce just that. This is what we call Shallow Cloning.

$ git clone --depth 1 [email protected]:laravel/laravel.git
Cloning into 'laravel'...
remote: Enumerating objects: 108, done.
remote: Counting objects: 100% (108/108), done.
remote: Compressing objects: 100% (88/88), done.
remote: Total 108 (delta 6), reused 49 (delta 1), pack-reused 0
Receiving objects: 100% (108/108), 41.80 KiB | 535.00 KiB/s, done.
Resolving deltas: 100% (6/6), done.

That’s a much speedier clone with only 108 objects, totalling a mere ±40KiB!

🤔 You could argue that 10MiB worth of objects is still OK to clone, but think of scenarios where you have a big “monorepo” with plenty of branches … then you’ll be talking about hundreds of wasted MiBs, if not GiBs.

~

Single Branch Cloning

When building a project you’re building only one certain branch. Information about the other branches is irrelevant at that time. To directly clone one specific branch we can use the --branch option to target said branch. With that alone we won’t get there though, we we still need to discard information about other branches. This is where the --single-branch option comes into play:

$ git clone --branch 3.0 --single-branch [email protected]:laravel/laravel.git
Cloning into 'laravel'...
remote: Enumerating objects: 20392, done.
remote: Total 20392 (delta 0), reused 0 (delta 0), pack-reused 20392
Receiving objects: 100% (20392/20392), 5.79 MiB | 853.00 KiB/s, done.
Resolving deltas: 100% (12731/12731), done.

Here we’ve cloned only the 3.0 branch of Laravel, resulting in roughly 10000 fewer objects to be downloaded.

By checking the contents of git branch -a we can also verify that other branch info has not been fetched:

$ cd laravel
$ git branch -a
* 3.0
  remotes/origin/3.0

~

Shallow Cloning + Single Branch Cloning

By combining both we can download only the latest files of a specific branch. Since the use of --single-branch is implied when using --depth, we can drop the former and our command will look like this:

$ git clone --depth 1 --branch <branchname> <repo>

Here’s an example downloading the Laravel 3.0 branch:

$ git clone --depth 1 --branch 3.0 [email protected]:laravel/laravel.git
Cloning into 'laravel'...
remote: Enumerating objects: 545, done.
remote: Counting objects: 100% (545/545), done.
remote: Compressing objects: 100% (465/465), done.
remote: Total 545 (delta 78), reused 293 (delta 45), pack-reused 0
Receiving objects: 100% (545/545), 1.34 MiB | 832.00 KiB/s, done.
Resolving deltas: 100% (78/78), done.

~

With this in place you’ll see your build times drop by minutes, especially when working on a monorepo with many branches.

Did this help you out? Like what you see?
Thank me with a coffee.

I don't do this for profit but a small one-time donation would surely put a smile on my face. Thanks!

☕️ Buy me a Coffee (€3)

To stay in the loop you can follow @bramus or follow @bramusblog on Twitter.

🐳 Building Docker Images on Google Cloud Build? Check out this trick to enable caching.

How to set up PHP for use in a GitHub Action

To use PHP in a GitHub action there’s the magnificent setup-php action. It also allows for installing extensions and setting several php.ini directives.

steps:
- name: Checkout
  uses: actions/[email protected]

- name: Setup PHP
  uses: shivammathur/[email protected]
  with:
    php-version: '7.4'
    extensions: mbstring, intl #optional, setup extensions
    ini-values: post_max_size=256M, short_open_tag=On #optional, setup php.ini configuration
    coverage: xdebug #optional, setup coverage driver
    pecl: false #optional, setup PECL

Here’s an example that uses a build matrix:

jobs:
  run:    
    runs-on: ${{ matrix.operating-system }}
    strategy:      
      matrix:
        operating-system: [ubuntu-latest, windows-latest, macOS-latest]
        php-versions: ['7.0', '7.1', '7.2', '7.3', '7.4']
    name: PHP ${{ matrix.php-versions }} Test on ${{ matrix.operating-system }}
    steps:
    - name: Checkout
      uses: actions/[email protected]

    - name: Setup PHP
      uses: shivammathur/[email protected]
      with:
        php-version: ${{ matrix.php-versions }}
        extensions: mbstring, intl #optional, setup extensions
        ini-values: post_max_size=256M, short_open_tag=On #optional, setup php.ini configuration
        coverage: xdebug #optional, setup coverage driver
        pecl: false #optional, setup PECL

When building against a matrix, the documentation recommends to cache composer data to speed up the process.

shivammathur/setup-php

💡 This example workflow from Spatie’s laravel-fractal Pacakge is also worth a peek. Also builds against a matrix with caching, but in a slightly different way.

Run Lighthouse in a CI Pipeline using lighthouse-ci

Lighthouse CI is a set of commands that make continuously running, asserting, saving, and retrieving Lighthouse results as easy as possible.

npm install -g @lhci/[email protected]
lhci autorun --upload.target=temporary-public-storage || echo "LHCI failed!"

Comes with default configurations for Travis, GitHub Actions, Circle CI, GitLab CI, and Jenkins.

lighthouse-ci

GitHub CI Workflow for PHP applications

Mattias Geniar has shared his GitHub Workflow to make GitHub do the CI work for PHP applications:

on: push
name: Run phpunit testsuite
jobs:
  phpunit:
    runs-on: ubuntu-latest
    container:
      image: mattiasgeniar/php73

    steps:
    - uses: actions/[email protected]
      with:
        fetch-depth: 1

    - name: Install composer dependencies
      run: |
        composer install --prefer-dist --no-scripts -q -o;
    - name: Prepare Laravel Application
      run: |
        cp .env.example .env
        php artisan key:generate
    - name: Compile assets
      run: |
        yarn install --pure-lockfile
        yarn run production --progress false
    - name: Set custom php.ini settings
      run: echo 'short_open_tag=off' >> /usr/local/etc/php/php.ini
    - name: Run Testsuite
      run: vendor/bin/phpunit tests/

Tests get run in a custom mattiasgeniar/php73 Docker container, which contains PHP 7.3 and extensions (including imagick).

With some fiddling you can easily adjust this to:

  1. Run these tests for pull requests too (just like you can automatically run linters on pull requests)
  2. Extend it to become a CI/CD pipeline to automatically deploy your code too

A github CI workflow tailored to modern PHP applications →

Continuous Improvement as seen through the lens of Systems Thinking

Recording of a 1994 talk by Dr. Russel Ackoff on Continuous Improvement as seen through the lens of Systems Thinking.

This talk really was way ahead of its time. It’s talking about Continuous Improvement long before I even heard this term in the business of web development; it’s expressed in a very clear and calm language; it has a typical TED-talk style written all over it; it’s packed with one-liners; …–
Yes, I am impressed.

The system is not the sum of the behavior of its parts, it’s the product of their interactions.

There’s also a lengthier talk by Dr. Ackoff on the subject of Systems Thinking available on YouTube.

(via @mvuijlst)

Detox: E2E Testing in React Native

High velocity native mobile development requires us to adopt continuous integration workflows, which means our reliance on manual QA has to drop significantly. Detox tests your mobile app while it’s running in a real device/simulator, interacting with it just like a real user.

Here’s a sample test for a login screen:

describe('Login flow', () => {
    
  it('should login successfully', async () => {
    await device.reloadReactNative();
    await expect(element(by.id('email'))).toBeVisible();
      
    await element(by.id('email')).typeText('[email protected]');
    await element(by.id('password')).typeText('123456');
    await element(by.label('Login')).tap();
      
    await expect(element(by.label('Welcome'))).toBeVisible();
    await expect(element(by.id('email'))).toNotExist();
  });
  
});

wix/detox (GitHub) →
Testing in React Native — Jest & Detox →

Fastlane Screencast: Integrate fastlane into your Ionic Framework build process

fastlane are an awesome bunch of tools. Josh Holtz has recently started Fastlane Screencast, a website with videos/tutorials for explaining and implementing built-in fastlane tools, fastlane actions, third-party fastlane plugins, continuous integration, and anything else that fastlane can possibly do.

The first video covers integrating fastlane into your Ionic Framework build process:

A second tutorial – covering dotenv and environment variables – is already up too 🙂

Fastlane Screencast →
Fastlane Screencast: Integrate fastlane into your Ionic Framework build process →

PHPCI – Continuous Integration for PHP Projects

phpci

PHPCI is a free and open source continuous integration tool specifically designed for PHP. Built with simplicity in mind and featuring integrations with all of your favourite testing tools, we’ve created the very best platform for testing your PHP projects.

Looks interesting. Host it yourself, or make use of one of the hosted plans.

PHPCI – Continuous Integration for PHP Projects →

(via)