Speed up build times with this little Git trick

When building applications on build pipelines like GitHub Actions, Google Cloud Build, CircleCI, etc. every second counts. Here’s this small trick I use to speed up build times: when cloning the repo from its Git source, I instruct Git to do a shallow clone of the single branch it is building.

πŸ’‘ If you’re running a prebuilt “git clone” step on your Build Pipeline it most likely already uses this trick. Be sure to double check though.

~

Shallow Cloning

When doing a git clone you, by default, get the entire history of the project along with that. Take this clone of Laravel for example:

$ git clone [email protected]:laravel/laravel.git
Cloning into 'laravel'...
remote: Enumerating objects: 19, done.
remote: Counting objects: 100% (19/19), done.
remote: Compressing objects: 100% (15/15), done.
remote: Total 32004 (delta 5), reused 11 (delta 3), pack-reused 31985
Receiving objects: 100% (32004/32004), 9.94 MiB | 6.98 MiB/s, done.
Resolving deltas: 100% (18934/18934), done.

That’s a whopping 32004 objects totalling Β±10MiB that have been downloaded, even though the default Laravel branch only counts 66 files spread across 36 directories.

The objects contained in this Β±10MiB make up the entire history of every file and folder the project. To build the project we don’t really need all that, as we’re only interested in the latest version of each file and folder. By leveraging the --depth argument of our git clone command, we can enforce just that. This is what we call Shallow Cloning.

$ git clone --depth 1 [email protected]:laravel/laravel.git
Cloning into 'laravel'...
remote: Enumerating objects: 108, done.
remote: Counting objects: 100% (108/108), done.
remote: Compressing objects: 100% (88/88), done.
remote: Total 108 (delta 6), reused 49 (delta 1), pack-reused 0
Receiving objects: 100% (108/108), 41.80 KiB | 535.00 KiB/s, done.
Resolving deltas: 100% (6/6), done.

That’s a much speedier clone with only 108 objects, totalling a mere Β±40KiB!

πŸ€” You could argue that 10MiB worth of objects is still OK to clone, but think of scenarios where you have a big β€œmonorepo” with plenty of branches … then you’ll be talking about hundreds of wasted MiBs, if not GiBs.

~

Single Branch Cloning

When building a project you’re building only one certain branch. Information about the other branches is irrelevant at that time. To directly clone one specific branch we can use the --branch option to target said branch. With that alone we won’t get there though, we we still need to discard information about other branches. This is where the --single-branch option comes into play:

$ git clone --branch 3.0 --single-branch [email protected]:laravel/laravel.git
Cloning into 'laravel'...
remote: Enumerating objects: 20392, done.
remote: Total 20392 (delta 0), reused 0 (delta 0), pack-reused 20392
Receiving objects: 100% (20392/20392), 5.79 MiB | 853.00 KiB/s, done.
Resolving deltas: 100% (12731/12731), done.

Here we’ve cloned only the 3.0 branch of Laravel, resulting in roughly 10000 fewer objects to be downloaded.

By checking the contents of git branch -a we can also verify that other branch info has not been fetched:

$ cd laravel
$ git branch -a
* 3.0
  remotes/origin/3.0

~

Shallow Cloning + Single Branch Cloning

By combining both we can download only the latest files of a specific branch. Since the use of --single-branch is implied when using --depth, we can drop the former and our command will look like this:

$ git clone --depth 1 --branch <branchname> <repo>

Here’s an example downloading the Laravel 3.0 branch:

$ git clone --depth 1 --branch 3.0 [email protected]:laravel/laravel.git
Cloning into 'laravel'...
remote: Enumerating objects: 545, done.
remote: Counting objects: 100% (545/545), done.
remote: Compressing objects: 100% (465/465), done.
remote: Total 545 (delta 78), reused 293 (delta 45), pack-reused 0
Receiving objects: 100% (545/545), 1.34 MiB | 832.00 KiB/s, done.
Resolving deltas: 100% (78/78), done.

~

With this in place you’ll see your build times drop by minutes, especially when working on a monorepo with many branches.

Did this help you out? Like what you see?
Thank me with a coffee.

I don't do this for profit but a small one-time donation would always put a smile on my face. Thanks!

β˜•οΈ Buy me a Coffee (€3)

🐳 Building Docker Images on Google Cloud Build? Check out this trick to enable caching.

Published by Bramus!

Bramus is a Freelance Web Developer from Belgium. From the moment he discovered view-source at the age of 14 (way back in 1997), he fell in love with the web and has been tinkering with it ever since (more …)

Leave a comment

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.