A checklist for all projects that are going live

The folks over at Spatie have open sourced the checklist they run each site/project through before hitting the “Go Live” switch. This kind of stuff should be open sourced more often.

The list is a good starting point to creating your own list, but of course your mileage may vary depending on the type of project you are building, the type of environment you run on, the scale your project runs at, the technology used, etc.

A checklist for all projects that are going live →

Transform HTML Content to Apple News / Google AMP / Facebook Instant Articles with Distro

distro

Automatically transform article HTML for third-party platforms such as Facebook Instant Articles, Apple News, Google AMP

“Just” make a curl request …

curl --data "
<h1>Distro.Mic example</h1>
<p>Text and media embed</p>
<iframe src=\"https://www.youtube.com/embed/M7lc1UVf-VE\"></iframe>
" https://distro.mic.com/1.0/format?output=apple-news

… and the content is “translated” to the specified format:

{
	"article": [{
		"text": "Distro.Mic example",
		"additions": [],
		"inlineTextStyles": [],
		"role": "heading1",
		"layout": "bodyLayout"
	}, {
		"text": "Text and media embed\n",
		"additions": [],
		"inlineTextStyles": [],
		"role": "body",
		"layout": "bodyLayout"
	}, {
		"role": "container",
		"components": [{
			"role": "embedwebvideo",
			"URL": "https://www.youtube.com/embed/M7lc1UVf-VE",
			"style": "embedMediaStyle",
			"layout": "embedMediaLayout"
		}],
		"layout": "embedLayout",
		"style": "embedStyle"
	}],
	"bundlesToUrls": {}
}

You can also install it as a dependency:

npm install distro-mic
import { format } from 'distro-mic';

const html = '<p>Article HTML</p>';
const output = format(html);

Distro →

Automatic Website Publishing with git-ftp.py on Mac OS X

On a recent project I collaborated on, deployment happened via git-ftp.py, a Python script which automatically publishes your git repository to an FTP server.

The script itself works with with an git-rev.txt file on the FTP server which keeps track of the last published commit. When deploying via git-ftp.py, the script only uploads the changes made since the last published commit.

~

Installing git-ftp

git-ftp.py relies on GitPython which itself can easily be installed using easy_install if you don’t want to worry about dependencies and the like.

  1. Install easy_install
    • Check your python version at the Terminal by running python (quit the python prompt by running exit() or hitting CTRL+D)
    • Download the correct .egg from http://pypi.python.org/pypi/setuptools/ (in my case: the one with 2.7 in its filename as my python version is 2.7.1)
    • Install it at the Terminal with sudo sh setuptools-0.6c11-py2.7.egg
  2. Install GitPython using easy_install
    • Just run sudo easy_install GitPython in Terminal and it’ll install GitPython along with all its dependencies for you
  3. Install git-ftp.py

~

Deploying with git-ftp.py

Before being able to deploy with git-ftp.py, you’ll have to provide it some FTP credentials. To do so, create a file ftpdata inside the (hidden) .git folder of your project (so the file is /path/to/project/.git/ftpdata). Set the contents of it to something like this:

[master]
username=projectname
password=projectpass
hostname=ftp.myproject.be
remotepath=/public_html
ssl=no

Note: you can add per-branch credentials if you want. Just duplicate the block and change [master] to the name of the branch you’re targetting.

Once configured, you can start deploying using this command:

python ~/Library/git-ftp.py

The script will output a list of all files that were uploaded.

Note: If you run git-ftp.py for the very first time on an FTP server containing an already published version of the project you should first place a git-rev.txt file on the server. Set the contents of the file to the SHA1 of the last commit which is already present on the server. Otherwise git-ftp.py will upload the whole repository which is not necessary.

~

Pro tip #1: set up an alias and save some time

In order to not having to type the entire publishing command all the time, set up an alias in .bash_profile. Run these commands at the Terminal:

echo "alias git-ftp='python ~/Library/git-ftp.py'" >> ~/.bash_profile
source ~/.bash_profile

That way you can deploying using this command:

git-ftp

~

Pro tip #2: Use a bare repository as a proxy

With git-ftp.py, it’s also possible to have a repository automatically publish when it’s being pushed upon. The git-ftp.py project has full instructions on how to set this up (haven’t used it myself).

~

Happy Deploying!

Did this help you out? Like what you see?
Consider donating.

I don’t run ads on my blog nor do I do this for profit. A donation however would always put a smile on my face though. Thanks!

☕️ Buy me a Coffee ($3)

Note: Whilst researching this I’ve stumbled upon git-ftp (not the same as git-ftp.py!), a shell script which – if I understand correctly – does about the same.

Alternatively – if you have Transmit – you could use Dock Send along with git-transmit.

If you’re more fond of the GitHub way of publishing (using a gh-pages branch), you’ll want to check out my own guide on Automatic website publishing with Git, GitHub-Style.

Finally, if you’re running all-Linux machines with SSH enabled, you’ll be better of with Capistrano.

Automatic website publishing with Git, GitHub-Style

One of the things I like about GitHub is the fact that it sports a gh-pages branch. Anything you push to it, is automatically published on your GitHub subdomain http://username.github.com/projectname/.

Inspired by this GitHub publishing flow, I’ve set up a likewise method on our web servers at work: a branch which gets published automatically onto our web server whenever we push code to it. This way we can eliminate the manual (and tedious) task of FTP’ing to the server (or opening up a network share) and copying the files onto the server in order to publish.

Somewhat related: Did you know, you can also (ab)use Dropbox as a web publishing tool? Not as robust as this method, yet it might be more fitting for some of you readers.

~

The Setup

The web server at work is a Windows 2008 R2 machine running a WAMP stack to serve the (mostly static) sites. Each hosted subsite is configured a vhost and is stored in its own subfolder on disk. Apache is being run as a separate user which has limited access to the filesystem.

<VirtualHost *:80>
    ServerName subdomain.ikdoeict.be
    UseCanonicalName On
    DocumentRoot "c:/apache2/htdocs/vhosts/subdomain.ikdoeict.be/wwwroot"
    php_admin_value open_basedir "c:/apache2/htdocs/vhosts/subdomain.ikdoeict.be;c:/php/temp"
</VirtualHost>

Next to the web server we also have a private Git server running Linux to/from wich code is pushed/pulled. Repositories are accessible via HTTPS and authenticating to this server is done via an HTTP Username & Password (not via a SSH key).

Of course your mileage may vary:

  • you might be running all-linux machines with shell access enabled (in which case you might be better off with Capistrano);
  • or you might have one single server function as both the web and Git server;
  • or you might have your code stored onto a public Git server (if you’re on GitHub, just set up a Post-Recieve URL and use this script as the update script);
  • or you might skip out on the gh-pages branch and want to deploy your master;
  • etc.

Either way: adjust/skip steps where necessary to reflect your setup/preferences 😉

~

Prerequisites

Install Git on the Web Server

As it’s a Windows server I’ve installed Git for Windows, choosing to run Git and the Unix Tools from the Windows Command Prompt.

Once installed, I’ve restarted the server to get the git command available anywhere on the server.

Create new repository on the Git server

Create a (bare) repository on the Git server and make sure that is externally accessible so that you can clone it on an other machine.

~

Gitting [sic] started

Clone, Branch & Develop on your local machine

Develop your site on your machine as you’d normally do (in my case: on the master branch).

git clone https://user@gitserver/project.git .
[build site]
git add .
git commit -m 'the one true commit'
git push origin master

Also create a gh-pages branch which will be the version to be deployed to the web server.

Note: If the gh-pages branch won’t differ from the master, feel free to skip out on it and deploy the master on the web server later on.

git branch gh-pages
git checkout gh-pages
[make changes, such as adding a Google Analytics Tracking Code]
git add .
git commit -m 'changes for online version'
git push origin gh-pages

Clone on the Web Server

On the web server, clone the project into the vhost’s DocumentRoot and activate the gh-pages branch so that the web server will serve that version.

cd c:/apache2/htdocs/vhosts/subdomain.ikdoeict.be/wwwroot
git clone https://user@gitserver/project.git -b gh-pages .

Note: If you already have a clone on the web server, or mistakingly have cloned the master, run these commands instead

git branch -f gh-pages origin/gh-pages
git checkout gh-pages
git branch -d master

Be sure to verify with your favorite text editor that you’ve got the correct version. If it’s correct you’ve successfully deployed your site onto the web server. Break out the champagne, but don’t open it yet, we’ve got some more work to do.

Security Alert!

You might not know this but whenever you clone a Git project onto disk, you’ll end up with a (hidden) .git directory in the root of your project. In that directory, everything about the repository clone is stored: commits, branches, hooks, ignore files, remotes, etc.

Now that you’ve deployed the project onto the web server, that .git directory will also be present in your vhost DocumentRoot, meaning that it — and its files — are now publicly accessible via http://subdomain.ikdoeict.be/.git/

To prevent this adjust your apache config to disallow the .git folder from being accessed over HTTP.

<VirtualHost *:80>
    ServerName student.ikdoeict.be
    UseCanonicalName On
    DocumentRoot "c:/apache/htdocs/vhosts/student.ikdoeict.be/wwwroot"
    php_admin_value open_basedir "c:/apache/htdocs/vhosts/student.ikdoeict.be;c:/php/temp"
    
    <Directory "c:/apache/htdocs/vhosts/student.ikdoeict.be/wwwroot/.git">
        Order allow,deny
        Deny from all
    </Directory>
    
</VirtualHost>

~

Getting the server to fetch updates

Deploying changes

Now, how to get changes onto the web server? Make changes on your local machine as you’d normally do, and push them upstream when commited. Be sure to merge your changes in the gh-pages branch as that’s the version that’s served on the web server.

git checkout master
[make changes]
git add .
git commit -m 'changes'
git push origin master
	
git checkout gh-pages
git merge master
git push origin gh-pages

On the web server, do a git pull to get the latest version

git pull origin gh-pages

~

Automating Deployment

Prerequisite: “standalone” git pull

In order to automate deployment, git pull should run fine without any user intervention such as requiring to enter a password.

As our setup requires HTTP username + password authentication it’ll ask for the password each time I push/pull something. In order to bypass this edit .git/config so that the remote url contains the password. If you’re doing a fresh clone, you can clone the repository as https://user:pass@gitserver/project.git straightaway.

Beware though: everything is stored plaintext! If other people have access to the machine, this isn’t a good idea! Also be sure to have implemented the security step above.

PHP, do your thing!

Up until now we can deploy changes onto the server, yet it still requires us to log in to the server and manually invoke a git pull. What if we could just open up a webpage in our browser which does the updating for us?

Luckily for us, PHP has a built-in command shell_exec, which allows you to execute a command via the shell and which returns the output. Given this, it’s fairly easy to knock up a PHP script that executes a git pull

<php
    echo nl2br(shell_exec('git pull origin gh-pages 2>&1'));

Note: the 2>&1 at the end of the commmand is to route encountered errors to the output.

Add this file on your local machine to your gh-pages branch …

git checkout gh-pages
[create update.php]
git add .
git commit -m 'update script'
git push origin gh-pages

… and — for the final time — do a manual git pull on the web server.

git pull origin gh-pages

Once the file is on the web server, you can from then on update the version on the server by visiting http://subdomain.ikdoeict.be/update.php. The file will give output when done.

Note: Since Apache is running as a limited user, you must give that user R/W permissions on your DocumentRoot so that it can perform the git pull and do all basic CRUD file manipulations. You can test this by making a local change, pushing it, and visiting update.php: if no error occurs, it’s all fine and dandy!

Putting the magic in automagic

Deploying new versions goes smooth by now, but it’s not fully automated … yet.

You might not know this but Git supports hooks. Hooks are executed after a certain action is performed and are stored in .git/hooks/. For example: after a commit, you could let a hook send out a tweet with your commit message.

One of the hooks that’s interesting for us is the post-receive hook, a server-side hook which is executed after a client has performed a push. We can take this hook, and let it call the update.php script for us. This way, we just have to push, and the web server will be updated automagically.

First, create the hook by renaming the sample provided

cd .git/hooks/
mv post-receive.sample post-receive
chmod +x post-receive

Second — and most important — adjust the hook’s content so that it calls the update.php script

#!/bin/sh
curl -s http://subdomain.ikdoeict.be/update.php

Tip: If you want some actual feedback to see if everything works fine, adjust update.php so that it sends you an e-mail.

Happy deploying!

Did this help you out? Like what you see?
Consider donating.

I don’t run ads on my blog nor do I do this for profit. A donation however would always put a smile on my face though. Thanks!

☕️ Buy me a Coffee ($3)

Using Dropbox as an automatic and free Web Publishing solution

In addition to having moved my mail to Google Apps for your Domain I decided to take it to the next level about a month ago by using Dropbox for storing both personal as professional (viz. 3RDS-related) data in the cloud. This decision made me more mobile than ever, as I can access any file, any time, any place; This as well from my PC as from my Mac as from any other Computer that’s connected to the internet. Over the weekend I raised the bar again and started experimenting with Dropbox resulting in a method by which one can (ab?)use that very same Dropbox as an automagic Web Publishing Solution (as an alternative to SVN and other technologies).

Continue reading “Using Dropbox as an automatic and free Web Publishing solution”