Static Site work flow

A few months ago I started up this blog. After working with Content Management Systems (CMS) for a while I wanted to try a different approach that enabled me to post quickly without getting trapped spending too much time looking at WYSIWYG editors. See my First Post Static Site and all the aspirational intents behind it.

Now I’m a few months into the adventure and I know a lot more now. I’ve made a few refinements to my work flow, and decided to write them up for future reference. I decided to use plantuml to generate a sequence diagram that will help me walk through the process. (More on plantuml later).

Work flow Sequence Diagram

(Alternate better looking svg version: Work flow Sequence Diagram)

In this diagram, there is a bold sequential number that I will following through with details.

1) Public Viewing Existing Site

Since I have already got some pages published, I thought it was important to show those current viewers. Although not many, it is good to note that none of the background work flow is exposed to them until ‘deploy’ at step 11.

2) Creating a new document

To create a new blog entry, I use the command

$ hugo new 2018/07/IOTHouseDrama.md
content/2018/07/IOTHouseDrama.md created

This will create a new markdown document in the content folder. I have defined a year and month folder to partition my blog entries over time. This partitioning is not reflected in the way pages are presented on the site, it just used to collect things.

3) Edit Markup

The markup file is pre-populated with some seed information from the archtypes/detaults.md file. I haven’t gone too deeply into this file yet, but I know several things from experience.

date = is honored when rendering. If you set a future date, the page will not be rendered until that date has passed.
draft = true is honored by deploy. If you set draft to true, it will not be rendered until the flag is set to false.

I’m sure I will learn more about archetypes as I move along but right know these are things I know because they have caused some head scratching while working out the deployment process.

4) `hugo server` …

Eventually you’ll want to render the markdown and see what it looks like. I try very hard to delay this as long as possible. I find that I get more content written if I don’t get distracted by what it looks like. This is one of the major reasons I was drawn to markdown.

I run hugo with several command line parameters. I didn’t want to list them all out in the sequence diagram so here is the whole thing:

hugo server --buildDrafts \
--buildExpired \
--buildFuture \
--bind 0.0.0.0 \
--disableFastRender \
--baseURL $(DEST_URL)

The --buildExpired, --buildFuture and --buildDrafts are all used to make sure that the local web server presents all pages – even if they will not be rendered in the final output because of one of these options. --bind 0.0.0.0 enables me to run the test server on a remote box (my Linux development box) that is independent of my typical day computer (OSx). I chose to set --disableFastRender so that there is no doubt that all changes I made during edits are going to be picked up immediately by the test web server. If I had many pages, I might choose differently, but right now, CPU cycles are cheap. Finally `–baseURL $(DEST_URL) is set to the DNS name of the the Linux box so that when I navigate between relative pages, they all reference back into the same test host.\

5) memory render

Not much here, other than the hugo go app will fire up a web server that will render content in memory and present it as a little web server for access locally.

6 – 10) write, view, etc

The creative process in action right here. The hugo web server is watching the source directories and if any of the content changes, the server will re-render the pages. Once I get the general outline sketched, I spend most of my time working in markdown.

11 – 12) hugo

At some point, when I think the content has been refined enough, I’ll move on to rendering the final output and publishing the static site. This command is pretty easy.

hugo

This command, by default will render the markdown in the ‘content’ directory into web page layout in the public folder. I haven’t found any cases where I need add command line parameters to this one.

13) gsutil rsync

Finally, I chose to use Google Cloud Platform bucket as a final resting place for my static site. Now that the site has been rendered, I need to copy all directories and files from the public folder over to it’s final location in a Google bucket.

Fortunately, Google provides a utility gsutil to do just this. One of the sub-commands is specifically written for keeping a source and target directory structure in sync. Very similar to the Unix command rsync

gsutil -m rsync -d -c -R public $(GOOGLE_BUCKET)

The gsutil parameters use are:

-m is recommended to take advantage of multi-thread or process. Although not needed for my puny little case, it is probably very important on larger data transfers. rsync this sub-command identifies the operation to be performed. In this case it is going to synchronize the contents of a source and target directory structure. It tries to do this in a way that minimizes the data transfer operations. For example, if a file has not been modified in the source directory, then there is no need to copy it to the target again. The -d option will delete any files on the target that are no longer at the source. This is dangerous and the help documentation marks it as so. In my particular use case, it’s exactly what I want. If I decide to delete a markup, png or svg file from my site, I also want the rendered HTML file or referenced images to be deleted from the final static site. Use this option with caution. The -c option causes rsync to compare ‘equality’ by using a check-sum calculation rather than the time-stamp of the file. I need this because I also use git and may edit the markup from several different computers. I don’t want files re-transferring just because a date-time does not appear to be the most recent. In my case, the only true change to a file would be indicated by a check-sum difference.