Michael Shogren's Blog

In the last post I outlined my decision to use Snap CI as my CI tool. I want to test my blog for broken links, well-formed HTML, valid Atom feed and possibly other things in the future. I want these tests to be automated so I can't forget or be too lazy to run them and I want the failing of these tests to prevent changes to my blog from being published.

I created a Snap-CI account and I have added a link to the builds for my blog to the ALSL Links navigation. Snap-CI automatically links to your Github account, and it allowed me to setup my first build of the blog repo in no time using Ruby 2.1.5. I followed this guide and created my Gemfile as suggested here.

The commands I added to my build definition were as follows:

bundle install
bundle exec jekyll build
bundle exec htmlproof ./_site --allow-hash-href true --check-html true --check-external-hash true

The first command downloads the required rubygems, the second step builds the site, and the third step uses HTML::Proofer to test the site's links and HTML. When I saved my build definition, the Snap-CI server immediately ran it against my existing code in the master branch. I found a couple of typos in the links and fixed them and the build ran again and passed. My concern that the version of ruby used and the commands used are not stored in source control still stands, since if I was using Travis they would be in my .travis.yml

Once I had a good build I turned my attention to ensuring that failing tests would prevent me from publishing the changes that caused the problem. I always liked the concept of a pre-tested commit that I first saw in TeamCity. It is done there in a way that is independent of the source control technology used. After some investigation and testing in Snap-CI I found that I could achieve something similar in Snap-CI using the Github flow. Each time I am writing a new post or working on a technical feature of my blog, I create a new branch. These feature branches should ideally be short-lived as continuous delivery depends on trunk based development. If I was coding something that had a separate deployment step I would prefer to use feature toggles and force all development to be done in master. I think in the case of a blog hosted in Github Pages I think an exception to this trunk-based development rule can be made. When I am done with my branch I can create a pull request from the branch. I set Snap-CI to monitor pull requests. When a pull request is created it tests the request merged into the trunk and notifies Github whether it is successful or not. I can then see that information on the pull request and decide whether or not to merge it into the master branch. Once it is merged I can delete the branch. Below is what the failed and successful builds look like when reviewing the pull request in Github.

An alternative to using pull requests and manually merging them is provided by Snap-CI. Automatic branch tracking will build the new branch when it is created and automatically merge it to another branch if the build succeeds. This target branch need not even be master, though I don't want to complicate things too much. This automation removes the manual step of merging a pull request so it is closer to the continuous delivery ideal of relentless automation. The automated creation of new build pipelines for each branch also solves a problem I noticed with Snap-CI's pull request tracking. When you change the build definition on the master branch you have to remember to update the build on the pull request pipeline. Snap-CI prompts you to do so and the process is simple enough but there is still the possibility of a pull request running a build definition that is out of date.

Once I had my build setup and working I added a build status badge to the README.md of my blog repo. I also took the opportunity to set up some Slack integrations so I will be notified when various things happen. It is receiving notifications for blog comments, changes in the ALSL Task Board in Trello, changes in Github and build notifications from Snap-CI.

I think that is enough detail for this post but I am not quite finished with the tests for the blog website. I am a little dissatisfied with HTML::Proofer, since it does not make any suggestions about links that are redirected the way the W3C Link Checker does. I will try to find a way to test my site against the link checker, the W3C Markup Validator and the W3C Feed Validator as soon as I can.

0 Comments

I wasn't going to blog much about the blog (too meta!) but I couldn't pass up the opportunity to get started on something very important to the ALSL project - Continuous Integration (CI). In fact what the ALSL project is going to strive for is Continuous Delivery (CD), but I don't know if we are ready yet. This blog needs to be tested for broken links, properly formed HTML, etc before it is published, and I am going to use a CI tool to do that, since the deployment or publishing is done automatically by Github Pages whenever I commit to the master branch.

I am not going to get into too much detail on CI or CD as the linked articles do a good job but I will reiterate two things:

1. I want to put everything in source control
2. I want to automate relentlessly.

A future post will be on provisioning a development environment for me to use and is going to get into both these points at great length, because I want even my environment to be easily setup in a repeatable and robust way, since in my experience I will have to do it more than once.

There are a vast list of CI tools out there and I have looked at quite a lot but the ones I have a little more to say on are these

• Snap CI
• Travis
• Appveyor
• TeamCity
• Visual Studio Team Services (VSTS)

My first requirement is a cheap hosted solution, and each of these is capable of that. However the hosted TeamCity platform at CodeBetter is a little limited, which is a shame, since I have extensive experience with TeamCity already, and like it very much. Appveyor and VSTS are focused on Windows, which may be helpful since ALSL might leverage .NET in the future. For now however, I would prefer the flexibility of a linux based CI tool. After all I should still be able to build Mono or .NET Core projects using Snap-CI or Travis, and one of the big advantages they have is that they use containers for the builds, and these containers are very configurable.

Travis is very popular and there are a couple of things I like about it. The most important thing is that your build definition is stored in source control as a .travis.yml file. This agrees with the first goal I have above on my journey towards CD. Another thing I like about it is the fact that it covers the most programming languages of any of the CI tools I have seen, and the documentation for customizing the build environment is excellent. The containers Travis uses for builds are built using Ubuntu, a Debian derived distribution I am very familiar with.

There is one thing missing from Travis that a really like about Snap-CI which is why I am going to try working with Snap-CI for now. That is The focus on build pipelines. The pipeline is defined by Jez Humble in his book Continuous Delivery as:

At an abstract level, a deployment pipeline is an automated manifestation of your process for getting software from version control into the hands of your users.

At the place I work, we have built a pipeline in TeamCity with a tree of 57 build and deployment stages that compiles, tests, deploys to an integration environment, test again and packages artifacts for our project. It is a good start, but it doesn't even deploy anything to an acceptance or production environment yet. My decision about which CI tool to use comes down to the fact that it looks much easier to do something like that in Snap-CI than in Travis. One the other hand, build commands are stored on the Snap-CI server and I am not sure if they are versioned and kept for ever. Also, Snap-CI uses one of the RedHat based distros for containers, so I might have to learn a bit more about yum in order to configure my build environments.

For now I have decided to try out Snap-CI, but I am not opposed to the idea of trying out other tools as the need arises. ALSL might eventually need builds done on Windows, for example. I will have a post up later about how I get on with setting up Snap-CI to run tests on this blog.

0 Comments

In my first post I talked about using this blog to explore modern software development and delivery practices. There is an awful lot to cover, and I only have vague plans, which are definitely subject to change. When I am organizing my thoughts for future blog posts I am really organizing tasks that need to be completed by my fictional software company ALSL. As this is very speculative in nature and difficult to time-box, and because I am doing this on my own, I am going to use a Kanban board to organize these tasks.

I already use a Kanban board to organize my tasks, and that is hosted by KanbanTool. I like that a lot and I have configured everything the way I like. It uses two swimlanes - one for work tasks that are not part of the project I work on (those tasks are in Rally), and one for personal to-dos. The columns I use are pretty standard, but I have divided today's tasks into four using the Eisenhower method, which helps me prioritize my day. That isn't necessarily going to be helpful for the tasks I carry out for the ALSL project, and KanbanTool doesn't allow publicly viewable boards.

For that reason I am going to try using Trello. I looked at it previously and I saw that it didn't do everything that KanbanTool did but that it was more flexible since it isn't really just a Kanban tool at all. Anecdotally, it is used fairly heavily by agile software projects, which is what ALSL really is, and its popularity means that it is easy to find information about integrating it with other software development tools such as source control and continuous integration tools. If Trello doesn't work out I might consider Pivotal Tracker, though that seems like it might be overkill for now.

Anyway, the ALSL Kanban board is up and I will create a link of the main page shortly. The first task I created is to improve the blog. I created it by following this article by Jonathan McGlone, but there is a lot left to do. I might not get it all done at once, but I definitely need to look at some things mentioned at the end of that article such as commenting and analytics, and I also need to sort out some basic styling based on HTML5 Boilerplate or Bootstrap. Another thing I need to do immediately is figure out what workflow I am going to use as far as drafting posts before I publish them. For drafts I am thinking of something along these lines - Public drafts in Jekyll.

I may or may not blog in much detail about the changes I make to the blog, but the next task that AI need to get done for ALSL is creating a development environment for me to use.

0 Comments