A long (long) time ago I wrote something called henshin to generate static sites. It solved the problem that jekyll wasn’t generic enough, there wasn’t much to configure. Well actually there was, but…

Anyway, it quickly became this insane annoyance to (remember how to) setup in a useful manner. Instead of using yaml to configure it you used ruby — because it is so much richer — which lead to you reïmplementing parts of henshin in a magic init.rb file.

Reading through the code scares me sometimes…

module Henshin

  class File

    # @abstract You will want to implement {#raw_text}, {#path} and maybe
    #   {#text}.
    #
    # This class implements all the functionality that is required to build or
    # serve a file. {Abstract} instances do not relate to a file in the file
    # system, use {Physical} in this case.
    class Abstract

      include  Henshin::Helpers, Safety, Comparable
      extend   Attributes

In the end I never even used it to make sites. It was always going to be quicker to write a little script than look back and work out how henshin was even meant to work.

Fast-forward to a few months ago and I decided to extract some of the common logic in scripts I’d written for generating little sites, for example hawx.me/stuff. These usually ended up being a few hundred lines of ruby that read some input, ran that through some templates and spat out some html.

A couple had the ability to serve the site locally and push it to a web server. These are generally the messy parts so I pulled them out into serveable and publishable, which do about 80% of the hard work of writing a script to generate a static site.

The great thing about this approach is that you don’t have to remember how to customise the available template engines, or how to change the way posts are read, or … anything. You only need to understand how two small gems work, the 20% glue code in between for generating the html is yours.

Code over config.

I made something a few weeks ago that I’ve been meaning to write about, but like all things that “I am going to write about” … I never did. So here we are, with the project a little stale. It was fun to do though.

It is simply a web proxy called Circumambient that spits out the request headers and timings to Redis using pub-sub. And when I say simple, it really is simple. The source is only just over a hundred lines. I’ve been playing with a few different uses and recently set it up to play nicely with Grafana, which I thought I would share.

First of all you will need Go installed. Then you can use go get to grab circumambient.

$ go get github.com/hawx/circumambient

Now we need to write a tool to subscribe to redis messages from circumambient and write them out to somewhere Grafana can read. I’m using InfluxDB for this, and NodeJs for the glue script.

Each request received is published to a redis channel called requests with a format of:

{
  "method": "...",
  "url": "...",
  "headers": {

  },
  "timestamp": 1397731289,
  "duration": 14331
}

And here’s the subscriber,

var host = 'localhost',
    port = '8086',
    username = 'root',
    password = 'root',
    database = 'test';

var influx = require('influx')(host, port, username, password, database),
    redis = require('redis').createClient();

influx.getDatabaseNames(function(err, names) {
  if (names.indexOf(database) == -1) {
    influx.createDatabase(database, function(err, success) {
      console.log("database created");
    });
  } else {
    console.log("database exists");
  }
});

function seriesNamer(obj) {
  return obj.url.path.slice(1).replace('/', '.') + '.' + obj.method.toLowerCase();
}

redis.on('message', function(channel, message) {
  if (channel == 'requests') {
    var obj = JSON.parse(message);
    var point = {
      time: new Date(obj.timestamp / 1e6),
      duration: obj.duration
    };

    var seriesName = seriesNamer(obj);

    influx.writePoint(seriesName, point, {}, function(err, success) {
      if (err == null) {
        console.log("wrote", seriesName);
      }
    });
  }
});

redis.subscribe('requests');

process.on('SIGINT', function() {
  redis.end();
  process.exit();
});

console.log('started app.js');

Now every time you hit your app through Circumambient we put the duration of the request into InfluxDB. Which can then be graphed in Grafana.

The idea is that instead of building with the view of “store everything we might need it in the future” you get an open system that small components sit around where each component is interested in one metric. If you need to now see x, you don’t have to hope that x was being recorded all along: you just write something to record x! (Obviously there is no way of recording the historical values of x, but that’s kinda the point.)

So, that’s Circumambient.