circumambient

I made something a few weeks ago that I’ve been meaning to write about, but like all things that “I am going to write about” … I never did. So here we are, with the project a little stale. It was fun to do though.

It is simply a web proxy called Circumambient that spits out the request headers and timings to Redis using pub-sub. And when I say simple, it really is simple. The source is only just over a hundred lines. I’ve been playing with a few different uses and recently set it up to play nicely with Grafana, which I thought I would share.

First of all you will need Go installed. Then you can use go get to grab circumambient.

$ go get github.com/hawx/circumambient

Now we need to write a tool to subscribe to redis messages from circumambient and write them out to somewhere Grafana can read. I’m using InfluxDB for this, and NodeJs for the glue script.

Each request received is published to a redis channel called requests with a format of:

{
  "method": "...",
  "url": "...",
  "headers": {

  },
  "timestamp": 1397731289,
  "duration": 14331
}

And here’s the subscriber,

var host = 'localhost',
    port = '8086',
    username = 'root',
    password = 'root',
    database = 'test';

var influx = require('influx')(host, port, username, password, database),
    redis = require('redis').createClient();

influx.getDatabaseNames(function(err, names) {
  if (names.indexOf(database) == -1) {
    influx.createDatabase(database, function(err, success) {
      console.log("database created");
    });
  } else {
    console.log("database exists");
  }
});

function seriesNamer(obj) {
  return obj.url.path.slice(1).replace('/', '.') + '.' + obj.method.toLowerCase();
}

redis.on('message', function(channel, message) {
  if (channel == 'requests') {
    var obj = JSON.parse(message);
    var point = {
      time: new Date(obj.timestamp / 1e6),
      duration: obj.duration
    };

    var seriesName = seriesNamer(obj);

    influx.writePoint(seriesName, point, {}, function(err, success) {
      if (err == null) {
        console.log("wrote", seriesName);
      }
    });
  }
});

redis.subscribe('requests');

process.on('SIGINT', function() {
  redis.end();
  process.exit();
});

console.log('started app.js');

Now every time you hit your app through Circumambient we put the duration of the request into InfluxDB. Which can then be graphed in Grafana.

The idea is that instead of building with the view of “store everything we might need it in the future” you get an open system that small components sit around where each component is interested in one metric. If you need to now see x, you don’t have to hope that x was being recorded all along: you just write something to record x! (Obviously there is no way of recording the historical values of x, but that’s kinda the point.)

So, that’s Circumambient.