SlashDeploy has been slowly gathering momentum over the past two months. How do I know this? Well I check the KPIs (Key Performance Indicator). Right now there is only one KPI: traffic to this blog. Hopefully the more eyes on blog turn into leads and then clients. I use gaug.es for simple page view tracking. I check the numbers myself every couple of days. Instead I want to receive a weekly summary. Second, I want to track all future KPIs in a single place. How can these business requirements be translated into a technical solution? First, this requires a single place for KPI data points. Second a way to summarize them into easy to read reports. stathat works perfectly for this use case. It is also free at this usage level. Next step is how to get data from gauges into stathat?

This is a straight forward. There are cron tasks that scrape data from various sources and push to another. The data sources are commonly HTTP APIs so there is nothing uncommon there. The next question is how to run this this cloud and adopt continuous delivery? It turns out that AWS Lambda in combination with a cron style event source is a fantastic solution to this problem.

Developing

I use docker for pretty much everything these days. I opted to use Node.js for my Lambda functions because I am most familiar with it compared to Java or Python. Writing and testing Lambda functions is just like testing any other piece of Node.js code. The KPIs are scraped from HTTP APIs so I went with request, bluebird, and mocha for the key libraries. Implementing the TDD flow with docker & node.js was a bit strange because I did not want to rebuild a image every time for every test run. I also did not want to commit node_modules to SCM. I came up with a halfway solution that meets my requirements. I plan to use this flow for future Node.js projects. It goes like so:

  1. Generate node_modules via npm install in a docker container.
  2. Tarball node_modules
  3. Commit tar vendor/node_modules.tar.gz
  4. Extract tar file to disk before building docker image (npm install not needed in the Dockerfile).
  5. Build docker image by simple COPY . /app.
  6. Use docker image in combination with current directory volume mount to run arbitrary file changes (e.g. red-green-refactor).

I prefer to make things as repeatable and prevent as many known possible failure cases. This makes the dependency tree the same for everything everywhere and removes npm from the test and deployment pipeline. The Makefile below coordinates this whole process. You can also check the source.

Note that Docker is only used for testing. It is not part of the production system. There are other few goodies in there:

  • make check: Smoke the system for software dependencies
  • make test: Runs tests with volume mount to prevent image rebuilding
  • make test-shellcheck: run build script through shellcheck.

Deploying

Deploying requires two CloudFormation stacks. The first stack creates the S3 bucket for Lambda to read code from. The second stack creates the Lambdas and associated resources. Unfortunately it is not possible to have the S3 bucket in the same stack because the bucket must exist with the release artifact before deploying the Lambda stack. I have been using ansible over the past month or two and found it work well for automating bits of infrastructure and simple deployment pipelines. Ansible and CloudFormation work extremely well together because Ansible can create/update CloudFormation stacks as needed and will wait for the operations to complete.

The process goes like this:

  1. Create a zip file with node_modules and appropriate source functions
  2. Run a smoke test against the release artifact
  3. Provision the prerequisite stack
  4. Upload the zip file to S3
  5. Provision the lambda stack with artifact on S3
  6. Verify deploy by test firing the lambda function(s)

Here is the complete ansible playbook and CloudFormation template. Deploys happen via SlashDeploy GitLab CI runners.

Take-aways

  • Adding a new Lambda requires updating the CloudFormation template. This may be annoying if there were hundreds of Lambda functions but it is not a real problem given the small scale.
  • Ansible + CloudFormation continue to prove to be a great pair for AWS related automation
  • I wish npm had a built in way to easily vendor dependencies. My solution works well enough but I think npm should take this area more seriously. Node is more prone to dependency problems because every npm install may generate a different dependency graph each time for each node in the dependency tree.
  • This solution does not use Lambda versions. Instead the running code is replaced right away. This would probably not work for Lambdas behind the API gateway.
  • Lambda with a scheduled event source is a perfect low maintenance cron type solution.

Finally view the complete source code. I encourage you check out Lambda and other things in the serverless space for your future projects. Thanks for reading!