On microservices and distributed architectures

On microservices and distributed architectures

Like the boiling frog, we often fail to appreciate just how significantly the infrastructure upon which we rely as developers has improved over the last decade. When I began working with the Rails framework in 2010, everything from the hardware we used for local development, to the infrastructure upon which we tested and deployed, was positively anemic by today’s standard.

My personal laptop, reasonably high-end for the time, had a 5400 RPM spinning disk and 2 GB of RAM. SSDs were exotic, even on servers. Nowadays, you can get bare metal servers with 512gb-1tb of RAM, 2x multi-core CPUs and terabytes of fast SSD storage for a price that is perfectly reasonable for even small companies. Similarly, you can easily and cheaply launch fleets of high-spec virtual servers with providers like Amazon Web Services and DigitalOcean at minutes’ notice.

In many ways, it seems to me that we are often basing architectural decisions on imagined constraints. In my experience, a decision to embrace a microservices architecture should not follow primarily from concerns about scalability.

High-performance logging from Nginx to Postgres with Rsyslog

High-performance logging from Nginx to Postgres with Rsyslog

While there are excellent purpose-built solutions for general log storage and search, such as Librato and the ELK Stack, there are sometimes reasons to write log data directly to Postgres. A good example (and one we have recent experience of at Superset) is storing access log data to present to users for analytics purposes. If the number of records is not explosive, you can benefit greatly from keeping such data closer to your primary application schema in PostgreSQL (even scaling it out to a separate physical node if need be by using something like Foreign Data Wrappers or the multi-database faculties of your application framework).

For our own application, we have a few geographically-distributed Nginx hosts functioning as “dumb” edge servers in a CDN. These Nginx hosts proxy back to object storage to serve up large media files to users. For each file access we want to record analytics information such as IP, user agent, and a client identifier (via the Nginx userd module, to differentiate unique users to the extent that it is possible).

A review of Sendy – driving the cost of maintaining your mailing list towards zero

A review of Sendy – driving the cost of maintaining your mailing list towards zero

I’ve been using Sendy on several of my sites over the last few years and wanted to write a review of it, since most of the reviews I’ve seen out there neglect a lot of what I think are its best features.

Sendy is a sort of self-hosted Mailchimp. It is a PHP application with very few dependencies (MySQL and a PHP-compatible web server like Apache) that you install on your own server. You pay a one-time fee of $59 US for a perpetual license to your major version. Major versions updates (of which there has only been one in the ~3 years I’ve held my Sendy license) are $29.

Implementing flexible, Stripe-style API authentication in Ruby on Rails

Having dealt with many uncomfortable API authentication schemes in the last decade, I wanted to share the scheme that we’ve found most sustainable for our Ruby on Rails applications. This scheme aligns with that used by services like Stripe. It allows clients to authenticate either via HTTP Basic Auth or an X-AUTH-TOKEN custom header.

What it will look like

API interactions with the Authentication scheme we are implementing will look as follows:

In using the -u flag, curl is authenticating via HTTP Basic Auth. The client’s API key can also be passed via an X-AUTH-TOKEN header, which you may find more convenient for implementing clients depending on the features and conventions of the underlying HTTP library you are using.

An intro to Stimulus JS: well-factored JavaScript for server-rendered applications

An intro to Stimulus JS: well-factored JavaScript for server-rendered applications

Stimulus is a JavaScript framework from Basecamp that provides consistent conventions and hooks for JavaScript that manipulates the DOM in server-rendered applications. It aims to fill some gaps that have always existed for developers who embrace a traditional server-rendered paradigm (and who may also be using libraries like Turbolinks or PJAX) but who also need to integrate one-off functionality in JavaScript. Stimulus does not at all concern itself with client-side (nor isomorphic) rendering and emphatically does not aim to be a heavy-client app framework like React, Angular or Vue.js. Instead it was created to support apps that are server-rendered first, and that rely on custom JavaScript where appropriate for UI enhancement.

In the existing paradigm, at least as concerns Ruby on Rails applications, it has been up to the developer to decide how to structure whatever custom JavaScript they have beyond UJS and SRJ responses. Stimulus gives us strong conventions to replace the ad-hoc or custom/bespoke approaches we may have previously taken to such work.