High-performance logging from Nginx to Postgres with Rsyslog

High-performance logging from Nginx to Postgres with Rsyslog

While there are excellent purpose-built solutions for general log storage and search, such as Librato and the ELK Stack, there are sometimes reasons to write log data directly to Postgres. A good example (and one we have recent experience of at Superset) is storing access log data to present to users for analytics purposes. If the number of records is not explosive, you can benefit greatly from keeping such data closer to your primary application schema in PostgreSQL (even scaling it out to a separate physical node if need be by using something like Foreign Data Wrappers or the multi-database faculties of your application framework).

For our own application, we have a few geographically-distributed Nginx hosts functioning as “dumb” edge servers in a CDN. These Nginx hosts proxy back to object storage to serve up large media files to users. For each file access we want to record analytics information such as IP, user agent, and a client identifier (via the Nginx userd module, to differentiate unique users to the extent that it is possible).

A review of Sendy – driving the cost of maintaining your mailing list towards zero

A review of Sendy – driving the cost of maintaining your mailing list towards zero

I’ve been using Sendy on several of my sites over the last few years and wanted to write a review of it, since most of the reviews I’ve seen out there neglect a lot of what I think are its best features.

Sendy is a sort of self-hosted Mailchimp. It is a PHP application with very few dependencies (MySQL and a PHP-compatible web server like Apache) that you install on your own server. You pay a one-time fee of $59 US for a perpetual license to your major version. Major versions updates (of which there has only been one in the ~3 years I’ve held my Sendy license) are $29.

Implementing flexible, Stripe-style API authentication in Ruby on Rails

Having dealt with many uncomfortable API authentication schemes in the last decade, I wanted to share the scheme that we’ve found most sustainable for our Ruby on Rails applications. This scheme aligns with that used by services like Stripe. It allows clients to authenticate either via HTTP Basic Auth or an X-AUTH-TOKEN custom header.

What it will look like

API interactions with the Authentication scheme we are implementing will look as follows:

In using the -u flag, curl is authenticating via HTTP Basic Auth. The client’s API key can also be passed via an X-AUTH-TOKEN header, which you may find more convenient for implementing clients depending on the features and conventions of the underlying HTTP library you are using.

An intro to Stimulus JS: well-factored JavaScript for server-rendered applications

An intro to Stimulus JS: well-factored JavaScript for server-rendered applications

Stimulus is a JavaScript framework from Basecamp that provides consistent conventions and hooks for JavaScript that manipulates the DOM in server-rendered applications. It aims to fill some gaps that have always existed for developers who embrace a traditional server-rendered paradigm (and who may also be using libraries like Turbolinks or PJAX) but who also need to integrate one-off functionality in JavaScript. Stimulus does not at all concern itself with client-side (nor isomorphic) rendering and emphatically does not aim to be a heavy-client app framework like React, Angular or Vue.js. Instead it was created to support apps that are server-rendered first, and that rely on custom JavaScript where appropriate for UI enhancement.

In the existing paradigm, at least as concerns Ruby on Rails applications, it has been up to the developer to decide how to structure whatever custom JavaScript they have beyond UJS and SRJ responses. Stimulus gives us strong conventions to replace the ad-hoc or custom/bespoke approaches we may have previously taken to such work.

Dedicated vs. Cloud vs. VPS vs. PaaS – a value comparison

Dedicated vs. Cloud vs. VPS vs. PaaS – a value comparison

I find myself increasingly evangelizing for awareness of infrastructure and ops practices because I believe that such awareness (or lack thereof) has cascading effects for application architecture and, ultimately, company success. Understanding the relative value of platforms can keep you on a path of rapid execution. Misunderstanding or neglecting it can get you into very dire situations.

I see many teams break out their applications into services prematurely, with immense consequence in terms of cognitive overhead and loss of velocity. Typically the decision is a consequence of a perception that they have hit a performance ceiling, when in fact they are still on some relatively weakly-provisioned PaaS.

I want to do a detailed value comparison to aid others in making informed infrastructure decisions.