Using Multiple Databases in Rails 6 to Log Access Requests

Imagine a situation where you need to log every attempt at accessing your application regardless of how many application servers are deployed and when. Text logs aren’t enough and can be difficult to aggregate across multiple disparate filesystems. Not to mention, retrieval of any given text-based record (or series thereof) after several years can be a significant undertaking. So you decide to log that information in your application database. But how do you scale the data storage for logs independently of data storage for your application?

We encountered this situation recently while working on a medical API to be consumed by a custom built mobile application. Targeted for deployment in the United States, the law known as HIPAA – the Health Insurance Portability and Accountability Act – will require us to log all attempts to access PHI, or protected health information.

To independently scale those logs from the application database, we took the interesting approach of using Rails 6’s multiple databases feature to set up a separate database for the application logs, allowing us to control, monitor and scale access request storage independently from the healthcare data that the application is responsible for storing and serving.

This article is a guide in how to set up your application to do the same thing. We’ll explore the application configuration needed, database migrations, and application code to make use of this feature.

Disclaimer: This is NOT a guide on HIPAA compliance and it should absolutely NOT be assumed that logging access requests is all you have to do to be compliant with the law. Don’t use this as a guide on compliance for HIPAA or any other mandate. Always get real guidance from an expert who knows the law well.

Configuration

The first step in utilizing Rails 6’s multiple database support is to modify your config/database.yml to designate a primary database, and then add a second named database configuration. That second database will have a parameter telling Active Record where to find database migrations, and one telling it the name of your database. This guide assumes both databases are on the same host for simplicity, but once you’ve seen the configuration, you should easily be able to extrapolate for a multi-host setup.

default: &default
  adapter: postgresql
  host: localhost

development:
  primary:
    <<: *default
    database: yourapp_dev
  access_logs:
    <<: *default
    database: yourapp_access_logs_development
    migrations_paths: db/access_logs_migrate

test:
  primary:
    <<: *default
    database: yourapp_test
  access_logs:
    <<: *default
    database: yourapp_access_logs_test
    migrations_paths: db/access_logs_migrate

production:
  primary:
    <<: *default
    database: yourapp_prod
  access_logs:
    <<: *default
    database: yourapp_access_logs_production
    migrations_paths: db/access_logs_migrate

Note that the parameter to specify the location of database migrations is called “migration paths”, plural, even though we only have one here. That particular idiosyncrasy gave me no small amount of frustration while trying to figure out why my Rails app wasn’t happy with my database.yml.

Migrations

Now create a new directory under db, and call it access_logs_migrate. Create a new migration file there with the following in it:

class CreateAccessLogs < ActiveRecord::Migration[6.0]
  def change
    # First set up UUID capabilities for PostgreSQL
    enable_extension('pgcrypto')
    enable_extension('uuid-ossp')

    create_table :access_logs, id: :uuid do |t|
      t.string        :ip
      t.string        :request_uri
      t.string        :request_method
      t.bigint        :profile_id
      t.bigint        :current_user_id
      t.timestamps
    end
  end
end

As you can see, there’s nothing special about this migration; we designated which database will have this table created in it by placing it under the directory we configured in database.yml.

Here we’re creating a table with UUIDs for a primary key (because you could conceivably run into the top limits of bigint otherwise and let me tell you, that’s not fun) and logging IP address, the requested resource and method, and in this case I’m also logging, if available, a profile ID and if the current user is logged in, their ID as well. Those last two params are optional and will likely vary based on your application.

From now on, when you run migrations, ActiveRecord will run migrations in both the normal location and now in your new directory that you just created. Calling rake db:migrate is all you need to do from here to get that table created.

Application Code

The general idea here is that we’re going to use a standard Rails model to log access requests in a before_action in application_controller.rb. We decided to put it here since the vast majority of requests need to be logged, and we can use skip_before_action to save some space on the few access requests that don’t need to be logged, like requests for options that a mobile application would use to generate a select list, for example.

Before you create a model, let’s create a class that it can inherit from to access the new database:

# app/models/access_log_base.rb

class AccessLogBase < ApplicationRecord
  self.abstract_class = true
  connects_to database: { writing: :access_logs, reading: :access_logs }
end

Now you can create a model that inherits from this class:

# app/models/access_log.rb

class AccessLog < AccessLogBase
  validates :ip, presence: true
  validates :request_uri, presence: true
  validates :request_method, presence: true

  def subject
    Profile.find(profile_id)
  end
end

In our use case, we created a subject method that allows us to locate the “subject” of a given access log – in other words, who the potential target was for any given request. This may be useful later on if we’re ever the subject of an audit or have to conduct an investigation for some reason.

Finally, in app/controllers/application_controller.rb:

# app/controllers/application_controller.rb

class ApplicationController < ActionController::API
  before_action :log_access_request

private

  def log_access_request
    AccessLog.create(
      ip: request.ip,
      request_uri: request.url,
      request_method: request.method,
      profile_id: params[:profile_id],
      current_user_id: current_user&.id
    )
  end
end

Since every controller in our application inherits from ApplicationController, our access requests will now automatically be logged on every request. But you may not always want that; in those cases, just skip_before_action :log_access_request in your controller of choice. In our case, it was less common that we’d not want it logged than it was that we would, which is why we made it automatic.

Wait, before action? Why not after for faster responses?

You may be wondering if making the log an after_action might speed up response times. While theoretically possible, we opted not to do this because in our case, we have to log every request by law. We can’t risk there being a situation where the information is served, but no record of that was recorded because to do so is a violation of HIPAA. In other words, if we can’t record the access log, we can’t serve it either, and having this as a before_action prevents that possibility because if recording the log fails, the rest of the request will fail too, meaning that PHI will never be accessed or otherwise served. This is a choice we’ve made to protect our client, and while it’s possible there could be performance implications in some circumstances, those issues are manageable as long as database storage availability and query response times are monitored and addressed accordingly.

Using multiple databases in your application used to involve quite a bit of extra work, or extra libraries, but now it’s quite simple and easy to use. While your use case will likely differ from ours, being able to independently scale your databases by splitting model data across multiple locations can have benefits that, thanks to Active Record, are now more easily accessible than ever.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.