Content posted here with the permission of the author Bandana Pandey, who is currently employed at Josh Software. Original post available here.
Today Performance is what that comes first, when we developers try to develop web services. One of the issue is that, when a web service tries to interact with database, in order to get the result it may take time depending on the number of records.
Prerequisites
For this blog, I am assuming that you have knowledge about Rails and basic idea about Redis.
Getting Started
Lets’s imagine we are building a back-end for the online movie app. Customers will use this app to view all the movies, their details, resulting in huge load on Database. So what if we could reduce the load on the database by caching the movies data. But for caching what should we use ?
There comes REDIS to our rescue.
Redis
Redis is the key-value store, which we can use for CACHING to speed things up and improve our performance.
But Redis is not just a plain key-value store, it is data structures server, means it not just limited to support strings as value, but also more complex data structures, such as Hashes, Lists, Sets, Sorted Sets. For detailed information refer this.
Strings
Strings are the most basis data type that we use for caching in Redis. They are binary safe and easy to use. So we mostly go for them.
But in our scenario Strings DataType was not enough as I have to store the whole list of movies and their respective details in Redis. Strings work well, but it stores the whole list in the string format as value. So, before sending the data , I have to parse them in JSON Format, such that they can be used by the views in order to present it to the User. But what if the data is huge, parsing strings to JSON or any other required format will be time consuming. So, string is not which can be used in our case.
By reading these memory optimization blog and documentation, I found that there is other Datatype that Redis supports, which can be helpful i.e, Hashes.
Hashes
Hashes are the perfect data structure to represent the objects. They are the map between string fields and string values. Also, they are stored in attribute: value format, just like how the tables data is mapped to object using ActiveRecord in Rails. Small hashes are encoded in a very small space, so we should always try to represent our data using hashes.
And in this way using hashes our data parsing issue is solved. Now, we fetch data, as it is from Redis using Hashes, and there is no conversion of data format is involved.
Also memory consumption, reading and writing performance can be improved using optimized storage of hashes over strings data type.
Now lets check the above theory using Benchmark in rails. Here, we are going to use redis-namespace and redis service which is explained later in this section.
Setting Data in Redis:
Benchmark.bm do |x|
#here data is in the json format
#Setting data using hash(value will be stored as hash)
x.report { RedisService.new(klass: Event).set_list(key: CMS_MOVIE_LIST, data: data) }
#Setting data using string(value will be stored as string)
x.report { RedisService.new(klass: Event).set(key: MOVIE_LIST, data: data) }
end
#user system total real
#0.030000 0.010000 0.040000 ( 0.011480) #Hashes
#0.150000 0.000000 1.150000 ( 0.447619) #Strings
Fetching Data from Redis:
Benchmark.bm do |x|
#Fetching data using hash(value will be stored as hash)
x.report { RedisService.new(klass: Event).get_list(key: CMS_MOVIE_LIST) }
#Fetching data using string(value will be stored as string)
x.report { RedisService.new(klass: Event).get(key: MOVIE_LIST) }
end
#user system total real
#0.010000 0.000000 0.010000 ( 0.008200) #Hashes
#0.090000 0.000000 0.090000 ( 0.032398) #Strings
This demonstrates, how our performance can be improved by using Hashes over Strings in Redis.
So in order to use the same things in our rails application, we are going to use redis-namespace. For detailed information about this, refer Redis::Namespace
Initializing Redis in Rails
We instruct our rails app to use redis as a cache store and set the redis_host in ENV variable like this:
REDIS_HOST: 'redis://localhost:6379'
Now, initialize a wrapper around redis using redis-namespace. Or have a service redis_service.rb using redis-namespace so that we can interact with our redis.
class RedisService
def initialize(klass:)
redis = Redis.new(url: ENV['REDIS_HOST'], timeout: 1)
@namespaced_redis = Redis::Namespace.new(klass, redis: redis)
end
def set(key:, data:, expire: nil)
#Command to Set value of a key
@namespaced_redis.set(key, data.to_json)
#Expire your redis key In 1 week
@namespaced_redis.expire(key, 1.weeks)
end
def set_list(key:, data:, expire: nil)
#Command to Set List of Data on a Redis Key
@namespaced_redis.set(key, Marshal.dump(data))
end
def get(key:)
#Command to Get only Value of Key in Json Format
JSON.parse(@namespaced_redis.get(key))
end
def get_list(key:)
#Command to Get List of Data from Redis
Marshal.load(@namespaced_redis.get(key))
end
def del(key:)
#Command to delete Key from Redis
@namespaced_redis.del(key)
end
def keys(pattern: nil)
@namespaced_redis.keys(pattern)
end
end
Marshal
In the above code, we are using marshal. It is a library in ruby which converts the collection of Ruby objects into byte stream. It is the fastest option available in ruby for data serialization. For detailed information refer this
Now we have generic Redis Service which we can use to perform different operations like add, delete, fetch data from Redis in our rails application.
Advantages of writing this service class:
- Code is DRY
- All the redis commands are there in it, and we can use them whenever and wherever we want in our rails app.
Now, we are going to use this, to fetch movies on the basis of city.
Managing Redis Cache in Rails
Here, the whole idea is that, when a customer wants list of movies in a particular city, firstly we are going to fetch the movies, by directly quering on database. Secondly, we will cache the response using redis-namespace wrapper, such that on subsequent quering, the data will be fetched from redis, and not from the Database, thus improving our application performance.
class MoviesController < ApplicationController
#Here we are going to use the RedisService to perform operations on redis
def index
#Check if the list of movie is there in redis
movies = RedisService.new(klass: Movie).get_list(key: "movies:#{params[:city]}")
#If there is no movies in redis
if movies.blank?
#Load Movies from Database
load_movies
#serialize the data
movies = serialize_resource(movies, V1::MoviesSerializer)
#Cache the serialized response in Redis, so that it can be used again
RedisService.new(klass: Movie).set_list(key: "movies:#{params[:city]}", data: movies, expire: 1.day)
end
#Returns the response
mobile_success_response(data: movies)
end
end
The above code is perfect, but there is one loophole in that, if any movie is added or it is updated in the database, it will not be shown to the customer if the data is fetched from Redis.
So in order to solve the above issue, what we have to do ?
We’ll write a callback in such a way that, whenever any movie is added or updated, we will delete the keys corresponding to movie list. So, during updation of any movie, if User wants the data, it will be fetched directly from database and then will be stored in redis cache. On the subsequent calls, it will be fetched from redis. Below is the callback, to achieve this:
class Movie < ApplicationRecord
after_commit :update_in_redis, on: [:create, :update]
after_commit :delete_from_redis, on: [:destroy]
def update_in_redis
redis = RedisService.new(klass: self.class)
#Delete all the keys matching the movies: pattern
redis.del(key: redis.keys(pattern: "movies:*"))
end
def delete_from_redis
redis = RedisService.new(klass: self.class)
#Delete a movie from redis if it is deleted from database
redis.del(key: self.id)
end
end
Hope this blog will be useful. For more information like this, Stay tuned 🙂
