Upstart Scripts in Ubuntu

Posted on February 14, 2012 by

Upstart is an event based tool that handles starting of services during system booting and shut-down the services before system is shut down. It also monitors the services while they running. It was designed to overcome the limitations in system V and dependency based init systems.

Limitations in existing systems

  1. They are not dynamic in nature. That means when we plug in a new devise like a monitor after the system has booted up, the system cannot recognize it without user intervention.
  2. There is no parallelism. This means when we start certain services, like mysql which is dependent on some other services, they should start before mysql service has started. To start these dependent services, it requires manual configuration in /etc/rc.X directories and is prone to errors.
  3. In some systems, a brute force approach is used to ensure all dependent services has started. This bring load on the system and is sometimes in-efficient.

To get more limitations on existing systems, you can read more information here. Upstart overcomes these limitations by starting services only when all conditions are met. And there would be no kernel panic if init fails as it is designed in a very clean and sensible way.

Triggering Jobs

Upstart scripts are located in /etc/init/ directory with a .conf extension. The scripts are called ‘System Jobs’ and run using sudo privileges. Just like system jobs we also have ‘User Jobs’ that are located at $HOME/.init/ directory. (Note: Only Ubuntu versions above 11.10 support user jobs). After internal upstart initialization, it emits an event called ‘startup’ which triggers rest of system and user jobs.

Writing Upstart scripts

An Upstart script is a combination of states and events. On particular events the service state changes from one to another. Currently upstart supports 10 states. They are waiting, starting, pre-start, spawned, post-start, running, pre-stop, stopping, killed and post-stop. More details here.  Here is a simple upstart script which starts node.js server whenever system boots.

# /etc/init/nodejs.conf

description "node.js server"
author      "Siva Gollapalli"

# used to be: start on startup
# until we found some mounts weren't ready yet while booting:
#start on started mountall
# If network interface is wireless
start on (local-filesystems and net-device-up IFACE=wlan0)
# If network interface is Ethernet uncomment below line and comment above line
#start on (local-filesystems and net-device-up IFACE=eth0)

stop on shutdown

# Automatically Respawn:
respawn
respawn limit 99 5

script
    # Not sure why $HOME is needed, but we found that it is:
    export HOME="/home/siva/work/myproject"

    exec /usr/local/bin/node $HOME/node/notify.js 13002 >> /var/log/node.log 2>&1
end script

post-start script
   # Optionally put a script here that will notify you node has (re)started
   # /root/bin/hoptoad.sh "node.js has started!"
end script

Stanzas and Configuration

‘description’, ‘author’, ‘start on’, ‘stop on’, ‘respawn’, ‘script’ and ‘post-start’ are called as stanzas.

start on (local-filesystems and net-device-up IFACE=wlan0)

This tells Upstart to start node.js service  when all local-filesystems and wireless network interface is up.

start on (local-filesystems and net-device-up IFACE=eth0)

This tells Upstart to  start node.js service when all local-filesystems are up and Ethernet network interface is up.

Before starting node.js service Upstart executes the pre-start block if it exists. Later it executes the script block which is the actual code used to start a service. After service has been started, upstart executes post-start block.

In our current scenario with node.js service, there are no pre-start block and the post-start block is there with no code. So, upstart just start node.js service without any pre and post actions.

Monitoring services

You may not have noticed that I forgot to explain what is ‘respawn’ and when to use :) ‘respawn’ is a stanza  which executes when the service has exited for some reason and you want start the service again automatically then this will be used.

‘respawn limit’ defines the limit of re-tries for a service. In our current scenario it is 99 times after a time interval of 5 seconds. That means, try 99 times every 5 seconds to get our service started.

Note: It is preferred not to ‘respawn’ unless and until you are sure that the code you have written is correct. Otherwise it could leads to problems on CPU utilization and memory constraints.

After system has been started execute following command to know status of the service


sudo initctl nodejs status

Stopping services

Up till now we have seen how to start a service. When system is about shut down Upstart executes pre-stop block similar to pre-start block and then executes post-stop block after service has been shut down which is similar to post-start. In out current scenario node.js service stops when system is shut down. We do not need to do anything special to stop services on shutdown. Its guaranteed to cleanly shutdown processes in the correct order!

I hope everyone enjoys this article and let me know if you face any complex scenarios to write. I will give a shot. Feedback and suggestions are welcome.

Posted in Tutorials | Tagged , | Leave a comment

Push notifications using express.js and socket.io

Posted on January 30, 2012 by

So we built a real-time bidding system! Why? Because one of our clients needed it. Our customer bridges the gap between vendors and customers. (Cannot reveal more specifics like the domain etc.)

  • Customer requests an order and vendors bid for that order.
  • Customer should be notified in real-time about any bids.
  • Customer should be able to negotiate with any of the bidders.
  • A negotiation request should be sent to all bidders, so they can be competitive.
  • A vendor should see a negotiation request.
  • A customer can confirm a bid for a vendor.
  • That vendor should get immediate notification and should confirm that bid!
  • blah, blah, blah…

Since this was web application, using websockets was the need of the hour! What we needed was to finalize was putting it all together. We had a choice of socket.io, express.js or backbone.js. (Feel free to remind me of others). Since our Rails server (v3.1) was configured with MongoDB (via Mongoid), we had to get this in place too.

Option 1

Use node.js and its mongodb node_module to setup a notification service between users. A user could connect to the node server (register his login name) and we could control messages to and from that user. The problem is however that the notifications are generated after Rails request processing – this is not a chat service. Shoot!

Option 2

Use express.js to setup a small rack application and route requests to that.  But I would have to manage flow of information – write models, controllers etc. AND it still doesn’t solve my problem of notifications being generated after Rails request processing. Further more, I would need to listen on the client side anyway!

What we did

We run a node server in addition to a Rails server. The node server is powered by socket.io and express.js.

  • socket.io is used for easy notification management from clients.
  • express.js route was to listen for notifications from the Rails server.

So, when a vendor or customer logs in, we connect to the node server and maintain that connected socket connections hash. The express.js routes would be for notifications generated from the Rails server. Using the node connections  hash, we can decide whom to send notifications! The added advantage is that I am not dependent on the database. Here is the gist if you want to see it all. Here are the deeper details:

First we need to configure our node server with express.js AND socket.io. The express.js server is listening on port 13002!


var express = require('express')
, app = express.createServer()
, io = require('socket.io').listen(app);

app.use(express.bodyParser());
app.listen(13002);

Now, lets see how we setup the routes for listening to notifications from the Rails server. We have a generic message sender method which takes the :action and the : recipient whom the message is intended for. The body contains whatever we want to send, typically json data. We shall soon see what the connections contain.


app.get('/', function (req, res) {
 res.send(404);
});

app.post('/message/:action/:to', function (req, res) {
 target = connections[req.params.to]
 if (target) {
  connections[req.params.to].emit(req.params.action, req.body);
  res.send(200);
 }
 else
  res.send(404);
});

We see that connections is critical here. This is where we hook up socket.io into express.js. The connections is a Hash that contains all the connected websockets!


var connections = {}

io.sockets.on('connection', function(socket) {
 socket.on('username', function(username) {
  connections[username] = socket;
 });
});

When a websocket connection is received, we save the connection data into our server connections hash. That way we know who is connected and who should be sent the notification!

Hey! How did the web-sockets get connected anyway? Simple – here is the HTML client side Javascript!


:javascript
  var socket = io.connect('#{NODEJS_HOST}');

  // Connectivity
  socket.emit('username', '#{current_user.name}');

  socket.on('reconnect', function () {
    console.log('Reconnected to the server');
    socket.emit('username', '#{current_user.name}');
  });

  socket.on('reconnecting', function () {
    console.log('Attempting to re-connect to the server');
  });

  // Custom Messages

  socket.on('bid', function(data) {
    $('#notifications').append("<div class='notifications'>" +
    "Bid: Rs." + data.amount + " for " + data.customer +
    "</div>");
  });

What happens here is that when the socket.io gets connected, the socket emits a method with the logged in users name! Thats the name we collect and store in the server connections hash!

We can now customize ANY message like we have done for ‘bid’. It can be passed data directly and can be processed. So, the only missing piece of the puzzle now is how we create and fire notifications from the Rails server!

require 'net/http'

module FX
  module Messenger

    def notify(action, user, data)
      url = "#{NODEJS_HOST}/message/#{action}/#{user}"
      res = Net::HTTP.post_form(URI.parse(URI.encode(url)), data)

      # 200 implies successfully sent.
      # There is nothing we can do if the targe user is not online(404)
      # For any other error, raise Exception
      unless ["200", "404"].include? res.code
        raise Exception.new("Error: #{res.code}")
      end
    end

A sample example of what we have done is on github. Its still in the basic stage (extracted from what we built) but I plan to add full fledge bidding control to it soon.

Ciao!

Posted in Javascript, Ruby on Rails | Tagged , , , , | 8 Comments

Authorize Net (AIM) payment integration with rails

Posted on January 16, 2012 by

Authorize Net (AIM) method enables internet merchants to accept online payments via credit card. We shall see how to integrate authorize net payment gateway inside a rails app to accept online payments using activemerchant library.

  # Gemfile
  gem 'activemerchant', :require => 'active_merchant'

Register for authorize net sandbox account click here

Payment gateway credentials

# config/authorize_net.yml
development: &development
    mode: test
    login: 9gdLh6T
    key: 67fu45xw6VP92LX1

production:
   <<: *development

test:
   <<: *development

Payment & creditcard form 

# app/views/payments/new
= form_for @payment, :url => payments_url do |f|
  = f.text_field :amount
  = fields_for :creditcard, @creditcard do |cc|
    = cc.text_field :name
    = cc.text_field :number
    = cc.select :month, Date::ABBR_MONTHNAMES.compact.each_with_index.collect{|m, i| [m, i+1]}, {:prompt => 'Select'}
    = cc.select :year, Array.new(15){|i| Date.current.year+i}, {:prompt => 'Select'}
    = cc.text_field :verification_value
  = f.submit 'Pay'

Payments Controller 

# app/controllers/payments_controller.rb
class PaymentsController < ApplicationController

  def new
    @payment = Payment.new
    @creditcard = ActiveMerchant::Billing::CreditCard.new
  end

  def create
    @payment = Payment.new(params[:payment])
    @creditcard = ActiveMerchant::Billing::CreditCard.new(params[:creditcard])
    @payment.valid_card = @creditcard.valid?
    if @payment.valid?
      @payment = @payment.process_payment(@creditcard)
      if @payment.success?
        @payment.save
        flash[:notice] = I18n.t('payment.success')
        redirect_to payments_url and return
      else
        flash[:error] = I18n.t('payment.failed')
      end
    end
    render :action => :new
  end
end

Generate & Migrate Payment Model

  rails g model payment status:string amount:float transaction_number:string
  rake db:migrate

Payment Model

# app/models/payment.rb
class Payment < ActiveRecord::Base

  PROCESSING, FAILED, SUCCESS = 1, 2, 3

  validates :valid_card, :inclusion => {:in => [true], :message => 'Invalid Credit Card'}
  validates :amount, :presence => true, :numericality => { :greater_than => 0 }

  def process_payment(creditcard)
    ActiveMerchant::Billing::Base.mode = auth['mode'].to_sym
    self.status = PROCESSING
    response = gateway.purchase(amount * 100, creditcard)

    if response.success?
      self.transaction_number = response.subscription_id
      self.status = SUCCESS
    else
      self.status = FAILED
    end
    return self
  rescue Exception => e
    self.status = FAILED
    return self
  end

  def success?
    self.status == SUCCESS
  end

  private
  def gateway
    ActiveMerchant::Billing::AuthorizeNetGateway.new(
      :login    => auth['login'],
      :password => auth['key'])
  end

  def auth
    @@auth ||= YAML.load_file("#{Rails.root}/config/authorize_net.yml")[Rails.env]
  end
end

Posted in Ruby on Rails | Leave a comment

Designing Rails API using Rabl and Devise

Posted on December 23, 2011 by

Most of us are aware that enabling API access for rails application is easy as Rails provides RESTful APIs by default. However, a little complexity arises when some responses are expected in xml format (maybe for some legacy system) and JSON format. Complexity increases when these API calls need authentication.

The initial obvious thought that comes to ones mind is to create different views for each format and add respond_to block in controller. No! You don’t have to do this.

Rabl

Rabl is a gem which comes in handy when you want to represent the response in both json and xml using a single template. These are the only 2 formats currently supported.

We can enable API for an existing application or new application – the steps are same! Lets see how we

  • enable APIs without authentication.
  • enable APIs with authentication.

Enabling API access without authentication

  • Add ‘rabl’ entry to your Gemfile
  • You might need to add json/xml parser gem to your Gemfile if don’t have them already
  • Execute ‘bundle update’

Now, lets enable json & xml api response for https://localhost/tasks for getting list of tasks. Create the content file

#app/views/tasks/index.rabl

collection @tasks
attributes :id, :name, :priority, :complete_at, :task_type

In the above code we only  mention the attributes that are need to be rendered. There is no need to change tasks_controller code if you haven’t used respond_to for specifying the supported format.  Otherwise, you need insert the following code inside the respond_to block of index method of tasks_controller

#app/controllers/tasks_controller.rb#index

def index
  # some code

  respond_to do |format|
   format.json{}
   fromat.xml{}
  end
end

All set! Now, lets now test the responses for json and xml format by using curl:

curl -v -H "Accept: application/json" -H "Content-type: application/json" https://localhost/tasks.json -k

[{"task":{"name":"Inform users", "task_type":"email", "priority":1, "id":2, "completed_at":null}}, {"task": {"name":"Update Bank or Building Society", "task_type":"document", "priority":1, "id":3, "completed_at":null}}]

curl -v -H “Accept: application/xml” -H “Content-type: application/xml” https://localhost/tasks.xml -k

<?xml version="1.0" encoding="UTF-8"?>
<tasks type="array">
  <task>
    <task-type>email</task-type>
    <name>Inform users</name>
    <completed_at nil="true"></completed_at>
    <id type="integer">2</id>
    <priority type="integer">1</priority>
  </task>

  <task>
    <task-type>document</task-type>
    <name>Update Bank or Society</name>
    <completed_at nil="true"></completed_at>
    <id type="integer">3</id>
    <priority type="integer">1</priority>
  </task>
</tasks>

So with single rabl template we can serve two different formats.

Enabling API access with authentication

Let us consider devise as authentication mechanism. Enable token_authenticatable along with other required devise modules in user model and run migration

#app/models/users.rb
devise :token_authenticatable

We also need to add a migration for this

#db/migrate/20111201074702_add_token_authenticatable_to_users.rb
class AddTokenAuthenticatableToUsers < ActiveRecord::Migration
  def self.up
    add_column :users, :authentication_token, :string
  end

  def self.down
    remove_column :users, :authentication_token
  end
end

Enable/uncomment the following line in devise configuration, :auth_token is the name of the parameter be passed in the user for authentication

config.token_authentication_key = :auth_token

For sign_in and sign_up, devise takes care of most of the things. But if you want authentication_token as a response for sign_up/sign_in you need to do the following

  • Add the fields in user model ‘attr_accessor’ to be shown/emitted in the response. For example, if you want email and authentication_token to be part of your response, we need to modify the model like this

#app/models/user.rb
attr_accessible :email, :authentication_token

  •  Modify RegistrationsController create method. The following snippet shows us an example for json format sign_up request with api_key check

#app/controllers/registrations_controller#create
format.json {
  if !params[:api_key].blank? and params[:api_key] == API_KEY
    resource = warden.authenticate!(:scope => resource_name, :recall => "#{controller_path}#new")
    sign_in(resource_name, resource)
    current_user.reset_authentication_token!
    respond_with resource, :location => after_sign_in_path_for(resource)
  else
    render :json => {'errors'=>{'api_key' => 'Invalid'}}.to_json, :status => 401
  end
}
format.any{super}

Suppose we have a pre-registered API_KEY as ‘mprmzayb’, we can use it to generate and retrieve an authentication token!

curl -v -H “Accept: application/json” -X POST -d ‘user”:{“email”:”email@email.com”, “password”:”password”, “password_confirmation”:”password”, “accept_terms”:1}}’ https://localhost/users.json?api_key=mprmzayb -k

{"user":{"accept_terms":true, "authentication_token":"VBXdiVeCDGQD3Rxtg9qp", "email":"email@email.com", "has_email_reminder":false}}

Now, if you need the task list API requires authentication (i.e. the authenticate_user! before_filter is enabled for index method of TasksController), we need to pass the authentication_token in the request from sign_in/sign_up request.

Assuming you have the index.rabl in place as shown above, lets test this out:

curl -v -H “Accept: application/json” -H “Content-type: application/json”  https://localhost/tasks.json?auth_token=VBXdiVeCDGQD3Rxtg9qp -k

[{"task":{"name":"Inform users","task_type":"email","priority":1,"id":2,"completed_at":null}},{"task":{"name":"Update Bank or Building Society","task_type":"document","priority":1,"id":3,"completed_at":null}}]

Yeah! Devise automatically takes care of authentication, so in the following senerios appropriate error message is returned.

  • If the user tries to access tasks after logout with old token

{"error":"Invalid authentication token."}

  • If the user tries to access tasks without authentication token

{"error":"You need to sign in or register before continuing."}

Enjoy building your APIs in Rails!

Posted in Ruby on Rails | Tagged , , | 10 Comments

Sencha Touch MVC application with Rails

Posted on December 14, 2011 by

For sometime, we have been toying around with different frameworks to build cross-platform mobile applications. Our latest research was with Sencha Touch. This post is a tutorial to get kick-started.

Installation

To run Sencha Touch application you need:

  1. A running web server. I used a Rails server running on localhost:3000
  2. A WebKit based browser like chrome or safari.
  3. Sencha Touch SDK.

After downloading the SDK, extract it to the public/javascripts directory of your web application. We typically need only  ”sencha-touch.js” and “resources/css/sencha-touch.css”. You may remove the remaining files.

Configuring the User Agent

I used Safari to test my application. To make it work, I needed to change its User agent to Safari iOS 4.1 iPhone. To change the User agent in Safari, you can go to Menu -> Develop -> User Agent and set the one you need.

In Chrome, refer how to change the user agent of browser.

Rembember Sencha Touch uses HTML5.  In case you require to support requests from a web-application as well as mobile devices (as is in most cases), in the rails application we need some modifications:

  • A method to check that the request is coming from a web browser or a mobile. To do so, modify your application controller as described at Railscasts episode 199. Here is a snippet:

# application_controller.rb
  class ApplicationController < ActionController::Base

    helper :all
    protect_from_forgery
    def mobile_device?
      user_agent = request.user_agent
      user_agent =~ /Mobile|webOS/
    end
   helper_method :mobile_device?
  end

Here we are checking that if user agent of request contains ‘Mobile’ or not, as

  • Safari on Windows has user agent as “Mozilla/5.0 (Windows; U; Windows NT 6.0; en-us) AppleWebKit/533.20.25 (KHTML, like Gecko) Version/5.0.4 Safari/533.20.27″.
  • Mozilla Firefox on Ubuntu has user agent as “Mozilla/5.0 (X11; Linux i686; rv:7.0.1) Gecko/20100101 Firefox/7.0.1″.
  • Safari installed on Ubuntu on Wine with user agent as Safari iOS4.1 iPhone has user agent as “Mozilla/5.0 (iPhone; U; CPU iPhone OS 4_1 like Mac OS X; en-us) AppleWebKit/532.9 (KHTML, like Gecko) Version/4.0.5 Mobile/8B117 Safari/6531.22.7″
  • Android Mobile phone has user agent as “Mozilla/5.0 (Linux; U; Android 2.3.4; en-gb; GT-I9100G Build/GINGERBREAD) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1″.
  • Update the application layout. (app/views/layouts/application.html.erb)

<!DOCTYPE html>
<html>
  <head>
  <title>Senchamvc</title>
  <%= csrf_meta_tag %>
  <% if mobile_device? %>
    <%= stylesheet_link_tag 'sencha-touch' %>
    <%= javascript_include_tag 'sencha-touch-1.1.1/sencha-touch' %>
    </head>
    <body>
    </body>
    </html>
  <% else %>
    <%= stylesheet_link_tag :all %>
    <%= javascript_include_tag :defaults %>
    </head>
    <body>
      <%= yield %>
    </body>
    </html>
  <% end %>

You can notice that for mobile devices we are not adding ‘yeild’ in body of layout as we have added for web application’s code, this is because all view rendering is going to be  done by the included javascripts.

Now lets see how to write code for our application using the Sencha Touch SDK.

We are building a very simple application. There are 2 models:

  • Business – This is any corporate, company or registered profit making entity.
  • Organization – This is any non-profit organization like NGOs, NPOs etc.

The larger product maps businesses with organizations they support. For the sake of our demo, we are simply using these 2 models to list information. This will help us understand Sencha Touch SDK.

First,create a directory named “app” under public/javascripts. In that create a directory structure like this:


Under the public/javascripts/app directory, create directories named models, controllers and views. These are exactly similar to a standard MVC Rails application.
So application_layout will look like this,

<!DOCTYPE html>
<html>
  <head>
    <title>Senchamvc</title>
    <%= csrf_meta_tag %>
    <% if mobile_device? %>
      <%= stylesheet_link_tag 'sencha-touch' %>
      <%= javascript_include_tag 'sencha-touch-1.1.1/sencha-touch' %>
      <%= javascript_include_tag 'app/app' %>
      <%= javascript_include_tag 'app/routes' %>
      <%= javascript_include_tag 'app/views/Viewport' %>
      <%= javascript_include_tag 'app/views/TabBarMvc' %>
      <%= javascript_include_tag 'app/models/business' %>
      <%= javascript_include_tag 'app/models/organization' %>
      <%= javascript_include_tag 'app/controllers/business' %>
      <%= javascript_include_tag 'app/controllers/organization' %>
      <%= javascript_include_tag 'app/views/business/index' %>
      <%= javascript_include_tag 'app/views/business/show' %>
      <%= javascript_include_tag 'app/views/organization/index' %>
      </head>
      <body></body>
      </html>
    <% else %>
      <%= stylesheet_link_tag :all %>
      <%= javascript_include_tag :defaults %>
      </head>
      <body>
        <%= yield %>
      </body>
      </html>
    <% end %>

Sencha application is initialized from app.js. My application initialization starts with listing of businesses so my public/javascripts/app/app.js looks like this:

new Ext.Application({
  name: 'senchamvc',
  defaultUrl: 'business/index',
  launch: function() {
    this.views.viewport = new senchamvc.views.Viewport();
  }
});

Now lets see what exactly we are doing here,

  • Ext is the global namespace for the whole Sencha Touch framework. Every class, function and configuration for the whole framework exists under this single global variable.
  • Ext.Application represents the entire application.
  • name: ‘senchamvc’ creates a global variable called ‘senchamvc’ – all of your Application’s classes (such as its Models, Views and Controllers) will reside under this single namespace.
  • defaultUrl: When the app is first loaded, this url will be redirected to.
  • when all of your JavaScript are loaded, your application’s launch function is called, at which time you can run the code that starts your app. The launch function is only expected to be run once.
  • Viewport extends from Ext.Container, it has as layout ( defaults layouts is Ext.Layout.Card). This means you can add items to it at any time, from anywhere in your code. Viewport for an application is defined in app/views/Viewport.js

Routing the requests

We routes the request to specific action of controller using app/routes.js

Ext.Router.draw(function(map) {
  map.connect(':controller/:action');
});

The Router is used to map url to controller/action pairs. Every Ext.application can set up Routes using the default Ext.Router instance. So request to url  “business/index” will be dispatched to index action of controller registered as “business”.

The Model

The Business model can be defined as shown below in the app/models/business.js file

senchamvc.models.Business = new Ext.regModel('Business', {
  fields: [
    {name: 'id' , type: 'int'},
    {name: 'name', type: 'string'},
    {name: 'address', type: 'string'}
   ]
});

Here,

  • Ext.regModel:  Registers a model definition i.e creates a new Model class from the specified config object
  • fields : specify the attributes of a model.

To increase performance, we can sync data from remote Rails server to a local data store at time of application initialization. For this I keep 2 data stores in my application, one is remote data store and another is local data store. So, I add this to my business.js model

senchamvc.stores.remoteBusinesses = new Ext.data.Store({
  id: 'remoteBusinesses',
  model: 'Business',
  proxy: {
    type: 'ajax',
    url: 'http://localhost:3000/businesses.json',
    reader: {
      type: 'json',
      root: 'businesses',
      record: 'business'
    },
    writer: {
      type: 'json',
      record: 'business'
    }
  }
});

senchamvc.stores.localBusinesses = new Ext.data.Store({
  id: 'localBusinesses',
  model: 'Business',
  proxy: {
    type: 'localstorage',
    id: 'businessses'
  }
});

Here we are defining the data stores with their proxies. The Store class contains the client side cache of model objects. Stores load data via a Proxy.

Now lets see various attributes of Store,

  • id: is the id we are assigning to data Store.
  • model: specify the model for which we are defining the Store.
  • proxy: Proxies are used by stores to handle the loading and saving of Model data.
  • proxy type: json: Configuration is automatically turned into an Ext.data.AjaxProxy instance, with the url we specified being passed into AjaxProxy’s constructor. So when we call load() on store makes a request to the url we configured.
  • proxy type: localstorage: The LocalStorageProxy uses the new HTML5 localStorage API to save Model data locally on the client browser.
  • reader in this case is JSON Reader is used by a Proxy to read a server response that is sent back in JSON format.
  •  the business data is nested an additional level inside the “businesses” as each “business” item has additional data surrounding it, so we specify “record” i.e within the JSON response that the record data itself can be found at and “root” i.e the name of the property which contains the Array of row objects.

Every time we sync data from the server, it is necessary to clear local data store before syncing it with remote data store. This is to avoid  data inconsistency. We can use far more superior techniques in production systems to sync only deltas. Here I am clearing this, so that we can study the concept of listeners! Update the business.js model

senchamvc.stores.remoteBusinesses.addListener('load', function () {
  var store = Ext.getStore('localBusinesses');
  store.getProxy().clear();
  store.data.clear();
  store.sync();
  this.each(function (record) {
    var business = senchamvc.stores.localBusinesses.add(record.data)[0];
  });
  senchamvc.stores.localBusinesses.sync();
});

The Controller

Moving onto controllers, we ensure that when we initialize the Business controller, we load both the data stores. The app/controllers/business.js looks like this:

Ext.regController('business',{
  init: function(options) {
    senchamvc.stores.remoteBusinesses.load();
    senchamvc.stores.localBusinesses.load();
  }
});

Here Ext.regController: Creates a new Controller class .

To show list page of businesses, in index method of business controller set the active item in Viewport as business index view.

index: function(options) {
  senchamvc.stores.localBusinesses.load();
  if (! this.indexView)
  {
    this.indexView = this.render({
    xtype: 'BusinessIndex',
    });
  }
  senchamvc.views.viewport.setActiveItem(this.indexView, options.animation);
},

You can notice here that we are rendering “BusinessIndex” instead of just index, this is because we are registering the views also, so if we want to register more than 1 index views of different controllers, it will lead to confusion so we are registering index view of business as BusinessIndex. While rendering we are mentioning xtype because every component has a specific xtype, which is its Ext-specific type name.

The View

In the view we can define title of page, store from which you want to show the data, action to be handled on tap event etc. Our Business index (app/views/business/index.js) is as follows,

senchamvc.views.BusinessIndex = Ext.extend(Ext.Panel, {
  dockedItems: [{
    xtype: 'toolbar',
    title: 'Businesses',
    items: []
   }],
  layout: {
    type: 'vbox',
    align: 'center',
    pack: 'center'
  },
  items: [{
    xtype: 'list',
    id: 'businessesindex',
    scroll: 'vertical',
    width: Ext.Element.getViewportWidth()*0.9,
    store: senchamvc.stores.localBusinesses,
    style: {
      background: '#ffffff'
    },
    itemTpl: new Ext.XTemplate(
      '<tpl for=".">',
        '<div>',
          '{name}',
        '</div>',
        '<div>',
          '{address}',
        '</div>',
      '</tpl>'
    ),
    onItemDisclosure: function(record){
    },
    onItemTap: function(item) {
      record = this.getRecord(item);
      Ext.dispatch({
        controller: 'business',
        action: 'show',
        id: record.getId(),
        animation: {type: 'slide', direction: 'left'}
      });
    }
  }],
  initComponent: function() {
    senchamvc.stores.localBusinesses.load();
    senchamvc.views.BusinessIndex.superclass.initComponent.apply(this, arguments);
  }
});
Ext.reg('BusinessIndex', senchamvc.views.BusinessIndex);

Now lets see what exactly we are doing in the view definition,

  • Panel is a container which is building block for user interfaces.
  • dockedItems: A component or series of components to be added as items to panel, items can be docked to either the top, right, left or bottom of a panel. This is typically used for things like toolbars or tab bars.
  • layout should be defined for child items to be correctly sized and positioned, if not then the default layout manager will be used which does nothing but render child components sequentially into the Container (no sizing or positioning will be performed in this situation).
  • items can be single or array of child Components to be added in Container.
  • within items we can specify the properties such as width, scrolling style etc. for Container.
  • One thing you will notice here that is we are defining 2 events, onItemDisclosure and onItemTap. This is because onItemDisclosure display a disclosure icon on each list item. This won’t bind a listener to the tap event. By setting dipatch configuration to an action, it will automatically add a tap event listeners to the disclosure button [not to  entire row]. So as to add listener to entire row we are defining listener for onItemTap event.
  • At the end we are registering this view as BusinessIndex so from a controller we can render this view using the name with which it is registered.

So my application is ready to go with data stored locally.

You can clone the complete example from github.

For more information you can refer to sencha touch application with ruby on rails.

References :

For adding Tabbar to mobile application I refered to sencha-touch-tabbar-in-a-mvc-application.

To generate an installable, such as for android *.apk, of your code please refer to phonegap documentation.

Also an excellent example for MVC application in sencha with phonegap.

At the end I was able to develop this application which shows list of businesses and organizations,


Posted in Mobile Development, Ruby, Sencha | Tagged | 1 Comment

Case Study: Improving Performance of MySQL, Thinking Sphinx in a Rails app

Posted on November 15, 2011 by

Recently, we faced a huge performance problem with one of our installed apps. The application is a Call Center ERP solution. The Call model is updated very frequently as there are about approximately 8,000 – 10,000 calls being made everyday. For every action there is an audit trail, so updates and inserts into database are huge!

The system is hosted in a private data center – with nginx + passenger setup. It runs on Rails 2.3.11 + MySQL and uses Thinking Sphinx for search indexing. It has been running fine for the past 3 years and over the last few months the client has scaled number of users considerably but nothing that we found alarming.

Lets talk numbers! I am using New Relic terminology here and if you are not familiar with it I do suggest you read and use it. New Relic RPM rocks! We earlier used to have about 25 rpm (requests per minute) which recently scaled to about 45 rpm. Not much increase when it comes to scale in terms of number of users.

However, to our surprise, we started getting a lot of complaints about the system being slow – typically slowing down drastically towards evening. The possible culprits:

  1. Code. There was minimal code and non-impacting changes since a few weeks before the complaints , so we were pretty sure there was no code issue.
  2. Infrastructure. Was a possibility, but the server hardware statistics checked out fine. Network card monitoring showed no packet drops.
  3. Network routing. Routers were changed but the problem persisted.
  4. Disk IO / swap space. No disk space issues.

So, we monitored the server using top and found that MySQL was taking up memory in spikes (sometime even more than 200%). To verify our hypothesis, we installed the latest version of New Relic RPM gem and upgraded to New Relic Pro version and went through the details. This is the image we saw:

There are a few things worth noticing. Firstly, there is one particular query (update_call_status) which is taking obscenely long (we saw web requests ranging from 17 seconds – 35 seconds). Poor users!

The orange portion was time spent in database. So, we set about looking into improving the database resources. After much research on improving an InnoDB MySQL performance, we added the following lines in the  /etc/my.cnf

innodb_buffer_pool_size = 1600M # earlier value was 800M
innodb_flush_log_at_txn_commit  = 0

Increasing the pool size allowed MySQL to improve its cache and the flush the commit log on transaction improved speed. Typically innodb_buffer_pool_size recommended value is about 60% of total memory. 0 second commit transaction flush is immediate but if you set it to 1 or more, its not guaranteed to flush the commit log exactly after that interval.

This was the next picture we got in RPM:

Notice that the orange has reduced but the blue (Ruby processing) has increased. We also noticed that the web request time has not reduced. It remains same as earlier. So, though we have indeed improved the resources of MySQL and lowered database time, its has not solved the problem!

Next we analyzed the Call Trace and found this:

Hang on — Uninstrumented time – wtf! RPM says its the time spent where we cannot analyze where the control went. We add tracer_method to thinking sphinx methods  (ThinkingSphinx::Search showed up) but we still got uninstrumented time. Now what more tracer methods could be add? !! Having limited options here, we dug into various gem code.

On expansion of the trace, the call log pointed to ‘after_commit’ gem for db_update_transaction. Hmm.. into the thinking sphinx scope now. We added some log statements into after_commit gem but it did not show any delay. So, we thought back and reviewed how we are using Thinking Sphinx here.

Delta Indexes! Could they be causing trouble? We added a log statement here to see if delta indexing is causing trouble:

# File: thinking-sphinx-2.0.1/lib/thinking_sphinx/active_record/delta.rb

def index_delta
  Rails.logger.info("+++ #{Time.now.to_i}"
  delta_object.index(self, instance)
  Rails.logger.info("+++ (exit) #{Time.now.to_i}"

VOILA! We nailed it – the time difference between these 2 log statements was over 10 seconds. But what does that explain and how do we solve it?

RTFM

We read the configuration about how we can manage delta indexed. There are 3 ways to do this:

  • Default: Spawn the indexer every time a delta is issued (UPDATE / INSERT) queries.
  • DateTimeDelta: Do not spawn the indexer immediately but at most after a threshold
  • DelayedDelta: Use delayed job for spawning the indexer

Now, in every request to the culprit – ‘update_call_status’, there are about 5 UPDATE and 3 INSERT calls. Using the default indexing strategy, that triggers the indexer about 8 times per second. Considering about 40 rpm for update_call_status, it means 320 calls to indexer per minute. Considering that this was a system running for about 8 hours a day…. oh I lost count, do the math!

Considering the symptoms “server works fine in the morning and slows down crazily by evening”, its starting to make sense. We had a index rebuild task around mid-night, so it would rebuild the TS index.

We reconfigured our delta indexes with datetime deltas with a threshold of 1 hour.

class Call < ActiveRecord::Base
define_index do
...
set_property  :delta => :datetime, :threshold => 1.hour
    ...
  end
  ...
end

Since we need the rake tasks, add the ts-datetime-delta gem to the Gemfile.

#Gemfile
gem 'ts-datetime-delta', :require => 'thinking_sphinx/deltas/datetime_delta'

Add a cron job to the frequency of the threshold value you have set:

0 * * * * (cd <path to your Rails.root>; rake ts:in:delta)

When we restarted our app server, we saw a totally different and pleasant sight!

We brought down the server response time to under 500ms and average web response times to under 1.5 seconds.

Problem resolved! Woot!

Posted in Case Studies, Ruby on Rails, Search | Tagged , , , , | Leave a comment

IndexTank – So long and thanks for all the fish!

Posted on October 17, 2011 by

IndexTank got acquired by LinkedIn just a while back and I received a newsletter saying that they may discontinue services in 6 months. My tweet about this got a few responses from Pat Allen (@pat – the creator of Flying Sphinx) and others, which ended up with me promising a brief comparison between Thinking Sphinx (TS), WebSolr (Solr) and IndexTank and why I love IndexTank!. There are plenty of documents and posts on how to setup, configure and use TS and Solr. This post does give information about using IndexTank and the various facets available with it that I find awesome and/or non-existent in TS or Solr.

My experiments with Full-text searching

When I first set out needing full text searching, I used Solr. It was pretty good though re-indexing took ages and to ensure consistency, I had to re-index every day via cron. Then I found Thinking Sphinx – and loved it because it managed delta indexes! Wow – no more daily re-index cron jobs. Even the re-indexing was way quicker.

TS was easy to configure and it generated a production.sphinx.yml, which I could tweak (at my own risk) to get it to index what I want – a bad practice but works well if you know what you are doing.

The big issue with both Solr and TS was that it required tight integration with models and my database. For example – in TS, if a relationship was changed, I had to ensure to trigger the parent / child delta index in order to ensure it gets indexed too.  Both TS and Solr add methods to ActiveRecord, which I find a little annoying.

These nuances gets my code too dependent on TS or Solr and switching from them to something else becomes a big pain!

IndexTank makes an entrance.

IndexTank is a NoSQL hosted service for full text search. What I liked best about IndexTank is that I had my application integrated in about 15 minutes – no frills, 3rd party storage. Wow!

The first thought is ‘performance hit’ … when I checked from Rails console — result was totally acceptable! Of course performance is relative but when you consider that the server load is reduced and a third party is doing a full text index on my data — the cost is covered.

Accessibility is awesome. I can create and update documents on IndexTank at will! That is I dont need to index / re-index or add delta’s to my database, because IndexTank is not dependent on my database or even my schema!

I can now index what I want and how I want. Here is a small example:


# config/initializers/indextank.rb
CLIENT = IndexTank::Client.new('http://:67xN9mHBV7BV8w@iej.api.indextank.com')
INDEX = CLIENT.indexes('idx')

I like the fact that there are no rake tasks for configuration, starting and stopping. Since its a 3rd party service and not a separate process on my server, my server resources are not utilized. The icing on the cake is that we can manage multiple indexes on IndexTank and hence segregate the entire data, not just subset it!

Using it is pretty simple:


# app/models/user.rb
INDEX.document("User:id:#{self.id}").add(:text => "#{self.name} #{self.address}")
# basically what ever I want

It’s interesting to note that self.name could be a attribute accessor but self.address could be a method which returns a GeoCoder formatted address – the point being that I can send processed output to IndexTank! Now, when I want search results, I simply do this


results = INDEX.search("something")
=> {"matches"=>2, "query"=>"something",
"facets"=>{}, "search_time"=>"0.009",
"results"=>[{"docid"=>"User:id:33", "query_relevance_score"=>-2217269.0},
{"docid"=>"User:id:38", "query_relevance_score"=>-2739353.0}]}

So, I now get the number of matches, the objects which matched sorted according to their relevance! ‘Relevance’ – what is that you ask? Its an amazing feature of IndexTank called scoring functions. The default scoring function is by time of creation but we can easily register new scoring functions so that we get the objects in the order we want. Wowie!

Note that ‘text’ is the default keyword for IndexTank for storing data. But there is a LOT more to this. I can not only provide my own keys instead of text, I can also use some facets like ‘categories’ to limit my search. For example, I can modify my code earlier for insertion like this:

INDEX.document("User:id:#{self.id}").add(:text => "#{self.name} #{self.address}",
:categories => {:type => 'admin'})

I added the ‘admin’ category for users. And when I search using the same query as above, I get this result

results = INDEX.search("something")
=> {"matches"=>2, "query"=>"something",
"facets"=>{'type' => {'admin' => 1, 'user' => 1}}, "search_time"=>"0.009",
"results"=>[{"docid"=>"User:id:33", "query_relevance_score"=>-2217269.0},
{"docid"=>"User:id:38", "query_relevance_score"=>-2739353.0}]}

Notice this line in particular in the results, that filters my search result.

"facets"=>{'type' => {'admin' => 1, 'user' => 1}}

A slight modifications in my search call renders sub-set results:

results = INDEX.search("something", :category_filters => { :type => 'admin'}))
=> {"matches"=>1, "query"=>"something",
"facets"=>{'type' => {'admin' => 1}, "search_time"=>"0.007",
"results"=>[{"docid"=>"User:id:33", "query_relevance_score"=>-2217269.0}
]}

Not that this is not doable in TS or Solr – but its not as usable as IndexTank. Please educate me otherwise. This post is getting long already, so I plan to detail out IndexTank faceting and scoring functions in my next post and keep this one as a comparison between IndexTank and TS or Solr.

Heroku Setup

First and foremost IndexTank has the basic version free! At the time of writing this post, the lowest version of FlyingSphinx is ‘wooden’ version (12$ per month) and WebSolr has the ‘Silver’ version. (20$ per month). This is pretty steep when it comes to prototyping, basic building of small apps. IndexTank free version has upto 100,000 documents storage which is free – which is more than enough to support a small application.

Further-more, for both TS and WebSolr, we need workers which are run by in the daily cron. This does increase the cost further.

The FlyingSphinx and Websolr configuration on Heroku is no simple task either. There are quite a few caveats to cater too and it can still cause some hiccups. Its indeed worth mentioning the the support you get from FlyingSphinx and WebSolr is indeed awesome. But I would expect it if I were paying a fee for the setup ;)

Hope this stirs up a healthy debate – (and secretly hoping that IndexTank services continue even after 6 months).

Posted in Ruby, Search | Tagged , , , | 11 Comments