Most of us are aware that enabling API access for rails application is easy as Rails provides RESTful APIs by default. However, a little complexity arises when some responses are expected in xml format (maybe for some legacy system) and JSON format. Complexity increases when these API calls need authentication.
The initial obvious thought that comes to ones mind is to create different views for each format and add respond_to block in controller. No! You don’t have to do this.
Rabl
Rabl is a gem which comes in handy when you want to represent the response in both json and xml using a single template. These are the only 2 formats currently supported.
We can enable API for an existing application or new application – the steps are same! Lets see how we
- enable APIs without authentication.
- enable APIs with authentication.
Enabling API access without authentication
- Add ‘rabl’ entry to your Gemfile
- You might need to add json/xml parser gem to your Gemfile if don’t have them already
- Execute ‘bundle update’
Now, lets enable json & xml api response for https://localhost/tasks for getting list of tasks. Create the content file
<br> #app/views/tasks/index.rabl</p> <p>collection @tasks<br> attributes :id, :name, :priority, :complete_at, :task_type<br>
In the above code we only mention the attributes that are need to be rendered. There is no need to change tasks_controller code if you haven’t used respond_to for specifying the supported format. Otherwise, you need insert the following code inside the respond_to block of index method of tasks_controller
<br>
#app/controllers/tasks_controller.rb#index</p>
<p>def index<br>
# some code</p>
<p> respond_to do |format|<br>
format.json{}<br>
fromat.xml{}<br>
end<br>
end<br>
All set! Now, lets now test the responses for json and xml format by using curl:
curl -v -H "Accept: application/json" -H "Content-type: application/json" https://localhost/tasks.json -k
<br>
[{"task":{"name":"Inform users", "task_type":"email", "priority":1, "id":2, "completed_at":null}}, {"task": {"name":"Update Bank or Building Society", "task_type":"document", "priority":1, "id":3, "completed_at":null}}]<br>
curl -v -H “Accept: application/xml” -H “Content-type: application/xml” https://localhost/tasks.xml -k
<br>
<?xml version="1.0" encoding="UTF-8"?><br>
<tasks type="array"><br>
<task><br>
<task-type>email</task-type><br>
<name>Inform users</name><br>
<completed_at nil="true"></completed_at><br>
<id type="integer">2</id><br>
<priority type="integer">1</priority><br>
</task></p>
<p> <task><br>
<task-type>document</task-type><br>
<name>Update Bank or Society</name><br>
<completed_at nil="true"></completed_at><br>
<id type="integer">3</id><br>
<priority type="integer">1</priority><br>
</task><br>
</tasks><br>
So with single rabl template we can serve two different formats.
Enabling API access with authentication
Let us consider devise as authentication mechanism. Enable token_authenticatable along with other required devise modules in user model and run migration
<br> #app/models/users.rb<br> devise :token_authenticatable<br>
We also need to add a migration for this
<br>
#db/migrate/20111201074702_add_token_authenticatable_to_users.rb<br>
class AddTokenAuthenticatableToUsers < ActiveRecord::Migration<br>
def self.up<br>
add_column :users, :authentication_token, :string<br>
end</p>
<p> def self.down<br>
remove_column :users, :authentication_token<br>
end<br>
end<br>
Enable/uncomment the following line in devise configuration, :auth_token is the name of the parameter be passed in the user for authentication
<br> config.token_authentication_key = :auth_token<br>
For sign_in and sign_up, devise takes care of most of the things. But if you want authentication_token as a response for sign_up/sign_in you need to do the following
- Add the fields in user model ‘attr_accessor’ to be shown/emitted in the response. For example, if you want email and authentication_token to be part of your response, we need to modify the model like this
<br> #app/models/user.rb<br> attr_accessible :email, :authentication_token<br>
- Modify RegistrationsController create method. The following snippet shows us an example for json format sign_up request with api_key check
<br>
#app/controllers/registrations_controller#create<br>
format.json {<br>
if !params[:api_key].blank? and params[:api_key] == API_KEY<br>
build_resource<br>
if resource.save<br>
sign_in(resource)<br>
resource.reset_authentication_token!<br>
render :template => '/devise/registrations/signed_up' #rabl template with authentication token<br>
else<br>
render :template => '/devise/registrations/new' #rabl template with errors<br>
end<br>
else<br>
render :json => {'errors'=>{'api_key' => 'Invalid'}}.to_json, :status => 401<br>
end<br>
}<br>
format.any{super}<br>
Suppose we have a pre-registered API_KEY as ‘mprmzayb’, we can use it to generate and retrieve an authentication token!
curl -v -H “Accept: application/json” -X POST -d ‘user”:{“email”:”email@email.com”, “password”:”password”, “password_confirmation”:”password”, “accept_terms”:1}}’ https://localhost/users.json?api_key=mprmzayb -k
<br>
{"user":{"accept_terms":true, "authentication_token":"VBXdiVeCDGQD3Rxtg9qp", "email":"email@email.com", "has_email_reminder":false}}<br>
Now, if you need the task list API requires authentication (i.e. the authenticate_user! before_filter is enabled for index method of TasksController), we need to pass the authentication_token in the request from sign_in/sign_up request.
Assuming you have the index.rabl in place as shown above, lets test this out:
curl -v -H “Accept: application/json” -H “Content-type: application/json” https://localhost/tasks.json?auth_token=VBXdiVeCDGQD3Rxtg9qp -k
<br>
[{"task":{"name":"Inform users","task_type":"email","priority":1,"id":2,"completed_at":null}},{"task":{"name":"Update Bank or Building Society","task_type":"document","priority":1,"id":3,"completed_at":null}}]<br>
Yeah! Devise automatically takes care of authentication, so in the following senerios appropriate error message is returned.
- If the user tries to access tasks after logout with old token
<br>
{"error":"Invalid authentication token."}<br>
- If the user tries to access tasks without authentication token
<br>
{"error":"You need to sign in or register before continuing."}<br>
Enjoy building your APIs in Rails!
Josh Software 
Awesome tutorial, thanks for sharing! Glad to see you using RABL, hope its been helpful.
@nesquena, its been really helpful that is the trigger for this blog post. Thanks a lot for making this gem.
Nice post
Great tutorial, thanks a lot !
i think there is probably a mistake in your filename #app/views/tasks/index.rabl
shouldn’t it be #app/views/tasks/index.json.rabl ? or #app/views/tasks/index.xml.rabl
with your filename, the attributes line wasn’t taken into account
No, don’t have to create two rabl files for each format. This is the major point of this blog post, single rabl file (i.e., app/views/tasks/index.rabl) is sufficient enough to render both the formats based on request content-type. Remember, you need to have gems for json and xml.
Thanks for the tutorial. In terms of Enabling API access with authentication. Assuming i am using omniauth and devise and creating a provider similar to your other helpful post on ‘multiple applications with devise, omniauth and single sign on’, will i need to enable token_authenticatable in devise as describe in this article or it will be sufficient to just use the “uid” in the authentications model of the provider or perhaps the app_secret in the clients model of the provider.
Will use any of that nullify the need to enable token_authenticatable in devise. I following your tutorial on ‘multiple applications with devise, omniauth and single sign on’ but will want the provider to enable api access with authentication.
Thanks.
@Ed,
If you are using omniauth, once you are already signed in, you would have the oauth_token already. So, all API calls would get authenticated automatically if you append the omniauth oauth_token with your call.
I guess you just have to remember to use the parameter as ‘oauth_token’ and not ‘auth_token’ as in the API above. (as is with OAuth 2.0 RFC)
Ok thanks for the explanation.
Awesome ! Thanks a lot !
Good Tutorials..
Hello,
Thanks for this very nice tutorial !!!
I’m working on an rabl / rails API and I’ve started devise integration but I’m facing some issues I do not understand. I’ve detailed the issues I encountered at http://stackoverflow.com/questions/9641079/token-authentication-with-rails-and-devise
Would you have any idea of what is wrong ?
Thanks again for your work,
Luc
It’s my fault, I’ll update the blog post.
You need to add the following code to create the user in you registration controller
if params[:api_key].blank? or params[:api_key] != API_KEY render :json => {'errors'=> {'api_key' => 'Invalid'}}.to_json, :status => 401 return end build_resource if resource.save sign_in(resource) resource.reset_authentication_token! render :template => '/devise/registrations/signed_up' #rabl template with authentication token else render :template => '/devise/registrations/new' #rabl template with errors endLet me know if you face any problem?
thanks a lot, the registration is working fine.
Regarding the sign_in part, my understanding is that the code to modify is in the “create” method of SessionsController, do you confirm ? In this case, are the login and password sent in clear for authentication ?
Do you get any error after registration? After registration, it automatically sign-in user you don’t have do anything.
Do you mean separate sign-in process? If yes, you need add following line in sessions_controller#create method after the sign_in(resource_name, resource)
current_user.reset_authentication_token!
Thanks a lot Sethupathi, that really help. I have both custom registration and authentication working.
Luc
Thanks for the nice tutorial!
But how does the rabl template (app/view//devise/registrations/new.rabl) with errors look like? How about the signed_up rabl template?
In case it helps others, here’s the curl command that worked for me:
curl -v -H “Accept: application/json” -X POST -d “user[email]=email@email.com&user[password]=password&user[password_confirmation]=password” http://localhost:3000/users.json
I assume you are using devise. A sample new.rabl is given below and the name can be different. It will be rendered in case of error while sign-up.
new.rabl
===========
object resource => :user
attributes :errors
Following is the sample json format rendered on error while signup:
“user”:{“errors”:{“password”:”doesn’t match confirmation”,”email”:”has already been taken”}}}
Signed_up.rabl can have all the required fields that needs be given back after successful signup. Following is an example
object current_user => :user
attributes :email, :authentication_token
Regarding curl example for sign_up, all the additional mandatory fields(Customized signup) for signup needs to posted.
Thanks Sethupathi! Your example rabl is very helpful.
With curl example, my signup is currently simple without api_key and accept_terms. But I agree those things are good (plus SSL) and will add 🙂
thanks for the tutorial.. do you mind post your ruby project? I think I miss something in my code (new.rabl, signed_up.rabl etc) so my response is always { }, also how do I get rid of this warning
WARNING: Can’t verify CSRF token authenticity
I would not be able to post the project since it is a private one. If you could share you controller code snipper and rabl file that would help in identify the issue. Or I need to write a sample project and post it.
two follow up questions, 1) if the user has previously registered, how would the user get his auth_token from sign_in? it appears the users/sign_in, which is handled by session_controller, does not return the auth_token 2) once the user got his auth_token, is there a quick method to sign-out the user (delete the auth_token) via curl command?
1) In SessionsController#create, method you need to have following as mentioned in the blog
resource.reset_authentication_token! and attribute accessor in the user model for this field. This will return the json with authentication_token.
2) You should write the following in SessionsController#destroy and call the url through curl. This will sign_out and clears the authentication token.
current_user.authentication_token = nil
current_user.save
super
This is very, very helpful — thank you. Perhaps you can help with a slightly different question. I have not yet moved to :token_authenticatable for my API, so I am just passing the session cookie in my requests, which works fine. So with JSON and RABL I can log users in, create new users, and most other operations.
But I am getting an error when I try to execute the Devise::RegistrationsController#update method. The first line of this method is:
self.resource = resource_class.to_adapter.get!(send(:"current_#{resource_name}").to_key)which is generating the error
NoMethodError (undefined method `to_key' for nil:NilClass)I am sure I need to override the update method, similarly to how you have done in your post with create, but I am a little unclear what code it should contain (my case is a little different), and if it needs to have anything to do with RABL, or if this is just a Devise thing. I am clearly confused 🙂 Any help greatly appreciated!
Tom
Could you please check after login/signup the “current_user” is set/available through out the session? I guess, it is not getting set that is the reason for the error. I haven’t check api access by passing the session cookie through rabl. I’ll try to check it out and let you know.
Hi Sethupathi,
Is the reason why you are calling resource.reset_authentication_token! in RegistrationController because you are login the user in during registration? I don’t currently do that (User needs to be invited before login in). In this case, should the resource.reset_authentication_token! go in the SessionsController.create function? How do I override the SessionsController.create to call resource.reset_authentication_token! ?
Also, I’m using the default non-API login process, where SessionsController.create handles the login and renders html. How do I retrieve the authentication_token and reuse it as part of the API calls later?
Yes, that is reason why was calling resource.reset_authentication_token!. In anycase, you need to override sessions_controller#create method for token_authentication api login. Following is a snippet for overriding sessions_controller#create for API and keep the format for html as it is.
format.json{ if !params[:api_key].blank? and params[:api_key] == API_KEY resource = warden.authenticate!(:scope => resource_name, :recall => "#{controller_path}#new") sign_in(resource_name, resource) current_user.reset_authentication_token! respond_with resource, :location => after_sign_in_path_for(resource) else render :json => {'errors'=>{'api_key' => 'Invalid'}}.to_json, :status => 401 end }Thanks…. I’ve been playing around… One interesting thing I noticed is that if I sign in via non RESTful method with Devise, I can call API’s that require authentication without needing the authentication token.
Also, in my custom SessionsController.create
I can’t call “super” right? I need to completely override Devise’s SessionsController.create?
Yes, if you are using it through same browser session. However it will be for same user.
You can call super from your SessionsController.create
format.html{ super }Thanks it saved my day.
I think I’m missing something here… How did you setup and register the api_key? This will be a big help, thanks. 🙂
API_KEY is just an additional check to access the initial sign up. To make sure no one else other than who knows API can signup through API access(whoever will affiliate with you will get/know the api key). If you want to keep it open you can get ride of it.
Do you have or know of any working example for this anywhere? Also what version of devise is this based on? I am having a number of issues with version 2 as I cant seam to find any good documentation or examples anywhere… 😦
I was using Devise 1.5.3, but it should work with 2.0 as well. I will spend sometime to create a sample rails app to demonstrate this in a day or two.
This would really be a big help to me and im sure to other too. This seam to be the one part of the devise documentation that is really lacking.
great tutorial. I have been struggling with this api authentication.
I was able to run the first cURL command :
curl -v -H "Accept: application/json" -X POST -d "user[email]=sth13@gmail.com&user[full_name]=ABCD&user[password]=password&user[password_confirmation]=password" http://localhost:3000/users.json?api_key=testAPI -kbut it didn’t return an authentication token needed for the following step. Any thoughts?
+++++ This is the response header:
Status Code: 201 Created
Cache-Control: max-age=0, private, must-revalidate
Connection: keep-alive
Content-Length: 222
Content-Type: application/json; charset=utf-8
Date: Mon, 24 Sep 2012 06:32:09 GMT
Etag: “121b4e32ff1af0841529869ff338eb56”
Location: /yells
Server: thin 1.4.1 codename Chromeo
Set-Cookie: _YELL-around_session=BAh7B0kiGXdhcmRlbi51c2VyLnVzZXIua2V5BjoGRVRbCEkiCVVzZXIGOwBGWwZpFyIiJDJhJDEwJDdMVEFVQUg5UVA3ZGVSMi9 SL1NDaHVJIg9zZXNzaW9uX2lkBjsARkkiJWRhMmUwNDU3MDI3NWY3NTUwNzEzOGExNjRlOWQxMjM0BjsAVA%3D%3D–afca0b737b55daebd5d9c351c36764fdc7d4faf0; path=/; HttpOnly
X-Rack-Cache: invalidate, pass
X-Request-Id: a2c44c169a0de9a6a1d6cb236f1bcd98
X-Runtime: 0.095680
X-UA-Compatible: IE=Edge,chrome=1
++++++ and the response body
{
“about”: null,
“category”: null,
“created_at”: “2012-09-24T06:32:09Z”,
“email”: “sth13@gmail.com”,
“full_name”: “ABCD”,
“id”: 18,
“image_url”: null,
“password”: “password”,
“provider”: null,
“uid”: null,
“updated_at”: “2012-09-24T06:32:09Z”
}
here is my devise/registrations/signed_up.rabl
object current_user => :user
attributes :email, :authentication_token
Did you add the attr_accessible for authentication_token in user model file?
attr_accessible :authentication_token
+++++ User model
devise :database_authenticatable, :token_authenticatable, :registerable, :omniauthable,
:recoverable, :rememberable, :trackable, :validatable
# Setup accessible (or protected) attributes for your model
attr_accessible :email, :password, :password_confirmation, :authentication_token,
:remember_me, :full_name, :provider, :uid, :category, :about
+++ User table in the database also contains authentication_token column.
authentication_token | varchar(255) | YES | | NULL |
As modifying the registration controller, would I need to update the route as well?
# current route
devise_for :users, :controllers => { omniauth_callbacks: “omniauth_callbacks” }
Yes, if you have modified/overridden devise registration controller you need update the route
devise_for :users, :controllers => { omniauth_callbacks: “omniauth_callbacks”:registrations => “registrations”}. Also, make sure you have added the following code in you sign_up and sign_in api calls
resource.reset_authentication_token!
I did try to change that too. Seems like the controller is causing an issue.
After I updated the route and run the cURL json post, I got this following error.
+++++ error:
Started POST “/users.json?api_key=testAPI” for 127.0.0.1 at 2012-09-25 00:52:43 -0500
Processing by RegistrationsController#create as JSON
Parameters: {“user”=>{“email”=>”sth17@gmail.com”, “full_name”=>”ABCD”, “password”=>”[FILTERED]”, “password_confirmation”=>”[FILTERED]”}, “api_key”=>”testAPI”}
WARNING: Can’t verify CSRF token authenticity
Completed 500 Internal Server Error in 1ms
ArgumentError (too few arguments):
app/controllers/registrations_controller.rb:3:in `format’
app/controllers/registrations_controller.rb:3:in `create’
+++ This is the Registration controller that I followed on your tutorial
class RegistrationsController ‘/devise/registrations/signed_up’ #rabl template with authentication token
else
render :template => ‘/devise/registrations/error’ #rabl template with errors
end
else
render :json => {‘hello error’=>{‘api_key’ => ‘Invalid’}}.to_json, :status => 401
end
}
format.any{super}
end
end
Oh .. I think the issue is that I currently don’t have the sign_in api set up yet.
So when the registration is triggered, this “sign_in(resource)” line causes the issue.
I found the real issue is the format json{} – which caused this error
ArgumentError (too few arguments):app/controllers/registrations_controller.rb:3:in `format'
app/controllers/registrations_controller.rb:3:in `create'
Seems like the syntax is different on the newer rails version.
this is my controller after replacing with response_to
class RegistrationsController '/devise/registrations/signed_up' #rabl template with authentication token
else
render :template => '/devise/registrations/error' #rabl template with errors
end
else
render :json => {'hello error'=>{'api_key' => 'Invalid'}}.to_json, :status => 401
end
end
end
class RegistrationsController ‘/devise/registrations/signed_up’ #rabl template with authentication token
else
render :template => ‘/devise/registrations/error’ #rabl template with errors
end
else
render :json => {‘hello error’=>{‘api_key’ => ‘Invalid’}}.to_json, :status => 401
end
end
end
Great. I guess, you are issue is resolved now. My long pending plan to have sample project for this. I think, it is high time for me to generate one.
Sample project would be helpful to other folks.
I have another question about passing parameters. Along with authentication_token, I want to test if I can pass two other parameters (lat & lng) to the tasks controllers
+++ This is my cURL request looks like
curl -v -H “Accept: application/json” -H “Content-type: application/json” https://localhost/tasks.json?auth_token=VBXdiVeCDGQD3Rxtg9qp&lat=44.9833&lng=-93.2667 -k
+++ Task controller
def index
@lat = params[:lat]
@lng = params[:lng]
@tasks = Task.near([@lat, @lng], 5) #Show task near this location.
end
+++ However, seems like it only passes the 1st parameter (authentication_token).Below was the log. Oddly, I was able to pull out the list of tasks when I load that link on browser.
Started GET “/tasks.json?auth_token=yJj6mUMYxbExnqFc8zMk” for 127.0.0.1 at 2012-09-26 01:19:22 -0500
Processing by TasksController#index as JSON
Parameters: {“auth_token”=>”yJj6mUMYxbExnqFc8zMk”, “task”=>{}}
Rendered tasks/index.rabl (0.3ms)
Completed 200 OK in 9ms (Views: 0.8ms | ActiveRecord: 4.0ms)
This seems weird, I have used with additional parameter and it works for me. Follow is an example where I could get the category values as parameter.
https://localhost:3000/tasks.json?auth_token=AprqKmzSnL7RedLf8pSH&category=test2
I will add this in my sample project as well
Strangely, I tried it to make GET request using RESTclient FF add-on and it worked like charm.
aha, you need to do url encoding for the ‘dot’ in the lat & lng with %2E through curl or wget. Browser automatically encodes it.
Eg. curl -v -H “Accept: application/json” -H “Content-type: application/json” https://localhost/tasks.json?auth_token=VBXdiVeCDGQD3Rxtg9qp&lat=44%2E9833&lng=-93%2E2667 -k
My issue got resolved a while ago and didn’t see your post .
Anyway, +1 .. Thank you so much for the help!
I am using SSO. and i dont want to use API for http get and http post. so when user gets logged out from SSO portal it send a request to other application running with SSO portal.as i dont want to use API den how can i call a method on the basis of request coming from SSO portal??
I don’t understand your question. This article is all about API usage.
please update your blog you said on http://stackoverflow.com/questions/9641079/token-authentication-with-rails-and-devise
Yasir, it is already updated as i mentioned. Just change in the condition from what you see it in stackoverflow reply.
Nice tutorial, however I didn’t understand how to Modify RegistrationsController create method.
A step by step details should be nice for newby (like me).
Finally, I got sometime to create a rails application for reference. Here is the github url: https://github.com/sethu/rabl-example.
I will try add comments in code the explain each steps.
I’m wondering what you would do for authentication when someone wants to consume the resource via json API call from their website. In such a case, their token is available for anyone to see (who goes View Source) on their website. So how could one restrict this to “have API key equal to “X” and respond to requests from “Y.com” ?
Oops, I have missed this. You can send the json request from server side.
Hi,
Great post! This is the thing i had been looking for. You have saved my life. But, i do a question that i am working on a Rails project alongwith an mobile app. It has authentication Devise and access control by CanCan. I want to know how i can role based access control by using CanCan in mobile app. Please help me.
Dev, it is straight forward. Once the user is authenticated by devise in rails app, the app knows the logged in user and you should be able to identify the user role. In ability.rb, you might have defined the access control. It should automatically work with load_and_authorize_resource in appropriate controllers. Let me know if you have any query.
First of all, i thank you for your reply. Second, I want to know how i can do role based access control in Mobile app, as my Rails project has this option by CanCan. Can it be possible to do this in REST API.
REST API request will go to rails app, you can manage the access control by identifying user using authentication token. If you are asking about how to do access control from mobile app before sending REST API request then you need to check with the mobile app framework spec which you are using.