Testing React-Redux App with Jest

Content posted here with the permission of the author, Kiran Deshmukh, who is currently employed at Josh Software. Original post available here.

We often get confused about selecting testing framework for our application. Currently, I am working on a ReactRedux based project. While selecting the testing framework, we compared some of the popular JavaScript testing frameworks. We found that Jest is the best fit for testing our application.

Jest is not limited to ReactJs testing. We can test any JavaScript code using Jest. It can be used to test asynchronous code.

In React – Redux project, we will have a single store containing state of the application. We will have actionCreators which return action type and payload(may be the response from API). Reducer will contain actual logic to update store for a particular action. Components listen to the reducer. So, when state of the reducer changes, component will be re-rendered.

Here, we will discuss how Jest helped us for testing actionCreators, reducers, components in our project. ActionCreators return the actions. So, we are not testing actions .

Suppose we have a file friendListActions.js which contains string literals for the actions:

const friendListActions = {
  fetchFriendList: 'FETCH_FRIEND_LIST',
  fetchingFriendListSucceeded: 'FETCHING_FRIEND_LIST_SUCCEEDED',
  fetchingFriendListFailed: 'FETCHING_FRIEND_LIST_FAILED'

};

export default friendListActions;

Suppose, we have following file friendListReducer.js.We are changing the state in the reducer based on the actions.

import friendListActions from 'friendListActions.js';

//Set the initial state for this reducer.
const initialState = {
  isLoading: false,
  errorMsg: null,
  friendList: []
};

//Here is our business logic to change state in the reducer.
const friendListReducer = (state = initialState, action) => {
  switch (action.type) {
    case friendListActions.fetchFriendList:
    case friendListActions.fetchingFriendListSucceeded:
    case friendListActions.fetchingFriendListFailed:
      return { ...state, ...action.payload }
    default:
      return state;
  }
}
export default friendListReducer;

Suppose we have the following file friendListActionCreators.js file containing action creators for fetching friend list. We are handling success as well as error response while fetching the friend list.

import friendListActions from 'friendListActions.js';

//This actionCreator is to initialise the fetching of friend list
export const fetchingFriendListInitiated = () => {
  //actionCreator is returning an action object.
  return {
    type: friendListActions.fetchFriendList,
    payload: {
      isLoading: true
    }
  }
}

//This actionCreator is used when friend list is fetched successfully.
export const fetchingFriendListSucceeded = ( friendList ) => {
  //actionCreator is returning an action object.
  return {
    type: friendListActions.fetchingFriendListSucceeded,
    payload: {
      isLoading: false,
      errorMsg: null,
      friendList
    }
  }
}

//This actionCreator is used when failed to fetch friend list.
export const fetchingFriendListFailed = ( errorMsg ) => {
  //actionCreator is returning an action object.
  return {
    type: friendListActions.fetchingFriendListFailed,
    payload: {
      isLoading: false,
      errorMsg
    }
  }
}

export const fetchFriendList = () => {
  return( dispatch => {
    dispatch( fetchingFriendListInitiated() )

    //Here we are fetching the friend list for user having 23 as id.
    return fetch("http://social-media.com/23/friends")
    .then(successResponse => {
      dispatch(
        fetchingFriendListSucceeded(
          successResponse.response
        )
      )
    })
    .catch(errorResponse => {
      dispatch( fetchingFriendListFailed(errorResponse.message))
    })
  })
}

We are making ‘fetch’ call to the respective API.

Let us observe the test cases for the reducer. We have the following file friendListReducer.test.js:

import reducer from 'friendListReducer.js';
import friendListActions from 'friendListActions.js';

const expectedInitialState = {
  isLoading: false,
  errorMsg: null,
  friendList: []
}

//'describe' is used to create 'test suite' containing multiple test cases.
describe('Friend List Reducer', () => {
  it('returns a state of reducer when succeeded to fetch the friend list', () => {
    let expectedPayload = {
      isLoading: false,
      errorMsg: null,
      friendList: [
          'John', 'Emraan', 'Sukanya'
      ]
    }

    expect(
      // "reducer" takes 2 arguments:
      // first argument: state of reducer before applying the action
      // second argument: Plain JavaScript Object containing "action" and "payload"
      reducer(expectedInitialState, {
        type: friendListActions.fetchingFriendListSucceeded,
        payload: expectedPayload
      })
    ).toEqual({ ...expectedInitialState, ...expectedPayload })
  })
})

State in the reducer should be changed when some action is performed. Here, we are testing whether this state is changing as per expected payload or not.

Here, we are using expect() and toEqual() methods provided by Jest. Also, it provides describe to create a test suite and it to create an individual test case.

Let us test which actions will be performed when friend list is fetched successfully from the API. Let we have file friendListActionCreators.test.js:

import configureMockStore from 'redux-mock-store';
import thunk from 'redux-thunk';
import * as fetch from 'jest-fetch-mock';

import friendListActions from 'friendListActions.js';
import { fetchFriendList } from 'friendListActionCreators.js';

//We are creating a mock store here.
const middlewares = [thunk];
const mockStore = configureMockStore(middlewares);

describe('fetchFriendListSucceeded()', () => {

  it('returns friend list in response', () => {
    const getFriendList = [
      'John','Emraan', 'Sukanya'
    ]

    const jsonResponse = {
      "method": "getFriendList",
      "response": getFriendList
    }

    //We are mocking only one http fetch response
    fetch.mockResponseOnce(JSON.stringify(jsonResponse));

    let store = mockStore({
      friendList: {}
    })

    let expectedActions = [
      {
        type: friendListActions.fetchFriendList,
        payload: {
          isLoading: true
        }
      },
      {
        type: friendListActions.fetchingFriendListSucceeded,
        payload: {
          isLoading: false,
          errorMsg: null,
          friendList: getFriendList
        }
      }
    ]
    //We are returning 'promise' due to asynchronous actions.
    return (
      store.dispatch(fetchFriendList())
      .then(() => {
        expect(store.getActions())
          .toEqual(expectedActions);
      })
    )
  })
})

Here, while testing, we should mock the Redux store. Since we are using fetch call to the API,our store is getting responses from asynchronous actions. As Redux only supports synchronous code, we require middle-ware f so that it supports asynchronous code. While writing test cases, we should mock middle-ware also.

Jest can’t mock http  fetch calls, Redux store and middle-ware which will be used in store. So, we need to add some other packages for mocking these things.

Here, we have used ‘redux-mock-store’ to create a mock store and ‘redux-thunk’ to mock the middle-ware. We are using ‘jest-fetch-mock’ package to mock http fetch calls. We have used ‘mockResponseOnce()’ method since we want to mock only one API call.

After mocking API call, it will return a static value. But, our actual code expects a promise object to be returned from API. So, for simulating the same behaviour, we are returning the promise object in the test case.

Snapshot testing:

When we don’t want to change UI components unexpectedly, Snapshot testing will be useful.Jest provides this  amazing feature.

Snapshot for the component is created when test case for the component is run for the first time. So, when I am running Snapshot test cases for the first time, they will pass successfully. This shows that Snapshot testing is not Test Driven Development(TDD). For making it TDD, we can use enzyme package along with it.

Jest creates a new folder __snapshots__ under the current working folder of the test cases and the snapshots will be stored here. These snapshots will be in human readable format. When I run test cases afterwards, the component will be compared with it’s existing snapshot. If some modifications are done in the component, test case will fail. If these changes are desired, we can change the existing snapshot.

Let us have a list component in list.js file:


import React, { Component } from 'react';

class List extends Component {

  render() {
    const flowers = ['Lily', 'Lotus', 'Rose']
    return (
<div>
<h2> List of flowers</h2>
<ul>
          {
            flowers.map(( flower, index ) =&gt;
	<li key="{" index="" }=""> { flower }</li>
)
          }</ul>
</div>
)
  }
}
export default List;

We will write test case for this component to create it’s snapshot:

import React from 'react';
import renderer from 'react-test-renderer'
import List from 'list.js';

it('renders list component correctly', () =&gt; {
  const tree = renderer.create(	<list>).toJSON()
  expect( tree ).toMatchSnapshot()
})

 

For creating snapshot of a component, first we have to create Json object of that component. So, we will create JavaScript object for the component and will convert it to Json. We are unable to create JavaScript object of the component in Jest. So, we have used ‘react-test-renderer’ .

Jest provides ‘toMatchSnapshot()’ method to create snapshot(if it is absent) for that component. Next time, when I want to test List component, it is compared with existing snapshot.

If there is any change in List component, the test case will fail. We can update the snapshot to reflect these changes in snapshot. We should commit these snapshot files along with other test files. You can find more information here.

If we want to add assertions in component testing, want to check manipulations in the components, we can use enzyme package along with Jest framework. enzyme will add TDD in out UI component testing.

In short, Jest will create component tree structure and we can traverse this component tree with the help of enzyme. This package doesn’t have it’s own assertion library. So, we can use assertion library provided by Jest.

We can test following things with the help of Jest and Enzyme:

  • We can test state changes in the components.
  • We can test conditional parameters passed in the component. E.g. Suppose the className of div tag is calculated at run-time based on the received props, we can test it.
  • We can test event handling in the component.
  • We can test component life cycle callbacks. Here we can test whether desired function is called from that life cycle hook or not.

Here are my observations about testing React – Redux application with Jest:

  1. Jest provides a very good assertion and mocking library. We can test asynchronous code with the help of it. If you are new to testing ReactJs application, Jest will be the best choice. Due to parallel testing, it is a great choice for large projects.
  2. We cannot mock http fetch calls with Jest. We can use package like ‘jest-fetch-mock’ for it.
  3. We cannot mock Redux store with Jest. We can use packages like ‘redux-mock-store‘ to create mock store and ‘redux-thunk’ to provide middleware for the store.
  4. Snapshot testing is one of the best features provided by Jest. It is useful to check whether UI is changed unexpectedly or not.
  5. Snapshot testing creates a component tree. We are unable to traverse through this component tree using Jest only. We can use enzyme package along with Jest for it.

To sum it up, I think, Jest is really good framework for testing ReactJs part of the application. Using some packages like ‘redux-mock-store’, ‘redux-thunk’, ‘jest-fetch-mock’, ‘enzyme’ along with Jest, we can test entire React-Redux application.

Advertisements
Posted in General | 1 Comment

Raspberry Pi with GOBOT (Golang) Part I

This all started with developing a simple software to support the robotic project. We had initially planned with Arduino and Gobot (A golang framework for robotics). After we tested some initial sample programs we realised that we can’t run the … Continue reading

Gallery | Tagged , , | Leave a comment

Rails views to ReactJS

Content posted here with the permission of the author, Yogesh Khater, who is currently employed at Josh Software. Original post available here.

Have you ever been introduced to a Rails application having JS/jQuery scattered around app/assets/javascript or app/views or even in app/helpers directory for that matter ? Wondering if there is any consistency present while adding any JS snippet, you think would it be better to have some conventions to follow in here too. Whilst there are lot of other options available to help us out, this blog is about using react-rails to bootstrap with ReactJS and then decoupling it from Rails to a complete UI only application.

♦ But why ?

Few reasons that I could think of,

  • Reactiveness, Of course.
  • Performance is certainly better with client-side rendering.
  • View layer testing gets easier because of the component based architecture.
  • Having Rails as an API only application has its own advantages like better performance (by removal of some middlewares), well documented APIs, faster deployments etc.

♦ Inception

To start with the decoupling, start introducing ReactJS into the application using rails-react gem. But before going further, prerequisite is that you’ve some idea about ReactJS, Redux and JSX.

Usage of react-rails gem is pretty simple. To start off, you could start conversion with one simple Rails view or even a div from the Rails view into a React component.

Lets say you’ve a Rails partial which renders the navigation bar containing some links and username, so creating a NavigationBar component would be like,

$ rails g react:component NavigationBar username:string --es6

which creates app/assets/javascripts/components directory having navigation_bar.jsx in it. After adding its rendering logic, you can replace

render partial: 'navigation_bar', locals: { username: current_user.username }

with

react_component('NavigationBar', username: current_user.username)

So by using above approach, you could start replacing partials and views into React components.

Just to make sure that our goal is to decouple the view layer from Rails, don’t pass too many props which are bound with Rails methods/variables. It would be difficult or rather time consuming to remove such references and repopulate the props values while separation.

♦ Architecture of components

As our final goal is to have well architected and maintainable ReactJS application, we should follow the basic conventions for it.

Assuming you would be using react-redux as a state maintainer, the directory structure that any React-Redux application follows basically has,

  • src/
    • components
    • containers
    • actions
    • actionCreators
    • reducers
    • store.js

So in this scenario also, you could add above directories along with components directory in app/assets/javascripts. You could refer to an example of such architecture.

♦ Using NPM packages over bundled gems

You might have installed some gems for packages like select2, moment.js etc in your Gemfile. But those can’t be used after the decoupling. So a better way is to start using NPM packages over such gems.

To do so, you can start using https://rails-assets.org which converts the NPM packages into respective gems and then adds them to the asset pipeline.

# Gemfile

source "https://rails-assets.org" do
  gem 'rails-assets-moment'
  gem 'rails-assets-select2'
  ...
end

♦ Using rails-api

In time, you would also need to start replacing your controller actions with API based controller actions. If your application is using Rails 5, then it has the builtin support for ActionController::API (a class extracted from ActionController::Base with minimal requirements supporting API actions), but for applications with Rails < 5, you would need to install rails-api gem.  Once the ActionController::API is available, you could add app/controllers/api/v1 directory to start with API based controller actions.

Note that while initialisation of your application, Rails requires all the middlewares by default using require 'rails/all' in config/application.rb. But after the decoupling, we won’t be needing all the middlewares. So do remember to remove require 'rails/all' and keep only the required middlewares.

BTW, inheriting controllers from ActionController::API won’t process requests through additional middlewares (like for rendering the views), so you don’t have to worry if you’ve decided to keep all the middlewares.

♦ The Decoupling

Say you’ve reached a level in your application where

  • Views has only single line for rendering the specific React component
  • No more partials
  • No more view helpers
  • No more HAML, ERB or any other Rails template code in view layouts
  • No more controller actions redirecting or rendering the Rails views.

then your application is in the right place to start the decoupling.

You can use create-react-app or any other boilerplate template to create a React application and start copying the required directories from app/assets/javascripts to it.

Some points to be considered after the migration,

  • Add the dependencies that you’ve mentioned in Gemfile under https://rails-assets.org group into the package.json.
  • You would need to add authentication module in your main component as that was handled by Rails previously.
  • Finally, add top-level components inside BrowserRouter (or any other navigational component) which are rendered in Rails views and then remove app/views.

Your ReactJS application will be up in no time and the best thing is that there won’t be any downtime required while switching from Rails views to ReactJS on your servers !!

Thanks for reading, Happy decoupling !!

Posted in General | Leave a comment

ProTips PostGres 1: Looking through the Postgres Window

Content posted here with the permission of the author, who is currently employed at Josh Software. Original post available here.

Problem – 

We encountered a situation where we had to select only one record for each values of a column (lets say ‘user_id’) ordered by a different column (lets say ‘price’) in a certain table (lets say ‘purchases’). Doing this active-record way was very inefficient. By active-record way, we had to:

  1. Fetch all the user_ids from purchases table
  2. Loop through all the user_ids, for each user_id
    1. Select once record ordered by price

This approach lead us to query the database number of user times + one query for selecting all the user_ids (N + 1 query problem).

Solution – 

We struggled to find a better solution in active-record for this problem and window functions of postgresql came to the rescue.

To understand postgres window functions, we must understand GROUP BY clause. We use GROUP BY clause to apply aggregate functions on a subset of rows of a table.

For example you have a table called “purchases” with the schema as shown below.

  CREATE TABLE purchases (
    purchase_id serial PRIMARY KEY,
    product_id INT NOT NULL,
    price DECIMAL (11, 2),
    user_id INT NOT NULL
  );

Lets insert data into this table.

 INSERT INTO purchases (product_id, price, user_id)
 VALUES
 (1, 10, 1),
 (2, 20, 1),
 (3, 30, 1),
 (4, 40, 2),
 (1, 10, 2),
 (5, 50, 2),
 (6, 30, 2)

 

To calculate average of all the purchase prices, we will use AVG aggregate function.

  SELECT AVG (price) FROM purchases;

This returned the average of prices for both users with id 1 and 2.

  avg
 --------
  27.1428571428571429

 

Now, if you want to find AVG (A postgres aggregate function) of the price of items that a specific user has purchased. You would use GROUP BY clause on user_id. Thus the following statement will return average for each user_id.

  SELECT AVG (price) FROM purchases GROUP BY user_id;

Above query return the following

 user_id          avg
 --------      --------
    1          20.0000000000000000
    2          32.5000000000000000

As you can see,

The AVG aggregate function has reduced the number of rows returned by the sql query.

The GROUP BY clause groups the rows of the table into subset of rows based on a column(s).

Similarly, a window function also operates on a subset of rows of a table but it does not reduce the number of rows. It returns the actual records of the table in the output with an extra column “avg” if you have applied AVG function to the prices.

In the query below the AVG function works as a window function that operates on a set of rows specified by the OVER (PARTITION BY) clause.

For example, below query will return the product_id, user_id, price and average from “purchases” table for each user group.

  SELECT
  product_id,
  user_id,
  price,
  AVG (price) OVER ( PARTITION BY user_id )
  FROM
  purchases

The output

  product_id    user_id   price   avg
 ------------ ---------- ------- -----
      1           1       10.00   20.0000000000000000
      2           1       20.00   20.0000000000000000
      3           1       30.00   20.0000000000000000
      4           2       40.00   32.5000000000000000
      1           2       10.00   32.5000000000000000
      5           2       50.00   32.5000000000000000
      6           2       30.00   32.5000000000000000

You can also apply other operations with PARTITION BY clause. Like if you want the results to be returned in an order, you can use ORDER BY clause. You can also use built in window functions like row_number(), rank(), etc.

These built-in window functions adds a number to each row based on their order. The row_number() function assigns a serial number to each of the rows returned. So if you want a limited number of records for each user_id, you can use it.

The query below will return only one record for each user_id, decreasingly ordered by their purchase price.

  SELECT * FROM
  (
    SELECT
    product_id,
    user_id,
    price,
    row_number() OVER ( PARTITION BY user_id ORDER BY price DESC )
    FROM
    purchases
  ) tmp
  WHERE tmp.row_number &amp;amp;lt;= 1

In the above query, I have ordered each partition by decreasing price and then added limit on the number of rows for each partition by using the row_number() function. Below is the output.

  product_id   user_id   price   row_number
------------- --------- ------- ------------
     3           1       30.00       1
     5           2       50.00       1

Postgres provides many built-in window functions. Some of them are explained below.

RANK() function

The RANK() function assigns ranking within an ordered partition.  If the values of the two rows are the same, the  RANK() function assigns the same rank, with the next ranking(s) skipped.

  SELECT
  product_id,
  user_id,
  price,
  RANK () OVER ( PARTITION BY user_id ORDER BY price )
  FROM purchases

The output

 product_id    user_id    price   rank
 ------------ ---------- ------- -----
    1            1        10.00    1
    1            1        10.00    1
    2            1        20.00    3
    3            1        30.00    4
    1            2        10.00    1
    6            2        30.00    2
    4            2        40.00    3
    5            2        50.00    4

 

DENSE RANK()

The DENSE_RANK() function assigns the ranking within an ordered partition, but the ranks are consecutive. In other words, the same ranks are assigned to multiple rows and no ranks are skipped.

  SELECT
  product_id,
  user_id,
  price,
  DENSE_RANK () OVER ( PARTITION BY user_id ORDER BY price )
  FROM purchases

The output

  product_id   user_id    price   rank
 ------------ ---------- -------  -----
      1         1        10.00     1
      1         1        10.00     1
      2         1        20.00     2
      3         1        30.00     3
      1         2        10.00     1
      6         2        30.00     2
      4         2        40.00     3
      5         2        50.00     4

There are many other window functions that are listed below.

  1. FIRST_VALUE()
  2. LAST_VALUE()
  3. NTH_VALUE()
  4. PERCENT_RANK()
  5. LAG()
  6. LEAD()

 

In the next blog I will share performance analysis of using postgres window functions vs active-record way for same task.

 

Posted in General | Leave a comment

Securing your web applications

Content posted here with the permission of the author, who is currently employed at Josh Software. Original post available here.

While developing web apps, how frequently do you think about securing it?

Just wait for a moment and think!

You do? Really? That’s Awesome!

This blog will act like a checklist for you. Go through it once, to figure out just in case you have missed anything OR maybe I might have missed something that you have taken care of. Either way, please be kind enough to drop a comment.

You don’t? No worries, let’s do it together. Let’s understand:

  • What are the various types of attacks?
  • How do they work?
  • How to secure your app from such attacks?

Alright, without any further ado, let’s get started.

Cross-Site Request Forgery (CSRF)

It’s a kind of attack that forces an end user to execute unwanted actions on a web application in which they’re currently authenticated. These attacks target state-changing request, not theft of data.

csrf explanation

Consider a scenario wherein a genuine admin user logs in to your web app (http://www.genuine-domain.com). When login form is submitted, server validates the credentials and sends a cookie in response. Now, for each subsequent request to your domain, browser will automatically send along this cookie. Server will verify the existence and contents of the cookie before considering the request authenticated.

All sounds good? Yeah? Think again!

Your web app is vulnerable to CSRF attacks right now. How?

Let’s say the admin user is authorized to delete some other user’s account. Also assume that URL for such endpoint is accessible over GET request. Now, open a new tab, type this URL and press enter. Voila! you just deleted a user.

Now think, how difficult is it for a malicious user to trick your customer into clicking such a URL? Not very difficult? Right?

Just by following certain basic rules, you can protect your app from such attacks:

  1. Show a confirmation prompt before every destructive action
  2. Use GET verb for read-only requests. Use POST/PUT/PATCH/DELETE for write operations.

How do non-get verbs help? Did you notice that only URLs exposed over GET verb can be opened from browser’s URL bar (and hence the links). By using non-get verbs, you are making the hackers job a little more difficult.

The hacker may still trick your customer into submitting a form, which would trigger a POST/PATCH/PUT/DELETE request.

We can step-up our security by including a one-time disposable secure token in every form rendered by the server. Now, if a form is submitted without a token OR with an invalid token, the server would simply reject it. Only our server and genuine client (browser tab) knows about this token. Additionally, verify the Origin and Referrer header whenever possible. Please note that Origin header may not always be present (specially in GET requests). Similarly, Referrer header is not guaranteed to be present in every request.

Cookie Stealing/Hijacking

The attacker can also modify the contents of the cookie OR just copy it. To mitigate this risk, your web app needs to ensure the following while setting the cookie:

  1. User “session” cookies – they are deleted when browser/client is shut down.
  2. Turn on the “secure” flag on the cookie – only accessible over HTTPS.
  3. Turn on the “HttpOnly” flag on the cookie – inaccessible using JavaScript API’s.

Network Sniffing

An attacker can always intercept your network requests and get access to sensitive information. To secure your data during transit, we need to encrypt it. This can be done by using HTTPS certificates.

However, using HTTPS alone is not sufficient. HTTPS encrypts data using some ciphers. You encryption is only as strong as your ciphers. Older version of HTTPS are known to have vulnerabilities. Therefore, I urge you to use HTTP Strict Transport Security (HSTS).

CORS (Cross-Origin Resource Sharing)

Similar to  HTML requests, we also need to protect our web app from AJAX based security threats. AJAX calls can be triggered either from your domain OR from a different domain. Luckily, most modern browsers block cross-domain AJAX calls by default. But, what if a genuine user wants to share resources across the domains?

Secure inter-domain AJAX based communication is handled by using CORS (Cross Origin Request Sharing). Here is an excellent blog on implementing CORS.

NOTE: CORS restrictions are only applicable on AJAX requests (i.e. not applicable on HTML requests).

Clickjacking

Clickjacking (User Interface redress attack OR UI redress attack OR UI redressing) is a malicious technique of tricking a Web user into clicking on something different from what the user perceives they are clicking on, thus potentially revealing confidential information or taking control of their computer while clicking on seemingly harmless web pages.

clickjacking explanation

In order to secure your web app from clickjacking attacks, the server needs to set X-Frame-Options: DENY. This means, your web app cannot be opened inside an iframe.

If you want to allow rendering of your app inside an iframe from your domain only, you can set:

X-Frame-Options: SAMEORIGIN
OR
X-Frame-Options: ALLOW-FROM <whitelisted-domain>

SQL Injection

A SQL injection attack consists of insertion or “injection” of a SQL query via the input data from the client to the application. A successful SQL injection exploit can read sensitive data from the database, modify database data (Insert/Update/Delete), execute administration operations on the database (such as shutdown the DBMS), recover the content of a given file present on the DBMS file system and in some cases issue commands to the operating system.

Bind variables are the best way to prevent SQL injection. When using bind parameters you do not write the actual values but instead insert placeholders into the SQL statement. That way the statements do not change when executing them with different values. The question mark (?) is the only placeholder character that the SQL standard defines.

Cross-Site Scripting (XSS)

Cross-Site Scripting, commonly known as XSS, is a vulnerability that is often found in web apps. XSS allows attackers to inject client-side scripts into public facing web pages and, in many cases, can be used by attackers to work their way past access controls.

This is done by tricking a browser so that it accepts data from an untrusted source, and this typically happens when attackers use familiar code (such as JavaScript, for example) as developers don’t scrub out these characters.

xss explanation

To protect your web app from XSS attacks, you need to:

  1. Never trust user input. i.e. Always escape/sanitize input before inserting into:
    1. HTML element’s content. e.g. innerHTML, $(‘#some-id’).html(), etc.
    2. Values of common attributes. e.g. class, data-*, etc.
    3. HTML or JS comments
    4. Style property values
    5. URL parameters
  2. Whenever possible use an Auto-Escaping Template System
  3. Avoid eval() or such dynamic evaluation methods
  4. Implement CSP

CSP makes it possible for web app developers to specify the domains that the browser should consider to be valid sources of executable scripts. A CSP compatible browser will then only execute scripts loaded in source files received from those whitelisted domains, ignoring all other script (including inline scripts and event-handling HTML attributes).

e.g. The following header instructs the CSP compatible browser to only allow scripts from site’s own origin. This excludes the sub-domains.

Content-Security-Policy: default-src ‘self’

Event after implementing all the above mentioned security measures, your web app is still not 100% secure. That’s when I say: Security is a myth.

Despite that, I recommend following ALL the above mentioned precautions to protect your web app. That’s because something is better than nothing and this is certainly more than something ;). If nothing else, at least this will discourage attackers from choosing you as a target.

Finally, a sincere appeal to all:

Implement all these security measures together. Don’t pick n choose.

Posted in General | Leave a comment

Ionic 3 : A Step Forward

Content posted here with the permission of the author, who is currently employed at Josh Software. Original post available here.

Most of the people have already switched from Ionic 1 to Ionic 2. You are maybe one of them and feel more comfortable and settled with Ionic 2. But now the latest version of Ionic i.e Ionic 3 has been released. The change from Ionic 2 to Ionic 3 is nothing as compared to that of Ionic 1 to Ionic 2.

Ionic 3 release is not the framework update. It’s just a major update in Ionic 2. It supports angular 4 and intern follows semantic versioning system of angular( The 3 dots system.).That is MAJOR.MINOR.PATCH. The major update implies that there will be some breaking changes and in case of minor/ patch level update, some API will be changed.

Ionic has come up with many features and improvements over Ionic 2. Basic 3 features I implemented in my app and it helped me a lot. My app is now working much faster. In this blog I am sharing details of these features and steps to implement them:

1. Ionic native plugin support:

The plugin support system has been changed in Ionic native 3.x. These changes are:

  • A native package needs to be installed for every plugin that is added to a project
  • The application root module requires that the native package for each installed plugin is imported and added to the provider’s array

Easy steps to add a new plugin to Ionic 3 project:

1. Install Cordova plugin with the help of ionic CLI command:

ionic cordova plugin add &lt;plugin-name&gt;

2. Install ionic native package for the installed plugin by npm:

npm install --save @ionic-native/&lt;plugin-name&gt;

Note that, we need to just add plugin name after @ionic-native/ path. This command will make entry of same package in package.json.

3. Import installed native package to project.

For that, we need to import it into app.module.ts Open root module of project – /src/app/app.module.ts. Add import statement like, (exmaple: camera plugin)

 import { Camera } from '@ionic-native/camera'; 

Now add entry in providers,

  @NgModule({
  ...
  providers: [
    Camera
  ]
  ...
  })

4. Now wherever you required this plugin, just import it. Fairly simple, right?

Ionic native 3 providing us a Cordova wrapper for more than 130 plugins. Instead of using plugin directly, ionic allows us to import respective plugin wrapper in root module once and use it whenever and wherever we want. This reduces app bundle size and make ionic app run faster

2. Lazy loading:

Loading something as and when required. For example, In a web page if we have 50 images, then for better loading time we would not prefer to load all images at once. As user scroll down, images should load. The Same concept is used in Ionic 3.

Instead of loading everything in a front, it will load the component when needed. This would help in starting ionic app. In big applications, when lots of things need to be loaded in the start, this feature will be helpful.

To implement lazy loading we need to follow some steps. Lets see how we can implement lazy loading in Ionic 3

1. Remove all import statement of a page

Lets take example for HomePage. We need to remove all references of HomePage from all other pages, providers, and modules. Start with src/app/app.module.ts. In this file we will have two arrays: entryComponents and declarations. All the components that we want to load in app, we specify them here. Now we want to load HomePage only when it is requested.

So, first of all remove all the reference of HomePage from app.module.ts includinng import statement.

@NgModule({
  declarations: [
    MyApp,
    HomePage
  ],
  ...
  entryComponents: [
    MyApp,
    HomePage
  ],
  ...
})

2. Create ngModule for HomePage

Create new file src/pages/home/home.module.ts. Add declaraion, import and export statement for HomePage.

import { NgModule } from '@angular/core';
import { IonicPageModule } from 'ionic-angular';
import { HomePage } from './home';

@NgModule({
declarations: [
    HomePage,
],
imports: [
    IonicPageModule.forChild(HomePage),
],
exports: [
    HomePage
]
})
export class HomePageModule {}

3. Add decorator in HomePage

Import IonicPage from ionic-angular and add @IonicPage() decorator to home.ts page.

import { Component } from '@angular/core';
import { NavController } from 'ionic-angular';
import { IonicPage } from 'ionic-angular';

@IonicPage()
@Component({
  selector: 'page-home',
  templateUrl: 'home.html'
})
export class HomePage {
  constructor(public navCtrl: NavController) {}
}

4. Remove all references of HomePage

Remove all the import statements to HomePage from all pages, provides, root files. Now its time to use HomePage module. We can use it by simply specifying HomePage in a string. For example, If you are using it like this:

rootPage:any = HomePage

Now, it should be replaced by,

rootPage:any = 'HomePage'

These are the simple steps to make our app run faster by implementing lazy loading. We just need to create separate ngModule for each component and decorate it with IonicPage.

3. Plugin mocking and browser development:

Till Ionic 2 we were not able to test any plugin in the browser of the ionic app. Ionic 3 provides full support for plugin mocking. This means that we can easily use and test more than 130 plugins in a browser. This makes ionic developer more comfortable to build entire app in browser without any help of device or emulator.

To use this feature we need to create mocks for our plugin to use and make them return proper data.

Native plugin mock is just a copy of actual plugin. It allows user to test the plugin functionality without using actual plugin in browser itself. Lets take exmaple of camera plugin. We will implement getPictures() method and return test data. Lets write camera plugin mock.

1. Create mock folder in root folder

mkdir src/mocks
cd mocks

2. Create camera-mock.ts and overwrite getPictures() method like following:

export class CameraMock {
  getPicture(params) {
    return new Promise((resolve, reject) =&gt; {
      resolve("BASE_64_IMAGE_DATA");
    });
  }
}

3. Import camera-mock.ts in app.module.ts and add it to providers array too

import { CameraMock } from "../mocks/camera-mock";
@NgModule({
...
providers: [
...
CameraMock
]
...
})

4. Now plugin mock is ready to use in any component. Just import it and use it.

I recommend you to use Ionic 3 native plugin support as well as load the components as and when required i.e. lazy loading, Surely you will get performance refinement. Also, there is proper documentation available for steps to migrate from Ionic 2 to Ionic 3. You can refer this.

Posted in General | Leave a comment

Record and export Audio files in browser using Recorder.js and upload it on cloud.

Content posted here with the permission of the author, who is currently employed at Josh Software. Original post available here.

In this article, Let’s see how audio recording is possible without any shim or plugin, but only by using the Recorder.js library in the browser and how to upload recorded file on s3 (Or any other cloud storage). Before going to do audio recording first we will see which technology we need and why.

There are resources available for handing these individually but not together. I used references from below links. It is advisable to read below links before proceeding.

Technology Used:

How does the recording process works?

To record audio in the browser, you will need obviously access to the microphone with the getUserMedia API and the AudioContext API, so be sure that your browser supports these APIs

  • Web Audio API

The Web Audio API is a JavaScript API for processing and synthesizing audio in web applications.The API supports loading audio file data in multiple formats, such as WAV, MP3, AAC, OGG and others. Browser’s support for different audio format varies(Check Here).

  • AudioContext API

The AudioContext is used for creating, processing and decoding audio. You need to create an instance of AudioContext class before you can do anything because all are processed / decoded, controlled for audio generation by this API.
How AudioContext declare?:

window.AudioContext = window.AudioContext or window.webkitAudioContext
  • getUserMedia API

Before going to do recording we should know whether our web application can access user’s camera or microphone. So getUserMedia is an API that gives a web page access to a user’s camera and microphone via JavaScript. getUserMedia() has been supported since Chrome 21, Opera 18, and Firefox 17.

Make sure your application is secured with HTTPS protocol to use “getUserMedia API”. It does not work on insecure origins. Starting with Chrome 47, getUserMedia() requests are only allowed from secure origins: HTTPS or localhost.
How to check getUserMedia

if navigator.getUserMedia
  # do something cool
else
  # fallback code
  • Recorderjs

Recorderjs is a JavaScript library that records audio input (Web Audio API AudioNode object) and encodes to audio file image (Blob object). It supports three encoding formats. By using Recorder.js, the recording process is very simple. All that you need to do is to request the microphone in browser, once the user allows the access to the microphone, a stream object is received in the success callback of the initialization.This stream needs to be handled with the createMediaStreamSource method from an instance of the AudioContext API.All the variables should be stored globally or at least accessible in the scope for the rest of functions.

To understand above explanation, added code here:

startUserMedia = (stream) ->
  input = window.audio_context.createMediaStreamSource(stream)
  console.log 'Media stream created.'
  window.recorder = new Recorder(input, 'audio/mp3')
  window.recorder.record()
  console.log 'Recorder initialised.'
  return

The Recorder.js class expects the handled stream as first argument namely the input. With this instance of Recorder.js you can trigger the record method that starts to recording the received Audio and it will run indefinitely until the stop method from the same instance of recorder is triggered.

Export Recorded Audio

Export the Audio Blob from the recorder audio using the exportWAV method of the recorder instance. The exportWAV method can export the audio in wav and mp3 format when the second parameter is specified with the correct mimetype (‘audio/wav’, ‘audio/mp3’). Don’t forget to use the clear method of the recorder to start with a new one later. With the generated blob you will be able to upload it as an audio file to the server, in the same browser or just to play it in the same document.
Added Code here:

createDownloadLink = ->
  recorder and recorder.exportWAV((blob) ->
    url = URL.createObjectURL(blob)
    li = document.createElement('li')
    au = document.createElement('audio')
    hf = document.createElement('a')

    au.controls = true
    au.src = url
    hf.href = url
    hf.download = 'audio.mp3'
    hf.innerHTML = hf.download
    li.appendChild au
    li.appendChild hf
    playRecord.appendChild li

How to record Audio with Recorder.js

To make the explanation as simple as possible, we’ll just write a haml document that implements all the previously described process. It consists of 2 buttons namely Start and Play that triggers the recording process with Recorder.js. When the user stops the recording, it will append an Audio Track to an UL list with a downloadable audio file with like “audio.mp3”:

HAML code:
Create 2 buttons as “Start Recording” and “Stop Recording”

!!!
%html
  %head
    %meta{:content =&gt; "text/html; charset=utf-8", "http-equiv" =&gt; "Content-Type"}
    %title Live input record and playback
    :css
      ul {
          list-style: none;
      }

      #playRecord audio {
          display: block;
          margin-bottom: 10px;
      }
  %body
    %h1 Recorder.js export example
    %button#start-btn Start recording
    %button#stop-btn{:disabled =&gt; "disabled"} Stop recording
    %h2 Stored Recordings
    %ul#playRecord

Click on Start button will start recording and clicking on Stop button will stop recording and export it.

$('#start-btn').on 'click', -&gt;
  if confirm "Once you start recording, you can't pause it."
    initAudioRecording()

$('#stop-btn').on 'click', -&gt;
  stopRecording(this)

stopRecording = (button) -&gt;
  if window.recorder != undefined
    window.recorder  window.recorder.stop()
    createRecording()
    return

createRecording = -&gt;
  recorder and recorder.exportWAV((blob)
    data = new FormData();
    data.append("audio", blob);
    console.log(data)
    $.ajax
      url: 'Your Controller path'
      type: 'POST'
      data: data
      contentType: false
      processData: false
      success: -&gt;
        console.log "Successfully uploaded recording."
  )
  return

startUserMedia = (stream) -&gt;
  input = window.audio_context.createMediaStreamSource(stream)
  # console.log 'Media stream created.'
  window.recorder = new Recorder(input, 'audio/mp3')
  window.recorder.record()
  # console.log 'Recorder initialised.'
  return

initAudioRecording = -&gt;
  try
    window.AudioContext = window.AudioContext or window.webkitAudioContext
    navigator.getUserMedia = navigator.getUserMedia or navigator.webkitGetUserMedia or navigator.mozGetUserMedia
    window.URL = window.URL or window.webkitURL
    window.audio_context = new AudioContext
    console.log 'navigator.getUserMedia ' + (if navigator.getUserMedia then 'available.' else 'not present!')
  catch e
    alert 'No web audio support in this browser!'
  navigator.getUserMedia { audio: true }, startUserMedia, (e) -&gt;
    console.log 'No live audio input: ' + e
    return
  return

Browser Support

  • Make sure you are using a recent version of browser.
  • Make sure your application is secured with HTTPS protocol to use “Web Audio API”. It does not work on insecure origins. As of Chrome 47, the getUserMedia API cannot be called from insecure origins.
  • There is no issue while implementing Web Audio API in development environment i.e. http://localhost:3000
  • You can test your browser compatibility test.Here

How to upload recorded file on s3

Till now we have seen how to get audio blob object, but the major challenge is how to upload this audio blob on s3(or on cloud).
See below coffee script code:

createRecording = ->
  recorder and recorder.exportWAV((blob) ->
    data = new FormData();
    data.append("audio", blob);
    $.ajax
      url: 'Your controller path to upload recording'
      type: 'POST'
      data: data
      contentType: false
      processData: false
      success: ->
        if confirm "Successfully uploaded recording."
          window.location.href = 'Redirect path after successful ajax call.'

To make it very simple to understand I will explain you above code:
I have created a method as “createRecording” which calls exportWAV() method from Recorder.js, so we get a blob object.
Our aim is to upload this blob object on s3, for that I have created FormData() object and append audio blob to FormData() object.
Send this FormData() object to server by ajax call with “POST” method.

NOTE:
As we may be sending large amount of data to server (in terms of MB), we should have proper nginx entry for client_max_body_size.We should change nginx configuration in file /etc/nginx/nginx.conf, to expected max size of our recorded file.

client_max_body_size 50M;

Let’s see server side implementation:

def your_controller_method
    @model_object.audio = params[:audio]
    @model_object.save
    render nothing: true
end

Once file object reaches to your server upload same to S3 (or any other cloud storage) using regular file upload method.

For uploading file we have used carrierwave gem(click here)

Posted in General | Leave a comment