Using Rack::Proxy to serve multiple React Apps on the same domain.

Content posted here with the permission of the author Rahul Ojha, who is currently employed at Josh Software. Original post available here

Background

We had 4 React apps running on the local server. All apps were running on different ports because we can not start multiple React Apps on the same port. We had to build a single sign-on server. Out of 4 react apps one will be responsible for managing authentication and based on roles or department redirecting to other apps. The Auth app will get token from Auth API and will store it to local storage, and other apps will be using this token for authentication while calling APIs.

Problem Faced:

Now while calling API from any React app we need to send the token with it. Since the token is stored in local storage by Auth App which is using a different port so the local storage will not be accessible for other react apps.

You may think “Why didn’t you go with Cookies”, Yes cookies are shareable among HTTP ports but its size is very less which is 4 KB. So if you have to store permissions and anything else which is shareable and exceeding to 4 KB the cookies will not be useful.

Solution:

We will be using a single domain and will access different React apps using namespace in URL.

Example:

localhost:9292/app1 => app1 => localhost:3001
localhost:9292/app2 => app2 => localhost:3002
localhost:9292/app3 => app3 => localhost:3003
localhost:9292/app4 => app4 => localhost:3004

Here we need to use a reverse proxy server to do this. We will use Ruby Rack to create a proxy server. Why not use Nginx here? You can but there is a problem, React does not add namespace in the assets path so assets will not be loaded and you will see a blank white page always, and there is no way in development to add the namespace in asset path without ejecting CRA. If you have specified any value to homepage key in package.json file, then it will be automatically prefixed with assets path while creating a build in production mode.

So if you are Ruby developer then you are going to enjoy rest of the blog. But if you are not a Ruby developer there is no issue because Ruby is nice and easily understandable. You just need to follow the steps.

Step 1:

Install ruby
You can use below link to install RVM and ruby. RVM is Ruby version manager.https://rvm.io/rvm/install

Step 2:

gem install rack

Step 3:

gem install rack-proxy

Step 4:

Create a file ./proxy_server.rb and add below code.

require 'rack-proxy'
class ProxyServer < Rack::Proxy
 def rewrite_env(env)
  request = Rack::Request.new(env)  
  if request.path.match('/app1')
    env["HTTP_HOST"] = "localhost:3001"
    @port = 3001
  elsif request.path.match('/app2')
    env["HTTP_HOST"] = "localhost:3002"
    @port = 3002
  elsif request.path.match('/app3')
    env["HTTP_HOST"] = "localhost:3003"
    @port = 3003
  elsif request.path.match('/app4')
    env["HTTP_HOST"] = "localhost:3004"
    @port = 3004
  # The below line is important to load assets.
  # So it detects app name from the first request to the app
  # which is stored in an instance variable and redirects
  # to that app for assets.
  elsif request.path.match('/static') || request.path.match('/assets')
    env['HTTP_HOST'] = "localhost:#{@port}"
  else
    env["HTTP_HOST"] = "localhost:3001"
    @port = 3001
  end
  env
 end
end

Step 5:

Add a file ./config.ru and add below code

require_relative './proxy_server'
run ProxyServer.new

Step 6:

Run below command on the command line where you have stored above files. This command is responsible for starting your rack server.

rackup

Now your Rack server has started and ready do proxy pass. You just need to go at localhost:9292 and your default app which is app1 as per proxy_server file will be loaded to the browser. Port 9292 is default port defined for Rack server.

Conclusion:

By using Rack server you can run multiple react apps on the same domain in development environment with local storage and cookies shareable among all applications. This solution is not restricted to only React apps, it can be used for any other client-side frameworks too.

Posted in General | Tagged , , , , | Leave a comment

How to make image loading lightning fast on web and mobile App

Content posted here with the permission of the author Shivam Kumar Singh, who is currently employed at Josh Software. Original post available here

Since internet is born, we have come a long way in improving web page loading time. Faster it gets, better it is. Most time consuming part of loading web & mobile app is loading images. Therefore faster image load is major challenge today.

One of application we worked recently has lots of images to render. As number of images increased, loading time of application increased as well.

So to tackle this problem we decided to compress images. We were using ImageMagick with carrierwave for all the image processing and remote file upload. So we decided to compress the image using ImageMagick libraries to reduce the file size but it compresses the image at the cost of image quality. We can not compromise on image quality therefore we were looking for alternatives.

We came across webp format developed by Google — which uses both lossy and lossless compression. WebP lossless images are 26% smaller in size compared to PNGs.

Can you figure out size of below images by looking at them ?

https://www.dropbox.com/s/aes6q2vldawol2u/seeds-4306035.webp?dl=0 (Image 2)

Image 1 is of type jpg and Image 2 is of type webp.

Quality wise they look very similar, right ? But size wise Image 1 is 4 times larger than Image 2

Image 1 size is 2.8 MB while Image 2 size is 551 KB.

How to configure it for Rails Application

Considering we already have ImageMagick installed on our system.

We have to install libwebp, a library used to add WebP encoding and decoding to your programs. Before that

Download dependency for libwebp.

sudo apt-get install libjpeg-dev libpng-dev libtiff-dev libgif-dev

Download latest libwebp and extract it.

wget http://downloads.webmproject.org/releases/webp/libwebp-0.4.3.tar.gz
tar xf libwebp-0.4.3.tar.gz

Run the following commands to install libwebp.

cd libwebp-0.4.3/
 ./configure
make
sudo make install

Run following command on the console to confirm successful installation.

2.0.0p0 :002 > WebP.decoder_version  => "0.4.3" 
2.0.0p0 :003 > WebP.encoder_version  => "0.4.3"

Now we need to add carrierwave-webpgem which is Ruby wrapper for libwebp.

gem 'carrierwave-webp'

Now that we have all of our prerequisites out of the way, we are now ready to convert our image to WebP.

Customize Carrierwave uploader to generate images in webp format.

class MediaUploader < CarrierWave::Uploader::Base
  include CarrierWave::MiniMagick
  include CarrierWave::WebP::Converter   
   version :jpeg do     
     process resize_to_fit: [720, 450]     
     process convert: :jpeg   
   end  
 version :webp do     
   process :convert_to_webp   
  end 
end
 

So this way we have generated two different versions of images, one in jpg with given resolution (for fallback) and other in webp format.

Format Specific remote image URL

def return_url(poster, format)
  if poster.image.send(format).exists?
    url = poster.image.send(format).url
  else
    poster.image.jpeg.url //JPEG as fallback option.
  end
end

There are few limitations as well for webp

  1. Not supported in iOS and Safari client.
  2. More effective for substantially large files

Conclusion:

With Webp format, we decreased our images size by 4 times thereby increased page load speed. If you have any questions let us know in comments.

Posted in General | Tagged , , , , | Leave a comment

How is data secure over https?

https

STEP1: Client HELLO

STEP2: Server HELLO

STEP3: Server authentication

STEP4: Secret key exchange

STEP5: Client HELLO finished

STEP6: Server HELLO finished

STEP7: Data exchange

Thanks for reading and hope you learned something new. Feel free to comment if you have any suggestions or corrections.

Posted in General | Leave a comment

Rails Sidekiq configuration for micro services on reverse proxy.

Content posted here with the permission of the author Rahul Ojha, who is currently employed at Josh Software. Original post available here.

This blog is intended to describe how to configure multiple sidekiq webs for API only apps using reverse proxy server.

Background

We have 4 API services running on ECS. We used a single domain for all the services and segregated them with the namespace in routing. For namespace routing and pointing to Docker containers, we have used Traefik. We can do the same thing using Nginx and some other reverse proxy too.

Example:

www.domain.com/rails-api-1/REST_ROUTES
www.domain.com/rails-api-2/REST_ROUTES
www.domain.com/rails-api-3/REST_ROUTES
www.domain.com/rails-api-4/REST_ROUTES

Problem Faced

Now the problem was when we try to access sidekiq webs with these namespace routes. It does not load rails assets because namespace path ‘rails-api-1 ’ is not considered in the root path, and sidekiq tries to find assets in the path `www.domain.com/sidekiq/` and it doesn’t exist actually.

Solution

Initially, we mounted sidekiq path in config/routes.rb like this.

mount Sidekiq::Web => '/sidekiq'

Later, we changed it to

mount Sidekiq::Web => '/rails-api-1-sidekiq'

Configuration for Traefik:

You need to add a new rule in docker labels

Key => traefik.sidekiq.frontend.rule 
Value => Host:www.domain.com;PathPrefix:/rails-ap1-1-sidekiq

Configuration for Nginx:

If you are using Nginx as a reverse proxy then you need to add these configuration for each service.

location /rails-ap1-1-sidekiq/ {
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header Host $http_host;
    proxy_redirect off;
    proxy_pass http://your_domain/rails-ap1-1-sidekiq/;
    access_log /var/log/nginx/access.log;
    error_log /var/log/nginx/error.log;
  }location /rails-ap1-2-sidekiq/ {
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header Host $http_host;
    proxy_redirect off;
    proxy_pass http://your_domain/rails-ap1-2-sidekiq/;
    access_log /var/log/nginx/access.log;
    error_log /var/log/nginx/error.log;
  }

Now you can access your sidekiq web at

https://www.domain.com/rails-api-1-sidekiq/retries

After that Sidekiq will look for assets in path /rails-api-1-sidekiq/. Here as per docker label rails-api-1-sidekiq indicates that it will point to the container rails-api-1 because it is configured forrails-api-1task and within that container, it will look for the assets in /rails-api-1-sidekiq/.path, You can see in below image

Similarly, you can setup multiple sidekiq on your reverse proxy.

Thank you!

Posted in General, Ruby on Rails | Leave a comment

Building Decentralised Link Shortner on Ethereum blockchain using Truffle

Original post available here.

Blockchain is emerging technology, it needs no introduction. By any chance if you are left behind and don’t know about blockchain then I recommend reading about blockchain first before reading this article. You can read about blockchain here or here or search on internet and you will find plenty of article to read from.

What is Ethereum ?

Launched in 2015, Ethereum is the world’s leading programmable blockchain. It is a global, open-source platform for decentralized applications. These decentralized applications (or “dapps”) gain the benefits of cryptocurrency and blockchain technology. Read more about ethereum here.

What is Truffle ?

Truffle is a framework for blockchain development, it streamlined smart contractcreationcompilationtesting, and deployment onto Ethereum. Read more about truffle here

What is Decentralised App ?

Decentralised application does not require centralised server to work (hence no maintenance cost). It interacts with smart contract deployed on blockchain network.

What is Smart Contract ?

Smart contracts are programs which govern the behaviour of accounts within the Ethereum state. We will write smart contract in Solidity language, Solidity is an object-oriented, high-level language for implementing smart contracts. Read more about solidity from here.

Getting Started:

A) Install npm, node & Truffle

Follow https://docs.npmjs.com/downloading-and-installing-node-js-and-npm for installing npn & node.

Then install truffle

npm install -g truffle

check if truffle installed successfully or not

$ truffle version
Truffle v5.0.21 (core: 5.0.21)
Solidity v0.5.0 (solc-js)
Node v11.0.0
Web3.js v1.0.0-beta.37

B) Create Project

Create new folder for project & initialise with truffle. We will use React Truflle box

$ mkdir link_shortner
$ cd link_shortner/
$ truffle unbox react
✔ Preparing to download
✔ Downloading
✔ Cleaning up temporary files
✔ Setting up box
Unbox successful. Sweet!
Commands:
Compile:        truffle compile
  Migrate:        truffle migrate
  Test contracts: truffle test

If you are new to Truffle then read about created directory from https://www.trufflesuite.com/docs/truffle/getting-started/creating-a-project

C) Install Ganache for blockchain setup on local machine https://www.trufflesuite.com/docs/ganache/overview


Link Shortner Smart Contract

Create LinkShortner.sol file inside contracts/ folder and write following content in it.

pragma solidity ^0.5.0;

contract LinkShortner {
  event LinkAdded(uint linkId, string url);
  uint lastLinkId;
struct LinkTemplate {
  address userAddress;
  string url;
 }

mapping (uint => LinkTemplate) public linkMapping;
constructor() public {
  lastLinkId = 0;
 }

function createNewLink(string memory url) public returns (uint, string memory) {
   lastLinkId++;
  linkMapping[lastLinkId] = LinkTemplate(msg.sender, url);
    emit LinkAdded(lastLinkId, url);
  return(lastLinkId, url);
 }

function getLink(uint linkId) public view returns(address, string memory) {
  LinkTemplate memory link = linkMapping[linkId];
  return(link.userAddress, link.url);
 }

function getLastLink() public view returns(address, string memory, uint) {
  LinkTemplate memory link = linkMapping[lastLinkId];
  return(link.userAddress, link.url, lastLinkId);
 }
}


Now deploy this contract on local blockchain network:


$ truffle compile
$ truffle migrate
Ganache Screenshot after contract deployment

React Application for interaction with Smart Contract

Open client/src/App.js file & Replace

import SimpleStorageContract from"./contracts/SimpleStorage.json";

with

import SimpleStorageContract from "./contracts/LinkShortner.json";

Creating new link

contract.methods.createNewLink(this.state.url).send({ from: accounts[0] })

Install Metamask chrome extension

and run React app

cd client
npm run start

Deploying contract on Ropsten test network

- Register new account on infura.io
- Create new project
- Get project api and connection link:
ROPSTEN_URL=https://ropsten.infura.io/v3/<your-api-key>

Goto Truffle project, install truffle-hdwallet-provider

npm install truffle-hdwallet-provider — save

Create `.env` file, put MNEMONIC and <network>_URL to file

MNEMONIC=wallet mnemonic 12 words
ROPSTEN_URL=https://ropsten.infura.io/v3/<your-api-key>

Update truffle-config with following content

const path = require("path");
require('dotenv').config()
const HDWalletProvider = require('truffle-hdwallet-provider')
const MNEMONIC = process.env.MNEMONIC
const ROPSTEN_URL = process.env.ROPSTEN_URLmodule.exports = {
  // See <http://truffleframework.com/docs/advanced/configuration>
  // to customize your Truffle configuration!
  contracts_build_directory: path.join(__dirname, "client/src/contracts"),
  networks: {
    ropsten: {
      provider: function() {
        return new HDWalletProvider(MNEMONIC, ROPSTEN_URL);
      },
      network_id: '3',
    },
    development: {
      host: "127.0.0.1",
      port: 7545,
      network_id: "*",
   },
   test: {
     host: "127.0.0.1",
     port: 7545,
     network_id: "*",
  }
 }
};

Run following command to deploy

truffle migrate --network ropsten

Sinatra API for reading Short Link on ethereum network

Create folder backend
Add following content in backend/app.rb

# Require the bundler gem and then call Bundler.require to load in all gems
# listed in Gemfile.
require 'bundler'
Bundler.require

require 'sinatra'

require 'ethereum'before do
  content_type 'application/json'
end

class Contract
  def initialize
    @client = Ethereum::HttpClient.new("https://ropsten.infura.io/v3/<API-KEY>")
    contract_json = JSON.parse(File.read('LinkShortner.json'))
    @contract_abi = contract_json['abi']
    @address = contract_json["networks"]["3"]["address"]
    @client.default_account = "0x3b8B0b23C4850FA8289da815a6abEE4Fc2DF941A"
  end

  def result(id)
    return nil unless id
    contract_instance.call.get_link(id.to_i)[1]
  enddef contract_instance
    Ethereum::Contract.create(name: "LinkShortner", address: @address, abi: @contract_abi,
                              client: @client)
  end
end

class App < Sinatra::Base
  get '/url' do
    response.headers["Access-Control-Allow-Origin"] = "*"
    return {url: Contract.new.result(params[:id])}.to_json
  end
end

Deploy sinatra API on heroku

heroku create
heroku buildpacks:set https://github.com/timanovsky/subdir-heroku-buildpack
heroku buildpacks:add heroku/ruby
heroku config:set PROJECT_PATH=backend
git push heroku master

Now use deployed API for reading short link

fetch("https://<heroku-app-url>/url?id="+id).then((response) => {
  return response.json();
}).then((response) => {
  const url = response.url
  console.log(url)
})

That’s it, now you have your link shortner decentralised app deployed on ethereum network. Generated short link can be shared with anyone, irrespective of browser. For creating short link Metamask plugin is required.

Code is hosted on github

Application is hosted at http://anilmaurya.github.io/link-shortner

Demo of Link Shortner

References:


https://medium.com/@nhancv/deploy-smart-contract-with-truffle-79d2bf218332

https://hackernoon.com/making-a-decentralized-url-shortener-using-ethereum-4fdfccf712a6

Posted in Blockchain, Tutorials | Tagged , , , | Leave a comment

An Efficient way to manage API calls with RxRetrofit

Prologue

Android is becoming popular day by day and thousands of apps are being published on Play Store. To survive in this competition, the User experience matters a lot. The app crash is the worst User experience, in other words we can say 1 billion dollar mistake. So I am going to explain how to use RxRetrofit efficiently to make API calls.

Problem

The most common mistake made by the Android developer is while making an API call and not handling the fact that the user can exit the app or press the back button, before the API call returns. The result is app crash!

In more detail, when an API is hit, that network call is being performed on the background thread and based on response the UI needs to be updated in UI thread. Now, when the background thread is running and the back button is pressed, the current screen (Activity or Fragment) will be popped out from the stack. After this when data is received from the API and an attempt to update the UI will result in a crash as the current screen (Activity or Fragment) will not be in the stack.

Solution

Make API calls using RxRetrofit !!

Example

RetrofitApiClient.createService(ApiInterface::class.java)
   .callLoginApi(“email”,”password”)
   .observeOn(AndroidSchedulers.mainThread()) // On UI thread
   .subscribeOn(Schedulers.io()) // On background thread 
   .subscribe({ response -> // On Success 

         // App will crash on the below line
         editTextOtp.setText(response?.body()?.otp)
         buttonLogin.setText("Verify OTP")

   }, {
       // On Error (Could not reach to the server)
   })

According to the above example, when you hit the Login API and immediately press the back button (Activity or Fragment removed from the stack) then you get the response from the API in the subscribe() method, you try to update the UI and the app will crash!

Use the Disposable interface of the RxJava which is used with RxRetrofit.

See the below example :

public interface Disposable {
   void dispose();  // Dispose the resource or an operation.
   boolean isDisposed(); // Returns true if this resource has been disposed.
}


Let’s implement it in the above example,

1) private var disposableLoginAPI: Disposable? = null // Globally
2) 
disposableLoginAPI = 
RetrofitApiClient.createService(ApiInterface::class.java)
   .callLoginApi(“email”,”password”)
   .observeOn(AndroidSchedulers.mainThread()) // On UI thread
   .subscribeOn(Schedulers.io()) // On background thread 
   .subscribe({ response -> // On Success 

         editTextOtp.setText(response?.body()?.otp)
         buttonLogin.setText("Verify OTP")

   }, {
       // On Error (Could not reach to the server)
   })

The subscribe() returns disposable so initialize the disposable with it.

3) override fun onStop() {
   super.onStop()

     // If it's not disposed then dispose  
    if (disposableLoginAPI?.isDisposed == false) {
         disposableLoginAPI?.dispose() // Magic!
    }  
} 

So once you are about to leave the screen (Activity or Fragment), in the onStop() method, dispose() method will dispose the API call and there will be no response in the subscribe() method, so UI will not be updated and the app will not crash.

For Multiple API calls

You can use CompositeDisposable to add multiple disposable objects:

val listOfDisposables=CompositeDisposable() // Init object

//Adding multiple disposables
listOfDisposables.add(disposableLogin!!)
listOfDisposables.add(disposableForgotPassword!!)

And finally dispose in the onStop() method.

override fun onStop() {
   super.onStop()

   listOfDisposables.dispose()  // Dispose all the disposables 
} 

Happy Coding!

Posted in General | Tagged , , | Leave a comment

AWS EC2: Increase volume size on the fly (Using AWS Management Console)

As time passes the production requirements grow. This happens all the time and recently again it happened and I got an alert from Nagios telling me that disk space is getting full. I immediately checked the server to find out what is consuming space!!

Initially, I thought, perhaps unnecessary files are consuming the space. I can just delete those files and get rid of the problem. But on checking it turned out that everything was important to keep in the EBS (Elastic Block Store) volume and nothing can be deleted.

The only solution was to increase the volume size. Thankfully it is AWS. They have enhanced the console to a great extent and increasing the volume size is a piece of cake now. Here are the simple steps:

1) Check the exact volume size which is initially their with ‘lsblk’.

 nvme0n1     259:0    0   50G  0 disk  
 └─nvme0n1p1 259:1    0   50G  0 part / 

This is the exact size of my volume it means 50 GB is available and it is using the exact 50 GB

To increase the volume size follow the below steps:

2) Login to your aws console and than go to the volumes in your EC2 dashboard and find the volume which you want to increase.

3) Select the volume and then click on the ‘Actions’ -> ’Modify Volume’ .

4) Increase the volume size edit the field and enter the new volume size (In my case i want to increase it to 60 GB).

5) Click on the ‘Modify’ button then you can see it will be reflacted to your aws console within a minute but it will be not in use until ‘reboot’.

6) Login to your server and check with ‘lsblk’ you will get to see output as below.

nvme0n1     259:0    0   60G  0 disk  
└─nvme0n1p1 259:1    0   50G  0 part /

Note: This denotes that disk size is 60 GB and 50 GB is allocated to nvme0n1p1.

7) Now to allocate 60 GB to nvme0n1p1 reboot your instance.

CAUTION: While rebooting the instance it will down your service.

8) After rebooting its done now you can check with ‘df -h’. You will see partition is using 60 GB.

/dev/nvme0n1p1   59G  24G   35G  42% /
Posted in General | Leave a comment