Using SVG to draw graphs easily and fast!

So often we look for the simple way out. For example, to generate graphs, we search for the best possible libraries and make the front-end un-necessarily heavy. Sometimes, we need to think out of the box and can use SVG to create simple graphs.

Anuja talks exactly about how they used SVG to create simple line and bar charts in their light-weight mobile friendly application.

Fun To Learn

Do you want to draw a graph in HTML? Are you a backend developer and not a UI developer? How will you go with this? You will surely search for some available library to draw a graph even if graphs are simple. In our project, we also have simple bar and line graphs for weekly and monthly tracking respectively. Initially our UI team designed bar graphs. As is often the case, after few days there was a requirement of line graphs.

I was worried about timelines. We had an option to ask UI team to give us the static graph and integrate the API. But we would need to explain them the requirement in detail and the design. This process seem too time consuming. So I thought, that instead of wasting time in that, it would be better to work on this and learn something new. So I decided to work on this new…

View original post 601 more words

Posted in General | Leave a comment

Searching on steroids

ElasticSearch is one of the popular Search Indexing engines today. Yogesh takes us on an interesting tutorial for integrating ElasticSearch into a Rails app – right from the initial thought-process till execution and performance.

Happy Coding !!!

There are plenty of indexing/search servers available out there like Solr, Sphinx, ElasticsearchGoogle Search Appliance and many more.

But out of all the above, Elasticsearch is gaining more attention to it because of it’s popularity. This post is going to tell you only about it and it’s integration in Rails.

So, how would someone describe what an Elasticsearch is ?

From God’s perspective, it’s an open source, distributed, RESTful, search engine i.e. it has a very advanced distributed model, speaks JSON natively, and exposes many advanced search features, all seamlessly expressed through JSON DSL and REST API’s.

♦ Inception

Standard way of using ES in any application is to use it as a secondary data store i.e. data will be stored in some kind of SQL/NoSQL database and then continuously upserting required documents from it into ES, pretty neat.

Some of us might think why not use database itself…

View original post 1,116 more words

Posted in General | Leave a comment

Houston, we got attacked

Yogesh talks about configuring and securing your Redis server to avoid attacks. Voice of experience rules!

Happy Coding !!!

Houston, you there ?? We’ve had a problem here.

One of our EC2 instance which has Redis server on it, got hacked out of nowhere.
Just before we knew it is hacked, we were screwing up with Redis configuration and thinking what could go wrong with it.

Houston – Roger that, give us more details.

Well, we’ve recently shifted our Redis server to new EC2 instance. The reason we had to do that was because, our Sidekiq processing got much bigger and we couldn’t afford it alongside the Nginx + Passenger. So we took a call to separate it out.

But while configuring Redis, we think we made some mistake😦.

Houston – What is that ?

Basically, we wanted the Redis to listen to all of our Passenger instances . Though by default, it listens on localhost because it’s  bind directive is set to  127.0.0.1 , it’s possible to listen on multiple interfaces by providing multiple IP…

View original post 355 more words

Posted in General | Leave a comment

Rails command line options I hardly ever use.

A good rake refresher!

Learning shall never stop..!!!

This is the blog post about few things in Rails command line that I hardly ever use.

rake notes

Left a comment like #FIXME or #TODO on a method, but never actually end up changing anything or fixing anything about the method. If this sounds familiar, then rake notes helps you avoid just that. It will find all the files with comments beginning with OPTIMIZE, FIXME, OPTIMIZE.

Screen Shot 2016-05-29 at 12.49.47 pm

Also, we can add search for specific annotation, for ex: searching for just FIXME comments would be as simple as firing rake notes:fixme.

Also it lets us search for any custom annotation that we might be using in our code, rake notes:custom ANNOTATION=ADDTESTCASE

Read more about rake notes on the guides.

rails console –sandbox

If you wish to test some code, but you also want the data to be reverted back to original after you have tested the…

View original post 442 more words

Posted in General | Leave a comment

Using Golang with Api.ai Artificial Intelligence service

Part 2 — In this article, we will try out their /tts service.

In part 1 of this series we explored their /query service. Here, we will explore their/tts service which takes text and returns the generated speech (as an audio WAV file).

I have created a folder “apiai” which will hold the Go source code “apiaifile.go”.

C:\go_projects\go\src\github.com\SatishTalim\apiai

We shall be running our program at the command prompt in the folder “apiai” as follows:

go run apiaifile.go

apiaifile.go

Here’s the complete source code:

package main

import (
        "fmt"
        "io"
        "log"
        "net/http"
        "os"
)

func main() {
        url := fmt.Sprintf("https://api.api.ai/v1/tts?v=20150910&text=The+Go+programming+language+rocks!")

        // Create file to store the .wav file
        output, err := os.Create("d.wav")
        if err != nil {
                log.Fatal("Create: ", err)
                return
        }
        defer output.Close()

        // Build the request
        req, err := http.NewRequest("GET", url, nil)
        if err != nil {
                log.Fatal("NewRequest: ", err)
                return
        }        

        // Replace Your_Client_Token with your actual token
        req.Header.Add("Authorization", "Bearer Your_Client_Token")
        req.Header.Add("Accept-language", "en-US" )

        // For control over HTTP client headers,
        // redirect policy, and other settings,
        // create a Client
        // A Client is an HTTP client
        client := &http.Client{}

        // Send the request via a client
        // Do sends an HTTP request and
        // returns an HTTP response
        resp, err := client.Do(req)
        if err != nil {
                log.Fatal("Do: ", err)
                return
        }
        
        // Callers should close resp.Body
        // when done reading from it
        // Defer the closing of the body
        defer resp.Body.Close()       
        
        if _, err := io.Copy(output, resp.Body); err != nil {
                log.Fatal("Copy: ", err)
                return
        }

        fmt.Println("File d.wav downloaded.")
}

https://api.api.ai/v1/tts?v=20150910&text=The+Go+programming+language+rocks!

The above URL processes the phrase “The Go programming language rocks!” and generates an audio file from it.

func Create(name string) (*File, error)

“Create” creates the named file, truncating it if it already exists.

func NewRequest(method, urlStr string, body io.Reader) (*Request, error)

“NewRequest” returns a new “Request” given a method, URL, and an optional body. “NewRequest” returns a “Request” suitable for use with “Client.Do”.

func (h Header) Add(key, value string)

“Add” adds the key, value pair to the header. It appends to any existing values associated with key.

// Replace Your_Client_Token by your actual Client access token 
req.Header.Add(“Authorization”, “Bearer Your_Client_Token”)
req.Header.Add("Accept-language", "en-US" )

A single header “Accept-language” is used to identify TTS language. For each API request, include the above HTTP headers.

client := &http.Client{}

A “Client” is an HTTP client.

resp, err := client.Do(req)

“Do” sends an HTTP request and returns an HTTP response. When “err” is nil, “resp” always contains a non-nil “resp.Body”.

defer resp.Body.Close()

Callers should close “resp.Body” when done reading from it. Use “Defer” for closing the body. “resp.Body” is of type “io.Reader”.

func Copy(dst Writer, src Reader) (written int64, err error)

“Copy” copies from src to dst until either EOF is reached on src or an error occurs. It returns the number of bytes copied and the first error encountered while copying, if any.

Here’s the file “d.wav” that got downloaded using our program.

That’s it!

Posted in Go, Tutorials | Tagged , , | Leave a comment

Using Golang with Api.ai Artificial Intelligence service

Part 1 — In this article, we will try out their /query service.

Api.ai is a service which provides a nice combination of both voice recognition and machine learning for developers. We shall use their free tier.

Api.ai has “Domains”. Domains are a whole collection of knowledge and data structures from Api.ai that are ready for use in every Api.ai agent (apps are called “agents” in Api.ai).

Api.ai has SDKs for most languages except Golang. Nevertheless, we shall use Go to write a program called “apiai.go” which will query the weather for a particular place and display the result.

I have created a folder “apiai” which will hold the Go source code “apiai.go”.

C:\go_projects\go\src\github.com\SatishTalim\apiai

We shall be running our program at the command prompt in the folder “apiai” as follows:

go run apiai.go

apiai.go

As mentioned in their docs, a sample query response is:

{
  "id": "cfcbd337-6b66-4393-a6a3-74fc5487cedb",
  "timestamp": "2016-02-16T00:30:13.529Z",
  "result": {
    "source": "agent",
    "resolvedQuery": "hi my name is Sam",
    "action": "greetings",
    "actionIncomplete": false,
    "parameters": {
      "name": "Sam"
    },
    "contexts": [
      {
        "name": "user_name",
        "parameters": {
          "name": "Sam"
        },
        "lifespan": 5
      },
      {
        "name": "greetings",
        "parameters": {
          "name": "Sam"
        },
        "lifespan": 5
      }
    ],
    "metadata": {
      "intentId": "c251ef97-0c43-404d-bf75-98e806f942be",
      "intentName": "Greetings"
    },
    "fulfillment": {
      "speech": "Hi Sam! How can I help you?"
    }
  },
  "status": {
    "code": 200,
    "errorType": "success"
  }
}

JSON-to-Go is an excellent tool that instantly converts JSON into a Go type definition. Using it, I get:

type QResponse struct {
        ID string `json:"id"`
        Timestamp time.Time `json:"timestamp"`
        Result struct {
                Source string `json:"source"`
                ResolvedQuery string `json:"resolvedQuery"`
                Action string `json:"action"`
                ActionIncomplete bool `json:"actionIncomplete"`
                Parameters struct {
                        Name string `json:"name"`
                } `json:"parameters"`
                Contexts []struct {
                        Name string `json:"name"`
                        Parameters struct {
                                Name string `json:"name"`
                        } `json:"parameters"`
                        Lifespan int `json:"lifespan"`
                } `json:"contexts"`
                Metadata struct {
                        IntentID string `json:"intentId"`
                        IntentName string `json:"intentName"`
                } `json:"metadata"`
                Fulfillment struct {
                        Speech string `json:"speech"`
                } `json:"fulfillment"`
        } `json:"result"`
        Status struct {
                Code int `json:"code"`
                ErrorType string `json:"errorType"`
        } `json:"status"`
}

Here’s the complete “apiai.go” program:

package main

import (
        "encoding/json"
        "fmt"
        "log"
        "net/http"
        "time"
)

type QResponse struct {
        ID string `json:"id"`
        Timestamp time.Time `json:"timestamp"`
        Result struct {
                Source string `json:"source"`
                ResolvedQuery string `json:"resolvedQuery"`
                Action string `json:"action"`
                ActionIncomplete bool `json:"actionIncomplete"`
                Parameters struct {
                        Name string `json:"name"`
                } `json:"parameters"`
                Contexts []struct {
                        Name string `json:"name"`
                        Parameters struct {
                                Name string `json:"name"`
                        } `json:"parameters"`
                        Lifespan int `json:"lifespan"`
                } `json:"contexts"`
                Metadata struct {
                        IntentID string `json:"intentId"`
                        IntentName string `json:"intentName"`
                } `json:"metadata"`
                Fulfillment struct {
                        Speech string `json:"speech"`
                } `json:"fulfillment"`
        } `json:"result"`
        Status struct {
                Code int `json:"code"`
                ErrorType string `json:"errorType"`
        } `json:"status"`
}

func main() {
        url := fmt.Sprintf("https://api.api.ai/v1/query?v=20150910&query=weather&lang=en&latitude=35.925&longitude=-86.8688889&sessionId=1234567890")

        // Build the request
        req, err := http.NewRequest("GET", url, nil)
        if err != nil {
                log.Fatal("NewRequest: ", err)
                return
        }        
        // Replace 9ea93023b7274cfbb392b289658cff0b by your Client access token
        req.Header.Add("Authorization", "Bearer 9ea93023b7274cfbb392b289658cff0b")

        // For control over HTTP client headers,
        // redirect policy, and other settings,
        // create a Client
        // A Client is an HTTP client
        client := &http.Client{}

        // Send the request via a client
        // Do sends an HTTP request and
        // returns an HTTP response
        resp, err := client.Do(req)
        if err != nil {
                log.Fatal("Do: ", err)
                return
        }
        
        // Callers should close resp.Body
        // when done reading from it
        // Defer the closing of the body
        defer resp.Body.Close()       
        
        // Fill the record with the data from the JSON
        var record QResponse   
        
        // Use json.Decode for reading streams of JSON data
        if err := json.NewDecoder(resp.Body).Decode(&record); err != nil {
                log.Println(err)
        }

        fmt.Println("Status = ", record.Status.Code)
        fmt.Println("Response = ", record.Result.Fulfillment.Speech)
}

https://docs.api.ai/docs/reference allow you to submit queries and get text-to-speech results.

All the URLs have the following base:

https://api.api.ai/v1/

The query endpoint is used to process natural language, either in the form of text or a sound file. The query requests return structured data in JSON format with an action and parameters for that action. Get /query — Takes natural language text and information as query parameters and returns information as JSON.

We are going to find out the weather at Franklin, Tennessee, USA.

Query parameters

Clipboard01

https://api.api.ai/v1/query?v=20150910&query=weather&lang=en&latitude=35.925&longitude=-86.8688889&sessionId=1234567890

With the parameters in the above table, our URL is as shown above.

fmt.Sprintf

“Sprintf” formats and returns a string without printing it anywhere.

func NewRequest(method, urlStr string, body io.Reader) (*Request, error)

“NewRequest” returns a new “Request” given a method, URL, and an optional body. “NewRequest” returns a “Request” suitable for use with “Client.Do”.

func (h Header) Add(key, value string)

“Add” adds the key, value pair to the header. It appends to any existing values associated with key.

// Replace 9ea93023b7274cfbb392b289658cff0b by your Client access token 
req.Header.Add(“Authorization”, “Bearer 9ea93023b7274cfbb392b289658cff0b”)

For each API request, include the above HTTP header.

client := &http.Client{}

A “Client” is an HTTP client.

resp, err := client.Do(req)

“Do” sends an HTTP request and returns an HTTP response. When “err” is nil, “resp” always contains a non-nil “resp.Body”. Callers should close “resp.Body” when done reading from it. Use “Defer” for closing the body. “resp.Body” is of type “io.Reader”.

defer resp.Body.Close()

Next, “NewDecoder” returns a new decoder that reads from “io.Reader”. A “Decoder” reads and decodes JSON objects from an input stream.

func NewDecoder(r io.Reader) *Decoder

“Decode” reads the next JSON-encoded value from its input and stores it in the value pointed to by v.

func (dec *Decoder) Decode(v interface{}) error

Finally, we extract the information from our populated “QResponse” struct variable “record”.

Here’s a sample output of the program:

Response = It’s currently 63 degrees and cloudy in Franklin, Tennessee. It will be intermittent clouds and around 62 degrees this afternoon. Expect a thunderstorms evening, with temperatures around 66 degrees.

That’s it!

Posted in Go, Tutorials | Tagged , , | 1 Comment

Use Go and Clarifai API to Recognize NSFW Images

The ‘Not Safe For Work’ (NSFW) model analyzes images and videos and returns probability scores on the likelihood that the image or video contains pornography. For those of you who don’t know what NSFW is, we’re talking about Not Safe For Work. R-rated content. Stuff that would probably get you fired if you were looking at it at work.

I was reading “How To Use Clarifai To Protect Your Eyes From Seeing Something They Can’t Unsee” and thought of writing a Go program instead of Python, as used in that blog post.

To write our Go program, we shall build upon what we learned in my previous article “Using Go with an image and video recognition API from Clarifai”.

Models

When images or videos are run through the tag endpoint, they are tagged using a model. A model is a trained classifier that can recognize what’s inside an image or video according to what it ‘knows’. Different models are trained to ‘know’ different things. Running an image or video through different models can produce drastically different results.

If you’d like to get tags for an image or video using a different model, you can do so by passing in a model parameter. If you omit this parameter, the API will use the default model for your application.

NSFW

The ‘Not Safe For Work’ model analyzes images and videos and returns probability scores on the likelihood that the image or video contains pornography.

The response for NSFW returns probabilities for nsfw (Not Safe For Work) and sfw (Safe For Work) that sum to 1.0. Generally, if the nsfw probability is less than 0.15, it is most likely Safe For Work. If the nsfw probability is greater than 0.85, it is most likely Not Safe For Work.

We shall be using the Model: nsfw-v1.0 and the following image available at the url — https://samples.clarifai.com/nsfw.jpg:

nsfw

Here’s the complete code for our Go program “mynsfw.go”.

package main

import (
        "encoding/json"
        "fmt"
        "net/http"
        "net/url"
        "strings"
)

const (
        clientID     = "Your ClientID"
        clientSecret = "Your Client Secret"
)

type TokenResp struct {
        AccessToken string `json:"access_token"`
        ExpiresIn   int    `json:"expires_in"`
        Scope       string `json:"scope"`
        TokenType   string `json:"token_type"`
}

type TagResp struct {
        StatusCode string `json:"status_code"`
        StatusMsg string `json:"status_msg"`
        Meta struct {
                Tag struct {
                        Timestamp float64 `json:"timestamp"`
                        Model string `json:"model"`
                        Config string `json:"config"`
                } `json:"tag"`
        } `json:"meta"`
        Results []struct {
                Docid uint64 `json:"docid"`
                URL string `json:"url"`
                StatusCode string `json:"status_code"`
                StatusMsg string `json:"status_msg"`
                LocalID string `json:"local_id"`
                Result struct {
                        Tag struct {
                                ConceptIds []string `json:"concept_ids"`
                                Classes []string `json:"classes"`
                                Probs []float64 `json:"probs"`
                        } `json:"tag"`
                } `json:"result"`
                DocidStr string `json:"docid_str"`
        } `json:"results"`
}

type NSFWResp struct {
        StatusCode string `json:"status_code"`
        StatusMsg string `json:"status_msg"`
        Meta struct {
                Tag struct {
                        Timestamp float64 `json:"timestamp"`
                        Model string `json:"model"`
                        Config string `json:"config"`
                } `json:"tag"`
        } `json:"meta"`
        Results []struct {
                Docid float64 `json:"docid"`
                URL string `json:"url"`
                StatusCode string `json:"status_code"`
                StatusMsg string `json:"status_msg"`
                LocalID string `json:"local_id"`
                Result struct {
                        Tag struct {
                                Classes []string `json:"classes"`
                                Probs []float64 `json:"probs"`
                        } `json:"tag"`
                } `json:"result"`
                DocidStr string `json:"docid_str"`
        } `json:"results"`
}

func main() {
        accessToken, err := requestAccessToken()
        if err != nil {
                fmt.Println(err)
        }
        
        nsfw, err := getNSFW(accessToken)
        if err != nil {
                fmt.Println(err)
        }

        if nsfw[0] > nsfw[1] {
            fmt.Println("Safe for work, you can trust Bob again (for now)!")
        } else {
            fmt.Println("Not safe for work. Classic Bob.")
        }
}

func requestAccessToken() (string, error) {
        form := url.Values{}
        form.Set("grant_type", "client_credentials")
        form.Set("client_id", clientID)
        form.Set("client_secret", clientSecret)
        formData := strings.NewReader(form.Encode())

        url := fmt.Sprintf("https://api.clarifai.com/v1/token/")

        req, err := http.NewRequest("POST", url, formData)
        if err != nil {
                return "", err
        }

        req.Header.Set("Content-Type", "application/x-www-form-urlencoded")

        httpClient := &http.Client{}

        resp, err := httpClient.Do(req)
        if err != nil {
                return "", err
        }

        defer resp.Body.Close()

        var record TokenResp

        if err := json.NewDecoder(resp.Body).Decode(&record); err != nil {
                return "", err
        }

        return record.AccessToken, nil
}

func getNSFW(token string) ([]float64, error) {
        // Analyze the image at https://samples.clarifai.com/nsfw.jpg
        url := fmt.Sprintf("https://api.clarifai.com/v1/tag/?model=nsfw-v1.0&url=https://samples.clarifai.com/nsfw.jpg")

        req, err := http.NewRequest("GET", url, nil)
        if err != nil {
                return nil, err
        }

        req.Header.Set("Content-Type", "application/json")
        req.Header.Set("Authorization", "Bearer "+token)

        httpClient := &http.Client{}

        resp, err := httpClient.Do(req)
        if err != nil {
                return nil, err
        }

        defer resp.Body.Close()
        
        var record NSFWResp

        if err := json.NewDecoder(resp.Body).Decode(&record); err != nil {
                return nil, err
        }
        
        return record.Results[0].Result.Tag.Probs, nil
}

The main thing to observe in the above program is the url:

url := fmt.Sprintf(“https://api.clarifai.com/v1/tag/?model=nsfw-v1.0&url=https://samples.clarifai.com/nsfw.jpg")

in the function:

func getNSFW(token string) ([]float64, error) {

and the new structure:

type NSFWResp struct {
        StatusCode string `json:”status_code”`
        StatusMsg string `json:”status_msg”`
        Meta struct {
                Tag struct {
                        Timestamp float64 `json:”timestamp”`
                        Model string `json:”model”`
                        Config string `json:”config”`
                } `json:”tag”`
        } `json:”meta”`
        Results []struct {
                Docid float64 `json:”docid”`
                URL string `json:”url”`
                StatusCode string `json:”status_code”`
                StatusMsg string `json:”status_msg”`
                LocalID string `json:”local_id”`
                Result struct {
                        Tag struct {
                                Classes []string `json:”classes”`
                                Probs []float64 `json:”probs”`
                        } `json:”tag”`
                } `json:”result”`
                DocidStr string `json:”docid_str”`
        } `json:”results”`
}

When you run the program:

go run mynsfw.go

The output you get is:

Safe for work, you can trust Bob again (for now)!

That’s it!

Posted in Go, Tutorials | Tagged , , | Leave a comment