Don’t test your code

I spoke at Agile India 2015 in March. Excellent conference but I was a quite disappointed that the videos are not publicly published. So, I decided to write about it here in detail.

The slide deck has been published at on slideshare But here goes the picto-blog. This is an experience report – the story of Josh Software and you can see a quick 2 minute preview here

Dont test your code.001The entire talk has the theme of war – because that was exactly what it has been. If you saw the preview, I have a sword there because I had promised my daughter that I would use it the next time I took the stage. It did fit the bill on this war theme.

Dont test your code.004

 

 

So, this talk does not talk about going in the wild and changing code on production without testing it. This is a story about about how manual QA can make you blow yourself up!

Dont test your code.005

 

 

 

This is a story about a perspective – towards cleaner code, good code quality and happy customers. This is what we learnt over the years when we started Josh Software in 2007 as a company that specialises in Ruby on Rails.

Dont test your code.009

 

When we started, we were like a refugee camp searching for clients and leading a hand to mouth existence. Sethu and I started with no experience (and no money or savings). This was our entrepreneur plunge in to the wild. Of course, I admit, it’s been an excellent journey and we are far better off than being “lost to the corporate cause” ;)

Dont test your code.011Anyway, back to Josh… in 2007, we had our first war at Josh. Just like World War 1, this was an internal battle between egos – those of our developers and the QA. It was a question of responsibility and while that battle raged, our customers suffered.

 

 

Dont test your code.012Of course, we matured and like all mature countries, we had our Second World War at Josh in 2008.  – this was a better organised war between our QA and Developers on one side and our client on the other. This was a war on scope of work, delays caused due to missed deadlines, finger-pointing and low budgets!

 

Now, the crisis had reached critical mass and we needed to the right thing – choosing the right customers and concentrating on quality over time and cost! This then brought the new order at Josh:

Dont test your code.016 Dont test your code.017

We also overhauled all our old arsenal to the new arsenal.

Dont test your code.018We used to use SVN, Trac, Redmine, and Bugzilla. While these are still good tools, it didn’t suit use. SVN and Trac had local server installation dependencies causing numerous problems. Redmine and Bugzilla couldn’t help in the way we wanted to manage projects.

 

Dont test your code.020

Now, we switched to Github as our code repository and started using Pivotal Tracker or Trello for our project management. We added Code Climate for our code quality checks and CircleCI for continuous integration!

A special mention for Slack as it’s fast evolving into an ideal communication platform.

The theme of Agile India this years was #noestimates. While I was a little taken aback at it’s emphasis in almost every talk, I also realised that it was exactly what we do:

Dont test your code.023We estimate (sorry Forecast) a budget and then race to deliver stories. We estimate the stories at run-time and keep flexibility for adding, removing and changing priorities for any stories in a current iteration too! Pivotal Tracker for the win.

 

 

Dont test your code.028We use CodeClimate and CircleCI to their full potential thereby removing the need for any manual QA. Now, we are in a unique situation, where we have only engineers in our office – everyone codes and everyone writes their tests. We have never stressed on TDD or BDD (or BDT, TBD, DDT, TDT etc. ;) As long as the code is covered, we are good to go. Dont test your code.026

 

Since we get emails about code-coverage and test-coverage, it’s easy to monitor and manage projects among the entire team. In fact there is always some healthy competition between teams to ensure good code quality and better test coverage.

 

Now, what remained was browser testing. Even though there are good automated tools for cross-browser testing, nothing beats the human eye!

Dont test your code.032Along with this, we also changed and improved our processes. I believe that Agile, scrum, sprints etc. are overrated. We have our own process and we always encourage teams to have their own strategy. We don’t have daily standups and any process control. We hope our customers talk to our developers directly and keep the project management tool at the centre of the development universe!

Dont test your code.033Well, we also pushed agile to a whole new level. We have started Agile billing – we don’t charge any advance from our customers. This means its a “pay as you go” model where the customer always pays for what they get!

We bill every 2 weeks and that also keeps our cash flows running smoothly. Above all, it builds a lot of trust with our customers. It’s only natural that this keeps everyone on their toes.

Dont test your code.035All these changes have ensured that Josh Software has now become very successful in any work undertaken and we can proudly say that we have customers who are proud of us! Writing this post itself is testament to that! :) We feel like winners indeed when our customers value us as much as we value them!

 

They do say that there are no winners in war, so it was only fair to end my talk/post with an awesome photo of Dalai Lama – “may the peace be with you”.

Dont test your code.036

Posted in Conferences | Tagged , , , | Leave a comment

Uniqueness Gotcha!!!

Gautam Rege:

validation_uniqueness_of does not guarantee uniqueness. Interesting write-up by Yogesh about things to remember while working with uniqueness in Rails.

Originally posted on Happy Coding!!!:

♦ The Problem

Consider the following relation where poll is having many options and each option of the poll must be having a unique description,

Now, when trying to create a poll with its options using nested attributes, uniqueness validation is not getting applied due to race condition.

> poll = Poll.new(options_attributes: [ { description: 'test' }, { description: 'test' } ])
> poll.save
=> true
>
> poll.options
=> [#<Option id: 1, description: "test">, #<Option id: 2, description: "test">]

♦ Why it is occurring ?

There is a section in UniquenessValidator class, which mentions that the ActiveRecord::Validations#save does not guarantee the prevention of duplicates because, uniqueness checks are performed at application level which are prone to race condition when creating/updating records at the same time. Also, while saving the records, UniquenessValidator is checking uniqueness against the records which are in database only, but not for the records which are…

View original 128 more words

Posted in General | Leave a comment

Using Service Classes to improve code-quality

Gautam Rege:

Good post from Rishi on how to refactor code into Service Objects to improve code maintainability and quality.

Originally posted on Learning shall never stop..!!!:

ServiceClasses are nothing but plain old ruby classes. But they go a long way to keep your controllers thin and models thin. To show what I mean by this, lets look at some code. This is the code in controller to download data following some set of rules.

This is bad. The controller should not look like this. Comes in Service classes.

The ideology behind service classes is that it should have only one purpose to serve and nothing else.

And I can clearly see a purpose which our new service class should serve and that is to provide the data for our template download and nothing else. I keep the service classes under app/services folder. Also I do not think I need to iterate much on the importance of the name of the service class you choose. It goes without saying that the name should tell the purpose it…

View original 237 more words

Posted in General | Leave a comment

Writing a Chrome extension

Gautam Rege:

Here is an excellent post on how to write Chrome extensions. It’s complete with an example using Pocket.

Originally posted on Into the Wild:

With a large number of services available online, we tend to to spend most of our time on our browser. Well browser do come handy for a large number of activities from reading news to making travel and dining reservations. What can be better than a browser that can do some cool stuff with a click of mouse or a simple key stroke… I guess nothing

Google Chrome is the best web browser around right now (see usage statistics), and part of Chrome’s appeal is owed to its excellent extensions. Its not really hard write simple extension to get your work done.
All it needs is a little javascript, html and may be some css if you want things to be a little more pretty.

So lets build one…

Purpose: Export Chrome bookmarks to Pocket account.

Requirements:

  • Some javascript skills
  • poor html skills
  • since we want to talk…

View original 548 more words

Posted in General | Leave a comment

Integrating legacy Visual Basic app with Rails

What is the best way to import data from legacy external sources into our Rails app and trigger certain tasks automatically in our rails app ? A very difficult question to answer and it varies in every case.

Well, here was our challenge – The earlier process was that the new entries were being dumped into a table using VBA code and then from our Rails admin portal, we had to click a button to process these entries and load them into a new table. A very painful and manual process. To eliminate the manual process, the idea now was to trigger the migration process straight from the VBA code ie. once new entries are loaded into the from_table, our rails system should be able to kickstart the migration job and transfer them to to_table.

The first solution that came to our mind was to write database level triggers to move the data from the from_table to to_table but there were ActiveRecord callbacks in our application which affected three other tables. So rewriting all that code at the database level did not seem right.

The second solution that came to our mind was to move the code from VBA to ruby, so that we could automate the whole process. But then there was some processing going on in the VBA code before creating the row and due to business constraints, we had to work with VBA code (let sleeping lions lie). Point taken.

The next solution to have a cron job running every 30 mins was rejected by the client. He did not want to wait for that long to see the data migrated. We could have written a cron job running every 1 min, but that would have created many processes. And since the migration sometimes took more than 1 minute (depending on the data), it would have been a dangerous possibility to have 2 processes parallely doing the same thing.

Now, the big question was how to pass a message from VBA script directly to our rails app?

With a little help from the database, we designed an unorthodox solution which was easy to develop and got the job done. There might be a better and cleaner solution and we are all ears, but I am proud of what we have cooked up. So here is the recipe.

So lets divide the task into 2 main parts

  1. Write a rake task to watch a table named “triggers” and trigger whenever new entry is created in this table
  2. Make sure that this rake task is monitored by monit so that we don’t need to start/stop/monitor this task manually

Step 1

The VBA script created a entry in triggers table with end_time as null
insert into triggers set start_time = current_timestamp
Our rake task was basically an infinite loop where we checked that if there is an entry in triggers table with end_time nil, then we trigger the ruby function to migrate entries.
desc "Migrate new entries"
task :monitor_entries => :environment do
  `echo "#{Process.pid}" > #{Rails.root}/log/monitor_entries.pid`
  `echo "#{Process.ppid}" > #{Rails.root}/log/monitor_entries.ppid`
  while(1)
    trigger_entries = Trigger.where(end_time: nil)
    if trigger_entries.length > 0
      results = FromTable.process
      trigger_entries.each {|trigger| trigger.update_attribute(:end_time, DateTime.now.utc) }
    else
      sleep 30
    end
  end
end

 

The lines for storing pid and ppid were important, so that we could use them in our monit script.

Step 2

Now it was time to make changes to `project.monitrc` file so that we did not have to monitor this background rake task.
check process job_trigger
  with pidfile /path/to/log/monitor_entries.pid
    start program = "/bin/bash -c 'export rvm_path=/path/to/.rvm; . $rvm_path/bin/rvm; cd /path/to/rails/app; bundle exec rake monitor_entries'"
    stop program = "/bin/bash -c 'kill $(cat /path/to/log/monitor_entries.pid)'"
We also added the below line to the to `:launch` block in mina deploy script, so that this task gets started on its own after every deployment.
queue "sudo monit restart job_trigger"
Thankfully everything went to plan. Would love to hear your ideas to improve this recipe :)
Posted in Ruby | Tagged , , | 2 Comments

Tips and Tricks using Git

Gautam Rege:

Here are some often used Tips and tricks with Git

Originally posted on Happy Coding!!!:

Earlier, for maintaining Linux kernel, Linus Torvalds and his team were using BitKeeper asSCM system, but they had to opt out for another system because their were some concerns about the OSL and the support for the free version of BitKeeper had been dropped. Torvalds and his team wanted a SCM which is more faster in terms of performance than the existing systems, he wanted it to be fast enough that it can apply a patch in 3 seconds and update its associated metadata, but none of the systems were meeting his needs at that time, so he decided to develop his own.

So, Torvalds and his team started the development of new SCM in April 2005. He wanted it to be named after himself, like Linux. He said that he considers himself as egotistical bastard, so he preferred the name Git (English meaning of git is an unpleasant person :-)

View original 1,100 more words

Posted in General | Leave a comment

Ruby through rails part 6: Bundler Dsl

Gautam Rege:

What happens when you write ‘gemspec’ in your Gemfile? Learn some Ruby by understanding the Bundler DSL in this series of posts.

Originally posted on narutosanjiv:

Earlier we seen detailed working of ‘gem'(present in Gemfile) command. For now, we are now going to introspect the working of ‘gemspec’ which we mostly used during building our rubygems. As i have mentioned in earlier serials that all command, which we used in Gemfile, found in the bundler file(lib/bundler/dsl.rb)

Let see how ‘gemsepec’ is used in gemfile.

Normally, when we write our rubygem, our directory name is the name of gem and gemspec found in same directory as “name_of_gem.gemspec”.

Let see the code of ‘gemspec’ defined in bundler(lib/bundler/dsl.rb).

Let understand above implementation with code example. Consider we have ‘dummy’ as rubygem.
Content of Gemfile of ‘dummy’ gem are:

As we do ‘bundle install’, then gemspec call without any options. Since we did not pass any options, default name, path, development_group get set as ‘.(current_directory)’, ‘{,*}’ and
‘development’ respectively. Line no 6 get ‘#{name}.gemspec’ file if name provided as nation…

View original 159 more words

Posted in General | Leave a comment