22 September 2016

State of Rails Releases

I am dealing with multiple Rails applications at once some of which are funded right and some of which have ran into maintenance modes. I just wanted to have a pictorial representation of what is the state of each of the Rails versions, unfortunately, I cannot get one. So I spent half an hour trying to decode the Rails official releases page and came up with this.

If you closely observe, the Rails releases 3.0.x and 3.1.x are history (I don't even want to pull up data for releases before that). It is high time you plan to upgrade your stack to a minimum of Rails 4.2 before you loose all the goodness the Rails community has to offer. Yes, I know it is a herculean task for folks in 3.x in which case I recommend you to do a complete rewrite of your application piecemeal by piecemeal!

Data Source: http://weblog.rubyonrails.org/releases/


26 July 2016

20 Tips for an Effective Code Review

It is a well-established fact that most of the bugs in the Software Development life cycle could be prevented literally right at the source (code). Since Code Review is almost an inevitable process in the Agile paradigm, keep in mind these 20 tips/guidelines (in no particular order) to become an effective reviewer of code. This is not restrictive to any one language but applicable to all. I've been reviewing code for many years and one of my core successes lie in stressing these points across the team. This is also the only way to effectively nurture and scale teams across the organisation.

  1. Identify the right tool: Identifying the right tool is very important. Because one should not be thrown off for adding a review just because the tool is not efficient enough. There are many open source tools out there. In most cases, you may have to host it yourself or you can also opt for services that do the hosting for you. If you are an Open Source Contributor, you would know how effective Github can be which also happens to be my personal favorite.
  2. Pre-conditions/Checklist: Any patch or a pull request that is submitted should have a minimum set of pre-conditions like it should have a Green build. A lot of Review tools have hooks to be configured to poll the SCM automatically and run the build. Build tools like Jenkins, Travis support these with minimal to no configuration. Ensure that you use them! Because it will definitely save you time and heartache instead of seeing stuff getting pushed to your trunk/master/production branch.
  3. Avoid Repeat Mistakes: As Gustavo Fring from The Breaking Bad rightly says "Never Repeat the same mistake twice", it is crucial that repetitive patterns are broken. In the context of code review, this means the developers should not get the same review comment that they had received earlier. This ensures that with each Iteration - the Quality of the patches improves so that if at all any new review comments are there - they are only new and anything that is given in the past are assumed to have been implemented in the later ones. If this is not happening, it is up to the Reviewer to go and identify to see where the leak is.
  4. Self Review: The person who is submitting the diff should first review it him/herself. Many obvious things like debugger statements, extra/missing files, ignorable files could be identified here. And I would recommend even doing a full fledged review of his/her own code as if he/she would do another one's. This culture also reduces the burden on the reviewer of concentrating on the Meat of the patch and not the obvious ones.
  5. Design Review: The Reviewer should also be able to decode the design introduction/changes that the Pull Request has and should be in a position to judge and give appropriate feedback. This is very important. 
  6. UI Review: Although software developers look at just the code and give feedback (because that is all they can be seeing in a Pull Request or a diff), they often neglect how the end product would look in a browser or the device where the code was intended to. It is extremely difficult to guess on how it will look. I recommend everyone to go the extra mile of looking how it looks and whether it relates to the original functionality. This is going to take some extra time. In my experience, this has insane returns in terms of identity and squashing obvious UI related issues. 
  7. Non-Logical Checklist: Code Review does not only involve in vetting the Logical Integrity but also some non-logical things like Naming convention, Spacing/Indentation, Object oriented compliance checks. Ensure that there is such a checklist in the first place.
  8. Keeping a pulse on the industry: This is true not only in this context but also on the overall wholeness of a Programmer. You should be up to date on what is going on in the Programming world, at least in the particular language you are part of. Knowledge of things like critical security patches, feature additions, language enhancements, performance improvements proves to be really powerful in assisting an effective review process.
  9. Encourage feedback: One does not always have to agree with what is said in the Review. If there are some contradictions - it is best they are addressed between the Reviewer and the Reviewed (or Reviewee). I also encourage that all the review comments are responded to. This process gives confidence to the entire team that any review comment will not go unanswered. 
  10. Avoid Oral Reviews: When a patch or pull requested gets created that too for teams and developers co-located or sitting next to each other, it is tempting to just go through it and give all the feedback orally. This may be fine if the team is small (only 2) and they fully own the codebase. However, this has some negative effects in terms of follow up and broadcasting. What I mean by broadcast is that there could be a review comment which could be applicable for the entire team.
  11. Learn from other reviews: Encourage to team members to not just read your own reviews and apply however to read the other reviews within the team. I've heard this famous quote - 'An intelligent person learns from their own mistakes, but a genius learns from the mistakes of others'. Let's make everyone in the team a Genius! 
  12. Dual Reviews: Similar to a Doubly refined sugar or Oil, the throughput and Quality of the code review could improve if it has a Second reviewer if that's possible.
  13. Review the Reviewer: It is a bit over-zealous to expect anyone coming new to the team who is relatively younger to the software development or who has not involved in Review process in the past to quickly catch up to all the nuances in the code review process. It would be nice if these guidelines are slowly implemented and mentoring/onboarding is in the organisation's culture. In simple terms, there could be a reviewer who can review whether reviewer complies to all the best practices out there.
  14. Over Engineering: At times it is tempting for Reviewers to comment on things that may look like Over Engineering work. These cases it is okay to voice your opinion to the reviewer.
  15. Enterprise Adherence: An Enterprise will have an adherence in different horizontals in terms on what tools they need to use, what style guide they need to follow, what frameworks has been used across different projects. It is up to the Enterprise Architect or the Senior Member of the team to proactively absorb all these facts and ensure that the entire review process is in Adherence with the overall Enterprise. This is crucial because each Atomic commit may slowly introduce things that could stray away from what the Enterprise would want. It may not look like a problem at all in the initial phase. However, should there be a consolidation happen across various projects - having multiple stacked apps across the enterprise would result in painful Refactors and often ends up leaving a huge amount of technical debt behind.
  16. Dependency Injection: Be wary of addition or removal of a new Library to the code. This again falls under the adherence of standards across the projects. Make sure that any introduction of a new library is well evaluated across the team and that it has enough support both in the near and long run. I have seen a lot of libraries which were started by individual contributors go unmaintained for years. Ensure that there is a strong community following and is very active.
  17. Against the Right branch: This may seem like something that may not belong here but in my personal experience I've faced this issue multiple times where a Reviewed creates a pull request against a different (or default) branch instead of the one that it actually has to go.
  18. Tech Debt Identification: During the course of the review, the reviewer may stumble upon an issue which involves a good amount of effort. In such cases, it is not advised to block it and hamper the delivery commitments. Instead, the right thing to do here is to add these things to a technical debt backlog where it could be groomed and picked up in future.
  19. Copy Paste excuses: "I did not do this - it was already there - I just copied/moved it" - Yes this is a very common statement every developer says when his code is challenged - however ensure that any code that is touched has to comply to the coding standards set by the team.
  20. Make Guidelines explicit: It is a very good process for all the developers on-boarding to a new team to have a set of guidelines (you could use this) explicit and review it from time to time. This could be done across the organisation.

The above list may look overwhelming. However, if you have the knack and right drive to implement some or all of these - the productivity of the Engineering team would increase by multi-folds.


08 June 2016

Jenkins bump from 1.x to 2.x

Jenkins is one of the amazing open source softwares especially after it forked itself from its predecessor Hudson. Amazing for multiple reasons - but the one that has really "amazed" me is the painless upgrades it provide. Trust me, I am a Rails developer and I know in & out on what a nightmare it is to perform an upgrade!

Recently I had to perform an upgrade for Jenkins and it was as dead simple as replacing a .war file and I was err.... DONE! All I did was stopped and started the process whilst replacing the war file. Once I rebooted, it all just worked. They seriously think about making their installations to run the latest software which I personally consider a true value.

Anyways, the reason me posting this is to help my fellow developers who are running into an issue where their slave would not start up after the upgrade process.

You would face an error something like this below when you go and inspect in the slave agent.

What this error means is that it could not invoke the slave because of an outdated java running in the slave. All you've got to do is upgrade it and you should be all good to go.

This was how the overall upgrade felt like :)


31 May 2016

Responsive Email design with Rails

It is almost imperative in the recent times, the emails we send out are expected to be responsive with a heavy number of users preferring to read or more like skim through emails from their smartphones. To find the ideal sweet spot that aids in not only developing fully responsive emails, but also to do it quickly and easily is vital. There are lot of factors should be taken into account both from a business perspective and from a developer standpoint. I am listing them here (in no particular order)

  1. Responsive design - works consistent across all the devices from mobile layout to the most stringent Outlook Email client.
  2. The UI should be consistent with ways to freeze the Headers, Footers and should follow a proper template similar to Rails Action View Layouts.
  3. Should be able to easily testable in developer mode with support for Plain text view besides supporting HTML View.
  4. Avoiding hardcoding of styles in each and every HTML tag. Hardcoding styles in the email has been the norm in Rails community and other web frameworks as well for a very long time.
  5. Should be easily testable in all types of browsers. Even a minor modification/tweak should be tested quickly instead of painfully sending emails again and again.
The following may seem a shorter list, but believe me - to quench all the above criterions I had to go through a lot of different phases with varied learning curves. To attack all the above problems - I would suggest the following tools and libraries to make our lives super simple.

  1. Zurb's "Foundation for emails" (previously called Ink) that provides with ready-made available templates to kick start and later customise on top of it to our heart's content.
  2. Premailer-Rails - A wonderful Rails pre-processor that makes the email design entirely stylesheet driven as opposed to hardcoding styles directly in the tag. Not only does it removes the pain of having hardcoded styles, it also provides a packages view to render the Plain text automagically - with 0 amount of code required from the developer.
  3. Letter Opener - A classical tool by Ryan to quickly preview the emails in development mode.
  4. Litmus - If you are into Responsvie email design, you have no reason not to subscribe to Litmus as they provide a comprehensive way to template, design and test your email in in-numerous email clients.
That is it! Combine these tools and with a slight learning curve, you can claim yourself as a fully responsive e-mail designer.


22 January 2016

Streaming vs Synchronus Replication in Postgres

I recently faced one strange issue in Rails which usually questioned some of the basic Relation Database principles. It gave me almost a sleepless night until I was able to get to the Root cause of the issue.

The problem

The problem was pretty straightforward. A Rake task generates an email and the email had two places where the count of documents was mentioned. Ideally they are supposed to be the same - but for some reason it was different.

The pain point

The reason this particular problem was painful because this has not occurred for few years and that it occurred only intermittently. The problem with intermittency is that there is always some theory behind. Here too there was something. Here are steps I had to perform to find the Root cause.

The approach

I first looked into the Rake task's log file which is outputted when my specific Email job runs. Things looked fine there - meaning it completed in under 90 seconds as expected.
The next step was to look at the production logs. The logs as expected was having 30 insert statements - Check. And it also has a read statement for the insert statements before and it was a typical count(*) query. The problem occurred at this point. The count(*) should have returned 30 but instead it returned 4. There comes another count(*) somewhere below in the code - but that returned 30 as expected!

The above step revealed that this problem is not with the Rails layer but something to do with our production database setup. So routed my energy towards there.
The production database environment is a Master-Slave configuration with Master taking Writes and Reads and Slave purely configured to take Reads. Both these nodes are load balanced via a PG Pool server. My initial gut said to investigate some time in the PGPool but that is not much useful as all PG Pool going to do is route traffic.
So I went and read about the Master - Slave Replication configuration. I read about two types of replication. One being synchronous replication and the other being Streaming Replication. Digging into that I found my root cause!

Synchronous vs Streaming Replication

Assume you have two databases A and B with A being a R/W Master and B being R-only Slave. If an insert or update command is issued, it goes and writes that entry to A as its configured for write. If the database returns after it ensures that all the slaves got this write - it is called as Synchronus or 2-Safe Replication. If A does not wait for this step however acknowledge whether it wrote successfully and later streams that value to B - this is called as Streaming Replication.

Both has their obvious own pros and cons. Streaming Replication is for Raw Speed and is also a very good configuration where there are too many writes. And Synchronous Replication although not as fast as Streaming provides 100% consistency. We unfortunately were in Streaming Replication mode. The 30 inserts happened so fast at A, that before even it could stream them to B, the count query intervened and read the half baked data from B. I am talking in terms of millisecond speed.

How did we fix it?

We isolated all our cron jobs to run in a dedicated node and pointed the database directly to the Master database server skipping the PG Pool in the process. In a single database configuration the concept of Streaming or Synchronous Replication does not apply. Hope this was helpful!


31 December 2015

Hotel Automation Controller - Interview coding problem

This is one of the problems I got via a friend who recently faced a company called Sahaj Software a clone company to Thoughtworks.

Problem Statement:

Hotel Automation Controller Problem  Statement

A very prestigious chain of Hotels is facing a problem managing their electronic equipments. Their equipments, like lights, ACs, etc are currently controlled manually, by the hotel staff, using switches. They want to optimise the usage of Power and also ensure that there is no inconvenience caused to the guests and staff.

So the Hotel Management has installed sensors, like Motion Sensors, etc at appropriate places and have approached you to program a Controller which takes inputs from these sensors and controls the various equipments.

The way the hotel equipments are organised and the requirements for the Controller is below:
  • A Hotel can have multiple floors
  • Each floor can have multiple main corridors and sub corridors
  • Both main corridor and sub corridor have one light each
  • Both main and sub corridor lights consume 5 units of power when ON
  • Both main and sub corridor have independently controllable ACs
  • Both main and sub corridor ACs consume 10 units of power when ON
  • All the lights in all the main corridors need to be switched ON between 6PM to 6AM, which is the Night time slot
  • When a motion is detected in one of the sub corridors the corresponding lights need to be switched ON between 6PM to 6AM (Night time slot)
  • When there is no motion for more than a minute the sub corridor lights should be switched OFF
  • The total power consumption of all the ACs and lights combined should not exceed (Number of Main corridors * 15) + (Number of sub corridors * 10) units of per floor. Sub corridor AC could be switched OFF to ensure that the power consumption is not more than the specified maximum value
  • When the power consumption goes below the specified maximum value the ACs that were switched OFF previously must be switched ON

Motion in sub corridors is input to the controller. Controller need to keep track and optimise the power consumption.

Write a program that takes input values for Floors, Main corridors, Sub corridors and takes different external inputs for motion in sub corridors and for each input prints out the state of all the lights and ACs in the hotel. For simplicity, assume that the controller is operating at the night time. Sample input and output below.

Initial input to the controller: Number of floors: 2
Main corridors per floor: 1

Sub corridors per floor: 2

Since the hotel management is trying this for the first time, they would be changing the requirements around which electronic equipments are controlled and the criteria based on which they are controlled, so the solution design should be flexible enough to absorb these requirement changes without significant change to the system.

The solution to this problem involves approaching in an object oriented manner. Also we need to see here that we should use a Command/Strategy Pattern given there could be changes in the behavior based on external factors. I have not included the timings from the problem but from on here it should be easily extensible.

Code below:


21 November 2015

Cloning remote PG database and loading in Local environment

For projects involving small to medium sized databases one may require to copy the remote (or production) database onto local environment. I was earlier doing this for my production application using custom pg_dump and then restoring with pg_restore. It was relatively straightforward but still consumed good amount of time. I wanted to automate this using capistrano and this is how I did it

You should note that this is extremely fast because it executes the command on the VPS - usually EC2 which has amazing internet speeds. And then copies it over scp as a single file. You can also add a compression step using the --format option in the pg_dump.

Hope this was helpful!


Dealing with Intermittent Build failures due to Memory - Jenkins + EC2

It is not uncommon to see a Jenkins build failure due to a memory choke now and then while running thousands and thousands of rspec examples. The examples may be too much for the memory allocated within the EC2 instance. One simple solution is to enable Swap memory. Going through the typical EC2 route you need to have a dedicated Swap Partition. However if you feel you don't want to go through that route, you can simply do it via a swap file.

Make sure you have root access and follow these instructions to enable swap memory usage. Typically you could go from 1 to 2 times the allocated RAM but that is not a hard rule. In the current scenario I am going to elicit what I did.

Done! you may check whether this is enabled by typing the free command.

Note: This will only persist until the machine is running. If you reboot this will go away. If you still want to persist the swap after a restart you may do

Hope you find this article helpful. This is a trimmed down version of a wonderful article from DigitalOcean.


03 November 2015

Stand still - and you are going backwards!

Imagine you are standing on a moving escalator that is supposed to take you from a Ground level to the Top level - only that its moving in the opposite direction. If you stand still on such an escalator you will hit the bottom soon. In order to not hit the bottom - you need to "at least" walk at the -ve speed of the elevator. For an observer it would still be standing still but thats the least one could do especially being in the software industry.

My blog once had crossed a million page views and the most active time I was during 2010. I think looking back - that was the time I had cracked any problems thrown at me. And I enjoyed it. I still could crack em only that I realise recently I've become rusty. However, I developed a variety of different skills down the road.

Alright I think I am diverging too much from what I wanted to say. In the IT world, one needs to constantly keep learning, keep pushing the limits, get hands dirty on a variety of technologies. If you are a manual tester - try poking at Automation testing. If you are an Automation Engineer - learn about performance/scalability testing. If you are an application developer - explore system development. If you are a software architect - see what is going on in that particular space and try to remain on top. That is the only way you could make sure that you are always on the 99th percentile in this industry.

One has to realise doing the mundane and similar work won't scale as we have already headed and running in the First order differentiation of time (speed). Second order differentiation (acceleration) will make us even more redundant. Can't imagine what we differentiated even one more step. The only way to sustain such challenging epochs is to learn. Learn learn learn!


Printing the permutation of characters from a classical mobile phone

One of the problems I recently faced.

Input: A set of numbers from a mobile keypad
Output: Print all the combinations of characters for that given number.

For example if the input is 26. Then possible outputs are am, an, ao, bm, bn, bo, cm, cn, co


There are many ways to solve this problem. One is using a tree. Another way is to simply do it with recursion. I solved this using a linear approach. If you convert the problem from the text domain to number domain - it will seem very simple.

For example - if the input is something like ['abc', 'def', 'pqrs'] (transformed from digits 237). Then the output would have a simple rule - the first character should come from the first element, second from the second and so on. Writing in an incremental number form for the indexes, you would get:

0 0 0
0 0 1
0 0 2
0 0 3
0 1 0
0 1 1
0 1 2
2 2 3

That's it - problem solved. Programatically expressed it here

17 August 2015

VIM Cheatsheet

A one page printable cheat sheet for VIM in PDF Format. Direct URL: https://goo.gl/Yu2rpG

Source and credits to RICHARD TORRUELLAS : http://vim.rtorr.com/


08 August 2015

Ransack namespace collision - The search method

I am a big fan of Ransack, the Rails gem for quickly building a quick and easy Search based on pure ActiveRecord queries. For projects that require some on the fly search without having to set up dedicated Search Engines like Elastic Search, SOLR, ThinkinSphinx - this is a very juicy alternative.

However I recently faced one problem when I introduced this to a project where there already was SOLR integrated. It was because of the collision of the search method.

This simply patch did the job for me. All you have to do is include it to your initializers folder say under config/initializers/ransack.rb

That's it, the Ransack library will no longer collide with your existing suite. As a matter of fact the method search deprecated already and the library itself wants us to use the Alias ransack


02 June 2015

Good bye Heroku !

I primarily work on Ruby on Rails and I use to deploy one of my pet projects to Production entirely on the Heroku's free tier. It was a wonderful two years to see having my application running there during which time I had around a thumping 10 Million page views! Heroku's free tier is one of the most used around the world for quickly showcasing development applications. However, my Production application was such that all my needs felt well under Heroku's free tier.

Here are some of the freebies I use to enjoy
  • Free Dedicated (techincally not) machine with 512 MB - The reason its technically not because you are allotted a free worker that would go to sleep if the application is not up for 30 minutes. However you could easily overcome this with something like Pingdom.
  • Free Postgres Database - Although there was no specific size limit to this other than a maximum number of 5000 rows, I would still say this is one of the best things Heroku offers. Not only your application can talk to it, anything anywhere from the world can talk to the database directly. And that too 100% free!
  • Free emails via Sendgrid - A maximum 200 emails per day! Sendgrid's free Addon allows me send upto 200 emails per day. My application's requirement was in the range 20-40.
  • Unlimited Job Schedulers - Although they would take one entire and the only worker allotted to you to spin, this is a boon considering you do not have a full fledged linux machine with you. I use to have only one rake task that takes a routine back up of my data as CSV and email it to me.
  • Free PG Backup - Again, if you want to routinely backup your data, you don't have to write any complex code. Heroku provides this out of the box and for free!

Now that Heroku is pulling down the curtain on the 24 hour availability or to pay $7 a month, I have no other option to switch to a much better alternative - that is Digital Ocean.

Sad to depart ways, but you made me learn a lot of stuff in these two + years.


27 November 2013

My first ruby gem horoscope is live

I wanted to write a rubygem on my own for a long time. Never had the time really to write one. However, recently I had some spare time during which I wrote this gem called horoscope. This gem will help drafting the horoscope of a person given the birth date, time and the place. Source is hosted at Github. As with every other open source project out there, feel free to fork it and add more to your liking

Screenshot at Github

Proud to release this and I am planning to add more features to this. Also, planning to release different gems. If you want something particular or specific, don't be shy, add it in the comments section and I would be more than happy to contribute!


18 November 2013

Tree Traversal in C++

We've already seen many implementations of tree and its uses in java. Lets look at a simple implementation of traversing on a tree in C++. Code has been given below which I think is quite self descriptive.