Poetry of Programming

Its about Ruby on Rails – Kiran Soumya


Invoker – HTTP Service Manager

Pow is good. Its easy and simple to setup. Recently I faced issue with Pow that suddenly broke due to node update or whatever reason that I couldn’t still figure out. I rebooted the mac, brew updates, node updates, uninstalled and installed it back. But dunno it refused to serve my working development app.

This gave me an opportunity to switch to Invoker. (To be honest, I am not in favor of updating tools or switching to new tools that break our years together work sometimes. We always stick to the version that works until and unless there is a requirement.)

Invoker supports not only Ruby/Rack, but any web-service written in PHP or Python or Java or Django or express.js. So that’s a good news for PHP, Python, JAVA, Django and express.js developers to run web applications on different local domains without /etc/hosts hacks.(+1)

To start off with Invoker, its just three steps process (+1)

gem install invoker
sudo invoker setup
invoker start local.ini

The installation alone requires Ruby >=1.9.3 and it works with any version of app like even ruby 1.8.7 app or any other language app on HTTP service. (+1)

Refer http://invoker.codemancers.com/ if you are migrating from Pow

Before running the last step listed above, you need to make sure to add all your working app paths to local.ini file. Thereby, keeping track of all the apps in one place.(+1)

Here are some useful process management commands (+1)

# List currently running processes managed by invoker
~> invoker list

# Will try to stop running delayed job by sending SIGINT to the process
~> invoker remove app_name

# If Process can’t be killed by SIGINT send a custom signal
~> invoker remove app_name -s 9

# add and start running
~> invoker add app_name

# Restart process given by command Label
~> invoker reload app_name

# Restart process given by command label using specific signal for killing
~> invoker reload app_name -s 9

Last but not least, Logs at one place though you have 3 or 5 or 10 services running at same time. Invoker takes away the pain to look into different terminals for logs of your multiple services.
(+1 * number of apps) 😀

It’s a cool journey of Invoker ! Try it out.


Release of Rockdove

Announcing the stable release of my gem Rockdove, version 0.3.1.

So what’s the noise about Rockdove ?

Rockdove is for those applications with a requirement to poll and process every incoming mail from Microsoft EWS 1.0 Mail Server !

It doesn’t just fly away blindly with the mail but parses each mail item for the actual content by stripping off replied or forwarded content. In most of the cases, strips off the signatures too if specified under proper convention with two dashes on top.

Rockdove is installed on our Production Rails App used by around 30k users triggering around 700 mails per day initially and the count seems to be increasing day by day with a number of 100’s…! Its proven to be well tested and stable.

The setting up of this gem is damn easy and pretty clear on github. If there are any queries on this, you can always drop a comment on this blog or raise a issue on github.

I would like to share some use-cases implemented in our Rails App (a enterprise social networking portal), using rockdove:

UseCase 1: Post via Email

Whenever a user emails to our mail server, mail@ewsdomain.com, the scenario is to create a new post in our social networking rails app. Rockdove fetches the mail item, strips off the signature and share the details of the sender and the mail content to our Model which does the App business logic of finding the sender under user list and posting the content under his/her name on the portal. If the sender email doesn’t exist and is not registered, we register those users automatically via email that matches our ewsdomain.com.


Rockdove::Config.configure do |config|

config.ews_url ‘https://mail.ewsdomain.com/ews/exchange.asmx’

config.ews_username ‘username’

config.ews_password ‘password’



Rockdove::CollectMail.watch do |rockdove_parsed_mails|
rockdove_parsed_mails.each do |email|

# Operation on your rails app Model, ex. Blog

#  Rockdove Accessors one can play with email, ex. email.subject, email.body,

:subject, :body, :has_attachments?, :attachments, :to_recipients, cc_recipients, :date_time_created, :date_time_sent, :from

UseCase 2: Comment to a Post via Email

In a scenario of thread of discussions and users receiving couple of notifications on the post, the user can simply reply to a notification via email and get his mail content posted as a comment/reply to the discussion. Rockdove fetches the mail, pulls in the main content and strips off the previous post replied upon (if the email contains the forwarded content, it will be stripped off too). The content is always retrieved in the text format (supporting markdown syntax).

UseCase 3: Post to a particular User-Group

User can post to group with subject containing the group-name in square brackets ex: [My Community]. Rockdove passes the content of the subject as it is to your Rails app for further processing.

UseCase 4: Email Attachments

Rockdove supports email attachments and saves them to a file based on the path specified by the model in your rails app.

rockdove_email.save_to_file(email_attachment, “#{Rails.root}/assets/images”, filename)

# above code can be refactored as per your app requirements

UseCase 5: Ignore Senders

You can provide Rockdove to ignore mails from a group of sender’s addresses. Rockdove will ignore and even delete those mails from your inbox. ex: postmaster@ewsdomain.com

Rockdove::Config.configure do |config|


config.ews_ignore_mails [“postmaster@ewsdomain.com”]



UseCase 6: Archive Mails that are processed

You can specify Rockdove whether these mails have to be archived on a different folder after processing them or delete them from your inbox.

Rockdove::Config.configure do |config|


config.ews_folder ‘Inbox’

config.ews_archive_folder ‘Archive’


UseCase 7: Handling OOO and Mail Delivery failures

Your rails app need not worry about how to handle the OOO and delivery failure emails. Rockdove takes care of automatically collecting and deleting those mails even before passing them to your rails app. Whistles to this feature on Rockdove as it has saved a storm of automated out of office mails in the form of posts and comments in our rails app.


Rspec to work with Logger

In order to set expectations on what we actually logged during the method call,assign under before(:each)

@test_logger = StringIO.new


and then match the log output using @test_logger.string


Issue with Guard

I came across this error this morning, “The user limit on the total number of inotify watches was reached or the kernel failed to allocate a needed resource. (Errno::ENOSPC)”

Tried Guard ignore path to .git or tried reducing the watch files under Guardfile, but alas! Guard by default handles that ignore path of .git and still it throws the same crazy error…

Googled the issues under Guard gem, then realized that you need to increase the watch count for the current user on your box to fix this.
# Check current maximum
$ cat /proc/sys/fs/inotify/max_user_watches
# Increase the maximum
$ echo 100000|sudo tee /proc/sys/fs/inotify/max_user_watches
$ cat /proc/sys/fs/inotify/max_user_watches

This increases max_user_watches temporarily.On Ubuntu, to increase at boot, edit /etc/sysctl.conf and add fs.inotify.max_user_watches=100000.


Installing RVM & Git through Proxy

RVM relies on GIT.

So set proxy for GIT first.

*  Set the http_proxy environment
*  Set a proxy command to bypass the connection:
gcc -o connect connect.c
mv connect ~/bin
echo “/home/kiran/bin/connect -H proxy.company.com:6030 $@” >> ~/bin/proxy
chmod +x ~/bin/proxy

echo “export GIT_PROXY_COMMAND=proxy” >> .bashrc

Now try git clone. If it doesnt work, try out the following the command.

export http_proxy=http://<username>:<password>@<proxy_ip>:<proxy_port>

This line below also works like a charm for GIT,

git config --global http.proxy proxy_addr:proxy_port

Once the GIT is configured, for RVM, you need to do one more change for curl,
Set the proxy inside your ~/.curlrc

proxy = proxy.company.com:proxy_port

and now you can install rvm with no issues.

For rvm install thru proxy:

rvm install X --proxy proxy.company.com:proxy_port

If two developers are under the same user-group, we can even clone/copy the .rvm folder within two users without explicit installations.

Some more references:
http://zipizap.wordpress.com/2010/11/02/cloning-rvm-to-other-user-you-can-just-copy-the-rvm-directory/ [This worked for me as well]


Open ID != OAuth

Open ID single sign-on Authentication for consumers

OpenID allows you to use an existing account to sign in to multiple websites, without needing to create new passwords.You may choose to associate information with your OpenID that can be shared with the websites you visit, such as a name or email address. With OpenID, you control how much of that information is shared with the websites you visit.With OpenID, your password is only given to your identity provider, and that provider then confirms your identity to the websites you visit. Other than your provider, no website ever sees your password, so you don’t need to worry about an unscrupulous or insecure website compromising your identity.OpenID is rapidly gaining adoption on the web, with over one billion OpenID enabled user accounts and over 50,000 websites accepting OpenID for logins. Several large organizations either issue or accept OpenIDs, including Google, Facebook, Yahoo!, Microsoft, AOL, MySpace, Sears, Universal Music Group, France Telecom, Novell, Sun, Telecom Italia, and many more.OpenID is the fast, easy and secure way to sign in to websites.Here are just a few benefits to using OpenID.

Accelerate Sign Up Process at Your Favorite Websites:

Most websites ask for an extended, repetitive amount of information in order to use their application. OpenID accelerates that process by allowing you to sign in to websites with a single click. Basic profile information (such as your name, birth date and location) can be stored through your OpenID and used to pre-populate registration forms, so you spend more time engaging with a website and less time filling out registration pages.

Reduce Frustration Associated with Maintaining Multiple Usernames and Passwords

Most web users struggle to remember the multiple username and password combinations required to sign in to each of their favorite websites, and the password recovery process can be tedious. But using the same password at each of your favorite websites poses a security risk. With OpenID, you can use a single, existing account (from providers like Google, Yahoo, AOL or your own blog) to sign in to thousands of websites without ever needing to create another username and password. OpenID is the safer and easier method to joining new sites.

Gain Greater Control Over Your Online Identity

OpenID is a decentralized standard, meaning it is not controlled by any one website or service provider. You control how much personal information you choose to share with websites that accept OpenIDs, and multiple OpenIDs can be used for different websites or purposes. If your email (Google, Yahoo, AOL), photo stream (Flickr) or blog (Blogger, WordPress, LiveJournal) serves as your primary online presence, OpenID allows you to use that portable identity across the web.

Minimize Password Security Risks

Many web users deploy the same password across multiple websites. And since traditional passwords are not centrally administered, if a security compromise occurs at any website you use, a hacker could gain access to your password across multiple sites. With OpenID, passwords are never shared with any websites, and if a compromise does occur, you can simply change the password for your OpenID, thus immediately preventing a hacker from gaining access to your accounts at any websites you visit.Because the focus of most OpenID providers (such as Google, Yahoo and AOL) is in identity management, they can be more thorough about protecting your online identity. Most website operators are less likely to be as dedicated to protecting your identity as the OpenID providers, whose focus is on securely hosting user identities.

OAUTH API Authorization between applications

An open protocol to allow secure API authorization in a simple and standard method from desktop and web applications.

If you’re building…

  • desktop applications
  • dashboard widgets or gadgets
  • Javascript or browser-based apps
  • webpage widgets

OAuth is a simple way to publish and interact with protected data. It’s also a safer and more secure way for people to give you access.

If you’re supporting…

  • web applications
  • server-side APIs
  • mashups

If you’re storing protected data on your users’ behalf, they shouldn’t be spreading their passwords around the web to get access to it. Use OAuth to give your users access to their data while protecting their account credentials.For more on OAuth, refer http://hueniverse.com/2007/10/beginners-guide-to-oauth-part-i-overview/


Import Tool – ScRUBYt

At first I tried String Wrapper tool using open-uri

Then I understood Tree wrappers tools that the HTML document can look very good in a browser, yet still be seriously malformed (unclosed/misused tags). It is a non-trivial problem to parse such a document into a structured format like XML, since XML parsers can work with well-formed documents only

But HTree and REXML is capable to transform the input into the nicest possible XML from our point of view: a REXML Document. ( REXML is Ruby’s standard XML/XPath processing library).

After preprocessing the page content with HTree, we have to unleash the full power of XPath, which is a very powerful XML document querying language, highly suitable for web extraction.

The powerful web scrapping tools in Ruby are mainly Mechanize and Hpricot. Hpricot is “a Faster HTML Parser for Ruby” out of other Rubyful-soup(HTree + XPath),scrAPI,ARIEL

www::Mechanize has the ability to automatically navigate through Web pages as a result of interaction (filling forms etc.) while keeping cookies, automatically following redirects and simulating everything else what a real user (or the browser in response to that) would do.

Mechanize is powerful lib BUT we cannot perfectly interact with JavaScript websites. That is, it cannot handle more than one redirects through javascript.

Using mechanize, attempts to gmail webscrap to get all mails (feed exists to get new mails only) and orkut scraps extraction could go through until there is no complex javascript to break.

At first tried to do google search and extract reditt articles using almost all libs.

Then Using Xpath and scRUBYt, I am able to extract details from finance.google.com in xml format.

To a certain extent scRUBYt which is combination of Hpricot and Mechanize on steriods, seems starting step to me to get finance.google.com portfolio.

Also,scRUBYt is faster than mechanize.


require 'rubygems'
require 'scrubyt'

google_data = Scrubyt::Extractor.define do
fetch 'http://finance.google.com'
click_link 'Portfolios'
fill_textfield 'Email', '<a href="mailto:kiransoumi@gmail.com">kiransoumi@gmail.com</a>'
fill_textfield 'Passwd', '----'
fetch 'http://finance.google.com/finance/portfolio?action=view&pid=1'
click_link 'Transactions'
#Construct the wrapper
stockinfo "/html/body/div/div/table/tbody/tr" do
   symbol "/td[1]/a[1]"
   qty "/td[5]"
   price "/td[6]"


google_data.to_xml.write($stdout, 1)

[MODE] Learning
[ACTION] fetching document: <a href="http://finance.google.com">http://finance.google.com</a>
[ACTION] clicking link: Portfolios
[ACTION] fetched <a href="https://www.google.com/accounts/ServiceLogin?hl=en&service=finance&nui=1&continue=http%3A%2F%2Ffinance.google.com%3A80%2Ffinance%2Fportfolio%3Faction%3Dview">https://www.google.com/accounts/ServiceLogin?hl=en&service=finance&nui=1&continue=http%3A%2F%2Ffinance.google.com%3A80%2Ffinance%2Fportfolio%3Faction%3Dview</a>
[ACTION] typing <a href="mailto:kiransoumi@gmail.com">kiransoumi@gmail.com</a> into the textfield named 'Email'
[ACTION] typing ---- into the textfield named 'Passwd'
[ACTION] submitting form...
[ACTION] fetched <a href="https://www.google.com/accounts/CheckCookie?continue=http%3A%2F%2Ffinance.google.com%3A80%2Ffinance%2Fportfolio%3Faction%3Dview&service=finance&hl=en&chtml=LoginDoneHtml">https://www.google.com/accounts/CheckCookie?continue=http%3A%2F%2Ffinance.google.com%3A80%2Ffinance%2Fportfolio%3Faction%3Dview&service=finance&hl=en&chtml=LoginDoneHtml</a>
[ACTION] fetching document: <a href="http://finance.google.com/finance/portfolio?action=view&pid=1">http://finance.google.com/finance/portfolio?action=view&pid=1</a>
[ACTION] clicking link: Transactions
[ACTION] fetched <a href="http://finance.google.com/finance/portfolio?action=viewt&pid=1">http://finance.google.com/finance/portfolio?action=viewt&pid=1</a>
Extraction finished succesfully!

stockinfo extracted 4 instances.
symbol extracted 4 instances.
qty extracted 4 instances.
price extracted 4 instances.


CSV in Ruby

To Create CSV file in Rails:

    @models = Model.find(:all, :conditions => ['...'])
    rawdata = StringIO.new
    CSV::Writer.generate(rawdata, ',') do |csv|
      csv << %w(Title Total)
      @models.each do |model|
        csv << [model.title, model.total]

      :type => 'text/csv; charset=iso-8859-1; header=present',
      :filename => 'rawdata.csv')

To retrieve, here is the lib…

FasterCSV is intended as a
replacement to Ruby‘s standard CSV library. It was designed to
address concerns users of that library had and it has three primary goals:

  1. Be significantly faster than CSV while remaining a pure Ruby library.

  2. Use a smaller and easier to maintain code base. (FasterCSV is larger now, but
    considerably richer in features. The parsing core remains quite small.)

  3. Improve on the CSV interface.
  4. def retrieve_csv
       csv_file = params[:csv_file]
        Record.transaction do
          fastercsv = FasterCSV.new( csv_file )
          while row = fastercsv.readline
            foo, bar = row
            Record.create!( :foo => foo, :bar => bar )
        redirect_to success_action_path
        # do something with the error
        flash[:error] = "CSV import failed"
        redirect_to retrieve_path


Can Ruby Live Without Rails?

I do not know, use, or have an opinion on the Ruby language yet. But since this language climbed up to the 13th place in the Tiobe index, it deserves to be taken seriously. Bruce Tate is a well know proponent of Ruby. While some people are quick to blame any Java developer who is looking into other languages,in my opinion it’s an attitude of weak people. If we want Java keep evolving, we need to look around. I’ve asked Bruce several questions about this programming language, and this is our blitz-interview.

Why this 13-year old language was not known in the programming community until the Ruby on Rails came about?

I looked into this extensively for my Beyond Java book. Three important points in time are

1) The emergence of a language. This does not always coincide with the emergence of the catalyst.

2) The emergence of the catalyst. With Oak/Java, you could argue that this catalyst was applets in Netscape.

3) Point of no return. We’ve never seen a language emerge, wait twenty years, and then explode. At some point, we decide that we know all there is to know about a language, and then nothing can really help it. I’d put Smalltalk and Lisp into the camp of good languages that will never dominate commercially.

Ruby is a little strange for two reasons: it has no major commercial support (outside of Japan), and it stayed below the radar in Japan for such a long period of time. But the real point that you’ve got to measure is the emergence of the catalyst. Right now, the whole Ruby community–book sales, education, components–everything is wrapped up in Ruby on Rails.

I’m interested in Ruby, and a whole lot of other people are in this camp too, because it’s a dynamic language with a catalyst. Other languages have better web development experiences–Seaside on Smalltalk, for example. But Rails has traction, and the combination of productivity in a clean language with good market share is tough to beat.

Is creation of Web applications the main/only area where this language shines (I’m talking about the real-world business applications)?

Specifically, Ruby is a great language for metaprogramming. Think domain specific languages, open classes, templates, etc. Ruby is a higher level language than Java, with some functional programming tendencies, and powerful idioms that Java doesn’t yet share. A few things that I notice about Ruby are:

  • Dependency injection happens, but not through a framework. You can redefine an object or class to do method injection trivially. (The concept is called open classes in Ruby.) This capability makes testing much easier than it is in Java.

  • Ruby has a JSP-like model for substitution, and you can use it as a template for HTML/XML documents, but also for code. For example, you can have a template with variable substitution, and append that text as code to a class. This capability makes application generators almost trivial. You’ve probably seen Rails scaffolding if you’ve seen a Rails demo. But you can extend this scaffolding to build a surprisingly complete application, given a powerful enough metamodel. The streamlined project does exactly this. (streamlinedframework.org).

  • You don’t see much emphasis at all on AOP. Ruby uses the language itself to handle crosscutting concerns. Ruby 2.0 will hook before and after, and then you really won’t need AOP. Just open up the class and add interceptors in the places you need them with a handful of code.

So Ruby is a fantastic applications language. I’m doing projects now with around 150 tables, a very sophisticated web interface with tree views and AJAX on 20% of the views, and very complex business logic around rebalancing trees and optimal distribution algorithms. Ruby’s symbolic programming model makes this logic easier to reason through, and Ruby’s superior testing help me tremendously. I’d do this project with 3 times the Java programmers, and it would take a little less than twice as long. (It won’t be as fast as I’d make it with Java, but I don’t need it to be. The latency, as always, is in the database.)

But all of this flexibility comes at a cost. I can’t see Ruby as a platform for building middleware or operating systems. Enterprise programming (distributed 2pc, hard core orm) will take some time, and more investment than you see at this point. Right now, Ruby is a great applications language. I’d expect to see Ruby grow as a rich client framework. But it’s not a one-size-fits-all tool.

I’d love to see better Java/Ruby integration. I think it’s going to be important over time.

Imagine, that you have the right to add five Ruby language elements to the next version of Java. What would they be? Can you include quick code samples as well?

  1. Closures.

Closures give you the ability to pass a code block as a parameter. At times, I’m simply looking for convenience. I can use a closure to print something 10 times:

10.times {puts "Hello"}

call do_something on each item in an array (this code example uses do and end in place of {}):

array.each do |item|

build a collection containing the squares of all items in an array:

array.collect {|number| number * number}

Other times, I want to customize the inside of a block of code. For example, when you deal with a file, you must make sure you handle exceptions, and clean up resources, leading to a repetitive ugly block of code. But with closures, you can do something like this:

File.open(filename) {|f| doSomethingWithFile(f)}

The open method can take care of all of the repetitive details for you. Closures help whenever you want to deal with blocks of something. Closures are a huge win for Ruby, and the design pattern is actually used frequently in Java within frameworks like Spring.

  1. Continuations

The second feature is the continuation. Using a continuation, you can capture the state of execution within one instance variable. Think of a continuation as a save_game in an adventure game. You can pick up the game, in progress, when you restore the game. This is a code example of a continuation:

def loop
for i in 1..10
puts i
callcc {|c| return c} if i == 5

This code captures the state of the loop method at the point where i is 5. If you called the method with the command:

continuation = loop

Ruby would produce


Then, you could type


and get the result


This type of processing is extremely important for the next-generation web server, and for implementing things like virtual machines (if you use a continuation-passing style, you can easily implement virtual machines with more stack depth, lightweight threads with cooperative multiprocessing, and many other interesting algorithms. RIFE builds their own version of continuations, so some Java frameworks already need this capability.

  1. Mix-ins

Within Ruby, you don’t need AOP as frequently. You can implement something like a Java interface, but a module can give you both the implementation plus a specification. Java uses whole frameworks to give you the same capability. In the age of pojo programming, it would be incredibly useful to say I want this POJO, and mix in security, persistence, and transactions. Ruby modules, which provide mix ins, can do just that.

4.Open classes

Open classes let you open up and redefine a class in any context. You don’t have to rewrite the Integer class, for example, to add in the processing for units that an ingeger, or fixnum in Ruby, might need. You just implement methods that handle the math and the syntax you need.

For example, you could say

class Fixnum

// A fixnum represents a time in miliseconds.

def days
// convert to miliseconds
self.hours * 24

def hours
// convert to miliseconds
self.minutes * 60

def minutes
// convert to miliseconds
self.seconds * 60

def seconds
// convert to miliseconds
self * 1000

def from_now
Time.now + self

def ago
Time.now – self


Now, I can say things like 10.days.ago, and 6.hours.from_now, adding a certain richness to my domain specific languages. This capability is extremely powerful for testing also.

  1. Full object orientation

In Ruby, everything is an object. When that’s true, frameworks become much easier to consume and write, because you don’t have to deal with so many mind-numbing special cases. Autoboxing gets you closer, but not all the way. Look at the API for an array. With a similar number of methods, the Ruby array is much, much more powerful. The reason is that the Ruby array doesn’t have to deal with every special case for primitive types.

Those are a good place to start, but there are many others as well.

I make my living working as a Java architect/developer. Can you give me a good pragmatic reason to learn Ruby? Is there a job market demand for Ruby skills?

Whenever you learn a new language, it changes the way you think. The Java programmers that I know that also know Ruby don’t use as much configuration, and take better advantage of their collections than Java counterparts do. They also look for more opportunities to do metaprogramming.

But there’s certainly a demand for Ruby developers, too. My latest book, From Java to Ruby, helps managers who need to do Ruby for technical reasons, understand the political implications of such a move. I wrote the book because of an increasing demand for literature for people seeking to use the right tool for the job, rather than blindly using the most popular choice of tools. I found that Ruby development efforts are out there.

I think we’re also constantly underestimating the possibilities for integration across languages. The ReST-based web services in Rails are very rich, and the integration options for JRuby, a Ruby virtual machine implemented in Java, can blow your mind. Think of Ruby scripting in a JSP, or Ruby rules in a Java rules engine, or Ruby tests for Java code. Minimally, Ruby is a fantastic scripting language, and Java developers can take advantage of this exploding frontier.