Back to blogosphere

November 2, 2008 Indraneel Leave a comment

September was a crazy month. PaidInterviews got launched at DEMO and I worked really very hard. By the end of September, things eased out. I got some breathing room. The financial crisis had changed the world by then. I took a couple of vacations for Durga Puja and Dewali. And oh boy, did I need them! I’m well rested and focused now. I’ll post a few things very soon.

Categories: Product Engineering Tags:

CubicTest – A UI test automation tool like no other

September 4, 2008 Indraneel 2 comments

I was just trying out CubicTest today. It is a really awesome tool for automation of tests through UI for Web Applications. It is implemented as a graphical plugin for eclipse. You can export the scripts to selenium core (HTML) or Watir scripts. And when you export the scripts the quality of the scripts are not like the XML generated by MS Word, they are good quality scripts. The tests can be run via the command line, thus making it easy to run in a continuous integration environment. I have seen dozens of UI test automation tools. I have used Selenium and Watir for at least a dozen products. But this tool is like no other. The only thing that I found was missing in the tool is the inherent support for a Test Object Repository. But that’s not a big deterrent. I am definitely going to use it for the new product we have started building.

Installing ffmpeg on OpenSolaris

September 1, 2008 Indraneel 3 comments

We have been using Joyent accelerators to host Alpha and Beta versions of some of our products. One of the products that we are building needed ffmpeg to convert videos to flv format for streaming. Joyent accelerators use OpenSolaris 5.11. I thought I would just get ffmpeg and install it from source. “make” went along fine for sometime but bombed with the following error:

In file included from /usr/include/sys/int_types.h:55,
from /usr/include/sys/stdint.h:38,
from /usr/include/stdint.h:38,
from ./libavcodec/bitstream.h:29,
from libavformat/rtpdec.c:25:
/usr/include/sys/feature_tests.h:353:2:
#error "Compiler or options invalid for pre-UNIX 03 X/Open applications and pre-2001 POSIX applications"

So I opened up “configure” and saw the following c99 directives:
check_cflags -std=c99 and
add_cflags -D_ISOC99_SOURCE -D_POSIX_C_SOURCE=200112

I look inside the file rtpdec.c inside the directory libavformat and I find this:
#define _XOPEN_SOURCE 500
This clearly conflicts with the “-std=c99″ settings.
So I change it to
#define _XOPEN_SOURCE 600
Two other files utils.c inside libavcodec directory and ffmpeg.c had the same problem. I applied the same fix there.
It compiled fine thereafter and that saved my day.

Categories: Ruby on Rails Tags: ,

How to avoid getting “Flagged as Spam” while sending legitimate emails

July 15, 2008 Indraneel 4 comments

The menace called SPAM is a double edged sword. On one hand we have to constantly fight to keep SPAM from reaching our mailboxes. On the other hand we have to be careful so that the legitimate emails that we send don’t end up being caught in the net of anti-spam software. Take the example of sending automated emails to people who sign up for the beta version of an exciting product that we are building. These emails are not unsolicited. The ultimate control for the delivery of such emails reside with the servers receiving the emails and the anti-spam policies and software they implement. However following the guidelines below will reduce the chances of getting flagged by anti-spam software drastically.

I. Don’t spoof your identity.

Be accurate in who you are and from where you are sending the email. It’s always good to send your emails from your own servers, using your own domain name. For example if you have an application running on a machine in the domain example.com and you are sending emails from a mail server in that domain, it’s preferable that the senders address be someone@example.com. If you try to hide your source and destination you’ll look like spam. Don’t’ add unnecessary headers to the emails.
The email with the following header went to the junk email folder.


Received: by *ip-xxx-xxx-171-173.ip.xxxxserver.net* (Postfix, from userid 99)
id 7C95F298101; Wed, 9 Jul 2008 05:16:24 \-0700 (MST)
Received: from ip.secureserver.net (ip-xxx-xxx-171-173.ip.xxxxserver.net [127.0.0.1])
by ip-xxx-xxx-171-173.ip.secureserver.net (Postfix) with ESMTP id 6385E2980F1
...
Date: Wed, 9 Jul 2008 05:16:23 \-0700
From: *admin@somedomain.com*

The domain names of the “Received: ” and “From:” fields don’t match.

II. Genuine domain names

Use a domain name which is identified by a verifiable IP address. For this reason, it is very important to have rDNS (Reverse DNS) entry, also known as a PTR record for the server from which you are sending emails. Most anti-spam software reject emails sent from servers that don’t have an rDNS entry.

The email with the following header went to the junk email folder.

Received: from *unknown* (HELO ip-xxx-xxx-171-173.ip.xxxxserver.net) (208.109.171.173)
by k2smtpout06-01.prod.xxxx.xxxxserver.net (xx.xx.189.102) with ESMTP; 08 Jul 2008 06:34:42 \-0000

“unknown” in the “Received” header means the receiving server could not determine the identity of the server sending the email which generally turns out to be the lack of an rDNS entry for the sending server.

III. Send well constructed emails

Emails with missing mime sections, invalid or missing message-ids, invalid or missing date headers, or subject etc., are frequently signs of spam.

IV. Encodings

Avoid needless encodings and charsets in the emails. Don’t use base-64 encoding unless you really need to.

Consider the Subject field in the header below:
Subject: =?iso-8859-2?B?U1BBTTpSZWdhaW4geW91ciBuYXR1cmFsIHdlbA==?=
=?iso-8859-2?B?bG5lc3M=?=
Content-Type: text/plain;
charset="iso-8859-2"

The character set in this email is ISO-8859-2 which is the unicode encoding that a lot of eastern European countries like Hungary, Poland et al. use. This message ended up in my junk mail folder.

The following is a part of the header which ended up in my junk mail:

X-OriginalArrivalTime: 13 Jun 2008 13:52:08.0648 (UTC) FILETIME=[AC00D480:01C8CD5C]
\--Apple-Mail-16-982198482
Content-Disposition: inline;
filename="Picture 6.png"
Content-Type: image/png;
x-mac-hide-extension=yes;
x-unix-mode=0644;
name="Picture 6.png"
Content-Transfer-Encoding: base64

V. HTML emails

Malformed HTML

If you’re using HTML emails then the least you can do is to make sure that the HTML is valid. Unbalanced and invalid tags are bound to flag an email as spam.

Invisible text in HTML

If you’re using HTML emails, do not use invisible text within those emails. Make sure your text colors and sizes are distinct enough and large enough to read. Invisible text (e.g – text color is the same as background color) is often identified as a sign of spam.

Consider the following logo in a HTML email:

Welcome Everyone

The following is the source for this:
< p style="color:#4d4d4d;font-family:Arial,Helvetica,Verdana,sans-serif;font-size:24px;font-weight:bold;margin:0;padding:5px 0 0 5px;">Welcome<span style="color:#808080;"> Everyone</span></p>

This is good and valid. However the following is not:

Since it’s some text disguised as an image.

VI. Keep it simple

Do not use cute spellings, Don’t space out your words, don’t put str@nge l3tters 0r characters into your emails. You are bound to look as spam if you do.

For example some people emphasize/stylize the text by writing:
L E G I T I M A T E .
Text like such will make the email likely to get caught in spam filtering.

If you found this post helpful, there is some more stuff on the same topic in a later post of mine over here

An Agile way of reporting bugs

July 6, 2008 Indraneel 1 comment

I have seen a lot of times that when I report bugs for a product that we are building, the developers don’t really understand the problem. Some bugs are so subtle and some so complicated that it takes an essay and half a dozen annotated screenshots to cover them. And even then I get “So what’s the problem, I have no idea what you’re talking about” from developers. Reporting bugs clearly, so that the person who’s supposed to fix it understands it completely without doubts is not trivial. So I resorted to a more vivid representation – short movies with live screen-captures and voice. And suddenly the daunting task of writing an essay and seemingly endless capturing of screenshots and annotating them vanished. If you haven’t guessed it yet, I use Jing http://www.jingproject.com to do just that. Jing sits as an icon atop all windows and it takes just two clicks to start recording the screen. It produces an .swf file – occupying as less space as possible. I promptly attach it to the issue I create in the bug-tracker.
I have found remarkable success with this. The developers are happy and so am I.

Categories: Software Quality Tags: ,

Selenium on Rails breaks on Rails 2.1

July 4, 2008 Indraneel 9 comments

We upgraded ourselves to Rails 2.1. And builds started failing. Bamboo showed me a “undefined method – register_template_handler” error.

.../​vendor/​ plugins/​selenium-on-rails/​lib/​selenium_on_rails/​selenese.rb:3: undefined method `register_template_handler' for ActionView::Base:Class (NoMethodError)

It happened for rselenese.rb too.
It seems they moved “register_template_handler” from ActionView::Base to ActionView::Template in Rails 2.1. So I went in and changed

ActionView::Base.register_template_handler 'sel', SeleniumOnRails::Selenese
To
ActionView::Template.register_template_handler 'sel', SeleniumOnRails::Selenese

And bingo! it worked fine as it was working earlier.

Categories: Ruby on Rails Tags:

Winning strategies

June 30, 2008 Indraneel 1 comment

I’ve been serving in the Agile army for sometime now. Being a foot-soldier I’ve been pretty close to ground-zero and hence to the ground realities.

Being Agile is hard. It may sound a little odd, but it is true.

Discipline

Being Agile requires quite a bit of discipline. In fact success with Agile depends on it. We were developing this shiny new product. For quite some time during one iteration our builds were failing continuously. Though we had a continuous integration platform, we lacked the discipline to make sure that builds passed everyday. Eventually when the build statistics showed us all reds for about 2 weeks continuously, we kinda woke up to it. It took us a few days to figure out what were actually wrong with the builds. Had we acted upon it the very first day the builds failed, it would have taken us a lot less time to figure out what was wrong. Moral of the story – “Having a kickass tool or platform does not mean we follow Agile methodology. The entire team has to make sure that the tools are used in the everyday life of product development. And that, requires discipline

Planning

Being Agile does not mean that we try to build an aircraft today, a wicker basket tomorrow and a flower vase the day after. Agile does not mean ‘no planning’. In fact Agile is a lot about planning.

Which bring me to iteration or sprint planning. A functionality freeze at the beginning of an iteration or sprint is not a “nice to have“. It’s a necessity for delivering a “quality” product “on time“. In one of the products that we were developing, the product manager saw the need to modify the functionality in the middle of the iteration. He had valid business reasons to do that, but that iteration was a nightmare. The quality of the product suffered badly, the developers were stressed and stretched to their extremes and the engineering manager spent sleepless nights. Unless billions are at stake, I don’t recommend doing it ever.

Design

There is no substitute for good design.
Bad Design + Flawless code = Bad product. Nobody would buy a machine which has a super fast processor but can only be started after opening the casing and finding the right wires to connect.
Excellent Design + Bad Code = Still a bad product but it’s relatively easy to fix it. Design flaws are very very expensive to fix. The chances of getting a better product increases geometrically when the design is given the enough amount of time to iterate and mature.

Teamwork

The whole team needs to have a holistic view of the product. People working in silos with blinders on don’t make a good team and are usually not at their productive best. This is a issue of epidemic proportions. I have seen it in mom-and-pop software shops as well as in Fortune 100 organizations. Good software requires more than collaboration. Each member of the team needs to understand and appreciate the big picture of the product, the purpose the product will serve and the kind of users who are likely to use the product . More often than not, engineers underestimate, misunderstand or simply ignore the value of having this holistic view. It’s not enough if only the functionality is implemented right.

Cross functional skills in the team are the need of the day. A job done by keeping in mind that one’s output will become someone else’s input is a job done well. A UI designer who knows a little about coding will be able to design the UI in a way which can make coding a breeze. An eye for detail can save a lot of time by
cutting down on rework. A developer who keeps an eye open for obvious design or UI errors can reduce rework manifolds. For example if the input boxes in a web-page are not properly aligned the developer should contact the UI developer or the designer and get it resolved before starting to code. It would take quite some rework if the non-aligned imput boxes made it to a QA build.

In the end, the success of a product depends on a lot of factors. Some of these we don’t have control over. But we gotta do our best with the factors we can control.

This is the first post in this series. There is more to come.