Continuous Integration and Automated Deployment for PHP January 16, 2009
Posted by Indraneel in Continuous Integration.Tags: Continuous Integration, Deployment
add a comment
I had set up the Continuous Integration for a few Ruby on Rails products including Workstreamr and PaidInterviews. It was easy with rake, the ci_reporter plugin, rcov and Bamboo. I also used Capistrano for automated deployments after each build. A new product development started a few days ago and I was called upon to setup the Continuous Integration platform again. It was different this time though. This new product was to be developed in PHP.
Rake is a general purpose build tool just like ant and maven. And Capistrano is the best deployment tool I have encountered so far. So the choice was easy. This is how I created a rake task to execute all the unit tests and produce test coverage reports with PHPUnit.
namespace(:build) do
basedir = "."
task :test do
Dir.foreach("test") { |filename|
begin
system("phpunit --log-xml #{basedir}/test/report/TEST-#{filename}.xml test/#{filename}") if File.ftype("test/#filename}")=="file"
rescue
end
}
end
task :report_coverage => test do
begin
system("phpunit --report #{basedir}/test/coverage test")
end
end
end
Yeah, yeah, I know, I could have done the same thing with a shell script. But when I start creating tasks for database migration and such, things are going to get messy and tracking dependencies wouldn’t be easy with shell script.
PHPUnit can also produce coverage reports. So I included the second task which would produce coverage reports.
Now for the deployment script.
set :application, "appname"
set :scm_username, 'myname'
set :scm_password, 'secret'
set :synchronous_connect, true
set :scm, :subversion
task :remote do
set :deploy_to, "/you/app/path/#{application}"
set :command_path, "/usr/bin"
set :repository, "http://your/subversion/project/appname/trunk/"
set :user, 'username'
set :password, 'secret'
set :app_server, "my.app.server"
role :app, "#{app_server}"
role :web, "#{app_server}"
role :db, "#{app_server}", :primary => true
set :runner, user
set :use_sudo, true
end
namespace(:deploy) do
task :start, :roles => :app do
sudo "apache2ctl -k start"
end
task :stop, :roles => :app do
sudo "apache2ctl -k stop"
end
task :restart, :roles => :app do
sudo "apache2ctl -k graceful"
end
end
And now I bundle all this up neatly in a shell script and give it to Bamboo. Here is what my build-deploy.sh script looks like:
#/bin/bash
rake build:report_coverage
cap remote deploy
The PC Quest article January 16, 2009
Posted by Indraneel in Personal.add a comment
Some of my comments appeared in the January 2009 edition of PC Quest. The online version of it is available over here.
Cloud Computing – Large scale computing for everyone November 2, 2008
Posted by Indraneel in Product Development.Tags: cloud computing, product deployment
add a comment
In the beginning of 2008, New York Times ingested 405,000 very large TIFF images, 3.3 million articles in SGML and 405,000 xml files mapping articles to rectangular regions in the TIFF’s using Amazon Web Services, Hadoop and some custom code. This data was converted to a more web-friendly 810,000 PNG images and 405,000 Javascript files containing JSON in less than 36 hours.
Why is this a big deal?
NASA has been computing on far greater scale than this for a long time. NASA’s weather simulation software is computationally a lot more complex than indexing documents.
So why is this a big deal? It’s a big deal because it was neither NASA nor CERN, not even Google. It was a business who did this without buying a single machine. They rented computing power on the fly. They rented slices of a cloud.
What is cloud computing?
A few days ago a journalist said “There is clear consensus that there is no consensus on what cloud computing is”. I like to think of cloud computing as the commercialization of computing resources like CPU cycles, storage, memory etc just like public utilities like electricity, water or natural gas. At the very core of the cloud is virtualization. Virtualization is a technique in which software is used to completely simulate or emulate hardware.
Types of clouds
I see two distinct categories of clouds that vendors are selling today:
- Infrastructure as a Service – IaaS vendors sell raw compute power – CPU cycles, memory, bandwidth etc. IaaS clouds are complex but with the complexity comes flexibility. Most cloud vendors allow root access to an instance. And hence, specialized knowledge is necessary to handle such flexibility.
- Platform as a service – PaaS refers to those clouds which provide frameworks and infrastructure on which users can build applications. PaaS clouds are built on IaaS clouds. Most PaaS clouds are very restrictive. They generally allow users to build applications on a particular set or sets of technologies. For example Google App Engine allows users to build applications using Python only. Portability is an inherent issue with PaaS clouds, because of the lack of standards in this domain. So if you have built an application using Google App Engine and BigTable you probably won’t be ableto port the data to any other cloud without spending a huge amount of time and money.
Inside the cloud
At a very high level clouds are made up of the following layers:
Inside the cloud
- At the very bottom is the hardware layer. Many cloud vendors build their clouds out of of the shelf server class software. For example Joyent uses Dell servers with quad core intel processors for their cloud. Plumbing refers to the networking elements in the cloud with all the fast router, switches and load balancers connected by fiber optic cabling. Clusters, made up of ordinary server class machines make up the skeleton of the cloud.
- Storage services refer the storage provided by the cloud. Most cloud vendors offer SAN or NAS storage. Provisioning is generally on the fly and users can ask for virtually unlimited amount of storage.
- As mentioned earlier, virtualization is at the very core of the cloud. Virtualization has made creation of a software machine as a clone of an existing one super fast. Think of the cluster (mentioned earlier) as one mega machine with one host OS managing all its resources. Creating virtual machines with pre-defined CPU and memory is fast and easy. Many vendors like Amazon Web Services use Xen virtualization.
- Platform services are bunch of pre-installed and packaged goodies that an user of the cloud gets whenever an instance of the cloud is brought up. The LAMP stack supported by AWS and Joyent is an example of platform services.
- No matter what the vendor says, if it takes more than 10-15 minutes to bring up an instance, then it is not a cloud. The web services layer is the one that enables users to templatize an instance, bring up a new instance from a template, take backups, restore from a backup etc. instantly, as and when needed.
Just in time deployment using the cloud
Deployment of products is messy business. Not so long ago, fledgling organizations had to first calculate the amount of computing resources needed for a launch, translate that into hardware requirements, call up the hardware vendors or the hosting company and wait till they provisioned the hardware and then installed and configured the software. Provisioning, installation and configuration took several days. It was a lose-lose scenario for everyone. Product success meant another cycle of calls and provisioning while the users suffered due to unresponsive software caused due to heavy load. Product failure meant huge losses due to unused hardware.
Not any more with the advent of the cloud. Now product launches can happen at the click of a button with just enough computing resources sitting behind a Virtual IP. The utilization of the resources are closely monitored. New, templatized instances of the cloud are instantiated whenever the threshold for the monitored utilization is reached. The users of the cloud pay for what they use at any instant of time. The users of the product never find it unresponsive, since computing resources are always adequate, just in time to meet the users’ needs.
Large scale computing for all
So long the ability to do large scale computing was within the reach of an elite club of businesses. Google, Amazon and Yahoo were amongst the very few in the club. Few businesses had the means to lay their hands on infrastructure of that scale. Cloud computing has changed that. Today, a ‘large’ Amazon EC2 instance with 4 EC2 compute units (which is equivalent to the capacity of 4 Opteron or Xeon processors) and 7.5 Gigs of memory costs as less as $288 per month. Users can choose from quite a few operating systems and scale up and down on the fly. Application development platforms like JBoss Enterprise Application Platform and Ruby on Rails come built into it. Clouds have opened the doors of large scale computing to virtually everyone.
Back to blogosphere November 2, 2008
Posted by Indraneel in Personal.Tags: Personal
add a comment
September was a crazy month. PaidInterviews got launched at DEMO and I worked really very hard. By the end of September, things eased out. I got some breathing room. The financial crisis had changed the world by then. I took a couple of vacations for Durga Puja and Dewali. And oh boy, did I need them! I’m well rested and focused now. I’ll post a few things very soon.
CubicTest – A UI test automation tool like no other September 4, 2008
Posted by Indraneel in Software Quality.Tags: Test Automation, Testing
2 comments
I was just trying out CubicTest today. It is a really awesome tool for automation of tests through UI for Web Applications. It is implemented as a graphical plugin for eclipse. You can export the scripts to selenium core (HTML) or Watir scripts. And when you export the scripts the quality of the scripts are not like the XML generated by MS Word, they are good quality scripts. The tests can be run via the command line, thus making it easy to run in a continuous integration environment. I have seen dozens of UI test automation tools. I have used Selenium and Watir for at least a dozen products. But this tool is like no other. The only thing that I found was missing in the tool is the inherent support for a Test Object Repository. But that’s not a big deterrent. I am definitely going to use it for the new product we have started building.
Installing ffmpeg on OpenSolaris September 1, 2008
Posted by Indraneel in Ruby on Rails.Tags: ffmpeg, OpenSolaris
3 comments
We have been using Joyent accelerators to host Alpha and Beta versions of some of our products. One of the products that we are building needed ffmpeg to convert videos to flv format for streaming. Joyent accelerators use OpenSolaris 5.11. I thought I would just get ffmpeg and install it from source. “make” went along fine for sometime but bombed with the following error:
In file included from /usr/include/sys/int_types.h:55,
from /usr/include/sys/stdint.h:38,
from /usr/include/stdint.h:38,
from ./libavcodec/bitstream.h:29,
from libavformat/rtpdec.c:25:
/usr/include/sys/feature_tests.h:353:2:
#error "Compiler or options invalid for pre-UNIX 03 X/Open applications and pre-2001 POSIX applications"
So I opened up “configure” and saw the following c99 directives:
check_cflags -std=c99 and
add_cflags -D_ISOC99_SOURCE -D_POSIX_C_SOURCE=200112
I look inside the file rtpdec.c inside the directory libavformat and I find this:
#define _XOPEN_SOURCE 500
This clearly conflicts with the “-std=c99″ settings.
So I change it to
#define _XOPEN_SOURCE 600
Two other files utils.c inside libavcodec directory and ffmpeg.c had the same problem. I applied the same fix there.
It compiled fine thereafter and that saved my day.
How to avoid getting “Flagged as Spam” while sending legitimate emails July 15, 2008
Posted by Indraneel in Product Development.Tags: Product Development
4 comments
The menace called SPAM is a double edged sword. On one hand we have to constantly fight to keep SPAM from reaching our mailboxes. On the other hand we have to be careful so that the legitimate emails that we send don’t end up being caught in the net of anti-spam software. Take the example of sending automated emails to people who sign up for the beta version of an exciting product that we are building. These emails are not unsolicited. The ultimate control for the delivery of such emails reside with the servers receiving the emails and the anti-spam policies and software they implement. However following the guidelines below will reduce the chances of getting flagged by anti-spam software drastically.
I. Don’t spoof your identity.
Be accurate in who you are and from where you are sending the email. It’s always good to send your emails from your own servers, using your own domain name. For example if you have an application running on a machine in the domain example.com and you are sending emails from a mail server in that domain, it’s preferable that the senders address be someone@example.com. If you try to hide your source and destination you’ll look like spam. Don’t’ add unnecessary headers to the emails.
The email with the following header went to the junk email folder.
Received: by *ip-xxx-xxx-171-173.ip.xxxxserver.net* (Postfix, from userid 99)
id 7C95F298101; Wed, 9 Jul 2008 05:16:24 \-0700 (MST)
Received: from ip.secureserver.net (ip-xxx-xxx-171-173.ip.xxxxserver.net [127.0.0.1])
by ip-xxx-xxx-171-173.ip.secureserver.net (Postfix) with ESMTP id 6385E2980F1
...
Date: Wed, 9 Jul 2008 05:16:23 \-0700
From: *admin@somedomain.com*
The domain names of the “Received: ” and “From:” fields don’t match.
II. Genuine domain names
Use a domain name which is identified by a verifiable IP address. For this reason, it is very important to have rDNS (Reverse DNS) entry, also known as a PTR record for the server from which you are sending emails. Most anti-spam software reject emails sent from servers that don’t have an rDNS entry.
The email with the following header went to the junk email folder.
Received: from *unknown* (HELO ip-xxx-xxx-171-173.ip.xxxxserver.net) (208.109.171.173)
by k2smtpout06-01.prod.xxxx.xxxxserver.net (xx.xx.189.102) with ESMTP; 08 Jul 2008 06:34:42 \-0000
“unknown” in the “Received” header means the receiving server could not determine the identity of the server sending the email which generally turns out to be the lack of an rDNS entry for the sending server.
III. Send well constructed emails
Emails with missing mime sections, invalid or missing message-ids, invalid or missing date headers, or subject etc., are frequently signs of spam.
IV. Encodings
Avoid needless encodings and charsets in the emails. Don’t use base-64 encoding unless you really need to.
Consider the Subject field in the header below:
Subject: =?iso-8859-2?B?U1BBTTpSZWdhaW4geW91ciBuYXR1cmFsIHdlbA==?=
=?iso-8859-2?B?bG5lc3M=?=
Content-Type: text/plain;
charset="iso-8859-2"
The character set in this email is ISO-8859-2 which is the unicode encoding that a lot of eastern European countries like Hungary, Poland et al. use. This message ended up in my junk mail folder.
The following is a part of the header which ended up in my junk mail:
X-OriginalArrivalTime: 13 Jun 2008 13:52:08.0648 (UTC) FILETIME=[AC00D480:01C8CD5C]
\--Apple-Mail-16-982198482
Content-Disposition: inline;
filename="Picture 6.png"
Content-Type: image/png;
x-mac-hide-extension=yes;
x-unix-mode=0644;
name="Picture 6.png"
Content-Transfer-Encoding: base64
V. HTML emails
Malformed HTML
If you’re using HTML emails then the least you can do is to make sure that the HTML is valid. Unbalanced and invalid tags are bound to flag an email as spam.
Invisible text in HTML
If you’re using HTML emails, do not use invisible text within those emails. Make sure your text colors and sizes are distinct enough and large enough to read. Invisible text (e.g – text color is the same as background color) is often identified as a sign of spam.
Consider the following logo in a HTML email:
Welcome Everyone
The following is the source for this:
< p style="color:#4d4d4d;font-family:Arial,Helvetica,Verdana,sans-serif;font-size:24px;font-weight:bold;margin:0;padding:5px 0 0 5px;">Welcome<span style="color:#808080;"> Everyone</span></p>
This is good and valid. However the following is not:
Since it’s some text disguised as an image.
VI. Keep it simple
Do not use cute spellings, Don’t space out your words, don’t put str@nge l3tters 0r characters into your emails. You are bound to look as spam if you do.
For example some people emphasize/stylize the text by writing:
L E G I T I M A T E .
Text like such will make the email likely to get caught in spam filtering.
An Agile way of reporting bugs July 6, 2008
Posted by Indraneel in Software Quality.Tags: Agile, Testing
1 comment so far
I have seen a lot of times that when I report bugs for a product that we are building, the developers don’t really understand the problem. Some bugs are so subtle and some so complicated that it takes an essay and half a dozen annotated screenshots to cover them. And even then I get “So what’s the problem, I have no idea what you’re talking about” from developers. Reporting bugs clearly, so that the person who’s supposed to fix it understands it completely without doubts is not trivial. So I resorted to a more vivid representation – short movies with live screen-captures and voice. And suddenly the daunting task of writing an essay and seemingly endless capturing of screenshots and annotating them vanished. If you haven’t guessed it yet, I use Jing http://www.jingproject.com to do just that. Jing sits as an icon atop all windows and it takes just two clicks to start recording the screen. It produces an .swf file – occupying as less space as possible. I promptly attach it to the issue I create in the bug-tracker.
I have found remarkable success with this. The developers are happy and so am I.
Selenium on Rails breaks on Rails 2.1 July 4, 2008
Posted by Indraneel in Ruby on Rails.Tags: Rails, Testing
8 comments
We upgraded ourselves to Rails 2.1. And builds started failing. Bamboo showed me a “undefined method – register_template_handler” error.
.../vendor/ plugins/selenium-on-rails/lib/selenium_on_rails/selenese.rb:3: undefined method `register_template_handler' for ActionView::Base:Class (NoMethodError)
It happened for rselenese.rb too.
It seems they moved “register_template_handler” from ActionView::Base to ActionView::Template in Rails 2.1. So I went in and changed
ActionView::Base.register_template_handler 'sel', SeleniumOnRails::Selenese
To
ActionView::Template.register_template_handler 'sel', SeleniumOnRails::Selenese
And bingo! it worked fine as it was working earlier.
Winning strategies June 30, 2008
Posted by Indraneel in Product Development.Tags: Agile, Product Development
1 comment so far
I’ve been serving in the Agile army for sometime now. Being a foot-soldier I’ve been pretty close to ground-zero and hence to the ground realities.
Being Agile is hard. It may sound a little odd, but it is true.
Discipline
Being Agile requires quite a bit of discipline. In fact success with Agile depends on it. We were developing this shiny new product. For quite some time during one iteration our builds were failing continuously. Though we had a continuous integration platform, we lacked the discipline to make sure that builds passed everyday. Eventually when the build statistics showed us all reds for about 2 weeks continuously, we kinda woke up to it. It took us a few days to figure out what were actually wrong with the builds. Had we acted upon it the very first day the builds failed, it would have taken us a lot less time to figure out what was wrong. Moral of the story – “Having a kickass tool or platform does not mean we follow Agile methodology. The entire team has to make sure that the tools are used in the everyday life of product development. And that, requires discipline“
Planning
Being Agile does not mean that we try to build an aircraft today, a wicker basket tomorrow and a flower vase the day after. Agile does not mean ‘no planning’. In fact Agile is a lot about planning.
Which bring me to iteration or sprint planning. A functionality freeze at the beginning of an iteration or sprint is not a “nice to have“. It’s a necessity for delivering a “quality” product “on time“. In one of the products that we were developing, the product manager saw the need to modify the functionality in the middle of the iteration. He had valid business reasons to do that, but that iteration was a nightmare. The quality of the product suffered badly, the developers were stressed and stretched to their extremes and the engineering manager spent sleepless nights. Unless billions are at stake, I don’t recommend doing it ever.
Design
There is no substitute for good design.
Bad Design + Flawless code = Bad product. Nobody would buy a machine which has a super fast processor but can only be started after opening the casing and finding the right wires to connect.
Excellent Design + Bad Code = Still a bad product but it’s relatively easy to fix it. Design flaws are very very expensive to fix. The chances of getting a better product increases geometrically when the design is given the enough amount of time to iterate and mature.
Teamwork
The whole team needs to have a holistic view of the product. People working in silos with blinders on don’t make a good team and are usually not at their productive best. This is a issue of epidemic proportions. I have seen it in mom-and-pop software shops as well as in Fortune 100 organizations. Good software requires more than collaboration. Each member of the team needs to understand and appreciate the big picture of the product, the purpose the product will serve and the kind of users who are likely to use the product . More often than not, engineers underestimate, misunderstand or simply ignore the value of having this holistic view. It’s not enough if only the functionality is implemented right.
Cross functional skills in the team are the need of the day. A job done by keeping in mind that one’s output will become someone else’s input is a job done well. A UI designer who knows a little about coding will be able to design the UI in a way which can make coding a breeze. An eye for detail can save a lot of time by
cutting down on rework. A developer who keeps an eye open for obvious design or UI errors can reduce rework manifolds. For example if the input boxes in a web-page are not properly aligned the developer should contact the UI developer or the designer and get it resolved before starting to code. It would take quite some rework if the non-aligned imput boxes made it to a QA build.
In the end, the success of a product depends on a lot of factors. Some of these we don’t have control over. But we gotta do our best with the factors we can control.
This is the first post in this series. There is more to come.
