New Linux Cluster

All the hype about bigdata is turning into some great new tools, dontcha think?

I have to admit that I've become more of a Microsoft fan than I ever thought I would. SQL Server 2012 was released today, and boy oh boy, the Reporting/Visualization tool they have makes a very compelling argument for using it.

Surprising.

Anyway, bigdata bigdata bigdata - I have to do something. So I'm learning new clustering tools first. I have build the cluster before I have something that can host hadoop.

All of the tools that I have used in the past are defunct. And I am now an ubuntu system admin at home. So it seemed obvious to choose the Ubuntu route.

There is a tool called Orchestra, which not only allows you to automate the entire system management process, but handles automated provisioning and installation as well. So it seemed like an obvious choice. In fact, Orchestra slid on to my master server in just about 10 minutes. So that was great. Within a few days I had provisioning working!

Now I'm at the point where its time to set up the environment for each node and install the supporting software packages. this means learning another new tool: JuJu.

To make things even worse though, I've been keeping my mind open as to the solution to use here, and there's an amazon EC2 solution from MIT called StarCluster that automates provisioning a cluster on the Amazon EC2 system.

I looked at that, but I didn't want to pay for Amazon to host what I wanted to build.

At this point I'm following along with the crew of people that are using the Orchestra toolchain, and trying to learn what I need to get thsi project booted.

More later.