Bundling Hardware and Software to Tackle Big Compute Problems
The New York Times highlights a big trend in data center computing: the move to bundled solutions rather than the acquisition of individual building blocks and then assembly of solutions by corporations and governments. In the February 8th 2010 article entitled Bundling Hardware and Software to Do Big Jobs, reporter Steve Lohr highlights how "Suppliers are offering customers assembled bundles of hardware and software to make it easier and less expensive for customers to cope with the Internet-era surge in data."
However, the real challenge is not just selling the software and hardware together. It is getting these large compute problems broken down so that subtasks can be processed simultaneously – a concept known as parallel computing – and hence available resources can more effectively reach at least a partial solution. As Kunle Olukotun, a computer scientist at Stanford says, "The huge challenge is to take all this data and generate useful knowledge from it. It is an enormous opportunity in science and business, but it also presents a massive computing problem."
Toomre Capital Markets LLC ("TCM") believes that the capital markets divisions of the financial world have been on the leading edge of this push to transform (on a real-time basis) data first into information and then into knowledge, simply because of the massive profits made in that sector. Wall Street has been able to throw tremendous resources at its data and compute problems. Note the many scientific and engineering graduate types who have left academia for the lure of Wall Street and high pay.
As the cost of these new compute servers come down, more and more sectors of the economy will be able to create the mathematical models that use industry specific data to assist in making more effective decisions in near real-time. Imagine being able to identify in near real-time what expense a consumer might save if he were not to use electricity just then. Other solutions will include how to allocate scarce resources ahead of a predicted major storm such as a blizzard or a hurricane. In short, what Wall Street has been doing for much of the last quarter century will shortly be migrating to Main Street in a much more intense fashion.