Storage systems and processing platforms have similar histories. Both started as single large monoliths, but both have moved strongly to parallelism to take advantage of the volume economies introduced by the personal computer industry. Large attached storage systems now have thousands of individual spindles; Top500 computing platforms increasingly have thousands of individual processors. The challenge for system architects, of course, is to maximize the parallelism between those thousands of spindles and those thousands of processors, whilst assuring reliability when one of either fails.
Read more...Every HPC application faces the temptation to employ heroic measures in order to eke out performance. A common example is the use of local disks on conventional cluster nodes. Used in parallel, they offer the potential of very high aggregate bandwidth, but at the expense of specialized programming and uncertain reliability.
Read more...