Application Workload Innovation

In my last post I introduced the concept of application workload evolution. As software applications become increasingly complex, they drive innovation from the underlying IT infrastructure that has to service those applications.

In this post I’d like to dive into two specific examples. The first example describes application workloads in a high-end context. The second example describes a mid-range, open-systems context.

Cache Technology

In the 1980s, storage devices (such as HDAs) were limited in their ability to keep up with application workload demand. Hard drives might take 25-30 milliseconds to respond to an application request. Larger numbers of applications were deployed onto more and more server hosts (or mainframe environments).  The ability for the storage infrastructure to satisfy the application workload increase became impossible.

At the time, customers had only one solution: buy a new system (e.g. a new mainframe) with a new storage system attached. This was too expensive but often the only choice.

As a result, EMC engineers in Hopkinton, Massachusetts developed a cached disk array known as Symmetrix. A high-level diagram of a cached approach is shown below.

From the application point of view, the “disk drive” processed application requests with an order of magnitude more speed: (e.g. 2-3 milliseconds as opposed to 20-30). In some cases, the Symmetrix product removed the need for a customer to purchase a second system. The new capability to service I/O requests from a disk array cache allowed the customer to run more applications on their existing systems than ever before.

Up The Road in Westborough

Less than ten miles away from the Symmetrix engineers in Hopkinton, the Westborough-based Data General team was also struggling to solve the “slow disk drive” problem. However, they were targeting a different application workload (which resulted in a different innovation).

At the time, UNIX systems were on the rise and mid-range database systems were becoming in vogue. Operating systems such as Solaris were out in front from a device driver perspective, forcing more and more requests down onto the disk infrastructure. Database workloads could be characterized as having more reads than writes, often by a 2-1 ratio (or higher). A cached approach would likely be too expensive for customers in the mid-range; a RAID5 approach, however, would fit the bill nicely. With a random-read-weighted workload the
striping of data across multiple disks enabled IO parallelism. This innovation
sped up the storage response time for the database workload.

At the time, RAID5 was an academic concept that proved to be highly challenging to implement, due in large part to the enormous number of failure permutations that could occur during a RAID5 write operation (sometimes referred to as a write hole).

The resulting RAID5 product implementation, initially known as HADA, and eventually better-known as CLARiiON, effectively satisfied the workload demands of typical mid-range database applications.

Innovation via Application Workload

This brings me back to the main point of the post. In the 1980s, the evolution of application workloads drove high-tech innovation. The same is still true today.  The diagram below shows a small sample of the variety of workloads that must be satisfied by today’s IT infrastructures.

Most of the historical discussion up to this point has focused on the upper portion of the Y-axis (performance). In a future post, I’ll take a look at the X-axis (service level) and relate how the service level needs of an application also play a big factor in driving innovation.

image credit: nyse.com

Wait! Before you go…

Choose how you want the latest innovation content delivered to you:

• Daily — RSS Feed — Email — Twitter — Facebook — Linkedin Today
• Weekly — Email Newsletter — Free Magazine — Linkedin Group

---

Steve Todd is an EMC Fellow, the Director of EMC’s Innovation Network, and a high-tech inventor and book author Innovate With Global Influence. An EMC Intrapreneur with over 200 patent applications and billions in product revenue, he writes about innovation on his personal blog, the Information Playground. Twitter: @SteveTodd