The broken development process and "The Golden Rule"

In many of the performance issues I help resolve there is strong resistance from developers and architects, in some ways a disbelief that their application could be anything other than perfect and performant.

Digging a little deeper, identical patterns of development appear. Put bluntly there is no attention paid to operational price/performance, huge assumptions are made and in many cases there is blind hope coupled with a certain arrogance.

"But it works fine on my PC" is a common chant, this little example of failed logic is indicative of the broken Java development process. Naturally it worked fine on a powerful development machine at 1'ish transactions per second, with no concurrency and trivial data volumes, it was bound to as developers unconsciously adapt to make it so. However this has no relation to production platforms, transactions rates and data volumes.

The golden rule is to consider "non-functional" performance requirements from the very beginning, in the design phase, and to carry these through development with constant measurement and refinement. Involving management, reporting on price/performance on an daily basis as the application takes shape, making technology and architecture choices visible and deciding on their value using empirical data as well as subjective "attractiveness".

It's not all about developer productivity (which is never measured BTW).

This "feedback loop" is in fact identical in nature to the unconscious tuning that happens in a developers personal "space", their development workstation, but broadened to encompass real volumetrics and real deployment platforms.

I plan to go into much more detail about this, but for 2007, remember this Golden Rule, take responsibility for performance, set goals and constantly measure.

We are approaching a time where throwing hardware at an issue will increasingly run out of steam, planning for operational performance helps delivery times and reduces costs for very little investment.

It's not 'your' code....

In many cases over 90% of the code running in a modern corporate Java application is not actually written by the principal developers but is rather "included" as part of containers vendor products, and OSS libraries.

This fact is directly related to the inability of many developers to understand and resolve poor performance.

Nobody can cold read this volume of code, it's impact with real data volumes at production transactions rates in a certain "(ab)use case" cannot be predicted theoretically.

The implicit trust we give to this code is quite amazing, if I ever get time I recount my .circumventBug2650 story.

Popular performance and profiling tools do a very bad job of exposing this behavior and promoting the understanding of this silent 90% majority. The traditional "plus.. plus... plus..." tree view cannot cope with the depth or the scale.

So when it comes to poor application performance it may not actually be our fault but rather a cumulative problem of layering other peoples code.

OK, it's time to "Call" it.

After seven years dealing with hundreds of customer Java performance issues for one of the industry's largest players I've decided to start a blog.

I think the name of the blog says it all.

The is no question that Java (read J2EE and the rest of the industry) has brought us significant improvements in quality, reach and integration, but like much else in life it is a transitional cycle.

It depends on your age, but most now see the cyclical nature of the IT industry, in fact many argue a more sophisticated genetic nature, whatever the model, most agree on an increasing frequency.

I do not disagree with any of these positions, they all match, however I believe the corporate reality, where most development is done, is far from the ideal.

The effort, time and money wasted daily in the industry is quite astounding. Many of the numerous voices and opinions that most you suffer are uninformed and selfish. There is so much flux, so much self supporting change, a job like mine gives you an interesting angle on this.

This is not a popular view at the moment, but it is realistic. Java is wonderful, and it rapidly became an industry, I was there. It developed a behavior of change for changes sake, because...... "change is money", "change is the only constant".... pick your quote, but realize that you are at the wrong end on it

I admit I'm a little jaded, but I continue to try... and I succeed..... but it is unsatisfying..... where I want to see genetic or exponential development I just see cycles, not even cycles in performance, technology or patterns but rather cycles in behavior.

This blog will have a humble cause, Java performance, but may at times appear rather strident, for this I apologize in advance. Throughout, I implore the reader to, stop, look and listen (to quote an icon). Think about your business and the wasted effort, time and money, think about your career, your need to rise above this technical malaise, the conflict and the lost opportunity.

Now on to the detail and the passion...... and more use of the words we and us rather than I.