The Poor (Broke) Man's Java Profiler

When I talk to customers about big performance issues they are experiencing in their applications I usually mention "The Poor Man's Profiler". Actually, when I use this phrase, I am talking about Java core dumps. Java comes with a cool feature where you can send a break signal to the Java process and get a core dumped to the working directory of the application. In that core, if you search for "Full thread dump", you will see what is happening in all the threads of the application. If you do that a few times during load, you can very poorly approximate a Java profiler. If you have an application that won't scale or a Java process that is taking minutes to complete an operation, its amazing how often you can find the issue with Java cores.

Here is how you do it.

1. Get the application running under load (or if its a single threaded long running operation, just let the operation get started).


2. Send a BREAK signal to the Java process. In Unix/Linux, you can do this with kill -3 ProccessId. In Windows, you can do this by using a program called SendSignal (Note I don't endorse this tool, but happen to find it useful).
   
   
   You'll see the process start to consume exactly one CPU in the system (on a uniprocessor box, 100% CPU, on a 4-way box, 25% CPU, for example). During this time the JVM is doing its job collecting up all the information for the core. At the end it will dump the core to the working directory of the application. Once it is done (could take some time) the CPU will go back to normal.


3. Let the process continue for a little while, so the application gets back into its normal state.


4. Then repeat the previous steps starting with the BREAK signal three or more times (the more the better).


5. Look at the information in the thread dump section (search for "Full thread dump") of the Java core and make sure it looks similar in all three (or more) core dumps. If so, you should have enough certainty to trust what you see in the core dumps.

In a future post, I'll talk about what to look for in the Full Thread Dump, but to get you started, if you see the same method repeatably on the top of the stack of multiple threads, that is what you want to go after first.

In the last two months, I used this approach successfully to find problems in an application server application (bad third party libraries) and a migration script (bad algorithm for handling large amount of data). Both showed rather clearly what needed to be fixed in the program. Also, since this comes with Java, you need no tools to use the "poor man's profiler".

If you have questions or if you find this tip useful, please post a comment on this topic.

Tuscany Graduated

A quick shout out to my colleagues and all the folks who have worked so hard to develop an open community which supports Service Component Architecture. The Tuscany project at Apache has recently been promoted from an incubator project to a fully endorsed ASF top level project. This promotion recognizes the community for fostering an open and diverse membership, demonstrating capability to deliver stable, working implementations of the technology and also garnering the votes and confidence of the Apache Board. Congratulations on your success! Super job.

Evans Data Study: RAD scores the highest user satisfaction

It was with great pleasure that I read the Evans Data Study that was released this week; the study was a survey of over 1200 IDE users, and the bottom line was that RAD had the highest user satisfaction scores among the IDEs rated (including Oracle JDeveloper, Microsoft Visual Studio, MyEclipse and NetBeans). A survey is a survey; I have no doubt that you could find users of all of the above IDEs who are passionate supporters of their product of choice – but this survey revealed that compared to the other IDEs (and I’m quoting here) “RAD is just head and shoulders above them when it comes to user satisfaction.”

Cool; that’s what we’ve been trying to deliver, and it’s gratifying to see that our users are pleased with what we deliver.

What was most heartening to me, though, was what the survey didn’t measure; namely RAD 7.5. I think RAD 7.0 (which is the latest GA version of RAD that’s available) is a pretty darn good product, and the survey seems to agree with me – but I am even more pumped about what we have coming down the road, in the form of RAD 7.5. As I’ve discussed in this blog, we’ve had RAD 7.5 available in open beta since late last year – that code was refreshed at the beginning of May, and we’re still working on it. The fact that we were able to deliver public beta code so early in the schedule is a testament to the state and the quality of the release; we’re still adding features and function, we’re still fixing defects and improving the quality even more, and we’ve also got the benefit of lots of additional user feedback, with enough time (for a change) to do something about it. Put it all together, and I’m convinced that RAD 7.5 will be the best RAD we’ve ever delivered.

Back to the survey; results were provided giving an overall satisfaction of the different IDEs, and the various capabilities within the IDEs were also rated (how does your debugger rate, how do your web design tools rate, etc). Although we were first place in the overall rating, the RAD results in the individual areas were mixed; we did great in some places (application modeling tools, tech support, profiler, web design tools), and less well in other aspects. There was not a lot of variability in the rating for editors, all the products scored well - but in that category, RAD was in last place, for example.

That actually doesn’t surprise me, as the editors were one of the areas I was most concerned about as well – and for that reason, the editors, and the editing experience were one of the areas we’ve spent the most time and energy on, in RAD 7.5. I was presenting at the IBM Impact conference in April, and showed off some of the new features in RAD 7.5 – let me share a couple of highlights with you:

• The various deployment descriptor editors in RAD 7.0 were attempting to abstract away the details of the actual changes being made to the underlying files – but we’ve heard from numerous customers that it’s more important to assist the developers… but still “leave them in touch” with the underlying files being updated. We have a great new editor framework (the source of several patents) that we’re using across RAD 7.5, that means developers who make a change in the editor, will have no problem know what (and where) they have changed in the actual file – but at the same time, we still provide lots of help and validation in editing the actual contents. These new editors are getting great feedback from our beta customers.
  
  
• We’ve had a big focus on refactoring operations and quick fixes, both of which surface via editors of all shapes and sizes. Eclipse has a great history when it comes to providing refactoring operations, and RAD 7.5 really takes and extends that model; we participate in many existing eclipse refactoring operations now (for example, a rename will do the right thing across the many J2EE artifacts, as well the just the Java code), and we have added several of our own, J2EE specific refactoring operations. These refactoring operations allow us to provide significant help to developers who are updating their code, while leaving the editors to do just that – provide a good, focused, editing experience. Again, the feedback from beta users to date has been very positive on the changes we’ve made.
  
  
• JavaScript is increasingly important in a web 2.0 world, and so we’re introducing significant new support for JavaScript in RAD 7.5 – we have a comprehensive new JavaScript editor, which we’ve integrated into all the right places; you’ll now get support like code assist, syntax validation, and colour highlighting whether you are editing a standalone .js file, or a JavaScript snippet embedded in the middle of an HTML file, while you’re using the RAD page design tools.
  

These are just a few examples of the changes, improvements, and new features we’ve been developing for quite a while in RAD 7.5 – and the Evans survey has simply validated many of the decisions we’ve made; if we can be number one in user satisfaction with all our old editors, I’m super confident that we’ll be able to make developers even happier, and even more productive, with RAD 7.5.

Next up for me is the RSDC conference, in Orlando – we’ll be presenting and demoing RAD 7.5 there, so come by and say hi if you’ll be there.

Tim

GWC hosting an "Ask the Expert" day next Wednesday

The Global WebSphere Community (a.k.a. http://websphere.org) is hosting their annual "Ask the Experts" day with IBM Development and SOA experts on May 28th. You need to register as a GWC member to participate, but registering is free and fast and painless.

The session starts at 10:00 am Eastern and ends at 10:00 pm Eastern, and is basically an opportunity for GWC to both ask questions and submit ideas about WebSphere products, directly to the product teams. Do you have a nagging issue or a great idea? This is your chance to share both with IBM.

Join in!

Jobs and lyrics...emotional design

I'm no psychologist, but I play one on the interwebs. Anyway, let's take a blogger favorite subject, Steve Jobs...or preferably, El Jobso. Of the many things he's famous for, it's memorizing his speeches, down to the last seemingly spontaneous slip-up & joke. That "scrolls like butter" comment? Scripted. Yeah. Believe it.

But you gotta admit, it works. Whom else scripts their words so well? Musicians for one. And from his public speeches, we know El Jobso admires musicians. Musicians memorize their lyrics to get us to feel a certain way...just as El Jobso does when he demos the next new delicious piece of gadgetry. It's probably the highest art to get someone to feel as you do when you need them to. It's like a great concert when an entire stadium all feels the same emotion on cue. It's pretty cool, isn't it? So, where is all this going and how does it relate to WebSphere? We're getting there.

Compared to Apple, Microsoft's products are less emotive...but they're damn effective. They're good enough not to get in the way (most of the time...bluescreen...doh) and ubiquitous. Compared to Apple Music, they're Muzak. The white noise of music. Hold music. You know, that stuff in the background that helps you pass the time, but isn't so amazing that it forces you stop, to listen, to *feel*. That would be distracting. But it does remove some of the monotony.

Anyway, there's obviously a place (and I would say a need) for both approaches. I love beautiful objects as much as the next person, but after awhile, the beauty fades into the background, and utility had better be underneath, supporting the facade. Given a choice, people I speak to usually value utility over form, but only slightly, and only then after the glow of newness has worn off. You don't want your tool to be so difficult to use or so ugly that your profession becomes *work*. Bleh.

Frankly speaking, the WebSphere products I've worked on and with emphasize utility a bit too strongly over form. That is, they haven't been strong (but we're working on it) on design focus. We first set out to make efficient, scalable systems that run the back offices of your world reliably. Our model is mainframe reliability. You know, the every credit card transaction, every ATM withdrawal kinda stuff. But not the prettiest GUIs. Some Google software fits into this category...gmail/search/adwords, none too pretty, but all effective.

I suppose all software designers strive for that ultimate goal: perfect form, perfect function, perfectly intuitive, consistency top to bottom, inside and out. Based on the behavior of something you already know how to do, the next feature functions with the same general behavior. Looks, proportion, cut, color, fit, finish, function...all there. Applied well, your product develops a personality of a good, reliable, effective beautiful tool. A killer app. Much like a great sports car (maybe that's why we have so many comparisons between software and cars? but that's a topic for a different post...).

Anyway, those are my ramblings. Until next time, let us know your thoughts in the comments.

On Consumability...

Hi all, go easy on me given this is my first time posting here. Anyway, re: the title...you might not think of software as food, so indulge me for a minute. We can agree that anything more palatable is more consumable, no? Assuming that, then consumability of any product is a good thing -- they're "tastier" so to speak.

But Aaron, I hear you say...how do you make software more consumable?

Good question. Here's one handy definition: Time to Value. That means, from the time you identify that you'll use software to solve your problem, measure how long the interval is until you actually *deliver* a working software-based solution. Toss in a metric for average time to deliver maintenance and viola! Time to Value. Simple. Believable. Hard as s**t. But ultimately, we believe, doable.

IBM's idea, and this is portable across all software, everywhere, irrespective of origin/company etc. is this:

How effectively can clients identify the right IT solutions to solve a business problem, find the right IBM products and technologies to use, get the solution up and running, and maintain that solution? Our goal is to enable our clients to easily and effectively use our products to solve their business problems. Making this a positive experience leads to delighted clients, which in turn leads to strong references, repeat business, and revenue growth.

We're curious about your thoughts and as the months wear on, our progress (or lack thereof) in areas that you most need consumability.

Web 2.0 Collaboration, Sharing and Social Networking on WebSphere at Impact 2008

In my session at the Inner Circle at Impact 2008, I'll talk about the Collaboration, Sharing, Social Networking, and Portal capabilities provided by Lotus Quickr, Lotus Connections and WebSphere Portal that are all powered by WebSphere Application Server. I'll give an overview of these WebSphere applications and how they enable Web 2.0 collaboration and social networking to increase organizational productivity and will give an overview of their architecture, concluding with a set of demos.

Join me at Impact 2008 to discuss SOA Performance

I will be speaking on SOA performance at Impact 2008, the Smart SOA Conference, in Las Vegas in April. Please join me there at my session on Wednesday. The abstract for my session (1446A):

This session will discuss SOA performance as it relates to the IBM WebSphere® portfolio. The portfolio view will begin with runtime aspects including basic Web services (including the new Web Services Feature Pack). The runtime aspect focus will continue and build to choreographing and integrating these services across IBM WebSphere Process Server, WebSphere Enterprise Service Bus, WebSphere Message Broker, and the WebSphere DataPower® SOA appliance. The portfolio view will then focus on the SOA lifecycle, business activity monitoring and situation detection, and governance covering WebSphere Business Monitor and the WebSphere Service Registry and Repository. In addition, historical views on SOA performance will describe how performance has improved over the past two years. Finally, the session will discuss best performance practices when implementing SOA solutions.

I was also lucky enough to be included in a set of videos "On the Road to Impact" along with Kyle Brown on Event Processing and Ruth Willenborg on Virtualization. Go ahead and watch the videos and feel free to comment.

IBM WebSphere Application Server Total Leadership in SPECjAppServer2004

As noted in the IBM press release, we recently published a SPECjAppServer 2004 result that puts IBM on top of BEA, Oracle, and Sun for the absolutely largest SPECjAppServer 2004 throughput ever achieved. This result comes after we dominated the same competitive crowd on per cpu core performance as I blogged about previously. Both of these results matter. The per cpu core leadership shows application server performance in the most competitive light. The total configuration shows application server scalability in the most competitive light. IBM WebSphere dramatically wins both.

Some numbers from this total leadership publish:

• The result of 14004 JOPS represents roughly 75,000 transactions per second against the DB2 database. Simply impressive.
  
• The closest competitor in total throughput was Oracle at 10519. The IBM result tops that Oracle result by 33%. The IBM result tops the highest Sun result (8439) by 66%. The IBM result tops the highest BEA result (8253) by 70%.
  

Congrats to John Stecher and team. You can read more about the total configuration publish (and John's team) on InfoQ.

SPEC is a non-profit organization that establishes, maintains and endorses standardized benchmarks to measure the performance of the newest generation of high-performance computers. Its membership comprises leading computer hardware and software vendors, universities, and research organizations worldwide. For complete details on benchmark results and the Standard Performance Evaluation Corporation, please see www.spec.org. Competitive claims reflect results published on www.spec.org as of February 18, 2008 when comparing SPECjAppServer2004 JOPS on all published results as well as derived metrics calculated by SPECjAppServer2004 JOPS / Number of Application Server Processor Cores.

The facts of RAD

MyEclipse has issued a press release that mentions RAD, and may cause some confusion, so I'd like to take this chance to set the facts straight.

First of all, the transition from WSAD to RAD was purely a naming one. WSAD was delivered prior to IBM's acquisition of Rational, and RAD 6.0 was simply the next version of "WSAD" released under the Rational brand. RAD is entitled as a free upgrade to all active WSAD customers; there is no new cost - describing the transition from WSAD to RAD as complex or costly is either uninformed, or intentionally misleading.

The plumbing that formed the core of both WSAD and RAD was donated by IBM to the eclipse WTP project, to provide a defacto standard interface that others could extend - meaning that any tool extensions will work correctly and naturally with RAD. MyEclipse uses this base, so the basic project structure is the same - but the vast majority of the components that form RAD are not part of WTP; they are only available in the IBM offerings. Those value add components and functional areas are where we're spending most of our time and energy on, and that's where the real benefit of RAD comes from.

RAD 7.0, which has been available for over a year, includes comprehensive support for J2EE & Web Application developers, with support for pure spec compliant applications, as well as integration for all the WebSphere extensions - for example we don't just support JSF, we also ship and have outstanding support for the IBM JWL widget library; the latest version of JWL (which is only available with RAD) makes it fall-off-a-log easy to create an Ajax application, with no handwritten javascript at all (see http://webspherecommunity.blogspot.com/2007/06/simple-ajax.html for more details).

That said, simply providing spec level support is not really a differentiator today; that's why we donated the code we did to WTP. The RAD value proposition is it's ability to improve the productivity of developers across the entire development life cycle. Some examples:

• Some other products provide simplistic support of UML, that requires iterative generation of java code, followed by import actions to keep the diagrams and code in sync. This round trip engineering approach was available back in Rational Rose, and is a long way from state of the art now. RAD provides much more than that - for example, we provide UML visualization, which allows you to view, edit, and update Java code using UML views. There's no separate model to get out of sync; you're editing the live code, but in UML.
• All modern IDEs include support for database interaction and definition, but RAD has the benefit of working with the DB2 developers to provide exceptional support (for all major databases); RAD includes graphical views of your data topology, a SQL scrapbook, impact analysis tools, the ability to write user-defined functions (UDFs) & stored procedures, and phenomenal support for embedded SQLJ.
• RAD has in-depth support for application profiling, analysis & testing; you can execute, monitor, and trace the flow and performance of your application, right out of the box.
  

Of course, we also have the benefit of working hand in hand with our IBM colleagues, to provide the very tightest integration with WebSphere. It's not hard to deploy an application to WebSphere Application Server, but only RAD includes copies of three versions of WAS, to simplify your unit testing on a "real" WebSphere server, at no additional cost. Because we actually embed the WAS server, we support both remote application deployment and local application deployment, which gives you better deployment performance because there's no copying of files necessary. RAD also includes complete support for WebSphere Portal Server, including creation wizards, views, and editors that enable you to develop, test, debug, and deploy portal and portlet applications.

When it comes to production application deployment, RAD includes an advanced Jython editor, designed to simplify the task of developing and testing WebSphere automation scripts. We also offer unique integration with the WebSphere admin console, allowing you to capture key actions in the admin console and turn them into the equivalent Jython script commands. Finally, there's an embedded Jython debugger, that will let you single step through the admin script, examine variable values, etc.

IBM remains strongly committed to RAD; the next version of RAD (7.5) has been available as an open beta since late last year; you can find more details here: http://webspherecommunity.blogspot.com/2007/11/rational-application-developer.html. As you'll see if you try this code, RAD 7.5 provides many compelling features that focus on developer efficiency such as JEE refactoring and quickfix operations, while offering enhanced integration with products from all IBM brands, but especially Rational, WebSphere, and DB2. As I said in my blog entry, I'm really pleased with the progress the team has made on RAD 7.5; even though this is just a beta, the code is great quality, the performance is good, and all early reports are extremely positive - come check it out!

Tim Francis,
WebSphere Tools & RAD Chief Architect

WebSphere Process Server and ESB 6.1 Available

The 6.1 versions WebSphere Process Server, Enterprise Service Bus, and Integration Developer are available for download from passport advantage. I personally have used the 6.1 version for both our internal SOA benchmarks and a recent ESB project. I can attest for major functional improvements -- I wouldn't use anything less than 6.1 for ESB projects. Also, since this moved the underlying application server to WebSphere Application Server 6.1 and Java 1.5, there are some major performance improvements in this release. Oh, and WebSphere Integration Developer is now based on Eclipse 3.2.2 which enables far more of my eclipse plugins to work along with Integration Developer.

Links to more info:

WebSphere Process Server 6.1 new features
WebSphere Enterprise Service Bus 6.1 new features
WebSphere Integration Developer 6.1 new features

Enjoy!

WebSphere SIP Performance and Enterprise QoS helps AT&T

Another effort from the my performance team members is documented in the latest IBM press release.

This effort shows how:

1. WebSphere Application Server is leading the way in critical new technologies (SIP)
2. The WebSphere Application Server has performance that is not only leading, but also carrier-grade while being enterprise capable - including High Availability.
3. Our commitment to driving open and trusted industry standard benchmarking

In WebSphere Application Server 6.1, we added support for the IETF Session Initiation Protocol (SIP). SIP is a request/response protocol for negotiating communication between two endpoints that has been widely adopted in the telecommunications industry. SIP is useful in Voice Over IP (VOIP), web click to call, conferencing, interactive voice response systems, instant messaging, and presence applications.


Below is a basic example of a session initiation between two parties. It shows a swim lane diagram of user@ibm.com starting a discussion with user@example.com with proxies inline to negotiate the session.

In the press release, we announce how AT&T plans to use our SIP technology:

"AT&T is using IBM WebSphere Application Server and BladeCenter systems as a Session Initiation Protocol (SIP) service logic execution environment platform to develop mission-critical services for deployment on AT&T's IP-based network."

The features needed to support AT&T include not only SIP, but also technology that makes such latency dependent applications such as VOIP possible, along with enterprise class high availability. Also, performance is quite important. Also mentioned in the press release, we detail an internal SIP benchmark we ran along with the results:

"IBM's WebSphere Application Server delivers telecom class infrastructures with a focus on SIP features. It is a carrier-grade application server that utilizes a converged HyperText Transfer Protocol (HTTP) and SIP container. It allows service providers to quickly offer new personalized and productivity enhancing services that subscribers demand. WebSphere Application Server 6.1.0.11 using Red Hat Linux and integrated in the IBM BladeCenter HT chassis, which is network equipment building systems (NEBS) compliant, has achieved an industry leading SIP performance measurement of 1296 calls per second, using a 13 message SIP call model (with 80 second hold time) which translates to over 4.6 million busy hour call attempts per blade. Using this call model, in a high availability, carrier-grade configuration, WebSphere Application Server achieved 660 calls per second per blade with session replication. These results were achieved while retaining extremely low end-to-end SIP message processing latency of under 50 milliseconds ninety five percent of the time. This exhibits the ability of WebSphere Application Server to handle the call volumes businesses demands while ensuring service quality."

What does this mean?

It means that our converged web container (supporting typical HTTP servlets, JSR 168 portlets, and now JSR 116 SIP Servlets) can perform SIP operations at carrier class speeds. We demonstrated this using an internally developed SIP call model benchmark. The benchmark is similar to benchmarks being used externally by other SIP platform vendors, but only IBM added high availability to the benchmarking scenario. IBM shows that we lead in performance where it counts – in enterprise ready high availability scenarios.

Based on my previous posts, you would assume we'd want to push SIP benchmarking to standardized benchmarking organizations so the measurements can be trusted by customers. Well, we are doing exactly that! IBM, along with other vendors, has formed a subcommittee at SPEC to develop an industry SIP benchmark. Erich Nahum (IBM) is chairing the group and has requested public feedback on the current proposal.

If you'd like to find out more about SIP, Erik Burckart has a two part article on SIP containing both an introduction to SIP as well as a guide to developing converged SIP applications.

Congrats again to my most excellent performance team members.

Update: Telephony Online coverage that includes comments on SIP performance.

The wonders of modern hardware

I was at an IBM conference a couple of weeks ago, where some new hardware ideas were being presented. Perhaps it's my engineering background, but there's something very cool about building things you can hold in your hand - and when you combine that with the latest new technological ideas (multicore, etc), it's very impressive. In that light, I'm happy to share a video that I saw recently.



http://video.google.com/videoplay?docid=5125780462773187994

Tim Francis

Rational Application Developer

I have not posted to this blog for a while, and I can now explain why. Today, at the WebSphere Technical Conference in Vienna, we announced the upcoming availability of an open beta for the next major version of Rational Application Developer (RAD 7.5). Those of you who've read my previous posts (hi Mom!) will not be surprised by the themes we're focusing on for RAD 7.5; better support for the complete application development lifecycle, including refactoring operations and quickfixes that cover the complete Java EE programing model.

The RAD 7.5 beta also provides support for the WebSphere EJB3 feature pack, the beta for which is available now for WebSphere 6.1. The RAD support for EJB 3.0 allows us to really exploit annotation based programming models. Previous versions of the spec focused on deployment descriptors, and the matching RAD releases had advanced editors to support that. In RAD 7.5, the focus is on Java developers, and using the Java editor as a base - but extending the already powerful Eclipse editor by adding significant new function and capability to support developing an annotation based application.

The intent is simple; most developers creating applications for WebSphere are quite comfortable using Java, and we have a great Java editor in RAD/eclipse - so it's a mistake to force them to switch to a whole new environment (with wizards and graphical editors) when they want to define some aspect of server interaction (such as Web Services or EJBs). This is a natural extension to what the spec defines, but the way we're supporting that pattern within RAD is new, easy to use, and productive.

The other aspect that I'm thrilled about is just the fact that we're running an open beta, less than a year after we shipped the last major version of RAD. We're not done yet, and we have more good ideas that we're still implementing - but the code is great quality, and the performance is good... and this shows a commitment to delivering a rock solid, robust, and well performing RAD. The intent is to encourage developers to explore the product, and not feel concerned that they'll get in trouble if they try something non-standard or off the golden path.

I'm really pleased with the progress the team has made on RAD 7.5; this is going to be the best version of RAD we've ever delivered, and the open beta gives you a chance to see where we're at, comment on what you see, and help us refine the product to make it exactly what you need. You can sign up for the beta here, and there will be newsgroups available to report problems, ask questions, and give us your feedback.

Tim Francis

InfoQ interview on IBM SPECjAppServer leadership and benchmarking in general

InfoQ recently did an interview with myself and John Stecher who leads our JEE / SPECjAppServer performance work. The interview explains in further detail our work that led to the 37% leadership over both BEA and Oracle that I posted here:

http://webspherecommunity.blogspot.com/2007/10/websphere-beats-bea-and-oracle.html

Here is the interview:

http://www.infoq.com/news/2007/11/ibm-specj

Feel free to post follow-up questions here or on the InfoQ forums.

InfoQ is an excellent news site for tracking important news in enterprise computing.

SPEC is a non-profit organization that establishes, maintains and endorses standardized benchmarks to measure the performance of the newest generation of high-performance computers. Its membership comprises leading computer hardware and software vendors, universities, and research organizations worldwide. For complete details on benchmark results and the Standard Performance Evaluation Corporation, please see www.spec.org/. Competitive claims reflect results published on www.spec.org as of October 03, 2007 when comparing SPECjAppServer2004 JOPS@Standard per CPU Core on all published results.

WebSphere Beats BEA and Oracle Performance By Over 37%

I wanted to take the time to congratulate some of my performance team members who have worked to prove our performance leadership, displacing both of the current top SPECjAppServer 2004 leaders - BEA and Oracle. You can see this documented in our latest launch press release here.

"A powerful IT infrastructure is also required to support the growing number of transactions across the enterprise, as well as with customers, partners and suppliers. IBM recently beat competitors by 37 percent in an industry benchmark that measures high-volume transaction processing that are typical in today's customer environments. IBM WebSphere Application Server established record-breaking SPECjAppServer2004 benchmark performance and scalability results involved more than 15,500 concurrent clients and produced 1,197.51 SPECjAppServer2004 JOPS@Standard (jAppServer Operations Per Second), which translates into more than 4.3 million business transactions over the course of the benchmark's hour long runtime."

This shows our commitment to standard benchmarking organizations as I described in my last blog post. By leading this benchmark that was designed and approved by all participating SPEC vendors, IBM customers truly benefit from the demonstrated performance leadership.

We chose to lead in a category that matters most to our customers - per core performance. If you look at the result and the number of J2EE server cores and divide it out, we lead Oracle by 37% and smash BEA by 56% with approximately 300 JOPS/core. Comparing per core (CPU) normalized performance is really the best way to compare one application server and hardware stack to another. This demonstrates IBM's leadership of the industry in J2EE performance without question.

This result also shows the combined power of an IBM stack. Specifically, we achieved this by running WebSphere Application Server v6.1 on the IBM 5.0 JVM on AIX 5L V5.3 with an IBM POWER6 p570 server powered by two dual-core IBM® POWER6® 4.7 GHz processors against IBM DB2 Universal Database v9.1 running on a single POWER6-based server.

Congrats again to my team who yet again dominates where it matters - in customer focused standards based benchmarks.

SPEC is a non-profit organization that establishes, maintains and endorses standardized benchmarks to measure the performance of the newest generation of high-performance computers. Its membership comprises leading computer hardware and software vendors, universities, and research organizations worldwide. For complete details on benchmark results and the Standard Performance Evaluation Corporation, please see www.spec.org/. Competitive claims reflect results published on www.spec.org as of October 03, 2007 when comparing SPECjAppServer2004 JOPS@Standard per CPU Core on all published results.

A few pictures from London

I went to London for a couple days at the end of last week for a customer visit. My first time in London, but unfortunately the quick turnaround didn't leave much time for sightseeing. Here's a couple of my favorites pictures that I snapped:

SCA Rock Rolls Forward

My colleagues who have been working over in open source land at Apache inform me that Apache Tuscany SCA Incubator project 1.0 level has been released. The formal announcement, source and binaries are located here. This is a significant milestone in the life of SCA as an open initiative.
Congratulations!

Perspective in Benchmarks - My thoughts on Microsoft StockTrader

I wanted to take the time to respond to claims coming from Microsoft. On June 4th, Greg Leake, Technical Marketing Manager for .NET of Microsoft started to discuss an effort with which he was involved. The effort was “porting” IBM’s WebSphere Performance Sample – Trade 6.1 to Microsoft .NET 3.0. Since then, Microsoft has been interviewed by trade press, has created video podcasts, and has started discussions on Microsoft forums and other development community sites. Please do not be fooled by these marketing statements.

In my opinion, Microsoft uses their report to focus on three points

- Attempt to prove that Microsoft is interoperable with WebSphere software
- Attempt to prove Microsoft’s SOA relevancy and readiness for the enterprise
- Attempt to prove Microsoft performance is better than WebSphere software performance

I’d like to state up front that I was personally involved with the development of IBM’s Trade benchmark starting with Trade 3 and ending with Trade 6.1. Others on my team have made significant contributions to the code as well to ensure that Trade continues to be an credible and useful tool to express performance concepts and release to release enhancements to IBM customers for J2EE (Java EE) programming.

Some level of interoperability

I applaud any effort by Microsoft or any other software vendor in pursuit of interoperability. However, the interoperability the Microsoft report speaks to is basic web services functionality (SOAP/WSDL/XML) only. It does not focus on interoperability of transactions, security, reliability, durability and does not use industry standard schemas that many of our customers need for cross enterprise (B2B) or intra-enterprise web services interoperability with .NET clients calling WebSphere servers. Independent of the Microsoft report, Microsoft and IBM have already focused on interoperability of these higher value qualities of service in Web Services through industry standard WS-I efforts as shown here and here. I am very happy to see IBM and Microsoft continue to focus on standards based interoperability. I am confident that WS-I will continue to facilitate customer focused web services interoperability in these higher value web service functionalities.

Is Microsoft enterprise ready?

Press coverage has cited the Microsoft report as helping Microsoft prove it is ready for the enterprise. From our quick look at Microsoft code, this doesn’t seem to be the case. Many of the “enterprise ready” features stressed by the Microsoft report are hand coded into the application. Areas such as multiple vendor database support, load balancing, high availability, scalability, and enterprise wide configuration for services contribute to the significantly higher lines of code count in the Microsoft application as compared to the WebSphere implementation. You should equate higher lines of code count to more maintenance and support costs long term and think about the value of this being provided in the application server product versus in the application itself. Some of the increased lines of code are for comments, but the comments themselves point out how many places where Microsoft deviated from using framework classes and instead implemented custom extensions to fill in gaps of functionality lacking in their framework.

As an author of Trade, I must admit an embarrassing fact. Four years ago, my team added web services capability to Trade as an optional component – to demonstrate web services as an alternative to remote stateless session bean methods. Since that time, I personally have worked hand in hand with many enterprise customers adopting web services. We have found that the Trade approach to web services wasn’t the best. Specifically the fine grained “services” in Trade average around 400-800 bytes of passed data. As you’ll see in my recent blog post, industry standard schemas for B2B typically have much larger payloads. While Trade was a fun exercise for my team to learn web services, it in no way mirrors what we know now our customers have told us are good web service practices. Current SOA principles motivate more coarse gained services. The embarrassing fact is that we never have gotten around to removing the poor examples of web services usage from Trade. However, it is interesting to note that Microsoft did not recognize or point out the obvious flaws in these web service patterns during their analysis – they merely parroted what they saw.

Microsoft entitled this paper as “Service Oriented”. Later in the press, Microsoft alludes to this “Service Oriented” benchmark test to help draw credibility to Microsoft’s SOA strategy. Based on what I just said about Trade web services, you’ll see why IBM has never talked about Trade and web services to help customers understand web service performance. Meanwhile, this Microsoft report focuses on these trivial payload web services to prove they can support “Service Oriented” benchmarks. Follow-up posts and press coverage followed the “Service Oriented” title, proclaiming that this in some way helps Microsoft’s SOA strategy. Drawing that conclusion is incorrect. The industry agrees that SOA is more than trivial web services. SOA is about a business centric IT architecture that provides for a more agile and flexible IT support of business needs. Realistic web service usage is a cornerstone of SOA, but web services alone do not define SOA.

Is Microsoft performance better than WebSphere performance?

I believe Microsoft is trying to draw IBM out of our commitment to standard benchmarking organizations to confuse customers about performance. You can draw your own conclusions from specific comments apparently made by Greg Leake in press coverage, asking for IBM to “collaborate”. Some have commented correctly on community discussions that the Microsoft report isn’t specific enough in terms of topology, hardware, and tuning to make any sensible conclusions based on the performance data. It is a compelling story that this Microsoft report weaves – Microsoft beating IBM on its own benchmark. However, IBM didn’t run Trade as a benchmark in the way shown in the paper’s results. As a customer, you should always be careful how much you trust proprietary benchmark results produced by a single vendor. These things can always be coerced to create FUD and confusion.

We have reviewed the paper and results and found inconsistencies with the best practices for how to run WebSphere Application Server. Assuming items Microsoft chose not to document along with improvements in performance allowed by following best practices, we in fact believe that IBM WebSphere Application Server would win across all the scenarios shown in the results.

You may well ask, if WebSphere Application Server would win, why wouldn’t you say so and publish a contrary IBM report. I don’t believe publishing a proprietary view of performance would help our customers – for all the reasons stated above. At best, if IBM was to respond to this paper, you could expect both vendors to degrade to the lowest common coding styles for implementing the benchmark so they would “win”. As shown already by Microsoft’s implementation, Microsoft wouldn’t choose to use their framework classes and features, but instead code optimal patterns in the application code. When those patterns are not tested and supported by the formal product, no customer wins by seeing the resulting performance.

IBM has a history of competing in standard benchmarking organizations such as SPEC. We do so because such organizations are standards based and unbiased and therefore trusted. Standards-based benchmarking processes give all participating vendors equal opportunity to review run rules, implementation code, and the results of other vendors. Given this, if you find Trade and SOA benchmarks useful, maybe it is time for IBM and Microsoft to jointly propose a SOA benchmark to a standard benchmarking organization. SOA benchmarking under standard benchmarking organizations is where our customers and the industry can truly benefit.

Summary

I personally talk to many customers about performance, SOA, and web services. I stand behind all that I say technically independent of marketing. I build strong long lasting relationships with these customers, many of whom know me personally. In good faith, I can stand in front of them with results from a standards benchmarking organization. I can stand in front of them showing our SOA leadership based on both customer references and analyst reports. On the other hand, I cannot in good faith show a one-off competitive benchmark run by a single vendor. I hope you can understand this position and it helps you discuss the coverage of this Microsoft report and any similar efforts by any vendor that follow.

This Microsoft report shows basic levels of interoperability and work within the WS-I shows higher levels of interoperability. The Microsoft report points out that Microsoft has to hand code many enterprise ready features that are just taken care of for you in WebSphere technology. The title of this Microsoft report and its follow-on press coverage attempts to confuse the industry on SOA, which points out how desperate Microsoft is to get press coverage in SOA. This Microsoft report doesn’t do a good job of showing the true performance story. Specifically on performance, I encourage all customers to put their trust in standard benchmarking organizations

New Redbook with SOA customer case study

A new Redbook has been published called "Implementing and Testing SOA on IBM System z: A Real Customer Case".

The Redbook does a good job of breaking down SOA implementation into a series of logical decisions, with real world implications. It covers not only the overall architecture, but the steps to get there. The final solution includes much of the WebSphere Business Process Modeling stack, including WebSphere Application Server, WebSphere Process Server, and WebSphere Portal, as well as tooling from WebSphere Integration Developer and WebSphere Business Modeler.

In particular, the Redbook shows how the SOA solutions fit into the existing ecosystem at the customer shop. SOA is evolutionary, not rip-and-replace. A good read.