Environment tuning and compression

I was going to write about getting to know your customers but I just saw an article on features within Websphere application server where the author made a comment about the high CPU utilization required to achieve compression, obviously either something has changed or he had fallen for the marketing hype about network edge devices taking over that role.

This trend stated quite a few years ago with Cisco bandwidth ques, L7 packet inspection and queueing based on application and priority from Packetshaper and many others. These devices progressed to application accelerators that now do all this is one box. My question was always, and still is, “are they really needed”?  

Let’s consider what application accelerators actually do.

1.    When a client requests to open a TCP connection to the server the edge device will open multiple TCP connection and hold them open, just like persistent connections. This reduces the time for the TCP handshake when that client requests further connections.

2.    The edge device will perform TCP send/receive buffer manipulation to increase the throughput across the network.

3.    The edge device could turn on compressions between its sister device the other side of the network, thus reducing the traffic across the long distance links. The traffic between the client, or the server, to the edge device may or may not be compressed.

4.    They perform L7 packet inspection so they can determine if the traffic is valid or not, or if the traffic has a higher priority on the network, as in QOS, such as VOIP traffic or business critical traffic the customer has defined for a higher priority.

5.    Then the device can try to accelerate the business critical traffic, even to the point of slowing other traffic to give the high priority traffic more use of the network. The issue I found was that some group always considered their traffic should have a higher priority, and had the funding to get network services to increase the priority on their application traffic. What happens when everyone does this and there is no longer low priority traffic to relegate to the lower QOS queues?

But what can you do if you do not have accelerators or cannot afford to install them within your network? You tune your environment to get better results.

An example of this was when the company decided to move an application hosting environment from one country to another. This was part of a consolidation effort to reduce the number of hosting environments and reduce service costs. The problem was that the users in the original hosting country complained that their performance would deteriorate when the server was moved away.

We took sniffer traces of their normal application traffic and then analyzed it with Netdata. This gave us the baseline on their application performance. We then took their traffic and imposed a higher latency into the charting of the sniffer traffic, basically modelling what impact the higher latency would have. Netdata showed that they would suffer a 4% drop in application performance. We reported back to management that they would have a 10% hit, just to give us some wiggle room, if we just moved the server and did nothing else.

We of course were asked what else we could do. We looked at Akami solutions, dedicated application accelerators, dedicated queues’ via Packetshapers or via the routers or we could just tune the environment. Have you noticed that we don’t tune anything now, we trust in the notion that if you plug TCP devices together, that it will all work, and it does but maybe not to the most optimum level. We moved the servers to their new location and as tuning was the low cost option that was what we did first. We tuned the TCP send/receive buffers, the application buffers – that space in memory that TCP uses to transfer data to and from the application, we tuned on compression at the Websphere level and a host of things I can remember. We again collected sniffer traces and ran them through Netdata. We went back to the end users with a survey and compared the data from the base line.

Netdata reported a 24% decrease in application response times, hence an increase in application performance, the end users all asked when we were going to move the systems and whatever we were doing had to be included. The move was a great success yet cost nothing but some engineering time to make system changes.

But let me go back to compression, as I mentioned we turned on compression on the websphere server. A lot of people said that it would affect CPU utilization so we did some testing with Lotus. Websphere has 9 levels of compression, 1 being the lowest level and 9 the highest. They found that at level 3 they could not detect any CPU utilization, yet achieved a 60% reduction in network traffic. At level 9 there was a 7% CPU hit but achieve 85% reduction in network traffic. Every browser tested, and that was all the majors, support PKzip as standard so there was no work required at the browser. In fact I believe the largest advantage was that the compression was done before the SSL encryption so all traffic was compressed before it got onto the network.

I actually found that most IBM products, if not all, supported compression as part of the standard product set, Websphere, MQ messaging, Disk channels just off the top of my head. In fact on another project we took the MQmessaging throughput from 400kb/sec to 6Mb/sec by turning on MQ compression and some TCP tuning between RS6000 servers in the US to MVS systems in Australia – don’t ever let them tell you that you can’t tune the TCP setting on an OSA express MVS interface.

I was always amazed the number of companies or application owners that did not really know what was happening in their own environment, but it took Netdata to really show me that there were tools that allowed you see what traffic your network really carried, what your application traffic actually looked like on the network, how transaction handling effected application performance and so many other things that I will discuss in other posts.

Stay safe