Proxy Servers and Production Issues

While investigating production issues, at two instances I found proxy servers to be the cause of bottleneck.

In the first instance an application hosted in New York was being used from different locations around the world but the users from France faced issues. We used HP BSM to run scripts from differnt locations and were able to replicate the issue from agents in France. Later we asked the infrastructure team to provide the architecture of proxy servers. There were 4 proxy servers in the France region. We further digged in, we ran 4 scripts from France location, each through a differnt proxy server. This helped us to exactly pin point which of the 4 proxy servers were causing issue. Out of the 4 servers, 2 of them had been recently upgraded to a new OS and the traffic going through those 2 proxy servers was giving issue. When we provided the proofs to the proxy team, they installed a patch to fix the issue and the production problem was thus resolved successfully.

In the second instance an application hosted in London was giving issues when accessed by users from USA. In this case also we used HP BSM to run scripts from different locations and were able to replicate the issue from agents in USA. When we provided the proofs to the infrastructure team, they asked for further proof using wireshark. We programmatically integrated wireshark with HP Vugen scripts and collected wireshark sessions. Wireshark further confirmed the issue with proxy servers, the requests being sent to proxy servers never returned any response. In this case infrastructure team never fixed the issue and it was proposed to bypass the proxy for the specific application which was more of a security risk. It was like a deadlock between security and infrastructure.