Capturing network traffic errors with Selenium automation

While running test in Continuous Integration (CI) environment we come across to capture network traffic for errors and performance test. Sometime screenshot capturing help in identifying UI issues in CI environment, but they are not helpful in identifying root cause of issues if they are because of network traffic failures.

Network traffic consist of all network communication between browser and server to load image files, CSS, JavaScript and resources

For every network communication there will be http status code. Http status code help in identifying the success(2xx) or failures(4xx client errors and 5xx server errors) for resource loading

Rule of the thumb to identify root cause of issues in CI environment for automation capture all 4xx client errors and 5xx server errors.

Network traffic with Poltergeist(headless)

Poltergeist is a driver for Capybara. It allows you to run your Capybara tests on a headless WebKit browser, provided by PhantomJS.

Poltergeist has inbuilt support to capture network traffic

page.driver.network_traffic

 To clear the network traffic

page.driver.clear_network_traffic or page.driver.reset

Then(/^I see network traffic after page load$/) do
  page.driver.network_traffic.eachdo |request|
    puts "nnnn########################################################"
    puts "n Request Data #{request.instance_variable_get(:@data)}"
    puts "n Request Error #{request.error}"
    #puts "n Request Response #{request.instance_variable_get(:@response_parts)}"
    request.response_parts.eachdo |response|
      puts "n Responce URL #{response.url}"
      puts "n Status #{response.status}"
    end
    puts "########################################################"
  end
  page.driver.clear_network_traffic
end

Network traffic with Firebug and NetExport plugins

package NetworkTrafficfirebug;

import java.io.File;
import java.io.IOException;

import org.openqa.selenium.*;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.openqa.selenium.firefox.FirefoxProfile;
import org.openqa.selenium.firefox.internal.ProfilesIni;

public class firefoxprofile {

   public static void main(String[] args) throws IOException {
      
      //FirefoxProfile profile = new FirefoxProfile();
      ProfilesIni profilesIni = new ProfilesIni();
      
      // Clone the named profile               
      FirefoxProfile profile = profilesIni.getProfile("default");
      
      //Set preferences for this particular profile
      //To see preference key and value type about:config in firefox address bar
      profile.setPreference("browser.cache.memory.enable",false);
      
      //To add an extension/plugins to install into this instance.
      File firebug = new File("src/main/resources/firebug.xpi");
      File netExport = new File("src/main/resources/netExport.xpi");
      profile.addExtension(firebug);
      profile.addExtension(netExport);
      
      //Setting Firebug preferences
      profile.setPreference("extensions.firebug.currentVersion", "3.0");
      profile.setPreference("extensions.firebug.addonBarOpened", true);
      profile.setPreference("extensions.firebug.console.enableSites", true);
      profile.setPreference("extensions.firebug.script.enableSites", true);
      profile.setPreference("extensions.firebug.net.enableSites", true);
      profile.setPreference("extensions.firebug.previousPlacement", 1);
      profile.setPreference("extensions.firebug.allPagesActivation", "on");
      profile.setPreference("extensions.firebug.onByDefault", true);
      profile.setPreference("extensions.firebug.defaultPanelName", "net");

      // Setting netExport preferences
      profile.setPreference("extensions.firebug.netexport.alwaysEnableAutoExport", true);
      profile.setPreference("extensions.firebug.netexport.autoExportToFile", true);
      profile.setPreference("extensions.firebug.netexport.Automation", true);
      profile.setPreference("extensions.firebug.netexport.showPreview", false);
      profile.setPreference("extensions.firebug.netexport.pageLoadedTimeout", 15000);
      profile.setPreference("extensions.firebug.netexport.timeout", 20000);
      profile.setPreference("extensions.firebug.netexport.defaultLogDir", "/tmp");
       

      WebDriver driver = new FirefoxDriver(profile);
      
        
        try {
           Thread.sleep(5000);
          
          // Now use the firefox instance driver to open a page URL
          driver.get("http://www.whiteboxtest.com/selenium-test1.php");
          // driver.navigate().to("http://www.google.com"); alternative to open a URL
          
          
          // Check the title of the page
            System.out.println("Page title is: " + driver.getTitle());
            
            Thread.sleep(20000);
      } catch (InterruptedException e) {
         // TODO Auto-generated catch block
         e.printStackTrace();
      }
      
        // Close and clean firefox driver instance
      driver.quit(); 
   }
}

Network traffic with BrowserProxyMob

package NetworkTrafficBpm;

import java.io.File;
import java.io.IOException;
import java.io.FileOutputStream;

import org.browsermob.core.har.Har;
import org.browsermob.proxy.ProxyServer;
import org.openqa.selenium.By;
import org.openqa.selenium.Proxy;

import org.openqa.selenium.*;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.openqa.selenium.remote.CapabilityType;
import org.openqa.selenium.remote.DesiredCapabilities;

public class firefoxprofile {

   public static void main(String[] args) throws Exception {
      
      // start the proxy
       ProxyServer proxys = new ProxyServer(4445);
       proxys.start();
 
       // get the Selenium proxy object
       Proxy seleniumProxy = proxys.seleniumProxy();
 
       // configure it as a desired capability
       DesiredCapabilities capabilities = new DesiredCapabilities();
       capabilities.setCapability(CapabilityType.PROXY, seleniumProxy);
 
       // start the browser up
       WebDriver driver = new FirefoxDriver(capabilities);
        
        try {
           Thread.sleep(5000);
           
           // create a new HAR with the label "apple.com"
           proxys.newHar("whiteboxtest.com");
          
          // Now use the firefox instance driver to open a page URL
          driver.get("http://www.whiteboxtest.com/selenium-test1.php");
          // driver.navigate().to("http://www.google.com"); alternative to open a URL
          
          
          // Check the title of the page
            System.out.println("Page title is: " + driver.getTitle());
            
            Thread.sleep(20000);
      } catch (InterruptedException e) {
         // TODO Auto-generated catch block
         e.printStackTrace();
      }
      
        // Close and clean firefox driver instance
      driver.quit(); 
      
      Har har = proxys.getHar();
        FileOutputStream fos = new FileOutputStream("/tmp/whiteboxtest.har");
        har.writeTo(fos);
        proxys.stop();
   }
}

http://www.whiteboxtest.com/selenium2WebDriver-Capture-Network-traffic.php