Automated acceptance tests using Selenium Grid without parallelization
Originally we built our automated acceptance tests within our agile development process on a continuous integration server using Selenium with only one Selenium Remote Control. The tests were executed on a fixed browser under a specific operating system. With the growth of the test cases in the test suite the execution time of the builds extended rapidly, so the tests could not directly identify defects and thus a part of their function was lost.
The common strategy to solve this problem is to install Selenium Grid on the machine that formerly hosted the one Selenium RC to parallelize the execution of the tests. By connecting only 4 Selenium RC’s to this Grid Hub the execution time of these tests is reduced by a factor of 4 without any additional hardware and without rewriting the tests. The only prerequisite for this is that the used testing framework supports a parallelized test execution, i.e. it must be able to start more than one test of a test suite simultaneously and assign the answers supplied by Selenium Grid to the right test again.
Although our used testing framework PHPUnit does not provide a parallel execution of tests yet we found a way to use the benefits of Selenium Grid in our testing environment. Our continuous integration server provides the possibility to set up more than one build agent to run the Selenium driven acceptance tests. If we would do this with one single Selenium RC these agents would stress this RC rapidly because there is no possibility to check its state. The agents would start new tests no matter how many tests are already running at this RC.
So we installed Selenium Grid with 4 connected Remote Controls as described above. We can now control the number of simultaneously running tests, because Selenium Grid starts only so many test suites as RC’s are connected. Other incoming requests are queued in the Selenium Grid Hub until one of the connected RC’s has finished its test suite. Unlike the common usage of Selenium Grid we have not yet a real parallelization with this solution, since the test suites from the build agents of the continuous integration server run simultaneously, but each is still to be processed sequentially. But we have always the option of switching to a real parallelization when our testing framework supports it.