Tag Archives: Scalability

Erlang R13A Benchmark

I made a little benchmark to check out the new Erlang Release R13A and the behavior of the multiple run queues. The benchmarking program was the same I used in another benchmark you may find here. You may also find the sources at that location. As already noted there the slope from 1 CPU to 2 CPUs is due to the “bad” implementation made to challange the Erlang SMP features. The mashine was an 8 core Intel Xeon 3 GHz with a 64bit 2.6.9 Linux, Erlang kernel polling active.

erlangr13a_8cpu

PHP SPL Data Structures Benchmark

Data structures and collections are one of the most wanted features for the Standard PHP Library SPL over the last few years. With PHP 5.3 we’ll finally get a little of what we want and this is good news. With data structures like stack,  queue, heap or priority queue implemented in C we expect PHP programming to become somewhat more efficient.

Inspired by this post http://blueparabola.com/blog/spl-deserves-some-reiteration we decided to run our own benchmarks to either verify or disapprove the results posted. Our benchmarks were executed on a 64bit RHEL with PHP 5.3.0beta1. As you may expect, we carefully excluded startup or compilation time and measured only the code blocks in question. We used getrusage() to determine CPU time consumption. A huge number of iterations guaranteed smooth results.

The first structure under consideration was the SplFixedArray. If you only need numerical indices you now can create an array of fixed size that does not have to grow while more and more items are inserted. Dealing with an SplFixedArray saves you about 10 percent of runtime compared to a plain old PHP array.

Next we tried the SplStack and SplQueue. It is usually easy to implement a stack and a queue with plain arrays. Using array_push(), array_pop() and array_shift() is straightforward. It may be a surprise to the average PHP programmer to learn about the runtime behaviour of these functions. Worst is array_shift() because of the internal rehashing and the experienced PHP programmer may – for critical code at least – try to access arrays by indices maintaining counters, for example. This is much more efficient. Compared to the functions, at least SplQueue is something like an upset, but it is possible to find comparable solutions with plain PHP.

bars_full_ok

There is a little danger to compare apples and pears when turning towards SplHeap and SplPriorityQueue. What is the proper representation of a heap implemented using plain old arrays only? It’s a sorted array, ok. But a heap is sorted for each insert, so, do we really have to sort the array for each insert? Who will do this in real life?

It’s the use case that decides about the sorting strategy. If you are supposed to carefully separate writing the heap and reading from it, it is sufficient to sort it once. That way you beat SPL. But if you have to mix reading and writing arbitrarily the SPL will beat plain arrays by far. This is shown in the pictures below. For the mixed strategy we read once for 5 inserts and the SplMinHeap scales very well. The same holds for SplMaxHeap and SplPriorityQueue.

splminheap_rw_separated

splminheap_rw_mixed

Lessons learned:

  • SPL rules
  • use SPL data structures where appropriate for a particular use case, they are efficient and comfortable
  • benchmarking is error prone
  • anyway, always benchmark performance critical code blocks

Webserver Scalability and Reliability

Everybody knows Apache and Tomcat but when I try to talk about such strange things like Yaws or Mochiweb nobody knows what I actually want. These two are HTTP server implementations written in the old fashioned functional language Erlang and running on the famous Open Telecom Platform or OTP. Erlang/OTP was developed in the late 80s as a fault tolerant and highly scalable system for telecom applications. Nowadays in the social networking community it is daily business to serve some 10.000s of PHP requests per second. So we are facing problems telcos have for a long time.

Apache is the canonical web server to serve PHP to the world. Thinking about technological alternatives in the backend domain we have a look at both Java and Erlang. A rather quick and easy test to compare the scalability of the technologies was to setup web servers delivering the same static document. The image below shows the results.

Web Server Scaling by Concurrency

Apache and Tomcat scale nearly linearly up to a concurrency of 1000. We find that Mochiweb has a breakdown when concurrency reaches 300 but afterwards still scales linearly as Yaws does at a lower level. Absolute performance is less interesting here. What counts is the scaling behavior. Mochiweb for example is not designed first hand to deliver static content but to act as a HTTP endpoint for arbitrary OTP applications. Neither Yaws nor Mochiweb seem to cache documents in the default setup. Also, we did not use HiPE.

Unfortunally we had not yet the chance to verify the measurements given in this post: http://www.sics.se/~joe/apachevsyaws.html where Yaws still scales linearly (in the average) when Apache has long gone away, at concurrency of 50.000 or even 80.000 and clearly seems to survive DDoS attacks.

In this picture Erlang/OTP may not be a recommendation for classical web delivery but to build reliable services either to provide internal or external API endpoints. An interesing alternative at least.