With all these ruby implementations floating around, I thought it would be nice to compare a few of them and see which one turns out to be the fastest. Although speed shouldn't be the only thing you consider when you need to pick a runtime, ruby's poor performance is often targeted as a problem (most often by Python users who are just jealous of our beautiful language and are looking for something they can use to make Ruby seem worse than their annoying, indentation based weapon of choice), it can be important sometimes.
Well, mostly because they're popular, and under active development. JRuby is nearly 100% compatible with the standard Ruby interpreter these days, and Rubinius is worked on every day. Most other RVM's floating around on the web are either abandoned or not in a usable stage at this point. Another reason is because they are the only RVM's I actively use.
I ran these tests on an eMac G4 (1.25GHz), running Mac OS X 10.4.9 with 1024MB of memory. The ruby runtime I use ruby 1.8.6 (2007-03-13 patchlevel 0), which comes from the MacPorts repository. For JRuby, I used Ruby 1.0.0RC2, on top of Apple's JVM 1.5.0_07 with a heap size of 512MB. The Rubinius version I used is built from revision 1225 from their Subversion repository (built on 05/31/2007).
First, let me give you a short overview of the tests I used to compare the different Ruby runtimes. The following list of tests gives you an idea of what the test focuses on.
| Test | Description | \Focus |
|---|---|---|
| fibonacci | Calculate the n'th number in the fibonacci sequence, without dynamic programming or other optimizations. | \Method calls, recursion |
| hello world | Print 'hello world!' and exit. | Startup time |
| pi | \Calculate an accurate estimate of the number π. | Floating point performance |
| primes | Calculate the n'th prime number. | Fixed point math performance |
Here are the raw results, the actual measurements of the test.
| Test | Ruby | JRuby | \Rubinius | Rubinius (precompiled) |
|---|---|---|---|---|
| fibonacci | \47,56 | 24,67 | 9,36 | 8,49 |
| hello world | 0,01 | 3,14 | 0,72 | 0,35 |
| pi | 4,45 | 13,92 | 77,82 | \76,03 |
| primes | 16,9 | 21,1 | \53,37 | 51,71 |
| Time taken (in seconds), lower is better. | ||||
These results need to be interpreted. It's not fair comparing just the measurements without making a few adjustments. First of all, including the startup time in the results is unfair when comparing these Ruby implementations. JRuby runs on top of the Java Virtual Machine, which takes forever to load on my aging G4 eMac. And even if it didn't, JRuby and Rubinius compile ruby code before executing it, which gives them a speed boost in the long run but might slow them down in these short tests. So lets see what happens when we substract the startup time from the results, and compare every result to Ruby's performance:
| Test | Ruby | JRuby | \Rubinius | Rubinius (precompiled) |
|---|---|---|---|---|
| fibonacci | \100% | 45% | 18% | 17% |
| pi | 100% | 252% | 1801% | \1768% |
| primes | 100% | 107% | 313% | 306% |
| Relative duration, lower is better. | ||||
It's important to note that both JRuby and Rubinius are still a work in progress. Their main focus has been compatibility and not speed so far. However, it is clear that their virtual machine approach is ready to end the reign of the Ruby interpreter.
In the first test, Rubinius shows us that its architecture is by far the fastest at calling functions and recursion. However it loses so badly at the other tests, something has to be done to improve it's mathematical performance.
JRuby is clearly more mature, and is still able to outrun the ruby interpreter by over 100% in the first test. It's floating point performance isn't as swift as it could be but it's not as bad as with Rubinius. And you don't use floating points that often, do you?
Ruby is still the winner if you look at the overall results, but it's taking serious damage in the first test, indicating it's not nearly as good at looking up objects and calling methods as either virtual machine. It would be interesting to see how YARV performs at these tests, which is set to replace the interpreter by the end of this year.