Update: it seems that the JITting process has improved quite a bit these last years (which is to be expected), and the differences are much smaller (and in some cases in favor of Java). Also, the discussion below is to be understood in the context of trigonometric functions rather than floating point operations in general.
In the same podcast I referenced in my previous entry (Java Posse #222) I heard the following affirmation (this is from memory, so it may not be 100% correct): “Performing floating point calculations in Java is slower than native code by a factor of 6-7!”. If I recall correctly the biggest problem was with some trigonometric functions.
This came as quite a surprise for me, given the consensus of JIT-ed code being very close to “compiled” code in performance (in the 90% range). This is why I considered the possible performance gains by things like Google Native Code less significant compared to its problems (dependence on the x86 platforms, possible security risks, etc).
An other field where I would have considered Java to be perfect before this information was the World Community Grid / BOINC project. Now, I’m all for using the extra power of computers for solving large problems, however the model used by BOINC is a little problematic (IMHO) from a security standpoint: they repeteadly download both data and executables for each project that you take part in (I observed this because of a whitelisting solution that I use which prompts whenever an unknown executable is launched). I didn’t look into the safeguards (if any) BOINC has, but given that you run native code, I don’t know if there is much they can do. This is why I would have considered Java to be perfect for the job.
Looking for some more concrete results (because I’m lazy to do my own benchmarking :-)), I come over this site: JavaNumerics, where they have a nice Benchmarks section. Unfortunately they don’t seem to have a comparison with native code, only different JVM’s on different platforms. I also found this paper (warning! PDF) which claims that numerical computation in Java is 50%-60% slower than the equivalent compiled code. This is quite a lot and a big waste for distributed projects.
The (sad) conclusion seems to be: Java doesn’t seem to be “there” yet for numerical calculations (just as it doesn’t seem to be “there” with JavaFX either :-(). Hopefully this will be resolved sooner rather than later and we can dedicate our computer to things like computing rainbow tables with an extra level of safety.
6 responses to “Java numerical calculation benchmark”
Hi,
About the quote from Java Posse 222 – you are correct that trig (or in fact transcendental) functions are the problem. In fact, in practice, much of the floating point performance is as good as or even in some cases better (for tight loops that can be optimized by hotspot) in Java as that of C/C++. However transcendental functions are much slower.
The situation has improved with Java 6, but performance is still not on a par with other languages. The problem is that the hardware functions on x86 hardware is not considered accurate enough for Java. While there is some truth to this (the routines are flawed on x86), the fact remains that Java is the only mainstream language that refuses to let the developer decide whether to take the faster, less accurate implementation in the hardware, or the slower more accurate IEEE routines in software.
I have rallied against this decision for a long time but have made little headway (the worst case I have seen is less than one part in a million innacuracy – not something that can even be observed in, say, a coordinate transformation).
Cheers
Dick Wall (Java Posse)
This Java vs C++ benchmark shows that things are not that bad.
This is abselutely BS! Really, it is. If anyone have bothered to run Scimark2 from http://math.nist.gov/scimark2/index.html
then you will find that in -client mode Java is approx 65% of -O3 optimized pure C code. In -server mode Java is 98% ov C code. This is on the "small" model, ie probably chached.
On the "large" model (uncached) Java is 82% of C code in -client mode. In server mode Java is 94% of the C-code. They use Mflops.
So, in a practical application Java will have about 95% of the speed of optimized C code (not C++) in pure number crunching performance.
The problem was with the trigonometric functions. From the descriptions it seems to me that only one test (FFT) uses trigonometric functions, so the test suite as a whole isn't a very good measurement for the performance of the trigonometric functions.
I would love to be proven wrong, but Dick is knowledgeable guy who generally knows what he's talking about.
Well as quoting the lead post "Performing floating point calculations in Java is slower than native code by a factor of 6-7!"
This is not true. The truth is that floating point calculation in Java is around 95% of native optimized (O3) C code (I stress C code, as C in numerical calculations is normally faster than C++ due to all the overhead).
But OK, the FFT timing in the test show that Java speed is 83% of C in the "small" model and 65% of C in the "large" model. So it is true that trig functions are slower in Java when using the Java Math lib.
Java vs C for SOR is 120% and Monte Carlo simulation is about 195 % (2 tims faster in Java)
From the headline "point out the half-truths and misinformation which floats around…" – well, then do this by at least running the only real test that compares Java to C head to head instead of adding more half-truths and misinformation. That is all 🙂
@Anonymous: fair enough, I didn't realize that the posting can be misunderstood as saying that all of the floating point operations are slower in Java. To remedy that, I added a disclaimer at the beginning of the article.
At the time I wrote the original posting I did search around for some tests related to floating point performance in java (as compared to C), however I didn't find any (I didn't know about Scimark2).
Thank you for drawing my attention to the problem and hopefully the comments and the disclaimer at the beginning of the article will create a more realistic picture to the future readers.
PS: I have a "you comment, I follow policy", which means that the links in the username for the comments (not the ones in the body) are linked without nofollow. So next time maybe you can specify your blog 🙂
PS PS 🙂 There is also a selfish reason for asking you to give homepage: frequently I found the commenters blogs/twitter accounts/homepage/etc interesting.