I didn’t have nearly as much faith in Objective-C. I figured that Java would beat it by at least 2x. So much thought has been put into Java VMs compared with the Objective-C runtime it would actually be a shame if Objective-C won this competition. The benchmark I grabbed was a benchmark we use to test JRockit against HotSpot. Its does string comparisons that result in the different cases. Universally people thought this would give Obj-C an edge because strings are so low level in C. However, that is misleading, because a string in Obj-C isn’t low level at all. Its a class, NSString, just like it is in Java. Well, here is the result of the benchmark on a 1.42ghz PowerMac G4 (similar results on my 1.5ghz Powerbook):
That’s a nice 6.4x speedup for applications that use a lot of strings! Here is the binary you can run right off this site:
BenchmarkApp.zip
Here is the Xcode project so you can see the source and build it yourself:
Benchmark.zip
I would normally now talk about how this whole thing took like 20 minutes to develop, including the user interface, but the only people that would care are people that already know how productive development is when using Xcode + IB + Cocoa. You might also take note that you can put both Obj-C and Java in the same executable without any trouble, just make sure you start with a Cocoa-Java application. Oh, one more thing, I also tried compiling it optimizing for speed instead of space — it ended up with the same benchmark result except the executable was smaller!?
UPDATED
After that experiment I talked to a Cocoa person and he said that many times people will go back and optimize all their code for manipulating strings and things like that using pure C code. So I rewrote the benchmark again, this time using standard ANSI C. Its a straight console application and here are the results:
/Users/sam/Projects/Xcode/Benchmark:> gcc -o benchmark benchmark.c; time ./benchmark
53678.991685
real 0m18.640s
user 0m17.480s
sys 0m0.200s
Without optimizations, the C application is actually a lot slower than the Java application. Lets add some optimizations:
/Users/sam/Projects/Xcode/Benchmark:> gcc -O3 -o benchmark benchmark.c; time ./benchmark
69006.049967
real 0m14.502s
user 0m14.230s
sys 0m0.050s
Still lags slightly. I’m sure there is probably some command line thing that could get the performance to exceed the Java version, however, I think my point is made. Java is really, really quite fast. So please Apple, stop using an inferior development laguage. Here is the C file:
benchmark.c.zip
@
Pingback: Android Dalvik VM performance is a threat to the iPhone | java rants
Unfortunately, what you're comparing here isn't the language; it's the efficiency of the string comparison routine in the relevant library; in fact, for Java I'm not even sure you're measuring *that* properly. Removing the actual comparisons from the code and replacing them with 1, 0 and -1, on a Mac Pro I get:
Java: 345065.5624568668 compare tests / second
C: inf (it's too fast for your timing method and you'd have to change the code!)
ObjC: 1644439.299682 compare tests / second
Also, I noticed that the numbers for Java really didn't change when I removed the comparisons, which makes me think that perhaps the Java compiler or JIT is optimising away the calls to the compareTo() method. You might say, well then it's faster… but of course that's only true for this synthetic benchmark, not in the real world. It's pretty clear from the numbers here that C and ObjC are faster languages, since all we're measuring here is function call/method overhead.
Interestingly I just fiddled with the Java code a bit and got it going faster, without the comparisons (up to 8.3×10^7 a second).
Still, as I say, I think your benchmark is of dubious validity; I'm not enough of a Java fiend to know how to stop the compiler/JIT from optimising away parts of your code that should really be being timed, and you can't really claim that Java is faster on the basis that it can optimise away the computation you were trying to time.