Oracle Coherence using POF, without a single line of code
People developing distributed Java applications know the importance of wire formats for objects. Native Java serialization has only one advantage—it is built in. It is relatively slow, not very compact, and has other quirks. Starting with version 3.2, Oracle Coherence is offering its own proprietary binary wire format for objects—POF serialization. POF is not only cross platform, but also much more compact and faster compared to built-in serialization. Both compactness and speed are extremely important for data grid application. The only disadvantage of POF is that you should write custom serialization/deserialization code for each of your mobile objects. Not only domain objects stored in cache should have serializers, but also entry processors, aggregators, etc. The amount of code you have to write may look daunting and force you to stick with built-in Java serialization.
But there is a simple way to get best of both worlds. In a recent project I have implemented a generic POF serializer. It uses reflection and doesn't require any changes in code, although you still need to register classes in coherence-pof-config.xml. Still it offers the advantages of the POF format – compact object size and performance. While performance is a bit degraded from using reflection (but still much faster than Java serialization), the sizes of serialized objects are similar to a handmade POF serializer.
Some people believe that reflection is slow. I have performed simple speed tests between Java reflection, a handmade POF serializer, and a reflection-based POF serializer—a very simple single threaded test doing serialization and deserialization of an object in a loop.
- Java serialization – 8K ops/sec
- Handmade POF – 46.4K ops/sec
- Reflection based POF – 35K ops/sec
Sweet! I got 4 times speed up, 8 times reduced size just by writing 5 lines in coherence-pof-config.xml. Numbers may be different for different application but the trend is obvious.
And one more good thing—you can combine a reflection-based POF serializer with handmade ones. You can start with using generic implementation for all objects, and later write a few handmade serializers to reclaim ~24% lost on serialization for hot objects.
If you are interested you can look at the implementation here, available under the Apache 2.0 license.
Labels: coherence, data grid, POF, ~Alexey Ragozin
at
3:36 AM
7 Comments:
Nice post, thanks for sharing this! You might be interested to look at this approach as well from Rob Varga:
http://coherence.politext.info/pof-serializer-generator-java-apt-plugin
Thanks for a link Patrick, looks interesting. But I really like to keep my domain objects clear from product specific annotations.
From the other side, ability to generate .NET and C++ would be great.
I will look forward for project evolution.
Have you tried this with some real scenario? Reflection will cache method lookups and the class loading will be done only in the first loop. by testing it with only one object/class you almost discard that overhead.
This post has been removed by the author.
Hi ZeoS.
I use this implementation with about 50 classes (number of classes was a motivation to write generic tool in first place). Reflection in java can be extremely fast if you do it right. JVM generates code for relflection accessors, which is further optimized by JIT like your own code.
Creating an accessors is expensive operation, of cause, and any reflection based code should carefully cache generated accessors.
In my implementation a class serialization scheme is generated on first attempt to serialize/deserialize object of given class. Such serialization scheme stores all field accessors needed and they are generated only once for each class.
In other words, my implementation has own reflection cache to avoid lookup overheads.
Where can I find information about this "lmap" tool you talked about? I know about "jhat" for heap analysis but not not lmap...
Googling ling did not find anything...
I assume that was jmap, not lmap, not sure where have you find reference to lmap in this post.
Post a Comment
Subscribe to Post Comments [Atom]
Links to this post:
Create a Link
<< Home