Comments on Grid Designer's Blog: Scalability Benchmark of Monte Carlo Simulation on...

Hi Max, Very interesting and cool. Is there any ...

2009-03-31T08:22:00.000-07:00

Hi Max,
Very interesting and cool. Is there any chance that the team plans to release the code or a howto (or have they already?).

thanks...

We think, this is possible. GoGrid should have all...

2008-08-27T01:29:00.000-07:00

We think, this is possible. GoGrid should have all the required features to provision and start nodes. GridGain can also be used on Windows 2003, at least as a limited testing edition right now. Some of MQ solutions should run on Windows too. At least it will be ActiveMQ.

So, the only thing that is needed is to refactor our test harness to use GoGrid API. We can share the source codes if you want.

However, if you use OLE automation, it may not be very comfortable to run them from Java environment. So, you may consider using Microsoft HPC solutions.

We can also arrange a conf call to discuss the details if you find it useful.

Can the same test be performed in an all Windows e...

2008-08-21T18:07:00.000-07:00

Can the same test be performed in an all Windows environment using something like GoGrid instead of EC2?

http://www.gogrid.com/

I need to gridify an exhaustive optimization search whose fitness function is the result of an OLE automation call upon an object available only on Windows.

2 James Strachan:We decided to use OpenMQ for a nu...

2008-08-14T04:12:00.000-07:00

2 James Strachan:
We decided to use OpenMQ for a number of reasons with both 4.x and 5.x versions of ActiveMQ.
See, for example, this forum: http://www.gridgainsystems.com/jiveforums/thread.jspa?threadID=343&tstart=0

BTW details of the ActiveMQ instability are here -...

2008-08-14T02:02:00.000-07:00

BTW details of the ActiveMQ instability are here - basically using the latest 5.x releases fix this

Hi Max. I've done a spotlight on this entry at Th...

2008-08-10T12:27:00.000-07:00

Hi Max. I've done a spotlight on this entry at TheAquarium [1]. We are always interested in adoption stories for our OpenSource software [2], please contact me at "stories at sun dot com" if you are interested.

- eduard/o

[1] http://blogs.sun.com/theaquarium/entry/openmq_in_512_node_setup
[2] http://blogs.sun.com/stories

2 amalter:We got "java.lang.OutOfMemoryError: unab...

2008-08-07T09:26:00.000-07:00

2 amalter:
We got "java.lang.OutOfMemoryError: unable to create new native thread" and other issues with ActiveMQ on 50+ nodes. The best results (~100 nodes) were reached by tuning ActiveMQ 4.1.2. Anyway OpenMQ works much better with GridGain.

2 Anonymous:
Amazon promises to release their persistent storage on next week. Currently it is not publicly available.
We did not use availability zones in our tests. Putting all nodes into the same zone would insignificantly reduce our expenses (as well as network latencies), but real-world applications are likely to use different availability zones for reliability purposes.

2 AnjanBacchuDev:
Actually, there are 5 options. We used Small Instances for Worker Nodes and Large Instances for OpenMQ and Head Node.

2 Manik Surtani:
Amazon doesn't plan to support IP multicast. That's why we used JMS.

Interesting stuff. Do you have any idea when EC2 p...

2008-08-07T07:14:00.000-07:00

Interesting stuff. Do you have any idea when EC2 plans to support IP multicast? It would be interesting to see how things scale if you were to use JBoss Cache as your "persistence" library (in-memory storage, distributed across the grid) but you would need IP multicast for an optimal JBC configuration.

Some thoughts hereGreat work Max. Keep up the insi...

2008-08-06T22:45:00.000-07:00

Some thoughts here
Great work Max. Keep up the insights!

Hi there, I'd like to repeat what you did for the ...

2008-08-06T21:49:00.000-07:00

Hi there,

I'd like to repeat what you did for the learning experience.

I will be sending you an email asking for the source code and whatever else that you can share so that I can repeat the experiment and, more important, the findings.

it will be useful to your readers to know what type of EC2 instances you used(there are 4 varieties of instances, AFAIK).

thank you,

BR,
~A

I thought that AWS announced a new solution to hel...

2008-08-06T14:47:00.000-07:00

I thought that AWS announced a new solution to help with the EC2/S3 roughness. Some sort of high-speed persistent storage that can be mounted in the filesystem.

No, I don't mean a vendor-supplied solution either.

Also did you make use of their new EC2 geographic locality feature whereby you can specify that you prefer instances to be in the same datacenter?

This is really interesting stuff from a number of ...

2008-08-06T08:40:00.000-07:00

This is really interesting stuff from a number of perspectives. I like seeing that GridGain scales with a known use-case. It's interesting to see what heights you can push EC2, and you gave all the underlying components a good stress test.

Can you detail a little more what issues you had with ActiveMQ. We explored the project in it's 4.x incarnation and found some stability issues also. However, I had heard 5.x was much better and we again were going to start looking into it. If anybody has some real world experience, I'd love to hear.

Anyway, thanks for the interesting technical story. I saw a bunch of people on DZone were getting on you guys and I don't think they give credit due when you guys give back like this. Yes it's a GridGain plug, but I happen to like your product, and you also pushed EC2 and OpenMQ to it's limits. Additionally, this benchmark provides a pretty damn good tutorial for anybody creating a highly coordinated distributed application of any kind on EC2.

Bravo!

-Adam Malter