|
Replies:
14
-
Last Post:
Mar 15, 2008 12:40 PM
by: jmicheli
|
|
|
|
|
|
|
Why are Remote method calls so expensive?
Posted:
Jan 23, 2008 10:46 PM
|
|
|
I have a Remote EJB, and a WAR callig it. The WAR and EJB are in the same container, but the WAR is not in the EJBs EAR, so it needs to be a remote call.
The result of the call is large -- a List of 2000+ items. Thankfully, this is cached in the client WAR, but we need to do this one time load.
But when the method returns the resulting list, my heap SURGES over 200M for this single call.
I added a little blurb of code that simply writes the List out to a file via an ObjectOutputStream, and the final file is ~1.6M in size.
Now, all of that 200M in the surge is garbage, but it's some big garbage. If I reduce my heap size, this call will give me an Out of Memory exception, so whatever it is doing with that memory, it's doing it all at once.
Why are these remote methods so expensive?
|
|
|
|
|
|
|
Re: Why are Remote method calls so expensive?
Posted:
Jan 24, 2008 12:43 AM
in response to: whartung
|
|
|
Why don't you create an Enterprise Application and then include your Web Project and your EJB Project.
In that way you can access your EJB through local calls from your Web Project...
I think your question is good, and I would luv to know more about how the EJB container/memory (also stateless Vs. stateful memory + clustering/LB) works...
Any good articles on that?
Cheers,
/TheSlingBlade
|
|
|
|
|
|
|
|
|
Re: Why are Remote method calls so expensive?
Posted:
Jan 24, 2008 8:18 AM
in response to: whartung
|
|
|
I deploy the WAR(s) seperately because it gives me finer control over deployment. It lets me make changes and update one aspect of the overall system without redeploying the entire system. It also speed development by just being able to redeploy the WAR that I'm working on instead of the entire deployment.
I did some testing. First, I tried implementing Externalizable on my core class. It was a very naive implementation that simply did a writeObject on everything. No real surprise, but this didn't do much. But it did show me getting the OOM error within my write method.
So what I did next was rather than return the List of objects, I instead serialized it on the server to a byte array, returned THAT, and then deserialized on the client.
This had two benefits.
1) It eliminated my memory spike. On a fresh server the spike goes from over 200M down to 30M. On a running server (after a redeploy), it seems to be a larger surge (40-50M). But it all GC's in the end.
2) It's 5 times faster this way. The old way took 25s the new way took 5s.
I'm fine with this solution, as it's a specific pain point within the system (I've been casually looking at ways to kill that 25s for a while now). Of course, I'm disheartened that I had to do this myself.
I suspect it's the "fault" of the CORBA IIOP protocol that I believe is used for remote EJBs. I recall waaay back trying an experiment with Weblogic where I switched from their internal T3 (I think) protocol to CORBA. We had a similar situation that downloaded a large blob of data for caching on the web tier. The CORBA version was MUCH slower than T3. Thankfully we were eventually able to make T3 work for us.
Is there any thought in adding a proprietary but more efficient protocol for Remote EJBs to GF much in the spirit of Weblogics T3? Or has the drive toward web services put a stake in the heart of the Remote EJB problem space? (If I had the time I'd send this blob over a web service just to see what happens, alas I don't. I'd like to think it would not be pretty.)
|
|
|
|
|
|
|
|
|
Re: Why are Remote method calls so expensive?
Posted:
Jan 24, 2008 10:16 AM
in response to: whartung
|
|
|
What happens if you call this EJB for the second time? Which version of GF are you using?
|
|
|
|
|
|
|
|
|
Re: Why are Remote method calls so expensive?
Posted:
Jan 24, 2008 2:45 PM
in response to: danielrohe
|
|
|
But it's not using RMI, I think it's using IIOP.
I was actually able to get a quick test run with a WebService. (NB, GF, and JAX-WS ftw!).
WebServices can only marshal object trees, and not graphs, so I had to break some relationships for my test.
But I was shocked to find that the WS was only marginally slower than my serialized byte array, and it used LESS memory than the byte array. The WS beat the tar out of the stock Remote EJB call.
I thought this was very interesting.
Mind, I'm not going to be switching over any time soon, but it'll be something to think about in the future.
|
|
|
|
|
|
|
|
|
Re: Why are Remote method calls so expensive?
Posted:
Jan 24, 2008 6:48 PM
in response to: whartung
|
|
|
Let's sort thing out for a fair comparison: Did you compare a vanilla EJB call with your list of objects with a vanilla JAX-WS call, or did you compare your custom (and quite extravagant) server-side serialization into a byte array, followed by a stock remote EJB call of the whole byte array, followed by a client-side deserialization of your byte array into objects? Your extra-steps both on the server-side and the client seem overly complex, and for sure they result in additional CPU cycles and memory usage (both on the server- and client-side). RMI over IIOP is a quite an efficient protocol compared to WS-*, although IIOP transfers more information per call (DII - dynamic invocation interface - the client does not need to know the full structure of the parameters/return values to unmarshal the call).
|
|
|
|
|
|
|
|
|
Re: Why are Remote method calls so expensive?
Posted:
Jan 24, 2008 9:06 PM
in response to: scotty69
|
|
|
I started with a bone simple Session Bean with a remote interface. The return type is a List<SomeClass>, the class consists of 30 fields, that are a mix of Strings, Integer, and BigDecimals plus 2 internal Lists with their own domain objects, which are basically simple beans with a few properties each. Each of the items (1570ish of them) have, perhaps, 3 or 4 child items each.
So, that's the data.
As I said, it's a simple Remote EJB. Make the call, it loads from the DB, and sends the List back.
The next scenario is rather than sending the List back, I return a byte array which is created with a writeObject of the resulting list to a ByteArrayOutputStream. Then I readObject in the WAR on subsequent ByteArrayInputStream. As I also mentioned, the size of the byte array is ~1.6MB.
For the third scenario, I create a new single method Session Bean and made it a WebService. Thats the only annotation I used -- @WebService along with @Stateless. The function calls the backend to fetch the data, iterates across it to break up the parent/child links (since a webservice can only send a tree of objects and not a graph), then return the tweaked list.
I then created a very simple web service client that's pretty much 100% wizard created from NetBeans. All I changed was adding simple time code and having it print the results size, instead of the result.
I deploy my projects, and restart the server.
Then I run my test.
For the web versions, I have a break point where the code checks for the cache (since that's what this code does, is load a cache). This allows me to easily reset the cache, and this happens before the big load. The web service client doesn't cache, so I just run it again.
I run the test twice, once after the fresh server restart, and then again. The first run times include any JPA caching or startup for the queries, and any other one time costs.
I monitor the memory using jconsole, and basically eyeball it based on the values it shows on the graph. I also force a GC from JConsole before each test.
I'm running:
Server 9.1_01 (build b06-fcs)
Java 5 on a Mac Pro running Mac OS 10.4
Run with normal EJB Remote
First Run: 25s 160MB surge
Second Run: 25s 240MB(!!) surge
Run with byte array EJB Remote
First Run: 7.1s 45MB surge
Second Run: 2.6s 48MB surge
Simple WebService
First Run: 7.6s 36MB surge
Second Run: 2.8s 8MB surge
So, those are the numbers I am seeing. I did not monitor CPU at all. I'm not paging.
For now, I'm sticking with my byte array version, as it has the least impact to my system. This change will let me lower the heap size on my production instance. The system normally puts along with a heap of around 90MB, but I have it set to 384MB because this call was (apparently) smashing in to a smaller heap of 256M.
I am, frankly, shocked (SHOCKED!) by these numbers. I am shocked at the time and memory impact of the Remote EJB, and I am AMAZED at the Webservice performance.
I'd LOVE to hear an explanation behind this.
|
|
|
|
|
|
|
|
|
Re: Why are Remote method calls so expensive?
Posted:
Jan 25, 2008 6:54 AM
in response to: whartung
|
|
|
I would try one more tweak to this scheme: wrap your ByteArrayOutputStream with a GZIPOutputStream, then with an ObjectOutputStream. This will probably shrink your 1.6 MB buffer significantly, especially if you have a good bit of text data in the stream. Then on the other end, wrap your ByteArrayInputStream with a GZIPInputStream, then an ObjectInputStream, to decode. I think that even lengthy computation on each end trumps fast network transmission every time, because it will nearly always be faster.
|
|
|
|
|
|
|
|
|
Re: Why are Remote method calls so expensive?
Posted:
Jan 25, 2008 9:43 AM
in response to: jkeatley
|
|
|
I actually tried that, but didn't notice any real difference. If anything it was slower. I think since I'm talking to localhost, there's less overhead for the socket during the transfer.
|
|
|
|
|
|
|
|
|
Re: Why are Remote method calls so expensive?
Posted:
Jan 24, 2008 10:47 PM
in response to: whartung
|
|
|
> But it's not using RMI, I think it's using IIOP.
>
And that shows your ignorance (which is understandable if you've not delved into the topic) about how EJB works.
EJB uses RMI on top of IIOP. IIOP provides the transport layer, RMI the protocol layer, of the network traffic.
|
|
|
|
|
|
|
|
|
Re: Why are Remote method calls so expensive?
Posted:
Jan 25, 2008 9:44 AM
in response to: jwenting
|
|
|
Ah, well, that may well be. I'm certainly not a protocol expert.
Either way, there's something, somewhere, that makes this process prettyinefficient in this case.
|
|
|
|
|
|
|
|
|
Re: Why are Remote method calls so expensive?
Posted:
Mar 14, 2008 8:42 PM
in response to: danielrohe
|
|
|
Hi Danielrohe,
Excuse me for sending you this email as I don't know you, but maybe you could help me.
Let me say first that it's the first time in 35 years in computer software I ask help on a forum, but now I'm completly lost. I have done all I could, and nothing works.
I have a terrible problem with Glassfish on Mac OS X 10.4.11.
I downloaded Glassfish yesterday, and I am absolutely unable to make work all the samples which use servlets or advanced functionalities.
The only samples which work are those using jsp.
I built the samples using ant.
The samples build without errors, the installation seems OK, but when I to acess the URL nothing happens at all, I have in the browser the contents of the base directory of the war archive.
There is absolutely no error message, I checked all the logs and files I could imagine, and I couldn't see a single trace of the problem.
In this case, the page displays immediately in the browser, as if the server did not loaded anything.
I checked all the XML files, and I could not find anything abnormal.
Of course, I reinstalled Glassfish threee times, and nothing has changed.
I even tryied to installed Tomcat servlet components in the ext directory, nothing changed.
My Java installation is fine, I use it continuously to build projects using Eclipse.
I must solve this problem for an important work I must deliver to a client very soon.
Could someone help me in this very bad situation ?
I have posted this message 3 other persons as well. Maybe someone will take the time to help me.
I thank you a lot in advance
Jacques
|
|
|
|
|
|
|
|
|
Re: Why are Remote method calls so expensive?
Posted:
Mar 15, 2008 11:59 AM
in response to: jmicheli
|
|
|
Hi Jacques. I think you are more likely to get a quick response if you send it with a diferent subject line that is more specific to your problem. - eduard/o
|
|
|
|
|
|
|
|
|
Difficulties with setup of Glassfish on mac OS X 10.4.11
Posted:
Mar 15, 2008 12:40 PM
in response to: pelegri
|
|
|
Hi Eduardo,
In fact, I'm new on forums (I know it may sound strange ), and I don't know exactly how to send a message in a forum. I only know how to reply.
Until now, I've always considered I'll be able to find the solution to my problems by myself and information I could gather on the Internet and in books.
I've been working in this industry for more than 35 years now.
By the way, I solved my problem, by simply using the Java EE 5 environment, and then replicating the apps in Glassfish.
With Java EE 5, even Glassfish works normally.
Of course, the samples which did not worked built with Glassfish don't work either in Sun EE 5 application server.
But I did slow down my tests since I got a result with EE 5, some hours ago, and I still don't know the reason of my troubles with the Glassfish build environment.
I'll take the time to investigate, maybe I'm not the only person to have experienced difficuties of that kind.
If you wish, I'll keep you informed.
Thank you for your answer, and Best Regards,
Jacques
|
|
|
|
|
java.net RSS
