NetBeans Forums

 FAQFAQ   SearchSearch   MemberlistMemberlist   RegisterRegister   ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 
  

Why is the profiler so much slower than, e.g., YourKit?

 
Post new topic   Reply to topic    NetBeans Forums -> Profiler Users
View previous topic :: View next topic  
Author Message
Scott Wells
Posted via mailing list.





PostPosted: Wed Apr 29, 2009 10:26 pm    Post subject: Why is the profiler so much slower than, e.g., YourKit? Reply with quote

I'm a big fan and advocate of the NetBeans profiler, but I'm having trouble
getting other folks to use it when we have a small set of licenses for YourKit
at work because of its performance when trying to profile large Enterprise
applications.

In all honesty, I've found myself using YourKit over NetBeans for profiling
recently just because YourKit instruments even HUGE applications almost
immediately whereas NetBeans can take LONG amounts of time before you can begin
gathering results. For example, we have an Enterprise app that's deployed into
JBoss and uses Spring, Hibernate, and a ton of other third-party libraries in
addition to our own large product code base. When I connect YourKit I can begin
using CPU sampling and/or tracing immediately. Sampling (10ms sampling times;
no invocation counts) adds very little overhead; tracing is comprehensive and
does seem to lengthen things about 6x (observed) over sampling. When I connect
NetBeans 6.5 (latest patch-level), even with 10ms sampling it went into a black
hole instrumenting classes and I wasn't able to start using the application
after 30 mins of continuous instrumentation. I finally just killed the server,
switched back to YourKit, and very quickly found the bottlenecks. For what it's
worth, I hadn't applied any filters on instrumented classes or entry points in
YourKit at all in this example, and I had actually applied pretty a few filters
in NetBeans Profiler (e.g., exclude core Java classes) in an attempt to speed
things up.

I've tried eager, lazy, and full instrumentation, sampling vs. tracing, etc., in
NetBeans Profile, plus I've tried to reduce the classes that need to be
instrumented. None of this seemed to help. Please let me know if there's
something I'm doing fundamentally wrong when trying to use this tool. Our
company is about to go purchase additional licenses for YourKit given how
productive that tool makes us when trying to analyze performance. I'd love to
be able to recommend a free alternative that provides similar analytical
capabilities, but if it can't get to the same answers any faster than I've found
with these experiments, I can't really make that recommendation.

Thanks in advance for any tips,
Scott



---------------------------------------------------------------------
To unsubscribe, e-mail: address-removed
For additional commands, e-mail: address-removed
Back to top
Jess Holle
Posted via mailing list.





PostPosted: Wed Apr 29, 2009 11:11 pm    Post subject: Why is the profiler so much slower than, e.g., YourKit? Reply with quote

I can't explain NetBeans' lackluster tracing and instrumentation performance vis-a-vis YourKit.

The sampling discrepancy is easy to explain, though -- NetBeans' profiler has no "sampling only" mode. Couple this gap with slow instrumentation (even for tracing) and a hard limit on the number of methods that can be instrumented and NetBeans' profiler runs out of gas for large apps that YourKit has no issues with.

If NetBeans added a sampling-only mode and defaulted to this initially, it would be hugely more useful. You could then examine potential hotspots from the rough data and add instrumentation on a targeted basis from this data. A good UI to make it dead simple to add such targeted instrumentation based on and directly from the rough data display would also really help.

I've been asking for this enhancement to the NetBeans profiler for years (and filed an enhancement issue accordingly). There's some sign of motion in this direction -- NetBeans 6.7 can perform lightweight sampling profiling on itself, which is really nice. In terms of NetBeans or VisualVM profiler functionality, however, there's been absolutely no improvement in this regard. I'd really hoped to see it in NB 6.7 given the self-sampling feature -- but I guess I was overly optimistic.

This gap forced me into YourKit.

--
Jess Holle

P.S. NetBeans backtraces cannot be made to reflect total time and always reflect self time (even though they are labeled simply "time" whereas everywhere else self time is labeled as "self time"), which is another critical gap for getting to the bottom of issues. I've filed that one as well quite some time back. If both issues were fixed I'd have never needed YourKit (which is a very nice product).

Scott Wells wrote:
Quote:
Quote:
I'm a big fan and advocate of the NetBeans profiler, but I'm having trouble
getting other folks to use it when we have a small set of licenses for YourKit
at work because of its performance when trying to profile large Enterprise
applications.

In all honesty, I've found myself using YourKit over NetBeans for profiling
recently just because YourKit instruments even HUGE applications almost
immediately whereas NetBeans can take LONG amounts of time before you can begin
gathering results. For example, we have an Enterprise app that's deployed into
JBoss and uses Spring, Hibernate, and a ton of other third-party libraries in
addition to our own large product code base. When I connect YourKit I can begin
using CPU sampling and/or tracing immediately. Sampling (10ms sampling times;
no invocation counts) adds very little overhead; tracing is comprehensive and
does seem to lengthen things about 6x (observed) over sampling. When I connect
NetBeans 6.5 (latest patch-level), even with 10ms sampling it went into a black
hole instrumenting classes and I wasn't able to start using the application
after 30 mins of continuous instrumentation. I finally just killed the server,
switched back to YourKit, and very quickly found the bottlenecks. For what it's
worth, I hadn't applied any filters on instrumented classes or entry points in
YourKit at all in this example, and I had actually applied pretty a few filters
in NetBeans Profiler (e.g., exclude core Java classes) in an attempt to speed
things up.

I've tried eager, lazy, and full instrumentation, sampling vs. tracing, etc., in
NetBeans Profile, plus I've tried to reduce the classes that need to be
instrumented. None of this seemed to help. Please let me know if there's
something I'm doing fundamentally wrong when trying to use this tool. Our
company is about to go purchase additional licenses for YourKit given how
productive that tool makes us when trying to analyze performance. I'd love to
be able to recommend a free alternative that provides similar analytical
capabilities, but if it can't get to the same answers any faster than I've found
with these experiments, I can't really make that recommendation.

Thanks in advance for any tips,
Scott



---------------------------------------------------------------------
To unsubscribe, e-mail: address-removed ([email]address-removed[/email])
For additional commands, e-mail: address-removed ([email]address-removed[/email])


Back to top
Tomas Hurka
Posted via mailing list.





PostPosted: Thu Apr 30, 2009 10:50 am    Post subject: Why is the profiler so much slower than, e.g., YourKit? Reply with quote

Hi Scott,

On 29 Apr 2009, at 19:53, Scott Wells wrote:

Quote:
I'm a big fan and advocate of the NetBeans profiler, but I'm having
trouble
getting other folks to use it when we have a small set of licenses
for YourKit
at work because of its performance when trying to profile large
Enterprise
applications.

In all honesty, I've found myself using YourKit over NetBeans for
profiling
recently just because YourKit instruments even HUGE applications
almost
immediately whereas NetBeans can take LONG amounts of time before
you can begin
gathering results. For example, we have an Enterprise app that's
deployed into
JBoss and uses Spring, Hibernate, and a ton of other third-party
libraries in
addition to our own large product code base. When I connect YourKit
I can begin
using CPU sampling and/or tracing immediately. Sampling (10ms
sampling times;
no invocation counts) adds very little overhead; tracing is
comprehensive and
does seem to lengthen things about 6x (observed) over sampling.
When I connect
NetBeans 6.5 (latest patch-level), even with 10ms sampling it went
into a black
hole instrumenting classes and I wasn't able to start using the
application
after 30 mins of continuous instrumentation. I finally just killed
the server,
switched back to YourKit, and very quickly found the bottlenecks.
For what it's
worth, I hadn't applied any filters on instrumented classes or entry
points in
YourKit at all in this example, and I had actually applied pretty a
few filters
in NetBeans Profiler (e.g., exclude core Java classes) in an attempt
to speed
things up.

I've tried eager, lazy, and full instrumentation, sampling vs.
tracing, etc., in
NetBeans Profile, plus I've tried to reduce the classes that need to
be
instrumented. None of this seemed to help.
Yes, you should reduce the number classes that need to be
instrumented. You can do it in two ways:
1) select correct instrumentation filer - if you want to profile whole
application, your instrumentation filer should just include classes
from your project. Note that this is done automatically if you use
'Profile Project' and select 'Entire Application'.
2) if you can select just part of your application as root methods -
you can use 'Root method selector' to specify classes, packages or
servlet(s), jsp(s) from your application. If you use 'Entire
Application' with 'Profile Project', root methods are set
automatically to be all your project classes.
There is no CPU sampling in NetBeans Profiler, sampling you used is
labeled 'Exact call tree, sampled timing' (Am I right?) - it still
requires instrumentation. However it lowers the profiler overhead,
once the code is instrumented.
To help you more I need to know, what type of NetBeans project you use
for your application. What setting you use 'Entire Application' or
'Part of Application'? Are you doing local or remote profiling? Do you
use 'Profile Project' or 'Attach Profiler'? Do you use dynamic attach?

Bye,
--
Tomas Hurka <mailto:address-removed>
NetBeans Profiler http://profiler.netbeans.org
VisualVM http://visualvm.dev.java.net
Software Engineer, Developer Platforms Group
Sun Microsystems, Praha Czech Republic


---------------------------------------------------------------------
To unsubscribe, e-mail: address-removed
For additional commands, e-mail: address-removed
Back to top
Jess Holle
Posted via mailing list.





PostPosted: Thu Apr 30, 2009 11:33 am    Post subject: Why is the profiler so much slower than, e.g., YourKit? Reply with quote

Tomas Hurka wrote:
Quote:
Yes, you should reduce the number classes that need to be instrumented. You can do it in two ways:
1) select correct instrumentation filer - if you want to profile whole application, your instrumentation filer should just include classes from your project. Note that this is done automatically if you use 'Profile Project' and select 'Entire Application'.
2) if you can select just part of your application as root methods - you can use 'Root method selector' to specify classes, packages or servlet(s), jsp(s) from your application. If you use 'Entire Application' with 'Profile Project', root methods are set automatically to be all your project classes.
This is impractical in many real world cases where you don't initially know at all where the problem is -- hence the need for a sampling profiler.

Also there are serious bugs in the UI for specifying root methods (i.e. the one that operates via dialogs rather than from the source code).

Finally, you don't have to go through all of this in YourKit -- it just works.

--
Jess Holle
Back to top
Tomas Hurka
Posted via mailing list.





PostPosted: Thu Apr 30, 2009 7:22 pm    Post subject: Why is the profiler so much slower than, e.g., YourKit? Reply with quote

Hi Scott,

On 30 Apr 2009, at 18:54, Scott Wells wrote:
Quote:
Tomas Hurka wrote:
Quote:
Yes, you should reduce the number classes that need to be
instrumented. You can do it in two ways:

1) select correct instrumentation filer - if you want to profile
whole
application, your instrumentation filer should just include classes
from your project. Note that this is done automatically if you use
'Profile Project' and select 'Entire Application'.

2) if you can select just part of your application as root methods -
you can use 'Root method selector' to specify classes, packages or
servlet(s), jsp(s) from your application. If you use 'Entire
Application' with 'Profile Project', root methods are set
automatically
to be all your project classes.
[..]

Quote:
It sounds like the bottom line is that there's no way to configure
NetBeans
today to perform in a manner that's similar to YourKit, i.e.,
virtually
instantaneous profiling of CPU, memory, threads, etc., with the
overhead
introduced based on how much information you're trying to capture
and amortized
over actual execution time, not as an impediment to even getting
started.
I think NetBeans Profiler works that way, but you did not provide
enough information so that we can help you with your use-case or
improve future versions of NetBeans Profiler based on your feedback.
Unfortunately it looks like you already make your mind that it is not
possible to configure NetBeans Profiler so that you can profile your
application.

Quote:
If you haven't done so, you guys might want to pull down a 30-day
eval of
YourKit and play with it to see what we mean.
I am surprised by this offer. Do you really think that we did not play
and evaluate other profilers?

Bye,
--
Tomas Hurka <mailto:address-removed>
NetBeans Profiler http://profiler.netbeans.org
VisualVM http://visualvm.dev.java.net
Software Engineer, Developer Platforms Group
Sun Microsystems, Praha Czech Republic


---------------------------------------------------------------------
To unsubscribe, e-mail: address-removed
For additional commands, e-mail: address-removed
Back to top
SCWells72



Joined: 05 May 2009
Posts: 2

PostPosted: Tue May 05, 2009 3:08 pm    Post subject: Split thread... Reply with quote

FYI, I've posted follow-up questions here. I guess it split things up because I was using the mailing list rather than the forum.

Scott
Back to top
Display posts from previous:   
Post new topic   Reply to topic    NetBeans Forums -> Profiler Users All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB
By use of this website, you agree to the NetBeans Policies and Terms of Use. © 2012, Oracle Corporation and/or its affiliates. Sponsored by Oracle logo