NetBeans Forums

 FAQFAQ   SearchSearch   MemberlistMemberlist   RegisterRegister   ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 
  

New Language Support Issues

 
Post new topic   Reply to topic    NetBeans Forums -> NetBeans Platform Users
View previous topic :: View next topic  
Author Message
remcollier



Joined: 17 Oct 2008
Posts: 7
Location: Dublin, Ireland
PostPosted: Mon Oct 05, 2009 5:02 pm    Post subject: New Language Support Issues Reply with quote
Hi all,

I am currently working on updating support for a programming language known as AFAPL (its an academic project that aims to support the programming of multi-agent systems). I have provided IDE-based support for AFAPL since NB3.6(ish) and last year switched from editing Java project source code to make use of Schielmann (probably spelt wrong). Recently, I started to make some modifications to the syntax highlighting to reflect various improvements in the language and noticed the new support for JavaCC, so I decided to switch to use it instead.,,

I followed the tutorial: http://wiki.netbeans.org/How_to_create_support_for_a_new_language

Everything makes sense to me, and its quite easy to follow and I have managed to get syntax highlighting/syntax error detection working in general. However, I am getting a couple of issues that I was hoping you could help with:

1) I use Java style commenting for AFAPL, and found that the SINGLE LINE COMMENT does not work when the only two characters in the file are // (this means that I cannot create a blank file, and start by typing a single line comment)
2) For multi line commenting, I enter /* and the whole NB platform freezes. There is no error in the output of the NB client that I am using to develop the module.
3) I have been getting AssertionErrors for various characters, for example, I had to include / and * in the LETTER token definition - is this the correct thing to do? Do I have to work out all possible characters - I had previously reused the Java1.5.jj file as a basis for my project?
4) I am occasionally getting an OutOfMemoryError (this is independent of the previous 3 issues) and have no idea what is causing it...

I am developing on a Mac using NB6.7.1 and run the project to test functionality.

I appreciate any help as I have been hitting my head against a brick wall with this for the last week - In fact, I have to say that I found Schielmann a lot easier to use w.r.t. this.

I am happy to paste/attach any code that will help solve the problems - we have used NB for many years (5+) now and it provides a basis for both ongoing research work and an aide for teaching students how to program agents.

I would really appreciate any help that can be offered...

Rem Collier
School of Computer Science and Informatics
University College Dublin, Ireland
Back to top
Gregg Wonderly
Posted via mailing list.
PostPosted: Mon Oct 05, 2009 5:21 pm    Post subject: New Language Support Issues Reply with quote
There have been many instances of the EDT being used incorrectly in the new
editor changes that went into netbeans in 6.X releases. It might be useful to
get a thread dump out of the IDE when it locks up to see if there is an EDT
deadlock, or something else easily recognizable from the thread contexts that
will help you discover where the lockup is occurring at.

Gregg Wonderly

remcollier wrote:
Quote:
Hi all,



I am currently working on updating support for a programming language known as AFAPL (its an academic project that aims to support the programming of multi-agent systems). I have provided IDE-based support for AFAPL since NB3.6(ish) and last year switched from editing Java project source code to make use of Schielmann (probably spelt wrong). Recently, I started to make some modifications to the syntax highlighting to reflect various improvements in the language and noticed the new support for JavaCC, so I decided to switch to use it instead.,,



I followed the tutorial: http://wiki.netbeans.org/How_to_create_support_for_a_new_language



Everything makes sense to me, and its quite easy to follow and I have managed to get syntax highlighting/syntax error detection working in general. However, I am getting a couple of issues that I was hoping you could help with:



1) I use Java style commenting for AFAPL, and found that the SINGLE LINE COMMENT does not work when the only two characters in the file are // (this means that I cannot create a blank file, and start by typing a single line comment)

2) For multi line commenting, I enter /* and the whole NB platform freezes. There is no error in the output of the NB client that I am using to develop the module.

3) I have been getting AssertionErrors for various characters, for example, I had to include / and * in the LETTER token definition - is this the correct thing to do? Do I have to work out all possible characters - I had previously reused the Java1.5.jj file as a basis for my project?

4) I am occasionally getting an OutOfMemoryError (this is independent of the previous 3 issues) and have no idea what is causing it...



I am developing on a Mac using NB6.7.1 and run the project to test functionality.



I appreciate any help as I have been hitting my head against a brick wall with this for the last week - In fact, I have to say that I found Schielmann a lot easier to use w.r.t. this.



I am happy to paste/attach any code that will help solve the problems - we have used NB for many years (5+) now and it provides a basis for both ongoing research work and an aide for teaching students how to program agents.



I would really appreciate any help that can be offered...



Rem Collier

School of Computer Science and Informatics

University College Dublin, Ireland





Back to top
remcollier



Joined: 17 Oct 2008
Posts: 7
Location: Dublin, Ireland
PostPosted: Tue Oct 06, 2009 9:14 am    Post subject: Reply with quote
Hi Gregg,

Thanks for the info, the full thread dump is below. Unfortunately, I have no real idea what I am looking for...

It seems that the "Parsing & Indexing Loop (200907230233)" is going into a PARKING state...

...

"Parsing & Indexing Loop (200907230233)" daemon prio=1 tid=0x000000012d898800 nid=0x12ba82000 waiting on condition [0x000000012ba81000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x0000000108e794b0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:220)
at org.netbeans.modules.parsing.impl.TaskProcessor$CompilationJob.run(TaskProcessor.java:577)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:637)

...
Rem

PS I have n

2009-10-06 10:08:11
Full thread dump Java HotSpot(TM) 64-Bit Server VM (14.1-b02-90 mixed mode):

"Inactive RequestProcessor thread [Was:parsing-event-collector/org.netbeans.modules.parsing.impl.event.EventSupport$1]" daemon prio=1 tid=0x000000010224b000 nid=0x1286a6000 in Object.wait() [0x00000001286a5000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x0000000109c13e70> (a java.lang.Object)
at org.openide.util.RequestProcessor$Processor.run(RequestProcessor.java:983)
- locked <0x0000000109c13e70> (a java.lang.Object)

"Inactive RequestProcessor thread [Was:TimedSoftReference/org.openide.util.TimedSoftReference]" daemon prio=1 tid=0x000000010622b800 nid=0x12cfb0000 in Object.wait() [0x000000012cfaf000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x0000000109bb97e8> (a java.lang.Object)
at org.openide.util.RequestProcessor$Processor.run(RequestProcessor.java:983)
- locked <0x0000000109bb97e8> (a java.lang.Object)

"Parsing & Indexing Loop (200907230233)" daemon prio=1 tid=0x000000012d898800 nid=0x12ba82000 waiting on condition [0x000000012ba81000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x0000000108e794b0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:220)
at org.netbeans.modules.parsing.impl.TaskProcessor$CompilationJob.run(TaskProcessor.java:577)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:637)

"Inactive RequestProcessor thread [Was:PropertyUtils.FilePropertyProvider.RP/org.netbeans.spi.project.support.ant.PropertyUtils$FilePropertyProvider$2]" daemon prio=1 tid=0x00000001065c7000 nid=0x12be03000 in Object.wait() [0x000000012be02000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x0000000109381888> (a java.lang.Object)
at org.openide.util.RequestProcessor$Processor.run(RequestProcessor.java:983)
- locked <0x0000000109381888> (a java.lang.Object)

"Inactive RequestProcessor thread [Was:AntProjectHelper.RP/org.netbeans.spi.project.support.ant.AntProjectHelper$RunnableImpl]" daemon prio=1 tid=0x0000000106400000 nid=0x129723000 in Object.wait() [0x0000000129722000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x0000000109054290> (a java.lang.Object)
at org.openide.util.RequestProcessor$Processor.run(RequestProcessor.java:983)
- locked <0x0000000109054290> (a java.lang.Object)

"DestroyJavaVM" prio=5 tid=0x000000010662c800 nid=0x101301000 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE

"Inactive RequestProcessor thread [Was:PropertyUtils.FilePropertyProvider.RP/org.netbeans.spi.project.support.ant.PropertyUtils$FilePropertyProvider$2]" daemon prio=1 tid=0x000000012f7b8000 nid=0x129620000 in Object.wait() [0x000000012961f000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x0000000108c678c0> (a java.lang.Object)
at org.openide.util.RequestProcessor$Processor.run(RequestProcessor.java:983)
- locked <0x0000000108c678c0> (a java.lang.Object)

"AWT-EventQueue-1" prio=6 tid=0x0000000106268800 nid=0x12936f000 runnable [0x000000012936c000]
java.lang.Thread.State: RUNNABLE
at com.agentfactory.netbeans.lexer.JavaParserTokenManager.getNextToken(JavaParserTokenManager.java:1561)
at com.agentfactory.netbeans.lexer.AFAPL2Lexer.nextToken(AFAPL2Lexer.java:27)
at org.netbeans.lib.lexer.LexerInputOperation.nextToken(LexerInputOperation.java:190)
at org.netbeans.lib.lexer.inc.TokenListUpdater.relex(TokenListUpdater.java:619)
at org.netbeans.lib.lexer.inc.TokenListUpdater.updateRegular(TokenListUpdater.java:276)
at org.netbeans.lib.lexer.inc.TokenHierarchyUpdate$UpdateItem.update(TokenHierarchyUpdate.java:345)
at org.netbeans.lib.lexer.inc.TokenHierarchyUpdate.processLevelInfos(TokenHierarchyUpdate.java:220)
at org.netbeans.lib.lexer.inc.TokenHierarchyUpdate.updateImpl(TokenHierarchyUpdate.java:192)
at org.netbeans.lib.lexer.inc.TokenHierarchyUpdate.update(TokenHierarchyUpdate.java:130)
at org.netbeans.lib.lexer.TokenHierarchyOperation.textModified(TokenHierarchyOperation.java:546)
at org.netbeans.spi.lexer.TokenHierarchyControl.textModified(TokenHierarchyControl.java:93)
at org.netbeans.lib.lexer.inc.DocumentInput.textModified(DocumentInput.java:151)
at org.netbeans.lib.lexer.inc.DocumentInput.insertUpdate(DocumentInput.java:140)
at org.netbeans.lib.editor.util.swing.PriorityDocumentListenerList.insertUpdate(PriorityDocumentListenerList.java:79)
at javax.swing.text.AbstractDocument.fireInsertUpdate(AbstractDocument.java:185)
at org.netbeans.editor.BaseDocument.fireInsertUpdate(BaseDocument.java:1623)
at org.netbeans.editor.BaseDocument.insertString(BaseDocument.java:778)
at org.netbeans.editor.BaseKit$DefaultKeyTypedAction.insertString(BaseKit.java:1093)
at org.netbeans.editor.BaseKit$DefaultKeyTypedAction$1.run(BaseKit.java:1045)
at org.netbeans.editor.GuardedDocument.runAtomicAsUser(GuardedDocument.java:354)
at org.netbeans.editor.BaseKit$DefaultKeyTypedAction.actionPerformed(BaseKit.java:1016)
at org.netbeans.editor.ext.ExtKit$ExtDefaultKeyTypedAction.actionPerformed(ExtKit.java:1040)
at org.netbeans.editor.BaseAction.actionPerformed(BaseAction.java:325)
at javax.swing.SwingUtilities.notifyAction(SwingUtilities.java:1641)
at javax.swing.JComponent.processKeyBinding(JComponent.java:2851)
at javax.swing.JComponent.processKeyBindings(JComponent.java:2886)
at javax.swing.JComponent.processKeyEvent(JComponent.java:2814)
at java.awt.Component.processEvent(Component.java:6125)
at java.awt.Container.processEvent(Container.java:2085)
at java.awt.Component.dispatchEventImpl(Component.java:4714)
at java.awt.Container.dispatchEventImpl(Container.java:2143)
at java.awt.Component.dispatchEvent(Component.java:4544)
at java.awt.KeyboardFocusManager.redispatchEvent(KeyboardFocusManager.java:1848)
at java.awt.DefaultKeyboardFocusManager.dispatchKeyEvent(DefaultKeyboardFocusManager.java:704)
at java.awt.DefaultKeyboardFocusManager.preDispatchKeyEvent(DefaultKeyboardFocusManager.java:969)
at java.awt.DefaultKeyboardFocusManager.typeAheadAssertions(DefaultKeyboardFocusManager.java:841)
at java.awt.DefaultKeyboardFocusManager.dispatchEvent(DefaultKeyboardFocusManager.java:668)
at java.awt.Component.dispatchEventImpl(Component.java:4586)
at java.awt.Container.dispatchEventImpl(Container.java:2143)
at java.awt.Window.dispatchEventImpl(Window.java:2475)
at java.awt.Component.dispatchEvent(Component.java:4544)
at java.awt.EventQueue.dispatchEvent(EventQueue.java:635)
at org.netbeans.core.TimableEventQueue.dispatchEvent(TimableEventQueue.java:104)
at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:296)
at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:211)
at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:201)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:196)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:188)
at java.awt.EventDispatchThread.run(EventDispatchThread.java:122)

"TimerQueue" daemon prio=5 tid=0x0000000102121000 nid=0x127d32000 in Object.wait() [0x0000000127d31000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x0000000108a74328> (a javax.swing.TimerQueue)
at javax.swing.TimerQueue.run(TimerQueue.java:236)
- locked <0x0000000108a74328> (a javax.swing.TimerQueue)
at java.lang.Thread.run(Thread.java:637)

"Thread-4" daemon prio=5 tid=0x00000001026f7800 nid=0x129101000 in Object.wait() [0x0000000129100000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x000000010870e2a0> (a java.util.LinkedList)
at java.lang.Object.wait(Object.java:485)
at java.util.prefs.AbstractPreferences$EventDispatchThread.run(AbstractPreferences.java:1461)
- locked <0x000000010870e2a0> (a java.util.LinkedList)

"*** JFluid Separate Command Execution Thread" daemon prio=5 tid=0x00000001061ab800 nid=0x1287ba000 in Object.wait() [0x00000001287b9000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000001088e19c0> (a java.lang.Object)
at java.lang.Object.wait(Object.java:485)
at org.netbeans.lib.profiler.ProfilerClient$SeparateCmdExecutionThread.run(ProfilerClient.java:102)
- locked <0x00000001088e19c0> (a java.lang.Object)

"Attach Listener" daemon prio=9 tid=0x00000001067c2800 nid=0x12926c000 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE

"AWT-Shutdown" prio=5 tid=0x00000001021d7000 nid=0x127b2c000 in Object.wait() [0x0000000127b2b000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x0000000107d622b8> (a java.lang.Object)
at java.lang.Object.wait(Object.java:485)
at sun.awt.AWTAutoShutdown.run(AWTAutoShutdown.java:259)
- locked <0x0000000107d622b8> (a java.lang.Object)
at java.lang.Thread.run(Thread.java:637)

"Java2D Disposer" daemon prio=10 tid=0x00000001062cc000 nid=0x127c2f000 in Object.wait() [0x0000000127c2e000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x0000000107d622d0> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
- locked <0x0000000107d622d0> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)
at sun.java2d.Disposer.run(Disposer.java:125)
at java.lang.Thread.run(Thread.java:637)

"AWT-AppKit" daemon prio=5 tid=0x0000000102179800 nid=0x7fff700f6be0 runnable [0x0000000000000000]
java.lang.Thread.State: RUNNABLE

"Timer-0" daemon prio=5 tid=0x000000010613e000 nid=0x114a07000 in Object.wait() [0x0000000114a06000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x0000000107e35a18> (a java.util.TaskQueue)
at java.util.TimerThread.mainLoop(Timer.java:509)
- locked <0x0000000107e35a18> (a java.util.TaskQueue)
at java.util.TimerThread.run(Timer.java:462)

"CLI Requests Server" daemon prio=5 tid=0x00000001021ae000 nid=0x114904000 runnable [0x0000000114903000]
java.lang.Thread.State: RUNNABLE
at java.net.PlainSocketImpl.socketAccept(Native Method)
at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:390)
- locked <0x0000000107e4d530> (a java.net.SocksSocketImpl)
at java.net.ServerSocket.implAccept(ServerSocket.java:453)
at java.net.ServerSocket.accept(ServerSocket.java:421)
at org.netbeans.CLIHandler$Server.run(CLIHandler.java:1005)

"Active Reference Queue Daemon" daemon prio=1 tid=0x0000000102012000 nid=0x114801000 in Object.wait() [0x0000000114800000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x0000000107d62348> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
- locked <0x0000000107d62348> (a java.lang.ref.ReferenceQueue$Lock)
at org.netbeans.modules.openide.util.ActiveQueue.run(ActiveQueue.java:53)
at java.lang.Thread.run(Thread.java:637)

"Low Memory Detector" daemon prio=5 tid=0x0000000102026000 nid=0x11440d000 runnable [0x0000000000000000]
java.lang.Thread.State: RUNNABLE

"CompilerThread1" daemon prio=9 tid=0x0000000102025800 nid=0x11430a000 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE

"CompilerThread0" daemon prio=9 tid=0x0000000102024800 nid=0x114207000 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE

"Signal Dispatcher" daemon prio=9 tid=0x0000000102024000 nid=0x114104000 runnable [0x0000000000000000]
java.lang.Thread.State: RUNNABLE

"Surrogate Locker Thread (CMS)" daemon prio=5 tid=0x0000000102023000 nid=0x114001000 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE

"Finalizer" daemon prio=8 tid=0x000000010200d800 nid=0x113b07000 in Object.wait() [0x0000000113b06000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x0000000107e2e070> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
- locked <0x0000000107e2e070> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)

"Reference Handler" daemon prio=10 tid=0x000000010608e800 nid=0x113a04000 in Object.wait() [0x0000000113a03000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x0000000107d62300> (a java.lang.ref.Reference$Lock)
at java.lang.Object.wait(Object.java:485)
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
- locked <0x0000000107d62300> (a java.lang.ref.Reference$Lock)

"VM Thread" prio=9 tid=0x0000000102006800 nid=0x113901000 runnable

"Gang worker#0 (Parallel GC Threads)" prio=9 tid=0x0000000106000000 nid=0x105902000 runnable

"Gang worker#1 (Parallel GC Threads)" prio=9 tid=0x0000000106001000 nid=0x105a05000 runnable

"Concurrent Mark-Sweep GC Thread" prio=9 tid=0x0000000106040800 nid=0x105d76000 runnable
"VM Periodic Task Thread" prio=10 tid=0x0000000102027000 nid=0x114510000 waiting on condition

"Exception Catcher Thread" prio=10 tid=0x0000000102001800 nid=0x101701000 runnable
JNI global references: 4139
Back to top
Gregg Wonderly
Posted via mailing list.
PostPosted: Tue Oct 06, 2009 2:14 pm    Post subject: New Language Support Issues Reply with quote
The key thread trace is the one below which shows that the AWT-EventQueue-1
thread (an Event Dispatch Thread or EDT) is busy doing work that is not related
to GUI painting and drawing. They appear to be using the EDT to analyze changes
to the source code. This should not be happening. The EDT should only be used
for painting and preliminary event delivery. The EDT is (in netbeans based on
what I've seen in all the stack traces) often misused as a form of locking to
keep things from being changed elsewhere instead of using appropriate locks
elsewhere. The most typical reason for doing this is because it keeps circular
wait from happening where one thread has a lock and makes something happen that
triggers an EDT event that then enters code using the same lock, but now with a
different thread wanting the lock.

This is not easy stuff to manage, but nevertheless when it is done incorrectly,
we get into these kinds of situations where the EDT is misused for locking and
the end result is clunky, unusable UI functionality.

Please open an issue titled with the problem you are experiencing, such as "EDT
parsing activity causes editor to lock up" or some such.

Gregg Wonderly

"AWT-EventQueue-1" prio=6 tid=0x0000000106268800 nid=0x12936f000 runnable
[0x000000012936c000]
java.lang.Thread.State: RUNNABLE
at
com.agentfactory.netbeans.lexer.JavaParserTokenManager.getNextToken(JavaParserTokenManager.java:1561)
at com.agentfactory.netbeans.lexer.AFAPL2Lexer.nextToken(AFAPL2Lexer.java:27)
at
org.netbeans.lib.lexer.LexerInputOperation.nextToken(LexerInputOperation.java:190)
at org.netbeans.lib.lexer.inc.TokenListUpdater.relex(TokenListUpdater.java:619)
at
org.netbeans.lib.lexer.inc.TokenListUpdater.updateRegular(TokenListUpdater.java:276)
at
org.netbeans.lib.lexer.inc.TokenHierarchyUpdate$UpdateItem.update(TokenHierarchyUpdate.java:345)
at
org.netbeans.lib.lexer.inc.TokenHierarchyUpdate.processLevelInfos(TokenHierarchyUpdate.java:220)
at
org.netbeans.lib.lexer.inc.TokenHierarchyUpdate.updateImpl(TokenHierarchyUpdate.java:192)
at
org.netbeans.lib.lexer.inc.TokenHierarchyUpdate.update(TokenHierarchyUpdate.java:130)
at
org.netbeans.lib.lexer.TokenHierarchyOperation.textModified(TokenHierarchyOperation.java:546)
at
org.netbeans.spi.lexer.TokenHierarchyControl.textModified(TokenHierarchyControl.java:93)
at org.netbeans.lib.lexer.inc.DocumentInput.textModified(DocumentInput.java:151)
at org.netbeans.lib.lexer.inc.DocumentInput.insertUpdate(DocumentInput.java:140)
at
org.netbeans.lib.editor.util.swing.PriorityDocumentListenerList.insertUpdate(PriorityDocumentListenerList.java:79)
at javax.swing.text.AbstractDocument.fireInsertUpdate(AbstractDocument.java:185)
at org.netbeans.editor.BaseDocument.fireInsertUpdate(BaseDocument.java:1623)
at org.netbeans.editor.BaseDocument.insertString(BaseDocument.java:778)
at
org.netbeans.editor.BaseKit$DefaultKeyTypedAction.insertString(BaseKit.java:1093)
at org.netbeans.editor.BaseKit$DefaultKeyTypedAction$1.run(BaseKit.java:1045)
at org.netbeans.editor.GuardedDocument.runAtomicAsUser(GuardedDocument.java:354)
at
org.netbeans.editor.BaseKit$DefaultKeyTypedAction.actionPerformed(BaseKit.java:1016)
at
org.netbeans.editor.ext.ExtKit$ExtDefaultKeyTypedAction.actionPerformed(ExtKit.java:1040)
at org.netbeans.editor.BaseAction.actionPerformed(BaseAction.java:325)
at javax.swing.SwingUtilities.notifyAction(SwingUtilities.java:1641)
at javax.swing.JComponent.processKeyBinding(JComponent.java:2851)
at javax.swing.JComponent.processKeyBindings(JComponent.java:2886)
at javax.swing.JComponent.processKeyEvent(JComponent.java:2814)
at java.awt.Component.processEvent(Component.java:6125)
at java.awt.Container.processEvent(Container.java:2085)
at java.awt.Component.dispatchEventImpl(Component.java:4714)
at java.awt.Container.dispatchEventImpl(Container.java:2143)
at java.awt.Component.dispatchEvent(Component.java:4544)
at java.awt.KeyboardFocusManager.redispatchEvent(KeyboardFocusManager.java:1848)
at
java.awt.DefaultKeyboardFocusManager.dispatchKeyEvent(DefaultKeyboardFocusManager.java:704)
at
java.awt.DefaultKeyboardFocusManager.preDispatchKeyEvent(DefaultKeyboardFocusManager.java:969)
at
java.awt.DefaultKeyboardFocusManager.typeAheadAssertions(DefaultKeyboardFocusManager.java:841)
at
java.awt.DefaultKeyboardFocusManager.dispatchEvent(DefaultKeyboardFocusManager.java:668)
at java.awt.Component.dispatchEventImpl(Component.java:4586)
at java.awt.Container.dispatchEventImpl(Container.java:2143)
at java.awt.Window.dispatchEventImpl(Window.java:2475)
at java.awt.Component.dispatchEvent(Component.java:4544)
at java.awt.EventQueue.dispatchEvent(EventQueue.java:635)
at org.netbeans.core.TimableEventQueue.dispatchEvent(TimableEventQueue.java:104)
at
java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:296)
at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:211)
at
java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:201)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:196)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:188)
at java.awt.EventDispatchThread.run(EventDispatchThread.java:122)


remcollier wrote:
Quote:
Hi Gregg,



Thanks for the info, the full thread dump is below. Unfortunately, I have no real idea what I am looking for...



It seems that the "Parsing & Indexing Loop (200907230233)" is going into a PARKING state...



...



"Parsing & Indexing Loop (200907230233)" daemon prio=1 tid=0x000000012d898800 nid=0x12ba82000 waiting on condition [0x000000012ba81000]

java.lang.Thread.State: WAITING (parking)

at sun.misc.Unsafe.park(Native Method)

- parking to wait for <0x0000000108e794b0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)

at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)

at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)

at java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:220)

at org.netbeans.modules.parsing.impl.TaskProcessor$CompilationJob.run(TaskProcessor.java:577)

at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)

at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)

at java.util.concurrent.FutureTask.run(FutureTask.java:138)

at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)

at java.lang.Thread.run(Thread.java:637)



...

Rem



PS I have n



2009-10-06 10:08:11

Full thread dump Java HotSpot(TM) 64-Bit Server VM (14.1-b02-90 mixed mode):



"Inactive RequestProcessor thread [Was:parsing-event-collector/org.netbeans.modules.parsing.impl.event.EventSupport$1]" daemon prio=1 tid=0x000000010224b000 nid=0x1286a6000 in Object.wait() [0x00000001286a5000]

java.lang.Thread.State: TIMED_WAITING (on object monitor)

at java.lang.Object.wait(Native Method)

- waiting on <0x0000000109c13e70> (a java.lang.Object)

at org.openide.util.RequestProcessor$Processor.run(RequestProcessor.java:983)

- locked <0x0000000109c13e70> (a java.lang.Object)



"Inactive RequestProcessor thread [Was:TimedSoftReference/org.openide.util.TimedSoftReference]" daemon prio=1 tid=0x000000010622b800 nid=0x12cfb0000 in Object.wait() [0x000000012cfaf000]

java.lang.Thread.State: TIMED_WAITING (on object monitor)

at java.lang.Object.wait(Native Method)

- waiting on <0x0000000109bb97e8> (a java.lang.Object)

at org.openide.util.RequestProcessor$Processor.run(RequestProcessor.java:983)

- locked <0x0000000109bb97e8> (a java.lang.Object)



"Parsing & Indexing Loop (200907230233)" daemon prio=1 tid=0x000000012d898800 nid=0x12ba82000 waiting on condition [0x000000012ba81000]

java.lang.Thread.State: WAITING (parking)

at sun.misc.Unsafe.park(Native Method)

- parking to wait for <0x0000000108e794b0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)

at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)

at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)

at java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:220)

at org.netbeans.modules.parsing.impl.TaskProcessor$CompilationJob.run(TaskProcessor.java:577)

at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)

at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)

at java.util.concurrent.FutureTask.run(FutureTask.java:138)

at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)

at java.lang.Thread.run(Thread.java:637)



"Inactive RequestProcessor thread [Was:PropertyUtils.FilePropertyProvider.RP/org.netbeans.spi.project.support.ant.PropertyUtils$FilePropertyProvider$2]" daemon prio=1 tid=0x00000001065c7000 nid=0x12be03000 in Object.wait() [0x000000012be02000]

java.lang.Thread.State: TIMED_WAITING (on object monitor)

at java.lang.Object.wait(Native Method)

- waiting on <0x0000000109381888> (a java.lang.Object)

at org.openide.util.RequestProcessor$Processor.run(RequestProcessor.java:983)

- locked <0x0000000109381888> (a java.lang.Object)



"Inactive RequestProcessor thread [Was:AntProjectHelper.RP/org.netbeans.spi.project.support.ant.AntProjectHelper$RunnableImpl]" daemon prio=1 tid=0x0000000106400000 nid=0x129723000 in Object.wait() [0x0000000129722000]

java.lang.Thread.State: TIMED_WAITING (on object monitor)

at java.lang.Object.wait(Native Method)

- waiting on <0x0000000109054290> (a java.lang.Object)

at org.openide.util.RequestProcessor$Processor.run(RequestProcessor.java:983)

- locked <0x0000000109054290> (a java.lang.Object)



"DestroyJavaVM" prio=5 tid=0x000000010662c800 nid=0x101301000 waiting on condition [0x0000000000000000]

java.lang.Thread.State: RUNNABLE



"Inactive RequestProcessor thread [Was:PropertyUtils.FilePropertyProvider.RP/org.netbeans.spi.project.support.ant.PropertyUtils$FilePropertyProvider$2]" daemon prio=1 tid=0x000000012f7b8000 nid=0x129620000 in Object.wait() [0x000000012961f000]

java.lang.Thread.State: TIMED_WAITING (on object monitor)

at java.lang.Object.wait(Native Method)

- waiting on <0x0000000108c678c0> (a java.lang.Object)

at org.openide.util.RequestProcessor$Processor.run(RequestProcessor.java:983)

- locked <0x0000000108c678c0> (a java.lang.Object)



"AWT-EventQueue-1" prio=6 tid=0x0000000106268800 nid=0x12936f000 runnable [0x000000012936c000]

java.lang.Thread.State: RUNNABLE

at com.agentfactory.netbeans.lexer.JavaParserTokenManager.getNextToken(JavaParserTokenManager.java:1561)

at com.agentfactory.netbeans.lexer.AFAPL2Lexer.nextToken(AFAPL2Lexer.java:27)

at org.netbeans.lib.lexer.LexerInputOperation.nextToken(LexerInputOperation.java:190)

at org.netbeans.lib.lexer.inc.TokenListUpdater.relex(TokenListUpdater.java:619)

at org.netbeans.lib.lexer.inc.TokenListUpdater.updateRegular(TokenListUpdater.java:276)

at org.netbeans.lib.lexer.inc.TokenHierarchyUpdate$UpdateItem.update(TokenHierarchyUpdate.java:345)

at org.netbeans.lib.lexer.inc.TokenHierarchyUpdate.processLevelInfos(TokenHierarchyUpdate.java:220)

at org.netbeans.lib.lexer.inc.TokenHierarchyUpdate.updateImpl(TokenHierarchyUpdate.java:192)

at org.netbeans.lib.lexer.inc.TokenHierarchyUpdate.update(TokenHierarchyUpdate.java:130)

at org.netbeans.lib.lexer.TokenHierarchyOperation.textModified(TokenHierarchyOperation.java:546)

at org.netbeans.spi.lexer.TokenHierarchyControl.textModified(TokenHierarchyControl.java:93)

at org.netbeans.lib.lexer.inc.DocumentInput.textModified(DocumentInput.java:151)

at org.netbeans.lib.lexer.inc.DocumentInput.insertUpdate(DocumentInput.java:140)

at org.netbeans.lib.editor.util.swing.PriorityDocumentListenerList.insertUpdate(PriorityDocumentListenerList.java:79)

at javax.swing.text.AbstractDocument.fireInsertUpdate(AbstractDocument.java:185)

at org.netbeans.editor.BaseDocument.fireInsertUpdate(BaseDocument.java:1623)

at org.netbeans.editor.BaseDocument.insertString(BaseDocument.java:778)

at org.netbeans.editor.BaseKit$DefaultKeyTypedAction.insertString(BaseKit.java:1093)

at org.netbeans.editor.BaseKit$DefaultKeyTypedAction$1.run(BaseKit.java:1045)

at org.netbeans.editor.GuardedDocument.runAtomicAsUser(GuardedDocument.java:354)

at org.netbeans.editor.BaseKit$DefaultKeyTypedAction.actionPerformed(BaseKit.java:1016)

at org.netbeans.editor.ext.ExtKit$ExtDefaultKeyTypedAction.actionPerformed(ExtKit.java:1040)

at org.netbeans.editor.BaseAction.actionPerformed(BaseAction.java:325)

at javax.swing.SwingUtilities.notifyAction(SwingUtilities.java:1641)

at javax.swing.JComponent.processKeyBinding(JComponent.java:2851)

at javax.swing.JComponent.processKeyBindings(JComponent.java:2886)

at javax.swing.JComponent.processKeyEvent(JComponent.java:2814)

at java.awt.Component.processEvent(Component.java:6125)

at java.awt.Container.processEvent(Container.java:2085)

at java.awt.Component.dispatchEventImpl(Component.java:4714)

at java.awt.Container.dispatchEventImpl(Container.java:2143)

at java.awt.Component.dispatchEvent(Component.java:4544)

at java.awt.KeyboardFocusManager.redispatchEvent(KeyboardFocusManager.java:1848)

at java.awt.DefaultKeyboardFocusManager.dispatchKeyEvent(DefaultKeyboardFocusManager.java:704)

at java.awt.DefaultKeyboardFocusManager.preDispatchKeyEvent(DefaultKeyboardFocusManager.java:969)

at java.awt.DefaultKeyboardFocusManager.typeAheadAssertions(DefaultKeyboardFocusManager.java:841)

at java.awt.DefaultKeyboardFocusManager.dispatchEvent(DefaultKeyboardFocusManager.java:668)

at java.awt.Component.dispatchEventImpl(Component.java:4586)

at java.awt.Container.dispatchEventImpl(Container.java:2143)

at java.awt.Window.dispatchEventImpl(Window.java:2475)

at java.awt.Component.dispatchEvent(Component.java:4544)

at java.awt.EventQueue.dispatchEvent(EventQueue.java:635)

at org.netbeans.core.TimableEventQueue.dispatchEvent(TimableEventQueue.java:104)

at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:296)

at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:211)

at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:201)

at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:196)

at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:188)

at java.awt.EventDispatchThread.run(EventDispatchThread.java:122)



"TimerQueue" daemon prio=5 tid=0x0000000102121000 nid=0x127d32000 in Object.wait() [0x0000000127d31000]

java.lang.Thread.State: TIMED_WAITING (on object monitor)

at java.lang.Object.wait(Native Method)

- waiting on <0x0000000108a74328> (a javax.swing.TimerQueue)

at javax.swing.TimerQueue.run(TimerQueue.java:236)

- locked <0x0000000108a74328> (a javax.swing.TimerQueue)

at java.lang.Thread.run(Thread.java:637)



"Thread-4" daemon prio=5 tid=0x00000001026f7800 nid=0x129101000 in Object.wait() [0x0000000129100000]

java.lang.Thread.State: WAITING (on object monitor)

at java.lang.Object.wait(Native Method)

- waiting on <0x000000010870e2a0> (a java.util.LinkedList)

at java.lang.Object.wait(Object.java:485)

at java.util.prefs.AbstractPreferences$EventDispatchThread.run(AbstractPreferences.java:1461)

- locked <0x000000010870e2a0> (a java.util.LinkedList)



"*** JFluid Separate Command Execution Thread" daemon prio=5 tid=0x00000001061ab800 nid=0x1287ba000 in Object.wait() [0x00000001287b9000]

java.lang.Thread.State: WAITING (on object monitor)

at java.lang.Object.wait(Native Method)

- waiting on <0x00000001088e19c0> (a java.lang.Object)

at java.lang.Object.wait(Object.java:485)

at org.netbeans.lib.profiler.ProfilerClient$SeparateCmdExecutionThread.run(ProfilerClient.java:102)

- locked <0x00000001088e19c0> (a java.lang.Object)



"Attach Listener" daemon prio=9 tid=0x00000001067c2800 nid=0x12926c000 waiting on condition [0x0000000000000000]

java.lang.Thread.State: RUNNABLE



"AWT-Shutdown" prio=5 tid=0x00000001021d7000 nid=0x127b2c000 in Object.wait() [0x0000000127b2b000]

java.lang.Thread.State: WAITING (on object monitor)

at java.lang.Object.wait(Native Method)

- waiting on <0x0000000107d622b8> (a java.lang.Object)

at java.lang.Object.wait(Object.java:485)

at sun.awt.AWTAutoShutdown.run(AWTAutoShutdown.java:259)

- locked <0x0000000107d622b8> (a java.lang.Object)

at java.lang.Thread.run(Thread.java:637)



"Java2D Disposer" daemon prio=10 tid=0x00000001062cc000 nid=0x127c2f000 in Object.wait() [0x0000000127c2e000]

java.lang.Thread.State: WAITING (on object monitor)

at java.lang.Object.wait(Native Method)

- waiting on <0x0000000107d622d0> (a java.lang.ref.ReferenceQueue$Lock)

at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)

- locked <0x0000000107d622d0> (a java.lang.ref.ReferenceQueue$Lock)

at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)

at sun.java2d.Disposer.run(Disposer.java:125)

at java.lang.Thread.run(Thread.java:637)



"AWT-AppKit" daemon prio=5 tid=0x0000000102179800 nid=0x7fff700f6be0 runnable [0x0000000000000000]

java.lang.Thread.State: RUNNABLE



"Timer-0" daemon prio=5 tid=0x000000010613e000 nid=0x114a07000 in Object.wait() [0x0000000114a06000]

java.lang.Thread.State: TIMED_WAITING (on object monitor)

at java.lang.Object.wait(Native Method)

- waiting on <0x0000000107e35a18> (a java.util.TaskQueue)

at java.util.TimerThread.mainLoop(Timer.java:509)

- locked <0x0000000107e35a18> (a java.util.TaskQueue)

at java.util.TimerThread.run(Timer.java:462)



"CLI Requests Server" daemon prio=5 tid=0x00000001021ae000 nid=0x114904000 runnable [0x0000000114903000]

java.lang.Thread.State: RUNNABLE

at java.net.PlainSocketImpl.socketAccept(Native Method)

at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:390)

- locked <0x0000000107e4d530> (a java.net.SocksSocketImpl)

at java.net.ServerSocket.implAccept(ServerSocket.java:453)

at java.net.ServerSocket.accept(ServerSocket.java:421)

at org.netbeans.CLIHandler$Server.run(CLIHandler.java:1005)



"Active Reference Queue Daemon" daemon prio=1 tid=0x0000000102012000 nid=0x114801000 in Object.wait() [0x0000000114800000]

java.lang.Thread.State: WAITING (on object monitor)

at java.lang.Object.wait(Native Method)

- waiting on <0x0000000107d62348> (a java.lang.ref.ReferenceQueue$Lock)

at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)

- locked <0x0000000107d62348> (a java.lang.ref.ReferenceQueue$Lock)

at org.netbeans.modules.openide.util.ActiveQueue.run(ActiveQueue.java:53)

at java.lang.Thread.run(Thread.java:637)



"Low Memory Detector" daemon prio=5 tid=0x0000000102026000 nid=0x11440d000 runnable [0x0000000000000000]

java.lang.Thread.State: RUNNABLE



"CompilerThread1" daemon prio=9 tid=0x0000000102025800 nid=0x11430a000 waiting on condition [0x0000000000000000]

java.lang.Thread.State: RUNNABLE



"CompilerThread0" daemon prio=9 tid=0x0000000102024800 nid=0x114207000 waiting on condition [0x0000000000000000]

java.lang.Thread.State: RUNNABLE



"Signal Dispatcher" daemon prio=9 tid=0x0000000102024000 nid=0x114104000 runnable [0x0000000000000000]

java.lang.Thread.State: RUNNABLE



"Surrogate Locker Thread (CMS)" daemon prio=5 tid=0x0000000102023000 nid=0x114001000 waiting on condition [0x0000000000000000]

java.lang.Thread.State: RUNNABLE



"Finalizer" daemon prio=8 tid=0x000000010200d800 nid=0x113b07000 in Object.wait() [0x0000000113b06000]

java.lang.Thread.State: WAITING (on object monitor)

at java.lang.Object.wait(Native Method)

- waiting on <0x0000000107e2e070> (a java.lang.ref.ReferenceQueue$Lock)

at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)

- locked <0x0000000107e2e070> (a java.lang.ref.ReferenceQueue$Lock)

at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)

at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)



"Reference Handler" daemon prio=10 tid=0x000000010608e800 nid=0x113a04000 in Object.wait() [0x0000000113a03000]

java.lang.Thread.State: WAITING (on object monitor)

at java.lang.Object.wait(Native Method)

- waiting on <0x0000000107d62300> (a java.lang.ref.Reference$Lock)

at java.lang.Object.wait(Object.java:485)

at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)

- locked <0x0000000107d62300> (a java.lang.ref.Reference$Lock)



"VM Thread" prio=9 tid=0x0000000102006800 nid=0x113901000 runnable



"Gang worker#0 (Parallel GC Threads)" prio=9 tid=0x0000000106000000 nid=0x105902000 runnable



"Gang worker#1 (Parallel GC Threads)" prio=9 tid=0x0000000106001000 nid=0x105a05000 runnable



"Concurrent Mark-Sweep GC Thread" prio=9 tid=0x0000000106040800 nid=0x105d76000 runnable

"VM Periodic Task Thread" prio=10 tid=0x0000000102027000 nid=0x114510000 waiting on condition



"Exception Catcher Thread" prio=10 tid=0x0000000102001800 nid=0x101701000 runnable

JNI global references: 4139





Back to top
David Strupl
Posted via mailing list.
PostPosted: Tue Oct 06, 2009 3:37 pm    Post subject: New Language Support Issues Reply with quote
Gregg,

I would like to slightly correct your evaluation. The EDT from the thread dump
bellow is clearly used to react to the user typing. In the editor in order to
repaint the modified region we are trying to re-lex only part that was modified
by the user (for the user to get proper colors etc).

The problem is that

com.agentfactory.netbeans.lexer.JavaParserTokenManager.getNextToken(JavaParserTokenManager.java:1561)

com.agentfactory.netbeans.lexer.AFAPL2Lexer.nextToken(AFAPL2Lexer.java:27)

must be fast and finish very quickly for the editor to be usable.

Please do not file bug titled "EDT parsing activity causes editor to lock up" or
similar and first try to debug the above mentioned calls to figure out why they
are looping ...

Thanks,

David

Gregg Wonderly napsal(a):
Quote:
The key thread trace is the one below which shows that the
AWT-EventQueue-1 thread (an Event Dispatch Thread or EDT) is busy doing
work that is not related to GUI painting and drawing. They appear to be
using the EDT to analyze changes to the source code. This should not be
happening. The EDT should only be used for painting and preliminary
event delivery. The EDT is (in netbeans based on what I've seen in all
the stack traces) often misused as a form of locking to keep things from
being changed elsewhere instead of using appropriate locks elsewhere.
The most typical reason for doing this is because it keeps circular wait
from happening where one thread has a lock and makes something happen
that triggers an EDT event that then enters code using the same lock,
but now with a different thread wanting the lock.

This is not easy stuff to manage, but nevertheless when it is done
incorrectly, we get into these kinds of situations where the EDT is
misused for locking and the end result is clunky, unusable UI
functionality.

Please open an issue titled with the problem you are experiencing, such
as "EDT parsing activity causes editor to lock up" or some such.

Gregg Wonderly

"AWT-EventQueue-1" prio=6 tid=0x0000000106268800 nid=0x12936f000
runnable [0x000000012936c000]
java.lang.Thread.State: RUNNABLE
at
com.agentfactory.netbeans.lexer.JavaParserTokenManager.getNextToken(JavaParserTokenManager.java:1561)

at
com.agentfactory.netbeans.lexer.AFAPL2Lexer.nextToken(AFAPL2Lexer.java:27)
at
org.netbeans.lib.lexer.LexerInputOperation.nextToken(LexerInputOperation.java:190)

at
org.netbeans.lib.lexer.inc.TokenListUpdater.relex(TokenListUpdater.java:619)

at
org.netbeans.lib.lexer.inc.TokenListUpdater.updateRegular(TokenListUpdater.java:276)

at
org.netbeans.lib.lexer.inc.TokenHierarchyUpdate$UpdateItem.update(TokenHierarchyUpdate.java:345)

at
org.netbeans.lib.lexer.inc.TokenHierarchyUpdate.processLevelInfos(TokenHierarchyUpdate.java:220)

at
org.netbeans.lib.lexer.inc.TokenHierarchyUpdate.updateImpl(TokenHierarchyUpdate.java:192)

at
org.netbeans.lib.lexer.inc.TokenHierarchyUpdate.update(TokenHierarchyUpdate.java:130)

at
org.netbeans.lib.lexer.TokenHierarchyOperation.textModified(TokenHierarchyOperation.java:546)

at
org.netbeans.spi.lexer.TokenHierarchyControl.textModified(TokenHierarchyControl.java:93)

at
org.netbeans.lib.lexer.inc.DocumentInput.textModified(DocumentInput.java:151)

at
org.netbeans.lib.lexer.inc.DocumentInput.insertUpdate(DocumentInput.java:140)

at
org.netbeans.lib.editor.util.swing.PriorityDocumentListenerList.insertUpdate(PriorityDocumentListenerList.java:79)

at
javax.swing.text.AbstractDocument.fireInsertUpdate(AbstractDocument.java:185)

at
org.netbeans.editor.BaseDocument.fireInsertUpdate(BaseDocument.java:1623)
at org.netbeans.editor.BaseDocument.insertString(BaseDocument.java:778)
at
org.netbeans.editor.BaseKit$DefaultKeyTypedAction.insertString(BaseKit.java:1093)

at
org.netbeans.editor.BaseKit$DefaultKeyTypedAction$1.run(BaseKit.java:1045)
at
org.netbeans.editor.GuardedDocument.runAtomicAsUser(GuardedDocument.java:354)

at
org.netbeans.editor.BaseKit$DefaultKeyTypedAction.actionPerformed(BaseKit.java:1016)

at
org.netbeans.editor.ext.ExtKit$ExtDefaultKeyTypedAction.actionPerformed(ExtKit.java:1040)

at org.netbeans.editor.BaseAction.actionPerformed(BaseAction.java:325)
at javax.swing.SwingUtilities.notifyAction(SwingUtilities.java:1641)
at javax.swing.JComponent.processKeyBinding(JComponent.java:2851)
at javax.swing.JComponent.processKeyBindings(JComponent.java:2886)
at javax.swing.JComponent.processKeyEvent(JComponent.java:2814)
at java.awt.Component.processEvent(Component.java:6125)
at java.awt.Container.processEvent(Container.java:2085)
at java.awt.Component.dispatchEventImpl(Component.java:4714)
at java.awt.Container.dispatchEventImpl(Container.java:2143)
at java.awt.Component.dispatchEvent(Component.java:4544)
at
java.awt.KeyboardFocusManager.redispatchEvent(KeyboardFocusManager.java:1848)

at
java.awt.DefaultKeyboardFocusManager.dispatchKeyEvent(DefaultKeyboardFocusManager.java:704)

at
java.awt.DefaultKeyboardFocusManager.preDispatchKeyEvent(DefaultKeyboardFocusManager.java:969)

at
java.awt.DefaultKeyboardFocusManager.typeAheadAssertions(DefaultKeyboardFocusManager.java:841)

at
java.awt.DefaultKeyboardFocusManager.dispatchEvent(DefaultKeyboardFocusManager.java:668)

at java.awt.Component.dispatchEventImpl(Component.java:4586)
at java.awt.Container.dispatchEventImpl(Container.java:2143)
at java.awt.Window.dispatchEventImpl(Window.java:2475)
at java.awt.Component.dispatchEvent(Component.java:4544)
at java.awt.EventQueue.dispatchEvent(EventQueue.java:635)
at
org.netbeans.core.TimableEventQueue.dispatchEvent(TimableEventQueue.java:104)

at
java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:296)

at
java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:211)

at
java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:201)

at
java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:196)
at
java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:188)
at java.awt.EventDispatchThread.run(EventDispatchThread.java:122)
Back to top
Gregg Wonderly
Posted via mailing list.
PostPosted: Wed Oct 07, 2009 4:27 pm    Post subject: New Language Support Issues Reply with quote
David, it is clear that the editor is attempting to "annotate" the view with
information. The problem is you just can't do this on the EDT, no matter what.
It is very tempting to use the EDT to isolate and synchronize actions and
reactions. But in the end, netbeans needs a different strategy then what it is
currently using.

The problem is that paging and swapping (think how the RSS requirement has grown
because of all this cursory activity of always active code) can lead to the EDT
being the thread that causes the process to "page out data to run more code" or
something else. It is these moments that heavily influence the "odd nature" of
many of these reports.

They happen sometimes for some people and not others. We always ask "how much
memory" and "which OS" kinds of questions. While these can often influence the
speed of execution, they should very rarely impact the reactive nature of the GUI.

However, netbeans suffers terribly in this regard because, it appears, that the
developers don't have a perspective of "never use the EDT for computation" and
instead believe "you should only use the EDT for computations that are fast".

Paging and swapping and VM management on the part of the OS is an outside force
that you have to deal realistically with.

Windows has a terrible VM implementation. The active window gets to
aggressively expand it's RSS by consuming pages from idle windows. What then
happens is that when you activate one of those windows and try and do something,
there is tons of text to activate, and this means that both text and data from
the previously active window must be evicted. Data is expensive because it has
to be written to swap. That sends the disk head to the swap area. But then
text has to be loaded from the executable and so the head goes flying back
across the disk to get some more text.

This churn on the drive goes on painfully, forever, on machines with 1-2GB of
ram in the 32bit versions of windows. If you are using the EDT to run text that
is being brought back in, guess what? You get a GUI lock up, plain and simple.

So when the EDT calls out to these other classes in other packages and JAR
files, that's the most likely time that these odd lockups and non-responsive
moments will occur.

Every issue with the computing environment is important and must be part of the
software design.

Gregg Wonderly

David Strupl wrote:
Quote:
Gregg,

I would like to slightly correct your evaluation. The EDT from the
thread dump bellow is clearly used to react to the user typing. In the
editor in order to repaint the modified region we are trying to re-lex
only part that was modified by the user (for the user to get proper
colors etc).

The problem is that

com.agentfactory.netbeans.lexer.JavaParserTokenManager.getNextToken(JavaParserTokenManager.java:1561)

com.agentfactory.netbeans.lexer.AFAPL2Lexer.nextToken(AFAPL2Lexer.java:27)

must be fast and finish very quickly for the editor to be usable.

Please do not file bug titled "EDT parsing activity causes editor to
lock up" or similar and first try to debug the above mentioned calls to
figure out why they are looping ...

Thanks,

David

Gregg Wonderly napsal(a):
Quote:
The key thread trace is the one below which shows that the
AWT-EventQueue-1 thread (an Event Dispatch Thread or EDT) is busy
doing work that is not related to GUI painting and drawing. They
appear to be using the EDT to analyze changes to the source code.
This should not be happening. The EDT should only be used for
painting and preliminary event delivery. The EDT is (in netbeans
based on what I've seen in all the stack traces) often misused as a
form of locking to keep things from being changed elsewhere instead of
using appropriate locks elsewhere. The most typical reason for doing
this is because it keeps circular wait from happening where one thread
has a lock and makes something happen that triggers an EDT event that
then enters code using the same lock, but now with a different thread
wanting the lock.

This is not easy stuff to manage, but nevertheless when it is done
incorrectly, we get into these kinds of situations where the EDT is
misused for locking and the end result is clunky, unusable UI
functionality.

Please open an issue titled with the problem you are experiencing,
such as "EDT parsing activity causes editor to lock up" or some such.

Gregg Wonderly

"AWT-EventQueue-1" prio=6 tid=0x0000000106268800 nid=0x12936f000
runnable [0x000000012936c000]
java.lang.Thread.State: RUNNABLE
at
com.agentfactory.netbeans.lexer.JavaParserTokenManager.getNextToken(JavaParserTokenManager.java:1561)

at
com.agentfactory.netbeans.lexer.AFAPL2Lexer.nextToken(AFAPL2Lexer.java:27)

at
org.netbeans.lib.lexer.LexerInputOperation.nextToken(LexerInputOperation.java:190)

at
org.netbeans.lib.lexer.inc.TokenListUpdater.relex(TokenListUpdater.java:619)

at
org.netbeans.lib.lexer.inc.TokenListUpdater.updateRegular(TokenListUpdater.java:276)

at
org.netbeans.lib.lexer.inc.TokenHierarchyUpdate$UpdateItem.update(TokenHierarchyUpdate.java:345)

at
org.netbeans.lib.lexer.inc.TokenHierarchyUpdate.processLevelInfos(TokenHierarchyUpdate.java:220)

at
org.netbeans.lib.lexer.inc.TokenHierarchyUpdate.updateImpl(TokenHierarchyUpdate.java:192)

at
org.netbeans.lib.lexer.inc.TokenHierarchyUpdate.update(TokenHierarchyUpdate.java:130)

at
org.netbeans.lib.lexer.TokenHierarchyOperation.textModified(TokenHierarchyOperation.java:546)

at
org.netbeans.spi.lexer.TokenHierarchyControl.textModified(TokenHierarchyControl.java:93)

at
org.netbeans.lib.lexer.inc.DocumentInput.textModified(DocumentInput.java:151)

at
org.netbeans.lib.lexer.inc.DocumentInput.insertUpdate(DocumentInput.java:140)

at
org.netbeans.lib.editor.util.swing.PriorityDocumentListenerList.insertUpdate(PriorityDocumentListenerList.java:79)

at
javax.swing.text.AbstractDocument.fireInsertUpdate(AbstractDocument.java:185)

at
org.netbeans.editor.BaseDocument.fireInsertUpdate(BaseDocument.java:1623)
at
org.netbeans.editor.BaseDocument.insertString(BaseDocument.java:778)
at
org.netbeans.editor.BaseKit$DefaultKeyTypedAction.insertString(BaseKit.java:1093)

at
org.netbeans.editor.BaseKit$DefaultKeyTypedAction$1.run(BaseKit.java:1045)

at
org.netbeans.editor.GuardedDocument.runAtomicAsUser(GuardedDocument.java:354)

at
org.netbeans.editor.BaseKit$DefaultKeyTypedAction.actionPerformed(BaseKit.java:1016)

at
org.netbeans.editor.ext.ExtKit$ExtDefaultKeyTypedAction.actionPerformed(ExtKit.java:1040)

at
org.netbeans.editor.BaseAction.actionPerformed(BaseAction.java:325)
at javax.swing.SwingUtilities.notifyAction(SwingUtilities.java:1641)
at javax.swing.JComponent.processKeyBinding(JComponent.java:2851)
at javax.swing.JComponent.processKeyBindings(JComponent.java:2886)
at javax.swing.JComponent.processKeyEvent(JComponent.java:2814)
at java.awt.Component.processEvent(Component.java:6125)
at java.awt.Container.processEvent(Container.java:2085)
at java.awt.Component.dispatchEventImpl(Component.java:4714)
at java.awt.Container.dispatchEventImpl(Container.java:2143)
at java.awt.Component.dispatchEvent(Component.java:4544)
at
java.awt.KeyboardFocusManager.redispatchEvent(KeyboardFocusManager.java:1848)

at
java.awt.DefaultKeyboardFocusManager.dispatchKeyEvent(DefaultKeyboardFocusManager.java:704)

at
java.awt.DefaultKeyboardFocusManager.preDispatchKeyEvent(DefaultKeyboardFocusManager.java:969)

at
java.awt.DefaultKeyboardFocusManager.typeAheadAssertions(DefaultKeyboardFocusManager.java:841)

at
java.awt.DefaultKeyboardFocusManager.dispatchEvent(DefaultKeyboardFocusManager.java:668)

at java.awt.Component.dispatchEventImpl(Component.java:4586)
at java.awt.Container.dispatchEventImpl(Container.java:2143)
at java.awt.Window.dispatchEventImpl(Window.java:2475)
at java.awt.Component.dispatchEvent(Component.java:4544)
at java.awt.EventQueue.dispatchEvent(EventQueue.java:635)
at
org.netbeans.core.TimableEventQueue.dispatchEvent(TimableEventQueue.java:104)

at
java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:296)

at
java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:211)

at
java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:201)

at
java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:196)
at
java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:188)
at java.awt.EventDispatchThread.run(EventDispatchThread.java:122)


Back to top
Jesse Glick
Posted via mailing list.
PostPosted: Fri Oct 09, 2009 9:30 pm    Post subject: Re: New Language Support Issues Reply with quote
Gregg Wonderly wrote:
Quote:
netbeans suffers terribly in this regard because, it appears,
that the developers don't have a perspective of "never use the EDT for
computation" and instead believe "you should only use the EDT for
computations that are fast".

Ideally the EQ would be used only for actual rendering, and the app would always remain responsive regardless of paging, network disk accesses, etc. As a practical matter
the NB Platform (not to mention IDE) would need to be rewritten from scratch to achieve this, and very likely in a higher-level language.

Quote:
Windows has a terrible VM implementation. The active window gets to
aggressively expand it's RSS by consuming pages from idle windows. What
then happens is that when you activate one of those windows and try and
do something, there is tons of text to activate, and this means that
both text and data from the previously active window must be evicted.

That is why nbexec.dll passes -Dsun.awt.keepWorkingSetOnMinimize=true.
Back to top
Gregg Wonderly
Posted via mailing list.
PostPosted: Fri Oct 09, 2009 10:02 pm    Post subject: Re: New Language Support Issues Reply with quote
Jesse Glick wrote:
Quote:
Gregg Wonderly wrote:
Quote:
netbeans suffers terribly in this regard because, it appears, that the
developers don't have a perspective of "never use the EDT for
computation" and instead believe "you should only use the EDT for
computations that are fast".

Ideally the EQ would be used only for actual rendering, and the app
would always remain responsive regardless of paging, network disk
accesses, etc. As a practical matter the NB Platform (not to mention
IDE) would need to be rewritten from scratch to achieve this, and very
likely in a higher-level language.

I understand that "we are where we are" regarding this issue and that there is
no "simple solution." I also realize that I've been pushing pretty hard on this
issue, and believe me, I am not trying to be a nuisance. But I also believe
that there needs to be a change in mindset about how this stuff gets done, and I
also think that there is a chance for the netbeans developers and community to
create a new pathway for managing this issue. I want all of the netbeans
developers and community to realize that this is something that needs a better,
more organized approach to adequately address.

There's lots of history and experience amongst everyone here that can be a great
resource.

Quote:
Quote:
Windows has a terrible VM implementation. The active window gets to
aggressively expand it's RSS by consuming pages from idle windows.
What then happens is that when you activate one of those windows and
try and do something, there is tons of text to activate, and this
means that both text and data from the previously active window must
be evicted.

That is why nbexec.dll passes -Dsun.awt.keepWorkingSetOnMinimize=true.

Yes, that's a useful thing, but it doesn't seem to make as big of a difference
any more from my experience. Those that want can read about this at

<http://support.microsoft.com/kb/293215>

For those that want to try something different, you can go into the windows
control panel under System->Advanced->Performance->Settings->Advanced (path for
vista anyways), and select the "Background services" option for processor
resource allocation to make windows no longer give preference to desktop
applications and instead to run like a server where all processes get treated
fairly.

For me, this seems to make switching between different windows smoother.

Gregg Wonderly
Back to top
David Strupl
Posted via mailing list.
PostPosted: Sat Oct 10, 2009 7:35 pm    Post subject: EDT Was: New Language Support Issues Reply with quote
Jesse Glick napsal(a):
Quote:
Gregg Wonderly wrote:
Quote:
netbeans suffers terribly in this regard because, it appears, that the
developers don't have a perspective of "never use the EDT for
computation" and instead believe "you should only use the EDT for
computations that are fast".

Ideally the EQ would be used only for actual rendering

Huh? The EDT must be used not only for rendering (reading the AWT tree structure
and painting it) but also for model manipulation (writing to the AWT tree
model). Also please note what does the acronym EDT means: Event Dispatch Thread.
The actions that are result of the user interaction (e.g. the user hitting the
keyboard) should be handled directly in the EDT if they are not too costly.

Gregg, I think you should check actors in Erlang or Scala. But using them would
mean you would have to rewrite Swing from scratch. Pointers for further study
can be found e.g. here:
http://www.sauria.com/blog/2009/10/05/the-cambrian-period-of-concurrency/

Br,

David

--
David Strupl
http://dstrupl.blogspot.com
Back to top
Gregg Wonderly
Posted via mailing list.
PostPosted: Sun Oct 11, 2009 10:20 pm    Post subject: EDT Was: New Language Support Issues Reply with quote
David Strupl wrote:
Quote:
Jesse Glick napsal(a):
Quote:
Gregg Wonderly wrote:
Quote:
netbeans suffers terribly in this regard because, it appears, that the
developers don't have a perspective of "never use the EDT for
computation" and instead believe "you should only use the EDT for
computations that are fast".
Ideally the EQ would be used only for actual rendering

Huh? The EDT must be used not only for rendering (reading the AWT tree structure
and painting it) but also for model manipulation (writing to the AWT tree
model). Also please note what does the acronym EDT means: Event Dispatch Thread.
The actions that are result of the user interaction (e.g. the user hitting the
keyboard) should be handled directly in the EDT if they are not too costly.

Gregg, I think you should check actors in Erlang or Scala. But using them would
mean you would have to rewrite Swing from scratch. Pointers for further study
can be found e.g. here:
http://www.sauria.com/blog/2009/10/05/the-cambrian-period-of-concurrency/

Hi David, thanks for jumping in with your perspective. I'm not suggesting that
we need to rewrite swing for scratch, but rather that some applications need to
be changed to better deal with the overall issues that can effect how the
application is perceived. The absolute best thing is to keep the least amount
of work from running in the EQ as possible. The bigger issue, is that there are
not real tools available that make this simple to do.

Frameworks have been built that do simplify the task from several perspectives.
But, my perception, is that overall, this issue has not been appreciated in
complexity nor for the barriers that the issue creates.

I've constructed some simple tools (simple from my perspective) that I use to
try and manage this. I've grown to live with doing things like

final int fidx = i;
runInSwing( new Runnable() { public void run() {
lab.setText("Step "+fidx );
}});

where I must declare a new final variable to expose something to an anonymous
instance of some class, and have many lines of overhead of code to do something
simple.

We have this thing called annotations now. I had hoped that Sun would be all
over solving problems like this. Since Java is now open source, this kind of
thing will have to be done by communities of people now.

The netbeans community could be at the forefront of such a thing. That's what
I'm most interested in pushing. With all the experience that people in the
community have had, I thought that there would be a great opportunity to focus
all that experience on a solution that might be the "answer" to getting swing
threading to finally not be a barrier and ball-n-chain around the developers
feet, but rather something that is done by APIs that surround a desktop
application framework such as what netbeans provides in the RCP.

Surely I am not the only one that strives hard to make usable desktop
applications by making sure that the EQ work is as focused on the GUI experience
as possible?

Gregg Wonderly
Back to top
Tim Boudreau
Posted via mailing list.
PostPosted: Mon Oct 12, 2009 4:07 am    Post subject: EDT Was: New Language Support Issues Reply with quote
Gregg Wonderly-2 wrote:
Quote:

Surely I am not the only one that strives hard to make usable desktop
applications by making sure that the EQ work is as focused on the GUI
experience
as possible?


Generally, yes this is a good idea.

For better or worse, single-core machines will be with us for some time in
the future. On a single-core machine, everything is effectively
single-threaded - there, in the end, multithreading buys you nothing. I
remember that Jesse did a bunch of prototyping various threading models in
the performance work for 3.5, and in the end the conclusion was that just
doing everything in the event thread actually was the best performing option
(context switching has built-in overhead, even more so on single core
machines).

While less true in the case of multi-core machines, it is still a
consideration.

In a purist sense, you are right, doing only things directly related to the
UI on the event thread is the only way to guarantee that...only things
directly related to the UI are done on the event thread. How one defines
"directly related to the UI" is where things get messy. In practice, there
are some things that are reliably *not* a good idea to do in the event
thread (socket I/O, non-trivial file I/O [possibly all file I/O - but this
gets complicated by the fact that even File.lastModified() or
File.isDirectory() is a blocking call]). For the rest, the programmer must
guess what they think will be slow and what they think will be fast - and
not only is that going to be a guess, it is a guess about a moving target.

Then there's the matter that synchronous code is much easier to read and
maintain, which adds a point in favor of not doing things asynchronously.

Any API call that does serious amounts of I/O should probably start with
"assert !EventQueue.isDispatchThread()". APIs also can and should guarantee
asynchronous calls where warranted - in such a world, there would be no
FileObject.getInputStream(), there would be
class FileObject {
public void read (FileReader reader) {...}
interface FileReader {
public void read (InputStream in);
}
}
and read(FileReader) might schedule the read on a background thread (or, on
a single CPU system, could run it synchronously w/o modifications). This
isolates the I/O nicely so that it can be scheduled appropriately. It has a
few problems:
- It's pretty much guaranteed that, sooner or later, somebody will make the
assumption that the call to FileReader.read() hasn't completed yet, after a
call to FileObject.read() and, say, try to read old data that just got
overwritten if it *was* called synchronously
- The code to use this API ends up being more complicated than a
synchronous alternative
- If there are multiple background threads involved, people using the API
are likely to make assumptions about ordering that are sometimes wrong
- Once you've scheduled something to happen in the background, there's no
*guarantee* that it will *ever* complete, or when - things get much less
deterministic
- Sometimes you Really Really Need to read or write data synchronously, or
read/write a whole bunch of files in one pass, so you probably will need an
API to do that synchronously anyway...and so that's what everybody will
actually use. Or you will end up returning a Condition from
FileObject.read() and somewhere you'll end up with yet another thread
waiting for a whole bunch of return values from a bunch of calls to
FileObject.read() to signal that they've completed in order to do something
- Too much of this stuff, and you really end up with a message based system
- which is not necessarily a bad thing, but is pretty alien to someone
expecting to do fairly straightforward Java coding

Those caveats aside, I frequently do use this general idiom (although at the
moment I'm making myself question the practice Smile).

Swing, however, and NetBeans APIs for both historical and practical reasons,
does not *enforce* any sort of threading model for I/O or heavy computation
- that is, there is nothing to stop someone from mutating a model from a
random thread. NetBeans has synchronous APIs call for doing I/O
(http://www.netbeans.org/issues/show_bug.cgi?id=65135 is an example of how
quite innocuous calls can lead to serious problems) as does the JDK. Ever
making all such I/O (every lookup of something in the System Filesystem,
every call to DataObject.find(), every call to Lookup.lookup(), etc. - a lot
of which will be sufficiently fast 99.999% of the time, and some of which
make it non-obvious whether blocking I/O is or isn't involved) be done
through APIs that enforce asynchronous (or at least non-EQ) access would
mean a truly massive effort. For new code, please do I/O in the background
- sure. But I doubt it's practical to do this kind of refit to the existing
codebase, and when introducing such constructs, it's worth considering how
hard it will be for someone else to understand the resulting code, since
that has a cost as well.


BTW, one somewhat unusual practice that can perform better than one might
expect is to do your work on the event thread, but serialize it into a
series of jobs that run sequentially on the event thread, allowing input
events to be handled in between. Which also gives you code ready to be run
on a background thread if it seems necessary at some point.

-Tim
--
View this message in context: http://www.nabble.com/New-Language-Support-Issues-tp25754545p25850298.html
Sent from the Netbeans RCP/Platform Users (Open API) mailing list archive at Nabble.com.
Back to top
Gregg Wonderly
Posted via mailing list.
PostPosted: Mon Oct 12, 2009 8:38 pm    Post subject: EDT Was: New Language Support Issues Reply with quote
Tim Boudreau wrote:
Quote:
For new code, please do I/O in the background
- sure. But I doubt it's practical to do this kind of refit to the existing
codebase, and when introducing such constructs, it's worth considering how
hard it will be for someone else to understand the resulting code, since
that has a cost as well.

BTW, one somewhat unusual practice that can perform better than one might
expect is to do your work on the event thread, but serialize it into a
series of jobs that run sequentially on the event thread, allowing input
events to be handled in between. Which also gives you code ready to be run
on a background thread if it seems necessary at some point.

Okay, I don't want to drag this on, but I do want something concrete to
demonstrate the issue. Below is a class that uses my Packer class
(http://packer.dev.java.net) for layout, and the CircularListModel stuff from my
swingutil project (http://swingutil.dev.java.net) but is otherwise standalone.

This class provides a simple example that demonstrates how the use of the EDT
can make a UI more clunky than it needs to be. There are two controls at the
bottom of the opened window that control the behavior of the application. The
checkbox determines whether the EDT is used in a blocking way to do I/O, or if
that I/O is done in the background. In either case (checked or not) the
component updates are done on the EDT as required. Second control is a time
delay value selection. It controls how long the EDT is delayed as I/O occurs to
demonstrate the hesitation that any use of the EDT for non GUI events can
create. If you set the time value to zero and then use the check box to use the
EDT or not, you might notice no particular difference in performance of the two
different mode of operations.

This is what is being argued about in this discussion. I contend that even that
experience is not a justification for using the EDT. The reason I make that
argument is based on what happens if you select a non-zero value. In the case
that you pick a non-zero value, try selecting something from one of the combo
boxes, or typing repeating text in the text fields etc. with the use of the EDT
and without the use of the EDT. Click the checkboxes as fast as you can. Click
a combo box and drag the mouse up and down. Notice the difference in the
behavior of the controls when you use the EDT vs when you don't.

It's those pauses and glitchy behavior in how selection and focus change that I
contend is really what should be focused on.

It's a personal issue for me, that I can't really feel good about deciding it
takes too much effort. I think it really is an important issue and that
deciding to waste 1000's of peoples minutes of productivity just so that I can
spend 10 or 20 less minutes of development time just doesn't seem equitable as
an argument in favor of just using the EDT.

Don't get me wrong, I don't think it will take 10 or 20 minutes to fix netbeans
so that it's perfect. I'm just saying as a general rule, it might take me 10 or
20 minutes of work in an application to make sure the EDT usage is minimized.
Part of this is because I always use the tools I've mentioned in previous posts,
and part of it is just how I organize my applications in terms of responsibilities.

There are ways to improve netbeans use of the EQ and stuff that the EDT has to
do. I'm not an expert on the internals of netbeans. However, I do see a pretty
predominate trend to not deal with the EDT use, and that kind of sets off alarms
in my head. If it is something that is going to be changed, the best thing, I
think, would be to do some surveys of the code and see if there is not an API
kind of development effort that might make it easier for people to manage this
issue.

Gregg Wonderly

------------------------------------------------------------------------------


/*
* To change this template, choose Tools | Templates
* and open the template in the editor.
*/

package edtexample;

import java.awt.event.ActionEvent;
import java.awt.event.ActionListener;
import java.awt.event.WindowAdapter;
import java.awt.event.WindowEvent;
import java.io.DataInputStream;
import java.io.DataOutputStream;
import java.io.IOException;
import java.net.InetAddress;
import java.net.InetSocketAddress;
import java.net.ServerSocket;
import java.net.Socket;
import java.net.UnknownHostException;
import java.util.LinkedList;
import java.util.logging.Level;
import java.util.logging.Logger;
import javax.swing.BorderFactory;
import javax.swing.JCheckBox;
import javax.swing.JComboBox;
import org.wonderly.awt.Packer;
import javax.swing.JFrame;
import javax.swing.JLabel;
import javax.swing.JList;
import javax.swing.JPanel;
import javax.swing.JScrollPane;
import javax.swing.JSeparator;
import javax.swing.JTextField;
import javax.swing.SwingUtilities;
import javax.swing.Timer;
import org.wonderly.swing.CircularListModel;
import org.wonderly.swing.ListListModel;
import org.wonderly.swing.event.LifeCycleElement;
import org.wonderly.swing.event.ElementLifeCycleManager;

/**
* This isn't very pretty code, I just threw it together to get
* something working.
* @author gregg
*/
public class Main extends JFrame implements ElementLifeCycleManager {
ElementLifeCycleManager mgr = this;
Logger log = Logger.getLogger(getClass().getName());
/**
* @param args the command line arguments
*/
public static void main(String[] args) throws Exception {
new Main(args);
}

private class Entity implements LifeCycleElement {
String val;
public String toString() {
return val;
}
public void setValue( int v ) {
this.val = "Value="+v;
}
}
LinkedList<Entity>ls = new LinkedList<Entity>();
public void freeElement( LifeCycleElement elem ) {
synchronized( ls ) {
ls.add( (Entity)elem );
}
}
public Entity nextElement( int i ) {
Entity e = null;
synchronized( ls ) {
if( ls.isEmpty() == false )
e = ls.pop();
}
if( e == null ) {
e = new Entity();
}
e.setValue(i);
return e;
}

private class SocketGen implements Runnable {
ServerSocket s;
volatile boolean done;
public void run() {
while(true) {
try {
log.info("bound on: "+getAddress());
Socket sk = s.accept();
log.info("got connection from: "+sk);
serviceSocket( sk );
} catch( Exception ex ) {
log.log(Level.SEVERE, ex.toString(), ex);
}
}
}
public SocketGen() throws IOException {
s = new ServerSocket(0);
new Thread(this,"SocketGen").start();
}
public InetSocketAddress getAddress() throws UnknownHostException {
return (InetSocketAddress)new InetSocketAddress(
InetAddress.getByName("127.0.0.1"), s.getLocalPort() );
}
private void serviceSocket( final Socket s ) throws IOException {
final DataOutputStream os = new DataOutputStream(
s.getOutputStream() );
final DataInputStream is = new DataInputStream( s.getInputStream() );
new Thread() {
int idx = 0;
public void run() {
try {
while( !done ) {
try {
log.info("reading value");
int idx = is.readInt();
log.info("Read value: "+idx+", returning result");
os.writeInt( ++idx );
os.flush();
} catch( Exception ex ) {
log.log(Level.SEVERE, ex.toString(), ex);
done = true;
}
}
} finally {
try {
s.close();
} catch (IOException ex) {
log.log(Level.SEVERE, ex.toString(), ex);
}
}
}
}.start();
}
}

volatile boolean locked;
Timer evTimer;
Object lock = new Object();
public void setLocking( boolean how ) {
if( how ) {
locked = true;
evTimer.start();
} else {
evTimer.stop();
locked = false;
}
synchronized( lock ) {
lock.notifyAll();
}
}
long sleepVal = 50;
private class SocketReader implements Runnable {
final Socket s;
final DataOutputStream os;
final DataInputStream is;
public SocketReader( InetSocketAddress addr ) throws IOException {
log.info("connecting to socket on "+addr );
s = new Socket( addr.getAddress(), addr.getPort() );
os = new DataOutputStream( s.getOutputStream() );
is = new DataInputStream( s.getInputStream() );
evTimer = new Timer(3,new ActionListener() {
public void actionPerformed( ActionEvent ev ) {
lockedUpdate();
}
});
locked = true;
evTimer.start();
new Thread( this ).start();
}
public void run() {
while( !done ) {
if( !locked ) {
unLockedUpdate();
} else {
synchronized( lock ) {
try {
lock.wait();
} catch (InterruptedException ex) {
log.log(Level.SEVERE, ex.toString(), ex);
}
}
}
}
}
volatile boolean done;
int v = 0;
public void lockedUpdate() {
try {
log.info("locked client writing "+v);
if( sleepVal > 0 ) Thread.sleep(sleepVal);
os.writeInt( v );
os.flush();
if( sleepVal > 0 ) Thread.sleep(sleepVal);
v = is.readInt();
log.info("locked client read: "+v);
mod.add(nextElement(v));
if( sleepVal > 0 ) Thread.sleep(sleepVal);
} catch( Exception ex ) {
log.log(Level.SEVERE, ex.toString(), ex);
}
}
public void unLockedUpdate() {
try {
log.info("unlocked client writing "+v);
if( sleepVal > 0 ) Thread.sleep(sleepVal);
os.writeInt( v );
os.flush();
if( sleepVal > 0 ) Thread.sleep(sleepVal);
final int vv = v = is.readInt();
log.info("unlocked client read: "+v);
SwingUtilities.invokeAndWait( new Runnable() { public void run() {
mod.add(nextElement(vv));
}});
if( sleepVal > 0 ) Thread.sleep(sleepVal);
} catch( Exception ex ) {
log.log(Level.SEVERE, ex.toString(), ex);
}
}
}
int idx = 0;
SocketGen sg;
SocketReader sr;
CircularListModel<Entity> mod;
private Main(String[] args) throws IOException {
super( "Example of EDT preformance");
Packer pk = new Packer( getContentPane() );
mod = new CircularListModel<Entity>(200);
final JList list = new JList( mod );
pk.pack( new JScrollPane( list ) ).gridx(0).gridy(0).fillboth();

addWindowListener(new WindowAdapter() {

public void windowClosing(WindowEvent ev) {
System.exit(0);
}
});
JPanel bp = new JPanel();
Packer bpk = new Packer( bp );
pk.pack( bp ).gridx(0).gridy(1).fillx();
int y = -1;
for( int i = 0; i < 5; ++i ) {
bpk.pack( new JLabel("Label #"+(i+1))
).gridx(0).gridy(++y).inset(2,2,0,0);
bpk.pack( new JTextField(10)
).gridx(1).gridy(y).inset(2,0,0,2).fillx();
ListListModel<String>cl = new ListListModel<String>();
bpk.pack( new JCheckBox("Selection "+(i+1))
).gridx(2).gridy(y).inset(2,0,0,2);
for( int j = i+1; j < i+10; ++j )
cl.add( "Choice #"+j);
JComboBox cb;
bpk.pack( cb = new JComboBox( cl )
).gridx(3).gridy(y).inset(2,0,0,2).fillx();
cb.setSelectedIndex(0);
}
pk.pack( new JSeparator() ).gridx(0).gridy(2).fillx().inset(5,5,5,5);
JPanel cp = new JPanel();
Packer cpk = new Packer( cp );
pk.pack( cp ).gridx(0).gridy(3).fillx().inset(3,3,3,3);
final JCheckBox b = new JCheckBox("Use EQ for I/O");
b.setSelected(true);
final JComboBox dly = new JComboBox();
cp.setBorder( BorderFactory.createEtchedBorder() );
cpk.pack( b ).gridx(0).gridy(0).fillx().west();
cpk.pack( new JLabel("Threading Sleep Amount")
).gridx(1).gridy(0).inset(0,10,0,0);
cpk.pack( dly ).gridx(2).gridy(0);
cpk.pack( new JLabel("millis") ).gridx(3).gridy(0);
final ListListModel<Integer> ls = new ListListModel<Integer>();
dly.setModel( ls );
for( int i = 0; i < 320; i += 40 ) {
ls.add(i);
}
dly.setSelectedIndex(2);
dly.addActionListener(new ActionListener() {

public void actionPerformed(ActionEvent ev) {
sleepVal = ls.get(dly.getSelectedIndex());
}
});
sleepVal = ls.get(dly.getSelectedIndex());
b.addActionListener(new ActionListener() {

public void actionPerformed(ActionEvent ev) {
setLocking( b.isSelected() );
}
});
sg = new SocketGen();
sr = new SocketReader( sg.getAddress() );
pack();
setLocationRelativeTo(null);
setVisible(true);
}
}
Back to top
Ivan Soleimanipour
Posted via mailing list.
PostPosted: Mon Oct 12, 2009 9:20 pm    Post subject: EDT Was: New Language Support Issues Reply with quote
On 10/12/09 13:38, Gregg Wonderly wrote:

Quote:
The checkbox determines whether the EDT is used in a
blocking way to do I/O,

The way this _used_ to be done before MT GUI's and Java was through the use
of poll() or select(). poll() would poll input of any kind, including
graphical events and you had only one thread to work with.

When you use poll (correctly) then reads on an EDT should not block.
Note that the polling is not done by you in the EDT but by the system
which gives equal access to event dispatch and drawing and any other
i/o events.

One problem is that Java doesn't have a good poll mechanism.
There is nio.selectors but it doesn't quite fit the bill as it's
designers decide to bundle in asynchronous reads as well.
I would've also have like to instruct the EDT thread to dispatch
selector events.

In sunstudio, to have the gui talk to dbx, I "simulate" poll using
blocking waiter threads.

The dispatching of "stuff is available for reading" events happens on a helper
"notifier" thread but the reading of the sockets happens on the EDT. The
debugger is quite responsive even under mad clicking of StepInto.
Or rather, I should say, I haven't traced any performance problems to
"EDT hogging".

One time we did a performance analysis and discovered (as Tim alluded that
Jesse did) that the overhead of context switching is quite significant.
The time it takes for the waiter thread to transfer the control
to the notifier and then to the EDT took a big chunk out of available cpu cycles.

In a sense the architecture we have is effectively what Tim called a
message passing architecture. The messages are passed to/from an external
process but the main work of this is happening on the EDT.

In fact the whole architecture of Sun's pre-NB IDE (workshop)
was message-passing based with several processes co-operating and each having it's
EDT to do gui's and message handling. So my ideal system
would be a system that allows for multiple general purpose EDT's
so-to-speak. But that in turn starts favoring SDI style UI's and
"tool"ism but that's another story.
Back to top
Jesse Glick
Posted via mailing list.
PostPosted: Mon Oct 12, 2009 11:45 pm    Post subject: Re: EDT Was: New Language Support Issues Reply with quote
Tim Boudreau wrote:
Quote:
I remember that Jesse did a bunch of prototyping various threading models in
the performance work for 3.5, and in the end the conclusion was that just
doing everything in the event thread actually was the best performing option

I sholuld clarify that "everything" in this context meant "manipulation of Node-like structures backed by java.io.File's on a local disk", and like any other benchmark it
is probably obsolete by now.
Back to top
Gregg Wonderly
Posted via mailing list.
PostPosted: Tue Oct 13, 2009 12:08 am    Post subject: EDT Was: New Language Support Issues Reply with quote
Ivan Soleimanipour wrote:
Quote:
On 10/12/09 13:38, Gregg Wonderly wrote:

Quote:
The checkbox determines whether the EDT is used in a blocking way to
do I/O,

The way this _used_ to be done before MT GUI's and Java was through the use
of poll() or select(). poll() would poll input of any kind, including
graphical events and you had only one thread to work with.

In the single processor, single core world, this was an okay thing because it
really did make things more efficient to not context switch for I/O operations.
Now days, multi-core and/or hyper-threading etc make it more interesting to
tailor to those specific processors because they provide such a dramatically
better experience, and the context switching that occurs between threads has
gotten to be a lot more performant in the past several years as we've learned
more about what works and doesn't work in OSes supporting
multi-core/MP/hyper-threaded processor machines.

Quote:
When you use poll (correctly) then reads on an EDT should not block.
Note that the polling is not done by you in the EDT but by the system
which gives equal access to event dispatch and drawing and any other
i/o events.

In general, it's the latency that is the biggest issue. Latency can come from
waiting for a blocking I/O operation to have data to read, or buffer space to
write to. It can also be due to the OS paging and swapping or higher priority
processing, which in the end, the application will be hard pressed to combat.
However, because the EDT is in general, running in a handful of pages from a
text and data perspective (until an event is dispatched), the EDT will hardly
ever be blocked from running, and GUI events in the OS have sufficient priority
generally to get to the application fairly quickly (microseconds of latency).

When many (10-100) milliseconds of latency are introduced for event
"processing", that's when the application starts to feel clunky, and when
milliseconds expand to seconds because I/O events or computationally intensive
operations (lexical analysis of a block of code for example), the user can't
multi-task to get other things done such as editing a property sheet or typing
into an editor window while scanning is happening because they checked out some
changes from the SCM, and now need to wait for the update to GEL in the index.

Quote:
One problem is that Java doesn't have a good poll mechanism.
There is nio.selectors but it doesn't quite fit the bill as it's
designers decide to bundle in asynchronous reads as well.
I would've also have like to instruct the EDT thread to dispatch
selector events.

In sunstudio, to have the gui talk to dbx, I "simulate" poll using
blocking waiter threads.

The dispatching of "stuff is available for reading" events happens on a
helper
"notifier" thread but the reading of the sockets happens on the EDT. The
debugger is quite responsive even under mad clicking of StepInto.
Or rather, I should say, I haven't traced any performance problems to
"EDT hogging".

Perhaps, in general, the average latency of what you are doing on the EDT is
seldom an issue. You might try unplugging your ethernet cable that provides the
network connection to a remote debugging session and then see how well the IDE
lets you continue to do your work, including shutting down the debugging session.

It is possible, through clever design, to circumvent some of the issues that I'm
discussing. It's those clever designs though, that are often so "odd" that they
then drive further clever designs that make it very difficult to refactor
something that really needs a different implementation.

Again, I'm trying to make the point that we need to think about API toolkits
that make more of this stuff happen naturally so that users don't have to worry
about knowing when the EDT is used or not. APIs should be designed to segment
sets of steps into things that would nominally happen in the EDT from things
that would happen in a background thread. The API would then make it easy for
implementations of that API to call into user code with the correct environment
already in place. The SwingWorker kinds of APIs already try and do that.

My SyncThread class provide three contexts to work on changing the content or
state of a component. The preparation for the change happens in setup() on the
EDT so that components can be updated to disable them, change selection, grab
selected text etc. The run() method can then be used to interact with the
outside world to get data, process data that you got in setup() etc. Finally,
the done() method lets you take the new state of the world and present it in the
components while running on the EDT.

It's a simple thing to take this class, subclass it, add an interface for some
type of event handler, and then take that kind of event and segment it's
processing into these sets of steps.

You mentioned toolism below, and creating APIs can get out of hand, but they
also can present the easy way to create plugable implementations that change how
things are done when needed instead of having to rewire lots of stuff because
too much knowledge of the implementation is spread around in too many places.

Quote:
One time we did a performance analysis and discovered (as Tim alluded that
Jesse did) that the overhead of context switching is quite significant.
The time it takes for the waiter thread to transfer the control
to the notifier and then to the EDT took a big chunk out of available
cpu cycles.

It can be an important thing to request that a small amount of context switches
happen. Passing events into the EQ happens naturally with the update of
component models in swing and in netbeans use of the listener and swing
component model mechanisms. You always have to do something in the EQ, it's
deciding how much to do outside of the EQ that controls the cost of context
switching. The more latency that is occurring outside of the EQ, the less the
switch to the EQ should really be visible to the user.

If you run my example where events can be set to pump continuously (time value
of 0) into the model, you can see a bit of a lag occur when you use the EDT as
well as when you don't. But, it doesn't make things happen near as oddly as
when the latency is in the EDT getting back to the next event. So it seems to
me that its the low bandwidth high latency things that really are the issue, and
those seem to be ever present in the issues that I've found and reported.

I am the most frustrated by the use of the EDT for doing things where one can't
control the latency involved. It's those things that I've found in the IDE
where I've had to wait for 10s of seconds to do the next thing in a recurring
fashion that very much affect my productivity.

Quote:
In a sense the architecture we have is effectively what Tim called a
message passing architecture. The messages are passed to/from an external
process but the main work of this is happening on the EDT.

In fact the whole architecture of Sun's pre-NB IDE (workshop)
was message-passing based with several processes co-operating and each
having it's
EDT to do gui's and message handling. So my ideal system
would be a system that allows for multiple general purpose EDT's
so-to-speak. But that in turn starts favoring SDI style UI's and
"tool"ism but that's another story.

Well, I think it's okay to choose to use multiple threads to process complex
eventing. I just think it's not a good idea to use the EDT to do all of the
processing. You should only do the things that really need the EDT to be done
effectively. There are simple things like sorting where (using SyncThread for
this illustration) one might code the following.

ServerIntf srvr = ...
final ListListModel<String> mod = new ListListModel<String>();
final JButton fill = ...
final JList list = new JList( mod );
fill.addActionListener( new ActionListener() {
public void actionPerformed( ActionEvent ev ) {
new SyncThread<List<String>,List<String>>( fill, list ) {
public List<String>run() {
try {
return srvr.getList();
} catch( Exception ex ) {
reportException(ex);
}
}
public void done() {
List<String> list = getValue();
Collections.sort( list );
mod.setContents( list );
}
}.start();
}
});

Some people would put the sort inside of the EDT execution as I have done here
for any number of reasons.

I think it is vital to put the sort into run() because you don't actually have
the ability to control the size of the list nor the implementation of the
Comparable and thus you can't manage the latency of that operation to make sure
it's not an issue for the users.

We've dragged this on for quite a stretch. Hopefully everyone has gained some
insight into the issues that exist on both the netbeans front and on the "use of
the EDT" front.

Gregg Wonderly
Back to top
Display posts from previous:   
Post new topic   Reply to topic    NetBeans Forums -> NetBeans Platform Users All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You can attach files in this forum
You can download files in this forum


Powered by phpBB
By use of this website, you agree to the NetBeans Policies and Terms of Use. © 2012, Oracle Corporation and/or its affiliates. Sponsored by Oracle logo