Antlr questions

[jeff_johnston_mn]

Great work! You filled in the blanks of some of the things I had not gotten too!

-Jeff Johnston


On Sat, Aug 1, 2009 at 12:52 PM, areeda <address-removed ([email]address-removed[/email])> wrote:

[jeff_johnston_mn]

[jeff_johnston_mn]

Hi Joe,

I do not mind at all if you work on improving the tutorial! I just wanted to get something started as there was nothing on the wiki for developers using ANTLR. I plan on implementing your changes in my project as well.

Your timing on this email is great because I spent a few hours last night trying to fix this method. So, the reason it resolves to an asterisk is because my lexer tokens changed...it used to resolve to whitespace I believe. And, the reason that I even did (do) that is because ANTLR thinks that it is at the end of the file, but Netbeans is reading a newline ("\n") character. What I was doing was just returning a token to satisfy the Netbeans lexing.

Although I admit it is a hack (I had actually forgotten I did that), it worked in Netbeans 6.5. Now in Netbeans 6.7 it no longer works. So now when I open up a blank document the lexing fails. Again, ANTLR thinks there is nothing left in the file (and returns EOF), but the document object has a newline character in it and the Netbeans lexing wants me to do something with it. If I return null I get the message "Chars: "\n" - these characters need to be tokenized.".

[jeff_johnston_mn]

Ok, finally found it...totally my fault. In the SqlLanguageHierarchy class my tokens were not being created. I am now using a static initializer to avoid making this mistake again.

I also switched the tutorial to use an enum class for the ANTLR tokens. I needed an enum for the code formatting so, in general, I thought this was a better approach.

Getting this error was not a complete waste of time though, as it did force me to learn how to build the Netbeans platform so I could debug my code! I wrote how to do that here:

http://wiki.netbeans.org/DevFaqNetBeansPlatformManager

-Jeff Johnston


On Tue, Aug 4, 2009 at 7:04 AM, Jeff Johnston <jeff.johnston.mn@gmail.com> wrote:

[servercimen]

Hello,
I followed your tutorial at http://wiki.netbeans.org/New_Language_Support_Tutorial_Antlr (only for the lexer part). However in the end I get following error whenever I open and try to edit an SQLR file.

java.lang.IllegalStateException: Lexer com.sqlraider.editor.SqlLexer@6adae2
returned null token but lexerInput.readLength()=1
lexer-state: null
tokenStartOffset=1, readOffset=2, lookaheadOffset=3
Chars: "\n" - these characters need to be tokenized.
Fix the lexer to not return null token in this state.

I think you pointed same type of error in your previous post. I am using the SqlLanguageHierarchy that is provided currently in the tutorial which, I think, includes the static initializer.