FeaturesPluginsDocs & SupportCommunityPartners

NetBeans Forums

 FAQFAQ   SearchSearch   MemberlistMemberlist   RegisterRegister   ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 
  

Antlr questions
Goto page 1, 2  Next
 
Post new topic   Reply to topic    NetBeans Forums -> NetBeans Platform Users
View previous topic :: View next topic  
Author Message
areeda



Joined: 28 Aug 2008
Posts: 88

PostPosted: Tue Jul 28, 2009 7:22 am    Post subject: Antlr questions Reply with quote

I am having fun with antlr and antlrWorks but getting down to the nitty-gritty I find I'm missing something.

1. How do you add antlr-runtime.jar to a module project? I've spent a fair amount of time searching with no luck.

2. How are people implementing the tokenId list? As I go through the thousands of sample files I'm finding things that weren't in the BNF description I was working from. Not a big deal really but when I add a new token do I have to regenerate the list?

Thanks,
Joe
Back to top
View user's profile Send private message
areeda



Joined: 28 Aug 2008
Posts: 88

PostPosted: Tue Jul 28, 2009 8:03 am    Post subject: Re: Antlr questions Reply with quote

areeda wrote:
1. How do you add antlr-runtime.jar to a module project? I've spent a fair amount of time searching with no luck.

I think I found the answer to this one.

If I put the runtime in the base directory and add the following lines to project.properties

cp.extra=\
${basedir}/antlr-runtime.jar

Is that a good way to do it?

Joe
Back to top
View user's profile Send private message
Jonny Heggheim
Posted via mailing list.





PostPosted: Tue Jul 28, 2009 8:13 am    Post subject: Antlr questions Reply with quote

You should make a new library wrapper module that contains the ANTLR
jar-file, and add the new ANTLR-module as a dependency.


Jonny Heggheim

On Tue, Jul 28, 2009 at 10:03 AM, areeda<address-removed> wrote:
Quote:

areeda wrote:
Quote:
1.
Back to top
jeff_johnston_mn



Joined: 12 Apr 2009
Posts: 38

PostPosted: Tue Jul 28, 2009 11:38 am    Post subject: Antlr questions Reply with quote

A great resource for questions like this is the developer FAQ wiki page:
http://wiki.netbeans.org/NetBeansDeveloperFAQ

Under there look for "What is a library wrapper module and how do I use it?". That should be exactly what you are looking for.


Quote:
How are people implementing the tokenId list?
Back to top
View user's profile Send private message
areeda



Joined: 28 Aug 2008
Posts: 88

PostPosted: Tue Jul 28, 2009 8:38 pm    Post subject: Re: Antlr questions Reply with quote

Jonny Heggheim wrote:
You should make a new library wrapper module that contains the ANTLR
jar-file, and add the new ANTLR-module as a dependency.

Sounds easy, but I'm missing something basic. I've spent the whole day reading about library wrappers and dependencies.

I see how to create a wrapper (I think) by using the wizard to add a "Java SE Library Descriptor" which creates an xml file. I put a copy of the antlr-runtime.jar in ${basedir}/ext.
Is that correct?

Now as to how to add a dependency on that, I'm completely lost. If go to the library properties or right click on Libraries and try to add a Module Dependency, my only choices are NetBeans modules my newly created wrapper is not there.

Also in the log file I have errors like:
Cannot create library: antlr-runtime of unknown type: j2se
Cannot create library: jaxb of unknown type: j2se
Cannot create library: javac-api of unknown type: j2se

and a few others.

I appreciate your help. I know I'm missing something very basic.

Joe
Back to top
View user's profile Send private message
jeff_johnston_mn



Joined: 12 Apr 2009
Posts: 38

PostPosted: Tue Jul 28, 2009 8:55 pm    Post subject: Antlr questions Reply with quote

You have to restart Netbeans for it to see the library you added...I get burned by that quite a bit.

-Jeff Johnston


On Tue, Jul 28, 2009 at 3:39 PM, areeda <address-removed ([email]address-removed[/email])> wrote:
Quote:

Jonny Heggheim wrote:
Quote:
You should make a new library wrapper module that contains the ANTLR

Quote:
jar-file, and add the new ANTLR-module as a dependency.



Sounds easy, but I'm missing something basic.
Back to top
View user's profile Send private message
areeda



Joined: 28 Aug 2008
Posts: 88

PostPosted: Wed Jul 29, 2009 1:15 am    Post subject: Re: Antlr questions Reply with quote

jeff_johnston_mn wrote:
You have to restart Netbeans for it to see the library you added...I get burned by that quite a bit.

-Jeff Johnston




It wasn't that easy. I do appreciate your quick answers and help Jeff.

I put about 10 hrs in today reading about modules, wrappers, and libraries. I am learning things and am making some progress.

I get an exception NoClassDefFoundError for AntlrCharStream, which used to say because of antlr.runtime.CharStream, now it doesn't say "because" any more. But now I'm without a clue, a cntrl-B in the lexer where it's constructed takes me to the right constructor, project builds without error, everything is in the same project. Both arguments are valid at the breakpoint.

I'm taking a break. I know this is a simple thing, I just have to find it. I guess I'll reread the ClassLoader stuff again tomorrow morning.

Joe
Back to top
View user's profile Send private message
jeff_johnston_mn



Joined: 12 Apr 2009
Posts: 38

PostPosted: Wed Jul 29, 2009 11:34 am    Post subject: Antlr questions Reply with quote

Once you bring in your library you do need to make those jar packages public. So on your library wrapper module go to the project properties, then API versioning, and then select the packages that you want to expose. I usually just select everything.

I should also amend that you probably only need to restart Netbeans if you add more than one jar to your library wrapper module. That makes sense to because Netbeans does not know anything about jars that you add manually.

If it helps you can look at my SQLRaider project as a reference under the lib-wrapper module. As you can see I brought in quite a few dependencies.
http://code.google.com/p/sqlraider/source/browse/#svn/trunk

Also be sure to do a clean build so that Netbeans picks everything up properly.

-Jeff Johnston



On Tue, Jul 28, 2009 at 8:16 PM, areeda <address-removed ([email]address-removed[/email])> wrote:
Quote:

jeff_johnston_mn wrote:
Quote:
You have to restart Netbeans for it to see the library you added...I get burned by that quite a bit.


Quote:
-Jeff Johnston









It wasn't that easy.
Back to top
View user's profile Send private message
areeda



Joined: 28 Aug 2008
Posts: 88

PostPosted: Wed Jul 29, 2009 3:14 pm    Post subject: Re: Antlr questions Reply with quote

jeff_johnston_mn wrote:

If it helps you can look at my SQLRaider project as a reference under the lib-wrapper module. As you can see I brought in quite a few dependencies.
http://code.google.com/p/sqlraider/source/browse/#svn/trunk


Jeff,

You are very kind. Thanks for all the help. When I get this working, I'll add a page of newb Q&A to the Wiki.

I think I've found my conceptual problem. The OpenOffice extension is a module not a module suite. It looks like the wrapped libraries are a separate module. I seem to remember reading that.

I downloaded the trunk of SQLRaider and will keep reading. I could not build it because I seem to be missing com-sqlraider-db.jar (complete svn checkout of trunk from inside NetBeans, open project, clean-build).

I have many more questions but I'll see how many I can answer on my own.

Joe
Back to top
View user's profile Send private message
areeda



Joined: 28 Aug 2008
Posts: 88

PostPosted: Wed Jul 29, 2009 5:12 pm    Post subject: That was it Reply with quote

As soon as I created a module suite everything fell into place.

Believe it or not the syntax coloring seems to be working. It went through my test cases and colored them just fine.

Thanks again Jeff for the hand holding.

Joe
Back to top
View user's profile Send private message
jeff_johnston_mn



Joined: 12 Apr 2009
Posts: 38

PostPosted: Wed Jul 29, 2009 6:06 pm    Post subject: Antlr questions Reply with quote

No problem! I'm pretty new to all this myself Smile.
Back to top
View user's profile Send private message
areeda



Joined: 28 Aug 2008
Posts: 88

PostPosted: Sat Aug 01, 2009 5:52 pm    Post subject: Some notes Reply with quote

I added a wiki page with some notes on what I learned implementing the stuff in Jeff's tutorial.

Joe
Back to top
View user's profile Send private message
Andreas Stefik
Posted via mailing list.





PostPosted: Sun Aug 02, 2009 6:48 pm    Post subject: Antlr questions Reply with quote

Joe,

Quote:
2.
Back to top
James Reid
Posted via mailing list.





PostPosted: Sun Aug 02, 2009 9:16 pm    Post subject: Antlr questions Reply with quote

What I did was create a simple translator in ANTLR that will auto generate the appropriate syntax. I use the following syntax.


//TOKEN TYPE: keyword
{list of tokens that are keywords}


//TOKEN TYPE: operator
...


Then I copy paste the .tokens file into another text file along with the token defs from my grammar. I then run a parser that reads through the .token stuff and puts it into a hash map. Then when it goes through the token defs it looks up the token map and outputs the correct syntax for the language hierarchy class.


It is pretty simple and quick. When I get more time I want to refine it so that it will auoto generate the language hierarchy class for me using the stringtemplate engine.


When I get back to my computer I'll paste the simple grammar file for the above and an example


James

Sent from my iPod

On Aug 2, 2009, at 2:47 PM, Andreas Stefik <address-removed ([email]address-removed[/email])> wrote:



Quote:
Joe,

Quote:
2. How are people implementing the tokenId list? As I go through the thousands of sample files I'm finding things that weren't in the BNF description I was working from. Not a big deal really but when I add a new token do I have to regenerate the list?

I've been away on vacation (as I move to Illinois to start teaching CS here), but I thought I'd poke in and say how we're doing this. This easiest, and lamest, way to do this is to copy paste the tokens in every time and hand type the right syntax around it. That works, but it stinks if you regenerate your grammar much. What we're doing in the sodbeans project now is basically this:

1. Write a little hash table that adds in many of the tokens you use and what category they would be in. This hash should have a key with the token name and should store a TokenId object of whatever type you created.

2. When your project loads, read in and parse the .tokens file generated by ANTLR. At the same time, update your previous hash with the token id (the numbers), generated by ANTLR and put them into the token ID objects in the hash.

3. When you are adding into your hash from the token file, we do basically this: a) if the token is already in the hash, set its token number, b) if it's not in the token hash, add a new token and give it a default value "separator, whitespace, or whatever you like).

This approach isn't perfect, but now we only have to change our token file when we add new syntax/semantics to our language, not when ANTLR arbitrarily changes the numbers on us as we refactor our grammar (which we do a lot). Hope that helps, an implementation of this can be found in the compiler module of the Sodbeans project. There's a few bugs in our current implementation, but they are trivial and I will probably fix them in a week or two, after my wife and I get settled.

Here's a link to our source:

https://sodbeans.svn.sourceforge.net/svnroot/sodbeans/

The code for that is in the compiler module.

Andreas



Back to top
Andreas Stefik
Posted via mailing list.





PostPosted: Mon Aug 03, 2009 12:52 am    Post subject: Antlr questions Reply with quote

James,

That's the way I did it too for quite a while. It's definitely simple and quick. I went for the more extensive solution eventually because I got bored of copy-pasting the code any time I changed something in the grammar. Either way works though.

Andreas

On Sun, Aug 2, 2009 at 4:15 PM, James Reid <address-removed ([email]address-removed[/email])> wrote:
Quote:
What I did was create a simple translator in ANTLR that will auto generate the appropriate syntax. I use the following syntax.
Back to top
Display posts from previous:   
Post new topic   Reply to topic    NetBeans Forums -> NetBeans Platform Users All times are GMT
Goto page 1, 2  Next
Page 1 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You can attach files in this forum
You can download files in this forum


Powered by phpBB