
| View previous topic :: View next topic |
| Author |
Message |
areeda
Joined: 28 Aug 2008 Posts: 88
|
Posted: Tue Jul 28, 2009 7:22 am Post subject: Antlr questions |
|
|
I am having fun with antlr and antlrWorks but getting down to the nitty-gritty I find I'm missing something.
1. How do you add antlr-runtime.jar to a module project? I've spent a fair amount of time searching with no luck.
2. How are people implementing the tokenId list? As I go through the thousands of sample files I'm finding things that weren't in the BNF description I was working from. Not a big deal really but when I add a new token do I have to regenerate the list?
Thanks,
Joe |
|
| Back to top |
|
 |
areeda
Joined: 28 Aug 2008 Posts: 88
|
Posted: Tue Jul 28, 2009 8:03 am Post subject: Re: Antlr questions |
|
|
| areeda wrote: | | 1. How do you add antlr-runtime.jar to a module project? I've spent a fair amount of time searching with no luck. |
I think I found the answer to this one.
If I put the runtime in the base directory and add the following lines to project.properties
cp.extra=\
${basedir}/antlr-runtime.jar
Is that a good way to do it?
Joe |
|
| Back to top |
|
 |
Jonny Heggheim Posted via mailing list.
|
Posted: Tue Jul 28, 2009 8:13 am Post subject: Antlr questions |
|
|
You should make a new library wrapper module that contains the ANTLR
jar-file, and add the new ANTLR-module as a dependency.
Jonny Heggheim
On Tue, Jul 28, 2009 at 10:03 AM, areeda<address-removed> wrote:
| Quote: |
areeda wrote:
|
| Back to top |
|
 |
jeff_johnston_mn
Joined: 12 Apr 2009 Posts: 38
|
Posted: Tue Jul 28, 2009 11:38 am Post subject: Antlr questions |
|
|
A great resource for questions like this is the developer FAQ wiki page:
http://wiki.netbeans.org/NetBeansDeveloperFAQ
Under there look for "What is a library wrapper module and how do I use it?". That should be exactly what you are looking for.
| Quote: | | How are people implementing the tokenId list? |
|
| Back to top |
|
 |
areeda
Joined: 28 Aug 2008 Posts: 88
|
Posted: Tue Jul 28, 2009 8:38 pm Post subject: Re: Antlr questions |
|
|
| Jonny Heggheim wrote: | You should make a new library wrapper module that contains the ANTLR
jar-file, and add the new ANTLR-module as a dependency.
| Sounds easy, but I'm missing something basic. I've spent the whole day reading about library wrappers and dependencies.
I see how to create a wrapper (I think) by using the wizard to add a "Java SE Library Descriptor" which creates an xml file. I put a copy of the antlr-runtime.jar in ${basedir}/ext.
Is that correct?
Now as to how to add a dependency on that, I'm completely lost. If go to the library properties or right click on Libraries and try to add a Module Dependency, my only choices are NetBeans modules my newly created wrapper is not there.
Also in the log file I have errors like:
Cannot create library: antlr-runtime of unknown type: j2se
Cannot create library: jaxb of unknown type: j2se
Cannot create library: javac-api of unknown type: j2se
and a few others.
I appreciate your help. I know I'm missing something very basic.
Joe |
|
| Back to top |
|
 |
jeff_johnston_mn
Joined: 12 Apr 2009 Posts: 38
|
Posted: Tue Jul 28, 2009 8:55 pm Post subject: Antlr questions |
|
|
You have to restart Netbeans for it to see the library you added...I get burned by that quite a bit.
-Jeff Johnston
On Tue, Jul 28, 2009 at 3:39 PM, areeda <address-removed ([email]address-removed[/email])> wrote:
| Quote: |
Jonny Heggheim wrote:
| Quote: | You should make a new library wrapper module that contains the ANTLR
|
| Quote: | jar-file, and add the new ANTLR-module as a dependency.
|
Sounds easy, but I'm missing something basic. |
|
| Back to top |
|
 |
areeda
Joined: 28 Aug 2008 Posts: 88
|
Posted: Wed Jul 29, 2009 1:15 am Post subject: Re: Antlr questions |
|
|
| jeff_johnston_mn wrote: | You have to restart Netbeans for it to see the library you added...I get burned by that quite a bit.
-Jeff Johnston
|
It wasn't that easy. I do appreciate your quick answers and help Jeff.
I put about 10 hrs in today reading about modules, wrappers, and libraries. I am learning things and am making some progress.
I get an exception NoClassDefFoundError for AntlrCharStream, which used to say because of antlr.runtime.CharStream, now it doesn't say "because" any more. But now I'm without a clue, a cntrl-B in the lexer where it's constructed takes me to the right constructor, project builds without error, everything is in the same project. Both arguments are valid at the breakpoint.
I'm taking a break. I know this is a simple thing, I just have to find it. I guess I'll reread the ClassLoader stuff again tomorrow morning.
Joe |
|
| Back to top |
|
 |
jeff_johnston_mn
Joined: 12 Apr 2009 Posts: 38
|
Posted: Wed Jul 29, 2009 11:34 am Post subject: Antlr questions |
|
|
Once you bring in your library you do need to make those jar packages public. So on your library wrapper module go to the project properties, then API versioning, and then select the packages that you want to expose. I usually just select everything.
I should also amend that you probably only need to restart Netbeans if you add more than one jar to your library wrapper module. That makes sense to because Netbeans does not know anything about jars that you add manually.
If it helps you can look at my SQLRaider project as a reference under the lib-wrapper module. As you can see I brought in quite a few dependencies.
http://code.google.com/p/sqlraider/source/browse/#svn/trunk
Also be sure to do a clean build so that Netbeans picks everything up properly.
-Jeff Johnston
On Tue, Jul 28, 2009 at 8:16 PM, areeda <address-removed ([email]address-removed[/email])> wrote:
| Quote: |
jeff_johnston_mn wrote:
| Quote: | You have to restart Netbeans for it to see the library you added...I get burned by that quite a bit.
|
It wasn't that easy. |
|
| Back to top |
|
 |
areeda
Joined: 28 Aug 2008 Posts: 88
|
Posted: Wed Jul 29, 2009 3:14 pm Post subject: Re: Antlr questions |
|
|
Jeff,
You are very kind. Thanks for all the help. When I get this working, I'll add a page of newb Q&A to the Wiki.
I think I've found my conceptual problem. The OpenOffice extension is a module not a module suite. It looks like the wrapped libraries are a separate module. I seem to remember reading that.
I downloaded the trunk of SQLRaider and will keep reading. I could not build it because I seem to be missing com-sqlraider-db.jar (complete svn checkout of trunk from inside NetBeans, open project, clean-build).
I have many more questions but I'll see how many I can answer on my own.
Joe |
|
| Back to top |
|
 |
areeda
Joined: 28 Aug 2008 Posts: 88
|
Posted: Wed Jul 29, 2009 5:12 pm Post subject: That was it |
|
|
As soon as I created a module suite everything fell into place.
Believe it or not the syntax coloring seems to be working. It went through my test cases and colored them just fine.
Thanks again Jeff for the hand holding.
Joe |
|
| Back to top |
|
 |
jeff_johnston_mn
Joined: 12 Apr 2009 Posts: 38
|
Posted: Wed Jul 29, 2009 6:06 pm Post subject: Antlr questions |
|
|
No problem! I'm pretty new to all this myself . |
|
| Back to top |
|
 |
areeda
Joined: 28 Aug 2008 Posts: 88
|
Posted: Sat Aug 01, 2009 5:52 pm Post subject: Some notes |
|
|
I added a wiki page with some notes on what I learned implementing the stuff in Jeff's tutorial.
Joe |
|
| Back to top |
|
 |
Andreas Stefik Posted via mailing list.
|
Posted: Sun Aug 02, 2009 6:48 pm Post subject: Antlr questions |
|
|
Joe,
|
| Back to top |
|
 |
James Reid Posted via mailing list.
|
Posted: Sun Aug 02, 2009 9:16 pm Post subject: Antlr questions |
|
|
What I did was create a simple translator in ANTLR that will auto generate the appropriate syntax. I use the following syntax.
//TOKEN TYPE: keyword
{list of tokens that are keywords}
//TOKEN TYPE: operator
...
Then I copy paste the .tokens file into another text file along with the token defs from my grammar. I then run a parser that reads through the .token stuff and puts it into a hash map. Then when it goes through the token defs it looks up the token map and outputs the correct syntax for the language hierarchy class.
It is pretty simple and quick. When I get more time I want to refine it so that it will auoto generate the language hierarchy class for me using the stringtemplate engine.
When I get back to my computer I'll paste the simple grammar file for the above and an example
James
Sent from my iPod
On Aug 2, 2009, at 2:47 PM, Andreas Stefik <address-removed ([email]address-removed[/email])> wrote:
| Quote: | Joe,
| Quote: | | 2. How are people implementing the tokenId list? As I go through the thousands of sample files I'm finding things that weren't in the BNF description I was working from. Not a big deal really but when I add a new token do I have to regenerate the list? |
I've been away on vacation (as I move to Illinois to start teaching CS here), but I thought I'd poke in and say how we're doing this. This easiest, and lamest, way to do this is to copy paste the tokens in every time and hand type the right syntax around it. That works, but it stinks if you regenerate your grammar much. What we're doing in the sodbeans project now is basically this:
1. Write a little hash table that adds in many of the tokens you use and what category they would be in. This hash should have a key with the token name and should store a TokenId object of whatever type you created.
2. When your project loads, read in and parse the .tokens file generated by ANTLR. At the same time, update your previous hash with the token id (the numbers), generated by ANTLR and put them into the token ID objects in the hash.
3. When you are adding into your hash from the token file, we do basically this: a) if the token is already in the hash, set its token number, b) if it's not in the token hash, add a new token and give it a default value "separator, whitespace, or whatever you like).
This approach isn't perfect, but now we only have to change our token file when we add new syntax/semantics to our language, not when ANTLR arbitrarily changes the numbers on us as we refactor our grammar (which we do a lot). Hope that helps, an implementation of this can be found in the compiler module of the Sodbeans project. There's a few bugs in our current implementation, but they are trivial and I will probably fix them in a week or two, after my wife and I get settled.
Here's a link to our source:
https://sodbeans.svn.sourceforge.net/svnroot/sodbeans/
The code for that is in the compiler module.
Andreas
|
|
|
| Back to top |
|
 |
Andreas Stefik Posted via mailing list.
|
Posted: Mon Aug 03, 2009 12:52 am Post subject: Antlr questions |
|
|
James,
That's the way I did it too for quite a while. It's definitely simple and quick. I went for the more extensive solution eventually because I got bored of copy-pasting the code any time I changed something in the grammar. Either way works though.
Andreas
On Sun, Aug 2, 2009 at 4:15 PM, James Reid <address-removed ([email]address-removed[/email])> wrote:
| Quote: | | What I did was create a simple translator in ANTLR that will auto generate the appropriate syntax. I use the following syntax. |
|
| Back to top |
|
 |
|
|
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum You can attach files in this forum You can download files in this forum
|
|
|
|
|
| |
| | | | |