NetBeans Forums

 FAQFAQ   SearchSearch   MemberlistMemberlist   RegisterRegister   ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 
  

[platform-dev] Re: Netbeans Friend package & OSGi guys :)
Goto page 1, 2  Next
 
Post new topic   Reply to topic    NetBeans Forums -> NetBeans Platform Users
View previous topic :: View next topic  
Author Message
Tim Boudreau
Posted via mailing list.





PostPosted: Fri Jan 29, 2010 11:36 pm    Post subject: [platform-dev] Re: Netbeans Friend package & OSGi guys :) Reply with quote

Daoud AbdelMonem Faleh wrote:
Quote:
Hi folks,

Today I was thinking of uses cases of Netbeans "Friend package" feature
and concluded that one valid uses case is when a set of plugins
collaborate using an internal API to expose some feature to out world
throw a public API. The internal API could be exposed to the plugin set
throw "Friend package".

Generally we use friend packages as a way of *developing* an API - i.e.
it's a recognition that sometimes, when you are developing an API, not
all of the potential use-cases are known. Effective APIs are often not
born perfect from day 1. Rather than release something that is
inadequate or worse, incorrect, friend packages allow the API to be
developed in a modular way, without committing to full backward
compatibility until the API is solid. While friend packages can be used
to expose an API just to friend modules forever, that is not its primary
purpose - rather it provides a way to develop an API that has real
clients, and respond to the requirements of those clients by improving
the API, before making it an official API with all of the commitments
that implies.

I think that would be the case to make to the OSGi folks, rather than
presenting it as a standard way to limit the visibility of an API in
general (although surely some people will use it that way...but some
people also use exceptions as a way of providing an alternate return
value from a method...).

-Tim
Quote:
Then I thought that OSGi leaks such feature and that OSGi spec should
have addressed this uses case so I twitted it here [1]
Some of OSGi experts argued [2,3] that such feature -while implemented
by Eclipse Equinox- is a bad practice in OSGi/Modularity.
I argued that if Equinox implemented it that it solve some uses case and
that's an argument to seek standardization.
As there have been great architectural discussion here lately, I turn
back to my modularity roots seeking more informations and pro/cons of
using such kind of coupling.

With Warm regards,
Daoud AbdelMonem Faleh.

[1] http://twitter.com/AntholoJ/status/8362980777
[2] http://twitter.com/njbartlett/status/8366817308
[3] http://twitter.com/kartben/status/8363080572
Back to top
Richard S. Hall
Posted via mailing list.





PostPosted: Sat Jan 30, 2010 12:04 am    Post subject: [platform-dev] Re: Netbeans Friend package & OSGi guys :) Reply with quote

On 1/29/10 6:35 PM, Tim Boudreau wrote:
Quote:
Daoud AbdelMonem Faleh wrote:
Quote:
Hi folks,

Today I was thinking of uses cases of Netbeans "Friend package" feature
and concluded that one valid uses case is when a set of plugins
collaborate using an internal API to expose some feature to out world
throw a public API. The internal API could be exposed to the plugin set
throw "Friend package".
Generally we use friend packages as a way of *developing* an API -
i.e. it's a recognition that sometimes, when you are developing an
API, not all of the potential use-cases are known. Effective APIs are
often not born perfect from day 1. Rather than release something that
is inadequate or worse, incorrect, friend packages allow the API to be
developed in a modular way, without committing to full backward
compatibility until the API is solid. While friend packages can be
used to expose an API just to friend modules forever, that is not its
primary purpose - rather it provides a way to develop an API that has
real clients, and respond to the requirements of those clients by
improving the API, before making it an official API with all of the
commitments that implies.

I think that would be the case to make to the OSGi folks, rather than
presenting it as a standard way to limit the visibility of an API in
general (although surely some people will use it that way...but some
people also use exceptions as a way of providing an alternate return
value from a method...).

Still, typical "friend" approaches require explicit listing of who is or
is not a friend. These lists are brittle and sometimes a friend in one
scenario is not a friend in another, so this brittleness limits re-use.

On the otherhand, if your design is to have a friend mechanism without
explicit "friend" designation, then this is possible to mimic with
mandatory attributes in OSGi. Although still less than optimal, it makes
it clear to the person doing the importing that they are choosing to
depend on an API that might change.

You can also use security in OSGi to limit visibility, but no one runs
with security enabled, so that is not an option for most people. :-)

Another approach to hiding API from some while exposing to others is
introducing a notion of composite modules. Think of a module that
logically contains other module and only exposes from them what it
wants. This is a more modular way of achieving a shared, but private
API. It is more flexible, since the friendship isn't baked into the
module doing the exporting, but rather into the subsystem/composite
declaration. Thus, the same modules can be used in different composites
exposing different parts (or none) of the API as needed.

OSGi is working on something similar to this approach for the R4.3 release.

-> richard

Quote:

-Tim
Quote:
Then I thought that OSGi leaks such feature and that OSGi spec should
have addressed this uses case so I twitted it here [1]
Some of OSGi experts argued [2,3] that such feature -while implemented
by Eclipse Equinox- is a bad practice in OSGi/Modularity.
I argued that if Equinox implemented it that it solve some uses case and
that's an argument to seek standardization.
As there have been great architectural discussion here lately, I turn
back to my modularity roots seeking more informations and pro/cons of
using such kind of coupling.

With Warm regards,
Daoud AbdelMonem Faleh.

[1] http://twitter.com/AntholoJ/status/8362980777
[2] http://twitter.com/njbartlett/status/8366817308
[3] http://twitter.com/kartben/status/8363080572
Back to top
Tim Boudreau
Posted via mailing list.





PostPosted: Sat Jan 30, 2010 6:27 am    Post subject: [platform-dev] Re: Netbeans Friend package & OSGi guys :) Reply with quote

Richard S. Hall wrote:
Quote:
On 1/29/10 6:35 PM, Tim Boudreau wrote:
Quote:
Daoud AbdelMonem Faleh wrote:
Quote:
Hi folks,

Today I was thinking of uses cases of Netbeans "Friend package" feature
and concluded that one valid uses case is when a set of plugins
collaborate using an internal API to expose some feature to out world
throw a public API. The internal API could be exposed to the plugin set
throw "Friend package".
Generally we use friend packages as a way of *developing* an API -
i.e. it's a recognition that sometimes, when you are developing an
API, not all of the potential use-cases are known. Effective APIs
are often not born perfect from day 1. Rather than release something
that is inadequate or worse, incorrect, friend packages allow the API
to be developed in a modular way, without committing to full backward
compatibility until the API is solid. While friend packages can be
used to expose an API just to friend modules forever, that is not its
primary purpose - rather it provides a way to develop an API that has
real clients, and respond to the requirements of those clients by
improving the API, before making it an official API with all of the
commitments that implies.

I think that would be the case to make to the OSGi folks, rather than
presenting it as a standard way to limit the visibility of an API in
general (although surely some people will use it that way...but some
people also use exceptions as a way of providing an alternate return
value from a method...).

Still, typical "friend" approaches require explicit listing of who is
or is not a friend. These lists are brittle
That's why it's really an indication of "under-developmentness" of an
API, not something to use without expecting incompatible changes.
Quote:
and sometimes a friend in one scenario is not a friend in another,
If that's the case, I suspect you really have two APIs and are mixing
them in one module.
Quote:
so this brittleness limits re-use.
Well, that's the point of friend-ness - to limit reuse of something that
is not ready for reuse, but needed by some third parties. When there is
a compatibility break, the person breaking compatibility knows whom to
communicate with; and third parties are keenly aware that they are
depending on something that might not be dependable. I am happy to add
any module someone asks me to as a friend of a module I am working on -
that way, I know the code is being used, and there is an open channel
for communication about changes and change requests, and no party gets
unpleasantly surprised.
Quote:
On the otherhand, if your design is to have a friend mechanism without
explicit "friend" designation, then this is possible to mimic with
mandatory attributes in OSGi. Although still less than optimal, it
makes it clear to the person doing the importing that they are
choosing to depend on an API that might change.
Similar to provides-requires tokens in NB?
Quote:
You can also use security in OSGi to limit visibility, but no one runs
with security enabled, so that is not an option for most people. :-)

Another approach to hiding API from some while exposing to others is
introducing a notion of composite modules. Think of a module that
logically contains other module and only exposes from them what it
wants. This is a more modular way of achieving a shared, but private
API. It is more flexible, since the friendship isn't baked into the
module doing the exporting, but rather into the subsystem/composite
declaration. Thus, the same modules can be used in different
composites exposing different parts (or none) of the API as needed.

OSGi is working on something similar to this approach for the R4.3
release.
I seem to remember someone in OSGi-land talking about this with me at
EclipseCon a couple of years ago - I forget whom. He was very
passionate about this subject, and I couldn't really figure out why.

My general feeling about the whole composite module idea was one of
unease. One of the benefits of 1:1 (or occasionally 1:>1 if bundling a
library) mapping between modules and JAR files is that a module is
always a compilation-unit - at least with respect to its own code
linking to itself and any libraries it contains. What does breaking
that contract get us, and is it worth the possibility of a module that
cannot link with itself?

Now, you can more-or-less do this with libraries already in NetBeans -
that is, my module can bundle a version of, say, Apache Commons; your
module can bundle a different version. As long as the two modules do
not have a relationship, your module will use yours and my module will
use mine - and presumably neither one exposes it as API for other
modules to call. Or we can agree on a version that will work for both
of us, create a wrapper module and both depend on that. But from what
you said - "since the friendship isn't baked into the module doing the
exporting" - I get the impression the multiple-library-versions scenario
isn't what you are talking about.

Let me see if I understand what you're suggesting. If module A
logically contains module X (exporting org.foo.common and org.foo.bar,
hiding org.foo.impl) and module B logically contains X (exporting
org.foo.common and org.foo.baz, hiding org.foo.impl),
- Are there two copies of the classes of module X?
- Will a module depending on A and B see the same Class objects for
org.foo.common?
- If A exports org.foo.common as a friend API and B exports it as a
stable API, and D depends on A and B, what is D's contract for the
stability of org.foo.common?
- Can anyone create their own composite module export org.foo.impl if
they want to?
- Can it be guaranteed that anything that links with classes in
org.foo.common from X will also link with org.foo.common from Y?

I can see why doing this is an interesting idea in the abstract. What I
don't see is what problem it solves, which cannot be solved simply by
dividing the code into more modules and expressing the dependencies
plainly. What can you do with composite modules, aside from create
unenforceable contracts between components, that you can't do simply by
splitting friend-code into one module, API code into another, and
implementation code either in one or the other, both or a separate third
module that injects the implementation? It seems like this would vastly
complicate any sort of dependency analysis in an application - for what
benefit?

(If I am understanding you correctly) an ability to export another
module's implementation classes definitely adds flexibility - but it
rather defeats the point of modularity. If you want that much
flexibility, you could just put everything on the boot class path and
pray - which is what that amounts to.

What am I misunderstanding here?

-Tim


Quote:

-> richard

Quote:

-Tim
Quote:
Then I thought that OSGi leaks such feature and that OSGi spec should
have addressed this uses case so I twitted it here [1]
Some of OSGi experts argued [2,3] that such feature -while implemented
by Eclipse Equinox- is a bad practice in OSGi/Modularity.
I argued that if Equinox implemented it that it solve some uses case
and
that's an argument to seek standardization.
As there have been great architectural discussion here lately, I turn
back to my modularity roots seeking more informations and pro/cons of
using such kind of coupling.

With Warm regards,
Daoud AbdelMonem Faleh.

[1] http://twitter.com/AntholoJ/status/8362980777
[2] http://twitter.com/njbartlett/status/8366817308
[3] http://twitter.com/kartben/status/8363080572
Back to top
Jesse Glick
Posted via mailing list.





PostPosted: Sat Jan 30, 2010 3:39 pm    Post subject: [platform-dev] Re: Netbeans Friend package & OSGi guys :) Reply with quote

Richard S. Hall wrote:
Quote:
typical "friend" approaches require explicit listing of who is or is not a friend.

That is the case in the NB module system. A module simply lists a few friends by code-name-base (akin to OSGi's Bundle-SymbolicName).

Quote:
These lists are brittle and sometimes a friend in one
scenario is not a friend in another, so this brittleness limits re-use.

Can you expand on this? I have never come across any such scenario.

Quote:
if your design is to have a friend mechanism without
explicit "friend" designation, then this is possible to mimic with
mandatory attributes in OSGi.

When you have an explicit list of friend modules, you are either working on those other modules personally, or know who is and why (because they had to ask you to add
them as friends). So, when you uncover a design flaw in your API and decide to rectify it by refactoring some part of the API, you can notify all of the other developers
(if any) of the planned change and verify that it is acceptable to them; and update the friends you develop yourself (perhaps as part of the same SCM transaction as the
API change). Similarly, if you find some odd public method in your API that you think no one is actually using, you can just look at the friends list, open sources for
all these in your IDE, and run a Find Usages query. There is a clear and bounded scope for the API, just as with package-private access.

If all you have done is mark your API with a mandatory attribute, none of this is possible. Your API could still be in use from completely unknown third parties. Maybe
they have been "warned" of the API's instability by the attribute, but you will still be nervous about what the effects of changes might be.

(A friend list as symbolic name is of course susceptible to "spoofing" - someone could make a module with one of the listed names that is unrelated to the one you know
about, and run it against your API so long as the real McCoy is not also needed. But this seems like deliberate malice. Generally if there is a security threat from
unauthorized API usage then mere Java class loader visibility is not going to suffice anyway.)

Quote:
Another approach to hiding API from some while exposing to others is
introducing a notion of composite modules. Think of a module that
logically contains other module and only exposes from them what it
wants. This is a more modular way of achieving a shared, but private
API.

Again, this fails to help with the use case for friend APIs, which is to ensure that an API under development is not used without your knowledge:

1. Someone could just extract the inner module and start using its API. (This might or might not count as "malice" depending on how obvious an operation extraction is.)

2. You cannot offer your API to selected "beta testers" who are building modules that are not physically contained in the application you build and distribute, but are
rather add-ons available from third-party sites.
Back to top
Richard S. Hall
Posted via mailing list.





PostPosted: Sat Jan 30, 2010 7:49 pm    Post subject: [platform-dev] Re: Netbeans Friend package & OSGi guys :) Reply with quote

On 1/30/10 1:26, Tim Boudreau wrote:
Quote:
Richard S. Hall wrote:
Quote:
On 1/29/10 6:35 PM, Tim Boudreau wrote:
Quote:
Daoud AbdelMonem Faleh wrote:
Quote:
Hi folks,

Today I was thinking of uses cases of Netbeans "Friend package"
feature
and concluded that one valid uses case is when a set of plugins
collaborate using an internal API to expose some feature to out world
throw a public API. The internal API could be exposed to the plugin
set
throw "Friend package".
Generally we use friend packages as a way of *developing* an API -
i.e. it's a recognition that sometimes, when you are developing an
API, not all of the potential use-cases are known. Effective APIs
are often not born perfect from day 1. Rather than release
something that is inadequate or worse, incorrect, friend packages
allow the API to be developed in a modular way, without committing
to full backward compatibility until the API is solid. While friend
packages can be used to expose an API just to friend modules
forever, that is not its primary purpose - rather it provides a way
to develop an API that has real clients, and respond to the
requirements of those clients by improving the API, before making it
an official API with all of the commitments that implies.

I think that would be the case to make to the OSGi folks, rather
than presenting it as a standard way to limit the visibility of an
API in general (although surely some people will use it that
way...but some people also use exceptions as a way of providing an
alternate return value from a method...).

Still, typical "friend" approaches require explicit listing of who is
or is not a friend. These lists are brittle
That's why it's really an indication of "under-developmentness" of an
API, not something to use without expecting incompatible changes.

That is certainly one way to cast it.

Quote:
Quote:
and sometimes a friend in one scenario is not a friend in another,
If that's the case, I suspect you really have two APIs and are mixing
them in one module.

I don't necessarily agree with this. The issue is the same for providing
a service that I want to share among some modules, but not others.

Quote:
Quote:
so this brittleness limits re-use.
Well, that's the point of friend-ness - to limit reuse of something
that is not ready for reuse, but needed by some third parties. When
there is a compatibility break, the person breaking compatibility
knows whom to communicate with; and third parties are keenly aware
that they are depending on something that might not be dependable. I
am happy to add any module someone asks me to as a friend of a module
I am working on - that way, I know the code is being used, and there
is an open channel for communication about changes and change
requests, and no party gets unpleasantly surprised.

I understand that this is how you choose to use the "friend" mechanism,
but in the end if it is introduced into a module system, then it becomes
a general mechanism that cannot be restricted to such a "evolving API"
use case.

Quote:
Quote:
On the otherhand, if your design is to have a friend mechanism
without explicit "friend" designation, then this is possible to mimic
with mandatory attributes in OSGi. Although still less than optimal,
it makes it clear to the person doing the importing that they are
choosing to depend on an API that might change.
Similar to provides-requires tokens in NB?

Sort of, but less general and specifically tied to an exported packaged.
It forces the importer to supply an extra token to import the exported
package, where normally it only needs to match the package name.

Quote:
Quote:
You can also use security in OSGi to limit visibility, but no one
runs with security enabled, so that is not an option for most people.
:-)

Another approach to hiding API from some while exposing to others is
introducing a notion of composite modules. Think of a module that
logically contains other module and only exposes from them what it
wants. This is a more modular way of achieving a shared, but private
API. It is more flexible, since the friendship isn't baked into the
module doing the exporting, but rather into the subsystem/composite
declaration. Thus, the same modules can be used in different
composites exposing different parts (or none) of the API as needed.

OSGi is working on something similar to this approach for the R4.3
release.
I seem to remember someone in OSGi-land talking about this with me at
EclipseCon a couple of years ago - I forget whom. He was very
passionate about this subject, and I couldn't really figure out why.

I was there last year and gave a talk with Tom Watson (IBM) on this
subject. I don't know if it is something to be passionate about, but it
is an worthwhile idea. It is similar to hierarchical component models,
but extending them to the module layer.

Quote:
My general feeling about the whole composite module idea was one of
unease. One of the benefits of 1:1 (or occasionally 1:>1 if bundling
a library) mapping between modules and JAR files is that a module is
always a compilation-unit - at least with respect to its own code
linking to itself and any libraries it contains. What does breaking
that contract get us, and is it worth the possibility of a module that
cannot link with itself?

Now, you can more-or-less do this with libraries already in NetBeans -
that is, my module can bundle a version of, say, Apache Commons; your
module can bundle a different version. As long as the two modules do
not have a relationship, your module will use yours and my module will
use mine - and presumably neither one exposes it as API for other
modules to call. Or we can agree on a version that will work for both
of us, create a wrapper module and both depend on that. But from what
you said - "since the friendship isn't baked into the module doing the
exporting" - I get the impression the multiple-library-versions
scenario isn't what you are talking about.

In OSGi this would isolating libraries and ultimately the services
provided by modules within a composite. It is not a form of true
isolation, but an approximation of it.

Quote:
Let me see if I understand what you're suggesting. If module A
logically contains module X (exporting org.foo.common and org.foo.bar,
hiding org.foo.impl) and module B logically contains X (exporting
org.foo.common and org.foo.baz, hiding org.foo.impl),
- Are there two copies of the classes of module X?

Yes.

Quote:
- Will a module depending on A and B see the same Class objects for
org.foo.common?

If the dependencies on A and B involve a "uses" constraint on
org.foo.common, then the dependency would not be allowed, since it would
be a class space violation. If not, it would end up getting
org.foo.common from one or the other, but not both.

Quote:
- If A exports org.foo.common as a friend API and B exports it as a
stable API, and D depends on A and B, what is D's contract for the
stability of org.foo.common?

There is not such concept as a friend in this approach. It is just an
isolation model for modules.

Quote:
- Can anyone create their own composite module export org.foo.impl if
they want to?

Yes.

Quote:
- Can it be guaranteed that anything that links with classes in
org.foo.common from X will also link with org.foo.common from Y?

You cannot see a package from two different sources at the same time.

Quote:

I can see why doing this is an interesting idea in the abstract. What
I don't see is what problem it solves, which cannot be solved simply
by dividing the code into more modules and expressing the dependencies
plainly. What can you do with composite modules, aside from create
unenforceable contracts between components, that you can't do simply
by splitting friend-code into one module, API code into another, and
implementation code either in one or the other, both or a separate
third module that injects the implementation? It seems like this
would vastly complicate any sort of dependency analysis in an
application - for what benefit?

You look at the issues from too simplistic of a perspective. First, as I
mention these boundaries extend to services as well as encompass the
module layer. Second, even at the module layer alone, you assume that
you are in control of everything. What if I want to reuse an existing
OSGi bundle which I want to share amongst some bundles, but not with
other bundles? Large applications are often composed of subsystems where
the subsystems are composed from reusable components and desire
isolation from each other. Again, think hierarchical component model.

However, I agree that there is an extra level of complexity to this.
Especially tacking it onto an existing model, like OSGi. If these ideas
are thought of from the beginning, though, they can often be done in a
reasonably intuitive way.

Quote:
(If I am understanding you correctly) an ability to export another
module's implementation classes definitely adds flexibility - but it
rather defeats the point of modularity. If you want that much
flexibility, you could just put everything on the boot class path and
pray - which is what that amounts to.

Again, what is public API at one level is implementation API at another
level.

Quote:
What am I misunderstanding here?

Not sure. :-)

-> richard

Quote:

-Tim


Quote:

-> richard

Quote:

-Tim
Quote:
Then I thought that OSGi leaks such feature and that OSGi spec should
have addressed this uses case so I twitted it here [1]
Some of OSGi experts argued [2,3] that such feature -while implemented
by Eclipse Equinox- is a bad practice in OSGi/Modularity.
I argued that if Equinox implemented it that it solve some uses
case and
that's an argument to seek standardization.
As there have been great architectural discussion here lately, I turn
back to my modularity roots seeking more informations and pro/cons of
using such kind of coupling.

With Warm regards,
Daoud AbdelMonem Faleh.

[1] http://twitter.com/AntholoJ/status/8362980777
[2] http://twitter.com/njbartlett/status/8366817308
[3] http://twitter.com/kartben/status/8363080572

Back to top
Richard S. Hall
Posted via mailing list.





PostPosted: Sat Jan 30, 2010 8:02 pm    Post subject: [platform-dev] Re: Netbeans Friend package & OSGi guys :) Reply with quote

On 1/30/10 10:38, Jesse Glick wrote:
Quote:
Richard S. Hall wrote:
Quote:
typical "friend" approaches require explicit listing of who is or is
not a friend.

That is the case in the NB module system. A module simply lists a few
friends by code-name-base (akin to OSGi's Bundle-SymbolicName).

Quote:
These lists are brittle and sometimes a friend in one scenario is not
a friend in another, so this brittleness limits re-use.

Can you expand on this? I have never come across any such scenario.

For example, reuse of standard modules which provide functionality that
maintains state either explicitly or in static variables. In such a
case, you may not want other people messing with your state.

Quote:
Quote:
if your design is to have a friend mechanism without explicit
"friend" designation, then this is possible to mimic with mandatory
attributes in OSGi.

When you have an explicit list of friend modules, you are either
working on those other modules personally, or know who is and why
(because they had to ask you to add them as friends). So, when you
uncover a design flaw in your API and decide to rectify it by
refactoring some part of the API, you can notify all of the other
developers (if any) of the planned change and verify that it is
acceptable to them; and update the friends you develop yourself
(perhaps as part of the same SCM transaction as the API change).
Similarly, if you find some odd public method in your API that you
think no one is actually using, you can just look at the friends list,
open sources for all these in your IDE, and run a Find Usages query.
There is a clear and bounded scope for the API, just as with
package-private access.

If all you have done is mark your API with a mandatory attribute, none
of this is possible. Your API could still be in use from completely
unknown third parties. Maybe they have been "warned" of the API's
instability by the attribute, but you will still be nervous about what
the effects of changes might be.

Agreed, I said it is less than optimal, but the whole notion of friends
is less than optimal. Of course, if the goal is purely to share API
among modules within the same project where there is one single entity
in control of all modules, then the friend concept can be sufficient.

Quote:
(A friend list as symbolic name is of course susceptible to "spoofing"
- someone could make a module with one of the listed names that is
unrelated to the one you know about, and run it against your API so
long as the real McCoy is not also needed. But this seems like
deliberate malice. Generally if there is a security threat from
unauthorized API usage then mere Java class loader visibility is not
going to suffice anyway.)

Quote:
Another approach to hiding API from some while exposing to others is
introducing a notion of composite modules. Think of a module that
logically contains other module and only exposes from them what it
wants. This is a more modular way of achieving a shared, but private
API.

Again, this fails to help with the use case for friend APIs, which is
to ensure that an API under development is not used without your
knowledge:

1. Someone could just extract the inner module and start using its
API. (This might or might not count as "malice" depending on how
obvious an operation extraction is.)

Yes.

Quote:

2. You cannot offer your API to selected "beta testers" who are
building modules that are not physically contained in the application
you build and distribute, but are rather add-ons available from
third-party sites.

Likewise, the friend mechanism breaks down when it comes to wanting to
hide details from reusable modules.

Ultimately, we are talking about different use cases that overlap to
some degree.

-> richard
Back to top
Tim Boudreau
Posted via mailing list.





PostPosted: Sat Jan 30, 2010 11:09 pm    Post subject: [platform-dev] Re: Netbeans Friend package & OSGi guys :) Reply with quote

Richard S. Hall wrote:
Quote:
On 1/30/10 1:26, Tim Boudreau wrote:
Quote:
Richard S. Hall wrote:
Quote:
On 1/29/10 6:35 PM, Tim Boudreau wrote:
Quote:
Daoud AbdelMonem Faleh wrote:
Quote:
Hi folks,

Today I was thinking of uses cases of Netbeans "Friend package"
feature
and concluded that one valid uses case is when a set of plugins
collaborate using an internal API to expose some feature to out world
throw a public API. The internal API could be exposed to the
plugin set
throw "Friend package".
Generally we use friend packages as a way of *developing* an API -
i.e. it's a recognition that sometimes, when you are developing an
API, not all of the potential use-cases are known. Effective APIs
are often not born perfect from day 1. Rather than release
something that is inadequate or worse, incorrect, friend packages
allow the API to be developed in a modular way, without committing
to full backward compatibility until the API is solid. While
friend packages can be used to expose an API just to friend modules
forever, that is not its primary purpose - rather it provides a way
to develop an API that has real clients, and respond to the
requirements of those clients by improving the API, before making
it an official API with all of the commitments that implies.

I think that would be the case to make to the OSGi folks, rather
than presenting it as a standard way to limit the visibility of an
API in general (although surely some people will use it that
way...but some people also use exceptions as a way of providing an
alternate return value from a method...).

Still, typical "friend" approaches require explicit listing of who
is or is not a friend. These lists are brittle
That's why it's really an indication of "under-developmentness" of an
API, not something to use without expecting incompatible changes.
That is certainly one way to cast it.

Quote:
Quote:
and sometimes a friend in one scenario is not a friend in another,
If that's the case, I suspect you really have two APIs and are mixing
them in one module.

I don't necessarily agree with this. The issue is the same for
providing a service that I want to share among some modules, but not
others.
I would read that as the "friend module" solution can be used to solve
both problems.

I would like to see an example where there is truly no option except to
have one module contain both a semi-private API and a public API, where
those two things provably could not be split into two modules, or where
it would be too labor intensive to do so.
Quote:

Quote:
Quote:
so this brittleness limits re-use.
Well, that's the point of friend-ness - to limit reuse of something
that is not ready for reuse, but needed by some third parties. When
there is a compatibility break, the person breaking compatibility
knows whom to communicate with; and third parties are keenly aware
that they are depending on something that might not be dependable. I
am happy to add any module someone asks me to as a friend of a module
I am working on - that way, I know the code is being used, and there
is an open channel for communication about changes and change
requests, and no party gets unpleasantly surprised.

I understand that this is how you choose to use the "friend"
mechanism, but in the end if it is introduced into a module system,
then it becomes a general mechanism that cannot be restricted to such
a "evolving API" use case.
True, that is a social convention that relies on human behavior. But
much of software development relies on human communication -
requirements coming in, notification of changes, etc.

[snip]
Quote:

Quote:
Quote:

Another approach to hiding API from some while exposing to others is
introducing a notion of composite modules. Think of a module that
logically contains other module and only exposes from them what it
wants. This is a more modular way of achieving a shared, but private
API. It is more flexible, since the friendship isn't baked into the
module doing the exporting, but rather into the subsystem/composite
declaration. Thus, the same modules can be used in different
composites exposing different parts (or none) of the API as needed.

OSGi is working on something similar to this approach for the R4.3
release.
I seem to remember someone in OSGi-land talking about this with me at
EclipseCon a couple of years ago - I forget whom. He was very
passionate about this subject, and I couldn't really figure out why.

I was there last year and gave a talk with Tom Watson (IBM) on this
subject. I don't know if it is something to be passionate about, but
it is an worthwhile idea. It is similar to hierarchical component
models, but extending them to the module layer.
Ah, the catnip of developers - as soon as you have
interface Foo {
void foo();
}
someone will simply be unable to resist creating
class MetaFoo {
Foo[] foos;
public void foo() {
for (Foo foo: foos) { foo.foo(); }
}
}
It's just so cool to do that. Smile
Quote:
Quote:
I can see why doing this is an interesting idea in the abstract.
What I don't see is what problem it solves, which cannot be solved
simply by dividing the code into more modules and expressing the
dependencies plainly. What can you do with composite modules, aside
from create unenforceable contracts between components, that you
can't do simply by splitting friend-code into one module, API code
into another, and implementation code either in one or the other,
both or a separate third module that injects the implementation? It
seems like this would vastly complicate any sort of dependency
analysis in an application - for what benefit?

You look at the issues from too simplistic of a perspective. First, as
I mention these boundaries extend to services as well as encompass the
module layer. Second, even at the module layer alone, you assume that
you are in control of everything. What if I want to reuse an existing
OSGi bundle which I want to share amongst some bundles, but not with
other bundles? Large applications are often composed of subsystems
where the subsystems are composed from reusable components and desire
isolation from each other. Again, think hierarchical component model.
*This* is what I wanted to see if you'd respond with Smile

So, it sounds like what you're shooting for is a model for module use
where the person who composes the application is more like someone
composing a Linux distribution. Now, do people exist who do that at the
application-level, or would if they could? It reminds me a little of
the hype you'd hear in the late 90's about how, Real Soon Now, there
would be 10 programmers needed in the world, and applications would be
built by "component assemblers" (read: trained monkeys).

The question would be whether assembling an application much in the way
an OS distro is composed is actually a practical need, something demand
will develop for once it is available and its usefulness is proven, or
just a cool sounding thing almost nobody actually uses. Beyond very
coarse boundaries (choosing an OS, an application server, a database, a
framework), this is not a thing I see anybody asking for. Generally,
application code is still fairly tightly coupled above the framework
level - the process of general use APIs sedimenting into a framework or
below that level is necessarily slow (it only took our industry about 30
years to debate pascal strings vs. c strings). So while I could take
some of Boeing's airplane materials design modules, some of Fabrizio
Guiduci's photo editing modules, the NetBeans instant messaging modules
and Nuance's audio file editor modules, and maybe throw in an email
client module for good measure, I do end up with something that will
run. But what is this franken-app good for? There is certainly
generally useful stuff in all of these applications, and luckily we have
some lists and projects and enhancement requests that get those things
into NetBeans (although the platformx project needs a reboot). But I
don't know if, between the OSGi folks and NetBeans Platform folks, that
we've turned enough people into framework developers for an "assembly"
model of application development to get off the ground (one might argue
that this is the only way to *get* it off the ground).
Quote:
However, I agree that there is an extra level of complexity to this.
Especially tacking it onto an existing model, like OSGi. If these
ideas are thought of from the beginning, though, they can often be
done in a reasonably intuitive way.

Quote:
(If I am understanding you correctly) an ability to export another
module's implementation classes definitely adds flexibility - but it
rather defeats the point of modularity. If you want that much
flexibility, you could just put everything on the boot class path and
pray - which is what that amounts to.

Again, what is public API at one level is implementation API at
another level.
Okay. I think we are actually discussing three separate subjects with
some subtopics, and it might be good to tease them apart:

1. Scoped dependencies - restricting access to an API to a specific set
of clients

1a. Whether that scoping has directionality:
1a1. bi-directional (like NetBeans friend modules),
1a2. uni-directional (like traditional module dependencies), or
1a3. non-directional (third party can expose anything they want)

2. Meta-modules - a meta-module which acts as a container for other
modules and declares what it exports to modules outside the meta-module
2a. This also introduces a non-trivial versioning issue. Is module X
containing modules A and B the same version if it contains A and B+1?
Short of turning version numbers into hashes, I don't see how you can
have meaningful versioning of such meta-modules, which is going to limit
the container's ability to enforce dependencies.

3. Giving one module the ability to explicitly export some of another
module's packages.
3a. Should a module be able to export a superset of the packages the
contained module exports?
3b. How do you enforce that the secondary exporter does not
accidentally export an inaccessible class - i.e. if it exports a class
with a method foo (Bar) and Bar is hidden, where and how is this dealt with?

I don't think any of these really has that much to do with the other (if
you had meta-modules and scoped dependencies, meta-modules could make
use of scoped dependencies - but either can exist without the other).

I can imagine in a large application, 2. being useful - in fact, in the
form of module "clusters", NetBeans has the outline of this already
(without any sort of runtime enforcement of cross-cluster dependencies
or their direction).

Re 1a., 1a3. - allowing a third party to export packages the package
author never intended to be called, this looks like a recipe for
disaster - and entirely defeats one of the main benefits of modularity -
managed dependencies. 1a1 is what we have now; 1a2 with some sort of
token system seems reasonable (although I'm not convinced you ever need
to mix such strategies in a single module - one of the benefits of
modularity is that it encourages loose coupling, enabling reuse - in
that sense it has a social engineering function in development
practices. The ability to mix such strategies encourages creation of
"god modules" whose functionality is diverse and coupling is tight,
which leads to fewer genuinely reusable modules being developed. Better
to encourage lots of small modules that do one thing if we want a
healthy module ecosystem.).

3. could be reasonable enough, although it might have some performance
impact, since a dependent module's classloader now needs to include some
subset of the closure of the depended-on module's dependencies.

For 3a. (really 1a3 again), I can only say that I can't think of a
situation where somebody is developing software they expect to be
reliable, where this can do anything but harm. If you want to do this
sort of thing in NetBeans, you can, with implementation dependencies
(where we try to enforce that for an implementation dependency, the two
modules were compiled against the exact versions present in the
application). I don't see any good coming from making this easy.

-Tim
Back to top
Jesse Glick
Posted via mailing list.





PostPosted: Sun Jan 31, 2010 12:31 am    Post subject: [platform-dev] Re: Netbeans Friend package & OSGi guys :) Reply with quote

Tim Boudreau wrote:
Quote:
I would like to see an example where there is truly no option except to
have one module contain both a semi-private API and a public API, where
those two things provably could not be split into two modules, or where
it would be too labor intensive to do so.

As it happens we ran into several such cases in the past couple of weeks working on #179289. For example, api.progress exposes a true public API in
org.netbeans.api.progress and org.netbeans.api.progress.aggregate which is meant for general consumption and intended to be stable. It also exposes some other classes in
org.netbeans.modules.progress.spi which are intended for use just from core.execution (to show the "exit dialog") and progress.ui (to implement the API as a set of GUI
components). Since the NB friend system currently operates only at the module level, not at the package level, it is impossible to enforce this restriction; as a
workaround, the latter package is just excluded from Javadoc. Since this package both refers to the API package and is referred to from the API package, splitting it off
into another module would perhaps be possible but would fall into the "labor-intensive" category. There were other similar examples.

While it could be useful for other problems, module composition such as Richard proposes (or as included in the Fortress language) is not really desirable here, either.
Composition forces modules into a hierarchy, but these three modules do not form a natural unit. Some apps may wish to include only api.progress as a technical
dependency, knowing that it will not be used extensively. Others may wish to include progress.ui also but have no need for core.execution and its dependencies. Others
might need core.execution but not want progress.ui. We would really prefer these all remain separate modules, and simply restrict usage of certain classes to designated
friends unless and until a clean, stable SPI can be designed that we would feel comfortable publishing.
Back to top
Richard S. Hall
Posted via mailing list.





PostPosted: Sun Jan 31, 2010 2:25 am    Post subject: [platform-dev] Re: Netbeans Friend package & OSGi guys :) Reply with quote

On 1/30/10 18:08, Tim Boudreau wrote:
Quote:
Richard S. Hall wrote:
Quote:
On 1/30/10 1:26, Tim Boudreau wrote:
Quote:
Richard S. Hall wrote:
Quote:
On 1/29/10 6:35 PM, Tim Boudreau wrote:
Quote:
Daoud AbdelMonem Faleh wrote:
Quote:
Hi folks,

Today I was thinking of uses cases of Netbeans "Friend package"
feature
and concluded that one valid uses case is when a set of plugins
collaborate using an internal API to expose some feature to out
world
throw a public API. The internal API could be exposed to the
plugin set
throw "Friend package".
Generally we use friend packages as a way of *developing* an API -
i.e. it's a recognition that sometimes, when you are developing an
API, not all of the potential use-cases are known. Effective APIs
are often not born perfect from day 1. Rather than release
something that is inadequate or worse, incorrect, friend packages
allow the API to be developed in a modular way, without committing
to full backward compatibility until the API is solid. While
friend packages can be used to expose an API just to friend
modules forever, that is not its primary purpose - rather it
provides a way to develop an API that has real clients, and
respond to the requirements of those clients by improving the API,
before making it an official API with all of the commitments that
implies.

I think that would be the case to make to the OSGi folks, rather
than presenting it as a standard way to limit the visibility of an
API in general (although surely some people will use it that
way...but some people also use exceptions as a way of providing an
alternate return value from a method...).

Still, typical "friend" approaches require explicit listing of who
is or is not a friend. These lists are brittle
That's why it's really an indication of "under-developmentness" of
an API, not something to use without expecting incompatible changes.
That is certainly one way to cast it.

Quote:
Quote:
and sometimes a friend in one scenario is not a friend in another,
If that's the case, I suspect you really have two APIs and are
mixing them in one module.

I don't necessarily agree with this. The issue is the same for
providing a service that I want to share among some modules, but not
others.
I would read that as the "friend module" solution can be used to solve
both problems.

No it wouldn't. If, for example, you wanted to share a Log Service among
a subsystem, but not have other modules use it, then the "friend"
concept is not useful here since the Log Service package is public API.

Quote:
I would like to see an example where there is truly no option except
to have one module contain both a semi-private API and a public API,
where those two things provably could not be split into two modules,
or where it would be too labor intensive to do so.
Quote:

Quote:
Quote:
so this brittleness limits re-use.
Well, that's the point of friend-ness - to limit reuse of something
that is not ready for reuse, but needed by some third parties. When
there is a compatibility break, the person breaking compatibility
knows whom to communicate with; and third parties are keenly aware
that they are depending on something that might not be dependable.
I am happy to add any module someone asks me to as a friend of a
module I am working on - that way, I know the code is being used,
and there is an open channel for communication about changes and
change requests, and no party gets unpleasantly surprised.

I understand that this is how you choose to use the "friend"
mechanism, but in the end if it is introduced into a module system,
then it becomes a general mechanism that cannot be restricted to such
a "evolving API" use case.
True, that is a social convention that relies on human behavior. But
much of software development relies on human communication -
requirements coming in, notification of changes, etc.

Yes, but you'd have to admit that providing mechanisms that don't rely
on human behavior are preferable.

Quote:

[snip]
Quote:

Quote:
Quote:

Another approach to hiding API from some while exposing to others
is introducing a notion of composite modules. Think of a module
that logically contains other module and only exposes from them
what it wants. This is a more modular way of achieving a shared,
but private API. It is more flexible, since the friendship isn't
baked into the module doing the exporting, but rather into the
subsystem/composite declaration. Thus, the same modules can be used
in different composites exposing different parts (or none) of the
API as needed.

OSGi is working on something similar to this approach for the R4.3
release.
I seem to remember someone in OSGi-land talking about this with me
at EclipseCon a couple of years ago - I forget whom. He was very
passionate about this subject, and I couldn't really figure out why.

I was there last year and gave a talk with Tom Watson (IBM) on this
subject. I don't know if it is something to be passionate about, but
it is an worthwhile idea. It is similar to hierarchical component
models, but extending them to the module layer.
Ah, the catnip of developers - as soon as you have
interface Foo {
void foo();
}
someone will simply be unable to resist creating
class MetaFoo {
Foo[] foos;
public void foo() {
for (Foo foo: foos) { foo.foo(); }
}
}
It's just so cool to do that. Smile
Quote:
Quote:
I can see why doing this is an interesting idea in the abstract.
What I don't see is what problem it solves, which cannot be solved
simply by dividing the code into more modules and expressing the
dependencies plainly. What can you do with composite modules, aside
from create unenforceable contracts between components, that you
can't do simply by splitting friend-code into one module, API code
into another, and implementation code either in one or the other,
both or a separate third module that injects the implementation? It
seems like this would vastly complicate any sort of dependency
analysis in an application - for what benefit?

You look at the issues from too simplistic of a perspective. First,
as I mention these boundaries extend to services as well as encompass
the module layer. Second, even at the module layer alone, you assume
that you are in control of everything. What if I want to reuse an
existing OSGi bundle which I want to share amongst some bundles, but
not with other bundles? Large applications are often composed of
subsystems where the subsystems are composed from reusable components
and desire isolation from each other. Again, think hierarchical
component model.
*This* is what I wanted to see if you'd respond with Smile
So, it sounds like what you're shooting for is a model for module use
where the person who composes the application is more like someone
composing a Linux distribution.

I don't think this is a valid analogy. Linux distros are more like a
grab bag of components and anyone is free to use anything.

Quote:
Now, do people exist who do that at the application-level, or would if
they could? It reminds me a little of the hype you'd hear in the late
90's about how, Real Soon Now, there would be 10 programmers needed in
the world, and applications would be built by "component assemblers"
(read: trained monkeys).

Believe it or not, the OSGi Alliance isn't looking into this for the fun
of it, but people are requesting such capabilities.

Quote:
The question would be whether assembling an application much in the
way an OS distro is composed is actually a practical need, something
demand will develop for once it is available and its usefulness is
proven, or just a cool sounding thing almost nobody actually uses.
Beyond very coarse boundaries (choosing an OS, an application server,
a database, a framework), this is not a thing I see anybody asking for.

I think you'd be incorrect. See above.

Quote:
Generally, application code is still fairly tightly coupled above the
framework level - the process of general use APIs sedimenting into a
framework or below that level is necessarily slow (it only took our
industry about 30 years to debate pascal strings vs. c strings). So
while I could take some of Boeing's airplane materials design modules,
some of Fabrizio Guiduci's photo editing modules, the NetBeans instant
messaging modules and Nuance's audio file editor modules, and maybe
throw in an email client module for good measure, I do end up with
something that will run. But what is this franken-app good for?
There is certainly generally useful stuff in all of these
applications, and luckily we have some lists and projects and
enhancement requests that get those things into NetBeans (although the
platformx project needs a reboot). But I don't know if, between the
OSGi folks and NetBeans Platform folks, that we've turned enough
people into framework developers for an "assembly" model of
application development to get off the ground (one might argue that
this is the only way to *get* it off the ground).

Don't get me wrong, I think moving to such a model in the general sense
is still daunting too. But in the less general sense, we certainly do
have reusable components in OSGi that need to be isolated from each
other, but still be part of the same system.

Quote:
Quote:
However, I agree that there is an extra level of complexity to this.
Especially tacking it onto an existing model, like OSGi. If these
ideas are thought of from the beginning, though, they can often be
done in a reasonably intuitive way.

Quote:
(If I am understanding you correctly) an ability to export another
module's implementation classes definitely adds flexibility - but it
rather defeats the point of modularity. If you want that much
flexibility, you could just put everything on the boot class path
and pray - which is what that amounts to.

Again, what is public API at one level is implementation API at
another level.
Okay. I think we are actually discussing three separate subjects with
some subtopics, and it might be good to tease them apart:

1. Scoped dependencies - restricting access to an API to a specific
set of clients

1a. Whether that scoping has directionality:
1a1. bi-directional (like NetBeans friend modules),
1a2. uni-directional (like traditional module dependencies), or
1a3. non-directional (third party can expose anything they want)

2. Meta-modules - a meta-module which acts as a container for other
modules and declares what it exports to modules outside the meta-module
2a. This also introduces a non-trivial versioning issue. Is module
X containing modules A and B the same version if it contains A and
B+1? Short of turning version numbers into hashes, I don't see how
you can have meaningful versioning of such meta-modules, which is
going to limit the container's ability to enforce dependencies.

We deal with packages in OSGi, the version of modules is less important
for the most part, but in the end, there is nothing conceptually wrong
with version ranges and variability in a "metamodule".

Quote:
3. Giving one module the ability to explicitly export some of another
module's packages.
3a. Should a module be able to export a superset of the packages the
contained module exports?
3b. How do you enforce that the secondary exporter does not
accidentally export an inaccessible class - i.e. if it exports a class
with a method foo (Bar) and Bar is hidden, where and how is this dealt
with?

I don't think any of these really has that much to do with the other
(if you had meta-modules and scoped dependencies, meta-modules could
make use of scoped dependencies - but either can exist without the
other).

I am not sure about your "metamodule", but my "composite module" doesn't
actually contain anything. It is just a description of the modules
contained in it and what it exposes and requires. It is largely an
encapsulation/isolation mechanism.

Quote:
I can imagine in a large application, 2. being useful - in fact, in
the form of module "clusters", NetBeans has the outline of this
already (without any sort of runtime enforcement of cross-cluster
dependencies or their direction).
Re 1a., 1a3. - allowing a third party to export packages the package
author never intended to be called, this looks like a recipe for
disaster - and entirely defeats one of the main benefits of modularity
- managed dependencies. 1a1 is what we have now; 1a2 with some sort
of token system seems reasonable (although I'm not convinced you ever
need to mix such strategies in a single module - one of the benefits
of modularity is that it encourages loose coupling, enabling reuse -
in that sense it has a social engineering function in development
practices. The ability to mix such strategies encourages creation of
"god modules" whose functionality is diverse and coupling is tight,
which leads to fewer genuinely reusable modules being developed.
Better to encourage lots of small modules that do one thing if we want
a healthy module ecosystem.).

I think some of this will go away when/if Java has true a true "module"
access modifier. Then such API can be labeled "module" and a composite
can map its constituent modules to the same run-time module so they are
only accessible to each other and no one else. Then we have enforcement
and no brittle lists. Of course, this will depend on what comes out of
JSR 294.

Quote:

3. could be reasonable enough, although it might have some performance
impact, since a dependent module's classloader now needs to include
some subset of the closure of the depended-on module's dependencies.

For 3a. (really 1a3 again), I can only say that I can't think of a
situation where somebody is developing software they expect to be
reliable, where this can do anything but harm. If you want to do this
sort of thing in NetBeans, you can, with implementation dependencies
(where we try to enforce that for an implementation dependency, the
two modules were compiled against the exact versions present in the
application). I don't see any good coming from making this easy.

Yeah, I'm not proposing that.

-> richard

Quote:

-Tim


Back to top
Tim Boudreau
Posted via mailing list.





PostPosted: Sun Jan 31, 2010 6:40 am    Post subject: [platform-dev] Re: Netbeans Friend package & OSGi guys :) Reply with quote

Richard S. Hall wrote:
Quote:
On 1/30/10 18:08, Tim Boudreau wrote:
Quote:
Richard S. Hall wrote:
Quote:
On 1/30/10 1:26, Tim Boudreau wrote:
Quote:
Richard S. Hall wrote:
Quote:
On 1/29/10 6:35 PM, Tim Boudreau wrote:
Quote:
Daoud AbdelMonem Faleh wrote:
Quote:
Hi folks,

Today I was thinking of uses cases of Netbeans "Friend package"
feature
and concluded that one valid uses case is when a set of plugins
collaborate using an internal API to expose some feature to out
world
throw a public API. The internal API could be exposed to the
plugin set
throw "Friend package".
Generally we use friend packages as a way of *developing* an API
- i.e. it's a recognition that sometimes, when you are developing
an API, not all of the potential use-cases are known. Effective
APIs are often not born perfect from day 1. Rather than release
something that is inadequate or worse, incorrect, friend packages
allow the API to be developed in a modular way, without
committing to full backward compatibility until the API is
solid. While friend packages can be used to expose an API just
to friend modules forever, that is not its primary purpose -
rather it provides a way to develop an API that has real clients,
and respond to the requirements of those clients by improving the
API, before making it an official API with all of the commitments
that implies.

I think that would be the case to make to the OSGi folks, rather
than presenting it as a standard way to limit the visibility of
an API in general (although surely some people will use it that
way...but some people also use exceptions as a way of providing
an alternate return value from a method...).

Still, typical "friend" approaches require explicit listing of who
is or is not a friend. These lists are brittle
That's why it's really an indication of "under-developmentness" of
an API, not something to use without expecting incompatible changes.
That is certainly one way to cast it.

Quote:
Quote:
and sometimes a friend in one scenario is not a friend in another,
If that's the case, I suspect you really have two APIs and are
mixing them in one module.

I don't necessarily agree with this. The issue is the same for
providing a service that I want to share among some modules, but not
others.
I would read that as the "friend module" solution can be used to
solve both problems.

No it wouldn't. If, for example, you wanted to share a Log Service
among a subsystem, but not have other modules use it, then the
"friend" concept is not useful here since the Log Service package is
public API.
X contains A, B and LogService. A and B are friends of LogService. X
contains A, B and LogService and exports A and B but not LogService. At
which point, X becomes superfluous. The case where this fails is when
LogService is somehow public in its own right. But any system that
allows a third party to specify a magic token that allows access to
LogService means you can never guarantee it is not being used by a third
party. Doesn't this put us back at square one?
Quote:

Quote:
I would like to see an example where there is truly no option except
to have one module contain both a semi-private API and a public API,
where those two things provably could not be split into two modules,
or where it would be too labor intensive to do so.
Jesse's examples of these are interesting. I suspect some combination
of friend modules and API and SPI modules (probably doing some
contortions to make it work) could solve that case, but I haven't dug
into it to seriously think through the issues - and anyway, these
changes were done to get rid of implementation dependencies in order to
support OSGi. If OSGi doesn't have a comparable "friend" category
(although the mandatory attributes you mentioned might be able to be
used for that), then any solution using friend modules is a non-starter.
Quote:
Quote:
Quote:

Quote:
Quote:
so this brittleness limits re-use.
Well, that's the point of friend-ness - to limit reuse of something
that is not ready for reuse, but needed by some third parties.
When there is a compatibility break, the person breaking
compatibility knows whom to communicate with; and third parties
are keenly aware that they are depending on something that might
not be dependable. I am happy to add any module someone asks me to
as a friend of a module I am working on - that way, I know the code
is being used, and there is an open channel for communication about
changes and change requests, and no party gets unpleasantly surprised.

I understand that this is how you choose to use the "friend"
mechanism, but in the end if it is introduced into a module system,
then it becomes a general mechanism that cannot be restricted to
such a "evolving API" use case.
True, that is a social convention that relies on human behavior. But
much of software development relies on human communication -
requirements coming in, notification of changes, etc.
Yes, but you'd have to admit that providing mechanisms that don't rely
on human behavior are preferable.
For the case of APIs that are genuinely under development, nothing we
are discussing replaces the need for communication as the API evolves.
If I read this right, friend-style modules without these brittle lists
is just choosing to ignore the "under development" use-case for
friend-style dependencies. I think there might be a need for some more
flexible scoping mechanism for declaring dependencies, *along with* the
friend-with-brittle-list mechanism, and use that mechanism exclusively
for the under-development case.

[snip]
Quote:
Quote:
Quote:
You look at the issues from too simplistic of a perspective. First,
as I mention these boundaries extend to services as well as
encompass the module layer. Second, even at the module layer alone,
you assume that you are in control of everything. What if I want to
reuse an existing OSGi bundle which I want to share amongst some
bundles, but not with other bundles? Large applications are often
composed of subsystems where the subsystems are composed from
reusable components and desire isolation from each other. Again,
think hierarchical component model.
*This* is what I wanted to see if you'd respond with Smile
So, it sounds like what you're shooting for is a model for module use
where the person who composes the application is more like someone
composing a Linux distribution.
I don't think this is a valid analogy. Linux distros are more like a
grab bag of components and anyone is free to use anything.
True that there are not restrictions on what can use what; but there
is, with most packaging systems, dependency management with versioning.
I was seeing a composite module as being rather similar to an
application in Debian's packaging scheme - if I install Mozilla, and it
requires library A >= 1.0, and then I install Evolution, and it requires
libjpeg >= 1.2, I typically end up with one copy of libjpeg and it is >=
1.2. The creator of the distribution decides what libraries and what
versions of them should be available, and applications will use those to
satisfy their dependencies.
Quote:

Quote:
Now, do people exist who do that at the application-level, or would
if they could? It reminds me a little of the hype you'd hear in the
late 90's about how, Real Soon Now, there would be 10 programmers
needed in the world, and applications would be built by "component
assemblers" (read: trained monkeys).
Believe it or not, the OSGi Alliance isn't looking into this for the
fun of it, but people are requesting such capabilities.
Well, people have been for 20 years. Remember when Java Beans were
going to solve this problem? Smile And before that, unix pipes (which
arguably does a better job at decoupling than anything we're discussing,
but with no version or dependency management at all).

Perhaps we are getting closer - I think we are. But it does seem like
about every ten years, we solve the problem of building software by
gluing together a bunch of black boxes, and then find out we haven't
solved it after all.

Quote:
Quote:
The question would be whether assembling an application much in the
way an OS distro is composed is actually a practical need, something
demand will develop for once it is available and its usefulness is
proven, or just a cool sounding thing almost nobody actually uses.
Beyond very coarse boundaries (choosing an OS, an application server,
a database, a framework), this is not a thing I see anybody asking for.

I think you'd be incorrect. See above.
Then you're right. I'd be very curious about what the people asking for
this think they are getting, and what problems they see it as solving.

Quote:
Quote:
Generally, application code is still fairly tightly coupled above the
framework level - the process of general use APIs sedimenting into a
framework or below that level is necessarily slow (it only took our
industry about 30 years to debate pascal strings vs. c strings). So
while I could take some of Boeing's airplane materials design
modules, some of Fabrizio Guiduci's photo editing modules, the
NetBeans instant messaging modules and Nuance's audio file editor
modules, and maybe throw in an email client module for good measure,
I do end up with something that will run. But what is this
franken-app good for? There is certainly generally useful stuff in
all of these applications, and luckily we have some lists and
projects and enhancement requests that get those things into NetBeans
(although the platformx project needs a reboot). But I don't know
if, between the OSGi folks and NetBeans Platform folks, that we've
turned enough people into framework developers for an "assembly"
model of application development to get off the ground (one might
argue that this is the only way to *get* it off the ground).

Don't get me wrong, I think moving to such a model in the general
sense is still daunting too. But in the less general sense, we
certainly do have reusable components in OSGi that need to be isolated
from each other, but still be part of the same system.
That seems reasonable. But is there a better example than a logger
(which seems like it would be general enough that running two
independent copies in the same app wouldn't actually change application
behavior) or a case of needing a specific version (perhaps one package
relied on a side effect of a bug that was fixed), for a case where some
package is used by two parties, but it would be harmful for them to
share it? Say we have two independent modules, and both use JavaMail.
When is it necessary for each to have their own version of it?
Quote:

Quote:
Quote:
However, I agree that there is an extra level of complexity to this.
Especially tacking it onto an existing model, like OSGi. If these
ideas are thought of from the beginning, though, they can often be
done in a reasonably intuitive way.

Quote:
(If I am understanding you correctly) an ability to export another
module's implementation classes definitely adds flexibility - but
it rather defeats the point of modularity. If you want that much
flexibility, you could just put everything on the boot class path
and pray - which is what that amounts to.

Again, what is public API at one level is implementation API at
another level.
Okay. I think we are actually discussing three separate subjects
with some subtopics, and it might be good to tease them apart:

1. Scoped dependencies - restricting access to an API to a specific
set of clients

1a. Whether that scoping has directionality:
1a1. bi-directional (like NetBeans friend modules),
1a2. uni-directional (like traditional module dependencies), or
1a3. non-directional (third party can expose anything they want)

2. Meta-modules - a meta-module which acts as a container for other
modules and declares what it exports to modules outside the meta-module
2a. This also introduces a non-trivial versioning issue. Is module
X containing modules A and B the same version if it contains A and
B+1? Short of turning version numbers into hashes, I don't see how
you can have meaningful versioning of such meta-modules, which is
going to limit the container's ability to enforce dependencies.

We deal with packages in OSGi, the version of modules is less
important for the most part, but in the end, there is nothing
conceptually wrong with version ranges and variability in a "metamodule".
That makes sense - if you version at the package level, then a metamodel
is just a list of packages and their versions - correct? So the
dependency is really a pass-through to whatever module provides the
package. In which case, the only purpose of the meta-module is restrict
access to some other contained module?
Quote:
Quote:
3. Giving one module the ability to explicitly export some of
another module's packages.
3a. Should a module be able to export a superset of the packages
the contained module exports?
3b. How do you enforce that the secondary exporter does not
accidentally export an inaccessible class - i.e. if it exports a
class with a method foo (Bar) and Bar is hidden, where and how is
this dealt with?

I don't think any of these really has that much to do with the other
(if you had meta-modules and scoped dependencies, meta-modules could
make use of scoped dependencies - but either can exist without the
other).

I am not sure about your "metamodule", but my "composite module"
doesn't actually contain anything. It is just a description of the
modules contained in it and what it exposes and requires. It is
largely an encapsulation/isolation mechanism.
I think we are defining them the same way. I wasn't suggesting a
composite module would contain code (not ruling that out either).
Quote:
Quote:
Re 1a., 1a3. - allowing a third party to export packages the package
author never intended to be called, this looks like a recipe for
disaster - and entirely defeats one of the main benefits of
modularity - managed dependencies. 1a1 is what we have now; 1a2 with
some sort of token system seems reasonable (although I'm not
convinced you ever need to mix such strategies in a single module -
one of the benefits of modularity is that it encourages loose
coupling, enabling reuse - in that sense it has a social engineering
function in development practices. The ability to mix such
strategies encourages creation of "god modules" whose functionality
is diverse and coupling is tight, which leads to fewer genuinely
reusable modules being developed. Better to encourage lots of small
modules that do one thing if we want a healthy module ecosystem.).
I can imagine in a large application, 2. being useful - in fact, in
the form of module "clusters", NetBeans has the outline of this
already (without any sort of runtime enforcement of cross-cluster
dependencies or their direction).

I think some of this will go away when/if Java has true a true
"module" access modifier. Then such API can be labeled "module" and a
composite can map its constituent modules to the same run-time module
so they are only accessible to each other and no one else. Then we
have enforcement and no brittle lists. Of course, this will depend on
what comes out of JSR 294.
Agreed - a lot of what both the NetBeans Platform and OSGi are about is
really doing is working around the limitations available access
modifiers in Java. In API talks, I usually describe this in terms of
the ability to have a package in a JAR which is private to that package
- one of the real benefits of this is being freed from using package
privacy to handle all internal communication in modules. That has
wonderful effects for separating API and SPI, and not ending up with
gargantuan API packages (think javax.swing or java.awt) which make it
hard for a developer to focus in on what classes they actually need.
Quote:
Quote:

3. could be reasonable enough, although it might have some
performance impact, since a dependent module's classloader now needs
to include some subset of the closure of the depended-on module's
dependencies.

For 3a. (really 1a3 again), I can only say that I can't think of a
situation where somebody is developing software they expect to be
reliable, where this can do anything but harm. If you want to do
this sort of thing in NetBeans, you can, with implementation
dependencies (where we try to enforce that for an implementation
dependency, the two modules were compiled against the exact versions
present in the application). I don't see any good coming from making
this easy.
Yeah, I'm not proposing that.
Good :-)

Given Jesse's examples, I'm beginning to be convinced that there are
use-cases for both "friend" dependencies and scoped dependencies. Still
not sure that the problems solved by meta-modules could not be solved
without them - or at least I suspect that cases where you really need
two copies of the same module in an application are likely to be very
rare corner cases - if you're depending on a set of packages without
caring who provides them, but really need an independent instance of
them, you could still do that just by having two copies of the JAR
file. If the module is depending on specific classes, but needs them
loaded in its own classloader, that seems doable without meta-modules -
if the depending-module needs to declare what packages it wants, and
their minimum version, the only thing the meta-module is doing is hiding
some packages. In many cases these could be moved to another module.

-Tim
Back to top
Richard S. Hall
Posted via mailing list.





PostPosted: Sun Jan 31, 2010 4:52 pm    Post subject: [platform-dev] Re: Netbeans Friend package & OSGi guys :) Reply with quote

On 1/31/10 1:40, Tim Boudreau wrote:
Quote:
Richard S. Hall wrote:
Quote:
On 1/30/10 18:08, Tim Boudreau wrote:
Quote:
Richard S. Hall wrote:
Quote:
On 1/30/10 1:26, Tim Boudreau wrote:
Quote:
Richard S. Hall wrote:
Quote:
On 1/29/10 6:35 PM, Tim Boudreau wrote:
Quote:
Daoud AbdelMonem Faleh wrote:
Quote:
Hi folks,

Today I was thinking of uses cases of Netbeans "Friend package"
feature
and concluded that one valid uses case is when a set of plugins
collaborate using an internal API to expose some feature to out
world
throw a public API. The internal API could be exposed to the
plugin set
throw "Friend package".
Generally we use friend packages as a way of *developing* an API
- i.e. it's a recognition that sometimes, when you are
developing an API, not all of the potential use-cases are
known. Effective APIs are often not born perfect from day 1.
Rather than release something that is inadequate or worse,
incorrect, friend packages allow the API to be developed in a
modular way, without committing to full backward compatibility
until the API is solid. While friend packages can be used to
expose an API just to friend modules forever, that is not its
primary purpose - rather it provides a way to develop an API
that has real clients, and respond to the requirements of those
clients by improving the API, before making it an official API
with all of the commitments that implies.

I think that would be the case to make to the OSGi folks, rather
than presenting it as a standard way to limit the visibility of
an API in general (although surely some people will use it that
way...but some people also use exceptions as a way of providing
an alternate return value from a method...).

Still, typical "friend" approaches require explicit listing of
who is or is not a friend. These lists are brittle
That's why it's really an indication of "under-developmentness" of
an API, not something to use without expecting incompatible changes.
That is certainly one way to cast it.

Quote:
Quote:
and sometimes a friend in one scenario is not a friend in another,
If that's the case, I suspect you really have two APIs and are
mixing them in one module.

I don't necessarily agree with this. The issue is the same for
providing a service that I want to share among some modules, but
not others.
I would read that as the "friend module" solution can be used to
solve both problems.

No it wouldn't. If, for example, you wanted to share a Log Service
among a subsystem, but not have other modules use it, then the
"friend" concept is not useful here since the Log Service package is
public API.
X contains A, B and LogService. A and B are friends of LogService. X
contains A, B and LogService and exports A and B but not LogService.
At which point, X becomes superfluous. The case where this fails is
when LogService is somehow public in its own right.

This was exactly the case i was alluding to...clearly if Log Service
isn't public then there is no issue. This is why i keep mentioning
"reusable" modules, since we are reusing them, we didn't create them.

Quote:
But any system that allows a third party to specify a magic token that
allows access to LogService means you can never guarantee it is not
being used by a third party. Doesn't this put us back at square one?

Not really, in a composite, you cannot guarantee that someone doesn't
have their own copy of Log Service somewhere, but you can guarantee that
your Log Service is your own private copy shared among the composite's
constituent modules.

Quote:
Quote:
Quote:
I would like to see an example where there is truly no option except
to have one module contain both a semi-private API and a public API,
where those two things provably could not be split into two modules,
or where it would be too labor intensive to do so.
Jesse's examples of these are interesting. I suspect some combination
of friend modules and API and SPI modules (probably doing some
contortions to make it work) could solve that case, but I haven't dug
into it to seriously think through the issues - and anyway, these
changes were done to get rid of implementation dependencies in order
to support OSGi. If OSGi doesn't have a comparable "friend" category
(although the mandatory attributes you mentioned might be able to be
used for that), then any solution using friend modules is a non-starter.

For me, this is not purely an API-based issue. I don't know how service
lookup happens in NetBeans, but in OSGi the service registry is a flat
space. So modules may not only want to limit visibility of packages, but
of services. Composites provide a way of doing both.

Quote:
Quote:
Quote:
Quote:

Quote:
Quote:
so this brittleness limits re-use.
Well, that's the point of friend-ness - to limit reuse of
something that is not ready for reuse, but needed by some third
parties. When there is a compatibility break, the person breaking
compatibility knows whom to communicate with; and third parties
are keenly aware that they are depending on something that might
not be dependable. I am happy to add any module someone asks me
to as a friend of a module I am working on - that way, I know the
code is being used, and there is an open channel for communication
about changes and change requests, and no party gets unpleasantly
surprised.

I understand that this is how you choose to use the "friend"
mechanism, but in the end if it is introduced into a module system,
then it becomes a general mechanism that cannot be restricted to
such a "evolving API" use case.
True, that is a social convention that relies on human behavior.
But much of software development relies on human communication -
requirements coming in, notification of changes, etc.
Yes, but you'd have to admit that providing mechanisms that don't
rely on human behavior are preferable.
For the case of APIs that are genuinely under development, nothing we
are discussing replaces the need for communication as the API
evolves. If I read this right, friend-style modules without these
brittle lists is just choosing to ignore the "under development"
use-case for friend-style dependencies. I think there might be a need
for some more flexible scoping mechanism for declaring dependencies,
*along with* the friend-with-brittle-list mechanism, and use that
mechanism exclusively for the under-development case.

Maybe.

Quote:

[snip]
Quote:
Quote:
Quote:
You look at the issues from too simplistic of a perspective. First,
as I mention these boundaries extend to services as well as
encompass the module layer. Second, even at the module layer alone,
you assume that you are in control of everything. What if I want to
reuse an existing OSGi bundle which I want to share amongst some
bundles, but not with other bundles? Large applications are often
composed of subsystems where the subsystems are composed from
reusable components and desire isolation from each other. Again,
think hierarchical component model.
*This* is what I wanted to see if you'd respond with Smile
So, it sounds like what you're shooting for is a model for module
use where the person who composes the application is more like
someone composing a Linux distribution.
I don't think this is a valid analogy. Linux distros are more like a
grab bag of components and anyone is free to use anything.
True that there are not restrictions on what can use what; but there
is, with most packaging systems, dependency management with
versioning. I was seeing a composite module as being rather similar
to an application in Debian's packaging scheme - if I install Mozilla,
and it requires library A >= 1.0, and then I install Evolution, and it
requires libjpeg >= 1.2, I typically end up with one copy of libjpeg
and it is >= 1.2. The creator of the distribution decides what
libraries and what versions of them should be available, and
applications will use those to satisfy their dependencies.

Yeah, but there is no way of saying I want to keep these packages and
services private to my application.

Quote:
Quote:

Quote:
Now, do people exist who do that at the application-level, or would
if they could? It reminds me a little of the hype you'd hear in the
late 90's about how, Real Soon Now, there would be 10 programmers
needed in the world, and applications would be built by "component
assemblers" (read: trained monkeys).
Believe it or not, the OSGi Alliance isn't looking into this for the
fun of it, but people are requesting such capabilities.
Well, people have been for 20 years. Remember when Java Beans were
going to solve this problem? Smile

I think you are confusing composability (JavaBeans) with
hierarchical/recursive composability (which is largely not done in
industrial/practical component models, although now SCA supports this, I
think, and I believe CCM did...otherwise it has been mostly fantasized
about in academic component models).

Quote:
And before that, unix pipes (which arguably does a better job at
decoupling than anything we're discussing, but with no version or
dependency management at all).

Perhaps we are getting closer - I think we are. But it does seem like
about every ten years, we solve the problem of building software by
gluing together a bunch of black boxes, and then find out we haven't
solved it after all.

I am not arguing that this is a solved problem, so I think we agree that
we have a ways to go.

Quote:

Quote:
Quote:
The question would be whether assembling an application much in the
way an OS distro is composed is actually a practical need, something
demand will develop for once it is available and its usefulness is
proven, or just a cool sounding thing almost nobody actually uses.
Beyond very coarse boundaries (choosing an OS, an application
server, a database, a framework), this is not a thing I see anybody
asking for.

I think you'd be incorrect. See above.
Then you're right. I'd be very curious about what the people asking
for this think they are getting, and what problems they see it as
solving.

To be clear, we are not talking about your day-to-day programming doing
some "hello world" app. Think of developing an app server where you need
to load arbitrary applications composed of custom and reusable modules.
In this case, the app server wants to isolate the applications so they
cannot interfere with each other, but at the same time it wants to share
some stuff with them (and possibly among them).

Quote:

Quote:
Quote:
Generally, application code is still fairly tightly coupled above
the framework level - the process of general use APIs sedimenting
into a framework or below that level is necessarily slow (it only
took our industry about 30 years to debate pascal strings vs. c
strings). So while I could take some of Boeing's airplane materials
design modules, some of Fabrizio Guiduci's photo editing modules,
the NetBeans instant messaging modules and Nuance's audio file
editor modules, and maybe throw in an email client module for good
measure, I do end up with something that will run. But what is this
franken-app good for? There is certainly generally useful stuff in
all of these applications, and luckily we have some lists and
projects and enhancement requests that get those things into
NetBeans (although the platformx project needs a reboot). But I
don't know if, between the OSGi folks and NetBeans Platform folks,
that we've turned enough people into framework developers for an
"assembly" model of application development to get off the ground
(one might argue that this is the only way to *get* it off the ground).

Don't get me wrong, I think moving to such a model in the general
sense is still daunting too. But in the less general sense, we
certainly do have reusable components in OSGi that need to be
isolated from each other, but still be part of the same system.
That seems reasonable. But is there a better example than a logger
(which seems like it would be general enough that running two
independent copies in the same app wouldn't actually change
application behavior) or a case of needing a specific version (perhaps
one package relied on a side effect of a bug that was fixed), for a
case where some package is used by two parties, but it would be
harmful for them to share it? Say we have two independent modules,
and both use JavaMail. When is it necessary for each to have their
own version of it?

As I said before, any modules that maintain state or provide any
stateful services. Believe me, my first response to these sorts of
requests (at the package-level) was to suggest using a separate
framework instance or even separate processes if you want isolation.
This suggestion was not met with enthusiasm. However, I do feel that
isolation at the service-level does make sense, so it was somewhat
difficult for me to keep arguing that it doesn't make sense at the
package level. (The main difference is that it is easier to do at the
service level because we don't have to fight with the underlying
mechanics of the JVM.)

Quote:
Quote:

Quote:
Quote:
However, I agree that there is an extra level of complexity to
this. Especially tacking it onto an existing model, like OSGi. If
these ideas are thought of from the beginning, though, they can
often be done in a reasonably intuitive way.

Quote:
(If I am understanding you correctly) an ability to export another
module's implementation classes definitely adds flexibility - but
it rather defeats the point of modularity. If you want that much
flexibility, you could just put everything on the boot class path
and pray - which is what that amounts to.

Again, what is public API at one level is implementation API at
another level.
Okay. I think we are actually discussing three separate subjects
with some subtopics, and it might be good to tease them apart:

1. Scoped dependencies - restricting access to an API to a specific
set of clients

1a. Whether that scoping has directionality:
1a1. bi-directional (like NetBeans friend modules),
1a2. uni-directional (like traditional module dependencies), or
1a3. non-directional (third party can expose anything they want)

2. Meta-modules - a meta-module which acts as a container for other
modules and declares what it exports to modules outside the meta-module
2a. This also introduces a non-trivial versioning issue. Is
module X containing modules A and B the same version if it contains
A and B+1? Short of turning version numbers into hashes, I don't
see how you can have meaningful versioning of such meta-modules,
which is going to limit the container's ability to enforce
dependencies.

We deal with packages in OSGi, the version of modules is less
important for the most part, but in the end, there is nothing
conceptually wrong with version ranges and variability in a
"metamodule".
That makes sense - if you version at the package level, then a
metamodel is just a list of packages and their versions - correct? So
the dependency is really a pass-through to whatever module provides
the package. In which case, the only purpose of the meta-module is
restrict access to some other contained module?

Pretty much, that's correct. It's an isolation mechanism.

Quote:
Quote:
Quote:
3. Giving one module the ability to explicitly export some of
another module's packages.
3a. Should a module be able to export a superset of the packages
the contained module exports?
3b. How do you enforce that the secondary exporter does not
accidentally export an inaccessible class - i.e. if it exports a
class with a method foo (Bar) and Bar is hidden, where and how is
this dealt with?

I don't think any of these really has that much to do with the other
(if you had meta-modules and scoped dependencies, meta-modules could
make use of scoped dependencies - but either can exist without the
other).

I am not sure about your "metamodule", but my "composite module"
doesn't actually contain anything. It is just a description of the
modules contained in it and what it exposes and requires. It is
largely an encapsulation/isolation mechanism.
I think we are defining them the same way. I wasn't suggesting a
composite module would contain code (not ruling that out either).

Yeah, if the composite actually contains code, that raises some
interesting possibilities, but it makes it more complicated too. If I
were starting over from scratch creating OSGi++, I'd certainly consider
that case.

Quote:
Quote:
Quote:
Re 1a., 1a3. - allowing a third party to export packages the package
author never intended to be called, this looks like a recipe for
disaster - and entirely defeats one of the main benefits of
modularity - managed dependencies. 1a1 is what we have now; 1a2
with some sort of token system seems reasonable (although I'm not
convinced you ever need to mix such strategies in a single module -
one of the benefits of modularity is that it encourages loose
coupling, enabling reuse - in that sense it has a social engineering
function in development practices. The ability to mix such
strategies encourages creation of "god modules" whose functionality
is diverse and coupling is tight, which leads to fewer genuinely
reusable modules being developed. Better to encourage lots of small
modules that do one thing if we want a healthy module ecosystem.).
I can imagine in a large application, 2. being useful - in fact, in
the form of module "clusters", NetBeans has the outline of this
already (without any sort of runtime enforcement of cross-cluster
dependencies or their direction).

I think some of this will go away when/if Java has true a true
"module" access modifier. Then such API can be labeled "module" and a
composite can map its constituent modules to the same run-time module
so they are only accessible to each other and no one else. Then we
have enforcement and no brittle lists. Of course, this will depend on
what comes out of JSR 294.
Agreed - a lot of what both the NetBeans Platform and OSGi are about
is really doing is working around the limitations available access
modifiers in Java. In API talks, I usually describe this in terms of
the ability to have a package in a JAR which is private to that
package - one of the real benefits of this is being freed from using
package privacy to handle all internal communication in modules. That
has wonderful effects for separating API and SPI, and not ending up
with gargantuan API packages (think javax.swing or java.awt) which
make it hard for a developer to focus in on what classes they actually
need.

Agreed.

Quote:
Quote:
Quote:

3. could be reasonable enough, although it might have some
performance impact, since a dependent module's classloader now needs
to include some subset of the closure of the depended-on module's
dependencies.

For 3a. (really 1a3 again), I can only say that I can't think of a
situation where somebody is developing software they expect to be
reliable, where this can do anything but harm. If you want to do
this sort of thing in NetBeans, you can, with implementation
dependencies (where we try to enforce that for an implementation
dependency, the two modules were compiled against the exact versions
present in the application). I don't see any good coming from
making this easy.
Yeah, I'm not proposing that.
Good :-)

Given Jesse's examples, I'm beginning to be convinced that there are
use-cases for both "friend" dependencies and scoped dependencies.
Still not sure that the problems solved by meta-modules could not be
solved without them - or at least I suspect that cases where you
really need two copies of the same module in an application are likely
to be very rare corner cases - if you're depending on a set of
packages without caring who provides them, but really need an
independent instance of them, you could still do that just by having
two copies of the JAR file. If the module is depending on specific
classes, but needs them loaded in its own classloader, that seems
doable without meta-modules - if the depending-module needs to declare
what packages it wants, and their minimum version, the only thing the
meta-module is doing is hiding some packages. In many cases these
could be moved to another module.

Again, keep in mind that for OSGi, modules and services go hand in hand.
It is not only about API, but about run-time services.

-> richard

Quote:

-Tim
Back to top
njbartlett



Joined: 31 Jan 2010
Posts: 1

PostPosted: Sun Jan 31, 2010 11:46 pm    Post subject: Re: [platform-dev] Re: Netbeans Friend package & OSGi guys :) Reply with quote

I was asked by Daoud to comment on this thread.

First, I agree with much of what Richard has said, although I don't believe that Composite Bundles are a solution for the use-case that Tim mentions at the start of the thread, namely experimental or unstable APIs. I see composites as mostly useful for the isolation they provide with respect to services, allowing multiple logical "applications" to be deployed to a single OSGi framework. This may be useful for the developers of complex infrastructure products such as application servers, but it is not something that most OSGi developers should concern themselves with.

Rather, I believe that the mandatory attribute on export is sufficient for dealing with this use-case. It provides the right balance between communicating the need for caution, while still enabling unexpected and innovative uses of the APIs. If an API is good enough to be used by my friends, then why not by my enemies?

Jesse's post describes a system of controls, so that the author of an API knows exactly who is using it (because they are *forced* to ask for permission to use it). The point of this is apparently to enable "push" notification to consumers when the API changes, but I don't believe this approach scales. It also seems inappropriate to encode development workflow practices into a runtime module system. There are other ways to communicate with users besides running a Find Usages query... e.g.an RSS feed. I lose track of the number of times I have been forced to use "internal" packages from Eclipse, particularly the JDT sub-project. As much as I would like Erich Gamma to phone me every time his team makes a change, I don't think that's really practical. When I use those packages, I get a pretty stern warning from Eclipse about it, so I know that I should probably import a tight version range and also run my tests carefully against each version. In other words, it's my responsibility to track changes in the API.

So, I don't think friend exports are necessary for Tim's use case. Furthermore, I believe they are actually harmful because they encourage sloppy module design. As Richard noted, once "friends" is introduced as a general mechanism there is no way to restrict its use to evolving APIs. It will inevitably be used to create module groups that have brittle, non-substitutable internal dependencies and are impossible to reuse. The desire to use "friends" within a group of modules under your control is a strong indicator that you have got the module boundaries wrong, and you should go back and refactor the module contents.

This problem cannot be dismissed by invoking social conventions. As with language design, adding a new feature to OSGi requires that its potential abuses are considered as well as its intended uses.

Incidentally the "x-friends" mechanism was mentioned as being supported by Equinox. It is, but for it to be enforced the framework must run in strict mode ("osgi.resolverMode=strict"), and almost nobody actually uses that mode. Eclipse SDK does not, for example. Most of the time, these flags just generate compilation warnings in PDE.

Regards
Neil
Back to top
Petr Nejedl
Posted via mailing list.





PostPosted: Mon Feb 01, 2010 9:53 am    Post subject: [platform-dev] Re: Re: Netbeans Friend package & OSGi guys :) Reply with quote

njbartlett wrote:
Quote:
Jesse's post describes a system of controls, so that the author of an API knows exactly who is using it (because they are *forced* to ask for permission to use it). The point of this is apparently to enable "push" notification to consumers when the API changes, but I don't believe this approach scales. It also seems inappropriate to encode development workflow practices into a runtime module system. There are other ways to communicate with users besides running a Find Usages query... e.g.an RSS feed. I lose track of the number of times I have been forced to use "internal" packages from Eclipse, particularly the JDT sub-project. As much as I would like Erich Gamma to phone me every time his team makes a change, I don't think that's really practical. When I use those packages, I get a pretty stern warning from Eclipse about it, so I know that I should probably import a tight version range and also run my tests carefully against each version. In other words, it's my responsib!
ili
Quote:
ty to track changes in the API.

And this is what "implementation dependency" is good for in the NetBeans
land.
With that, you see all the classes of the module you have such a
dependency on,
but you have to explicitly state which _build_ (not only version) of the
module
you depend on.

--
Nenik
Back to top
Daoud AbdelMonem Faleh
Posted via mailing list.





PostPosted: Tue Feb 02, 2010 8:23 am    Post subject: [platform-dev] Re: Re: Netbeans Friend package & OSGi guys :) Reply with quote

Clearly there is no consensus on this matter. Am still keening to hear
from Jaroslav Tulach as he has already worked on integrating Netbeans
module system and OSGi frameworks and the author of one of the most
successful books on API Design.

With Warm Regards,
Daoud AbdelMonem Faleh.
Back to top
Jaroslav Tulach
Posted via mailing list.





PostPosted: Tue Feb 02, 2010 10:23 am    Post subject: [platform-dev] Common ground & Netbeans Friend package & OSGi guys :) Reply with quote

Dne
Back to top
Display posts from previous:   
Post new topic   Reply to topic    NetBeans Forums -> NetBeans Platform Users All times are GMT
Goto page 1, 2  Next
Page 1 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB
By use of this website, you agree to the NetBeans Policies and Terms of Use. © 2012, Oracle Corporation and/or its affiliates. Sponsored by Oracle logo