| View previous topic :: View next topic |
| Author |
Message |
ae00019
Joined: 01 Aug 2011 Posts: 6
|
Posted: Mon Aug 01, 2011 12:28 am Post subject: Derby error |
|
|
I use the following code in my class to connect with derby db:
CachedRowSet rowSet = null;
Class.forName(driver);
rowSet = new CachedRowSetImpl();
rowSet.setUrl(url); // set database URL
rowSet.setUsername(uname); // set username
rowSet.setPassword(dpass); // set password
I was wondering why I get the following error:
No buffer space available (maximum connections reached?): connect.
Is there a better implementation than the above for avoiding that error?
Thank you
|
|
| Back to top |
|
 |
dingfelder
Joined: 06 Oct 2010 Posts: 112 Location: new zealand
|
Posted: Mon Aug 01, 2011 2:53 am Post subject: |
|
|
this link talks about your issue and a possible solution:
http://kb2.adobe.com/cps/400/kb400700.html
they are saying:
This error typically occurs when a database query is called and there are a large number of TCP connections in TIME_WAIT state -- the interval between when the connection is closed and when its release back into the pool for reuse. By default this interval is four (4) minutes. If there is a large number of connections on the server, then each connection can wait up to 4 minutes before their resources are released for reuse by other connections.
Solution
Decreasing the TIME_WAIT interval will allow TCP to release closed connections faster, thus increasing the number of available resources for open connections. The TIME_WAIT value is governed by the TCPTimedWaitDelay Windows Registry setting.
Now... not sure if you are on windows or if there is a better way to tweak the time wait via db settings but maybe it will get you started in the right direction?
|
|
| Back to top |
|
 |
mdeggers
Joined: 28 Jan 2009 Posts: 208
|
Posted: Mon Aug 01, 2011 4:44 pm Post subject: |
|
|
Are you cleaning up and closing your connections after doing your database
access?
In web applications using connection pools this error is often due to not:
1. Cleaning up your objects, even when there's an exception
2. Releasing connection back to the pool, even when there's an exception
A good discussion on this can be found here:
http://blog.christopherschultz.net/index.php/2009/03/16/properly-handling-pooled-jdbc-connections/
I wrote and posted a simple base class for non-transaction database
access (using a connection pool). Take a look at the following for an
example:
http://forums.netbeans.org/viewtopic.php?t=40604
Even if you're not writing a web application or using JNDI-based connection
pools, cleaning up properly is a good idea.
. . . . just my two cents.
/mde/
|
|
| Back to top |
|
 |
ae00019
Joined: 01 Aug 2011 Posts: 6
|
Posted: Mon Aug 01, 2011 7:17 pm Post subject: |
|
|
Thank you very much for your replies!!
@mdeggers: I am using CachedRowSetImpl, so i am not opening any db connection as you are describing it.
Every time i need to make a call to the database i use the following code:
Class.forName(driver);
rowSet = new CachedRowSetImpl();
rowSet.setUrl(url); // set database URL
rowSet.setUsername(uname); // set username
rowSet.setPassword(dpass); // set password
String sqlCmnd = "Select * from Table";
rowSet.setCommand(sqlCmnd);
rowSet.execute();
I do not really understand which piece of the above code opens a connection to the db, and therefore causing the maximum connections reached problem.
Should I close the rowSet everytime or set it to null??
Thank you!!!
|
|
| Back to top |
|
 |
mdeggers
Joined: 28 Jan 2009 Posts: 208
|
Posted: Mon Aug 01, 2011 11:03 pm Post subject: |
|
|
I've not used CachedRowSet before, but it looks sort of interesting.
According to the javadoc (which I recommend that you read), a
CachedRowSet object only makes a connection to the database while reading
or updating information. Otherwise it's just another object.
However, it looks like a lot of methods throw SQLExceptions, and you do
need to call close() in order to release the objects.
Taking the above into consideration, here's a quick (untested) hack of the
BaseDAO I posted in another thread. Clean it up, and then possibly extend
it to implement your actual queries and updates.
I've not included the imports, so this class will not compile as is. However
it should give you a base idea as to what is going on (provided I
understand what is going on).
| Code: | package some.package;
// import lots of stuff
public class BaseDAO {
// the CachedRowSet
protected CachedRowSetImpl crsi = null;
// logging here - pick your poison
private Log log = LogFactory.getLog(this.getClass());
// database stuff - maybe read from a file rather than hardcoded?
protected String dburl = "database url";
protected String dbuser = "database user";
protected String dbpass = "database password";
protected void getConnection() {
// create the object
try {
crsi = new CachedRowSet();
} catch (SQLException ex) {
log.fatal("Something really bad happened"
+ ex.toString()); // you do log . . .
cleanup();
}
// set stuff up
if (crsi != null) {
try {
csri.setUrl(dburl);
csri.setUsername(dbuser);
csri.setPassword(dbpass);
} catch (SQLException ex) {
log.fatal("Could not populate with database info"
+ ex.toString()); // logging is a good thing
cleanup();
}
}
}
protected void cleanup() {
if (csri != null) {
try {
csri.close();
} catch (SQLException ex) {
log.warn("Could not close cached result set"
+ ex.toString());
} finally {
csri = null; // make sure we can recover resources
}
}
}
}
|
Then extend the class with one that does the actual work. Details would
change according to how long you need to maintain the result set, but
when you're done, you need to clean up.
Again, I've not explored this, and the code above is pretty much stream
of consciousness writing.
However, it seems to make sense.
. . . . just my two cents.
/mde/
|
|
| Back to top |
|
 |
ae00019
Joined: 01 Aug 2011 Posts: 6
|
Posted: Tue Aug 02, 2011 12:40 am Post subject: |
|
|
Thank you for your help!!!
I do all that you have advised but i still get the same error
|
|
| Back to top |
|
 |
mdeggers
Joined: 28 Jan 2009 Posts: 208
|
Posted: Tue Aug 02, 2011 2:54 am Post subject: |
|
|
Maybe if you post your entire class you use to connect and query the
database?? It's hard to determine without more information. That's why
logging exceptions is so important.
You could also run it through the debugger and see exactly where it stops.
. . . . just my two cents.
/mde/
|
|
| Back to top |
|
 |
mdeggers
Joined: 28 Jan 2009 Posts: 208
|
Posted: Tue Aug 02, 2011 8:24 am Post subject: |
|
|
OK, I just hacked together a quick project and it works as expected.
It really doesn't use the interesting features of CachedRowSetImpl, but
at least it works.
As an exercise, you could modify the code in SalesRecordInfo.java
to return a CachedRowSetImpl object, and then make sure you clean up
(with an appropriate cleanup routine) when you're done with it.
I hope you find the code useful. Be sure to read the quick notes I wrote
in a notes directory, telling you a little bit about the database and the
environment this runs on.
Hope this helps.
. . . . just my two cents.
/mde/
| Description: |
| Quick project for CachedRowSetImpl. It does not make good use of this class, but it does work. |
|
 Download |
| Filename: |
crs.zip |
| Filesize: |
908.22 KB |
| Downloaded: |
96 Time(s) |
|
|
| Back to top |
|
 |
ae00019
Joined: 01 Aug 2011 Posts: 6
|
Posted: Tue Aug 02, 2011 11:55 pm Post subject: |
|
|
Still the same problem.
Is there a way to monitor the open database connections in Netbeans 7.0?
Thank you!!
|
|
| Back to top |
|
 |
mdeggers
Joined: 28 Jan 2009 Posts: 208
|
Posted: Wed Aug 03, 2011 1:05 am Post subject: |
|
|
If you have the JConsole plugin installed, you can launch JConsole and
take a look at the MBeans.
JConsole is usually at netbeans_jdkhome/bin/jconsole.
I've attached some png flies to show you how it looks on my system.
Fedora 15
Gnome 3
JDK 1.6.0_26
NetBeans 7.0.1
1. Start up JConsole
2. Attach to the correct process
3. Click on the MBeans tab
4. Expand the org.apache.derby node
5. Expand the NetworkServer node
6. Expand the serialized name below
7. Click on the Attributes node to see the values
8. Periodically click on the the Refresh button to update the information
I've attached some screen shots.
netbeans-derby.png shows NetBeans connected to one of several Derby
databases.
jconsole-derby-02.png shows you the default look when you attach
JConsole to a Derby server (or any process, actually).
jconsole-derby-05.png shows you browsing the attributes under the
Derby server, with all of the interesting information available.
I hope this helps you debug your connections. The project I posted in my
previous response works cleanly on my system.
The first query: asr = sri.getAllRecords();
returns 194 rows.
The second query: nsr = sri.getNameRecord("Rice Ice");
returns 22 rows.
. . . . just my two cents.
/mde/
| Description: |
| Looking at the MBeans under the MBean tab |
|
| Filesize: |
97.76 KB |
| Viewed: |
4246 Time(s) |

|
| Description: |
| Attached to Derby, default view |
|
| Filesize: |
65.45 KB |
| Viewed: |
4246 Time(s) |

|
| Description: |
| NetBeans connected to a Derby database |
|
| Filesize: |
219.87 KB |
| Viewed: |
4246 Time(s) |

|
|
|
| Back to top |
|
 |
jerry0503222
Joined: 01 Aug 2011 Posts: 14
|
Posted: Wed Aug 03, 2011 8:01 am Post subject: |
|
|
Are you cleaning up and closing your connections after doing your database
access?
In web applications using connection pools this error is often due to not:
1. Cleaning up your objects, even when there's an exception
2. Releasing connection back to the pool, even when there's an exception
____________________________________________
microsoft office professional 2010,outlook 2010
|
|
| Back to top |
|
 |
ae00019
Joined: 01 Aug 2011 Posts: 6
|
Posted: Thu Aug 04, 2011 2:01 am Post subject: |
|
|
Thank you so much for your help mdeggers!!
I still get the same error despite the fact that maximum 3 connections are open at a time!!
I have attached the screenshot from the JConsole preview.
I have noticed that AccumulatedConnectionCount is really big, is that ok?
THANK YOU!!!!
| Description: |
|
| Filesize: |
208.45 KB |
| Viewed: |
4204 Time(s) |

|
|
|
| Back to top |
|
 |
ae00019
Joined: 01 Aug 2011 Posts: 6
|
Posted: Thu Aug 04, 2011 12:22 pm Post subject: |
|
|
Could anybody pls explain why AccumulatedConnectionCount (pls refer to previous post) it gets so big? And if this is related to the error :
java.sql.SQLTransientConnectionException: java.net.SocketException : Error connecting to server localhost on port 1527 with message No buffer space available (maximum connections reached?): connect.
Thank you!!!
|
|
| Back to top |
|
 |
mdeggers
Joined: 28 Jan 2009 Posts: 208
|
Posted: Thu Aug 04, 2011 3:48 pm Post subject: |
|
|
Sorry for the delay - life gets in the way sometimes :-p.
Overview
The number of connections could very well be a problem, depending on
how fast you generated them.
The reason behind this problem is how TCP connections operate. I'll
roughly summarize RFC 793.
1. A connection is made using a three way handshake
2. Data is sent
3. A connection is closed using a full duplex handshake
This last step is important. The side initiating the closing (your
Derby server) ends up in a final TCP_WAIT state. This is done at the
OS / Networking layer to make sure all of the data has been reliably
delivered.
By default, the TCP_WAIT state lasts for 240 seconds (2 times the
maximum segment life). While the socket is in a TCP_WAIT state, it
cannot be reused by the OS (obviously).
If you make and break lots of connections in a short amount of time,
you can run out of socket resources.
To check that, you can download TCPView for Windows at:
http://technet.microsoft.com/en-us/sysinternals/bb897437
If you see a lot of connections in TIME_WAIT when running your program
(or shortly after), you have a problem.
There are several solutions to this problem, depending on your use
case. I'll outline 4 common ones below.
Use Cases
1. Use Case 1 - Web application with lots of users
This is probably the most common use case. Your program is part of a
web application, and lots of people make lots of database requests.
Fortunately, there is a relatively standard solution to this
problem. It's called database connection pooling.
Application servers (such as Glassfish) and servlet containers (such
as Tomcat), provide a way to create pools of database
connections. The server controls the number of connections, can grow
the pool if needed, shrink the pool if needed, and makes sure that the
connections are valid.
To the database, the client (your application running in a server) has
a small number of more or less permanent connections. To your
application's users, it looks like you have a virtually unlimited
number of connections.
Configuration is server-specific, but there is good documentation on
how to do this for all major servers.
Your application usually (at least on Glassfish and Tomcat) accesses
the connection pool via JNDI. To your application, it looks just
exactly like a standard JDBC connection. You should close and clean up
the connection in exactly the same manner.
Here's one example on how to do this:
http://blog.christopherschultz.net/index.php/2009/03/16/properly-handling-pooled-jdbc-connections/
Here's a connection method that can be placed inside a base class:
| Code: | // instance variables, so you can write the cleanup routine
protected Connection conn = null;
protected ResultSet rs = null;
protected Statement stmt = null;
protected PreparedStatement ps = null;
protected void getConnection() {
DataSource ds = null;
try {
Context icontext = new InitialContext();
Context envCtx = (Context) icontext.lookup("java:comp/env");
// dsname is a string with the JNDI name
ds = (DataSource) envCtx.lookup(dsname);
} catch (NamingException ex) {
log.fatal("Failed to get JNDI name: " + dsname);
log.fatal(ex);
cleanup();
}
if (ds != null) {
try {
conn = ds.getConnection();
} catch (SQLException ex) {
log.fatal("Failed to get database connection");
log.fatal(ex);
cleanup();
}
}
}
|
Using connection pooling, a relatively small and permanent number of
database connections can serve a large number of web clients. You can
monitor your active connections via JMX to make sure the connection
pool is sized appropriately.
2. Use Case 2 - Few clients making many, rapid requests
This one is a bit more challenging, but again the solution is to use
connection pooling. Unfortunately, since you don't have a server to
manage the connection pool yourself, you'll have to create your own.
One implementation of connection pooling that's available is Apache's
DBCP (database connection pool). Its home page is:
http://commons.apache.org/dbcp/
I've not used it in a client, but there are examples. One such example
is:
http://www.freshblurbs.com/jakarta-commons-dbcp-tutorial
Basically, a connection pool is created during application start up,
and then you use this to get database connections. Finally, at
application shutdown, you need to close all of the connections.
This will reduce the number of connections that have to be made. It
will reduce the resource starvation and improve performance.
3. Use Case 3 - Many clients making few requests
If this is your use case, then the socket problem is an artifact of
testing. Try not to make so many requests in such a short amount of
time. However, you may still benefit from using a connection pool as
above.
4. Use Case 4 - Many clients making many requests
This one is the most difficult to scale. You have several approaches,
most of which are beyond the scope of this forum.
A. Clustering your database
As far as I know, Derby does not have clustering support. You can use
a third party piece of software to implement a Derby database
cluster. One link containing references to that is:
http://www.oracle.com/technetwork/java/javadb/documentation/index.html
B. Shard your database
I haven't seen references to sharding a database across several Derby
servers. There was a mention of it on the Derby mailing list, but an
initial search did not reveal much.
C. Move to a NOSQL database
If your data is document-based, you may benefit from using a NOSQL
database such as:
MongoDB
CouchDB
Hadoop MapReduce
NOSQL databases involve a fundamental change in architecture, so you
should examine your use case before jumping in.
Using CachedRowSetImpl with Connection Pools
Use Case 1
It looks like you can do this fairly easily (at least with the JNDI
pools). Sun (now Oracle) has a tutorial here:
http://java.sun.com/developer/Books/JDBCTutorial/chapter5.html
In some containers you might have to use the full JNDI name
(java:comp/env/jdbc/name).
From a brief look at the tutorial, it looks like once the execute()
method is fired, the CachedRowSetImpl object closes the
connection. This would release it back to the pool.
Once you are done, you'll need to clean up the CachedRowSetImpl object
as before.
Use Case 2
From the javadoc, it also looks like you can also use a
java.sql.Connection object. So you could use setCommand("sql
statement"), and then use execute(java.sql.Connection conn) to use a
connection from the connection pool.
Once you're done, you'll have to clean up both CachedRowSetImpl as
before, as well as the underlying Connection.
Sizing
Getting the right sizing for a connection pool is mostly trial and
error. From your other post, it sounds like a minPoolSize of 3 and a
maxPoolSize of 5 (it automatically grows and shrinks, see the
documentation) might work well.
. . . just my two cents.
/mde/
|
|
| Back to top |
|
 |
Bob McConnell Posted via mailing list.
|
Posted: Thu Aug 04, 2011 4:15 pm Post subject: Derby error |
|
|
From: mdeggers
| Quote: | The number of connections could very well be a problem, depending on
how fast you generated them.
|
| Quote: | To check that, you can download TCPView for Windows at:
|
No need to go to all that trouble. Simply open a command window and run
'netstat -an'. That will show you all the sockets in use and their
current state. The 'n' tells it not to bother with reverse DNS lookups
but show the numeric addresses.
Bob McConnell
|
|
| Back to top |
|
 |
|