This patch does the following:
- Adds binary datatype support (bytea)
- Changes getXXXStream()/setXXXStream() methods to be spec compliant
- Adds ability to revert to old behavior
Details:
Adds support for the binary type bytea. The ResultSet.getBytes() and
PreparedStatement.setBytes() methods now work against columns of bytea
type. This is a change in behavior from the previous code which assumed
the column type was OID and thus a LargeObject. The new behavior is
more complient with the JDBC spec as BLOB/CLOB are to be used for
LargeObjects and the getBytes()/setBytes() methods are for the databases
binary datatype (which is bytea in postgres).
Changes the behavior of the getBinaryStream(), getAsciiStream(),
getCharacterStream(), getUnicodeStream() and their setXXXStream()
counterparts. These methos now work against either the bytea type
(BinaryStream) or the text types (AsciiStream, CharacterStream,
UnicodeStream). The previous behavior was that these all assumed the
underlying column was of type OID and thus a LargeObject. The
spec/javadoc for these methods indicate that they are for LONGVARCHAR
and LONGVARBINARY datatypes, which are distinct from the BLOB/CLOB
datatypes. Given that the bytea and text types support upto 1G, they
are the LONGVARBINARY and LONGVARCHAR datatypes in postgres.
Added support for turning off the above new functionality. Given that
the changes above are not backwardly compatible (however they are more
spec complient), I added the ability to revert back to the old behavior.
The Connection now takes an optional parameter named 'compatible'. If
the value of '7.1' is passed, the driver reverts to the 7.1 behavior.
If the parameter is not passed or the value '7.2' is passed the behavior
is the new behavior. The mechanism put in place can be used in the
future when/if similar needs arise to change behavior. This is
patterned after how Oracle does this (i.e. Oracle has a 'compatible'
parameter that behaves in a similar manner).
Misc fixes. Cleaned up a few things I encountered along the way.
Note that in testing the patch I needed to ignore whitespace differences
in order to get it to apply cleanly (i.e. patch -l -i byteapatch.diff).
Also this patch introduces a new file
(src/interfaces/jdbc/org/postgresql/util/PGbytea.java).
Barry Lind
>there is still an unpatched reference to pg_description in
>getColumns(), in both jdbc1 and jdbc2.
This was introduced by Jeroen's patch (see
http://fts.postgresql.org/db/mw/msg.html?mid=1032468). Attached
is a patch that returns getColumns() to using "select
obj_description()" instead of direct access to pg_description,
as per the request by Tom.
I've incorporated Jeroen's fix to left outer join with
pg_attrdef instead of inner join, so getColumns() also returns
columns without a default value.
I have, however, not included Jeroen's attempt to combine
multiple queries into one huge multi-join query for better
performance, because:
1) I don't know how to do that using obj_description() instead
of direct access to pg_description
2) I don't think a performance improvement (if any) in this
method is very important
Because of the outer join, getColumns() will only work with a
backend >= 7.1. Since the conditional coding for 7.1/7.2 and
jdbc1/jdbc2 is already giving me headaches I didn't pursue a
pre-7.1 solution.
Regards,
Ren? Pijlman <rene@lab.applinet.nl>
ConnectionTest.testTransactionIsolation() in the JDBC driver's
test suite. This reduces the number of failures of the test
suite from 7 to 6. The patch fixes the test case itself, rather
than the driver.
In addition to the change described in my posting below, I fixed
the part of the test with autocommit enabled. The author of the
test assumed that setting the transaction isolation level would
have no effect, but in fact it does. Perhaps the test case
worked with pre-7.1 behaviour, when the JDBC driver set the
isolation level in every transaction, instead of using "set
session characteristics". Anyway, now it works with a backend
built from current CVS and the behaviour is JDBC compliant.
I also extended the test case by changing the isolation level
before beginning a transaction and verifying it inside the
transaction.
Regards,
Ren? Pijlman
suite. This reduces the number of failures from 9 to 7.
Both ConnectionTest and JBuilderTest did not create their own
tables, which caused these test cases to fail with "relation ...
does not exist". It appears these test cases relied on tables
created by the example code elsewhere in the source tree. I've
added the necessary "create table" and "drop table" statements
to the test cases, using the column definitions from the example
code.
While working on that I modified the helper method createTable
in JDBC2Tests.java to take a table parameter, rather than using
table names passed via the properties in build.xml. I'm not sure
what that was good for, and in fact, except for the default
table name "jdbctest", this functionality wasn't used at all.
Ren? Pijlman
>
>> On Mon, 3 Sep 2001 22:01:17 -0500, you wrote:
>> public boolean isWritable(int column) throws SQLException
>> {
>> return !isReadOnly(column);
>> }
Actually, I think this change has a consequence for this method
in the same class:
public boolean isDefinitelyWritable(int column)
throws SQLException
{
return isWritable(column);
}
This is from the JDBC spec
(http://java.sun.com/j2se/1.3/docs/api/java/sql/ResultSetMetaData.html):
isReadOnly() - Indicates whether the designated column is
definitely not writable.
isWritable() - Indicates whether it is possible for a write on
the designated column to succeed.
isDefinitelyWritable() - Indicates whether a write on the
designated column will definitely succeed.
At this time we don't really implement the fine semantics of
these methods. I would suggest the following defaults:
isReadOnly() false
isWritable() true
isDefinitelyWritable() false
And that would mean that your patch is correct, but
isDefinitelyWritable() would need to be patched accordingly:
public boolean isDefinitelyWritable(int column)
throws SQLException
{
return false;
}
Again, both in jdbc1 and jdbc2.
Regards,
Ren? Pijlman <rene@lab.applinet.nl>
>public boolean isWritable(int column) throws SQLException
>{
> if (isReadOnly(column))
> return true;
> else
> return false;
>}
The author probably intended:
public boolean isWritable(int column) throws SQLException
{
return !isReadOnly(column);
}
And if he would have coded it this way he wouldn't have made
this mistake :-)
>hence, isWritable() will always return false. this is something
>of a problem :)
Why exactly? In a way, true is just as incorrect as false, and
perhaps it should throw "not implemented". But I guess that
would be too non-backwardly-compatible.
>let me know if i can provide further information.
Will you submit a patch?
Regards,
Ren? Pijlman <rene@lab.applinet.nl>
flawed in the following ways:
1. Only returned columns that had a default value defined, rather than all
columns in a table
2. Used 2 * N + 1 queries to find out attributes, comments and typenames
for N columns.
By using some outer join syntax it is possible to retrieve all necessary
information in just one SQL statement. This means this version is only
suitable for PostgreSQL >= 7.1. Don't know whether that's a problem.
I've tested this function with current sources and 7.1.3 and patched both
jdbc1 and jdbc2. I haven't compiled nor tested the jdbc1 version though, as
I have no JDK 1.1 available.
Note the discussion in http://fts.postgresql.org/db/mw/msg.html?mid=1029626
regarding differences in obtaining comments on database object in 7.1 and
7.2. I was unable to use the following syntax (or similar ones):
select
...,
description
from
...
left outer join col_description(a.attrelid, a.attnum) description
order by
c.relname, a.attnum;
(the error was parse error at or near '(') so I had to paste the actual
code for the col_description function into the left outer join. Maybe
someone who is more knowledgable about outer joins might provide me with a
better SQL statement.
Jeroen van Vianen
the JDBC driver.
I've done this by extracting it into a new method object called
QueryExecutor (should go into org/postgresql/core/) and then taking it
apart into different methods in that class.
A short summary:
* Extracted ExecSQL() from Connection into a method object called
QueryExecutor.
* Moved ReceiveFields() from Connection to QueryExecutor.
* Extracted parts of the original ExecSQL() method body into smaller
methods on QueryExecutor.
* Bug fix: The instance variable "pid" in Connection was used in two
places with different meaning. Both were probably in dead code, but it's
fixed anyway.
Anders Bengtsson
for the changed files and a few new files:
- test/jdbc2/BatchExecuteTest.java
- util/MessageTranslator.java
- jdbc2/PBatchUpdateException.java
As an aside, is this the best way to submit a patch consisting
of both changed and new files? Or is there a smarter cvs command
which gets them all in one patch file?
This patch fixes batch processing in the JDBC driver to be
JDBC-2 compliant. Specifically, the changes introduced by this
patch are:
1) Statement.executeBatch() no longer commits or rolls back a
transaction, as this is not prescribed by the JDBC spec. Its up
to the application to disable autocommit and to commit or
rollback the transaction. Where JDBC talks about "executing the
statements as a unit", it means executing the statements in one
round trip to the backend for better performance, it does not
mean executing the statements in a transaction.
2) Statement.executeBatch() now throws a BatchUpdateException()
as required by the JDBC spec. The significance of this is that
the receiver of the exception gets the updateCounts of the
commands that succeeded before the error occurred. In order for
the messages to be translatable, java.sql.BatchUpdateException
is extended by org.postgresql.jdbc2.PBatchUpdateException() and
the localization code is factored out from
org.postgresql.util.PSQLException to a separate singleton class
org.postgresql.util.MessageTranslator.
3) When there is no batch or there are 0 statements in the batch
when Statement.executeBatch() is called, do not throw an
SQLException, but silently do nothing and return an update count
array of length 0. The JDBC spec says "Throws an SQLException if
the driver does not support batch statements", which is clearly
not the case. See testExecuteEmptyBatch() in
BatchExecuteTest.java for an example. The message
postgresql.stat.batch.empty is removed from the language
specific properties files.
4) When Statement.executeBatch() is performed, reset the
statement's list of batch commands to empty. The JDBC spec isn't
100% clear about this. This behaviour is only documented in the
Java tutorial
(http://java.sun.com/docs/books/tutorial/jdbc/jdbc2dot0/batchupdates.html).
Note that the Oracle JDBC driver also resets the statement's
list in executeBatch(), and this seems the most reasonable
interpretation.
5) A new test case is added to the JDBC test suite which tests
various aspects of batch processing. See the new file
BatchExecuteTest.java.
Regards,
Ren? Pijlman
system. Some systems did not understand the 'l' section, and in general
it wasn't entirely appropriate.
On SCO OpenServer, the man pages won't be installed at all until someone
figures out their man system.
longer compiles, due to objects being referenced in this patch that do
not exist in JDK1.1.
Barry Lind
---------------------------------------------------------------------------
The JDBC driver requires
permission java.net.SocketPermission "host:port", "connect";
in the policy file of the application using the JDBC driver
in the postgresql.jar file. Since the Socket() call in the
driver is not protected by AccessController.doPrivileged() this
permission must also be granted to the entire application.
>>>>
>>>> permission java.net.SocketPermission "host:port", "connect";
>>>>
>>>>in the policy file of the application using the JDBC driver
>>>>in the postgresql.jar file. Since the Socket() call in the
>>>>driver is not protected by AccessController.doPrivileged() this
>>>>permission must also be granted to the entire application.
>>>>
>>>>The attached diff fixes it so that the connect permission can be
>>>>restricted just the the postgresql.jar codeBase if desired.
David Daney
org.postgresql.util.Serialize and org.postgresql.jdbc2.PreparedStatement
that fixes the ability to "serialize" a simple java class into a
postgres table.
The current cvs seems completely broken in this support, so the patch
puts it into working condition, granted that there are many limitations
with serializing java classes into Postgres.
The code to do serialize appears to have been in the driver since
Postgres 6.4, according to some comments in the source. My code is not
adding any totally new ability to the driver, rather just fixing what
is there so that it actually is usable. I do not think that it should
affect any existing functions of the driver that people regularly
depend on.
The code is activated if you use jdbc2.PreparedStatement and try to
setObject some java class type that is unrecognized, like not String or
not some other primitive type. This will cause a sequence of function
calls that results in an instance of Serialize being instantiated for
the class type passed. The Serialize constructor will query pg_class
to see if it can find an existing table that matches the name of the
java class. If found, it will continue and try to use the table to
store the object, otherwise an SQL exception is thrown and no harm is
done. Serialize.create() has to be used to setup the table for a java
class before anything can really happen with this code other than an
SQLException (unless by some freak chance a table exists that it thinks
it can use).
I saw a difference in Serialize.java between 7.1.3 and 7.2devel that I
didn't notice before, so I had to redo my changes from the 7.2devel
version (why I had to resend this patch now). I was missing the
fixString stuff, which is nice and is imporant to ensure the inserts
will not fail due to embedded single quote or unescaped backslashes. I
changed that fixString function in Serialize just a little since there
is no need to muddle with escaping newlines: only escaping single quote
and literal backslashes is needed. Postgres appears to insert newlines
within strings without trouble.
This patch moves the logic that looks up TypeOid, PGTypeName, and
SQLTypeName from Field to Connection. It is moved to connection since
it needs to differ from the jdbc1 to jdbc2 versions and Connection
already has different subclasses for the two driver versions. It also
made sense to move the logic to Connection as some of the logic was
already there anyway.
Barry Lind
following email.
> > The problem: When I call getBigDecimal() on a ResultSet, it
> > sometimes throws an exception:
> >
> > Bad BigDecimal 174.50
> > at org.postgresql.jdbc2.ResultSet.getBigDecimal(ResultSet.java:373)
> > at org.postgresql.jdbc2.ResultSet.getBigDecimal(ResultSet.java:984)
> > ...blah blah blah...
> > org.postgresql.util.PSQLException: Bad BigDecimal 174.50
Barry Lind
so it may be a transit problem. Also removed the 'txt' suffix
in case that was confusing some transport layer trying to be
too inteligent for our own good.
This may have been because the Array.java class from the
previous patch didn't seem to have made it into the snapshot
build for some reason. This patch should at least fix that issue.
Greg Zoller
> Shouldn't
>
> throw new PSQLException("metadata unavailable");
>
> in getTypeInfo() be something like:
>
> throw new PSQLException("postgresql.meta.unavailable");
>
> to allow translation of the error message in the
> errors*.properties files?
You're right. Attached is an updated patch that also includes a message
in error.properties. I've attempted a French message in
errors_fr.properties but beware that I haven't written French in quite a
few years. Don't know Italian, German, or Dutch so I can't do those.
Liam Stewart
attempt at a patch to 7.1.2 to support Array.
[I think I've solved the mangled patch problem. Hotmail seems to
try to format the text file, so gzipping it should solve this
problem.]
In this patch I've incorporated Barry's feedback. Specifically:
1) OIDs are no longer hard-coded into Array.java. In order to
support this change I added a getOID(String) method to Field.java
which receives a PostgreSQL field type and returns a value from
java.sql.Types. I couldn't get away from using OIDs altogether
because the JDBC spec for Array specifies that some methods return
a ResultSet. This requires I construct Field objects,
which means I need OIDs. At least this approach doesn't hard
code these values. A Hashtable cache has been added to Field
so that an SQL lookup isn't necessary (following the model already
in Field.java).
2) Rewired the base formatting code in ResultSet.java to use 'to'
methods, which are then exposed as static methods in ResultSet.
These methods are used in Array to format the data without
duplications in the code.
3) Artifact call to first() in ResultSet.getArray() removed.
Greg Zoller
includes two changes in the JDBC driver:
1) When connected to a backend >= 7.2: use obj_description() and
col_description() instead of direct access to pg_description.
2) In DatabaseMetaData.getTables()/getColumns()/getProcedures():
when there is no comment on the object, return null in the
REMARKS column of the ResultSet, instead of the default string
"no remarks".
Change 2 first appeared as a side-effect of change 1, but it is
actually more compliant with the JDBC spec: "String object
containing an explanatory comment on the table/column/procedure,
which may be null". The default string "no remarks" was strictly
speaking incorrect, as it could not be distinguished from a real
user comment "no remarks". So I removed the default string
completely.
Change 2 might break existing code that doesn't follow the JDBC
spec and isn't prepared to handle a null in the REMARKS column
of getTables()/getColumns()/getProcedures.
Patch tested with jdbc2 against both a 7.1 and a CVS tip
backend. I did not have a jdbc1 environment to build and test
with, but since the touched code is identical in jdbc1 and jdbc2
I don't foresee any problems.
Regards,
Ren? Pijlman
* Merges identical code from org.postgresql.jdbc[1|2].Statement into
org.postgresql.Statement.
* Moves escapeSQL() method from Connection to Statement (the only place
it's used)
* Minor cleanup of the new isolation level stuff.
* Minor cleanup of version string handling.
Anders Bengtsson
Here is a context diff from latest cvs
And I see why you couldn't apply the last diff, the setCatalog diff has
been backed out, that was causing the compile problem in the first
place.
This following one needs to be applied to allow the current cvs to
compile
Dave Cramer
1) improves performance of commit/rollback by reducing number of round
trips to the server
2) uses 7.1 functionality for setting the transaction isolation level
3) backs out a patch from 11 days ago because that code failed to
compile under jdk1.1
Details:
1) The old code was doing the following for each commit:
commit
begin
set transaction isolation level xxx
thus a call to commit was performing three round trips to the database.
The new code does this in one round trip as:
commit; begin; set transaction isolation level xxx
In a simple test program that performs 1000 transactions (where each
transaction does one simple select inside that transaction) has the
following before and after timings:
Client and Server on same machine
old new
--- ---
1.877sec 1.405sec 25.1% improvement
Client and Server on different machines
old new
--- ---
4.184sec 2.927sec 34.3% improvement
(all timings are an average of four different runs)
2) The driver was using 'set transaction isolation level xxx' at the
begining of each transaction, instead of using the new 7.1 syntax of
'set session characteristics as transaction isolation level xxx' which
only needs to be done once instead of for each transaction. This is
done conditionally (i.e. if server is 7.0 or older do the old behaviour,
else do the new behaviour) to not break backward compatibility. This
also required the movement of some code to check/test database version
numbers from the DatabaseMetaData object to the Connection object.
3) Finally while testing, I discovered that the code that was checked in
11 days ago actually didn't compile. The code in the patch for
Connection.setCatalog() used Properties.setProperty() which only exists
in JDK1.2 or higher. Thus compiling the JDBC1 driver failed as this
method doesn't exist. Thus I backed out that patch.
Barry Lind
connection implementations (org.postgresql.jdbc[1|2].Connection) into
their superclass (org.postgresql.Connection).
It also changes the close() methods of Connection and PG_Stream, so that
PG_Stream no longer is responsible for sending the termination packet 'X'
to the backend. I figured that protocol-level stuff like that belonged in
Connection more than in PG_Stream.
Anders Bengtsson
in Connection - note: I've updated setCatalog(String catalog) from my previous
diff so it checks whether it is already connected to the specified catalog.
Jason Davies
Here's a patch against the current CVS. The changes from the previous
patch are mostly related to the changed interface for PG_Stream.
Anders Bengtsson
null terminated strings. The FE/BE protocol sends in some cases null
terminated strings to the client. The docs for the FE/BE protocol state
that there is no limit on the size of a null terminated string sent to
the client and a client should be coded using an expanding buffer to
deal with large strings. The old code did not do this and gave an error
if a null terminated string was greater than either 4 or 8K. It appears
that with the advent of TOAST very long SQL statements are becoming more
common, and apparently some error messages from the backend include the
SQL statement thus easily exceeding the 8K limit in the old code.
In fixing I also cleaned up some calls in the JDBC fastpath code that
were not doing character set conversion under multibyte, and removed
some methods that were no longer needed. I also removed a potential
threading problem with a shared variable that was being used in
Connection.java.
Thanks to Steve Wampler for discovering the problem and sending the
initial diffs that were the basis of this patch.
thanks,
--Barry
* NULLs are sorted differently in 7.2
* table correlation names are supported
* GROUP BY, ORDER BY unrelated is supported since 6.4
* ESCAPE/LIKE only supported since 7.1
* outer joins only since 7.1
* preferred term for procedure is "function"
* preferred term for catalog is "database"
* supports SELECT for UPDATE since 6.5
* supports subqueries
* supports UNION; supports UNION ALL since 7.1
* update some of the max lengths to match reality
* rearrange some functions to match the order in the spec
for easier maintenance
redirections between the build files, which didn't work completely. Now
you just go to the directory of your choice and run make. Clean up the
build files to have a logical order, fix the unnecessary rebuilds, prevent
the deleting targets from removing files they're not responsible for. Ant
1.3 does not have a bug. It deletes directories just fine if you follow
the documentation.
