That patch broke the ability to read data from binary cursors.
--Barry Lind
Modified Files:
pgsql/src/interfaces/jdbc/org/postgresql/Connection.java
pgsql/src/interfaces/jdbc/org/postgresql/ResultSet.java
pgsql/src/interfaces/jdbc/org/postgresql/core/QueryExecutor.java
pgsql/src/interfaces/jdbc/org/postgresql/jdbc1/Connection.java
pgsql/src/interfaces/jdbc/org/postgresql/jdbc1/ResultSet.java
pgsql/src/interfaces/jdbc/org/postgresql/jdbc2/Connection.java
pgsql/src/interfaces/jdbc/org/postgresql/jdbc2/ResultSet.java
pgsql/src/interfaces/jdbc/org/postgresql/jdbc2/UpdateableResultSet.java
select 'id' as xxx from table
The issue is:
When the driver gets a data type which does not map into the SQL.Types
it attempts to load the object into a java object. Eventually throwing
an exception indicating that the type "unknown" was not found.
Since the backend defaults "unknown" types to text it was suggested that
the jdbc driver do the same.
This patch does just that.
I have tested it on the above select statement as well as a small
program that serializes, and deserializes a class
Dave Cramer
DatabaseMetaData.getColumn(). I proposed a patch that would change the
number of queries to find out all columns in a table from 2 * N + 1 to 1 (N
being the number of columns reported) by using some outer joins. I also
fixed the fact that getColumns() only returned columns that had a default
defined. OTOH, I did not use to change the code required for obtaining a
column's remarks (by using col_description() for 7.2 and requested by Tom
Lane).
Finally, I have found a way to get all the column details in a single query
*and* use col_description() for 7.2 servers. A patch is attached. It
overrules Ren? Pijlman's fix for this that was committed just today, but
still used N + 1 queries (sorry Ren? ;-) )
I also fixed the return values for TABLE_CAT and TABLE_SCHEM from "" to
null, to be more standard compliant (and requested in Ren?'s mail found at
http://fts.postgresql.org/db/mw/msg.html?mid=1034253).
As always, the JDBC1 version has not been tested as I have no JDK 1.1
Jeroen van Vianen
summary of changes:
. removal of the tablename property from build.xml
. addition of a dropTable method in JDBC2Tests and cleanups of many
methods in the same
. all tests now use non-deprecated assertXYZ methods instead of the
deprecated assert method
. failure in TimestampTest (testSetTimestamp) fixed. The failure is
because testSetTimestamp was inserting a timestamp with hour 7 but
checkTimeTest was expecting a timestamp with hour 8. AFAICS, there are
no issues wrt daylight savings time and timestamps being pushed in and
pulled out (but more explicit tests should be added in the future)
. failure in TimeTest (testGetTime) fixed. Times to be inserted were
interpreted in the localtime zone but checking was done with the
assumption that the insertion was done in GMT.
. formatting changes in a few of the source files (because I found
it convenient to have consistent formatting while working on them). The
formatting is consistent with the new format for java source files in
PostgreSQL.
Liam Stewart
the JDBC driver.
This method is currently unimplemented and always returns
ResultSetMetaData.columnNullable. This is obviously incorrect
when a column is defined with NOT NULL or PRIMARY KEY. And we
have to think of check constraints, views, functions etc.
The patch simply changes the return value to
ResultSetMetaData.columnNullableUnknown. This is until someone
comes up with a real implementation of course.
On Fri, 14 Sep 2001 17:53:50 +0200, Tomisaw Kity?ski wrote:
>Hello there,
>
>could someone tell me, please, do I have any chance to get
>proper implementation of above method in JDBC (1.1+) soon?
>
>Current "return 1" works fine on most tables, however it seems
>to be a little bit incorrect with some of them ;)
Ren? Pijlman
by escape processing in the SQL statement. I've tested this for a
while now and it appears to work well. Previously string data
with {d was getting corrupt as the {d was being stripped regardless
of whether it was an escape code or not.
I also added checking for time and timestamp escape processing strings
as per 11.3 in the specification. The patch is against the latest
CVS.
Thomas O'Dowd
> null bytes to be literally '\0', the following can happen:
> 1. User inputs string value as "<null byte>##" where ## are digits in the
> range of 0 to 7.
> 2. PQescapeString converts this to "\0##"
> 3. Escaped string is used in a context that causes "\0##" to be evaluated as
> an octal escape sequence.
I agree that this is a problem, though it is not possible to do
anything harmful with it. In addition, it only occurs if there are
any NUL characters in its input, which is very unlikely if you are
using C strings.
The patch below addresses the issue by removing escaping of \0
characters entirely.
> If the goal is to "safely" encode null bytes, and preserve the rest of the
> string as it was entered, I think the null bytes should be escaped as \\000
> (note that if you simply use \000 the same string truncation problem
> occurs).
We can't do that, this would require 4n + 1 bytes of storage for the
result, breaking the interface.
Florian Weimer
driver's test suite. With previous patches applied, this reduces
the number of failures of the test suite from 6 to 4. The patch
fixes the test case itself, rather than the driver.
Details:
1) The driver correctly provided DatabaseMetaData about the sort
order of NULLs. This was confirmed by Peter Eisentraut on
pgsql-hackers. I fixed the test to accept/require the current
behaviour, and made it dependent on the backend version. See
nullsAreSortedAtStart(), nullsAreSortedAtEnd(),
nullsAreSortedHigh() and nullsAreSortedLow().
2) DatabaseMetaData.supportsOrderByUnrelated() correctly
returned true (an ORDER BY clause can contain columns that are
not in the SELECT clause), but the test case required false.
Fixed that.
3) Replaced deprecated assert() of junit.framework.TestCase by
assertEquals(), assertTrue() and assertNotNull(). This is
because assert will be a new keyword in Java 1.4.
4) Replaced assert(message,false) by the more elegant
fail(message).
Regards,
Ren? Pijlman <rene@lab.applinet.nl>
This patch does the following:
- Adds binary datatype support (bytea)
- Changes getXXXStream()/setXXXStream() methods to be spec compliant
- Adds ability to revert to old behavior
Details:
Adds support for the binary type bytea. The ResultSet.getBytes() and
PreparedStatement.setBytes() methods now work against columns of bytea
type. This is a change in behavior from the previous code which assumed
the column type was OID and thus a LargeObject. The new behavior is
more complient with the JDBC spec as BLOB/CLOB are to be used for
LargeObjects and the getBytes()/setBytes() methods are for the databases
binary datatype (which is bytea in postgres).
Changes the behavior of the getBinaryStream(), getAsciiStream(),
getCharacterStream(), getUnicodeStream() and their setXXXStream()
counterparts. These methos now work against either the bytea type
(BinaryStream) or the text types (AsciiStream, CharacterStream,
UnicodeStream). The previous behavior was that these all assumed the
underlying column was of type OID and thus a LargeObject. The
spec/javadoc for these methods indicate that they are for LONGVARCHAR
and LONGVARBINARY datatypes, which are distinct from the BLOB/CLOB
datatypes. Given that the bytea and text types support upto 1G, they
are the LONGVARBINARY and LONGVARCHAR datatypes in postgres.
Added support for turning off the above new functionality. Given that
the changes above are not backwardly compatible (however they are more
spec complient), I added the ability to revert back to the old behavior.
The Connection now takes an optional parameter named 'compatible'. If
the value of '7.1' is passed, the driver reverts to the 7.1 behavior.
If the parameter is not passed or the value '7.2' is passed the behavior
is the new behavior. The mechanism put in place can be used in the
future when/if similar needs arise to change behavior. This is
patterned after how Oracle does this (i.e. Oracle has a 'compatible'
parameter that behaves in a similar manner).
Misc fixes. Cleaned up a few things I encountered along the way.
Note that in testing the patch I needed to ignore whitespace differences
in order to get it to apply cleanly (i.e. patch -l -i byteapatch.diff).
Also this patch introduces a new file
(src/interfaces/jdbc/org/postgresql/util/PGbytea.java).
Barry Lind
>there is still an unpatched reference to pg_description in
>getColumns(), in both jdbc1 and jdbc2.
This was introduced by Jeroen's patch (see
http://fts.postgresql.org/db/mw/msg.html?mid=1032468). Attached
is a patch that returns getColumns() to using "select
obj_description()" instead of direct access to pg_description,
as per the request by Tom.
I've incorporated Jeroen's fix to left outer join with
pg_attrdef instead of inner join, so getColumns() also returns
columns without a default value.
I have, however, not included Jeroen's attempt to combine
multiple queries into one huge multi-join query for better
performance, because:
1) I don't know how to do that using obj_description() instead
of direct access to pg_description
2) I don't think a performance improvement (if any) in this
method is very important
Because of the outer join, getColumns() will only work with a
backend >= 7.1. Since the conditional coding for 7.1/7.2 and
jdbc1/jdbc2 is already giving me headaches I didn't pursue a
pre-7.1 solution.
Regards,
Ren? Pijlman <rene@lab.applinet.nl>
ConnectionTest.testTransactionIsolation() in the JDBC driver's
test suite. This reduces the number of failures of the test
suite from 7 to 6. The patch fixes the test case itself, rather
than the driver.
In addition to the change described in my posting below, I fixed
the part of the test with autocommit enabled. The author of the
test assumed that setting the transaction isolation level would
have no effect, but in fact it does. Perhaps the test case
worked with pre-7.1 behaviour, when the JDBC driver set the
isolation level in every transaction, instead of using "set
session characteristics". Anyway, now it works with a backend
built from current CVS and the behaviour is JDBC compliant.
I also extended the test case by changing the isolation level
before beginning a transaction and verifying it inside the
transaction.
Regards,
Ren? Pijlman
suite. This reduces the number of failures from 9 to 7.
Both ConnectionTest and JBuilderTest did not create their own
tables, which caused these test cases to fail with "relation ...
does not exist". It appears these test cases relied on tables
created by the example code elsewhere in the source tree. I've
added the necessary "create table" and "drop table" statements
to the test cases, using the column definitions from the example
code.
While working on that I modified the helper method createTable
in JDBC2Tests.java to take a table parameter, rather than using
table names passed via the properties in build.xml. I'm not sure
what that was good for, and in fact, except for the default
table name "jdbctest", this functionality wasn't used at all.
Ren? Pijlman
>
>> On Mon, 3 Sep 2001 22:01:17 -0500, you wrote:
>> public boolean isWritable(int column) throws SQLException
>> {
>> return !isReadOnly(column);
>> }
Actually, I think this change has a consequence for this method
in the same class:
public boolean isDefinitelyWritable(int column)
throws SQLException
{
return isWritable(column);
}
This is from the JDBC spec
(http://java.sun.com/j2se/1.3/docs/api/java/sql/ResultSetMetaData.html):
isReadOnly() - Indicates whether the designated column is
definitely not writable.
isWritable() - Indicates whether it is possible for a write on
the designated column to succeed.
isDefinitelyWritable() - Indicates whether a write on the
designated column will definitely succeed.
At this time we don't really implement the fine semantics of
these methods. I would suggest the following defaults:
isReadOnly() false
isWritable() true
isDefinitelyWritable() false
And that would mean that your patch is correct, but
isDefinitelyWritable() would need to be patched accordingly:
public boolean isDefinitelyWritable(int column)
throws SQLException
{
return false;
}
Again, both in jdbc1 and jdbc2.
Regards,
Ren? Pijlman <rene@lab.applinet.nl>
>public boolean isWritable(int column) throws SQLException
>{
> if (isReadOnly(column))
> return true;
> else
> return false;
>}
The author probably intended:
public boolean isWritable(int column) throws SQLException
{
return !isReadOnly(column);
}
And if he would have coded it this way he wouldn't have made
this mistake :-)
>hence, isWritable() will always return false. this is something
>of a problem :)
Why exactly? In a way, true is just as incorrect as false, and
perhaps it should throw "not implemented". But I guess that
would be too non-backwardly-compatible.
>let me know if i can provide further information.
Will you submit a patch?
Regards,
Ren? Pijlman <rene@lab.applinet.nl>
flawed in the following ways:
1. Only returned columns that had a default value defined, rather than all
columns in a table
2. Used 2 * N + 1 queries to find out attributes, comments and typenames
for N columns.
By using some outer join syntax it is possible to retrieve all necessary
information in just one SQL statement. This means this version is only
suitable for PostgreSQL >= 7.1. Don't know whether that's a problem.
I've tested this function with current sources and 7.1.3 and patched both
jdbc1 and jdbc2. I haven't compiled nor tested the jdbc1 version though, as
I have no JDK 1.1 available.
Note the discussion in http://fts.postgresql.org/db/mw/msg.html?mid=1029626
regarding differences in obtaining comments on database object in 7.1 and
7.2. I was unable to use the following syntax (or similar ones):
select
...,
description
from
...
left outer join col_description(a.attrelid, a.attnum) description
order by
c.relname, a.attnum;
(the error was parse error at or near '(') so I had to paste the actual
code for the col_description function into the left outer join. Maybe
someone who is more knowledgable about outer joins might provide me with a
better SQL statement.
Jeroen van Vianen
the JDBC driver.
I've done this by extracting it into a new method object called
QueryExecutor (should go into org/postgresql/core/) and then taking it
apart into different methods in that class.
A short summary:
* Extracted ExecSQL() from Connection into a method object called
QueryExecutor.
* Moved ReceiveFields() from Connection to QueryExecutor.
* Extracted parts of the original ExecSQL() method body into smaller
methods on QueryExecutor.
* Bug fix: The instance variable "pid" in Connection was used in two
places with different meaning. Both were probably in dead code, but it's
fixed anyway.
Anders Bengtsson
for the changed files and a few new files:
- test/jdbc2/BatchExecuteTest.java
- util/MessageTranslator.java
- jdbc2/PBatchUpdateException.java
As an aside, is this the best way to submit a patch consisting
of both changed and new files? Or is there a smarter cvs command
which gets them all in one patch file?
This patch fixes batch processing in the JDBC driver to be
JDBC-2 compliant. Specifically, the changes introduced by this
patch are:
1) Statement.executeBatch() no longer commits or rolls back a
transaction, as this is not prescribed by the JDBC spec. Its up
to the application to disable autocommit and to commit or
rollback the transaction. Where JDBC talks about "executing the
statements as a unit", it means executing the statements in one
round trip to the backend for better performance, it does not
mean executing the statements in a transaction.
2) Statement.executeBatch() now throws a BatchUpdateException()
as required by the JDBC spec. The significance of this is that
the receiver of the exception gets the updateCounts of the
commands that succeeded before the error occurred. In order for
the messages to be translatable, java.sql.BatchUpdateException
is extended by org.postgresql.jdbc2.PBatchUpdateException() and
the localization code is factored out from
org.postgresql.util.PSQLException to a separate singleton class
org.postgresql.util.MessageTranslator.
3) When there is no batch or there are 0 statements in the batch
when Statement.executeBatch() is called, do not throw an
SQLException, but silently do nothing and return an update count
array of length 0. The JDBC spec says "Throws an SQLException if
the driver does not support batch statements", which is clearly
not the case. See testExecuteEmptyBatch() in
BatchExecuteTest.java for an example. The message
postgresql.stat.batch.empty is removed from the language
specific properties files.
4) When Statement.executeBatch() is performed, reset the
statement's list of batch commands to empty. The JDBC spec isn't
100% clear about this. This behaviour is only documented in the
Java tutorial
(http://java.sun.com/docs/books/tutorial/jdbc/jdbc2dot0/batchupdates.html).
Note that the Oracle JDBC driver also resets the statement's
list in executeBatch(), and this seems the most reasonable
interpretation.
5) A new test case is added to the JDBC test suite which tests
various aspects of batch processing. See the new file
BatchExecuteTest.java.
Regards,
Ren? Pijlman
system. Some systems did not understand the 'l' section, and in general
it wasn't entirely appropriate.
On SCO OpenServer, the man pages won't be installed at all until someone
figures out their man system.
longer compiles, due to objects being referenced in this patch that do
not exist in JDK1.1.
Barry Lind
---------------------------------------------------------------------------
The JDBC driver requires
permission java.net.SocketPermission "host:port", "connect";
in the policy file of the application using the JDBC driver
in the postgresql.jar file. Since the Socket() call in the
driver is not protected by AccessController.doPrivileged() this
permission must also be granted to the entire application.
>>>>
>>>> permission java.net.SocketPermission "host:port", "connect";
>>>>
>>>>in the policy file of the application using the JDBC driver
>>>>in the postgresql.jar file. Since the Socket() call in the
>>>>driver is not protected by AccessController.doPrivileged() this
>>>>permission must also be granted to the entire application.
>>>>
>>>>The attached diff fixes it so that the connect permission can be
>>>>restricted just the the postgresql.jar codeBase if desired.
David Daney
org.postgresql.util.Serialize and org.postgresql.jdbc2.PreparedStatement
that fixes the ability to "serialize" a simple java class into a
postgres table.
The current cvs seems completely broken in this support, so the patch
puts it into working condition, granted that there are many limitations
with serializing java classes into Postgres.
The code to do serialize appears to have been in the driver since
Postgres 6.4, according to some comments in the source. My code is not
adding any totally new ability to the driver, rather just fixing what
is there so that it actually is usable. I do not think that it should
affect any existing functions of the driver that people regularly
depend on.
The code is activated if you use jdbc2.PreparedStatement and try to
setObject some java class type that is unrecognized, like not String or
not some other primitive type. This will cause a sequence of function
calls that results in an instance of Serialize being instantiated for
the class type passed. The Serialize constructor will query pg_class
to see if it can find an existing table that matches the name of the
java class. If found, it will continue and try to use the table to
store the object, otherwise an SQL exception is thrown and no harm is
done. Serialize.create() has to be used to setup the table for a java
class before anything can really happen with this code other than an
SQLException (unless by some freak chance a table exists that it thinks
it can use).
I saw a difference in Serialize.java between 7.1.3 and 7.2devel that I
didn't notice before, so I had to redo my changes from the 7.2devel
version (why I had to resend this patch now). I was missing the
fixString stuff, which is nice and is imporant to ensure the inserts
will not fail due to embedded single quote or unescaped backslashes. I
changed that fixString function in Serialize just a little since there
is no need to muddle with escaping newlines: only escaping single quote
and literal backslashes is needed. Postgres appears to insert newlines
within strings without trouble.
This patch moves the logic that looks up TypeOid, PGTypeName, and
SQLTypeName from Field to Connection. It is moved to connection since
it needs to differ from the jdbc1 to jdbc2 versions and Connection
already has different subclasses for the two driver versions. It also
made sense to move the logic to Connection as some of the logic was
already there anyway.
Barry Lind
following email.
> > The problem: When I call getBigDecimal() on a ResultSet, it
> > sometimes throws an exception:
> >
> > Bad BigDecimal 174.50
> > at org.postgresql.jdbc2.ResultSet.getBigDecimal(ResultSet.java:373)
> > at org.postgresql.jdbc2.ResultSet.getBigDecimal(ResultSet.java:984)
> > ...blah blah blah...
> > org.postgresql.util.PSQLException: Bad BigDecimal 174.50
Barry Lind
so it may be a transit problem. Also removed the 'txt' suffix
in case that was confusing some transport layer trying to be
too inteligent for our own good.
This may have been because the Array.java class from the
previous patch didn't seem to have made it into the snapshot
build for some reason. This patch should at least fix that issue.
Greg Zoller
> Shouldn't
>
> throw new PSQLException("metadata unavailable");
>
> in getTypeInfo() be something like:
>
> throw new PSQLException("postgresql.meta.unavailable");
>
> to allow translation of the error message in the
> errors*.properties files?
You're right. Attached is an updated patch that also includes a message
in error.properties. I've attempted a French message in
errors_fr.properties but beware that I haven't written French in quite a
few years. Don't know Italian, German, or Dutch so I can't do those.
Liam Stewart
attempt at a patch to 7.1.2 to support Array.
[I think I've solved the mangled patch problem. Hotmail seems to
try to format the text file, so gzipping it should solve this
problem.]
In this patch I've incorporated Barry's feedback. Specifically:
1) OIDs are no longer hard-coded into Array.java. In order to
support this change I added a getOID(String) method to Field.java
which receives a PostgreSQL field type and returns a value from
java.sql.Types. I couldn't get away from using OIDs altogether
because the JDBC spec for Array specifies that some methods return
a ResultSet. This requires I construct Field objects,
which means I need OIDs. At least this approach doesn't hard
code these values. A Hashtable cache has been added to Field
so that an SQL lookup isn't necessary (following the model already
in Field.java).
2) Rewired the base formatting code in ResultSet.java to use 'to'
methods, which are then exposed as static methods in ResultSet.
These methods are used in Array to format the data without
duplications in the code.
3) Artifact call to first() in ResultSet.getArray() removed.
Greg Zoller
includes two changes in the JDBC driver:
1) When connected to a backend >= 7.2: use obj_description() and
col_description() instead of direct access to pg_description.
2) In DatabaseMetaData.getTables()/getColumns()/getProcedures():
when there is no comment on the object, return null in the
REMARKS column of the ResultSet, instead of the default string
"no remarks".
Change 2 first appeared as a side-effect of change 1, but it is
actually more compliant with the JDBC spec: "String object
containing an explanatory comment on the table/column/procedure,
which may be null". The default string "no remarks" was strictly
speaking incorrect, as it could not be distinguished from a real
user comment "no remarks". So I removed the default string
completely.
Change 2 might break existing code that doesn't follow the JDBC
spec and isn't prepared to handle a null in the REMARKS column
of getTables()/getColumns()/getProcedures.
Patch tested with jdbc2 against both a 7.1 and a CVS tip
backend. I did not have a jdbc1 environment to build and test
with, but since the touched code is identical in jdbc1 and jdbc2
I don't foresee any problems.
Regards,
Ren? Pijlman
* Merges identical code from org.postgresql.jdbc[1|2].Statement into
org.postgresql.Statement.
* Moves escapeSQL() method from Connection to Statement (the only place
it's used)
* Minor cleanup of the new isolation level stuff.
* Minor cleanup of version string handling.
Anders Bengtsson
Here is a context diff from latest cvs
And I see why you couldn't apply the last diff, the setCatalog diff has
been backed out, that was causing the compile problem in the first
place.
This following one needs to be applied to allow the current cvs to
compile
Dave Cramer
1) improves performance of commit/rollback by reducing number of round
trips to the server
2) uses 7.1 functionality for setting the transaction isolation level
3) backs out a patch from 11 days ago because that code failed to
compile under jdk1.1
Details:
1) The old code was doing the following for each commit:
commit
begin
set transaction isolation level xxx
thus a call to commit was performing three round trips to the database.
The new code does this in one round trip as:
commit; begin; set transaction isolation level xxx
In a simple test program that performs 1000 transactions (where each
transaction does one simple select inside that transaction) has the
following before and after timings:
Client and Server on same machine
old new
--- ---
1.877sec 1.405sec 25.1% improvement
Client and Server on different machines
old new
--- ---
4.184sec 2.927sec 34.3% improvement
(all timings are an average of four different runs)
2) The driver was using 'set transaction isolation level xxx' at the
begining of each transaction, instead of using the new 7.1 syntax of
'set session characteristics as transaction isolation level xxx' which
only needs to be done once instead of for each transaction. This is
done conditionally (i.e. if server is 7.0 or older do the old behaviour,
else do the new behaviour) to not break backward compatibility. This
also required the movement of some code to check/test database version
numbers from the DatabaseMetaData object to the Connection object.
3) Finally while testing, I discovered that the code that was checked in
11 days ago actually didn't compile. The code in the patch for
Connection.setCatalog() used Properties.setProperty() which only exists
in JDK1.2 or higher. Thus compiling the JDBC1 driver failed as this
method doesn't exist. Thus I backed out that patch.
Barry Lind
connection implementations (org.postgresql.jdbc[1|2].Connection) into
their superclass (org.postgresql.Connection).
It also changes the close() methods of Connection and PG_Stream, so that
PG_Stream no longer is responsible for sending the termination packet 'X'
to the backend. I figured that protocol-level stuff like that belonged in
Connection more than in PG_Stream.
Anders Bengtsson
in Connection - note: I've updated setCatalog(String catalog) from my previous
diff so it checks whether it is already connected to the specified catalog.
Jason Davies
