PostgreSQL Weekly News – June 26 2011

== PostgreSQL Weekly News – June 26 2011 ==

== PostgreSQL Product News ==

HTSQL 2.1-RC1, a high-level query language for relational databases,

MicroOLAP Database Designer 1.8.0 for PostgreSQL released.

psycopg2 2.4.2, a Python connector for PostgreSQL, released.

== PostgreSQL Jobs for June ==

== PostgreSQL Local ==

CHAR(11), the PostgreSQL Conference on Clustering, High Availability
and Replication is now open for online registration and bookings.
July 11-12 2011 in Cambridge, UK.

PostgreSQL Conference China 2011 will be held in Guangzhou July
15-16, 2011.

PDXPUG is hosting PgDay on Sunday, July 24, 2011, one day before
OSCON, in Portland, Oregon, USA. Details here:

Postgres Open 2011, a conference focused on disruption of the database
industry through PostgreSQL, will take place September 14-16, 2011 in
Chicago, Illinois at the Westin Michigan Avenue hotel.

PG-Day Denver 2011 will be held on Saturday, September 17th, 2011 at
the Auraria Campus near downtown Denver, Colorado.

PostgreSQL Conference West (#PgWest) will be held September 27th-30th,
2011 at the San Jose Convention center in San Jose, California, USA.

PostgreSQL Conference Europe 2011 will be held on October 18-21 in

pgbr will be in Sao Paulo, Brazil November 3-4, 2011.

PGConf.DE 2011 is the German-speaking PostgreSQL Conference and will
take place on November 11th in the Rheinisches Industriemuseum in
Oberhausen, Germany. Call for Papers is open.

== PostgreSQL in the News ==

Planet PostgreSQL:

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm Pacific time.
Please send English language ones to, German language
to, Italian language to Spanish language

== Reviews ==

Noah Misch reviewed the patch to fix XML handling for the bug which
produced NULL where empty string was appropriate.
== Applied Patches ==

Tom Lane pushed:

- Fix thinko in previous patch to always update
pg_class.reltuples/relpages. I mis-simplified the test where
ANALYZE decided if it could get away without doing anything: under
the new regime, that’s never allowed. Per bug #6068 from Jeff
Janes. Back-patch to 8.4, just like previous patch.

- Add REL9_1_STABLE to the set of branches tracked by git_changelog.

- Fix thinko in previous patch for optimizing EXISTS-within-EXISTS.
When recursing after an optimization in
pull_up_sublinks_qual_recurse, the available_rels value passed down
must include only the relations that are in the righthand side of
the new SEMI or ANTI join; it’s incorrect to pull up a sub-select
that refers to other relations, as seen in the added test case. Per
report from BangarRaju Vadapalli. While at it, rethink the idea of
recursing below a NOT EXISTS. That is essentially the same
situation as pulling up ANY/EXISTS sub-selects that are in the ON
clause of an outer join, and it has the same disadvantage: we’d
force the two joins to be evaluated according to the syntactic
nesting order, because the lower join will most likely not be able
to commute with the ANTI join. That could result in having to form
a rather large join product, whereas the handling of a correlated
subselect is not quite that dumb. So until we can handle those
cases better, #ifdef NOT_USED that case. (I think it’s okay to pull
up in the EXISTS/ANY cases, because SEMI joins aren’t so inflexible
about ordering.) Back-patch to 8.4, same as for previous patch in
this area. Fortunately that patch hadn’t made it into any shipped
releases yet.

- Fix missed use of “cp -i” in an example, per Fujii Masao. Also be
more careful about markup: use & not just &.

- Apply upstream fix for blowfish signed-character bug
(CVE-2011-2483). A password containing a character with the high
bit set was misprocessed on machines where char is signed (which is
most). This could cause the preceding one to three characters to
fail to affect the hashed result, thus weakening the password. The
result was also unportable, and failed to match some other blowfish
implementations such as OpenBSD’s. Since the fix changes the output
for such passwords, upstream chose to provide a compatibility hack:
password salts beginning with $2x$ (instead of the usual $2a$ for
blowfish) are intentionally processed “wrong” to give the same hash
as before. Stored password hashes can thus be modified if necessary
to still match, though it’d be better to change any affected
passwords. In passing, sync a couple other upstream changes that
marginally improve performance and/or tighten error checking.
Back-patch to all supported branches. Since this issue is already
public, no reason not to commit the fix ASAP.

- Minor editing for README-SSI. Fix some grammatical issues, try to
clarify a couple of proofs, make the terminology more consistent.

- Fix symlink for errcodes.h so it works in VPATH builds from
tarballs. backend/Makefile was treating errcodes.h as a header
always generated during build, but actually it’s a header provided
in tarballs. Hence, must use the absolute-symlink recipe, not the
relative-symlink one. Per bug #6072 from Hartmut Raschick.

- Add missing -I switch for VPATH builds. Per bug #6073 from Hartmut

- Update alternative expected file for recent sequence test changes.

- Revert “Don’t select log_cnt in sequence regression tests.” This
reverts commit addf11f9a264417aa467d4e135b9a8afc59f172a. The right
fix for the problem is to update the alternative expected file, not
to lobotomize the test case.

- Undo overly enthusiastic de-const-ification. s/const//g wasn’t
exactly what I was suggesting here … parameter declarations of the
form “const structtype *param” are good and useful, so put those
occurrences back. Likewise, avoid casting away the const in a
“const void *” parameter.

Peter Eisentraut pushed:

- Produce HISTORY file consistently as ASCII. The release notes may
contain non-ASCII characters (for contributor names), which lynx
converts to the encoding determined by the current locale. The get
output that is deterministic and easily readable by everyone, we
make lynx produce LATIN1 and then convert that to ASCII with
transliteration for the non-ASCII characters.

- Message style and spelling improvements

- Add a bit of markup.

- Add information schema views role_udt_grants, udt_privileges,

- Replace tabs by spaces in makefile comment.

- Add the possibility to pass –flag arguments to xgettext calls. The
–flag argument can be used to tell xgettext the arguments of which
functions should be flagged with c-format in the PO files, instead
of guessing based on the presence of format specifiers, which fails
if no format specifiers are present but the translation accidentally
introduces one. Appropriate flag settings have been added for each
message catalog. based on a patch by Christoph Berg for bug #6066

- Make _ be automatically included in GETTEXT_TRIGGERS. Since it’s
globally defined in c.h, it should be treated as a gettext trigger

- Replace := by = in files. It currently doesn’t make a
difference, but it’s inconsistent with most other usage, and it
might interfere with a future patch, so I’ll change it all in a
separate commit. Also, replace tabs with spaces for alignment.

- Refactor common gettext triggers. Put gettext trigger words that
are common to the backend and backend modules into a makefile
variable to include everywhere, to avoid error-prone repetitions.

- Remove some cruft from files

- Add missing file to GETTEXT_FILES. This doesn’t actually change the
resulting set of strings, but better be correct.

Michael Meskes pushed:

- Fixed string in German translation that causes segfault. Applied
patch by Christoph Berg <> to replace placeholder “%s” by
correct string.

Alvaro Herrera pushed:

- Remove extra copying of TupleDescs for heap_create_with_catalog.
Some callers were creating copies of tuple descriptors to pass to
that function, stating in code comments that it was necessary
because it modified the passed descriptor. Code inspection reveals
this not to be true, and indeed not all callers are passing copies
in the first place. So remove the extra ones and the misleading
comments about this behavior as well.

Heikki Linnakangas pushed:

- Fix bug introduced by recent SSI patch to merge ROLLED_BACK and
MARKED_FOR_DEATH flags into one. We still need the ROLLED_BACK flag
to mark transactions that are in the process of being rolled back.
To be precise, ROLLED_BACK now means that a transaction has already
been discounted from the count of transactions with the oldest xmin,
but not yet removed from the list of active transactions. Dan Ports

- Fix bug in PreCommit_CheckForSerializationFailure. A transaction
that has already been marked as PREPARED cannot be killed. Kill the
current transaction instead. One of the prepared_xacts regression
tests actually hits this bug. I removed the anomaly from the
duplicate-gids test so that it fails in the intended way, and added
a new test to check serialization failures with a prepared
transaction. Dan Ports

- Adjust the alternative expected output file for prepared_xacts test
case, used when max_prepared_transactions=0, for the recent changes
in the test case.

- Remove pointless const qualifiers from function arguments in the SSI
code. As Tom Lane pointed out, “const Relation foo” doesn’t
guarantee that you can’t modify the data the “foo” pointer points
to. It just means that you can’t change the pointer to point to
something else within the function, which is not very useful.

Robert Haas pushed:

- Performance tuning advice for SSI. Kevin Grittner, with additional
wordsmithing by me.

- Add notion of a “transform function” that can simplify function
calls. Initially, we use this only to eliminate calls to the
varchar() function in cases where the length is not being reduced
and, therefore, the function call is equivalent to a RelabelType
operation. The most significant effect of this is that we can avoid
a table rewrite when changing a varchar(X) column to a varchar(Y)
column, where Y > X. Noah Misch, reviewed by me and Alexey Klyukin

- Make deadlock_timeout PGC_SUSET rather than PGC_SIGHUP. This allows
deadlock_timeout to be reduced for transactions that are
particularly likely to be involved in a deadlock, thus detecting it
more quickly. It is also potentially useful as a poor-man’s
deadlock priority mechanism: a transaction with a high
deadlock_timeout is less likely to be chosen as the victim than one
with a low deadlock_timeout. Since that could be used to game the
system, we make this PGC_SUSET rather than PGC_USERSET. At some
point, it might be worth thinking about a more explicit priority
mechanism, since using this is far from fool-proof. But let’s see
whether there’s enough use case to justify the additional work
before we go down that route. Noah Misch, reviewed by Shigeru

- Add smallserial pseudotype. This is just like serial and bigserial,
except it generates an int2 column rather than int4 or int8. Mike
Pultz, reviewed by Brar Piening and Josh Kupershmidt

- Add some regression tests for serial pseudotypes. Josh Kupershmidt

- Make the visibility map crash-safe. This involves two main changes
from the previous behavior. First, when we set a bit in the
visibility map, emit a new WAL record of type XLOG_HEAP2_VISIBLE.
Replay sets the page-level PD_ALL_VISIBLE bit and the visibility map
bit. Second, when inserting, updating, or deleting a tuple, we can
no longer get away with clearing the visibility map bit after
releasing the lock on the corresponding heap page, because an
intervening crash might leave the visibility map bit set and the
page-level bit clear. Making this work requires a bit of interface
refactoring. In passing, a few minor but related cleanups: change
the test in visibilitymap_set and visibilitymap_clear to throw an
error if the wrong page (or no page) is pinned, rather than silently
doing nothing; this case should never occur. Also, remove duplicate
definitions of InvalidXLogRecPtr. Patch by me, review by Noah

- Don’t select log_cnt in sequence regression tests. It’s not
entirely stable. Per suggestion from Josh Kupershmidt.

- Documentation improvements for pg_locks with respect to SSI.
Explain that querying pg_locks does not simultaneously lock both the
normal lock manager and the predicate lock manager. Per discussion
with Kevin Grittner.

Bruce Momjian pushed:

- Fix pg_upgrade status message capitalization mistake. Backpatch to
9.1 and 9.0. Dan McGee

- Adjust pg_upgrade check for pg_upgrade_support to happen after the
binary directory has been validated. Backpatch to 9.1. Dan McGee

- Rename pg_upgrade directory validation function, for clarity.

- In pg_upgrade, fix odd function parameter wrapping.

- In pg_upgrade, check that the binary and data directories are the
same major version. Backpatch to 9.1. Dan McGee

- In pg_upgrade docs, clarify that link mode uses “hard” links.
Backpatch to 9.1 and 9.0.

Joe Conway pushed:

- Async dblink functions require a named connection, and therefore
Problem found by Peter Eisentraut and patch by Fujii Masao.

== Rejected Patches (for now) ==

No one was disappointed this week :-)

== Pending Patches ==

Alexander Korotkov and Heikki Linnakangas traded revisions of a patch
to build GiST indexes more quickly.

Cedric Villemain sent in another revision, along with a review, of the
patch to constrain backend temp file space.

Alexey Klyukin sent in another revision of the patch to allow
validating configuration files without starting the postmaster.

Pavel Stehule sent in a patch for PL/pgsql to remove an unnecessary
ccache search when a array variable is updated.

Alvaro Herrera sent in two more revisions of the patch to make
keywords in pg_hba.conf field-specific.

Per review from Steve Singer, Pavel Stehule sent in another revision
of the patch to enhance error reporting in PL/pgsql.

Peter Geoghegan sent in another revision of the patch which wakes on
postmaster death.

Simon Riggs sent in two more revisions of a patch to fix an issue in a
privious patch, which was intended to take weaker locks on some kinds
of DDL.

Robert Haas and KaiGai Kohei traded patches to rework DROP.

Andrew Dunstan sent in a WIP patch to replace pgindent.

Peter Geoghegan sent in two more revisions of a patch to wake on
postmaster death.

Magnus Hagander sent in another revision of a patch to fix some
infelicities in how pg_dump uses malloc.

Florian Pflug sent in two more revisions of a patch to fix xml

Dan Ports and Kevin Grittner traded patches to skip some calls to

Bruce Momjian sent in two revisions of a patch to change the port for

Robert Haas sent in a patch intended to reduce spinlock contention.

Mark Kirkwood reviewed and revised the patch to control backend temp
file space.

Comments are closed.