PostgreSQL Weekly News – April 24 2011

== PostgreSQL Weekly News – April 24 2011 ==

New Survey: Have you used pg_upgrade?

== PostgreSQL Product News ==

MicroOLAP Database Designer 1.8.0 Beta 2 for PostgreSQL released.

== PostgreSQL Jobs for April ==

== PostgreSQL Local ==

PostgreSQL Turkey User Group is organizing a one-day event at April
30, 2011. Follow @PgDayTR for details in Turkish.

Open Database Camp will be on May 7-9, 2011 in Sardinia, Italy

PGCon will be May 19-20, 2011 at the University of Ottawa, preceded by
two days of tutorials on May 17-18.

PG Session 2, on PostGIS, will be held on June 23rd in Paris. The CfP
is open!

pgbr will be in Sao Paulo, Brazil November 3-4, 2011.

== PostgreSQL in the News ==

Planet PostgreSQL:

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm Pacific time.
Please send English language ones to, German language
to, Italian language to Spanish language

== Reviews ==

== Applied Patches ==

Robert Haas pushed:

– recoveryStopsHere() must check the resource manager ID. Before
commit c016ce728139be95bb0dc7c4e5640507334c2339, this wasn’t needed,
but now that multiple resource manager IDs can percolate down
through here, we have to make sure we know which one we’ve got.
Otherwise, we can confuse (for example) an XLOG_XACT_COMMIT record
with an XLOG_CHECKPOINT_SHUTDOWN record. Review by Jaime Casanova

– Only allow typed tables to hang off composite types, not e.g.
tables. This also ensures that we take a relation lock on the
composite type when creating a typed table, which is necessary to
prevent the composite type and the typed table from getting out of
step in the face of concurrent DDL. Noah Misch, with some changes.

– Allow ALTER TABLE name {OF type | NOT OF}. This syntax allows a
standalone table to be made into a typed table, or a typed table to
be made standalone. This is possibly a mildly useful feature in its
own right, but the real motivation for this change is that we need
it to make pg_upgrade work with typed tables. This doesn’t actually
fix that problem, but it’s necessary infrastructure. Noah Misch.

– Typo fix.

– Fix use of incorrect constant RemoveRoleFromObjectACL. This could
cause failures when DROP OWNED BY attempt to remove default
privileges on sequences. Back-patching to 9.0. Shigeru Hanada

– Allow ALTER TYPE .. ADD ATTRIBUTE .. CASCADE to recurse to
descendants. Without this, adding an attribute to a typed table
with an inheritance child fails, which is surprising. Noah Misch,
with minor changes by me.

Andrew Dunstan pushed:

– Attempt to remedy buildfarm breakage caused by commit f536d4194.

– Silence compiler warning about casting HANDLE to long on WIN64.

– Silence a few compiler warnings from gcc on MinGW. Most of these
cast DWORD to int or unsigned int for printf type handling. This is
safe even on 64 bit architectures because a DWORD is always 32 bits.
In one case a variable is initialised to keep the compiler happy.

Tom Lane pushed:

– Improve findoidjoins to cover more cases. Teach the program and
script to deal with OID-array referencing columns, which we now have
several of. Also, modify the recommended usage process to specify
that the program should be run against the regression database
rather than template1. This lets it find numerous joins that cannot
be found in template1 because the relevant catalogs are entirely
empty. Together these changes add seventeen formerly-missed cases
to the oidjoins regression test.

– Improve cost estimation for aggregates and window functions. The
previous coding failed to account properly for the costs of
evaluating the input expressions of aggregates and window functions,
as seen in a recent gripe from Claudio Freire. (I said at the time
that it wasn’t counting these costs at all; but on closer
inspection, it was effectively charging these costs once per output
tuple. That is completely wrong for aggregates, and not exactly
right for window functions either.) There was also a hard-wired
assumption that aggregates and window functions had procost 1.0,
which is now fixed to respect the actual cataloged costs. The
costing of WindowAgg is still pretty bogus, since it doesn’t try to
estimate the effects of spilling data to disk, but that seems like a
separate issue.

– Update oidjoins regression test for 9.1 catalog schema additions.

– Fix handling of collations in multi-row VALUES constructs. Per spec
we ought to apply select_common_collation() across the expressions
in each column of the VALUES table. The original coding was just
taking the first row and assuming it was representative. This patch
adds a field to struct RangeTblEntry to carry the resolved
collations, so initdb is forced for changes in stored rule

– Refrain from canonicalizing a client_encoding setting of “UNICODE”.
While “UTF8” is the correct name for this encoding, existing JDBC
drivers expect that if they send “UNICODE” it will read back the
same way; they fail with an opaque “Protocol error” complaint if
not. This will be fixed in the 9.1 drivers, but until older drivers
are no longer in use in the wild, we’d better leave “UNICODE” alone.
Continue to canonicalize all other inputs. Per report from Steve
Singer and subsequent discussion.

– Revert “Prevent incorrect updates of pg_index while reindexing
pg_index itself.” This reverts commit
4b6106ccfea21e86943f881edcf3cfc03661a415 of 2011-04-15. There’s a
better way to do it, which will follow shortly.

– Avoid changing an index’s indcheckxmin horizon during REINDEX.
There can never be a need to push the indcheckxmin horizon forward,
since any HOT chains that are actually broken with respect to the
index must pre-date its original creation. So we can just avoid
changing pg_index altogether during a REINDEX operation. This
offers a cleaner solution than my previous patch for the problem
found a few days ago that we mustn’t try to update pg_index while we
are reindexing it. System catalog indexes will always be created
with indcheckxmin = false during initdb, and with this modified code
we should never try to change their pg_index entries. This avoids
special-casing system catalogs as the former patch did, and should
provide a performance benefit for many cases where REINDEX formerly
caused an index to be considered unusable for a short time.
Back-patch to 8.3 to cover all versions containing HOT. Note that
this patch changes the API for index_build(), but I believe it is
unlikely that any add-on code is calling that directly.

– Make plan_cluster_use_sort cope with no IndexOptInfo for the target
index. The original coding assumed that such a case represents
caller error, but actually get_relation_info will omit generating an
IndexOptInfo for any index it thinks is unsafe to use. Therefore,
handle this case by returning “true” to indicate that a
seqscan-and-sort is the preferred way to implement the CLUSTER
operation. New bug in 9.1, no backpatch needed. Per bug #5985 from
Daniel Grace.

– Set indcheckxmin true when REINDEX fixes an invalid or not-ready
index. Per comment from Greg Stark, it’s less clear that HOT chains
don’t conflict with the index than it would be for a valid index.
So, let’s preserve the former behavior that indcheckxmin does get
set when there are potentially-broken HOT chains in this case. This
change does not cause any pg_index update that wouldn’t have
happened anyway, so we’re not re-introducing the previous bug with
pg_index updates, and surely the case is not significant from a
performance standpoint; so let’s be as conservative as possible.

– Fix bugs in indexing of in-doubt HOT-updated tuples. If we find a
DELETE_IN_PROGRESS HOT-updated tuple, it is impossible to know
whether to index it or not except by waiting to see if the deleting
transaction commits. If it doesn’t, the tuple might again be LIVE,
meaning we have to index it. So wait and recheck in that case.
Also, we must not rely on ii_BrokenHotChain to decide that it’s
possible to omit tuples from the index. That could result in
omitting tuples that we need, particularly in view of yesterday’s
fixes to not necessarily set indcheckxmin (but it’s broken even
without that, as per my analysis today). Since this is just an
extremely marginal performance optimization, dropping the test
shouldn’t hurt. These cases are only expected to happen in system
catalogs (they’re possible there due to early release of
RowExclusiveLock in most catalog-update code paths). Since
reindexing of a system catalog isn’t a particularly
performance-critical operation anyway, there’s no real need to be
concerned about possible performance degradation from these changes.
The worst aspects of this bug were introduced in 9.0 — 8.x will
always wait out a DELETE_IN_PROGRESS tuple. But I think dropping
index entries on the strength of ii_BrokenHotChain is dangerous even
without that, so back-patch removal of that optimization to 8.3 and

– Avoid possible divide-by-zero in gincostestimate. Per report from
Jeff Janes.

– Make a code-cleanup pass over the collations patch. This patch is
almost entirely cosmetic — mostly cleaning up a lot of neglected
comments, and fixing code layout problems in places where the patch
made lines too long and then pgindent did weird things with that. I
did find a bug-of-omission in equalTupleDescs().

– De-kludge contrib/btree_gin for collations. Using
DEFAULT_COLLATION_OID in the comparePartial functions was not only a
lame hack, but outright wrong, because the compare functions for
collation-aware types were already responding to the declared index
collation. So comparePartial would have the wrong expectation about
the index’s sort order, possibly leading to missing matches for
prefix searches.

– Make GIN and GIST pass the index collation to all their support
functions. Experimentation with contrib/btree_gist shows that the
majority of the GIST support functions potentially need collation
information. Safest policy seems to be to pass it to all of them,
instead of making assumptions about which ones could possibly need

– Fix contrib/btree_gist to handle collations properly. Make use of
the collation attached to the index column, instead of hard-wiring
DEFAULT_COLLATION_OID. (Note: in theory this could require
reindexing btree_gist indexes on textual columns, but I rather doubt
anyone has one with a non-default declared collation as yet.)

– Fix char2wchar/wchar2char to support collations properly. These
functions should take a pg_locale_t, not a collation OID, and should
call mbstowcs_l/wcstombs_l where available. Where those functions
are not available, temporarily select the correct locale with
uselocale(). This change removes the bogus assumption that all
locales selectable in a given database have the same wide-character
conversion method; in particular, the collate.linux.utf8 regression
test now passes with LC_CTYPE=C, so long as the database encoding is
UTF8. I decided to move the char2wchar/wchar2char functions out of
mbutils.c and into pg_locale.c, because they work on wchar_t not
pg_wchar_t and thus don’t really belong with the mbutils.c
functions. Keeping them where they were would have required
importing pg_locale_t into pg_wchar.h somehow, which did not seem
like a good plan.

– Adjust comments about collate.linux.utf8 regression test. This test
should now work in any database with UTF8 encoding, regardless of
the database’s default locale. The former restriction was really
“doesn’t work if default locale is C”, and that was because of not
handling mbstowcs/wcstombs correctly.

– Hash indexes had better pass the index collation to support
functions, too. Per experimentation with contrib/citext, whose hash
function assumes that it’ll be passed a collation.

Heikki Linnakangas pushed:

– Silence compiler warning about unused variable on Windows.

– Quotes in strings injected into bki file need to escaped. In
particular, “People’s Republic of China” locale on Windows was
causing initdb to fail. This fixes bug #5818 reported by yulei. On
master, this makes the mapping of “People’s Republic of China” to
just “China” obsolete. In 9.0 and 8.4, just fix the escaping.
Earlier versions didn’t have locale names in bki file.

Peter Eisentraut pushed:

– Avoid unused variable warnings for certain configurations.

– Add gitignore entries for Windows MSVC builds

– Fix typo

– Treat as optional in This is how
treats it and how it’s documented.

– Refix the unaccent regression test on MSVC properly … for some
value of “properly”. Instead of overriding REGRESS_OPTS, set the
variables ENCODING and NO_LOCALE, which is more expressive and
allows overriding by the user. Fix to handle that.

– Fix PL/Python traceback for error in separate file. It assumed that
the lineno from the traceback always refers to the PL/Python
function. If you created a PL/Python function that imports some
code, runs it, and that code raises an exception, PLy_traceback
would get utterly confused. Now we look at the file name reported
with the traceback and only print the source line if it came from
the PL/Python function. Jan Urbański

– Add fill-column setting to emacs example configurations. This
matches the maximum line length that pgindent uses.

– Small update to emacs example configuration. Since both tarballs
and git now result in a “postgresql” directory rather than a “pgsql”
directory, adjust the example configuration to look for the former.

– Normalize whitespace in the arguments of <indexterm>. Strip leading
and trailing whitespace and replace interior whitespace by a single
space. This avoids problems with the index generator producing
duplicate index entries for terms that differ only in whitespace.
Commit dca30da3433c40b5f92f1704c496cda052decef9 actually fixed all
the indexterm elements that would cause this problem at the moment,
but in case it sneaks in again, we’re set.

Bruce Momjian pushed:

– Add C comment about why we throw an error if the pg_upgrade old/new
database counts don’t match.

– Throw error for mismatched pg_upgrade clusters. If someone removes
the ‘postgres’ database from the old cluster and the new cluster has
a ‘postgres’ database, the number of databases will not match. We
actually could upgrade such a setup, but it would violate the 1-to-1
mapping of database counts, so we throw an error instead.
Previously they got an error during the upgrade, and not at the
check stage; Peter Geoghegan 9.0.4 does the same.

– pg_upgrade C comment addition. Document why we do the missing new
database check during the check phase.

– Improve doc wording for SQL syntax of LIMIT/OFFSET.

– In pg_upgrade, only compile copy_file() on non-Win32 systems. Per
report from Andrew Dunstan.

== Rejected Patches (for now) ==

No one was disappointed this week 🙂

== Pending Patches ==

Shigeru HANADA sent in another revision of the patch to fix foreign
table privileges.

Robert Haas sent in a patch for 9.2 to allow for time-delayed

Noah Misch sent in another revision of the patch to resolve an
incompatibility between pg_dump –binary-upgrade and ALTER TYPE …

Bruce Momjian sent in three revisions of a patch to make pg_upgrade
disable autovacuum.

Dan Ports sent in a patch to fix the UPDATE performance under SSI.

Leonardo Francalanci sent in a patch to allow switching tables from

Alex Hunsaker sent in two revisions of a patch to enable the new perl
5.14 to work with the build system.

Gianni Ciolli sent in a patch to fix a performance issue in NOTIFY.

Andrew Dunstan sent in a patch to do a consolidate cleanup on win32.

Peter Eisentraut sent in a patch to allow make check to work in

Comments are closed.