PostgreSQL Weekly News – February 20 2011
== PostgreSQL Weekly News – February 20 2011 ==
== PostgreSQL Product News ==
pgpool-II 3.0.2, a connection pooler and more, released.
PL/Proxy 2.2, a write-scaling clustering system for PostgreSQL, released.
== PostgreSQL Jobs for February ==
== PostgreSQL Local ==
PGDay is scheduled at this year’s Southern California Linux Exposition
(SCALE) held in the LAX Hilton Hotel in the city of Los Angeles,
California, on Friday February 25th, 2011.
PostgreSQL Conference East 2011: NYC March 22nd – 25th
Open Database Camp will be on May 7-9, 2011 in Sardinia, Italy
PGCon will be May 19-20, 2011 at the University of Ottawa, preceded by
two days of tutorials on May 17-18.
== PostgreSQL in the News ==
Planet PostgreSQL: http://planet.postgresql.org/
PostgreSQL Weekly News is brought to you this week by David Fetter
Submit news and announcements by Sunday at 3:00pm Pacific time.
Please send English language ones to firstname.lastname@example.org, German language
to email@example.com, Italian language to firstname.lastname@example.org. Spanish language
== Reviews ==
== Applied Patches ==
Bruce Momjian pushed:
- Clarify documentation for libpq’s PQescapeBytea to mention the new
hex format. Modify PQescapeStringConn() docs to be consisent with
other escaping functions. Add mention problems with pre-9.0
versions of libpq using not understanding bytea hex format to the
9.0 release notes. Backpatch to 9.0 docs.
- Adjust pg_upgrade error message, array freeing, and add error check.
- Fix bug in 9.1 pg_upgrade processing of old/new relations; adjust
- In pg_upgrade, no need to initialize global struct values; they are
always zeros. Also no need to free memory before we exit.
- In documentation example, use concat_values() instead of concat()
because concat() is a built-in function. Erik Rijkers
- Document that plpgsql upsert example can fail if insert triggers
return unique exception, or if not used in read committed
- Remove doc mention about read committed in upsert example.
- Typo fix. Erik Rijkers
- Document that text search comparisons happen after text search
configuration rules. Erik Rijkers
- Improve text search prefix doc addition.
- Initialize variable to quiet compiler.
- Document that an SQL-standard-compliant query that expects a unique
row from information_schema.referential_constraints might return
several in Postgres because we allow duplicate-named constraints in
the same schema.
- Documentation update for UTF16 surrogate pairs in UTF8 encoding.
- Adjust documentation wording of window function ORDER BY to not
mention avg() because it was inaccurate.
- Merge two documentation permission chapters into a single chapter.
Tom Lane pushed:
- Change the naming convention for extension files to use double
dashes. This allows us to have an unambiguous rule for
deconstructing the names of script files and secondary control
files, without having to forbid extension and version names from
containing any dashes. We do have to forbid them from containing
double dashes or leading/trailing dashes, but neither restriction is
likely to bother anyone in practice. Per discussion, this seems
like a better solution overall than the original design.
- Convert contrib modules to use the extension facility. This isn’t
fully tested as yet, in particular I’m not sure that the
“foo–unpackaged–1.0.sql” scripts are OK. But it’s time to get
some buildfarm cycles on it. sepgsql is not converted to an
extension, mainly because it seems to require a very nonstandard
installation process. Dimitri Fontaine and Tom Lane.
- Avoid use of CREATE OR REPLACE FUNCTION in extension installation
files. It was never terribly consistent to use OR REPLACE (because
of the lack of comparable functionality for data types, operators,
etc), and experimentation shows that it’s now positively pernicious
in the extension world. We really want a failure to occur if there
are any conflicts, else it’s unclear what the extension-ownership
state of the conflicted object ought to be. Most of the time,
CREATE EXTENSION will fail anyway because of conflicts on other
object types, but an extension defining only functions can succeed,
with bad results.
- Assorted fixups for “unpackaged” conversion scripts. From first
pass of testing. Notably, there seems to be no need for
adminpack–unpackaged–1.0.sql because none of the objects that the
old module creates would ever be dumped by pg_dump anyway (they are
all in pg_catalog).
- Support replacing MODULE_PATHNAME during extension script file
execution. This avoids the need to find a way to make PGXS’
.sql.in-to-.sql rule insert the right thing. We’ll just deprecate
use of that hack for extensions.
- More fixups for “unpackaged” conversion scripts.
- Remove no-longer-needed special case hacks in MSVC build scripts.
- Fix obsolete references to old-style contrib installation methods.
- Small improvements to external-projects documentation.
- Rearrange extension-related views as per recent discussion. The
original design of pg_available_extensions did not consider the
possibility of version-specific control files. Split it into two
views: pg_available_extensions shows information that is generic
about an extension, while pg_available_extension_versions shows all
available versions together with information that could be
version-dependent. Also, add an SRF pg_extension_update_paths() to
assist in checking that a collection of update scripts provide sane
update path sequences.
- Fix MSVC build scripts for recent extension-related changes.
Untested, but we’ll soon see if the buildfarm likes this.
- Rethink naming of contrib/intagg extension. Initially it was called
int_aggregate after the old SQL file, but since the documentation
just says “intagg” and that’s also the directory name, let’s conform
to that instead.
- Fix obsolete comment. Comment about MaxAllocSize was not updated
when the TOAST-header macros were replaced in 8.3 “varvarlena”
changes. Per report from Frederik Ramm.
- Bring hstore’s comment into line with style of other contrib
comments. All the other ones that are primarily a new datatype say
“data type for <purpose>”, so make this one similar.
- Add CheckTableNotInUse calls in DROP TABLE and DROP INDEX. Recent
releases had a check on rel->rd_refcnt in heap_drop_with_catalog,
but failed to cover the possibility of pending trigger events at
DROP time. (Before 8.4 we didn’t even check the refcnt.) When the
trigger events were eventually fired, you’d get “could not open
relation with OID nnn” errors, as in recent report from strk.
Better to throw a suitable error when the DROP is attempted. Also
add a similar check in DROP INDEX. Back-patch to all supported
- Fix corner case for binary upgrade: extension functions in
pg_catalog. Normally, pg_dump summarily excludes functions in
pg_catalog from consideration. However, some extensions may create
functions in pg_catalog (adminpack already does that, and extensions
for procedural languages will likely do it too). In binary-upgrade
mode, we have to dump such functions, or the extension will be
incomplete after upgrading. Per experimentation with adminpack.
- Add FOREACH IN ARRAY looping to plpgsql. (I’m not entirely sure
that we’ve finished bikeshedding the syntax details, but the
functionality seems OK.) Pavel Stehule, reviewed by Stephen Frost
and Tom Lane.
- Make a no-op ALTER EXTENSION UPDATE give just a NOTICE, not ERROR.
This seems a bit more user-friendly.
- Add backwards-compatible declarations of some core GIN support
functions. These are needed to support reloading dumps of 9.0
installations containing contrib/intarray or contrib/tsearch2.
Since not only regular dump/reload but binary upgrade would fail, it
seems worth the trouble to carry these stubs for awhile. Note that
the contrib opclasses referencing these functions will still work
fine, since GIN doesn’t actually pay any attention to the declared
signature of a support function.
- Fix bogus test for hypothetical indexes in
get_actual_variable_range(). That function was supposing that
indexoid == 0 for a hypothetical index, but that is not likely to be
true in any non-toy implementation of an index adviser, since
assigning a fake OID is the only way to know at EXPLAIN time which
hypothetical index got selected. Fix by adding a flag to
IndexOptInfo to mark hypothetical indexes. Back-patch to 9.0 where
get_actual_variable_range() was added. Gurjeet Singh
- Fix blatantly uninitialized variable in recent commit. Doesn’t
anybody around here pay attention to compiler warnings?
- Fix contrib/pg_trgm to have smoother updates from 9.0. Take care of
some loose ends in the update-from-unpackaged script, and apply some
ugly hacks to ensure that it produces the same catalog state as the
fresh-install script. Per discussion, this seems like a safer plan
than having two different catalog states that both call themselves
“pg_trgm 1.0″, even if it’s not immediately clear that the subtle
differences would ever matter. Also, fix the stub function
gin_extract_trgm() so that it works instead of just bleating.
Needed because this function will get called during a regular dump
and reload, if there are any indexes using its opclass. The user
won’t have an opportunity to update the extension till later, so
telling him to do so is unhelpful.
- Fix upgrade of contrib/btree_gist from 9.0. The initial version of
the update-from-unpackaged script neglected to include the <>
operators that were added to the opclasses during 9.1. To make this
script produce the same final state as the regular install script,
use the same ALTER OPERATOR FAMILY trick as in pg_trgm.
- Fix upgrade of contrib/intarray and contrib/unaccent from 9.0. Take
care of a couple of discrepancies between what you get from a fresh
install and what the first-draft update-from-unpackaged scripts
- Fix upgrade of contrib/xml2 from 9.0. Update script was being
sloppy about two functions that have been changed since 9.0.
- Fix tsmatchsel() to account properly for null rows. ts_typanalyze.c
computes MCE statistics as fractions of the non-null rows, which
seems fairly reasonable, and anyway changing it in released versions
wouldn’t be a good idea. But then ts_selfuncs.c has to account for
that. Failure to do so results in overestimates in columns with a
significant fraction of null documents. Back-patch to 8.4 where
this stuff was introduced. Jesper Krogh
- One more hack to make contrib upgrades from 9.0 match fresh 9.1
installs. intarray and tsearch2 both reference core support
functions in their GIN opclasses, and the signatures of those
functions changed for 9.1. We added backwards-compatible pg_proc
entries for the functions in order to allow 9.0 dump files to be
restored at all, but that hack leaves the opclasses pointing at
pg_proc entries different from what they’d point to if the contrib
modules were installed fresh in 9.1. To forestall any possibility
of future problems, fix the opclasses to match fresh installs via
the expedient of direct UPDATEs on pg_amproc in the
update-from-unpackaged scripts. (Yech … but the alternatives are
worse, or require far more effort than seems justified right now.)
Note: updating pg_amproc is sufficient because there will be no
pg_depend entries corresponding to these dependencies, since the
referenced functions are all pinned.
- Fix parallel pg_restore to handle comments on POST_DATA items
correctly. The previous coding would try to process all
SECTION_NONE items in the initial sequential-restore pass, which
failed if they were dependencies of not-yet-restored items. Fix by
postponing such items into the parallel processing pass once we have
skipped any non-PRE_DATA item. Back-patch into 9.0; the original
parallel-restore coding in 8.4 did not have this bug, so no need to
change it. Report and diagnosis by Arnd Hannemann.
- Un-break building with BTREE_BUILD_STATS. This has been broken for
awhile, but not clear it’s worth back-patching. Euler Taveira de
- Create the catalog infrastructure for foreign-data-wrapper handlers.
Add a fdwhandler column to pg_foreign_data_wrapper, plus HANDLER
options in the CREATE FOREIGN DATA WRAPPER and ALTER FOREIGN DATA
WRAPPER commands, plus pg_dump support for same. Also invent a new
pseudotype fdw_handler with properties similar to language_handler.
This is split out of the “FDW API” patch for ease of review; it’s
all stuff we will certainly need, regardless of any other details of
the FDW API. FDW handler functions will not actually get called
yet. In passing, fix some omissions and infelicities in
foreigncmds.c. Shigeru Hanada, Jan Urbanski, Heikki Linnakangas
- Implement an API to let foreign-data wrappers actually be
functional. This commit provides the core code and documentation
needed. A contrib module test case will follow shortly. Shigeru
Hanada, Jan Urbanski, Heikki Linnakangas
- Add contrib/file_fdw foreign-data wrapper for reading files via
COPY. This is both very useful in its own right, and an important
test case for the core FDW support. This commit includes a small
refactoring of copy.c to expose its option checking code as a
separately callable function. The original patch submission
duplicated hundreds of lines of that code, which seemed pretty
unmaintainable. Shigeru Hanada, reviewed by Itagaki Takahiro and
- Minor logic fix for new levenshtein implementation. Alexander
Simon Riggs pushed:
- Add version-sensitive SQL for psql when constraints NOT VALID. Bug
report and fix by Andres Freund.
- PITR can stop at a named restore point when recovery target = time
though must not update the last transaction timestamp. Plus comment
and message cleanup for recent named restore point. Fujii Masao,
minor changes by me.
- Additional required docs for pg_constraint catalog entry. Noted by
- Hot Standby feedback for avoidance of cleanup conflicts on standby.
Standby optionally sends back information about oldestXmin of
queries which is then checked and applied to the WALSender’s
proc->xmin. GetOldestXmin() is modified slightly to agree with
GetSnapshotData(), so that all backends on primary include WALSender
within their snapshots. Note this does nothing to change the
snapshot xmin on either master or standby. Feedback piggybacks on
the standby reply message. vacuum_defer_cleanup_age is no longer
used on standby, though parameter still exists on primary, since
some use cases still exist. Simon Riggs, review comments from Fujii
Masao, Heikki Linnakangas, Robert Haas
- Separate messages for standby replies and hot standby feedback.
Allow messages to be sent at different times, and greatly reduce the
frequency of hot standby feedback. Refactor to allow additional
- Make a hard state change from catchup to streaming mode. More
useful state change for monitoring purposes, plus a required change
for synchronous replication patch.
Robert Haas pushed:
- Delete stray word from comment.
- Avoid a few more SET DATA TYPE table rewrites. When the new type is
an unconstrained domain over the old type, we don’t need to rewrite
the table. Noah Misch and Robert Haas
- Rename max_predicate_locks_per_transaction. The new name,
max_pred_locks_per_transaction, is shorter. Kevin Grittner, per
- Assorted corrections to the patch to add WAL receiver replies. Per
reports from Fujii Masao.
- pg_ctl promote. Fujii Masao, reviewed by Robert Haas, Stephen
Frost, and Magnus Hagander.
- WAL receiver shouldn’t try to send a reply when dying. Per report
from, and discussion with, Fujii Masao.
- Remove rule to build sepgsql-regtest.pp. Instead, document how to
build it manually. Per discussion. KaiGai Kohei, with some
wordsmithing by me.
- Typo fixes. Thom Brown
- Add some words of caution to elog.c. Stephen Frost, somewhat
rewritten by me
- Add client_hostname field to pg_stat_activity. Peter Eisentraut,
reviewed by Steve Singer, Alvaro Herrera, and me.
- Prevent possible compiler warnings. Simon Riggs reports that
rnode.dbNode and rnode.spcNode were generating unused variable
warnings on gcc 4.4.3 with CFLAGS=-O1
Peter Eisentraut pushed:
- Allow make check in PL directories. Also add make check-world
target, and refactor pg_regress invocation code in makefiles a bit.
- Better support for thread-support flag detection with clang. When
testing the stderr produced by various thread-support flags, also
run a compilation in addition to a link, because clang warns on
certain flags when compiling but not when linking.
- Also process psqlrc when running psql -l. This was previously not
very useful, but with many people customizing the linestyle, it is
nice for a consistent appearance.
- Fix for warnings-free compilation with Python 3.2. The first
argument of PyEval_EvalCode() was changed from PyCodeObject* to
PyObject* because of PEP 384.
- Set psql client encoding from locale by default. Add a new libpq
connection option client_encoding (which includes the existing
PGCLIENTENCODING environment variable), which besides an encoding
name accepts a special value “auto” that tries to determine the
encoding from the locale in the client’s environment, using the
mechanisms that have been in use in initdb. psql sets this new
connection option to “auto” when running from a terminal and not
overridden by setting PGCLIENTENCODING. original code by Heikki
Linnakangas, with subsequent contributions by Jaime Casanova, Peter
Eisentraut, Stephen Frost, Ibrar Ahmed
- Invalidate PL/Python functions with composite type argument when the
type changes. The invalidation will cause the type information to
be refetched, and everything will work. Jan Urbański, reviewed by
Alvaro Herrera pushed:
- Make pg_upgrade compile again
- Cleanup ClusterInfo initialization in pg_upgrade
- Use $INDENT rather than indent throughout the pgindent code. This
allows the user to change the path to be used more easily. Also,
change URL in README.
- Convert Postgres arrays to Perl arrays on PL/perl input arguments.
More generally, arrays are turned in Perl array references, and row
and composite types are turned into Perl hash references. This is
done recursively, in a way that’s natural to every Perl programmer.
To avoid a backwards compatibility hit, the string representation of
each structure is also available if the function requests it.
Authors: Alexey Klyukin and Alex Hunsaker. Some code cleanups by
- pgindent run on plperl.c
- Use $INDENT instead of `which` to find the indent binary. Per
discussion after my commit o yesterday.
ITAGAKI Takahiro pushed:
- Export the external file reader used in COPY FROM as APIs. They are
expected to be used by extension modules like file_fdw. There are
no user-visible changes. Itagaki Takahiro. Reviewed and tested by
Kevin Grittner and Noah Misch.
- Add transaction-level advisory locks. They share the same locking
namespace with the existing session-level advisory locks, but they
are automatically released at the end of the current transaction and
cannot be released explicitly via unlock functions. Marko Tiikkaja,
reviewed by me.
- Fix an uninitialized field in DR_copy. Shigeru HANADA
Magnus Hagander pushed:
- Properly initialize variables. Kevin Grittner
== Rejected Patches (for now) ==
No one was disappointed this week
== Pending Patches ==
ITAGAKI Takahiro sent in another revision of the patch to implement
Stephen Frost and Robert Haas traded revisions of the patch to allow
logging the current role.
Marti Raudsepp sent in another revision of the patch which makes it so
a key lock is only acquired for columns that can be referenced.
Heikki Linnakangas sent in another revision of the patch to add a FDW
API for SQL/MED.
Thom Brown sent in a patch to make array_cat consistent with the ||
operator for arrays with respect to NULLs.
ITAGAKI Takahiro sent in another revision of the patch to implement
transaction-scope advisory locks.
Kevin Grittner sent in two revisions of a patch to fix an issue with
uninitialized structures from the SSI patch.
ITAGAKI Takahiro sent in another revision of the COPY ENCODING patch.
Per feedback from Heikki Linnakangas, Simon Riggs sent in a patch to
add a server_name parameter, plus mechanism to send info from
standby to master. This will help with synchronous replication.
YAMAMOTO Takashi sent in a patch to fix an issue with SSI.
Tom Lane sent in two versions of a patch to fix pg_tgrm’s
Mark Kirkwood sent in a WIP patch to allow the backend to constrain
temporary file space.
Tom Lane sent in a WIP patch to update KNN-GiST.
Magnus Hagander sent in another revision of the patch to include
Robert Haas sent in a patch to fix an issue with an assertion failure
on UNLOGGED VIEWs.
Simon Riggs sent in a patch to add server_name for synchronous
Simon Riggs sent in anothre revision of the patch to allow for
Radoslaw Smogura sent in a patch to add void_send and void_receive,
used in COPY … BINARY.
Comments are closed.