PostgreSQL Weekly News – June 19 2011
== PostgreSQL Weekly News – June 19 2011 ==
The first Commitfest of the 9.2 cycle has begun. Review a patch!
PostgreSQL 9.1 beta 2 is out. Test!
== PostgreSQL Product News ==
pgbouncer 1.4.2, a lightweight connection pooler for PostgreSQL, released.
== PostgreSQL Jobs for June ==
== PostgreSQL Local ==
NYCPUG presents “Data Storage at foursqare” June 22, 2011 6:30PM at
Spotify. RSVP and details below.
PG Session 2, on PostGIS, will be held on June 23rd in Paris. The CfP
CHAR(11), the PostgreSQL Conference on Clustering, High Availability
and Replication is now open for online registration and bookings.
July 11-12 2011 in Cambridge, UK.
PostgreSQL Conference China 2011 will be held in Guangzhou July
PDXPUG is hosting PgDay on Sunday, July 24, 2011, one day before
OSCON, in Portland, Oregon, USA. Details here:
Postgres Open 2011, a conference focused on disruption of the database
industry through PostgreSQL, will take place September 14-16, 2011 in
Chicago, Illinois at the Westin Michigan Avenue hotel.
PG-Day Denver 2011 will be held on Saturday, September 17th, 2011 at
the Auraria Campus near downtown Denver, Colorado.
PostgreSQL Conference West (#PgWest) will be held September 27th-30th,
2011 at the San Jose Convention center in San Jose, California, USA.
PostgreSQL Conference Europe 2011 will be held on October 18-21 in
pgbr will be in Sao Paulo, Brazil November 3-4, 2011.
PGConf.DE 2011 is the German-speaking PostgreSQL Conference and will
take place on November 11th in the Rheinisches Industriemuseum in
Oberhausen, Germany. Call for Papers is open.
== PostgreSQL in the News ==
Planet PostgreSQL: http://planet.postgresql.org/
PostgreSQL Weekly News is brought to you this week by David Fetter
Submit news and announcements by Sunday at 3:00pm Pacific time.
Please send English language ones to email@example.com, German language
to firstname.lastname@example.org, Italian language to email@example.com. Spanish language
== Reviews ==
Robert Haas reviewed KaiGai Kohei’s patch to rework DROP by making
‘missing_ok’ support get_object_address.
== Applied Patches ==
Robert Haas pushed:
- More foreign table documentation improvements. Shigeru Hanada, with
some additional wordsmithing by me
- Markup fix. Fujii Masao
- In ALTER EXTENSION docs, document left_type/right_type. Noted by
- Document GUC context for synchronous_standby_names/hot_standby_feedback.
- Add doc cross-reference to search_path discussion of
current_schemas(). Brendan Jurd
- Clarify that NATURAL without matching columns is like CROSS JOIN.
As suggested by Grzegorz Szpetkowski.
- Remove parentheses from mention of current_schemas function. This
is more consistent with what we do elsewhere, and hopefully avoids
creating the perception that current_schemas takes no arguments. As
suggested by Brendan Jurd
- Improve accuracy of ON_ERROR_STOP documentation. Per a gripe from
- Fix typo. Per Josh Kupershmidt and Tom Lane.
- Fix minor thinko in ProcGlobalShmemSize(). There’s no need to add
space for startupBufferPinWaitBufId, because it’s part of the
PROC_HDR object for which this function already allocates space.
This has been wrong for a while, but the only consequence is that
our shared memory allocation is increased by 4 bytes, so no
- Fix crash in CREATE UNLOGGED TABLE. The code that created the init
fork neglected to make sure that the relation was open at the smgr
level before attempting to invoke smgr. This didn’t happen every
time; only when the relcache entry was rebuilt along the way. Per
report from Garick Hamlin.
- Add overflow checks to int4 and int8 versions of generate_series().
The previous code went into an infinite loop after overflow. In
fact, an overflow is not really an error; it just means that the
current value is the last one we need to return. So, just arrange
to stop immediately when overflow is detected. Back-patch all the
Alvaro Herrera pushed:
- Expand warnings on locks acquired by CREATE INDEX CONCURRENTLY. The
previous wording wasn’t explicit enough, which could misled readers
into thinking that the locks acquired are more restricted in nature
than they really are. The resulting optimism can be damaging to
morale when confronted with reality, as has been observed in the
field. Greg Smith
- Fix aboriginal copy-paste mistake in error message. Spotted by
- Mention DROP TABLE as well as ALTER TABLE NO INHERIT … when
talking about how good they are in replacement of bulk DELETE in
partitioned setups. The original wording was a bit confusing. Per
an observation from David Wheeler.
- Fix grammatical mistake introduced by previous commit. Per note
from Tom Lane
- Fix typo
Bruce Momjian pushed:
- Document that bits to the right of the CIDR mask “should” be zero,
- In pg_upgrade, check there are no prepared transactions.
- In pg_upgrade, document that link mode has to have data directories
on the same file system, and that authentication should lock out
normal users. Per suggestsion from #postgresql irc channel.
Backpatch to 9.1.
- In pg_upgrade, clean up code layout in validateDirectoryOption().
Peter Eisentraut pushed:
- Move parse2.pl to parse.pl. We have a SCM, so we don’t need to keep
old versions of files around.
- Remove unused variable. The variable became obsolete in commit
68739ba856c52e6721d6cffec21f1bf0327a9a7b, but only gcc 4.6 shows the
- Add comment that attributes.is_nullable was removed from SQL
standard. We don’t have to remove the column if no one is bothered,
but it’s useful to comment on it in case someone looks for it in
newer standards versions.
- Allow psql \d tab completion to complete all relation kinds. This
matches what \d actually accepts.
- Document that psql’s \d also works with composite types. This has
always been true, it was just never documented.
- Start using flexible array members. Flexible array members are a
C99 feature that avoids “cheating” in the declaration of
variable-length arrays at the end of structs. With Autoconf
support, this should be transparent for older compilers. We start
with one use in gist.h because gcc 4.6 started to raise a warning
there. Over time, it can be expanded to other places in the source,
but they will likely need some review of sizeof and offsetof usage.
The current change in gist.h appears to be safe in this regard.
- Avoid compiler warnings due to possibly unused variables. gcc 4.6
complains about these because of the new option
-Wunused-but-set-variable which comes in with -Wall, so cast them to
void, which avoids the warning.
- Make external_pid_file world readable
- Capitalization fixes
Heikki Linnakangas pushed:
- Renumber 2PC resource managers so that compared to 9.0, predicate
lock rmgr is added to the end, and existing resource managers keep
their old ids. We’re not going to guarantee on-disk compatibility
for 2PC state files over major releases, but it seems better to
avoid changing the ids them anyway. It will help anyone who might
want to write external tools to inspect the state files to work with
files from different versions, if nothing else. Per complaint from
- Oops, forgot to change the order of entries in 2PC callback arrays
when I renumbered the resource managers. This should fix the
- Make non-MVCC snapshots exempt from predicate locking. Scans with
non-MVCC snapshots, like in REINDEX, are basically non-transactional
operations. The DDL operation itself might participate in SSI, but
there’s separate functions for that. Kevin Grittner and Dan Ports,
with some changes by me.
- The rolled-back flag on serializable xacts was pointless and
redundant with the marked-for-death flag. It was only set for a
fleeting moment while a transaction was being cleaned up at
rollback. All the places that checked for the rolled-back flag
should also check the marked-for-death flag, as both flags mean that
the transaction will roll back. I also renamed the marked-for-death
into “doomed”, which is a lot shorter name.
- pgindent run of recent SSI changes. Also, remove an unnecessary
#include. Kevin Grittner
- Update README-SSI. Add a section to describe the “dangerous
structure” that SSI is based on, as well as the optimizations about
relative commit times and read-only transactions. Plus a bunch of
other misc fixes and improvements. Dan Ports
Tom Lane pushed:
- Fix assorted issues with build and install paths containing spaces.
Apparently there is no buildfarm critter exercising this case after
all, because it fails in several places. With this patch, build,
install, check-world, and installcheck-world pass for me on OS X.
- Suppress -arch switches in the output of ExtUtils::Embed. We
previously found out that OS X’s standard perl installation tries to
put -arch switches into Perl link commands, evidently in hopes of
building universal binaries. But it doesn’t work to add such
switches in plperl’s link step if they weren’t being used earlier,
so this is basically unworkable. When using gcc the result is only
some warnings; but LLVM fails entirely, so this issue isn’t as
cosmetic as we originally thought. Hence, back-patch commit
d69a419e682c2d39c2355105a7e5e2b90357c8f0 into pre-9.0 branches.
- Fix oversights in pg_basebackup’s -z (compression) option. The
short-form -z switch didn’t work, for lack of telling getopt_long
about it; and even if specified long-form, it failed to do anything,
because the various tests elsewhere in the file would take
Z_DEFAULT_COMPRESSION (which is -1) as meaning “don’t compress”.
Per bug #6060 from Shigehiro Honda, though I editorialized on his
patch a bit.
- Fix failure to account for memory used by tuplestore_putvalues().
This oversight could result in a tuplestore using much more than the
intended amount of memory. It would only happen in a code path that
loaded a tuplestore via tuplestore_putvalues(), and many of those
won’t emit huge amounts of data; but cases such as holdable cursors
and plpgsql’s RETURN NEXT command could have the problem. The fix
ensures that the tuplestore will switch to write-to-disk mode when
it overruns work_mem. The potential overrun was finite, because we
would still count the space used by the tuple pointer array, so the
tuplestore code would eventually flip into write-to-disk mode
anyway. When storing wide tuples we would go far past the expected
work_mem usage before that happened; but this may account for the
lack of prior reports. Back-patch to 8.4, where
tuplestore_putvalues was introduced. Per bug #6061 from Yann
- Rework parsing of ConstraintAttributeSpec to improve NOT VALID
handling. The initial commit of the ALTER TABLE ADD FOREIGN KEY NOT
VALID feature failed to support labeling such constraints as
deferrable. The best fix for this seems to be to fold NOT VALID
into ConstraintAttributeSpec. That’s a bit more general than the
documented syntax, but it allows better-targeted syntax error
messages. In addition, do some mostly-but-not-entirely-cosmetic
code review for the whole NOT VALID patch.
- Use single quotes in preference to double quotes for protecting
pathnames. Per recommendation from Peter Eisentraut. Neither
choice is bulletproof, but this is the existing style and it does
help prevent unexpected environment variable substitution.
- Remove another no-longer-needed inclusion of predicate.h.
- Index tuple data arrays using Anum_xxx symbolic constants instead of
“i++”. We had already converted most places to this style, but this
patch gets the last few that were still doing it the old way. The
main advantage is that this exposes a greppable name for each target
column, rather than having to rely on comments (which a couple of
places failed to provide anyhow). Richard Hopkins, additional work
by me to clean up update_attstats() too
- Obtain table locks as soon as practical during pg_dump. For some
reason, when we (I) added table lock acquisition to pg_dump, we
didn’t think about making it happen as soon as possible after the
start of the transaction. What with subsequent additions, there was
actually quite a lot going on before we got around to that; which
sort of defeats the purpose. Rearrange the order of calls in
dumpSchema() to close the risk window as much as we easily can.
Back-patch to all supported branches.
- Don’t use “cp -i” in the example WAL archive_command. This is a
dangerous example to provide because on machines with GNU cp, it
will silently do the wrong thing and risk archive corruption.
Worse, during the 9.0 cycle somebody “improved” the discussion by
removing the warning that used to be there about that, and instead
leaving the impression that the command would work as desired on
most Unixen. It doesn’t. Try to rectify the damage by providing an
example that is safe most everywhere, and then noting that you can
try cp -i if you want but you’d better test that. In back-patching
this to all supported branches, I also added an example command for
Windows, which wasn’t provided before 9.0.
Simon Riggs pushed:
- Respect Hot Standby controls while recycling btree index pages.
Btree pages were recycled after VACUUM deletes all records on a page
and then a subsequent VACUUM occurs after the RecentXmin horizon is
reached. Using RecentXmin meant that we did not respond correctly to
the user controls provide to avoid Hot Standby conflicts and so
spurious conflicts could be generated in some workload combinations.
We now reuse pages only when we reach RecentGlobalXmin, which can be
much later in the presence of long running queries and is also
controlled by vacuum_defer_cleanup_age and hot_standby_feedback.
Noah Misch and Simon Riggs.
Andrew Dunstan pushed:
- Define FLEXIBLE_ARRAY_MEMBER for MSVC.
- Set FLEXIBLE_ARRAY_MEMBER to empty for MSVC. Per gripe from Tom
Lane. I have tested this with VC 2008, and assume it will work with
Magnus Hagander pushed:
- Remove redundant lib*dll.def rules from .gitignore. Since we now
have a global rule in the root .gitignore, there’s no need to keep
directory-specific ones as well. Noted by Peter Eisentraut
== Rejected Patches (for now) ==
Florian Pflug’s patch to implement ANY/ALL(array) OP EXPR. The right
way to do this is to provide commutators for the operators that don’t
have them yet.
== Pending Patches ==
Jaime Casanova and David Fetter traded patches which error out when
casting to or from a DOMAIN.
KaiGai Kohei sent in another revision of the patch to allow access to
the user-space access control vector for sepgsql.
Richard Hopkins sent in a patch to clarify the CreateComments
function, in the process removing an unneeded variable from it.
Florian Pflug sent in a patch to throw errors documenting the fact
that ANY/ALL(…) OP EXPR is not yet implemented.
KaiGai Kohei sent in another revision of the patch to add security
label support for shared database objects.
Merlin Moncure sent in another revision of the patch to clean up
psql’s describe.c by making all the SQL emitted by -E (ECHO_HIDDEN)
mode cut-and-pasteable with terminating semicolons. In passing,
remove trailing newlines.
Cedric Villemain sent in a set of experimental patches intended to get
estimates of what’s cached in order to make better execution plans.
Shigeru HANADA sent in two revisions of a patch to add SQL/MED
per-column GENERIC OPTIONs.
Alexander Korotkov sent in another revision of the patch to collect
frequency statistics for arrays.
Kevin Grittner sent in another revision of the patch to get SSI to
play nicer with 2PC.
Fujii Masao sent in another revision of the patch to enable cascading
Jun Ishiduka sent in a patch intended to help allow making an online
base backup from a hot standby.
Shigeru HANADA sent in a patch to enable per-column generic options
for foreign tables.
KaiGai Kohei sent in three revisions of a patch to rework DROP to
KaiGai Kohei sent in a patch to rework DROP by consolidating the
routines to handle DropStmt.
Robert Haas sent in a patch to remove a redundant store by
ProcGlobalShmemSize() of startupBufferPinWaitBufId.
Simon Riggs sent in a patch to create one-shot plans which get
executed immediately, the results discarded and the timing reported
back to the planner.
Bruce Momjian sent in two revisions of a patch to allow only apps with
the name binary-upgrade to connect to the cluster while it is in
binary upgrade mode, i.e. to disallow any apps not so named.
Mark Kirkwood and Cedric Villemain traded new revisions of the patch
to make it possible to constrain backend temporary file space.
Fujii Masao sent in a patch to fix a leak in dblink.
Noah Misch sent in another revision of the patch to avoid index
rebuilds for no-rewrite ALTER TABLE … ALTER TYPE operations.
Simon Riggs sent in three revisions of a patch to allow recycling
index pages in the case of hot standby.
Leonardo Francalanci sent in two more revisions of the patch to use
less space in xl_xact_commit.
Alvaro Herrera sent in three more revisions of the patch to allow
creating CHECK constraints as NOT VALID.
Alexander Korotkov sent in another revision of the patch to do fast
GiST index builds.
Martin Pihlak sent in a WIP patch to fix an infelicity with libpq and
Ahmed Shinwari sent in a patch to fix an issue with Windows when
connecting using SSPI auth.
Bernd Helmle and Joseph Adams traded patches to add a JSON type.
Alexander Korotkov and Florian Pflug traded new revisions of a patch
to create a validator for configuration files.
Radoslaw Smogura sent in two revisions of a patch to make debug builds
Radoslaw Smogura sent in a patch intended to provide a general
infrastructure for streaming data in and out of PostgreSQL.
Shigeru HANADA sent in another revision of the patch to handle
deadlock_timeout at times less than PGC_SIGHUP.
Robert Haas sent in three more revisions of a patch to make the
visibility map crash-safe.
Peter Geoghegan sent in two more revisions of a patch to wake when the
postmaster dies, both on win32 and *n*x.
Hitoshi Harada sent in another revision of the patch to optimize
aggregation by parameterzing.
Andrew Tipton sent in another revision of the patch to add GiST
support for BOX @> POINT queries.
Kevin Grittner sent in a doc patch describing SSI tuning points.
Brendan Jurd sent in another revision of a patch to make keywords in
Josh Kupershmidt sent in another revision of the patch to enable \dd
in psql to show constraint comments.
Radoslaw Smogura sent in another revision of a patch to fix some
infelicities in XPATH evaluation which could cause things to be
evaluated as null rather than empty string in XML context.
Jeff Davis and Robert Haas traded patches to refactor
heap_hot_search_buffer() so that index_getnext() can use it, and
modifies index_getnext() to do so.
Noah Misch sent in another revision of the patch to identify no-op
length coercions and not coerce them.
Kevin Grittner sent in a patch to fix some SSI issues with SLRU.
Comments are closed.