Index corruption after proper shut down

From: Strahinja Kustudić <strahinjak(at)nordeus(dot)com>
To: "pgsql-admin(at)postgresql(dot)org" <pgsql-admin(at)postgresql(dot)org>
Subject: Index corruption after proper shut down
Date: 2013-11-15 10:44:41
Message-ID: CADKbJJWBF4cg5h2Ww+okz6QPU1egvvgp+QU70v-ZEHM+2+KCQw@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-admin

Hi all,

Last week we migrated 200+ of our servers from one rack to another and the
procedure was dead simple: power off server from the OS, unplug it, move it
to a different rack, plug it in and start it. The problem was that after
the boot some of the servers had corrupted indexes.

Servers are Dell PowerEdge R420 with H700 RAID controller with BBU, Centos
5.9 x64 with Postgres 9.1.9 running on two Intel 330 120GB SSDSC2CT120 (one
for data, and one for indexes) on XFS (noatime,nobarrier,noquota). Relevant
Postgres configuration is:

wal_level = minimal
fsync = on
wal_sync_method = fdatasync
full_page_writes = on
synchronous_commit = off
wal_buffers = -1

Also we disabled disk write cache on all drives with the MegaCli64 utility,
since the RAID controller should be the one caching since it has a BBU.

Does anyone have any idea, why could we get index corruption?

Thanks in advance

Regards,
Strahinja

Responses

Browse pgsql-admin by date

  From Date Subject
Next Message Della Rahawarin 2013-11-15 15:20:06 Error Installing PostgreSQL 9.3.1
Previous Message Karl Hafner 2013-11-15 09:33:21 Re: