Re: Allow reading LSN written by walreciever, but not flushed yet

From: Fujii Masao <masao(dot)fujii(at)oss(dot)nttdata(dot)com>
To: Andrey Borodin <x4mmm(at)yandex-team(dot)ru>, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
Cc: PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Allow reading LSN written by walreciever, but not flushed yet
Date: 2025-05-13 11:13:00
Message-ID: 35d5d9bb-f0ef-43f4-b860-1d0d4fbbfe5b@oss.nttdata.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On 2025/05/13 0:47, Andrey Borodin wrote:
> Moved off from "Small fixes needed by high-availability tools"
>
>> On 12 May 2025, at 01:33, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
>>
>> On Fri, May 2, 2025 at 6:30 PM Andrey Borodin <x4mmm(at)yandex-team(dot)ru> wrote:
>>>
>>> 3. Allow reading LSN written by walreciever, but not flushed yet
>>>
>>> Problem: if we have synchronous_standby_names = ANY(node1,node2), node2 might be ahead of node1 by flush LSN, but before by written LSN. If we do a failover we choose node2 instead of node1 and loose data recently committed with synchronous_commit=remote_write.

In this case, doesn't the flush LSN typically catch up to the write LSN on node2
after a few seconds? Even if the walreceiver exits while there's still written
but unflushed WAL, it looks like WalRcvDie() ensures everything is flushed by
calling XLogWalRcvFlush(). So, isn't it safe to rely on the flush LSN when selecting
the most advanced node? No?

>>> Caveat: we already have a function pg_last_wal_receive_lsn(), which in fact returns flushed LSN, not written. I propose to add a new function which returns LSN actually written. Internals of this function are already implemented (GetWalRcvWriteRecPtr()), but unused.

GetWalRcvWriteRecPtr() returns walrcv->writtenUpto, which can move backward
when the walreceiver restarts. This behavior is OK for your purpose?

Regards,

--
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message David Rowley 2025-05-13 11:26:32 Re: Add explicit initialization for all PlannerGlobal fields
Previous Message Amit Kapila 2025-05-13 11:04:05 Re: Improve monitoring of shared memory allocations