| From: | Anthony Hsu <erwaman(at)gmail(dot)com> |
|---|---|
| To: | João Rito <joaovictor(dot)rito(at)gmail(dot)com> |
| Cc: | shihao zhong <zhong950419(at)gmail(dot)com>, Andrey Borodin <x4mmm(at)yandex-team(dot)ru>, Dmytro Astapov <dastapov(at)gmail(dot)com>, Álvaro Herrera <alvherre(at)kurilemu(dot)de>, pgsql-hackers(at)lists(dot)postgresql(dot)org, Fujii Masao <masao(dot)fujii(at)gmail(dot)com>, Heikki Linnakangas <hlinnaka(at)iki(dot)fi> |
| Subject: | Re: Set 1s WaitLatch timeout if standby limit has expired in ResolveRecoveryConflictWithBufferPin |
| Date: | 2026-11-15 23:42:26 |
| Message-ID: | CALQc50j7hQ4kRkKTy599VPFOTj97Bs+eZVKfrDNySTPbe5-Brw@mail.gmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
Hi João,
Thanks for the review and comments. I have addressed your comments:
> 1. The fix looks correct to me. ...
> 2. The #define ... would read better at the top of the file, next to the
other static state in standby.c.
Moved #define to top of the file.
> 3. While the limit stays expired, the startup process now sends the
> conflict signal to all backends once per second for as long as a pin
> is held. ... it may be worth saying in the commit message or the comment
that this is intended
Updated the commit message to mention this.
> 4. Tests
Added a test to 031_recovery_conflict.pl based on Dmytro's repro.
> 5. Commit message: it could add "Reported-by"/"Reviewed-by" trailers ...
> and a "Backpatch-through" line.
Added these lines. I put "Backpatch-through: 15" since I think this issue
could occur in older PG versions too, and PG15 is the oldest
still-supported version.
> 6. Process: CF entries 6444 and 6445 both point to this same thread,
> ... It might be good to close one of them so reviews are not split.
I just closed 6444 - we can use 6445 going forward.
Best,
Anthony
On Tue, Nov 10, 2026 at 3:48 PM João Rito <joaovictor(dot)rito(at)gmail(dot)com> wrote:
> Hi Anthony, Shihao,
>
> Thanks for the patch. I reviewed v3 (CF entry 6445) and tested it locally.
>
> What I tested
> - v3 applies cleanly with "git apply" on master at 9f191dce61e (no
> rebase needed). Build with --enable-cassert --enable-debug has no
> warnings, before or after the patch.
> - Reproducer: primary plus streaming standby (pg_basebackup -R),
> max_standby_streaming_delay = 2s, log_recovery_conflict_waits = on. I
> followed Dmytro's steps, with one change: session A is "COPY (SELECT
> t.id, repeat('y', 1000000) FROM bufferpin_test t) TO STDOUT | sleep",
> so it sits in ClientWrite holding the pin. After the VACUUM on the
> primary I waited 5s so the limit had expired and the startup process
> was in BufferCleanup, then opened session B (BEGIN; DECLARE c CURSOR
> ...; FETCH 5 FROM c), and then killed the "sleep" so session A goes
> away.
> - Without the patch (master 9f191dce61e): session B is never
> cancelled. 30s after session A was gone the startup process was still
> in BufferCleanup, B was still idle in transaction, and replay was
> behind by 34640 bytes. The log has only "recovery still waiting after
> 1000.189 ms: recovery conflict on buffer pin" and no "finished
> waiting" line.
> - With v3: in both runs session B got "FATAL: terminating connection
> due to conflict with recovery" while session A was still stuck in
> ClientWrite, i.e. before I released A and within my 2s check after B
> took its pin. The startup process resumed once A was released
> ("recovery finished waiting after 7059.663 ms: recovery conflict on
> buffer pin" in the first run, 7043.339 ms in the second), and the
> replay lag went back to 0.
> - make -C src/test/recovery check
> PROVE_TESTS=t/031_recovery_conflict.pl passes with the patch (18
> tests).
>
> Comments
> 1. The fix looks correct to me. ProcWaitForSignal() is just
> WaitLatch(MyLatch, WL_LATCH_SET | WL_EXIT_ON_PM_DEATH, 0, ...)
> followed by ResetLatch() and CHECK_FOR_INTERRUPTS(), so the new branch
> is the same call with WL_TIMEOUT and 1000 ms. After the timeout,
> LockBufferForCleanup() rechecks the pin count and calls
> ResolveRecoveryConflictWithBufferPin() again, where the limit_expired
> branch resends the signal. Other cases are unchanged: when the limit
> has not expired the STANDBY_TIMEOUT and STANDBY_DEADLOCK_TIMEOUT path
> is untouched, and with max_standby_streaming_delay = -1 (ltime == 0)
> the wait stays unbounded, which is intended.
> 2. The #define is placed between the large header comment and the
> function, so the comment block no longer sits directly above
> ResolveRecoveryConflictWithBufferPin(). It would read better at the
> top of the file, next to the other static state in standby.c.
> 3. While the limit stays expired, the startup process now sends the
> conflict signal to all backends once per second for as long as a pin
> is held. This is cheap, and a backend that cannot process the signal
> (like session A in ClientWrite) just gets it again, but it may be
> worth saying in the commit message or the comment that this is
> intended. It also means the deadlock-check part of the existing XXX
> comment is not affected, since the expired path never armed
> STANDBY_DEADLOCK_TIMEOUT.
> 4. Tests: the patch has none. A TAP test seems possible with the
> pattern above (two sessions with BackgroundPsql, the second one
> pinning after the limit has expired and the first one still holding),
> but the first backend has to be blocked somewhere it cannot process
> the signal, which is why I used ClientWrite. If you think it is too
> timing sensitive for the tree, a short comment saying how it was
> verified would still help. I can share my script if useful.
> 5. Commit message: it could add "Reported-by"/"Reviewed-by" trailers
> (Dmytro, Andrey, Shihao) and a "Backpatch-through" line. Since v3 does
> not touch proc.c or proc.h it should be easy to backpatch, as Shihao
> says, and Andrey and Dmytro saw it on 17 and 18.
> 6. Process: CF entries 6444 and 6445 both point to this same thread,
> and both are in "Needs review" for PG20-3. It might be good to close
> one of them so reviews are not split.
>
> Best regards,
> João Rito
>
> Em ter., 10 de nov. de 2026 às 20:46, shihao zhong
> <zhong950419(at)gmail(dot)com> escreveu:
> >
> > Hi Andrey,
> >
> > Sorry I missed your email. I closed 6445 because it is
> > duplicate with https://commitfest.postgresql.org/patch/6444/
> >
> > But I think this is an important issue and we need to fix it.
> >
> > Thanks,
> > Shihao
>
| Attachment | Content-Type | Size |
|---|---|---|
| v4-0001-Set-1s-WaitLatch-timeout-if-standby-limit-has-exp.patch | application/octet-stream | 7.8 KB |
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Jianqiao Han | 2026-11-16 00:00:00 | Re: [PATCH] Normalize numeric/bigint scientific notation after rounding |
| Previous Message | Zsolt Parragi | 2026-11-15 21:22:59 | Re: Re: Assertion failure in WAL summarizer |