| From: | Ajin Cherian <itsajin(at)gmail(dot)com> |
|---|---|
| To: | Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> |
| Cc: | Zhijie Hou <houzhijie22(at)gmail(dot)com>, Bertrand Drouvot <bertranddrouvot(dot)pg(at)gmail(dot)com>, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, Bharath Rupireddy <bharath(dot)rupireddyforpostgres(at)gmail(dot)com>, Chao Li <li(dot)evan(dot)chao(at)gmail(dot)com>, Fujii Masao <masao(dot)fujii(at)gmail(dot)com>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org> |
| Subject: | Re: Add a hook for handling logical decoding messages on subscribers. |
| Date: | 2026-11-19 00:47:53 |
| Message-ID: | CAFPTHDaWcPj8PV69+SBn4NF+VwMYWiDtf3PwcWC0d2OWgB0x6g@mail.gmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
On Tue, Nov 17, 2026 at 6:06 AM Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> wrote:
>
> On Sun, Nov 15, 2026 at 9:13 AM Zhijie Hou <houzhijie22(at)gmail(dot)com> wrote:
> >
> I've attached the updated patches.
>
Hi Sawada-san,
I found an issue in 0002 with non-transactional messages.
I found this while reviewing the patch with Claude, which also helped
write the TAP test below.
In apply_handle_message(), the non-transactional branch commits the
handler's work locally but calls replorigin_xact_clear(false) first,
so the commit doesn't advance the replication origin. Until the
publisher receives feedback that this commit has been flushed, both
the origin and the slot's confirmed_flush point before the message.
Feedback lags, since apply workers default to synchronous_commit =
off. If the worker exits in that window, the message is decoded and
sent again on restart, and the handler runs again. The easiest way to
hit it is a non-transactional message followed by a transaction that
conflicts on the subscriber. Every retry re-applies the message before
failing on the conflict again. With wal_retrieve_retry_interval =
200ms I saw 42 handler invocations for a single message in 8 seconds.
Handlers are documented as needing to be idempotent, but repeating
once per retry seems avoidable, since normal transactions don't have
this problem.
The fix is to record the message's LSN as the origin LSN instead of
clearing it:
- replorigin_xact_clear(false);
+ replorigin_xact_state.origin_lsn = msg.lsn;
+ replorigin_xact_state.origin_timestamp = 0;
With origin_timestamp = 0, RecordTransactionCommit() uses the local
commit time, so the commit still doesn't inherit the previous
transaction's timestamp, which was the reason for the clear. With this
change the handler runs once across all retries, and 001_basic.pl
still passes.
Attached is a TAP test (099_nontx_redelivery.pl) that reproduces the
issue: it fails on v9 and passes with the change above.
Regards,
Ajin Cherian
Fujitsu Australia
| Attachment | Content-Type | Size |
|---|---|---|
| 099_nontx_redelivery.pl | application/octet-stream | 2.0 KB |
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Chao Li | 2026-11-19 01:05:22 | Re: [PATCH] Add hint to wal_level errors for replication slots and pg_log_standby_snapshot |
| Previous Message | Joao Rito | 2026-11-19 00:45:20 | [PATCH] doc: Fix autovacuum_max_workers and autovacuum_worker_slots descriptions |