CopyMultiInsertInfo.bufferedBytes is not increased in binary format cases

From: Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com>
To: PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: CopyMultiInsertInfo.bufferedBytes is not increased in binary format cases
Date: 2025-09-08 23:25:47
Message-ID: CAD21AoDvNTmmSL5_tzFLYTdXk7N26raaLs7sUYXoLZsz+HscAg@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

When using the multi insertion method during COPY FROM, we buffer
input tuples until the number of tuples exceeds MAX_BUFFERED_TUPLES
(=1000) or the total buffered tuple size exceeds MAX_BUFFERED_BYTES
(=64kB). We use line_buf.len as input tuple size as follows and add it
to bufferedBytes:

/* Add this tuple to the tuple buffer */
CopyMultiInsertInfoStore(&multiInsertInfo,
resultRelInfo, myslot
cstate->line_buf.len,
cstate->cur_lineno);

However, while it works in text or csv cases, in binary format cases,
bufferedBytes is not increased as we don't use line_buf. So we always
buffer 1000 tuples regardless of tuple size. Is it intentional
behavior? I could not find any related discussion on the original
discussion threads.

Regards,

--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com

Browse pgsql-hackers by date

  From Date Subject
Next Message Melanie Plageman 2025-09-08 23:44:02 Re: Checkpointer write combining
Previous Message Melanie Plageman 2025-09-08 22:28:46 Re: eliminate xl_heap_visible to reduce WAL (and eventually set VM on-access)