Re: PG18 GIN parallel index build crash - invalid memory alloc request size

From: Tomas Vondra <tomas(at)vondra(dot)me>
To: Gregory Smith <gregsmithpgsql(at)gmail(dot)com>
Cc: PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: PG18 GIN parallel index build crash - invalid memory alloc request size
Date: 2025-11-04 19:27:52
Message-ID: bfde94f6-ade6-4129-846b-8724c1630050@vondra.me
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On 10/29/25 19:47, Tomas Vondra wrote:
> ...
> Unsurprisingly, there were a couple more palloc/repalloc calls (in
> ginPostingListDecodeAllSegments) that could fail with long TID lists
> produced when merging worker data. The attached v4 fixes this.
>
> However, I see this as a sign that allowing huge allocations is not the
> right way to fix this. The GIN code generally assumes, and I don't think
> reworking this in a bugfix seems a bit too invasive. And I'm not really
> certain this is the last place that could hit this.
>
> Another argument against 0001 is using more memory does not really help
> anything. It's not any faster or simpler. It's more like "let's use the
> memory we have" rather than "let's use the memory we need".
>
> So I'm planning to get rid of 0001, and fix that by 0002 or 0002+0003.
> That seems like a better and (unexpectedly) less invasive fix.
>

I ended up pushing 0002, for the reasons explained above. This fixes the
allocation issue simply by not needing too much memory. The 0003 is more
of an optimization, so I pushed that only to master.

regards

--
Tomas Vondra

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Andrew Dunstan 2025-11-04 19:32:41 Spacing of options in getopt_long processing
Previous Message Álvaro Herrera 2025-11-04 19:26:48 Re: [PATCH] Add pg_get_tablespace_ddl() function to reconstruct CREATE TABLESPACE statement