Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

From: Dilip Kumar <dilipbalaut(at)gmail(dot)com>
To: Andres Freund <andres(at)anarazel(dot)de>
Cc: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Robert Haas <robertmhaas(at)gmail(dot)com>, Justin Pryzby <pryzby(at)telsasoft(dot)com>, Ashutosh Sharma <ashu(dot)coek88(at)gmail(dot)com>, Maciek Sakrejda <m(dot)sakrejda(at)gmail(dot)com>, Bruce Momjian <bruce(at)momjian(dot)us>, Alvaro Herrera <alvherre(at)alvh(dot)no-ip(dot)org>, Andrew Dunstan <andrew(at)dunslane(dot)net>, Heikki Linnakangas <hlinnaka(at)iki(dot)fi>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Thomas Munro <thomas(dot)munro(at)gmail(dot)com>
Subject: Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints
Date: 2022-08-05 04:52:35
Message-ID: CAFiTN-t1rJiwUdfruaxh1LbJOfZov7s4Mc8Jgcwe9ngtRKOB-w@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Fri, Aug 5, 2022 at 5:36 AM Andres Freund <andres(at)anarazel(dot)de> wrote:
>
> Hi,
>
> On August 4, 2022 4:20:16 PM PDT, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> >Yeah, the assumption that P_NEW would automatically match the source block
> >was making me itchy too. An explicit test-and-elog seems worth the
> >cycles.
>
> Is there a good reason to rely on P_NEW at all?

I think there were 2 arguments for which we used bufmgr instead of
smgrextend for the destination database

1) (Comment from Andres) The big benefit would be that the *target*
database does not have to be written out / shared buffer is
immediately populated. [1]
2) (Comment from Robert) We wanted to avoid writing new code which
bypasses the shared buffers.

[1]https://stream.boxwatch.ru/message-id/20210905202800.ji4fnfs3xzhvo7l6%40alap3.anarazel.de

Both from an efficiency and robustness POV it seems like it'd be
better to use smgrextend to bulk extend just after smgrcreate() and
then fill s_b u using RBM_ZERO (or whatever it is called). That bulk
smgrextend would later be a good point to use fallocate so the FS can
immediately size the file correctly, without a lot of pointless writes
of zeroes.

Yeah okay, so you mean since we already know the nblocks in the source
file so we can directly do smgrextend in bulk before the copy loop and
then we can just copy block by block using bufmgr with proper blkno
instead of P_NEW. Yeah I think this looks optimized to me and this
will take care of the above 2 points I mentioned that we will still
have the target database pages in the shared buffers and we are not
bypassing the shared buffers also.

--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Dilip Kumar 2022-08-05 05:13:46 Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints
Previous Message John Naylor 2022-08-05 04:47:23 Re: fix typos