Reduce path length for locking leaf B-tree pages during insertion

In our B-tree implementation appropriate leaf page for new tuple
insertion is acquired using _bt_search() function.  This function always
returns leaf page locked in shared mode.  In order to obtain exclusive
lock, caller have to relock the page.

This commit makes _bt_search() function lock leaf page immediately in
exclusive mode when needed.  That removes unnecessary relock and, in
turn reduces lock contention for B-tree leaf pages.  Our experiments
on multi-core systems showed acceleration up to 4.5 times in corner
case.

Discussion: https://postgr.es/m/CAPpHfduAMDFMNYTCN7VMBsFg_hsf0GqiqXnt%2BbSeaJworwFoig%40mail.gmail.com
Author: Alexander Korotkov
Reviewed-by: Yoshikazu Imai, Simon Riggs, Peter Geoghegan

src/backend/access/nbtree/nbtinsert.c

@@ -216,23 +216,12 @@ top:
 
 	if (!fastpath)
 	{
-		/* find the first page containing this key */
+		/*
+		 * Find the first page containing this key.  Buffer returned by
+		 * _bt_search() is locked in exclusive mode.
+		 */
 		stack = _bt_search(rel, indnkeyatts, itup_scankey, false, &buf, BT_WRITE,
 						   NULL);
-
-		/* trade in our read lock for a write lock */
-		LockBuffer(buf, BUFFER_LOCK_UNLOCK);
-		LockBuffer(buf, BT_WRITE);
-
-		/*
-		 * If the page was split between the time that we surrendered our read
-		 * lock and acquired our write lock, then this page may no longer be
-		 * the right place for the key we want to insert.  In this case, we
-		 * need to move right in the tree.  See Lehman and Yao for an
-		 * excruciatingly precise description.
-		 */
-		buf = _bt_moveright(rel, buf, indnkeyatts, itup_scankey, false,
-							true, stack, BT_WRITE, NULL);
 	}
 
 	/*