Add new documentation file begin_concurrent.md.

FossilOrigin-Name: fdbf97e611ed23f86dc0df9d53421fb04226af0d1c7171d55c11acd0dbe935f0

doc/begin_concurrent.md

@@ -0,0 +1,107 @@
+
+Begin Concurrent
+================
+
+## Overview
+
+Usually, SQLite allows at most one writer to proceed concurrently. The
+BEGIN CONCURRENT enhancement allows multiple writers to process write
+transactions simultanously if the database is in "wal" or "wal2" mode,
+although the system still serializes COMMIT commands.
+
+When a write-transaction is opened with "BEGIN CONCURRENT", actually 
+locking the database is deferred until a COMMIT is executed. This means
+that any number of transactions started with BEGIN CONCURRENT may proceed
+concurrently. The system uses optimistic page-level-locking to prevent
+conflicting concurrent transactions from being committed.
+
+When a BEGIN CONCURRENT transaction is committed, the system checks whether 
+or not any of the database pages that the transaction has read have been
+modified since the BEGIN CONCURRENT was opened. In other words - it asks 
+if the transaction being committed operates on a different set of data than
+all other concurrently executing transactions. If the answer is "yes, this
+transaction did not read or modify any data modified by any concurrent
+transaction", then the transaction is committed as normal. Otherwise, if the
+transaction does conflict, it cannot be committed and an SQLITE_BUSY_SNAPSHOT
+error is returned. At this point, all the client can do is ROLLBACK the
+transaction.
+
+If SQLITE_BUSY_SNAPSHOT is returned, messages are output via the sqlite3_log
+mechanism indicating the page and table or index on which the conflict
+occurred. This can be useful when optimizing concurrency.
+
+## Application Programming Notes
+
+In order to serialize COMMIT processing, SQLite takes a lock on the database
+as part of each COMMIT command and releases it before returning. At most one
+writer may hold this lock at any one time. If a writer cannot obtain the lock,
+it uses SQLite's busy-handler to pause and retry for a while:
+
+  <a href=https://www.sqlite.org/c3ref/busy_handler.html>
+      https://www.sqlite.org/c3ref/busy_handler.html
+  </a>
+
+If there is significant contention for the writer lock, this mechanism can be
+inefficient. In this case it is better for the application to use a mutex or
+some other mechanism that supports blocking to ensure that at most one writer
+is attempting to COMMIT a BEGIN CONCURRENT transaction at a time. This is
+usually easier if all writers are part of the same operating system process.
+
+If all database clients (readers and writers) are located in the same OS
+process, and if that OS is a Unix variant, then it can be more efficient to
+the built-in VFS "unix-excl" instead of the default "unix". This is because it
+uses more efficient locking primitives.
+
+The key to maximizing concurrency using BEGIN CONCURRENT is to ensure that
+there are a large number of non-conflicting transactions. In SQLite, each
+table and each index is stored as a separate b-tree, each of which is
+distributed over a discrete set of database pages. This means that:
+
+  * Two transactions that write to different sets of tables never 
+    conflict, and that
+
+  * Two transactions that write to the same tables or indexes only 
+    conflict if the values of the keys (either primary keys or indexed 
+    rows) are fairly close together. For example, given a large 
+    table with the schema:
+
+      <pre>     CREATE TABLE t1(a INTEGER PRIMARY KEY, b BLOB);</pre>
+
+    writing two rows with adjacent values for "a" probably will cause a
+    conflict (as the two keys are stored on the same page), but writing two
+    rows with vastly different values for "a" will not (as the keys will likly
+    be stored on different pages).
+
+Note that, in SQLite, if values are not explicitly supplied for an INTEGER
+PRIMARY KEY, as for example in:
+
+>
+     INSERT INTO t1(b) VALUES(&lt;blob-value>);
+
+then monotonically increasing values are assigned automatically. This is
+terrible for concurrency, as it all but ensures that all new rows are 
+added to the same database page. In such situations, it is better to
+explicitly assign random values to INTEGER PRIMARY KEY fields.
+
+This problem also comes up for non-WITHOUT ROWID tables that do not have an
+explicit INTEGER PRIMARY KEY column. In these cases each table has an implicit
+INTEGER PRIMARY KEY column that is assigned increasing values, leading to the
+same problem as omitting to assign a value to an explicit INTEGER PRIMARY KEY
+column.
+
+For both explicit and implicit INTEGER PRIMARY KEYs, it is possible to have
+SQLite assign values at random (instead of the monotonically increasing
+values) by writing a row with a rowid equal to the largest possible signed
+64-bit integer to the table. For example:
+
+     INSERT INTO t1(a) VALUES(9223372036854775807);
+
+Applications should take care not to malfunction due to the presence of such
+rows.
+
+The nature of some types of indexes, for example indexes on timestamp fields,
+can also cause problems (as concurrent transactions may assign similar
+timestamps that will be stored on the same db page to new records). In these
+cases the database schema may need to be rethought to increase the concurrency
+provided by page-level-locking.
+

manifest

@@ -1,5 +1,5 @@
-C Merge\slatest\strunk\schanges\sinto\sthis\sbranch.
+C Add\snew\sdocumentation\sfile\sbegin_concurrent.md.
-D 2019-01-02T16:08:14.536
+D 2019-01-11T15:06:39.819
 F .fossil-settings/empty-dirs dbb81e8fc0401ac46a1491ab34a7f2c7c0452f2f06b54ebb845d024ca8283ef1
 F .fossil-settings/ignore-glob 35175cdfcf539b2318cb04a9901442804be81cd677d8b889fcc9149c21f239ea
 F Makefile.in d8b254f8bb81bab43c340d70d17dc3babab40fcc8a348c8255881f780a45fee6
@@ -37,6 +37,7 @@ F configure e2b4021dcb8ee6f9a91e7f4011316957868e8065019ac994f68e27363120c272 x
 F configure.ac 3552d3aecade98a9d4b64bceb48ffb7726cbc85902efde956812942f060fbd0a
 F contrib/sqlitecon.tcl 210a913ad63f9f991070821e599d600bd913e0ad
 F doc/F2FS.txt c1d4a0ae9711cfe0e1d8b019d154f1c29e0d3abfe820787ba1e9ed7691160fcd
+F doc/begin_concurrent.md 4bee2c3990d1eb800f1ce3726a911292a8e4b889300b2ffd4b08d357370db299
 F doc/lemon.html 24956ab2995e55fe171e55bdd04f22b553957dc8bb43501dbb9311e30187e0d3
 F doc/pager-invariants.txt 27fed9a70ddad2088750c4a2b493b63853da2710
 F doc/vfs-shm.txt e101f27ea02a8387ce46a05be2b1a902a021d37a
@@ -1808,7 +1809,7 @@ F vsixtest/vsixtest.tcl 6a9a6ab600c25a91a7acc6293828957a386a8a93
 F vsixtest/vsixtest.vcxproj.data 2ed517e100c66dc455b492e1a33350c1b20fbcdc
 F vsixtest/vsixtest.vcxproj.filters 37e51ffedcdb064aad6ff33b6148725226cd608e
 F vsixtest/vsixtest_TemporaryKey.pfx e5b1b036facdb453873e7084e1cae9102ccc67a0
-P 48ca30f9d7817d87a5e9a069fdc51b1a34e00585f8a35771895dd743c7bfe07c 0465d2fc0d3f8beaa1b6b0bd7bd51d69a524f30d889c9402e7d02cc06164a310
+P 5bf212f1a754fde90af0af82dd76f636754829c7372e19df1fd4fa579f0926aa
-R 71a7bae1fe784545f177910e02fe6935
+R 7b08b63629f0fa82321f4b409f15d3b0
 U dan
-Z c6a4d4a7e90e89246e3383ca332fbf2a
+Z ca9bd1f59646fbf4e9e98b47096634fe

manifest.uuid

@@ -1 +1 @@
-5bf212f1a754fde90af0af82dd76f636754829c7372e19df1fd4fa579f0926aa
+fdbf97e611ed23f86dc0df9d53421fb04226af0d1c7171d55c11acd0dbe935f0