database/sqlite
1
0
Fork 0
mirror of https://github.com/sqlite/sqlite.git synced 2025-07-29 08:01:23 +03:00
Code Activity

Refactor and expand the worker1 docs, consolidating them into the top of their file instead of scattered around the internals. Accommodate an API change from yesterday in demo-oo1.js.

Browse Source 
FossilOrigin-Name: 0a65747047322b7b585e281ac275e437ce3f46e1d06105c19117213929a906ad
This commit is contained in:
stephan
2022-08-25 11:39:12 +00:00
parent 407f75378e
commit 9afff9f3c5
6 changed files with 303 additions and 176 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
ext/wasm/api/sqlite3-api-oo1.js
View File

@ -412,7 +412,7 @@ self.sqlite3ApiBootstrap.initializers.push(function(sqlite3){
Stmt.getColumnNames() to append the column names to it
(regardless of whether the query produces any result
rows). If the query has no result columns, this value is
unchanged.
unchanged. (TODO: support this in execMulti() as well.)
The following options to execMulti() are _not_ supported by
this method (they are simply ignored):

447
ext/wasm/api/sqlite3-api-worker1.js
View File

@ -22,13 +22,14 @@
*/
/**
This function implements a Worker-based wrapper around SQLite3 OO
API #1, colloquially known as "Worker API #1".
sqlite3.initWorker1API() implements a Worker-based wrapper around
SQLite3 OO API #1, colloquially known as "Worker API #1".
In order to permit this API to be loaded in worker threads without
automatically registering onmessage handlers, initializing the
worker API requires calling initWorker1API(). If this function
is called from a non-worker thread then it throws an exception.
worker API requires calling initWorker1API(). If this function is
called from a non-worker thread then it throws an exception. It
must only be called once per Worker.
When initialized, it installs message listeners to receive Worker
messages and then it posts a message in the form:
@ -50,8 +51,275 @@
Promise-based wrapper for this API (`sqlite3-worker1-promiser.js`)
is more comfortable to use in that regard.
The documentation for the input and output worker messages for
this API follows...
TODO: hoist the message API docs from deep in this code to here.
====================================================================
Common message format...
Each message posted to the worker has an operation-independent
envelope and operation-dependent arguments:
```
{
type: string, // one of: 'open', 'close', 'exec', 'config-get'
messageId: OPTIONAL arbitrary value. The worker will copy it as-is
into response messages to assist in client-side dispatching.
dbId: a db identifier string (returned by 'open') which tells the
operation which database instance to work on. If not provided, the
first-opened db is used. This is an "opaque" value, with no
inherently useful syntax or information. Its value is subject to
change with any given build of this API and cannot be used as a
basis for anything useful beyond its one intended purpose.
args: ...operation-dependent arguments...
// the framework may add other properties for testing or debugging
// purposes.
}
```
Response messages, posted back to the main thread, look like:
```
{
type: string. Same as above except for error responses, which have the type
'error',
messageId: same value, if any, provided by the inbound message
dbId: the id of the db which was operated on, if any, as returned
by the corresponding 'open' operation.
result: ...operation-dependent result...
}
```
====================================================================
Error responses
Errors are reported messages in an operation-independent format:
```
{
type: 'error',
messageId: ...as above...,
dbId: ...as above...
result: {
operation: type of the triggering operation: 'open', 'close', ...
message: ...error message text...
errorClass: string. The ErrorClass.name property from the thrown exception.
input: the message object which triggered the error.
stack: _if available_, a stack trace array.
}
}
```
====================================================================
"config-get"
This operation fetches the serializable parts of the sqlite3 API
configuration.
Message format:
```
{
type: "config-get",
messageId: ...as above...,
args: currently ignored and may be elided.
}
```
Response:
```
{
type: 'config',
messageId: ...as above...,
result: {
persistentDirName: path prefix, if any, of persistent storage.
An empty string denotes that no persistent storage is available.
bigIntEnabled: bool. True if BigInt support is enabled.
persistenceEnabled: true if persistent storage is enabled in the
current environment. Only files stored under persistentDirName
will persist, however.
}
}
```
====================================================================
"open" a database
Message format:
```
{
type: "open",
messageId: ...as above...,
args:{
filename [=":memory:" or "" (unspecified)]: the db filename.
See the sqlite3.oo1.DB constructor for peculiarities and transformations,
persistent [=false]: if true and filename is not one of ("",
":memory:"), prepend sqlite3.capi.sqlite3_web_persistent_dir()
to the given filename so that it is stored in persistent storage
_if_ the environment supports it. If persistent storage is not
supported, the filename is used as-is.
}
}
```
Response:
```
{
type: 'open',
messageId: ...as above...,
result: {
filename: db filename, possibly differing from the input.
dbId: an opaque ID value which must be passed in the message
envelope to other calls in this API to tell them which db to
use. If it is not provided to future calls, they will default to
operating on the first-opened db. This property is, for API
consistency's sake, also part of the contaning message envelope.
Only the `open` operation includes it in the `result` property.
persistent: true if the given filename resides in the
known-persistent storage, else false. This determination is
independent of the `persistent` input argument.
}
}
```
====================================================================
"close" a database
Message format:
```
{
type: "close",
messageId: ...as above...
dbId: ...as above...
args: OPTIONAL: {
unlink: if truthy, the associated db will be unlinked (removed)
from the virtual filesystems. Failure to unlink is silently
ignored.
}
}
```
If the dbId does not refer to an opened ID, this is a no-op. The
inability to close a db (because it's not opened) or delete its
file does not trigger an error.
Response:
```
{
type: 'close',
messageId: ...as above...,
result: {
filename: filename of closed db, or undefined if no db was closed
}
}
```
====================================================================
"exec" SQL
All SQL execution is processed through the exec operation. It offers
most of the features of the oo1.DB.exec() method, with a few limitations
imposed by the state having to cross thread boundaries.
Message format:
```
{
type: "exec",
messageId: ...as above...
dbId: ...as above...
args: string (SQL) or {... see below ...}
}
```
Response:
```
{
type: 'exec',
messageId: ...as above...,
dbId: ...as above...
result: {
input arguments, possibly modified. See below.
}
}
```
The arguments are in the same form accepted by oo1.DB.exec(), with
the exceptions noted below.
A function-type args.callback property cannot cross
the window/Worker boundary, so is not useful here. If
args.callback is a string then it is assumed to be a
message type key, in which case a callback function will be
applied which posts each row result via:
postMessage({type: thatKeyType,
rowNumber: 1-based-#,
row: theRow,
columnNames: anArray
})
And, at the end of the result set (whether or not any result rows
were produced), it will post an identical message with
(row=undefined, rowNumber=null) to alert the caller than the result
set is completed. Note that a row value of `null` is a legal row
result for certain arg.rowMode values.
(Design note: we don't use (row=undefined, rowNumber=undefined) to
indicate end-of-results because fetching those would be
indistinguishable from fetching from an empty object unless the
client used hasOwnProperty() (or similar) to distinguish "missing
property" from "property with the undefined value". Similarly,
`null` is a legal value for `row` in some case , whereas the db
layer won't emit a result value of `undefined`.)
The callback proxy must not recurse into this interface. An exec()
call will type up the Worker thread, causing any recursion attempt
to wait until the first exec() is completed.
The response is the input options object (or a synthesized one if
passed only a string), noting that options.resultRows and
options.columnNames may be populated by the call to db.exec().
*/
self.sqlite3ApiBootstrap.initializers.push(function(sqlite3){
@ -85,10 +353,15 @@ sqlite3.initWorker1API = function(){
Internal helper for managing Worker-level state.
*/
const wState = {
/** First-opened db is the default for future operations when no
dbId is provided by the client. */
defaultDb: undefined,
/** Sequence number of dbId generation. */
idSeq: 0,
/** Map of DB instances to dbId. */
idMap: new WeakMap,
xfer: [/*Temp holder for "transferable" postMessage() state.*/],
/** Temp holder for "transferable" postMessage() state. */
xfer: [],
open: function(opt){
const db = new DB(opt.filename);
this.dbs[getDbId(db)] = db;
@ -122,6 +395,8 @@ sqlite3.initWorker1API = function(){
},
/** Map of DB IDs to DBs. */
dbs: Object.create(null),
/** Fetch the DB for the given id. Throw if require=true and the
id is not valid, else return the db or undefined. */
getDb: function(id,require=true){
return this.dbs[id]
|| (require ? toss("Unknown (or closed) DB ID:",id) : undefined);
@ -154,37 +429,6 @@ sqlite3.initWorker1API = function(){
on error.
*/
const wMsgHandler = {
/**
Proxy for the DB constructor. Expects to be passed a single
object or a falsy value to use defaults:
{
filename [=":memory:" or "" (unspecified)]: the db filename.
See the sqlite3.oo1.DB constructor for peculiarities and transformations,
persistent [=false]: if true and filename is not one of ("",
":memory:"), prepend
sqlite3.capi.sqlite3_web_persistent_dir() to the given
filename so that it is stored in persistent storage _if_ the
environment supports it. If persistent storage is not
supported, the filename is used as-is.
}
The response object looks like:
{
filename: db filename, possibly differing from the input.
dbId: an opaque ID value which must be passed in the message
envelope to other calls in this API to tell them which db to
use. If it is not provided to future calls, they will default
to operating on the first-opened db.
persistent: true if the given filename resides in the
known-persistent storage, else false. This determination is
independent of the `persistent` input argument.
}
*/
open: function(ev){
const oargs = Object.create(null), args = (ev.args || Object.create(null));
if(args.simulateError){ // undocumented internal testing option
@ -205,28 +449,11 @@ sqlite3.initWorker1API = function(){
rc.dbId = getDbId(db);
return rc;
},
/**
Proxy for DB.close(). ev.args may be elided or an object with
an `unlink` property. If that value is truthy then the db file
(if the db is currently open) will be unlinked from the virtual
filesystem, else it will be kept intact, noting that unlink
failure is ignored. The result object is:
{
filename: db filename _if_ the db is opened when this
is called, else the undefined value
dbId: the ID of the closed b, or undefined if none is closed
}
It does not error if the given db is already closed or no db is
provided. It is simply does nothing useful in that case.
*/
close: function(ev){
const db = getMsgDb(ev,false);
const response = {
filename: db && db.filename,
dbId: db && getDbId(db)
filename: db && db.filename
};
if(db){
wState.close(db, ((ev.args && 'object'===typeof ev.args)
@ -234,52 +461,7 @@ sqlite3.initWorker1API = function(){
}
return response;
},
/**
Proxy for oo1.DB.exec() which expects a single argument of type
string (SQL to execute) or an options object in the form
expected by exec(). The notable differences from exec()
include:
- The default value for options.rowMode is 'array' because
the normal default cannot cross the window/Worker boundary.
- A function-type options.callback property cannot cross
the window/Worker boundary, so is not useful here. If
options.callback is a string then it is assumed to be a
message type key, in which case a callback function will be
applied which posts each row result via:
postMessage({type: thatKeyType,
rowNumber: 1-based-#,
row: theRow,
columnNames: anArray
})
And, at the end of the result set (whether or not any result
rows were produced), it will post an identical message with
(row=undefined, rowNumber=null) to alert the caller than the
result set is completed. Note that a row value of `null` is
a legal row result for certain `rowMode` values.
(Design note: we don't use (row=undefined, rowNumber=undefined)
to indicate end-of-results because fetching those would be
indistinguishable from fetching from an empty object unless the
client used hasOwnProperty() (or similar) to distinguish
"missing property" from "property with the undefined value".
Similarly, `null` is a legal value for `row` in some case ,
whereas the db layer won't emit a result value of `undefined`.)
The callback proxy must not recurse into this interface, or
results are undefined. (It hypothetically cannot recurse
because an exec() call will be tying up the Worker thread,
causing any recursion attempt to wait until the first
exec() is completed.)
The response is the input options object (or a synthesized
one if passed only a string), noting that
options.resultRows and options.columnNames may be populated
by the call to db.exec().
*/
exec: function(ev){
const rc = (
'string'===typeof ev.args
@ -287,6 +469,8 @@ sqlite3.initWorker1API = function(){
if('stmt'===rc.rowMode){
toss("Invalid rowMode for 'exec': stmt mode",
"does not work in the Worker API.");
}else if(!rc.sql){
toss("'exec' requires input SQL.");
}
const db = getMsgDb(ev);
if(rc.callback || Array.isArray(rc.resultRows)){
@ -330,21 +514,7 @@ sqlite3.initWorker1API = function(){
}
return rc;
}/*exec()*/,
/**
Returns a JSON-friendly form of a _subset_ of sqlite3.config,
sans any parts which cannot be serialized. Because we cannot,
from here, distingush whether or not certain objects can be
serialized, this routine selectively copies certain properties
rather than trying JSON.stringify() and seeing what happens
(the results are horrid if the config object contains an
Emscripten module object).
In addition to the "real" config properties, it sythesizes
the following:
- persistenceEnabled: true if persistent dir support is available,
else false.
*/
'config-get': function(){
const rc = Object.create(null), src = sqlite3.config;
[
@ -355,6 +525,7 @@ sqlite3.initWorker1API = function(){
rc.persistenceEnabled = !!sqlite3.capi.sqlite3_web_persistent_dir();
return rc;
},
/**
TO(RE)DO, once we can abstract away access to the
JS environment's virtual filesystem. Currently this
@ -391,58 +562,12 @@ sqlite3.initWorker1API = function(){
wState.xfer.push(response.buffer.buffer);
return response;**/
}/*export()*/,
toss: function(ev){
toss("Testing worker exception");
}
}/*wMsgHandler*/;
/**
UNDER CONSTRUCTION!
A subset of the DB API is accessible via Worker messages in the
form:
{ type: apiCommand,
args: apiArguments,
dbId: optional DB ID value (else uses a default db handle),
messageId: optional client-specific value
}
As a rule, these commands respond with a postMessage() of their
own. The responses always have a `type` property equal to the
input message's type and an object-format `result` part. If
the inbound object has a `messageId` property, that property is
always mirrored in the result object, for use in client-side
dispatching of these asynchronous results. For example:
{
type: 'open',
messageId: ...copied from inbound message...,
dbId: ID of db which was opened,
result: {
dbId: repeat of ^^^, for API consistency's sake,
filename: ...,
persistent: false
},
...possibly other framework-internal/testing/debugging info...
}
Exceptions thrown during processing result in an `error`-type
event with a payload in the form:
{ type: 'error',
dbId: DB handle ID,
[messageId: if set in the inbound message],
result: {
operation: "inbound message's 'type' value",
message: error string,
errorClass: class name of the error type,
input: ev.data
}
}
The individual APIs are documented in the wMsgHandler object.
*/
self.onmessage = function(ev){
ev = ev.data;
let result, dbId = ev.dbId, evType = ev.type;

4
ext/wasm/demo-oo1.js
View File

@ -136,8 +136,10 @@
db.exec({
sql: "select a, twice(a), twice(''||a) from t order by a desc limit 3",
columnNames: columnNames,
rowMode: 'stmt',
callback: function(row){
log("a =",row.get(0), "twice(a) =", row.get(1), "twice(''||a) =",row.get(2));
log("a =",row.get(0), "twice(a) =", row.get(1),
"twice(''||a) =",row.get(2));
}
});
log("Result column names:",columnNames);

4
ext/wasm/testing-worker1-promiser.js
View File

@ -44,7 +44,7 @@
w.onerror = (event)=>error("worker.onerror",event);
return w;
},
//debug: (...args)=>console.debug('worker debug',...args),
debug: 1 ? undefined : (...args)=>console.debug('worker debug',...args),
onunhandled: function(ev){
error("Unhandled worker message:",ev.data);
},
@ -95,7 +95,7 @@
.assert(r.persistent
? (dbFilename!==r.filename)
: (dbFilename==r.filename))
.assert(ev.dbId)
.assert(ev.dbId === r.dbId)
.assert(ev.messageId)
.assert(promiserConfig.dbId === ev.dbId);
}).then(runTests2);

18
manifest
View File

@ -1,5 +1,5 @@
C Change\sDB.exec()\srowMode\sdefault\sfrom\s'stmt'\sto\s'array',\sper\s/chat\sdiscussion.\sAdd\sDB.exec()\srowMode\soption\sfor\sfetching\sa\sspecific\scolumn\sby\sname.\sAdd\sresult\scolumn\snames\sto\sworker1\sexec()\scallback\sinterface,\sas\sthere's\sotherwise\sno\sway\sto\sget\sthat\sinfo\sfrom\sa\sworker.
D 2022-08-24T20:57:37.430
C Refactor\sand\sexpand\sthe\sworker1\sdocs,\sconsolidating\sthem\sinto\sthe\stop\sof\stheir\sfile\sinstead\sof\sscattered\saround\sthe\sinternals.\sAccommodate\san\sAPI\schange\sfrom\syesterday\sin\sdemo-oo1.js.
D 2022-08-25T11:39:12.535
F .fossil-settings/empty-dirs dbb81e8fc0401ac46a1491ab34a7f2c7c0452f2f06b54ebb845d024ca8283ef1
F .fossil-settings/ignore-glob 35175cdfcf539b2318cb04a9901442804be81cd677d8b889fcc9149c21f239ea
F LICENSE.md df5091916dbb40e6e9686186587125e1b2ff51f022cc334e886c19a0e9982724
@ -483,10 +483,10 @@ F ext/wasm/api/post-js-footer.js b64319261d920211b8700004d08b956a6c285f3b0bba814
F ext/wasm/api/post-js-header.js 0e853b78db83cb1c06b01663549e0e8b4f377f12f5a2d9a4a06cb776c003880b
F ext/wasm/api/sqlite3-api-cleanup.js 1a12e64060c2cb0defd34656a76a9b1d7ed58459c290249bb31567c806fd44de
F ext/wasm/api/sqlite3-api-glue.js 67ca83974410961953eeaa1dfed3518530d68381729ed1d27f95122f5baeabd3
F ext/wasm/api/sqlite3-api-oo1.js 324b2f6817ff3711b59bd9505157f7a91fe319249d3b8b525c8254427c10504a
F ext/wasm/api/sqlite3-api-oo1.js a207d53bcc11955cc8f844aa59b30caaba8f57573222f62010628cfa4b1f4444
F ext/wasm/api/sqlite3-api-opfs.js 011799db398157cbd254264b6ebae00d7234b93d0e9e810345f213a5774993c0
F ext/wasm/api/sqlite3-api-prologue.js 2d5c5d3355f55eefe51922cec5bfedbec0f8300db98a17685ab7a34a03953c7a
F ext/wasm/api/sqlite3-api-worker1.js 11a9e8b22147d948e338b25d21697178b4414dc0578fc9613aa5fc4bfe62f208
F ext/wasm/api/sqlite3-api-worker1.js 38db6c3d77798a0ef8be78ae6d421ef144e2e9602cdabdd7a93c7fcb7a2d449f
F ext/wasm/api/sqlite3-wasi.h 25356084cfe0d40458a902afb465df8c21fc4152c1d0a59b563a3fba59a068f9
F ext/wasm/api/sqlite3-wasm.c 0d81282eaeff2a6e9fc5c28a388c5c5b45cf25a9393992fa511ac009b27df982
F ext/wasm/common/SqliteTestUtil.js eb96275bed43fdb364b7d65bcded0ca5e22aaacff120d593d1385f852f486247
@ -494,7 +494,7 @@ F ext/wasm/common/emscripten.css 3d253a6fdb8983a2ac983855bfbdd4b6fa1ff267c28d695
F ext/wasm/common/testing.css 572cf1ffae0b6eb7ca63684d3392bf350217a07b90e7a896e4fa850700c989b0
F ext/wasm/common/whwasmutil.js 41b8e097e0a9cb07c24c0ede3c81b72470a63f4a4efb07f75586dc131569f5ae
F ext/wasm/demo-oo1.html 75646855b38405d82781246fd08c852a2b3bee05dd9f0fe10ab655a8cffb79aa
F ext/wasm/demo-oo1.js 04e947b64a36ed8d6fe6d5e3ccee16ffc8b4461dd186e84f4baf44d53cc3aa72
F ext/wasm/demo-oo1.js 77b837b0fe13b542cee893c8caf84009482986bd43cf775197dfeb1e62ec0a2b
F ext/wasm/fiddle/emscripten.css 3d253a6fdb8983a2ac983855bfbdd4b6fa1ff267c28d69513dd6ef1f289ada3f
F ext/wasm/fiddle/fiddle-worker.js bccf46045be8824752876f3eec01c223be0616ccac184bffd0024cfe7a3262b8
F ext/wasm/fiddle/fiddle.html 550c5aafce40bd218de9bf26192749f69f9b10bc379423ecd2e162bcef885c08
@ -511,7 +511,7 @@ F ext/wasm/scratchpad-opfs-worker2.js 5f2237427ac537b8580b1c659ff14ad2621d169404
F ext/wasm/sqlite3-worker1-promiser.js 92b8da5f38439ffec459a8215775d30fa498bc0f1ab929ff341fc3dd479660b9
F ext/wasm/sqlite3-worker1.js 0c1e7626304543969c3846573e080c082bf43bcaa47e87d416458af84f340a9e
F ext/wasm/testing-worker1-promiser.html 6eaec6e04a56cf24cf4fa8ef49d78ce8905dde1354235c9125dca6885f7ce893
F ext/wasm/testing-worker1-promiser.js f4b0895b612606d04ae371d03a9ffe9ffa94a2a840da6e92742b2adf86f0783c
F ext/wasm/testing-worker1-promiser.js c62b5879339eef0b21aebd9d75bc125c86530edc17470afff18077f931cb704a
F ext/wasm/testing1.html 528001c7e32ee567abc195aa071fd9820cc3c8ffc9c8a39a75e680db05f0c409
F ext/wasm/testing1.js 2def7a86c52ff28b145cb86188d5c7a49d5993f9b78c50d140e1c31551220955
F ext/wasm/testing2.html a66951c38137ff1d687df79466351f3c734fa9c6d9cce71d3cf97c291b2167e3
@ -2009,8 +2009,8 @@ F vsixtest/vsixtest.tcl 6a9a6ab600c25a91a7acc6293828957a386a8a93
F vsixtest/vsixtest.vcxproj.data 2ed517e100c66dc455b492e1a33350c1b20fbcdc
F vsixtest/vsixtest.vcxproj.filters 37e51ffedcdb064aad6ff33b6148725226cd608e
F vsixtest/vsixtest_TemporaryKey.pfx e5b1b036facdb453873e7084e1cae9102ccc67a0
P 509f8839201ec1ea4863bd31493e6c29a0721ca6340755bb96656b828758fea7
R 4d9ca8433788ec1804950abdc80ba467
P 1bb37e5c477b9eb098362f74a45a55be23d450fe45cdff58c1cbff08b5b3998f
R 317eb3fcfe686c9323e52473ace36982
U stephan
Z aac659391ac05e725777e807ed0e05f4
Z 2516659ac708c3f3e5c0bf51001da2da
# Remove this line to create a well-formed Fossil manifest.

2
manifest.uuid
View File

@ -1 +1 @@
1bb37e5c477b9eb098362f74a45a55be23d450fe45cdff58c1cbff08b5b3998f
0a65747047322b7b585e281ac275e437ce3f46e1d06105c19117213929a906ad