+Back to the SQLite Home Page
++(This page was last modified on [lrange $rcsid 3 4] GMT) +
" + +puts { +SQLite stores each SQL table and index in a separate GDBM file. +The name of the GDBM file used to store a particular table is usually +just the table name with ".tbl" appended. +Consider an example:
+} + +proc Code {body} { + puts {}
+ regsub -all {&} [string trim $body] {\&} body
+ regsub -all {>} $body {\>} body
+ regsub -all {<} $body {\<} body
+ regsub -all {\(\(\(} $body {} body
+ regsub -all {\)\)\)} $body {} body
+ puts $body
+ puts {The example above creates a new SQL database with a single +table named tbl1 and a single index named idx1. +Three files were created for this database. tbl1.tbl stores +all the data for the tbl1 table and idx1.tbl stores +all the information needed by the index idx1. The remaining file +sqlite_master.tbl holds the data for the special +built-in table called sqlite_master. Every SQLite database +has an sqlite_master table. This table contains the schema +for the database. You can query the sqlite_master table +using ordinary SQL commands, but you cannot write to the +sqlite_master table.
+ +The GDBM file used to store an SQL table is usually +just the name of the table with .tbl appended. But there +are exceptions. First, the name of the table is converted to +all lower case letters before being used to construct the filename. +This is because SQL table names are not case sensitive but Unix filenames are. +Second, if the table name contains any characters other than +alphanumerics and underscores, the exceptional characters are encoded +as a single '+' sign. For example:
+} + +Code { +$ (((sqlite ex1))) +sqlite> (((create table 'Strange Table Name!'(a int, b char(30));))) +sqlite> .exit +$ (((ls ex1))) +idx1.tbl sqlite_master.tbl strange+table+name+.tbl tbl1.tbl +$ +} + +puts { +Each record of a GDBM file contains a key and a data. +Both key and data are arbitary bytes of any length. The information +from an SQL table is mapped into a GDBM file as follows:
+ +The GDBM key for each record of an SQL table file is a +randomly chosen integer. The key size thus depends on the size +of an integer on the host computer. (Typically this means "4 bytes".) +
+ +If the SQL table contains N columns, then the data entry +for each record begins with N integers. Each integer is the +offset in bytes from the beginning of the GDBM data to the +start of the data for the corresponding column. If the column +contains a NULL value, then its corresponding integer will +be zero. All column data is stored as null-terminated ASCII +text strings.
+ +Consider a simple example:
+} + +Code { +$ (((rm -rf ex1))) +$ (((sqlite ex1))) +sqlite> (((create table t1(a int, b text, c text);))) +sqlite> (((insert into t1 values(10,NULL,'hello!');))) +sqlite> (((insert into t1 values(-11,'this is','a test');))) +sqlite> (((.exit))) +$ (((gdbmdump ex1/t1.tbl))) +key : 223100ae "1.. +data : 0c000000 10000000 18000000 2d313100 74686973 ............-11.this + 20697300 61207465 737400 is.a test. + +key : a840e996 .@.. +data : 0c000000 00000000 0f000000 31300068 656c6c6f ............10.hello + 2100 !. + +$ +} + +puts { +In the example above, we have created a new table named t1 +that contains two records. The gdbmdump program is used to +dump the contents of the t1 GDBM file +in a human readable format. The source code to gdbmdump +is included with the SQLite distribution. Just type "make gdbmdump" +to build it.
+ +We can see in the dump of t1 that each record +is a separate GDBM entry with a 4-byte random key. The keys +shown are for a single sample run. If you try +this experiment yourself, you will probably get completely different +keys.
+ +
Because the t1 table contains 3 columns, the data part of +each record begins with 3 integers. In both records of the example, +the first integer +has the value 12 since the beginning of the data for the first column +begins on the 13th byte of the record. You can see how each column's +data is stored as a null-terminated string. For the second record, +observe that the offset integer is zero for the second column. This +indicates that the second column contains NULL data.
+ +Each SQL index is also represented using a single GDBM file. +There is one entry in the GDBM file for each unique SQL key in the +table that is being indexed. The GDBM key is an +arbitrary length null-terminated string which is SQL key that +is used by the index. The data is a list of integers that correspond +to GDBM keys of entries in data table that have the corresponding +SQL key.
+ +To illustrate, we will create an index on the example table +shown above, and add a new entry to this table that has a duplicate +SQL key.
+} + +Code { +$ (((sqlite ex1))) +sqlite> (((create index i1 on t1(a);))) +sqlite> (((insert into t1 values(10,'another','record');))) +sqlite> (((.exit))) +$ (((gdbmdump ex1/t1.tbl))) +key : 223100ae "1.. +data : 0c000000 10000000 18000000 2d313100 74686973 ............-11.this + 20697300 61207465 737400 is.a test. + +key : a840e996 .@.. +data : 0c000000 00000000 0f000000 31300068 656c6c6f ............10.hello + 2100 !. + +key : c19e3119 ..1. +data : 0c000000 0f000000 17000000 31300061 6e6f7468 ............10.anoth + 65720072 65636f72 6400 er.record. +$ +} + +puts { +We added the new record to the t1 table because we wanted to +have two records with the same value on column a since that +column is used by the i1 index. You can see from the dump +above that the new t1 record is assigned another random +GDBM key.
+ +Now let's look at a dump of the index file.
+} + +Code { +$ (((gdbmdump ex1/i1.tbl))) +key : 313000 10. +data : a840e996 c19e3119 .@....1. + +key : 2d313100 -11. +data : 223100ae "1.. + +$ +} + +puts { +The GDBM file for the index contains only two records because +the t1 table contains only two distinct values for +column a. You can see that the GDBM keys for each record +are just the text values for a columns of table t1. +The data for each record of the index is a list of integers +where each integer is the GDBM key for an entry in the t1 +table that has the corresponding value for the a column.
+} + +puts { +
+Back to the SQLite Home Page
+
}
puts "This page was last modified on [lrange $rcsid 3 4] GMT
"
puts "The SQLite source code was last modifed on [exec cat last_change] GMT"
puts {
@@ -27,56 +28,77 @@ There are currently no known errors in the code.
If you find bugs or missing features, please submit a comment to the SQLite mailing list.
} +} puts {SQLite is an SQL database built atop the +
SQLite is an SQL database engine built on top of the GDBM library. -The SQLite distribution includes both a interactive command-line -access program (sqlite) and a C library (libsqlite.a) +SQLite includes a standalone command-line +access program (sqlite) +and a C library (libsqlite.a) that can be linked -with a C/C++ program to provide SQL database access without having -to rely on an external RDBMS.
+with a C/C++ program to provide SQL database access without +an separate RDBMS. -The C interface to SQLite is very simple, consisting of only -four functions, a single opaque data structure, and a handful of -constants that define error return codes. -See c_interface.html for details. -A Tcl interface -to SQLite is also available and is included in the source tree. -Documentation on the Tcl interface is pending. -Interfaces for perl and python may be supplied in future releases.
+The standalone program sqlite can be used -to interactively create, update and/or query an SQLite database. -The sources to the sqlite program are part of the source tree and -can be used as an example of how to interact with the SQLite C -library. For more information on the sqlite program, -see sqlite.html.
- -A history of changes to SQLite is found -here.
- -SQLite now implements most of the SQL language. -The following are the known limitations:
- --
New features are still being added to the SQLite code base. -Nevertheless, the code appears to be stable and relatively -bug-free. At least one large database has -be loaded into SQLite and appears to work.
+A change history is available online. +There are currently no known bugs or memory leaks +in the library. Gcov +is used to verify test coverage. The test suite currently exercises +all code except for a few areas which are unreachable or which are +only reached when malloc() fails. The code has been tested +for memory leaks and is found to be clean.
-SQLite has so far been tested only on RedHat 6.0 Linux. But we -know of no reason why it will not work on any other Unix platform, -or on Windows95/98/NT.
++Among the SQL features that SQLite does not currently implement are:
+ ++
The following documentation is currently available:
+ +The SQLite source code is 35% comment. These comments are +another important source of information.
} puts { @@ -104,15 +126,41 @@ build directory, run configure from the build directory and then type "make". For example:} +puts {-$ tar xzf sqlite.tar.gz ;# Unpacks into directory named "sqlite" -$ mkdir bld ;# Create a separate build directory +$ tar xzf sqlite.tar.gz Unpacks into directory named "sqlite" +$ mkdir bld Create a separate build directory $ cd bld $ ../sqlite/configure -$ make ;# Builds "sqlite" and "libsqlite.a" -$ make test ;# Optional: run regression tests +$ make Builds "sqlite" and "libsqlite.a" +$ make test Optional: run regression tests
Download the source archive and compile the sqlite +program as described above. The type:
+ ++} puts {+bash$ sqlite ~/newdb Directory ~/newdb created automatically +sqlite> create table t1( + ...> a int, + ...> b varchar(20) + ...> c text + ...> ); End each SQL statement with a ';' +sqlite> insert into t1 + ...> values(1,'hi','y''all'); +sqlite> select * from t1; +1|hello|world +sqlite> .mode columns Special commands begin with '.' +sqlite> .header on Type ".help" for a list of commands +sqlite> select * from t1; +a b c +------ ------- ------- +1 hi y'all +sqlite> .exit +base$ +
The default output mode is "list". In +list mode, each record of a query result is written on one line of +output and each field within that record is separated by a specific +separator string. The default separator is a pipe symbol ("|"). +List mode is especially useful when you are going to send the output +of a query to another program (such as AWK) for additional processing.
} + +Code { +sqlite> (((.mode list))) +sqlite> (((select * from tbl1;))) +hello|10 +goodbye|20 +sqlite> +} + +puts { +You can use the ".separator" dot command to change the separator +for list mode. For example, to change the separator to a comma and +a space, you could do this:
} + +Code { +sqlite> (((.separator ", "))) +sqlite> (((select * from tbl1;))) +hello, 10 +goodbye, 20 +sqlite> +} +In "line" mode, each field in a record of the database is shown on a line by itself. Each line consists of the field name, an equal sign and the field data. Successive records are @@ -173,20 +203,14 @@ separated by a blank line. Here is an example of line mode output:
} Code { -sql> (((.mode line))) -sql> (((select * from tbl1;))) +sqlite> (((.mode line))) +sqlite> (((select * from tbl1;))) one = hello two = 10 one = goodbye two = 20 -sql> -} - -puts { -Line mode used to be the default mode setting. But after some -experience using the utility, it was decided that "list" mode made -a better default and so now the default mode is "list".
+sqlite> } puts { @@ -194,13 +218,13 @@ puts { data aligned in columns. For example:} Code { -sql> (((.mode column))) -sql> (((select * from tbl1;))) +sqlite> (((.mode column))) +sqlite> (((select * from tbl1;))) one two ---------- ---------- hello 10 goodbye 20 -sql> +sqlite> } puts { @@ -209,13 +233,13 @@ Data that is too wide to fit in a column is truncated. You can adjust the column widths using the ".width" command. Like this:} Code { -sql> (((.width 12 6))) -sql> (((select * from tbl1;))) +sqlite> (((.width 12 6))) +sqlite> (((select * from tbl1;))) one two ------------ ------ hello 10 goodbye 20 -sql> +sqlite> } puts { @@ -231,40 +255,11 @@ examples above, the column labels are on. To turn them off you could do this:} Code { -sql> (((.header off))) -sql> (((select * from tbl1;))) +sqlite> (((.header off))) +sqlite> (((select * from tbl1;))) hello 10 goodbye 20 -sql> -} - -puts { -The third output mode supported by sqlite is called "list". In -list mode, each record of a query result is written on one line of -output and each field within that record is separated by a specific -separator string. The default separator is a pipe symbol ("|"). -List mode is especially useful when you are going to send the output -of a query to another program (such as AWK) for additional processing.
} - -Code { -sql> (((.mode list))) -sql> (((select * from tbl1;))) -hello|10 -goodbye|20 -sql> -} - -puts { -You can use the ".separator" dot command to change the separator -for list mode. For example, to change the separator to a comma and -a space, you could do this:
} - -Code { -sql> (((.separator ", "))) -sql> (((select * from tbl1;))) -hello, 10 -goodbye, 20 -sql> +sqlite> } puts { @@ -286,11 +281,11 @@ query results will be written to that file. Use ".output stdout" to begin writing to standard output again. For example:} Code { -sql> (((.mode list))) -sql> (((.separator |))) -sql> (((.output test_file_1.txt))) -sql> (((select * from tbl1;))) -sql> (((.exit))) +sqlite> (((.mode list))) +sqlite> (((.separator |))) +sqlite> (((.output test_file_1.txt))) +sqlite> (((select * from tbl1;))) +sqlite> (((.exit))) $ (((cat test_file_1.txt))) hello|10 goodbye|20 @@ -310,10 +305,10 @@ can enter ".tables". } Code { -sql> (((.tables))) +sqlite> (((.tables))) tbl1 tbl2 -sql> +sqlite> } puts { @@ -341,20 +336,20 @@ CREATE statement used to make that table and all if its indices. We have:} Code { -sql> (((.schema))) +sqlite> (((.schema))) create table tbl1(one varchar(10), two smallint) CREATE TABLE tbl2 ( f1 varchar(30) primary key, f2 text, f3 real ) -sql> (((.schema tbl2))) +sqlite> (((.schema tbl2))) CREATE TABLE tbl2 ( f1 varchar(30) primary key, f2 text, f3 real ) -sql> +sqlite> } puts { @@ -378,6 +373,60 @@ ORDER BY type DESC, nameThe %s in the query above is replaced by the argument to ".schema", of course.
+Use the ".dump" command to convert the entire contents of a +database into a single ASCII text file. This file can be converted +back into a database by piping it back into sqlite.
+ +A good way to make an archival copy of a database is this:
+} + +Code { +$ (((echo '.dump' | sqlite ex1 | gzip -c >ex1.dump.gz))) +} + +puts { +This generates a file named ex1.dump.gz that contains everything +you need to reconstruct the database at a later time, or on another +machine. To reconstruct the database, just type:
+} + +Code { +$ (((zcat ex1.dump.gz | sqlite ex2))) +} + +puts { +The text format used is the same as used by +PostgreSQL, so you +can also use the .dump command to export an SQLite database +into a PostgreSQL database. Like this:
+} + +Code { +$ (((createdb ex2))) +$ (((echo '.dump' | sqlite ex1 | psql ex2))) +} + +puts { +You can almost (but not quite) go the other way and export +a PostgreSQL database into SQLite using the pg_dump utility. +Unfortunately, when pg_dump writes the database schema information, +it uses some SQL syntax that SQLite does not understand. +So you cannot pipe the output of pg_dump directly +into sqlite. +But if you can recreate the +schema separately, you can use pg_dump with the -a +option to list just the data +of a PostgreSQL database and import that directly into SQLite.
+} + +Code { +$ (((sqlite ex3The ".explain" dot command can be used to set the output mode @@ -390,8 +439,8 @@ instructions that would have been used to execute the SQL command are returned like a query result. For example:
} Code { -sql> (((.explain))) -sql> (((explain delete from tbl1 where two<20;))) +sqlite> (((.explain))) +sqlite> (((explain delete from tbl1 where two<20;))) addr opcode p1 p2 p3 ---- ------------ ----- ----- ------------------------------------- 0 ListOpen 0 0