created programs\README.md

programs/Makefile

@@ -24,7 +24,7 @@
 # zstd : Command Line Utility, supporting gzip-like arguments
 # zstd32 : Same as zstd, but forced to compile in 32-bits mode
 # zstd_nolegacy : zstd without support of decompression of legacy versions
-# zstd-small: minimal zstd without dictBuilder and bench
+# zstd-small : minimal zstd without dictionary builder and benchmark
 # zstd-compress : compressor-only version of zstd
 # zstd-decompress : decompressor-only version of zstd
 # ##########################################################################

programs/README.md

@@ -0,0 +1,93 @@
+zstd - Command Line Interface
+================================
+
+Command Line Interface (CLI) can be created using the `make` command without any additional parameters.
+There are however other Makefile targets that create different variations of CLI:
+- `zstd` : default CLI supporting gzip-like arguments; includes dictionary builder, benchmark, and support for decompression of legacy zstd versions
+- `zstd32` : Same as `zstd`, but forced to compile in 32-bits mode
+- `zstd_nolegacy` : Same as `zstd` except of support for decompression of legacy zstd versions
+- `zstd-small` : CLI optimized for minimal size; without dictionary builder, benchmark, and support for decompression of legacy zstd versions
+- `zstd-compress` : compressor-only version of CLI; without dictionary builder, benchmark, and support for decompression of legacy zstd versions
+- `zstd-decompress` : decompressor-only version of CLI; without dictionary builder, benchmark, and support for decompression of legacy zstd versions
+
+
+#### Aggregation of parameters
+CLI supports aggregation of parameters i.e. `-b1`, `-e18`, and `-i1` can be joined into `-b1e18i1`. 
+
+
+#### Dictionary builder in Command Line Interface
+Zstd offers a training mode, which can be used to tune the algorithm for a selected
+type of data, by providing it with a few samples. The result of the training is stored
+in a file selected with the `-o` option (default name is `dictionary`),
+which can be loaded before compression and decompression.
+
+Using a dictionary, the compression ratio achievable on small data improves dramatically.
+These compression gains are achieved while simultaneously providing faster compression and decompression speeds.
+Dictionary work if there is some correlation in a family of small data (there is no universal dictionary). 
+Hence, deploying one dictionary per type of data will provide the greater benefits.
+Dictionary gains are mostly effective in the first few KB. Then, the compression algorithm
+will rely more and more on previously decoded content to compress the rest of the file.
+
+Usage of the dictionary builder and created dictionaries with CLI:
+1) Create the dictionary : `zstd --train FullPathToTrainingSet/* -o dictionaryName`
+2) Compress with dictionary: `zstd FILE -D dictionaryName`
+3) Decompress with dictionary: `zstd --decompress FILE.zst -D dictionaryName`
+
+
+
+#### Benchmark in Command Line Interface
+CLI includes in-memory compression benchmark module for zstd.
+The benchmark is conducted using given filenames which are read into memory and joined together.
+It makes benchmark more precise as it eliminates I/O overhead.
+Many filenames can be supplied as multiple parameters, parameters with wildcards or
+names of directories can be used as parameters with the `-r` option.
+
+The benchmark measures ratio, compressed size, compression and decompression speed.
+One can select compression levels starting from `-b` and ending with `-e`.
+The `-i` parameter selects minimal time used for each of tested levels.
+
+
+
+#### Usage of Command Line Interface
+The full list of options can be obtained with `-h` or `-H` parameter:
+```
+Usage :
+      zstd [args] [FILE(s)] [-o file]
+
+FILE    : a filename
+          with no FILE, or when FILE is - , read standard input
+Arguments :
+ -#     : # compression level (1-19, default:3)
+ -d     : decompression
+ -D file: use `file` as Dictionary
+ -o file: result stored into `file` (only if 1 input file)
+ -f     : overwrite output without prompting
+--rm    : remove source file(s) after successful de/compression
+ -k     : preserve source file(s) (default)
+ -h/-H  : display help/long help and exit
+
+Advanced arguments :
+ -V     : display Version number and exit
+ -v     : verbose mode; specify multiple times to increase log level (default:2)
+ -q     : suppress warnings; specify twice to suppress errors too
+ -c     : force write to standard output, even if it is the console
+ -r     : operate recursively on directories
+--ultra : enable levels beyond 19, up to 22 (requires more memory)
+--no-dictID : don't write dictID into header (dictionary compression)
+--[no-]check : integrity check (default:enabled)
+--test  : test compressed file integrity
+--[no-]sparse : sparse mode (default:enabled on file, disabled on stdout)
+
+Dictionary builder :
+--train ## : create a dictionary from a training set of files
+ -o file : `file` is dictionary name (default: dictionary)
+--maxdict ## : limit dictionary to specified size (default : 112640)
+ -s#    : dictionary selectivity level (default: 9)
+--dictID ## : force dictionary ID to specified value (default: random)
+
+Benchmark arguments :
+ -b#    : benchmark file(s), using # compression level (default : 1)
+ -e#    : test all compression levels from -bX to # (default: 1)
+ -i#    : minimum evaluation time in seconds (default : 3s)
+ -B#    : cut file into independent blocks of size # (default: no block)
+ ```

projects/README.md

@@ -4,11 +4,11 @@ projects for various integrated development environments (IDE)
 #### Included projects
 
 The following projects are included with the zstd distribution:
-- cmake - CMake project contributed by Artyom Dymchenko
+- `cmake` - CMake project contributed by Artyom Dymchenko
-- VS2005 - Visual Studio 2005 project
+- `VS2005` - Visual Studio 2005 project
-- VS2008 - Visual Studio 2008 project
+- `VS2008` - Visual Studio 2008 project
-- VS2010 - Visual Studio 2010 project (which also works well with Visual Studio 2012, 2013, 2015)
+- `VS2010` - Visual Studio 2010 project (which also works well with Visual Studio 2012, 2013, 2015)
-- build - command line scripts prepared for Visual Studio compilation without IDE
+- `build` - command line scripts prepared for Visual Studio compilation without IDE
 
 
 #### How to compile zstd with Visual Studio

tests/README.md

@@ -2,24 +2,24 @@ programs and scripts for automated testing of zstd
 ================================
 
 This directory contains the following programs and scripts:
-- datagen : Synthetic and parametrable data generator, for tests
+- `datagen` : Synthetic and parametrable data generator, for tests
-- fullbench  : Precisely measure speed for each zstd inner functions
+- `fullbench`  : Precisely measure speed for each zstd inner functions
-- fuzzer  : Test tool, to check zstd integrity on target platform
+- `fuzzer`  : Test tool, to check zstd integrity on target platform
-- paramgrill : parameter tester for zstd
+- `paramgrill` : parameter tester for zstd
-- test-zstd-speed.py : script for testing zstd speed difference between commits
+- `test-zstd-speed.py` : script for testing zstd speed difference between commits
-- test-zstd-versions.py : compatibility test between zstd versions stored on Github (v0.1+)
+- `test-zstd-versions.py` : compatibility test between zstd versions stored on Github (v0.1+)
-- zbufftest  : Test tool, to check ZBUFF integrity on target platform
+- `zbufftest`  : Test tool to check ZBUFF integrity on target platform
-- zstreamtest : Fuzzer test tool for zstd streaming API
+- `zstreamtest` : Fuzzer test tool for zstd streaming API
 
 
-#### test-zstd-versions.py - script for testing zstd interoperability between versions
+#### `test-zstd-versions.py` - script for testing zstd interoperability between versions
 
 This script creates `versionsTest` directory to which zstd repository is cloned.
 Then all taged (released) versions of zstd are compiled.
 In the following step interoperability between zstd versions is checked.
 
 
-#### test-zstd-speed.py - script for testing zstd speed difference between commits
+#### `test-zstd-speed.py` - script for testing zstd speed difference between commits
 
 This script creates `speedTest` directory to which zstd repository is cloned.
 Then it compiles all branches of zstd and performs a speed benchmark for a given list of files (the `testFileNames` parameter).