803 Commits

Author	SHA1	Message	Date
Paul Cruz	e929d3b787	added priority decision making for adapt compression level	2017-07-21 09:26:35 -07:00
Paul Cruz	9259c7afa4	semi working version that stabilizes	2017-07-20 18:45:33 -07:00
Paul Cruz	82e488770c	fixed bug where writeSize could be zero	2017-07-20 16:38:02 -07:00
Paul Cruz	a19916425d	reworked adaptCompressionLevel to only account for completion information	2017-07-20 16:19:16 -07:00
Nick Terrell	7d3ac0710d	[linux] Update patches for v3	2017-07-20 13:33:55 -07:00
Paul Cruz	7ab758a640	changed how completion is actually sampled	2017-07-20 10:53:51 -07:00
Stella Lau	13a01ffb27	Fix off-by-one in size calculations	2017-07-19 17:24:09 -07:00
Stella Lau	2427a154cb	Minor refactoring	2017-07-19 16:56:28 -07:00
Paul Cruz	dcf609f835	make adaptCompressionLevel oscillate less	2017-07-19 16:36:33 -07:00
Paul Cruz	2a22c7915e	call ZSTD_compressBegin() once	2017-07-19 16:00:54 -07:00
Paul Cruz	6767abe652	fixing error when file size is multiple of job size (in which case, the srcSize of the last job is 0)	2017-07-19 14:54:15 -07:00
Stella Lau	030264ca51	Experiment with integrating ZSTD_count with findBestMatch	2017-07-19 14:14:26 -07:00
Paul Cruz	42382c1216	added some debug statements, adjusted end condition	2017-07-19 13:30:07 -07:00
Paul Cruz	5a85c57e30	set up new calculations compression completion progress	2017-07-19 11:47:17 -07:00
Paul Cruz	f1ac518b59	split compression into smaller blocks	2017-07-19 11:23:40 -07:00
Paul Cruz	338951cd48	moved compression adapt to avoid warning	2017-07-19 10:23:46 -07:00
Paul Cruz	4497ecf297	change compression level only right before actually performing compression. When waiting, only update waiting statistics.	2017-07-19 10:14:00 -07:00
Paul Cruz	e11bf55d0b	added mechanism for measuring how much of a job has been created	2017-07-19 10:10:47 -07:00
Paul Cruz	559ea4ff25	split up read process into smaller chunks	2017-07-19 09:59:17 -07:00
Paul Cruz	6119cd2164	added additional print for help menu	2017-07-19 09:43:17 -07:00
Stella Lau	4352e09cb0	Avoid recounting match lengths with ZSTD_count	2017-07-18 18:35:25 -07:00
Stella Lau	1fa223859f	Switch to using ZSTD_count instead of function pointer	2017-07-18 18:05:10 -07:00
Paul Cruz	3d7f1afadd	changed createCCtx() to split into initialization and creation	2017-07-18 17:32:36 -07:00
Paul Cruz	2c4e4ddc50	added mutex for stats struct	2017-07-18 15:55:58 -07:00
Paul Cruz	ad66faf16a	added progress check for filewriting, put important shared data behind mutex when being read from/written to	2017-07-18 15:23:11 -07:00
Stella Lau	19258f51c1	Make the meaning of LDM_MEMORY_USAGE consistent across tables	2017-07-18 14:25:39 -07:00
Paul Cruz	a34bc30237	setting up basic readme	2017-07-18 13:31:02 -07:00
Paul Cruz	29c36cf051	rename completion variable, split up fwrite operations in order to track progress	2017-07-18 13:30:29 -07:00
Paul Cruz	ae47eab2fd	changed test cases to use -s setting on the diffs	2017-07-18 12:58:50 -07:00
Stella Lau	fc41a87964	Experiment with using a lag when hashing	2017-07-17 18:13:09 -07:00
Paul Cruz	5af04c57b0	change parameters for compression level adapt	2017-07-17 17:59:50 -07:00
Paul Cruz	b3c9e02bb6	added signal to other threads whenever error occurs	2017-07-17 15:34:58 -07:00
Stella Lau	a00e406231	Remove version archive	2017-07-17 15:17:32 -07:00
Stella Lau	15a041adbf	Add function to get valid entries only from table	2017-07-17 15:16:58 -07:00
Paul Cruz	6be22f1f84	swap buffers instead of copying memory over	2017-07-17 14:39:10 -07:00
Paul Cruz	708238e07e	open file outside of adaptCCtx, pass to the output thread	2017-07-17 14:01:13 -07:00
Stella Lau	4bb42b02c1	Add basic chaining table	2017-07-17 11:53:54 -07:00
Paul Cruz	044e40db5a	removed freeCCtx() calls from createCCtx() so that it is not called twice during errors	2017-07-17 11:19:23 -07:00
Paul Cruz	50ce4eaeb6	added error detection for pthread initialization, added compression completion measurement, fixed const values	2017-07-17 10:12:44 -07:00
Stella Lau	ca300ce6e0	Decouple hash table from compression function	2017-07-14 17:17:00 -07:00
Paul Cruz	1ab3f06f00	updated tests to use different seeds when executing different tests	2017-07-14 16:29:29 -07:00
Stella Lau	6e443b4960	Move hash table access for own functions	2017-07-14 14:27:55 -07:00
Stella Lau	2d8e6c6608	Add more statistics	2017-07-14 12:31:01 -07:00
Stella Lau	55f960e8db	Add percentages to offset histogram	2017-07-14 11:00:20 -07:00
Stella Lau	4db7f12ef3	Add offset histogram	2017-07-14 10:52:03 -07:00
Paul Cruz	0c8b9436b7	removed goto statements for the most part	2017-07-13 16:38:20 -07:00
Stella Lau	175a6c6029	[ldm] Minor refactoring	2017-07-13 16:16:31 -07:00
Stella Lau	361c06df75	Add min/max offset to stats	2017-07-13 15:29:41 -07:00
Paul Cruz	65a4ce2635	added tests for forced compression level	2017-07-13 14:57:24 -07:00
Paul Cruz	0d9665cef5	added additional tests for performance, allowed force compression level for testing purposes	2017-07-13 14:46:54 -07:00