database/mariadb-columnstore-engine
1
0
Fork 0
mirror of https://github.com/mariadb-corporation/mariadb-columnstore-engine.git synced 2025-12-24 14:20:59 +03:00
Code Activity
214 Commits 373 Branches 90 Tags
68849fc5061cd7606103f3386b2eedd63ea7014d
Commit Graph

214 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Patrick LeBlanc
68849fc506 Added a tool to verify metadata contents against actual storage. 2019-05-28 17:11:35 -05:00
Ben Thompson
9351962942 Add sync forceFlush and modified unit_test to be in working state again. 2019-05-28 15:18:19 -05:00
Patrick LeBlanc
b9d7a2e4c4 Fixed the deadlock in our sanity check code in synchronizer. 2019-05-28 10:51:15 -05:00
Patrick LeBlanc
9a1dc8a4c5 Removed some redundant assertions around deleting files, 
commented the one in replicator::remove() to get ready for milestone-2,
and to make the unit test work.
 2019-05-28 09:57:26 -05:00
Patrick LeBlanc
be01adb150 Added some debugging lines to track down spurious threads. Looks like 
they're created by the s3 lib.  The lines are commented but left in
it we need them later.
 2019-05-24 15:52:24 -05:00
Patrick LeBlanc
0cad8308f1 Untangled cache flushing behavior s.t. it should no longer deadlock. 
Now, something that grabs the sync or cache locks should never block
while holding the lock.  Need to review the code to make sure of that.

Also made the recursive mutexes non-recursive again.
 2019-05-24 13:02:09 -05:00
Patrick LeBlanc
b7b9ce9f93 Merge branch 'develop' of github.com:pleblanc1976/storage-manager into develop 2019-05-23 13:46:38 -05:00
Patrick LeBlanc
8e4f8db2a2 Changed the lock scope in IOC::read() to make sure journal files 
can't be written to while they're being read.
 2019-05-23 13:43:51 -05:00
Patrick LeBlanc
e4f5d30ffa Added assertion around deleteing a failed download. 2019-05-23 13:43:16 -05:00
Patrick LeBlanc
6bd44eae5a Added printing the element count with the cache size on SIGUSR1. 
Not atomic, but good enough for debugging purposes.
 2019-05-23 13:42:31 -05:00
Patrick LeBlanc
a32e6c7bb4 Fixed a possible size mismatch in Sync, which could be caused 
by a truncated logical file.
 2019-05-23 13:41:24 -05:00
Patrick LeBlanc
51d06f1d93 Fixed the reason cache size was blowing up using S3. 2019-05-23 13:38:58 -05:00
Ben Thompson
6d6c51cdd6 Remove assertions that are bad. 2019-05-22 15:05:03 -05:00
Patrick LeBlanc
05e98265a8 Updated libmarias3 2019-05-22 08:50:37 -05:00
Patrick LeBlanc
0c4e549b7a Fixed a couple compiler warnings/errors. 2019-05-22 08:50:19 -05:00
Patrick LeBlanc
68ffceaa68 Merge branch 'develop' of github.com:pleblanc1976/storage-manager into develop 2019-05-21 16:06:08 -05:00
Patrick LeBlanc
0dcbc10b5e Added assertions around the file deletes in syncwithjournal. 2019-05-21 16:05:27 -05:00
Patrick LeBlanc
88de938913 Added logging to an error path in the s3 lib that looked like it should 
have one.
 2019-05-21 16:04:33 -05:00
Patrick LeBlanc
1e1fa5dbd5 Added a sigusr1 handler that prints current cache size. 2019-05-21 16:03:56 -05:00
Ben Thompson
d0a56fd3d4 Change how makespace is called on write when new objects are created. added some inserts and comments on potential bug. 2019-05-21 14:34:25 -05:00
Patrick LeBlanc
04aaff4c2b Undid the undoing of a fix that was OK to begin with. Cache is in fact 
supposed to delete the new file not the old one after a flush.
 2019-05-20 16:11:36 -05:00
Patrick LeBlanc
168a1dd194 Undid a bad fix I made where Cache would potentially delete the 
wrong file on a flush.
 2019-05-20 15:51:45 -05:00
Patrick LeBlanc
b8d4891a18 Added some commented debugging printouts & some assertions around 
deletion to make sure we're deleting things.
 2019-05-20 10:52:47 -05:00
Patrick LeBlanc
28495c7e90 Fixed a bug in sync with journal where a cloud object would 
not be deleted after a journal merge.
 2019-05-17 16:02:34 -05:00
Patrick LeBlanc
8b29e3f9ce Fixed a bug where Cache could delete the wrong file after a flush. 2019-05-17 12:11:57 -05:00
Patrick LeBlanc
77b682f9cd Fixed a few problems in Sync. 
- flushObject() could wait on an element in objNames which
   doesn't exist, and so it would not get processed.
 - process() would not always delete its job from objNames
 - Added add'l debugging printouts and a fallback case for when
   syncWithJournal does not find a journal file.
 - noticed a name collision between the scoped ops in syncWithJournal (harmless)
 2019-05-17 09:09:24 -05:00
Patrick LeBlanc
eb3608bf70 Fixed a race in Cache usage during delete & truncate ops. 
An object could be flushed between the existence check & the delete
done by delete() & truncate().  Made it atomic within Cache.
 2019-05-17 09:07:49 -05:00
Patrick LeBlanc
886acd8b08 Removed the LIBS=-lm from Cmakelists. For some reason not necessary 
in this release.  Also, for some reason, we are now linking dynamically
instead of statically.
 2019-05-16 13:47:13 -05:00
Patrick LeBlanc
8ec7e4a9af Fixed a couple bugs in S3Storage. 
- there was no closing of any file descriptors (!!)
 - there was the typo in the connection pruning code s.t. nothing
   would get pruned.
 - added -lm to the libmarias3 stuff, was getting a linker error, not sure why.
 2019-05-16 13:31:00 -05:00
Patrick LeBlanc
b1478f347b Strengthen a minor assert in SharedCloser utility. 2019-05-16 11:56:42 -05:00
Patrick LeBlanc
cd83b26d6b Was incrementing the wrong iterator, causing infinite loop. 2019-05-16 11:55:43 -05:00
Patrick LeBlanc
15dbd1eec8 Fixed a one-char type in an error msg in s3storage. 2019-05-13 12:13:59 -05:00
Patrick LeBlanc
e23051ea0c Added a comment to IOC. We could use some noise reduction 
in copyFile.
 2019-05-13 12:13:12 -05:00
Patrick LeBlanc
aad393dd34 Fixed a bug in Sync where we were using an object through a 
reference to a shared pointer.  A deref of the shared ptr could
trigger the dtor of the object while threads are waiting.  Fixed that
by properly copying the shared ptr.
 2019-05-13 12:11:16 -05:00
Patrick LeBlanc
19859de510 Added a test to the s3 module, attempt to copy a non-existant file. 2019-05-13 12:10:48 -05:00
Patrick LeBlanc
2cd2c8b0a6 Fixed error reporting in s3storage. Was printing the access key, 
not the key of the object.
 2019-05-10 13:55:39 -05:00
Patrick LeBlanc
a087c85776 Add error handling for a specific error code returned by 
libmarias3.
 2019-05-10 13:49:18 -05:00
Patrick LeBlanc
53a88fdc09 Fixed a minor bug in error reporting. 2019-05-10 13:49:00 -05:00
Patrick LeBlanc
353392f98b Update the ref on libmarias3. 2019-05-10 13:48:41 -05:00
Patrick LeBlanc
09eec724c8 Wrote simplified versions of Cache::read() and Downloader::download() 
to make it easier to find & fix bugs.  Then fixed a bug that
would result in a stalled read().  Later, it will be worth it
re-complicate things to get more lock granulatity.
 2019-05-03 16:40:41 -05:00
Patrick LeBlanc
3983247982 Fixed a bug in IOC::truncate() where it appeared to assume 
the object to truncate is in the cache.
 2019-05-03 16:39:54 -05:00
Patrick LeBlanc
df4c3eee37 Added a way to name a threadpool for debugging purposes. 2019-05-03 16:39:28 -05:00
Patrick LeBlanc
5b80dbc7c2 Fixed a couple minor things in ThreadPool. 
Most important fix was an off-by-one error where a thread would
be created and immediately return.
 2019-05-03 10:25:02 -05:00
Patrick LeBlanc
701534e3d3 Integrated S3 lib's s3-to-s3 copy fcn. 2019-04-25 14:33:56 -05:00
Patrick LeBlanc
6e2247e2b1 Updated ref for Andrew's lib 2019-04-25 13:24:53 -05:00
Patrick LeBlanc
1ee3c760b1 Changed Sync s.t. it starts processing jobs every 10 seconds instead 
of every time there's an event.  Big improvement in efficiency.
 2019-04-25 12:07:30 -05:00
Patrick LeBlanc
fc717886c6 Fixed a really dumb bug in threadpool that would leave no threads 
left to process jobs.  Also added an accessor to see how big
the job queue is.
 2019-04-25 12:05:52 -05:00
Patrick LeBlanc
7eebc7f06f Fixed a really dumb error where I had Sync writing 0's over the data. 
Also added a couple things out of paranoia.
 2019-04-24 11:33:34 -05:00
Patrick LeBlanc
4d49f9cc1e Somehow forgot to do this. Made open() impl O_TRUNC. Also added 
a couple assertions for debugging's sake.
 2019-04-23 15:37:15 -05:00
Patrick LeBlanc
5ad6246575 Debugged mergeJournal() using some data that failed to merge in a 
'real' run.  Added the files that failed to merge, & added to the unit test.
 2019-04-23 10:37:54 -05:00