database/mariadb-columnstore-engine
1
0
Fork 0
mirror of https://github.com/mariadb-corporation/mariadb-columnstore-engine.git synced 2025-12-10 22:42:30 +03:00
Code Activity
250 Commits 370 Branches 90 Tags
f7014c25e6fc91340aa387854a8f7a023a320be0
Commit Graph

42 Commits

Author SHA1 Message Date
Patrick LeBlanc
f7014c25e6 Made Cache populate Sync with jobs for the pre-existing stuff in the cache 
on startup.
 2019-07-12 14:59:23 -05:00
Patrick LeBlanc
d43f9d855c Merge branch 'cache-accounting' into develop 2019-07-11 12:32:29 -05:00
Patrick LeBlanc
530d9fa5ac Updated a comment re a sneaky bug that means an entry in the LRU 
does not exist on the filesystem.  Haven't figured that out yet.
Must be a narrow race b/c it happens rarely.
 2019-07-05 16:48:30 -05:00
Patrick LeBlanc
46265585ed Fixed a bug in the new makespace code. toBeDeleted wasn't being updated 
during one of the races fixed a couple commits ago.
 2019-07-05 11:03:50 -05:00
Patrick LeBlanc
aa65090a61 Fixed another race. Also shortened the interval Sync creates new jobs. 
In this one, if a read, then a delete happened on an object being flushed
by makeSpace(), it would leave the toBeDeleted struct with an invalid
list iterator.
 2019-07-03 17:18:14 -05:00
Patrick LeBlanc
56a51605af Fixed more races and edge cases I found. 2019-07-03 13:31:49 -05:00
Patrick LeBlanc
2722da0b40 Checkpointing a bunch of edge-case fixes I made and mistakenly mushed 
together with the cache size consistency code.
 2019-07-02 11:12:29 -05:00
Ben Thompson
dedffcd01e Add logic for graceful shutdown 2019-07-01 11:39:24 -05:00
Patrick LeBlanc
715d041a85 Fixed several bugs exposed by setting the cache size to an 
unreasonably small value.  Had to make a compromise to avoid a deadlock
though.  read/write/append/truncate will now be able to exceed the cache size
limit temporarily.  The cache will be reconciled at the end of the respective
operation.

Ex, given a cache of 100MB, and a read() of 500MB, all 500MB
of data being read will stay in the cache until it is read, then
400MB of it will be evicted.  Same on the write side.
 2019-06-28 13:20:31 -05:00
Patrick LeBlanc
91585b4c6d Checkpointing some fixes. 2019-06-27 14:58:04 -05:00
Patrick LeBlanc
0cad8308f1 Untangled cache flushing behavior s.t. it should no longer deadlock. 
Now, something that grabs the sync or cache locks should never block
while holding the lock.  Need to review the code to make sure of that.

Also made the recursive mutexes non-recursive again.
 2019-05-24 13:02:09 -05:00
Patrick LeBlanc
6bd44eae5a Added printing the element count with the cache size on SIGUSR1. 
Not atomic, but good enough for debugging purposes.
 2019-05-23 13:42:31 -05:00
Patrick LeBlanc
04aaff4c2b Undid the undoing of a fix that was OK to begin with. Cache is in fact 
supposed to delete the new file not the old one after a flush.
 2019-05-20 16:11:36 -05:00
Patrick LeBlanc
168a1dd194 Undid a bad fix I made where Cache would potentially delete the 
wrong file on a flush.
 2019-05-20 15:51:45 -05:00
Patrick LeBlanc
8b29e3f9ce Fixed a bug where Cache could delete the wrong file after a flush. 2019-05-17 12:11:57 -05:00
Patrick LeBlanc
eb3608bf70 Fixed a race in Cache usage during delete & truncate ops. 
An object could be flushed between the existence check & the delete
done by delete() & truncate().  Made it atomic within Cache.
 2019-05-17 09:07:49 -05:00
Patrick LeBlanc
8ec7e4a9af Fixed a couple bugs in S3Storage. 
- there was no closing of any file descriptors (!!)
 - there was the typo in the connection pruning code s.t. nothing
   would get pruned.
 - added -lm to the libmarias3 stuff, was getting a linker error, not sure why.
 2019-05-16 13:31:00 -05:00
Patrick LeBlanc
09eec724c8 Wrote simplified versions of Cache::read() and Downloader::download() 
to make it easier to find & fix bugs.  Then fixed a bug that
would result in a stalled read().  Later, it will be worth it
re-complicate things to get more lock granulatity.
 2019-05-03 16:40:41 -05:00
Patrick LeBlanc
5443f8662c Checkpointing some paranoid assertions & some fixes I noticed. 2019-04-12 15:08:01 -05:00
Patrick LeBlanc
867c12f07f Fixed a bug in the cache. 2019-04-09 09:41:26 -05:00
Patrick LeBlanc
64bbf44227 Realized Cache::_makeSpace() could deadlock, made a mutex a recursive_mutex. 2019-04-08 10:39:30 -05:00
Patrick LeBlanc
6f234f45c0 More fixes around sync, error handling, and logging. 2019-04-05 16:38:37 -05:00
Patrick LeBlanc
1879499ad3 Fixed a few random things. 2019-04-05 14:10:59 -05:00
Patrick LeBlanc
d5dfe5ee67 Fixed all compiler warnings. For debugging builds, also made 
the compiler treat warnings as errors.
 2019-04-04 16:52:47 -05:00
Patrick LeBlanc
fe2c5b64d0 Added a couple assertions in Cache to keep us honest during testing. 
Added a 'real' test for IOC::copyFile().  Looks good, will merge it.
 2019-04-04 11:06:17 -05:00
Patrick LeBlanc
e164cbc364 Got a basic unit test to work for IOC::read() 2019-03-28 14:13:21 -05:00
Patrick LeBlanc
de1d849d14 Fixed Cache::reset() 2019-03-26 14:04:28 -05:00
Patrick LeBlanc
4ff769ab24 Got a pretty decent unit test for Sync working. 2019-03-22 12:04:36 -05:00
Patrick LeBlanc
323fd18953 Fixed some stuff, got a basic unit test for sync to work. 2019-03-21 15:23:23 -05:00
Patrick LeBlanc
07b4bdd19c Got the synchronizer stuff to build. 2019-03-21 12:42:10 -05:00
Patrick LeBlanc
6366a54bbc Revamped/simplified the Sync impl. Won't build yet. 2019-03-18 14:49:25 -05:00
Patrick LeBlanc
6f294b8c07 Made Cache populate itself on init. 2019-03-15 14:56:11 -05:00
Patrick LeBlanc
931cb9fb54 First cut of synchronizer is done, won't build yet. 2019-03-15 14:08:19 -05:00
Patrick LeBlanc
9906ff9efb Check pointing some changes I made all over the place 
working on Synchronizer.  Won't build yet.
 2019-03-11 16:09:07 -05:00
Patrick LeBlanc
13ade9e724 Added more Cache tests, fixed a couple bugs. 2019-03-08 10:35:36 -06:00
Patrick LeBlanc
0626b30c88 Finished initial cut of the Cache class 2019-03-07 16:14:21 -06:00
Patrick LeBlanc
df6675db01 Checkpointing additions to Cache 2019-03-07 13:18:38 -06:00
Patrick LeBlanc
1627f46ee5 Commented some debugging output 2019-03-07 08:54:48 -06:00
Patrick LeBlanc
c386881f40 Simple unit test for Cache::read(), and corresponding fixes. 2019-03-06 14:41:30 -06:00
Patrick LeBlanc
3c84e8d749 Checkpointing work on Cache::read() functionality. 
It builds but 0% chance it works here.
 2019-03-06 12:53:04 -06:00
Patrick LeBlanc
2619aa8983 Added to the unit test 
Effectively this is just a test of Config, to make sure it's
doing envvar substitutions & number suffix expansions right.
 2019-02-21 16:27:22 -06:00
Patrick LeBlanc
402a49c27a Added 'k, m, g' suffix processing to Config class 
Started writing the Cache class.  Mostly stubs right now.
 2019-02-21 16:19:29 -06:00