1
0
mirror of https://github.com/mariadb-corporation/mariadb-columnstore-engine.git synced 2025-12-12 11:01:17 +03:00
Commit Graph

26 Commits

Author SHA1 Message Date
Patrick LeBlanc
fdbbfd44e0 Checkpointing some add'l KPIs to track down bits of interest. 2019-08-08 14:54:24 -05:00
Patrick LeBlanc
b952c2797d Merging in the kpis-1 branch.
Squashed commit of the following:

commit dfa92945f3d8ffb304ee9272ab0dd0a500070da2
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Fri Aug 2 09:21:13 2019 -0500

    Made unit_tests compile, made the printKPI fcn in CloudStorage virtual.

commit e3a9f0ac5559db1016911fbde0796a9086f1b8f8
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Thu Aug 1 16:07:59 2019 -0500

    Added the signal handler to print KPIs (SIGUSR2), made code use the
    KPIs.  It builds here, not tested or proof-read yet.

commit c3b89a57fe852394f7dbf37fb5780707b8bcc2ac
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Thu Aug 1 13:30:24 2019 -0500

    Checkpointing.  Got it to build but not complete yet.

commit 35b76f232a29652cec65916f5c95ed240a6eaf36
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Thu Aug 1 13:00:33 2019 -0500

    Checkpointing.  doesn't build yet.
2019-08-02 14:15:39 -05:00
Patrick LeBlanc
1c5979576e Merging the prefix-ownership feature.
Squashed commit of the following:

commit 4a4c3dab2e6acf942bbdfd4d760c000bc9cbfc6a
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Tue Jul 30 10:50:08 2019 -0500

    Standardized a couple status msgs.

commit 1b76f7e6411424c9633dcd4ebe7f61e9fce2f0ac
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Tue Jul 30 09:19:31 2019 -0500

    Fixed the thread-joining-itself problem.

commit 4fdb79e87496eab64c4c5af72321bc57423297ba
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Mon Jul 29 17:15:05 2019 -0500

    Checkpointing.  Need to release ownership differently, realized
    I have 1 thread trying to join itself.

commit 04d0183735e9697d76a2472c6135d90755ca61b5
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Mon Jul 29 16:12:33 2019 -0500

    Checkpointing a whole lot of fixes.  test000 works here.

commit 72e9c998c62b095cad1cf33f885f6c7697bde214
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Fri Jul 26 16:29:15 2019 -0500

    Checkpointing.  Started debugging.  Several small fixes.  Doesn't work yet.

commit ab728e1481debec94d676e697954b1d164302a0c
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Fri Jul 26 13:15:47 2019 -0500

    Checkpointing.  Got everything to build.

commit a2c6d07cdc12c45530c1d5cf4205d3aee8738d80
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Fri Jul 26 12:59:26 2019 -0500

    Checkpointing.  Got the library to build.

commit 9f6bf19a64f512e17e6139b0fc04850cdcdb3b3a
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Fri Jul 26 12:24:46 2019 -0500

    Checkpointing.  Still WIP.  Feature touches everything.

commit a79ca8dc88a99f812432d5dca34ed54474df1933
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Thu Jul 25 16:38:20 2019 -0500

    Checkpointing more changes.

commit a9e81af3e4e00f8a3d30b3796a2c3aa94c04f7c0
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Thu Jul 25 15:07:44 2019 -0500

    Checkpointing changes to the other classes that need to be aware
    of separately managed prefixes.

commit d85dfaa401b49a7bb714701649dec303eb7c068c
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Wed Jul 24 14:24:23 2019 -0500

    Added the new class to CMakeLists.

commit 66d6d550b13be94ada107311574378bd848951cd
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Wed Jul 24 14:23:49 2019 -0500

    Checkpointing.  Got the new class to build except for a
    to-be-implemented fcn in Cache.

commit e1b62dba7f05b37b9f12681a53d6632c6ce66d54
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Wed Jul 24 14:23:09 2019 -0500

    Added some documentation to the object_size param.

commit e671cf37c49ed084fbdec1bac50fbaa5ad7c43f9
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Wed Jul 24 10:34:54 2019 -0500

    Checkpointing a new class to manage ownership of prefixes.

commit e5f234ff4c05b5157d37fa17c44d7f626f5e4eb3
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Tue Jul 23 15:36:48 2019 -0500

    Fixed some copy/paste typos in Metadatafile config err msgs.
2019-07-30 10:51:05 -05:00
Patrick LeBlanc
5f43f55ea5 Changed a bunch of file perms. Source files don't need exec perms. 2019-07-23 13:20:08 -05:00
Patrick LeBlanc
d43f9d855c Merge branch 'cache-accounting' into develop 2019-07-11 12:32:29 -05:00
Patrick LeBlanc
90696b8887 Appears that the U18 version of boost needs an explicit include
of scoped_ptr
2019-07-11 11:46:47 -05:00
Patrick LeBlanc
d0c457a068 Parameterized the journal size threshold to trigger a flush. 2019-07-05 14:20:39 -05:00
Patrick LeBlanc
62c853595f Changed the sync trigger to include outstanding journal size.
Right now it's hardcoded at 50MB for testing.  Will parameterize later.
2019-07-05 11:02:55 -05:00
Patrick LeBlanc
aa65090a61 Fixed another race. Also shortened the interval Sync creates new jobs.
In this one, if a read, then a delete happened on an object being flushed
by makeSpace(), it would leave the toBeDeleted struct with an invalid
list iterator.
2019-07-03 17:18:14 -05:00
Ben Thompson
dedffcd01e Add logic for graceful shutdown 2019-07-01 11:39:24 -05:00
Patrick LeBlanc
715d041a85 Fixed several bugs exposed by setting the cache size to an
unreasonably small value.  Had to make a compromise to avoid a deadlock
though.  read/write/append/truncate will now be able to exceed the cache size
limit temporarily.  The cache will be reconciled at the end of the respective
operation.

Ex, given a cache of 100MB, and a read() of 500MB, all 500MB
of data being read will stay in the cache until it is read, then
400MB of it will be evicted.  Same on the write side.
2019-06-28 13:20:31 -05:00
Ben Thompson
9351962942 Add sync forceFlush and modified unit_test to be in working state again. 2019-05-28 15:18:19 -05:00
Patrick LeBlanc
0cad8308f1 Untangled cache flushing behavior s.t. it should no longer deadlock.
Now, something that grabs the sync or cache locks should never block
while holding the lock.  Need to review the code to make sure of that.

Also made the recursive mutexes non-recursive again.
2019-05-24 13:02:09 -05:00
Patrick LeBlanc
aad393dd34 Fixed a bug in Sync where we were using an object through a
reference to a shared pointer.  A deref of the shared ptr could
trigger the dtor of the object while threads are waiting.  Fixed that
by properly copying the shared ptr.
2019-05-13 12:11:16 -05:00
Patrick LeBlanc
1ee3c760b1 Changed Sync s.t. it starts processing jobs every 10 seconds instead
of every time there's an event.  Big improvement in efficiency.
2019-04-25 12:07:30 -05:00
Patrick LeBlanc
0be099769b Cleaned up a kludge, and made the retry behavior requeue a job
rather than wait forever to it to work in the same thread.
2019-04-08 10:40:28 -05:00
Patrick LeBlanc
a813581567 Got it to build. Getting into to some header file madness, but moving on. 2019-03-27 17:44:50 -05:00
Patrick LeBlanc
4ff769ab24 Got a pretty decent unit test for Sync working. 2019-03-22 12:04:36 -05:00
Patrick LeBlanc
323fd18953 Fixed some stuff, got a basic unit test for sync to work. 2019-03-21 15:23:23 -05:00
Patrick LeBlanc
07b4bdd19c Got the synchronizer stuff to build. 2019-03-21 12:42:10 -05:00
Patrick LeBlanc
2990d320aa Checkpointing some unit test stuff, won't build yet. 2019-03-20 13:17:16 -05:00
Patrick LeBlanc
97d2844994 Made Sync take into account that object names may change
beteween when a job is created and when it is run.
2019-03-19 17:24:32 -05:00
Patrick LeBlanc
6366a54bbc Revamped/simplified the Sync impl. Won't build yet. 2019-03-18 14:49:25 -05:00
Patrick LeBlanc
931cb9fb54 First cut of synchronizer is done, won't build yet. 2019-03-15 14:08:19 -05:00
Patrick LeBlanc
9906ff9efb Check pointing some changes I made all over the place
working on Synchronizer.  Won't build yet.
2019-03-11 16:09:07 -05:00
Patrick LeBlanc
df6675db01 Checkpointing additions to Cache 2019-03-07 13:18:38 -06:00