4f86b0f909
Fix develop build breakages
2019-08-29 09:44:14 +01:00
1982b38e8a
Added a couple warnings of what values should not be changed after postConfig.
2019-08-28 16:30:35 -05:00
2a78777e06
Documented all of the vars in the SM config file. Also add'l # parsing to Config.
...
Realized our Config class wouldn't handle tera- (T/t) processing but
no reason it shouldn't. Ex, they want a 1TB cache, they can specify
'1t' as the cache_size.
2019-08-28 15:19:03 -05:00
471b00846d
Added the known-good (WIP) config file the unit test will use.
2019-08-28 13:04:59 -05:00
0f065864c4
Started writing unit tests to verify SM does the right thing
...
when given short data (sender crashed).
2019-08-28 12:59:09 -05:00
fc8f68e391
A couple random things
...
- Re-added the StorageManager/Enabled key in the default config file.
Thought I had switched all that stuff to only look at DBRootStorageType
but I was wrong.
- Made the unit_tests bin for SM stay in the SM dir
- Collapsed one level of build dirs for SM. For some reason it was
putting test data in storage-manager/storage-manager/testData
- Limited a single read() call to 100MB as a paranoid safety measure.
Normal usage won't read that much at once anyway.
2019-08-28 12:56:52 -05:00
e203ea262f
Merge pull request #839 from mariadb-corporation/MCOL-3426
...
MCOL-3426: Fix writes that start past current EOF.
2019-08-28 10:55:51 -05:00
bb015a600b
Fix some error case handling to not update counts or write metadata when not required.
2019-08-28 10:48:50 -05:00
fab69d1717
Merge pull request #838 from pleblanc1976/j-merge-opt
...
Wrote a version of mergejournal that uses more mem but reduces the io op requirement.
2019-08-27 14:59:44 -05:00
e5a6e8401e
MCOL-3460. Found that indeed this is a worthwhile optimization.
...
Write and appendtask were breaking the incoming data into 1MB chunks
to call IOC::write/append() with. Keeping the limit for safety,
but bumping it to 100MB.
2019-08-27 14:40:40 -05:00
4d8523cf86
MCOL-3426: Fix writes that start past current EOF.
2019-08-26 13:52:51 -05:00
281443cb0a
Wrote a version of mergejournal that uses more mem but reduces the io op
...
requirement. Added a safety valve; if the journal file is > 100MB, it
will fall back to the previous IO op heavy but mem friendly version.
This also includes a compiler warning fix for smls & smput.
2019-08-22 16:38:27 -05:00
b9dbc079ac
Cleanup changes and make more flexible to run
2019-08-22 13:18:28 -05:00
5249066ada
Checkout unit_test fixes.
2019-08-21 16:04:09 -05:00
77f311c086
Handle if you set common_prefix_depth to -1 as described in .cnf file
2019-08-21 16:03:47 -05:00
050f8c0be5
Lock condition possible exposed by unit_test.
2019-08-21 16:00:57 -05:00
83789d7180
iocBytesRead was not getting set in ctor
2019-08-21 16:00:13 -05:00
6454b53572
Fix SM_TRACE flag.
2019-08-21 15:59:29 -05:00
fa0da4b9bb
Merge pull request #834 from pleblanc1976/develop
...
Bring in the fixes to make S3 lib work with google cloud storage.
2019-08-21 15:50:27 -05:00
ea1cf03e5f
Merge pull request #833 from pleblanc1976/toolkit-useability
...
MCOL-3438. SM Toolkit usability
2019-08-21 15:49:28 -05:00
d36d8b6831
Bring in the fixes to make S3 lib work with google cloud storage.
2019-08-20 15:52:40 -05:00
47a662b231
MCOL-3438. Made it more sensible. Ownership is still confusing even to
...
me, the author, but this makes the view of the SM-managed files consistent
for the sm* toolkit.
2019-08-20 15:06:21 -05:00
94e77a33c5
MCOL-3441 Fix in-tree builds
...
A copy command was failing for in-tree builds.
2019-08-20 11:15:54 +01:00
a7fd3bf5d1
A few fixes in Synchronizer. If you have a prefix with a '_'
...
char, the key breakout wouldn't be correct until this.
2019-08-19 17:02:29 -05:00
bcd50eaf6b
Made IOC only return files that end with .meta
2019-08-19 15:04:19 -05:00
bba8009c75
Checkpointing a possible insanity-fix to Ownership.
2019-08-19 13:37:10 -05:00
7ddb755ee1
Merge pull request #829 from pleblanc1976/toolkit-fix
...
SM toolkit fix (mcol-3402)
2019-08-19 10:16:49 -05:00
2c2cb219f2
A little cleanup & added a comment to a change made earlier.
2019-08-16 16:41:48 -05:00
815a0fda9f
All tools will now work when SM is on- and offline.
2019-08-16 16:01:59 -05:00
ee42d56b03
Reversed a previous change. Will make that part of another ticket.
2019-08-16 12:10:43 -05:00
95953624de
Checkpointing some changes.
2019-08-16 10:16:52 -05:00
78eed09942
Remove mhash dep from cmake
2019-08-15 12:46:34 -05:00
b40b55d145
Made the SM cmakelist init/update submodules to be nice to buildbot.
2019-08-14 16:56:37 -05:00
73c455c949
Cleared out the region/bucket values I was using for testing. Also
...
made the doc around 'endpoint' a little more clear, I hope.
2019-08-14 15:20:58 -05:00
b497d99ab3
Some SM cmakelists fixes. Looks like it all builds and installs to the right place.
2019-08-14 14:30:24 -05:00
1b1e956c63
Checkpointing changes for merging storagemanager -> CS.
2019-08-14 13:05:50 -05:00
543c61cb3e
Added instructions for getting pre-merge history.
2019-08-14 11:39:14 -05:00
fba0c1728c
Relocating everything in the repo s.t. it can be merged into
...
the columnstore repo.
2019-08-14 11:07:24 -05:00
