Roman Nozdrin
23a969dbe2
MCOL-5175 Increase the maximum effective length of S3 secret used as SHA256 key producing S3 signature ( #2860 )
...
Co-authored-by: Roman Nozdrin <rnozdrin@mariadb.com >
2023-06-07 15:24:42 +03:00
Leonid Fedorov
030144127e
Remove boost shared array [develop 23.02] ( #2812 )
...
* remove boost/shared_array include
* replace boost::shared_array<T> to std::shared_ptr<T[]>
2023-04-17 20:56:09 +03:00
Leonid Fedorov
2f153184c3
Fixes of bugs from ASAN warnings, part one ( #2796 )
2023-03-30 18:29:04 +03:00
Sam James
20b5dbb617
Add missing includes
...
These seem to have all fallen out of a recent Boost update to 1.81 which
dropped some internal includes. All of these uses within columnstore
relied on these transitive includes, so explicitly include what we need
to fix build.
Signed-off-by: Sam James <sam@gentoo.org >
2023-01-17 01:18:41 +00:00
Leonid Fedorov
d42485656c
Fix clang 16 warnings for comfort build
2023-01-12 22:11:28 +03:00
benthompson15
4630db9206
MCOL-5177: Error handling for HTTP 301 response from AWS S3. ( #2570 )
2022-09-27 11:49:20 -05:00
Leonid Fedorov
51a04ba960
Fixing some test for StorageManagerm commenting all broken rest
2022-09-16 18:51:10 +00:00
Leonid Fedorov
184a4b370b
Revert blind config read patch
2022-06-09 09:59:47 +00:00
benthompson15
652c9299b7
MCOL-4989: Fix error handling for misconfigured StorageManager startup. ( #2385 )
2022-06-02 08:50:41 -05:00
Leonid Fedorov
65252df4f6
C++20 fixes
2022-03-28 12:32:29 +00:00
benthompson15
dc4f6865d1
MCOL-3983: always set threadpool max threads in storagemanager ( #2269 )
2022-02-23 12:21:43 -06:00
Leonid Fedorov
3919c541ac
New warnfixes ( #2254 )
...
* Fix clang warnings
* Remove vim tab guides
* initialize variables
* 'strncpy' output truncated before terminating nul copying as many bytes from a string as its length
* Fix ISO C++17 does not allow 'register' storage class specifier for outdated bison
* chars are unsigned on ARM, having if (ival < 0) always false
* chars are unsigned by default on ARM and comparison with -1 if always true
2022-02-17 13:08:58 +03:00
Leonid Fedorov
04752ec546
clang format apply
2022-01-21 16:43:49 +00:00
Leonid Fedorov
01f3ceb437
replace header guards with #pragma once
2022-01-21 15:24:58 +00:00
Leonid Fedorov
5c5f103f98
MCOL-4839: Fix clang build ( #2100 )
...
* Fix clang build
* Extern C returned to plugin_instance
Co-authored-by: Leonid Fedorov <l.fedorov@mail.corp.ru >
2021-08-23 10:45:10 -05:00
benthompson15
1eea9f9e47
MCOL-4598: Fix the syslog setup script. Add syslog options for broken/non-syslog setup.
2021-04-29 16:34:53 -05:00
benthompson15
401c0fd5c7
MCOL-3542: Add ability to set libmarias3 options via StorageManager cnf file.
2021-03-31 16:13:03 -05:00
benthompson15
8dde6c5cb0
MCOL-4386: remove debug logging
2021-03-30 13:10:31 -05:00
benthompson15
a953409df5
MCOL-4386: Create new StorageManager setting for EC2 using assigned IAM credentials.
2021-03-30 13:10:12 -05:00
benthompson15
438d6949d5
MCOL-4437: change VLArray definition for ScopedCloser
2020-12-10 09:33:05 -06:00
Alexey Antipovsky
b25fee320a
Remove variable-length arrays (-Wvla)
2020-11-17 15:03:10 +03:00
Alexey Antipovsky
0e29b0b0f9
Fix -Wtype-limits
2020-11-17 15:03:10 +03:00
Alexander Barkov
ab44ef6ddb
MCOL-4170 Refactor services/systemd units to finish their bootstrap ...
2020-11-09 12:01:16 +04:00
Alexander Barkov
ffd96d0315
A preparatory patch for MCOL-4170 Refactor services/systemd units to finish their bootstrap ...
...
Moving signal initialization code into separate functions.
2020-11-06 11:58:31 +04:00
benthompson15
8ff81e5d45
MCOL-4347: catch exceptions during smcat/smls/smput/smrm on IOC::get()
2020-10-15 15:49:51 -05:00
benthompson15
ac258dc94d
MCOL-4347: catch exceptions at init and shutdown SM
2020-10-15 15:49:47 -05:00
benthompson15
d56e2453d6
MCOL-4347: Better handling of getConnection() returning NULL from libmarias3 init.
2020-10-15 15:49:44 -05:00
benthompson15
eb79e604e0
Fix for IAM/STS to have sts_region configurable and not tied to S3 bucket region.
2020-10-15 15:40:53 -05:00
benthompson15
00d264cc3f
Fix warning messages.
2020-10-15 15:34:31 -05:00
benthompson15
b3f5142141
renamed assumeRole to assume_role.
2020-10-15 15:34:25 -05:00
benthompson15
473af161c3
MCOL-3976:AWS S3 support for IAM roles. Initial commit.
2020-10-15 15:34:13 -05:00
benthompson15
8a89cb8986
MCOL-3494: cleaner way to do this.
2020-08-26 13:09:26 -05:00
benthompson15
0b32ad8107
Set errno before returning.
2020-08-26 13:09:06 -05:00
benthompson15
2951dbed39
MCOL-3494: make option for testing configuration to skip retryable errors.
2020-08-26 13:08:51 -05:00
Patrick LeBlanc
4b46f55d6a
Fixed a race in storagemanager. On startup it would run a connectivity
...
test to verify it can use S3. The key of the object it tests with is
constant, so there can be interference between multiple SM instances that
happen to be doing this at the same time. The fix is to include a
UUID in the key.
2020-08-25 09:59:07 -04:00
benthompson15
eac7dab096
MCOL-4030: first commit of warning removals unneed const and missing virtual dtors.
2020-06-23 13:51:36 -05:00
Patrick LeBlanc
4bddc92092
MCOL-4010 - fixes compilation errors on x64 w/-Werror
...
Merged in Sergei's patch.
2020-06-01 12:52:43 -04:00
Patrick LeBlanc
2e65619cb1
Fixed an error msg; the function name was wrong.
2020-05-27 16:37:51 -04:00
Patrick LeBlanc
cd5e872104
Removed unnecessary debugging printouts/logging, fixed a couple add'l bugs.
2020-05-27 16:37:51 -04:00
Patrick LeBlanc
6fd24d2d06
Added code to delete orphaned objects from the cache & from cloud storage.
2020-05-27 16:37:51 -04:00
Patrick LeBlanc
33558881ab
Fixed the problem with using the correct length for new objects in copyFile.
...
The metadata should contain the merged length, the object name should
contain the pre-merged length.
2020-05-27 16:37:51 -04:00
Patrick LeBlanc
c2c23b8098
Snapshotting the changes so far, we'll have to come back to it later.
2020-05-27 16:37:51 -04:00
Patrick LeBlanc
faa35ebeeb
Tentative commit, hunting down a source of misbehavior.
...
Conflicts:
storage-manager/src/IOCoordinator.cpp
2020-05-27 16:37:46 -04:00
Patrick LeBlanc
359beb9c96
Suppressed logging self-correcting problems.
...
It will start logging as an err if it does not self-correct after 10 attempts,
and will escalate to crit after 20 attempts.
Also fixed a silly error where it was checking the file size after it
deleted the file.
2020-05-27 16:36:18 -04:00
Patrick LeBlanc
5d8470b91f
Fixed a length parameter after a partial write.
2020-05-27 16:36:18 -04:00
Patrick LeBlanc
43585d55cd
Added error detection to the new ::rename calls.
2020-05-27 16:36:18 -04:00
Patrick LeBlanc
c29050899d
Took out the cache repopulate idea. Now we will only have warnings
...
of problems.
I realized we can't reliably tell how big the cache is while the system
is running. There's a window where write/append has added / is adding
a journal file but hasn't told Cache about it yet. This capability will
have to wait for now.
This shouldn't be a problem because in theory, we will no longer have
data whose size is not consistent with metadata stored outside of the file.
If we do, it means there was either a hard failure, or SM was killed.
Either way, SM will be restarted and the cache will populate its meta
fresh then.
2020-05-27 16:36:18 -04:00
Patrick LeBlanc
3497191d0c
Fixed a silly error.
2020-05-27 16:36:18 -04:00
Patrick LeBlanc
31e06e77a2
Fixed the fix.
...
Actually need to rename the file after you pick a new name for it LOL!
2020-05-27 16:36:18 -04:00
Patrick LeBlanc
4b9c1d9169
Cleaned up a little code of previous commit, added retry loops and
...
a little better error handling to the code that writes journal entries.
2020-05-27 16:36:18 -04:00