Alexander Barkov
ab44ef6ddb
MCOL-4170 Refactor services/systemd units to finish their bootstrap ...
2020-11-09 12:01:16 +04:00
Alexander Barkov
ffd96d0315
A preparatory patch for MCOL-4170 Refactor services/systemd units to finish their bootstrap ...
...
Moving signal initialization code into separate functions.
2020-11-06 11:58:31 +04:00
benthompson15
8ff81e5d45
MCOL-4347: catch exceptions during smcat/smls/smput/smrm on IOC::get()
2020-10-15 15:49:51 -05:00
benthompson15
ac258dc94d
MCOL-4347: catch exceptions at init and shutdown SM
2020-10-15 15:49:47 -05:00
benthompson15
d56e2453d6
MCOL-4347: Better handling of getConnection() returning NULL from libmarias3 init.
2020-10-15 15:49:44 -05:00
benthompson15
eb79e604e0
Fix for IAM/STS to have sts_region configurable and not tied to S3 bucket region.
2020-10-15 15:40:53 -05:00
benthompson15
0bb2361254
Fix iam_role_name usage comments.
2020-10-15 15:34:49 -05:00
benthompson15
039521fe91
Fix phrasing of error and cnf comments
2020-10-15 15:34:45 -05:00
benthompson15
bddea43178
add post-install setting of iam_role_name and sts_endpoint thru ENV variables.
2020-10-15 15:34:42 -05:00
benthompson15
00d264cc3f
Fix warning messages.
2020-10-15 15:34:31 -05:00
benthompson15
b3f5142141
renamed assumeRole to assume_role.
2020-10-15 15:34:25 -05:00
benthompson15
473af161c3
MCOL-3976:AWS S3 support for IAM roles. Initial commit.
2020-10-15 15:34:13 -05:00
Patrick LeBlanc
9bc0f2e5ab
Replaced the special quotes in storagemanager.cnf with regular ascii
...
ones.
2020-09-04 16:46:27 -04:00
benthompson15
8a89cb8986
MCOL-3494: cleaner way to do this.
2020-08-26 13:09:26 -05:00
benthompson15
0b32ad8107
Set errno before returning.
2020-08-26 13:09:06 -05:00
benthompson15
2951dbed39
MCOL-3494: make option for testing configuration to skip retryable errors.
2020-08-26 13:08:51 -05:00
Patrick LeBlanc
4b46f55d6a
Fixed a race in storagemanager. On startup it would run a connectivity
...
test to verify it can use S3. The key of the object it tests with is
constant, so there can be interference between multiple SM instances that
happen to be doing this at the same time. The fix is to include a
UUID in the key.
2020-08-25 09:59:07 -04:00
benthompson15
eac7dab096
MCOL-4030: first commit of warning removals unneed const and missing virtual dtors.
2020-06-23 13:51:36 -05:00
Patrick LeBlanc
4bddc92092
MCOL-4010 - fixes compilation errors on x64 w/-Werror
...
Merged in Sergei's patch.
2020-06-01 12:52:43 -04:00
Patrick LeBlanc
2e65619cb1
Fixed an error msg; the function name was wrong.
2020-05-27 16:37:51 -04:00
Patrick LeBlanc
cd5e872104
Removed unnecessary debugging printouts/logging, fixed a couple add'l bugs.
2020-05-27 16:37:51 -04:00
Patrick LeBlanc
6fd24d2d06
Added code to delete orphaned objects from the cache & from cloud storage.
2020-05-27 16:37:51 -04:00
Patrick LeBlanc
33558881ab
Fixed the problem with using the correct length for new objects in copyFile.
...
The metadata should contain the merged length, the object name should
contain the pre-merged length.
2020-05-27 16:37:51 -04:00
Patrick LeBlanc
c2c23b8098
Snapshotting the changes so far, we'll have to come back to it later.
2020-05-27 16:37:51 -04:00
Patrick LeBlanc
faa35ebeeb
Tentative commit, hunting down a source of misbehavior.
...
Conflicts:
storage-manager/src/IOCoordinator.cpp
2020-05-27 16:37:46 -04:00
Patrick LeBlanc
359beb9c96
Suppressed logging self-correcting problems.
...
It will start logging as an err if it does not self-correct after 10 attempts,
and will escalate to crit after 20 attempts.
Also fixed a silly error where it was checking the file size after it
deleted the file.
2020-05-27 16:36:18 -04:00
Patrick LeBlanc
5d8470b91f
Fixed a length parameter after a partial write.
2020-05-27 16:36:18 -04:00
Patrick LeBlanc
43585d55cd
Added error detection to the new ::rename calls.
2020-05-27 16:36:18 -04:00
Patrick LeBlanc
c29050899d
Took out the cache repopulate idea. Now we will only have warnings
...
of problems.
I realized we can't reliably tell how big the cache is while the system
is running. There's a window where write/append has added / is adding
a journal file but hasn't told Cache about it yet. This capability will
have to wait for now.
This shouldn't be a problem because in theory, we will no longer have
data whose size is not consistent with metadata stored outside of the file.
If we do, it means there was either a hard failure, or SM was killed.
Either way, SM will be restarted and the cache will populate its meta
fresh then.
2020-05-27 16:36:18 -04:00
Patrick LeBlanc
3497191d0c
Fixed a silly error.
2020-05-27 16:36:18 -04:00
Patrick LeBlanc
31e06e77a2
Fixed the fix.
...
Actually need to rename the file after you pick a new name for it LOL!
2020-05-27 16:36:18 -04:00
Patrick LeBlanc
4b9c1d9169
Cleaned up a little code of previous commit, added retry loops and
...
a little better error handling to the code that writes journal entries.
2020-05-27 16:36:18 -04:00
Patrick LeBlanc
aeec468814
Fixed a number of bugs in storage manager, and added code to detect
...
and recover from being killed while writing new objects.
Conflicts:
storage-manager/src/Synchronizer.cpp
2020-05-27 16:36:08 -04:00
Roman Nozdrin
b8caaaa760
Rebased MCOL-3836 on top of e7bc7e93
.
2020-05-12 15:56:10 +00:00
Jose
98abf95eae
MCOL-3991 MCS is now single package and properly uninstalls
2020-05-12 13:36:24 +00:00
Patrick LeBlanc
49b8a4afea
MCOL-3983 workaround. No time right now for a good fix. For now
...
using a workaround and a note.
2020-05-06 11:28:21 -04:00
Roman Nozdrin
6987ac40e1
MCOL-3982 Avoid using autotools for libmarias3 and remove libxml2 from libmarias3 dependencies.
2020-05-06 12:45:34 +00:00
benthompson15
75d1dbc592
MCOL-3494: initial commit of test program to verify storagemanager.cnf during postConfigure.
2020-04-30 16:59:56 -05:00
Patrick LeBlanc
bb3e76be7b
Accidentally clobbered some documentation with the last commit.
2020-04-22 17:29:12 -04:00
Patrick LeBlanc
78c53df1b9
MCOL-3889 part 1. Updated the storagemanager.cnf file.
2020-04-22 17:14:41 -04:00
Patrick LeBlanc
90d4b03fdb
MCOL-3743: cmdline params to postConfig to configure SM
...
Made some changes. It's working the way it should.
2020-03-17 13:55:08 -04:00
Patrick LeBlanc
eb52184a5d
Merge pull request #1043 from benthompson15/MCOL-3748
...
MCOL-3748: Modify signal handling for different default actions.
2020-02-28 16:19:48 -05:00
Patrick LeBlanc
635438599c
Merge pull request #1042 from benthompson15/MCOL-3736
...
MCOL-3736: Change AND to OR. Loop could exit with ops still in pendin…
2020-02-28 16:19:41 -05:00
Patrick LeBlanc
d8f22609b4
MCOL-3745: Investigate slow-ddl problem after restart
...
Found a way to mitigate it through configuration. Only relevant to low
upstream bandwidth envs. Documented it.
2020-01-31 11:35:11 -05:00
Patrick LeBlanc
bf356cdb86
Merge pull request #959 from benthompson15/MCOL-3638
...
MCOL-3638: Fix level of various storagemanager logging items.
2020-01-03 09:42:45 -06:00
Sergei Golubchik
586391e1ca
compilation failure
...
error: reference to 'mutex' is ambiguous
note: candidates are: 'class boost::mutex'
note: 'class std::mutex'
2019-12-19 18:13:39 +01:00
benthompson15
d811697cee
MCOL-3651: Another update to comments since we can't install to users home directory.
2019-12-06 11:28:32 -06:00
benthompson15
99eb56382b
MCOL-3651: Update the comments in the default cnf file to explain common_prefix_depth.
2019-12-06 11:19:26 -06:00
benthompson15
1b56000b7b
MCOL-3651: Change common_prefix_depth default from 4 to 3.
2019-12-06 11:10:32 -06:00
benthompson15
207fb3f588
Merge pull request #960 from LinuxJedi/unified_build
...
Unified build for ES and MCS
2019-12-05 01:19:23 +01:00