1
0
mirror of https://github.com/minio/mc.git synced 2025-09-19 06:01:34 +03:00

42 Commits

Author SHA1 Message Date
Sveinn ( keyb1nd )
8d713ff927 Giving mirror the option to exit early when --remove is not set (#5116)
* Giving mirror the option to exit early when --remove is not set
* Making sure always to cancel the context on exit
* Adding an internal flag to enable local listing only based on active/active and remove flags
* Renaming localListingOnly to sourceListingOnly
2025-02-04 12:32:54 -08:00
jiuker
e44a1c612d fix: mc mirror got zero exit code when access denied (#4952) 2024-06-07 08:48:20 -07:00
Anis Eleuch
5a0d5116f4 mirror: Fix removal of FS target without a trailing slash (#4560) 2023-05-04 11:10:16 -07:00
adfost
fd7142e223 update license headers to 2022 (#4360) 2022-11-14 18:38:39 -08:00
Harshavardhana
3a7a724753 error out when bucket not found with proper exitcode (#3959)
fixes #3953
2022-02-08 12:23:17 -08:00
Anis Elleuch
0c58d62a1f Fix mirror with relative fs path (#3929)
mc diff <alias>/<bucket>/ ./dir/ is not calculated correctly because the
difference code relies on the format of the argument, for example,
whether it is dir/ or ./dir/ or ../my/dir/.

This commit will calculate the absolute path of the passed argument if
it is a filesystem path and uses it through the code.
2022-01-28 10:19:21 -08:00
Harshavardhana
a59f9e4d30 add gofumpt support, automated cleanup (#3911) 2022-01-06 10:55:40 -08:00
Harshavardhana
3144cb97ed remove retry in difference() comparator (#3903)
it is potentially harmful when retrying
a lot of buckets where in this might
trigger a retry - continous Listing()

however this retry is not needed as the
callers restart as needed when --watch
or --active-active are used.
2021-12-27 09:16:15 -08:00
Harshavardhana
1402987ccb change license to AGPLv3 2021-05-11 17:45:08 -07:00
Harshavardhana
3792e9f906 multiple bug fixes for mirror, ls, rm (#3491)
- fixes errant top level directory creation, regression introduced in #3313
- fixes `mc ls -r` regression introduced in #3476
- fixes other bugs such as `mc mirror` would perpetually hang if the
  destination bucket doesn't exist, create and proceed instead.
- removes DirOpt style implementations for List, ListIncompleteUploads
  these are not useful anymore.
2020-11-15 13:24:00 -08:00
Harshavardhana
91c9364f3c add context for mirror failures (#3464) 2020-10-21 13:42:18 -07:00
Cesar N
7f2df96e48 Change ListOptions' fields to public (#3433) 2020-10-01 09:50:56 -07:00
Harshavardhana
93209e842c re-implement metadata handling to preserve nanosecs as well (#3402) 2020-09-12 10:20:03 -07:00
Anis Elleuch
442facedd1 Implement rewind flag in ls and cp commands (#3304)
--rewind will go back in time and select specific object versions that are
the latest at the specified time.

--versions will display all the versions

cp will just ignore if the latest version is a delete marker
ls will print delete markers with a (deleted) at the end
2020-07-23 11:18:38 -07:00
Anis Elleuch
a419bb4130 Use minio-go v7 and madmin master (#3295) 2020-07-14 11:15:19 -07:00
Harshavardhana
08685bae6a make sure active/active enables watch by default (#3271)
- effectively use context everywhere
- effectively look for source-mtime in all
  variations
- `mc diff` should do metadata diff as well
2020-06-23 16:56:08 -07:00
ebozduman
d5b6931a1e Apply global context for all apis (#3240) 2020-06-16 16:55:31 -07:00
Harshavardhana
707f46c32b rename multi-master to active-active (#3255) 2020-06-15 18:16:31 -07:00
Anis Elleuch
4cc297ba4e diff: Disable comparing modtimes when multimaster context is not found (#3226) 2020-05-21 23:50:29 -07:00
Anis Elleuch
c7166eac3a mirror: Use source time attr in multi-master mode (#3162) 2020-04-20 17:03:47 -07:00
Harshavardhana
8cae137525 fix all the build errors 2020-04-03 22:22:20 -07:00
Harshavardhana
b70a79e39b fix: handle symlinks by reading/following as needed (#3053)
Avoid using filepath.EvalSymlinks and let the caller
fail appropriately for symlinks.

Fixes #3011
2020-01-23 02:33:58 -08:00
Harshavardhana
6d1a41458b Use colorjson to indent json properly in cp (#3030)
Additionally also do not initialize progress bar in json
or quiet mode in mirror.

Fixes #3029
Fixes #3017
2020-01-03 12:14:39 -08:00
Krishna Srinivas
2d8bd7aecc Support for multi master sites (#2991) 2019-12-08 16:25:50 -08:00
Harshavardhana
69bea2eb96 Continue listing upon error during diff and mirror (#2988) 2019-12-08 14:34:52 -08:00
Harshavardhana
6cfbc809cf Add mirror command to compare metadata and copy (#2965)
mirror now uses the newly introduced
ListObjectsV2WithMetadata API in minio-go
2019-11-21 22:25:46 -08:00
Harshavardhana
dff077e34b Add support for attr flag in mirror (#2892) 2019-09-19 08:22:45 -07:00
ebozduman
83a9d8b91d Adds missing codebase MinIO and min.io changes (#2745) 2019-04-16 22:31:54 -07:00
Tufan Barış YILDIRIM
a1355e50e2 Documented --json for diff command (#2736) 2019-03-31 20:01:44 -07:00
Anis Elleuch
182ad295d8 Quit calculating difference for any listing error (#2440)
If we have any listing problem, we should stop comparing difference
between source & target lists because it is already so hard to know
what mc should do next. So, let's inform users and let them deal
with the problem.
2018-04-26 11:21:10 -07:00
kannappanr
1fd03fb81f Rearrange exclude code to filter source and target URLs (#2322)
Move exclude code handler to mirror-url and mirror-main

Fixes #2319
2017-12-04 15:47:29 -08:00
kannappanr
9831612507 Add exclude option to mirror command (#2269)
Added exclude options to exclude unwanted system files.
Exclude option excludes the source files/objects that match
the passed shell file name pattern

Fixes #1903
2017-10-12 14:52:21 -07:00
Harshavardhana
685299ab75 mirror: Add timestamp check similar to aws s3 sync (#2188)
Currently `mc mirror` requires source to be
uploaded only if the size is different than the
object on remote server. This PR adds support to
check for last modified time of the source object
as well such that we upload if there are timestamp
differences as well.

Fixes #2187
2017-06-21 16:34:19 -07:00
Anis Elleuch
8ce6f43e35 Stop mirror/diff when difference() sends error (#2049)
The caller of difference() now receives errors so it can
stop mirroring or difference calculation.
2017-03-01 11:06:26 -08:00
Anis Elleuch
ace3f452c3 mirror: Mirror buckets if only alias is specified (#2006) 2017-02-12 11:20:05 -08:00
Aaron Walker
bca93f6c83 Mirror: normalize keys before comparing (#1978)
vendorize golang.org/x/text/unicode/norm dependency

vendorize golang.org/x/text/transform dependency

add example of multiple encodings
2017-01-24 11:51:59 -08:00
Harshavardhana
9cf76b1233 api: StatObject if listing returned missing file. (#1945) 2017-01-04 18:36:09 -08:00
Harshavardhana
64166319ee cp/diff/mirror: Cleanup code and deprecate unnecessary flags. (#1914)
This fix also fixes an apparent crash found in diff.

```
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x4bdc07]

goroutine 84 [running]:
panic(0x810ea0, 0xc4200120e0)
        /home/builder/.gimme/versions/go1.7.3.linux.amd64/src/runtime/panic.go:500 +0x1a1
github.com/minio/mc/cmd.objectDifference.func1(0xc4200880a8, 0xc42042ee56, 0xc420315ec0, 0xc4200880b0, 0xc42042ee57, 0xc420315f20, 0xc42042ee54, 0xc42042ee55, 0xc420315501, 0xc4200880a0, ...)
        /home/builder/mygo/src/github.com/minio/mc/cmd/difference.go:109 +0x197
created by github.com/minio/mc/cmd.objectDifference
        /home/builder/mygo/src/github.com/minio/mc/cmd/difference.go:180 +0x2f5
```
2016-12-06 01:41:22 -08:00
Harshavardhana
cf89d74561 Vendorize new minio-go code. (#1891)
This fixes #1889.

```sh
mc mirror -w
```

Now runs continous even when the source is not available.
Reconnects indefinitely in a binomially increasing fashion
and also allows jitter so that the retry has a nice
distribution as well.

This allows for 'mc mirror -w' to run continously without
further monitoring.
2016-11-20 16:31:53 -08:00
Harshavardhana
38aec4d875 mirror and cp should print size in --json mode. (#1890)
```sh
mc cp -r --json /usr/bin /tmp | jq .
...
...
{
  "status": "success",
  "source": "/usr/bin/gcm-viewer",
  "target": "/tmp/bin/gcm-viewer",
  "size": 87128,
  "totalCount": 2602,
  "totalSize": 677422730
}
```

This allows scripts to build progress around this data.
2016-11-20 14:46:24 -08:00
Bala FA
6de2b4559e Refactor rm command. (#1855)
This patch makes rm command to use minio-go.RemoveObjects() to removeobjects recursively.
2016-10-17 20:24:09 -07:00
Harshavardhana
2ea2ec90d2 Rename mc --> cmd 2016-08-17 18:26:18 -07:00