minio/mc
1
0
Fork 0
mirror of https://github.com/minio/mc.git synced 2025-11-12 01:02:26 +03:00
Code Activity
3,548 Commits 6 Branches 322 Tags
1dd2b4494d4cce65b51a30d5800f3a8f4ea044f9
Commit Graph

88 Commits

Author SHA1 Message Date
Ashish Kumar Sinha
9a25e66e87 Add --ignore-existing/-p flag in mc event add command (#2773) 
If the event is already present the server throws an error. With the --ignore-existing/ -p flag these error messages are discarded
 2019-07-16 10:40:06 +01:00
Harshavardhana
5d0a9c0803 Make sure to preserve region specific URLs (#2791) 
This is to ensure that restricted access
to certain buckets is allowed on certain
regions.

This is a continuation of last commit
 2019-06-11 13:59:10 -07:00
Harshavardhana
e978c71b7c Add staticcheck based builds (#2788) 
Additionally this PR also supports
multi-platform builds to avoid cross
platform build issues.
 2019-06-03 11:22:31 -07:00
Harshavardhana
f4bb0b8850 Update minio-go to latest release v6.0.26 (#2782) 2019-05-29 08:27:18 -07:00
Harshavardhana
b1aa2232ef Disable http2.0 for now until upstream bugs are fixed (#2764) 2019-05-01 15:42:36 -07:00
ebozduman
83a9d8b91d Adds missing codebase MinIO and min.io changes (#2745) 2019-04-16 22:31:54 -07:00
Harshavardhana
b27cd5ada8 Make csv-input optional for default cases (#2741) 
Fixes #2740
 2019-04-03 18:31:23 +05:30
Harshavardhana
c34dec5495 Set correct compressionType for compression select request (#2730) 2019-03-28 10:52:13 -07:00
poornas
0490231b34 For non-canned policies, show policy status as custom (#2716) 2019-03-19 09:11:16 +05:30
poornas
5b585c4759 Additional flags for mc sql command (#2687) 
add sql select options
change flag names and remove parquet option
add csv-output-header option
 2019-03-11 22:23:45 -07:00
Ashish Kumar Sinha
2e612b23d6 Add metadata in mc copy (#2671) 
Example : mc cp --attr key1=value1,key2=value2 
play/mybucket/burningman2011/ ~/latest/
 2019-03-11 12:47:28 +05:30
Anis Elleuch
4a130c60a8 rb: Fix removing an empty bucket when passed as argument (#2703) 
rb was relying on client.List() to find all elements that needs to be
removed. However it doesn't include 'alias/bucket' in the listing
when 'alias/bucket' itself is passed as an argument in rb command.

This PR will always attempt to delete the passed argument as this is
what the user wants in the first place.

This PR also fixes teardown() in functional-tests.sh to remove
the bucket using rb instead rm since rm does not remove buckets
anymore.
 2019-03-08 17:07:06 +05:30
poornas
c63c5732e9 Fix policy set error (#2701) 
When policy is being set,json does not need to be indented - it
is causing a policy has invalid resource error
 2019-03-02 10:38:16 -08:00
poornas
dba70c0b92 Stat: display Expires header if present (#2697) 2019-03-01 13:03:53 -08:00
Sidhartha Mani
c75abbb3f9 use colorjson for all other mc outputs (#2683) 
- Update to MarshalIndent instead of Marshal
- Allow unmarshalling
 2019-02-28 14:33:13 -08:00
Harshavardhana
19e14db50a Avoid ListBuckets to keep Stat() ops on bucket simpler (#2699) 
Due to multi-user feature on Minio and mc adoption in
many different restrictive environments, most users disable
access to ListBuckets() calls. We can avoid such network
operations as they only add marginal value.
 2019-02-28 14:23:47 -08:00
poornas
e538599e2f Add mc rb command to remove empty bucket (#2662) 
Change mc rm --recursive behavior to
delete objects in bucket but not the
bucket itself.

Fixes #2635
 2019-02-16 20:12:38 -08:00
Harshavardhana
3a22044471 Vendorize all the latest dependencies with go mod (#2674) 2019-02-08 18:32:17 -08:00
Praveen raj Mani
30965763d6 Support setting of custom policies from a JSON file (#2649) 
Fixes #2633
 2019-01-28 13:30:31 -08:00
Harshavardhana
ede0911de2 Support parquet select implementation (#2643) 2019-01-22 11:13:58 -08:00
Harshavardhana
d8d3a3dec1 Remove incorrect OutputSerialization 2019-01-07 03:08:41 -08:00
Harshavardhana
59ef9fe468 Support SSE without keys (#2626) 2019-01-04 11:56:43 -08:00
Anis Elleuch
124b9fa4d9 Avoid remove freeze when server returns an error (#2631) 
Minio can return during deletion sometimes, like missing quorum,
but mc can block in that case. The reason is that Remove() in
client-s3 doesn't actively listen for errors reported by minio-go
RemoveObjects() API  because that would also block this latter.
 2018-12-27 17:38:17 -08:00
Harshavardhana
2073642656 Allow copying file onto itself (#2617) 
This allows encrypting an unencrypted file
using server side CopyObject API.
 2018-12-06 10:18:16 -08:00
Kale Blankenship
8137e79d0c Close readers returned by Client.Get (#2595) 
Change Client.Get to return an io.ReadCloser instead of io.Reader
so that it can be closed.
 2018-11-08 17:02:14 -08:00
Praveen raj Mani
449cad0eb8 Migrate from "mime" to "github.com/minio/minio/pkg/mimedb" (#2563) 
The content-type resolutions are standardized across platforms.

Fixes #2559
 2018-10-02 17:17:44 -07:00
Harshavardhana
a83ffe321c Implement select command (#2538) 2018-09-30 23:18:33 -07:00
Harshavardhana
12627c35e4 Fix parallel upload performance issues (#2521) 
Tested by transferring 29GiB data, around 246569 files
over 100Gig network.

- master branch
```
time mc mirror /root myminio/testbucket/
real	2m10s
```

- after changes
```
time mc mirror /root myminio/testbucket/
real	1m2s
```

By default runtime.NumCPU() workers are started.
 2018-08-30 22:31:08 +05:30
Harshavardhana
7f01db2cda create bucket automatically when prefix is created (#2488) 2018-07-27 15:29:53 -07:00
Harshavardhana
4ab7f30f99 Handle directories properly in cp and mirror (#2478) 
Fixes #2477
 2018-06-29 16:56:02 -07:00
Harshavardhana
29f2dbca6b Fix 'mc mb' to guide the user when bucket is not passed (#2476) 
Additionally this PR also allows for creating
prefixes on S3 compatible object storage backends
in the following form

```
mc mb s3/harshavardhana/object/prefix/
```
 2018-06-29 14:56:07 -07:00
Harshavardhana
afcfe758f3 Fix joinPath function to handle directories properly (#2459) 2018-06-06 11:29:05 +05:30
Anis Elleuch
c0b89a7aec Add support of empty directories in ls/rm/find commands (#2452) 2018-06-04 17:20:07 -07:00
Harshavardhana
75e9691616 Support server side copy object with progress (#2438) 
Support server side copy object with progress
and also uses a more optimized version to perform
multipart server side copy operation.
 2018-04-26 11:02:06 +05:30
poornas
fe82b0381c change encryption key parsing to allow spaces in sse-c key (#2408) 
Also vendoring in minio-go updates
 2018-03-14 02:47:14 -07:00
poornas
0ede95b65e add encrypt option to cp, mirror,pipe,stat, cat and rm commands (#2400) 2018-03-08 17:18:41 +05:30
Nitish Tiwari
ea580b0b65 Add storage-class support (#2403) 
This PR adds storage-class support for mc cp and mirror command.
Storage class values are passed via the -storage-class or
--sc flag.

Fixes #1859
 2018-03-03 10:28:53 -08:00
kannappanr
b1dd99c94d Detect duplicate config entry in mc add event (#2402) 
Fixes #2324
 2018-02-26 14:28:33 -08:00
kannappanr
57bf0c0e79 Fix moving standard metadata as user defined header in s3Client's PUT method (#2401) 
Standard headers like Content-Disposition, Content-Encoding,
Cache-Control, Content-Language were made part of user metadata
causing it to fail while mirroring, if the source has these headers
set.

Fixes #2358
 2018-02-24 15:12:01 -08:00
poornas
65e22bee24 Add api as option for mc config host add command. (#2381) 2018-02-10 14:39:12 -08:00
poornas
5d703df69d vendorize minio-go to version 4.0.6 (#2357) 2018-01-16 12:43:49 +05:30
poornas
5dee56d5d0 fix for prefix based rm (#2354) 
mc rm --r with prefix not terminated by / doesn't get deleted.
example: mc rm --r --force --older-than=10 play/test/prefix

Fixes: #2341
 2018-01-12 09:23:03 +05:30
poornas
275db95170 mc: Remove MC_MULTIPART_THREADS env variable (#2326) 
Remove this var as PR#2289 will manage threads automatically based on
workload to speed up mc cp/mirror operations.
 2017-12-06 09:30:20 -08:00
poornas
1e7b4c9e4a Add mc rm support to recursively remove site (#2265) 
This requires --dangerous flag in addition to --recursive and --force

Change namespace identification check to be more robust to user
input errors like mc rm --r --force play//
 2017-12-01 18:25:08 -08:00
poornas
ab6b2a83ac Add new stat sub command (#2275) 2017-10-31 13:56:40 -07:00
Harshavardhana
aa8d5ed0cc Add newS3Config helper to auto-probe signature type. (#2292) 2017-10-20 13:51:56 -07:00
poornas
a86080a0b7 Add context in Put api call for cp/mirror. (#2291) 2017-10-20 13:50:35 -07:00
poornas
26c7c3e500 ls: show ETag in json formatted output (#2285) 2017-10-13 18:23:13 -07:00
poornas
224e602e59 Allow multipart threads to be configured via environment variable (#2254) 2017-10-02 19:45:08 -07:00
Harshavardhana
fe583468ce Vendorize new changes from minio-go (#2246) 
This PR brings in changes for

- PutObjectStream with no content-length is heavily
  optimized so `mc pipe` will not run out of memory
  for large uploads.
- Fixes a bug in Signature v2 for HTTP requests.
- Support us-gov-west-1 region as well.
- Auto region handling is inside minio-go now.
 2017-09-03 13:56:22 -07:00