1
0
mirror of https://github.com/mariadb-corporation/libmarias3.git synced 2025-08-09 19:22:43 +03:00

29 Commits

Author SHA1 Message Date
Andrew Hutchings
d23e1ad9df Fix ms3_get_content_type() (#128)
Now uses the legacy curl method of header callbacks. #ifdef guard
removed.
2024-09-27 10:23:30 +01:00
Andrew Hutchings
93b1b1e5fd Add Content-Type support for object mime
This adds two new functions `ms3_set_content_type` and
`ms3_get_content_type`.
2024-09-10 15:19:06 +03:00
Andrew Hutchings
70b9a93fd6 Fix building
* RTD theme no longer builds with latest Sphinx. This updates it.
* Fix types based docs compile issues.
* Fix compiler warning not seeing a size check.
2024-09-10 13:45:32 +03:00
Monty
8829031736 Update docs for ms3_debug() API change
Author: Andrew Hutchings <andrew@mariadb.org>
2024-09-10 11:09:39 +03:00
Alexey Antipovsky
f74150b056 feat(curl,timeouts): MCOL-5785: Add functionality to set curl timeouts (#117) 2024-08-08 15:40:43 +03:00
Markus Mäkelä
d2dcab4e95 Add custom callback mechanism (#113)
* Allow custom read callbacks to be used

This allows handling of data in smaller chunks if the ms3_get result is
too large to fit into memory.

* Use the configured buffer size in curl

This way the chunks that curl will return will be the same size that the
internal buffer will be. For the custom read callback, this will allow
the data to be handled in smaller chunks.

* Fix copyright year in read_cb.c

The year is 2023.
2023-06-26 15:13:43 +03:00
Otto Kekäläinen
96053cf234 Fix spelling: occured -> occurred
All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer Amazon Web
Services, Inc.
2023-03-11 12:05:28 -08:00
Andrew Hutchings
d766a449c5 Add new XML parser and fix bugs
LibXML has been removed in favour of a modified version of xml.c
(modified to work with AWS responses and our code base usage).

Also fixes:

* AC_MSG_ERROR usage (fixes #83)
* Some places where we weren't using ms3_c alloc functions
* ms3_get doing an ms3_list with an empty key (fixes #82)
2020-04-20 19:49:52 +01:00
Andrew Hutchings
a3523c694a Revert "Remove domain bucket access method"
This reverts commit 7a02ffa83c.
2019-10-31 16:19:01 +00:00
Andrew Hutchings
7a02ffa83c Remove domain bucket access method
"Buckets created in Regions launched after March 20, 2019 are not
reachable via the https://bucket.s3.amazonaws.com naming scheme."

This patch enhances the one Markus wrote to just completely remove the
domain based access method.
2019-10-30 11:32:42 +00:00
Andrew Hutchings
cfde4a9c5d Improve linked list memory handling
Now does 2 mallocs per 1024 list entries. Makes ms3_list_free() a NULL
operation (all handled internally now).
2019-06-28 10:34:31 +01:00
Andrew Hutchings
9121dad663 Fix S3 compatible compatibility
* Fix path based access
* Improved autodetection of type of method to use
2019-06-26 15:36:46 +01:00
Andrew Hutchings
e850cd25b9 Fix warnings and add force protocol option
Add the warning -Wdeclaration-after-statement and fix everything it
finds.

Add MS3_OPT_FORCE_PROTOCOL_VERSION to ms3_set_option() with a test to
force protocol version when the autodetection does the wrong thing.
2019-05-31 15:22:51 +01:00
Andrew Hutchings
598c364742 Cleanups for removing C99 support and 3.0 release
* Remove bool from public interface
* Fix CLang compiling
* Remove deprecated functions
* Bump version

Fixes mariadb-corporation/libmarias3#51
2019-05-13 10:18:44 +01:00
Andrew Hutchings
2f818d5904 Add ms3_list_dir()
This function acts like ms3_list() but only looks in the prefix up to
the first slash and includes directory names.

Also remove the 1MB lower limit on chunk size
2019-05-07 12:01:13 +01:00
Andrew Hutchings
e8a54506bc Add ms3_copy() and ms3_move() functions
These implement internal S3 copy functionality
2019-04-23 07:36:28 +01:00
Andrew Hutchings
a8366f602a Add library deinit and improve init_malloc
Add deinit to cleanup library and make return code for init_malloc
consistent.
2019-04-22 07:47:22 +01:00
Andrew Hutchings
11f5226197 Add init function with custom malloc
You can now specify a custom allocator

Fixes mariadb-corporation/libmarias3#39
2019-04-21 10:29:29 +01:00
Andrew Hutchings
a4b269d14b Add ms3_init() to replace ms3_thread_init()
Helps with confusing name.

Fixes mariadb-corporation/libmarias3#40
2019-04-21 09:31:17 +01:00
Andrew Hutchings
c5d3f728ee Add V1 bucket list API support
* Will use V1 by default for non-Amazon S3 servers
* V1 or V2 can be forced using ms3_set_option()
* Updated tests to test for both
* Set version and ABI version correctly
2019-03-29 09:37:16 +00:00
Andrew Hutchings
cf300d35ca Add ms3_set_option() and several options
* Add ms3_set_option() API call
* Add options for http:// instead of https://, disabling SSL
verification and buffer chunk size.
* Deprecated ms3_buffer_chunk_size() (use ms3_set_option() instead)
* Added server/curl error messages to debugging output
2019-03-29 08:35:58 +00:00
Andrew Hutchings
1f08de4351 Add ms3_server_error() and remove ms3_init()
* ms3_server_error() gets the last server error or Curl error
* Remove the deprecated ms3_init(), it could not work with
ms3_server_error() and no one uses it
* Test cleanups
* Version bump for semver
2019-03-28 20:54:39 +00:00
Andrew Hutchings
7b3b3d6701 Add ms3_buffer_chunk_size()
Allows you to increase the buffer chunk size for the receive buffer to
reduce reallocs.

Fixes mariadb-corporation/libmarias3#26
2019-03-28 17:37:04 +00:00
Andrew Hutchings
b0ad2d9274 Add function ms3_free()
To free data allocated by ms3_get.

Fixes mariadb-corporation/libmarias3#17
2019-03-28 17:17:33 +00:00
Andrew Hutchings
a99add660a Add connection reuse functionality
CURL will automatically reuse connections if the CURL object is reused.
This commit adds functionality so that a thread can have a CURL object
that can be reused without dramatically altering the API.

It add two new functions:

* ms3_library_init() - to initialize curl for multithreaded operations
* ms3_thread_init() - to initialize a connection object for a thread

This patch also fixes tests that are affecting each other.
2019-03-27 08:23:27 +00:00
Andrew Hutchings
f2b554a081 Make ms3_list() use paginated calls
Before this the list response was capped at 1000 items.

This fix also makes ms3_list() thread safe.
2019-03-26 11:00:35 +00:00
Andrew Hutchings
8176772266 Move error codes to public API
Fixes mariadb-corporation/libmarias3#8
2019-03-26 08:13:42 +00:00
Andrew Hutchings
639dd0ea6e Fix static analysis errors
Tested using valgrind, scanbuild and cppcheck.

* Found that mhash has a 4GB limit, imposed a hard limit on ms3_put()
* Fixed some leaks on error conditions
* Other minor issues
2019-03-23 06:39:37 +00:00
Andrew Hutchings
1c1840dbe0 First version dump 2019-03-22 08:00:38 +00:00