Skip to content
Projects
Groups
Snippets
Help
This project
Loading...
Sign in / Register
Toggle navigation
U
uppy-s3_multipart
Project
Overview
Details
Activity
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
0
Issues
0
List
Board
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
open-source
uppy-s3_multipart
Commits
5e44c6a2
Unverified
Commit
5e44c6a2
authored
May 17, 2019
by
Janko Marohnić
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
Tweak the documentation
parent
e1760e54
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
83 additions
and
93 deletions
+83
-93
README.md
README.md
+83
-93
No files found.
README.md
View file @
5e44c6a2
# Uppy::S3Multipart
# Uppy::S3Multipart
Provides a Rack application that implements endpoints for the
[
AwsS3M
ultipart
]
Provides a Rack application that implements endpoints for the
[
aws-s3-m
ultipart
]
Uppy plugin. This enables multipart uploads directly to S3, which is
Uppy plugin. This enables multipart uploads directly to S3, which is
recommended when dealing with large files, as it allows resuming interrupted
recommended when dealing with large files, as it allows resuming interrupted
uploads.
uploads.
...
@@ -10,12 +10,12 @@ uploads.
...
@@ -10,12 +10,12 @@ uploads.
Add the gem to your Gemfile:
Add the gem to your Gemfile:
```
rb
```
rb
gem
"uppy-s3_multipart"
,
"~> 0.
2
"
gem
"uppy-s3_multipart"
,
"~> 0.
3
"
```
```
## Setup
## Setup
In order to allow direct multipart uploads to your S3 bucket,
we
need to update
In order to allow direct multipart uploads to your S3 bucket,
you
need to update
the bucket's CORS configuration. In the AWS S3 Console go to your bucket, click
the bucket's CORS configuration. In the AWS S3 Console go to your bucket, click
on "Permissions" tab and then on "CORS configuration". There paste in the
on "Permissions" tab and then on "CORS configuration". There paste in the
following:
following:
...
@@ -51,11 +51,63 @@ CORS settings to be applied.
...
@@ -51,11 +51,63 @@ CORS settings to be applied.
This gem provides a Rack application that you can mount inside your main
This gem provides a Rack application that you can mount inside your main
application. If you're using
[
Shrine
]
, you can initialize the Rack application
application. If you're using
[
Shrine
]
, you can initialize the Rack application
via the
`uppy_s3_multipart`
Shrine plugin, otherwise you can initialize it
via the
[
Shrine plugin
](
#shrine
)
.
directly.
### App
At its core, you initialize an
`Uppy::S3Multipart::App`
with an
`Aws::S3::Bucket`
object:
```
rb
require
"uppy/s3_multipart"
bucket
=
Aws
::
S3
::
Bucket
.
new
(
name:
"my-bucket"
,
access_key_id:
"..."
,
secret_access_key:
"..."
,
region:
"..."
,
)
UPPY_S3_MULTIPART_APP
=
Uppy
::
S3Multipart
::
App
.
new
(
bucket:
bucket
)
```
The instance of
`Uppy::S3Multipart::App`
is a Rack application that can be
mounted in your router (
`config/routes.rb`
in Rails). It should be
mounted at
`/s3/multipart`
:
```
rb
# config/routes.rb (Rails)
Rails
.
application
.
routes
.
draw
do
# ...
mount
UPPY_S3_MULTIPART_APP
=>
"/s3/multipart"
end
```
This will add the routes that the
`aws-s3-multipart`
Uppy plugin expects:
```
POST /s3/multipart
GET /s3/multipart/:uploadId
GET /s3/multipart/:uploadId/:partNumber
POST /s3/multipart/:uploadId/complete
DELETE /s3/multipart/:uploadId
```
Since your app will now play the role of Uppy Companion, in your Uppy
configuration you can point
`companionUrl`
to your app's URL:
```
js
// ...
uppy
.
use
(
Uppy
.
AwsS3Multipart
,
{
companionUrl
:
'/'
,
})
```
### Shrine
### Shrine
If you're using Shrine, you can use the
`uppy_s3_multipart`
Shrine plugin that
ships with this gem to simplify the setup.
In your Shrine initializer load the
`uppy_s3_multipart`
plugin:
In your Shrine initializer load the
`uppy_s3_multipart`
plugin:
```
rb
```
rb
...
@@ -71,20 +123,15 @@ Shrine.storages = {
...
@@ -71,20 +123,15 @@ Shrine.storages = {
Shrine
.
plugin
:uppy_s3_multipart
# load the plugin
Shrine
.
plugin
:uppy_s3_multipart
# load the plugin
```
```
The plugin will provide a
`Shrine.uppy_s3_multipart`
method that creates a new
The plugin will provide a
`Shrine.uppy_s3_multipart`
method that creates the
`Uppy::S3Multipart::App`
instance, which is a Rack app that you can mount
`Uppy::S3Multipart::App`
instance, which you can then mount inside your router:
inside your main application:
```
rb
```
rb
#
Rails (config/routes.rb
)
#
config/routes.rb (Rails
)
Rails
.
application
.
routes
.
draw
do
Rails
.
application
.
routes
.
draw
do
# ...
mount
Shrine
.
uppy_s3_multipart
(
:cache
)
=>
"/s3/multipart"
mount
Shrine
.
uppy_s3_multipart
(
:cache
)
=>
"/s3/multipart"
end
end
# Rack (config.ru)
map
"/s3/multipart"
do
run
Shrine
.
uppy_s3_multipart
(
:cache
)
end
```
```
Now in your Uppy configuration point
`companionUrl`
to your app's URL:
Now in your Uppy configuration point
`companionUrl`
to your app's URL:
...
@@ -96,9 +143,9 @@ uppy.use(Uppy.AwsS3Multipart, {
...
@@ -96,9 +143,9 @@ uppy.use(Uppy.AwsS3Multipart, {
})
})
```
```
In the
`upload-success`
Uppy callback
you can then construct the Shrine
In the
`upload-success`
Uppy callback
, you can construct the Shrine uploaded
uploaded file data (this example assumes your temporary Shrine S3 storage has
file data (this example assumes your temporary Shrine S3 storage has
`prefix:
`prefix:
"cache"`
set):
"cache"`
set):
```
js
```
js
uppy
.
on
(
'upload-success'
,
function
(
file
,
response
)
{
uppy
.
on
(
'upload-success'
,
function
(
file
,
response
)
{
...
@@ -115,67 +162,15 @@ uppy.on('upload-success', function (file, response) {
...
@@ -115,67 +162,15 @@ uppy.on('upload-success', function (file, response) {
})
})
```
```
**
See
[
Adding Direct S3 Uploads
]
for an example of a complete Uppy setup with
See
[
Adding Direct S3 Uploads
]
for an example of a complete Uppy setup with
Shrine. From there you can swap the
`presign_endpoint`
+
`
AwsS
3`
code with the
Shrine. From there you can swap the
`presign_endpoint`
+
`
aws-s
3`
code with the
`uppy_s3_multipart`
+
`
AwsS3Multipart`
setup.
**
`uppy_s3_multipart`
+
`
aws-s3-multipart`
setup.
Note that
**
Shrine won't extract metadata from directly upload files on
Note that
**
Shrine won't extract metadata from directly upload files on
assignment
**
by default. Instead, it will just copy metadata that was extracted
assignment
**
by default. Instead, it will just copy metadata that was extracted
on the client side. See
[
this section
][
metadata direct uploads
]
for the
on the client side. See
[
this section
][
metadata direct uploads
]
for the
rationale and instructions on how to opt in.
rationale and instructions on how to opt in.
### App
You can also use
`uppy-s3_multipart`
without Shrine, by initializing the
`Uppy::S3Multipart::App`
directly:
```
rb
require
"uppy/s3_multipart"
resource
=
Aws
::
S3
::
Resource
.
new
(
access_key_id:
"..."
,
secret_access_key:
"..."
,
region:
"..."
,
)
bucket
=
resource
.
bucket
(
"my-bucket"
)
UPPY_S3_MULTIPART_APP
=
Uppy
::
S3Multipart
::
App
.
new
(
bucket:
bucket
)
```
You can mount it inside your main app in the same way:
```
rb
# Rails (config/routes.rb)
Rails
.
application
.
routes
.
draw
do
mount
UPPY_S3_MULTIPART_APP
=>
"/s3/multipart"
end
# Rack (config.ru)
map
"/s3/multipart"
do
run
UPPY_S3_MULTIPART_APP
end
```
This will add the routes that the
`AwsS3Multipart`
Uppy plugin expects:
```
POST /s3/multipart
GET /s3/multipart/:uploadId
GET /s3/multipart/:uploadId/:partNumber
POST /s3/multipart/:uploadId/complete
DELETE /s3/multipart/:uploadId
```
Now in your Uppy configuration point
`companionUrl`
to your app's URL:
```
js
// ...
uppy
.
use
(
Uppy
.
AwsS3Multipart
,
{
companionUrl
:
'/'
,
})
```
### Configuration
### Configuration
This section describe various configuration options that you can pass to
This section describe various configuration options that you can pass to
...
@@ -183,40 +178,41 @@ This section describe various configuration options that you can pass to
...
@@ -183,40 +178,41 @@ This section describe various configuration options that you can pass to
#### `:bucket`
#### `:bucket`
The
`:bucket`
option is mandatory and accepts an instance of
`Aws::S3::Bucket`
.
The
`:bucket`
option is mandatory and accepts an instance of
`Aws::S3::Bucket`
:
It's easiest to create an
`Aws::S3::Resource`
, and call
`#bucket`
on it.
```
rb
```
rb
require
"uppy/s3_multipart"
require
"uppy/s3_multipart"
resource
=
Aws
::
S3
::
Resource
.
new
(
bucket
=
Aws
::
S3
::
Bucket
.
new
(
name:
"<BUCKET>"
,
access_key_id:
"<ACCESS_KEY_ID>"
,
access_key_id:
"<ACCESS_KEY_ID>"
,
secret_access_key:
"<SECRET_ACCESS_KEY>"
,
secret_access_key:
"<SECRET_ACCESS_KEY>"
,
region:
"<REGION>"
,
region:
"<REGION>"
,
)
)
bucket
=
resource
.
bucket
(
"<BUCKET>"
)
Uppy
::
S3Multipart
::
App
.
new
(
bucket:
bucket
)
Uppy
::
S3MUltipart
::
App
.
new
(
bucket:
bucket
)
```
```
If you want to use
[
Minio
]
, you can easily configure
your
`Aws::S3::Bucket`
to
If you want to use
[
Minio
]
, you can easily configure
the
`Aws::S3::Bucket`
to
point to
your Minio server:
use
your Minio server:
```
rb
```
rb
resource
=
Aws
::
S3
::
Resource
.
new
(
bucket
=
Aws
::
S3
::
Bucket
.
new
(
name:
"<MINIO_BUCKET>"
,
access_key_id:
"<MINIO_ACCESS_KEY>"
,
# "AccessKey" value
access_key_id:
"<MINIO_ACCESS_KEY>"
,
# "AccessKey" value
secret_access_key:
"<MINIO_SECRET_KEY>"
,
# "SecretKey" value
secret_access_key:
"<MINIO_SECRET_KEY>"
,
# "SecretKey" value
endpoint:
"<MINIO_ENDPOINT>"
,
# "Endpoint" value
endpoint:
"<MINIO_ENDPOINT>"
,
# "Endpoint" value
region:
"us-east-1"
,
region:
"us-east-1"
,
force_path_style:
true
,
)
)
bucket
=
resource
.
bucket
(
"<MINIO_BUCKET>"
)
# name of the bucket you created
Uppy
::
S3Multipart
::
App
.
new
(
bucket:
bucket
)
```
```
See the
[
`Aws::S3::Client#initialize`
]
docs for all supported configuration
Except for
`:name`
, all options passed to
[
`Aws::S3::Bucket#initialize`
]
are
options. In the Shrine plugin this option is inferred from the S3 storage.
forwarded to
[
`Aws::S3::Client#initialize`
]
, see its documentation for
additional options.
In the Shrine plugin this configuration is inferred from the S3 storage.
#### `:prefix`
#### `:prefix`
...
@@ -227,14 +223,7 @@ to be uploaded to.
...
@@ -227,14 +223,7 @@ to be uploaded to.
Uppy
::
S3Multipart
::
App
.
new
(
bucket:
bucket
,
prefix:
"cache"
)
Uppy
::
S3Multipart
::
App
.
new
(
bucket:
bucket
,
prefix:
"cache"
)
```
```
In the Shrine plugin this option is inferred from the S3 storage:
In the Shrine plugin this option is inferred from the S3 storage.
```
rb
Shrine
.
storages
=
{
cache:
Shrine
::
Storage
::
S3
.
new
(
prefix:
"cache"
,
**
options
),
store:
Shrine
::
Storage
::
S3
.
new
(
**
options
),
}
```
#### `:options`
#### `:options`
...
@@ -482,6 +471,7 @@ License](https://opensource.org/licenses/MIT).
...
@@ -482,6 +471,7 @@ License](https://opensource.org/licenses/MIT).
[
Client
]:
#client
[
Client
]:
#client
[
content_disposition
]:
https://github.com/shrinerb/content_disposition
[
content_disposition
]:
https://github.com/shrinerb/content_disposition
[
`Rack::Request`
]:
https://www.rubydoc.info/github/rack/rack/master/Rack/Request
[
`Rack::Request`
]:
https://www.rubydoc.info/github/rack/rack/master/Rack/Request
[
`Aws::S3::Bucket#initialize`
]:
https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3/Bucket.html#initialize-instance_method
[
`Aws::S3::Client#initialize`
]:
https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3/Client.html#initialize-instance_method
[
`Aws::S3::Client#initialize`
]:
https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3/Client.html#initialize-instance_method
[
`Aws::S3::Client#create_multipart_upload`
]:
https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3/Client.html#create_multipart_upload-instance_method
[
`Aws::S3::Client#create_multipart_upload`
]:
https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3/Client.html#create_multipart_upload-instance_method
[
`Aws::S3::Client#list_parts`
]:
https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3/Client.html#list_parts-instance_method
[
`Aws::S3::Client#list_parts`
]:
https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3/Client.html#list_parts-instance_method
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment