Initial commit

.gitignore

+pkg/
+Gemfile.lock

CODE_OF_CONDUCT.md

+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+In the interest of fostering an open and welcoming environment, we as
+contributors and maintainers pledge to making participation in our project and
+our community a harassment-free experience for everyone, regardless of age, body
+size, disability, ethnicity, gender identity and expression, level of experience,
+nationality, personal appearance, race, religion, or sexual identity and
+orientation.
+
+## Our Standards
+
+Examples of behavior that contributes to creating a positive environment
+include:
+
+* Using welcoming and inclusive language
+* Being respectful of differing viewpoints and experiences
+* Gracefully accepting constructive criticism
+* Focusing on what is best for the community
+* Showing empathy towards other community members
+
+Examples of unacceptable behavior by participants include:
+
+* The use of sexualized language or imagery and unwelcome sexual attention or
+advances
+* Trolling, insulting/derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or electronic
+  address, without explicit permission
+* Other conduct which could reasonably be considered inappropriate in a
+  professional setting
+
+## Our Responsibilities
+
+Project maintainers are responsible for clarifying the standards of acceptable
+behavior and are expected to take appropriate and fair corrective action in
+response to any instances of unacceptable behavior.
+
+Project maintainers have the right and responsibility to remove, edit, or
+reject comments, commits, code, wiki edits, issues, and other contributions
+that are not aligned to this Code of Conduct, or to ban temporarily or
+permanently any contributor for other behaviors that they deem inappropriate,
+threatening, offensive, or harmful.
+
+## Scope
+
+This Code of Conduct applies both within project spaces and in public spaces
+when an individual is representing the project or its community. Examples of
+representing a project or community include using an official project e-mail
+address, posting via an official social media account, or acting as an appointed
+representative at an online or offline event. Representation of a project may be
+further defined and clarified by project maintainers.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported by contacting the project team at janko.marohnic@gmail.com. All
+complaints will be reviewed and investigated and will result in a response that
+is deemed necessary and appropriate to the circumstances. The project team is
+obligated to maintain confidentiality with regard to the reporter of an incident.
+Further details of specific enforcement policies may be posted separately.
+
+Project maintainers who do not follow or enforce the Code of Conduct in good
+faith may face temporary or permanent repercussions as determined by other
+members of the project's leadership.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
+available at [http://contributor-covenant.org/version/1/4][version]
+
+[homepage]: http://contributor-covenant.org
+[version]: http://contributor-covenant.org/version/1/4/

Gemfile

+source "https://rubygems.org"
+
+gemspec
+
+gem "pry"
+
+gem "puma"
+gem "tilt"

LICENSE.txt

+The MIT License (MIT)
+
+Copyright (c) 2018 Janko Marohnić
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

README.md

+# Uppy::S3Multipart
+
+Provides a Rack application that implements endpoints for the [AwsS3Multipart]
+Uppy plugin. This enables multipart uploads directly to S3, which is
+recommended when dealing with large files, as it allows resuming interrupted
+uploads.
+
+## Installation
+
+Add the gem to your Gemfile:
+
+```rb
+gem "uppy-s3_multipart"
+```
+
+## Setup
+
+Once you've created your S3 bucket, you need to set up CORS for it. The
+following script sets up minimal CORS configuration needed for multipart
+uploads on your bucket using the `aws-sdk-s3` gem:
+
+```rb
+require "aws-sdk-s3"
+
+client = Aws::S3::Client.new(
+  access_key_id:     "<YOUR KEY>",
+  secret_access_key: "<YOUR SECRET>",
+  region:            "<REGION>",
+)
+
+client.put_bucket_cors(
+  bucket: "<YOUR BUCKET>",
+  cors_configuration: {
+    cors_rules: [{
+      allowed_headers: ["Authorization", "Content-Type", "Origin", "ETag"],
+      allowed_methods: ["GET", "POST", "PUT", "DELETE"],
+      allowed_origins: ["*"],
+      max_age_seconds: 3000,
+    }]
+  }
+)
+```
+
+## Usage
+
+This gem provides a Rack application that you can mount inside your main
+application. If you're using [Shrine], you can initialize the Rack application
+via the `uppy_s3_multipart` Shrine plugin, otherwise you can initialize it
+directly.
+
+### Shrine
+
+In the initializer load the `uppy_s3_multipart` plugin:
+
+```rb
+require "shrine"
+require "shrine/storage/s3"
+
+Shrine.storages = {
+  cache: Shrine::Storage::S3.new(...),
+  store: Shrine::Storage::S3.new(...),
+}
+
+# ...
+Shrine.plugin :uppy_s3_multipart # load the plugin
+```
+
+The plugin will provide a `Shrine.uppy_s3_multipart` method, which returns an
+instance of `Uppy::S3Multipart::App`, which is a Rack app that you can mount
+inside your main application:
+
+```rb
+# Rails (config/routes.rb)
+Rails.application.routes.draw do
+  mount Shrine.uppy_s3_multipart(:cache) => "/s3"
+end
+
+# Rack (config.ru)
+map "/s3" do
+  run Shrine.uppy_s3_multipart(:cache)
+end
+```
+
+This will add the routes that the `AwsS3Multipart` Uppy plugin expects:
+
+```
+POST   /s3/multipart
+GET    /s3/multipart/:uploadId
+GET    /s3/multipart/:uploadId/:partNumber
+POST   /s3/multipart/:uploadId/complete
+DELETE /s3/multipart/:uploadId
+```
+
+Finally, in your Uppy configuration pass your app's URL as the `serverUrl`:
+
+```js
+// ...
+uppy.use(Uppy.AwsS3Multipart, {
+  serverUrl: "https://your-app.com/",
+})
+```
+
+Both the plugin and method accepts `:options` for specifying additional options
+to the aws-sdk calls (read further for more details on these options):
+
+```rb
+Shrine.plugin :uppy_s3_multipart, options: {
+  create_multipart_upload: { acl: "public-read" } # static
+}
+
+# OR
+
+Shrine.uppy_s3_multipart(:cache, options: {
+  create_multipart_upload: -> (request) { { acl: "public-read" } } # dynamic
+})
+```
+
+### Standalone
+
+You can also initialize `Uppy::S3Multipart::App` directly:
+
+```rb
+require "uppy/s3_multipart"
+
+resource = Aws::S3::Resource.new(
+  access_key_id:     "...",
+  secret_access_key: "...",
+  region:            "...",
+)
+
+bucket = resource.bucket("my-bucket")
+
+UPPY_S3_MULTIPART_APP = Uppy::S3Multipart::App.new(bucket: bucket)
+```
+
+and mount it in your app in the same way:
+
+```rb
+# Rails (config/routes.rb)
+Rails.application.routes.draw do
+  mount UPPY_S3_MULTIPART_APP => "/s3"
+end
+
+# Rack (config.ru)
+map "/s3" do
+  run UPPY_S3_MULTIPART_APP
+end
+```
+
+In your Uppy configuration point the `serverUrl` to your application:
+
+```js
+// ...
+uppy.use(Uppy.AwsS3Multipart, {
+  serverUrl: "https://your-app.com/",
+})
+```
+
+The `Uppy::S3Mutipart::App` initializer accepts `:options` for specifying
+additional options to the aws-sdk calls (read further for more details on these
+options):
+
+```rb
+Uppy::S3Multipart::App.new(bucket: bucket, options: {
+  create_multipart_upload: { acl: "public-read" }
+})
+
+# OR
+
+Uppy::S3Multipart::App.new(bucket: bucket, options: {
+  create_multipart_upload: -> (request) { { acl: "public-read" } }
+})
+```
+
+### Custom implementation
+
+If you would rather implement the endpoints yourself, you can utilize
+`Uppy::S3Multipart::Client` to make S3 requests.
+
+```rb
+require "uppy/s3_multipart/client"
+
+client = Uppy::S3Multipart::Client.new(bucket: bucket)
+```
+
+#### `create_multipart_upload`
+
+Initiates a new multipart upload.
+
+```rb
+client.create_multipart_upload(key: "foo", **options)
+#=> { upload_id: "MultipartUploadId", key: "foo" }
+```
+
+Accepts:
+
+* `:key` -- object key
+* additional options for [`Aws::S3::Client#create_multipart_upload`]
+
+Returns:
+
+* `:upload_id` -- id of the created multipart upload
+* `:key` -- object key
+
+#### `#list_parts`
+
+Retrieves currently uploaded parts of a multipart upload.
+
+```rb
+client.list_parts(upload_id: "MultipartUploadId", key: "foo", **options)
+#=> [ { part_number: 1, size: 5402383, etag: "etag1" },
+#     { part_number: 2, size: 5982742, etag: "etag2" },
+#     ... ]
+```
+
+Accepts:
+
+* `:upload_id` -- multipart upload id
+* `:key` -- object key
+* additional options for [`Aws::S3::Client#list_parts`]
+
+Returns:
+
+* array of parts
+
+  - `:part_number` -- position of the part
+  - `:size` -- filesize of the part
+  - `:etag` -- etag of the part
+
+#### `#prepare_upload_part`
+
+Returns the endpoint that should be used for uploading a new multipart part.
+
+```rb
+client.prepare_upload_part(upload_id: "MultipartUploadId", key: "foo", part_number: 1, **options)
+#=> { url: "https://my-bucket.s3.amazonaws.com/foo?partNumber=1&uploadId=MultipartUploadId&..." }
+```
+
+Accepts:
+
+* `:upload_id` -- multipart upload id
+* `:key` -- object key
+* `:part_number` -- number of the next part
+* additional options for [`Aws::S3::Client#upload_part`] and [`Aws::S3::Presigner#presigned_url`]
+
+Returns:
+
+* `:url` -- endpoint that should be used for uploading a new multipart part via a `PUT` request
+
+#### `#complete_multipart_upload`
+
+Finalizes the multipart upload and returns URL to the object.
+
+```rb
+client.complete_multipart_upload(upload_id: upload_id, key: key, parts: [{ part_number: 1, etag: "etag1" }], **options)
+#=> { location: "https://my-bucket.s3.amazonaws.com/foo?..." }
+```
+
+Accepts:
+
+* `:upload_id` -- multipart upload id
+* `:key` -- object key
+* `:parts` -- list of all uploaded parts, consisting of `:part_number` and `:etag`
+* additional options for [`Aws::S3::Client#complete_multipart_upload`]
+
+Returns:
+
+* `:location` -- URL to the uploaded object
+
+#### `#abort_multipart_upload`
+
+Aborts the multipart upload, removing all parts uploaded so far.
+
+```rb
+client.abort_multipart_upload(upload_id: upload_id, key: key, **options)
+#=> {}
+```
+
+Accepts:
+
+* `:upload_id` -- multipart upload id
+* `:key` -- object key
+* additional options for [`Aws::S3::Client#abort_multipart_upload`]
+
+## Contributing
+
+You can run the test suite with
+
+```
+$ bundle exec rake test
+```
+
+This project is intended to be a safe, welcoming space for collaboration, and
+contributors are expected to adhere to the [Contributor
+Covenant](http://contributor-covenant.org) code of conduct.
+
+## License
+
+The gem is available as open source under the terms of the [MIT
+License](https://opensource.org/licenses/MIT).
+
+[AwsS3Multipart]: https://uppy.io/docs/aws-s3-multipart/
+[Shrine]: https://shrinerb.com
+[`Aws::S3::Client#create_multipart_upload`]: https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3/Client.html#create_multipart_upload-instance_method
+[`Aws::S3::Client#list_parts`]: https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3/Client.html#list_parts-instance_method
+[`Aws::S3::Client#upload_part`]: https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3/Client.html#upload_part-instance_method
+[`Aws::S3::Presigner#presigned_url`]: https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3/Presigner.html#presigned_url-instance_method
+[`Aws::S3::Client#complete_multipart_upload`]: https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3/Client.html#complete_multipart_upload-instance_method
+[`Aws::S3::Client#abort_multipart_upload`]: https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3/Client.html#abort_multipart_upload-instance_method

Rakefile

+require "bundler/gem_tasks"
+require "rake/testtask"
+
+Rake::TestTask.new do |t|
+  t.libs << "test"
+  t.test_files = FileList["test/**/*_test.rb"]
+  t.warning = false
+end
+
+task :default => :test

lib/shrine/plugins/uppy_s3_multipart.rb

+require "uppy/s3_multipart"
+
+require "securerandom"
+
+class Shrine
+  module Plugins
+    module UppyS3Multipart
+      def self.configure(uploader, options = {})
+        uploader.opts[:uppy_s3_multipart_options] = (uploader.opts[:uppy_s3_multipart_options] || {}).merge(options[:options] || {})
+      end
+
+      module ClassMethods
+        def uppy_s3_multipart(storage_key, **options)
+          s3 = find_storage(storage_key)
+
+          ::Uppy::S3Multipart::App.new(
+            bucket:  s3.bucket,
+            prefix:  s3.prefix,
+            options: opts[:uppy_s3_multipart_options],
+            **options
+          )
+        end
+      end
+    end
+
+    register_plugin(:uppy_s3_multipart, UppyS3Multipart)
+  end
+end

lib/uppy-s3_multipart.rb

+require "uppy/s3_multipart"

lib/uppy/s3_multipart.rb

+require "uppy/s3_multipart/app"
+require "uppy/s3_multipart/client"

lib/uppy/s3_multipart/app.rb

+require "uppy/s3_multipart/client"
+
+require "roda"
+
+require "securerandom"
+require "cgi"
+
+module Uppy
+  module S3Multipart
+    class App
+      def initialize(bucket:, prefix: nil, options: {})
+        @router = Class.new(Router)
+        @router.opts[:client]  = Client.new(bucket: bucket)
+        @router.opts[:prefix]  = prefix
+        @router.opts[:options] = options
+      end
+
+      def call(env)
+        @router.call(env)
+      end
+
+      class Router < Roda
+        plugin :all_verbs
+        plugin :json
+        plugin :json_parser
+        plugin :halt
+
+        route do |r|
+          # POST /multipart
+          r.post "multipart" do
+            content_type = r.params["type"]
+            filename     = r.params["filename"]
+
+            extension = File.extname(filename.to_s)
+            key       = SecureRandom.hex + extension
+            key       = "#{opts[:prefix]}/#{key}" if opts[:prefix]
+
+            # CGI-escape the filename because aws-sdk's signature calculator trips on special characters
+            content_disposition = "inline; filename=\"#{CGI.escape(filename)}\"" if filename
+
+            result = client_call(:create_multipart_upload, key: key, content_type: content_type, content_disposition: content_disposition)
+
+            { uploadId: result.fetch(:upload_id), key: result.fetch(:key) }
+          end
+
+          # GET /multipart/:uploadId
+          r.get "multipart", String do |upload_id|
+            key = param!("key")
+
+            result = client_call(:list_parts, upload_id: upload_id, key: key)
+
+            result.map do |part|
+              { PartNumber: part.fetch(:part_number), Size: part.fetch(:size), ETag: part.fetch(:etag) }
+            end
+          end
+
+          # GET /multipart/:uploadId/:partNumber
+          r.get "multipart", String, String do |upload_id, part_number|
+            key = param!("key")
+
+            result = client_call(:prepare_upload_part, upload_id: upload_id, key: key, part_number: part_number)
+
+            { url: result.fetch(:url) }
+          end
+
+          # POST /multipart/:uploadId/complete
+          r.post "multipart", String, "complete" do |upload_id|
+            key   = param!("key")
+            parts = param!("parts")
+
+            parts = parts.map do |part|
+              begin
+                { part_number: part.fetch("PartNumber"), etag: part.fetch("ETag") }
+              rescue KeyError
+                r.halt 400, { error: "At least one part is missing \"PartNumber\" or \"ETag\" field" }
+              end
+            end
+
+            result = client_call(:complete_multipart_upload, upload_id: upload_id, key: key, parts: parts)
+
+            { location: result.fetch(:location) }
+          end
+
+          # DELETE /multipart/:uploadId
+          r.delete "multipart", String do |upload_id|
+            key = param!("key")
+
+            client_call(:abort_multipart_upload, upload_id: upload_id, key: key)
+
+            {}
+          end
+        end
+
+        def client_call(operation, **options)
+          client = opts[:client]
+
+          overrides = opts[:options][operation] || {}
+          overrides = overrides.call(request) if overrides.respond_to?(:call)
+
+          options = options.merge(overrides)
+
+          client.send(operation, **options)
+        end
+
+        def param!(name)
+          value = request.params[name]
+
+          request.halt 400, { error: "Missing \"#{name}\" parameter" } if value.nil?
+
+          value
+        end
+      end
+    end
+  end
+end

lib/uppy/s3_multipart/client.rb

+require "aws-sdk-s3"
+
+module Uppy
+  module S3Multipart
+    class Client
+      attr_reader :bucket
+
+      def initialize(bucket:)
+        @bucket = bucket
+      end
+
+      def create_multipart_upload(key:, **options)
+        multipart_upload = object(key).initiate_multipart_upload(**options)
+
+        { upload_id: multipart_upload.id, key: multipart_upload.object_key }
+      end
+
+      def list_parts(upload_id:, key:, **options)
+        multipart_upload = multipart_upload(upload_id, key)
+        multipart_parts  = multipart_upload.parts(**options).to_a
+
+        multipart_parts.map do |part|
+          { part_number: part.part_number, size: part.size, etag: part.etag }
+        end
+      end
+
+      def prepare_upload_part(upload_id:, key:, part_number:, **options)
+        presigned_url = presigner.presigned_url "upload_part",
+          bucket: bucket.name,
+          key: object(key).key,
+          upload_id: upload_id,
+          part_number: part_number,
+          body: "",
+          **options
+
+        { url: presigned_url }
+      end
+
+      def complete_multipart_upload(upload_id:, key:, parts:, **options)
+        multipart_upload = multipart_upload(upload_id, key)
+        multipart_upload.complete(
+          multipart_upload: { parts: parts },
+          **options
+        )
+
+        { location: object(key).presigned_url(:get) }
+      end
+
+      def abort_multipart_upload(upload_id:, key:, **options)
+        multipart_upload = multipart_upload(upload_id, key)
+
+        # aws-sdk-s3 docs recommend retrying the abort in case the multipart
+        # upload still has parts
+        loop do
+          multipart_upload.abort(**options)
+          break unless multipart_upload.parts.any?
+        end
+
+        {}
+      end
+
+      private
+
+      def multipart_upload(upload_id, key)
+        object(key).multipart_upload(upload_id)
+      end
+
+      def object(key)
+        bucket.object(key)
+      end
+
+      def presigner
+        Aws::S3::Presigner.new(client: client)
+      end
+
+      def client
+        bucket.client
+      end
+    end
+  end
+end

test/app_test.rb

+require "test_helper"
+require "aws-sdk-s3"
+require "rack/test_app"
+
+describe Uppy::S3Multipart::App do
+  def app
+    Rack::TestApp.wrap(Rack::Lint.new(@endpoint))
+  end
+
+  before do
+    resource = Aws::S3::Resource.new(stub_responses: true)
+    @bucket  = resource.bucket("my-bucket")
+
+    @endpoint = Uppy::S3Multipart::App.new(bucket: @bucket)
+    @s3       = @bucket.client
+  end
+
+  describe "POST /multipart" do
+    it "creates a multipart upload" do
+      response = app.post "/multipart"
+
+      assert_equal :create_multipart_upload, @s3.api_requests[0][:operation_name]
+      assert_match /^\w{32}$/,               @s3.api_requests[0][:params][:key]
+    end
+
+    it "returns multipart upload id and key" do
+      @s3.stub_responses(:create_multipart_upload, { upload_id: "foo", key: "bar" })
+
+      response = app.post "/multipart"
+
+      assert_equal 200,                response.status
+      assert_equal "application/json", response.headers["Content-Type"]
+
+      assert_equal "foo",      response.body_json["uploadId"]
+      assert_match /^\w{32}$/, response.body_json["key"]
+    end
+
+    it "handles 'type' JSON body parameter" do
+      response = app.post "/multipart", json: { type: "text/plain" }
+
+      assert_equal :create_multipart_upload, @s3.api_requests[0][:operation_name]
+      assert_equal "text/plain",             @s3.api_requests[0][:params][:content_type]
+    end
+
+    it "handles 'filename' JSON body parameter" do
+      response = app.post "/multipart", json: { filename: "nature.jpg" }
+
+      assert_equal :create_multipart_upload,          @s3.api_requests[0][:operation_name]
+      assert_equal "inline; filename=\"nature.jpg\"", @s3.api_requests[0][:params][:content_disposition]
+
+      assert_match /^\w{32}\.jpg$/, response.body_json["key"]
+    end
+
+    it "handles :prefix option" do
+      @endpoint = Uppy::S3Multipart::App.new(bucket: @bucket, prefix: "prefix")
+
+      response = app.post "/multipart"
+
+      assert_match /^prefix\/\w{32}$/, response.body_json["key"]
+    end
+
+    it "handles :options as a hash" do
+      @endpoint = Uppy::S3Multipart::App.new(bucket: @bucket, options: {
+        create_multipart_upload: { acl: "public-read" }
+      })
+
+      response = app.post "/multipart"
+
+      assert_equal :create_multipart_upload, @s3.api_requests[0][:operation_name]
+      assert_equal "public-read",            @s3.api_requests[0][:params][:acl]
+    end
+
+    it "handles :options as a block" do
+      @endpoint = Uppy::S3Multipart::App.new(bucket: @bucket, options: {
+        create_multipart_upload: -> (request) {
+          assert_kind_of Rack::Request, request
+          { acl: "public-read" }
+        }
+      })
+
+      response = app.post "/multipart"
+
+      assert_equal :create_multipart_upload, @s3.api_requests[0][:operation_name]
+      assert_equal "public-read",            @s3.api_requests[0][:params][:acl]
+    end
+  end
+
+  describe "GET /multipart/:uploadId" do
+    it "fetches multipart parts" do
+      response = app.get "/multipart/foo", query: { key: "bar" }
+
+      assert_equal :list_parts, @s3.api_requests[0][:operation_name]
+      assert_equal "foo",       @s3.api_requests[0][:params][:upload_id]
+      assert_equal "bar",       @s3.api_requests[0][:params][:key]
+    end
+
+    it "returns multipart parts" do
+      @s3.stub_responses(:list_parts, { parts: [ { part_number: 1, size: 123, etag: "etag1" } ] })
+
+      response = app.get "/multipart/foo", query: { key: "bar" }
+
+      assert_equal 200,                response.status
+      assert_equal "application/json", response.headers["Content-Type"]
+
+      assert_equal 1,       response.body_json[0]["PartNumber"]
+      assert_equal 123,     response.body_json[0]["Size"]
+      assert_equal "etag1", response.body_json[0]["ETag"]
+    end
+
+    it "returns error response when 'key' parameter is missing" do
+      response = app.get "/multipart/foo"
+
+      assert_equal 400,                response.status
+      assert_equal "application/json", response.headers["Content-Type"]
+
+      assert_equal "Missing \"key\" parameter", response.body_json["error"]
+    end
+
+    it "handles :options as a hash" do
+      @endpoint = Uppy::S3Multipart::App.new(bucket: @bucket, options: {
+        list_parts: { max_parts: 5 }
+      })
+
+      response = app.get "/multipart/foo", query: { key: "bar" }
+
+      assert_equal :list_parts,   @s3.api_requests[0][:operation_name]
+      assert_equal 5,             @s3.api_requests[0][:params][:max_parts]
+    end
+
+    it "handles :options as a block" do
+      @endpoint = Uppy::S3Multipart::App.new(bucket: @bucket, options: {
+        list_parts: -> (request) {
+          assert_kind_of Rack::Request, request
+          { max_parts: 5 }
+        }
+      })
+
+      response = app.get "/multipart/foo", query: { key: "bar" }
+
+      assert_equal :list_parts,   @s3.api_requests[0][:operation_name]
+      assert_equal 5,             @s3.api_requests[0][:params][:max_parts]
+    end
+  end
+
+  describe "GET /multipart/:uploadId/:partNumber" do
+    it "returns presigned url for part upload" do
+      response = app.get "/multipart/foo/1", query: { key: "bar" }
+
+      assert_equal 200,                response.status
+      assert_equal "application/json", response.headers["Content-Type"]
+
+      assert_match URI.regexp, response.body_json["url"]
+    end
+
+    it "returns error response when 'key' parameter is missing" do
+      response = app.get "/multipart/foo/1"
+
+      assert_equal 400,                response.status
+      assert_equal "application/json", response.headers["Content-Type"]
+
+      assert_equal "Missing \"key\" parameter", response.body_json["error"]
+    end
+
+    it "handles :options as a hash" do
+      @endpoint = Uppy::S3Multipart::App.new(bucket: @bucket, options: {
+        prepare_upload_part: { expires_in: 10 }
+      })
+
+      response = app.get "/multipart/foo/1", query: { key: "bar" }
+
+      assert_match "X-Amz-Expires=10", response.body_json["url"]
+    end
+
+    it "handles :options as a block" do
+      @endpoint = Uppy::S3Multipart::App.new(bucket: @bucket, options: {
+        prepare_upload_part: -> (request) {
+          assert_kind_of Rack::Request, request
+          { expires_in: 10 }
+        }
+      })
+
+      response = app.get "/multipart/foo/1", query: { key: "bar" }
+
+      assert_match "X-Amz-Expires=10", response.body_json["url"]
+    end
+  end
+
+  describe "POST /multipart/:uploadId/complete" do
+    it "completes the multipart upload" do
+      response = app.post "/multipart/foo/complete", query: { key: "bar" },
+        json: { parts: [{ PartNumber: 1, ETag: "etag1" }] }
+
+      assert_equal :complete_multipart_upload, @s3.api_requests[0][:operation_name]
+      assert_equal "foo",                      @s3.api_requests[0][:params][:upload_id]
+      assert_equal "bar",                      @s3.api_requests[0][:params][:key]
+      assert_equal 1,                          @s3.api_requests[0][:params][:multipart_upload][:parts][0][:part_number]
+      assert_equal "etag1",                    @s3.api_requests[0][:params][:multipart_upload][:parts][0][:etag]
+    end
+
+    it "returns presigned URL to the object" do
+      response = app.post "/multipart/foo/complete", query: { key: "bar" }, json: { parts: [] }
+
+      assert_equal 200,                response.status
+      assert_equal "application/json", response.headers["Content-Type"]
+
+      assert_match URI.regexp, response.body_json["location"]
+    end
+
+    it "returns error response when 'key' parameter is missing" do
+      response = app.post "/multipart/foo/complete", json: { parts: [] }
+
+      assert_equal 400,                response.status
+      assert_equal "application/json", response.headers["Content-Type"]
+
+      assert_equal "Missing \"key\" parameter", response.body_json["error"]
+    end
+
+    it "returns error response when 'parts' parameter is missing" do
+      response = app.post "/multipart/foo/complete", query: { key: "bar" }
+
+      assert_equal 400,                response.status
+      assert_equal "application/json", response.headers["Content-Type"]
+
+      assert_equal "Missing \"parts\" parameter", response.body_json["error"]
+    end
+
+    it "returns error response when a part is missing 'PartNumber' field" do
+      response = app.post "/multipart/foo/complete", query: { key: "bar" },
+        json: { parts: [{ ETag: 1 }] }
+
+      assert_equal 400,                response.status
+      assert_equal "application/json", response.headers["Content-Type"]
+
+      assert_equal "At least one part is missing \"PartNumber\" or \"ETag\" field", response.body_json["error"]
+    end
+
+    it "returns error response when a part is missing 'ETag' field" do
+      response = app.post "/multipart/foo/complete", query: { key: "bar" },
+        json: { parts: [{ PartNumber: 1 }] }
+
+      assert_equal 400,                response.status
+      assert_equal "application/json", response.headers["Content-Type"]
+
+      assert_equal "At least one part is missing \"PartNumber\" or \"ETag\" field", response.body_json["error"]
+    end
+
+    it "handles :options as a hash" do
+      @endpoint = Uppy::S3Multipart::App.new(bucket: @bucket, options: {
+        complete_multipart_upload: { request_payer: "bob" }
+      })
+
+      response = app.post "/multipart/foo/complete", query: { key: "bar" }, json: { parts: [] }
+
+      assert_equal :complete_multipart_upload, @s3.api_requests[0][:operation_name]
+      assert_equal "bob",                      @s3.api_requests[0][:params][:request_payer]
+    end
+
+    it "handles :options as a block" do
+      @endpoint = Uppy::S3Multipart::App.new(bucket: @bucket, options: {
+        complete_multipart_upload: -> (request) {
+          assert_kind_of Rack::Request, request
+          { request_payer: "bob" }
+        }
+      })
+
+      response = app.post "/multipart/foo/complete", query: { key: "bar" }, json: { parts: [] }
+
+      assert_equal :complete_multipart_upload, @s3.api_requests[0][:operation_name]
+      assert_equal "bob",                      @s3.api_requests[0][:params][:request_payer]
+    end
+  end
+
+  describe "DELETE /multipart/:uploadId" do
+    it "aborts the multipart uplaod" do
+      response = app.delete "/multipart/foo", query: { key: "bar" }
+
+      assert_equal :abort_multipart_upload, @s3.api_requests[0][:operation_name]
+      assert_equal "foo",                   @s3.api_requests[0][:params][:upload_id]
+      assert_equal "bar",                   @s3.api_requests[0][:params][:key]
+    end
+
+    it "returns empty result" do
+      response = app.delete "/multipart/foo", query: { key: "bar" }
+
+      assert_equal 200,                response.status
+      assert_equal "application/json", response.headers["Content-Type"]
+
+      assert_equal Hash.new, response.body_json
+    end
+
+    it "returns error response when 'key' parameter is missing" do
+      response = app.delete "/multipart/foo"
+
+      assert_equal 400,                response.status
+      assert_equal "application/json", response.headers["Content-Type"]
+
+      assert_equal "Missing \"key\" parameter", response.body_json["error"]
+    end
+
+    it "handles :options as a hash" do
+      @endpoint = Uppy::S3Multipart::App.new(bucket: @bucket, options: {
+        abort_multipart_upload: { request_payer: "bob" }
+      })
+
+      response = app.delete "/multipart/foo", query: { key: "bar" }
+
+      assert_equal :abort_multipart_upload, @s3.api_requests[0][:operation_name]
+      assert_equal "bob",                   @s3.api_requests[0][:params][:request_payer]
+    end
+
+    it "handles :options as a block" do
+      @endpoint = Uppy::S3Multipart::App.new(bucket: @bucket, options: {
+        abort_multipart_upload: -> (request) {
+          assert_kind_of Rack::Request, request
+          { request_payer: "bob" }
+        }
+      })
+
+      response = app.delete "/multipart/foo", query: { key: "bar" }
+
+      assert_equal :abort_multipart_upload, @s3.api_requests[0][:operation_name]
+      assert_equal "bob",                   @s3.api_requests[0][:params][:request_payer]
+    end
+  end
+end

test/client_test.rb

+require "test_helper"
+require "aws-sdk-s3"
+require "uri"
+
+describe Uppy::S3Multipart::Client do
+  before do
+    resource = Aws::S3::Resource.new(stub_responses: true)
+    bucket   = resource.bucket("my-bucket")
+
+    @client = Uppy::S3Multipart::Client.new(bucket: bucket)
+    @s3     = bucket.client
+  end
+
+  describe "#create_multipart_upload" do
+    it "creates a multipart upload" do
+      @client.create_multipart_upload(key: "foo")
+
+      assert_equal :create_multipart_upload, @s3.api_requests[0][:operation_name]
+      assert_equal "foo",                    @s3.api_requests[0][:params][:key]
+      assert_equal "my-bucket",              @s3.api_requests[0][:params][:bucket]
+    end
+
+    it "returns multipart upload id and key" do
+      @s3.stub_responses(:create_multipart_upload, { upload_id: "bar" })
+
+      result = @client.create_multipart_upload(key: "foo")
+
+      assert_equal "bar", result[:upload_id]
+      assert_equal "foo", result[:key]
+    end
+
+    it "forwards additional option to the aws-sdk call" do
+      @client.create_multipart_upload(key: "foo", content_type: "text/plain")
+
+      assert_equal :create_multipart_upload, @s3.api_requests[0][:operation_name]
+      assert_equal "text/plain",             @s3.api_requests[0][:params][:content_type]
+    end
+  end
+
+  describe "#list_parts" do
+    it "fetches the list of parts" do
+      @client.list_parts(upload_id: "bar", key: "foo")
+
+      assert_equal :list_parts, @s3.api_requests[0][:operation_name]
+      assert_equal "foo",       @s3.api_requests[0][:params][:key]
+      assert_equal "bar",       @s3.api_requests[0][:params][:upload_id]
+      assert_equal "my-bucket", @s3.api_requests[0][:params][:bucket]
+    end
+
+    it "returns part numbers, sizes, and etags" do
+      @s3.stub_responses(:list_parts, parts: [ { part_number: 1, size: 123, etag: "etag1" } ])
+
+      result = @client.list_parts(upload_id: "bar", key: "foo")
+
+      assert_equal 1,       result[0][:part_number]
+      assert_equal 123,     result[0][:size]
+      assert_equal "etag1", result[0][:etag]
+    end
+
+    it "forwards additional options to the aws-sdk call" do
+      @client.list_parts(upload_id: "bar", key: "foo", max_parts: 5)
+
+      assert_equal :list_parts, @s3.api_requests[0][:operation_name]
+      assert_equal 5,           @s3.api_requests[0][:params][:max_parts]
+    end
+  end
+
+  describe "#prepare_upload_part" do
+    it "returns a presigned url for uploading a multipart part" do
+      result = @client.prepare_upload_part(upload_id: "bar", key: "foo", part_number: 1)
+
+      uri = URI.parse(result.fetch(:url))
+
+      assert_match %r{^my-bucket\.}, uri.host
+      assert_match %r{^/foo$},       uri.path
+      assert_match %r{uploadId=bar}, uri.query
+      assert_match %r{partNumber=1}, uri.query
+    end
+
+    it "forwards additional options to the aws-sdk call" do
+      result = @client.prepare_upload_part(upload_id: "bar", key: "foo", part_number: 1, content_md5: "foobar")
+
+      uri = URI.parse(result.fetch(:url))
+
+      assert_match %r{X-Amz-SignedHeaders=content-md5}, uri.query
+    end
+  end
+
+  describe "#complete_multipart_upload" do
+    it "completes the multipart upload" do
+      @client.complete_multipart_upload(upload_id: "bar", key: "foo", parts: [
+        { part_number: 1, etag: "etag1" }
+      ])
+
+      assert_equal :complete_multipart_upload, @s3.api_requests[0][:operation_name]
+      assert_equal "foo",                      @s3.api_requests[0][:params][:key]
+      assert_equal "bar",                      @s3.api_requests[0][:params][:upload_id]
+      assert_equal "my-bucket",                @s3.api_requests[0][:params][:bucket]
+      assert_equal 1,                          @s3.api_requests[0][:params][:multipart_upload][:parts][0][:part_number]
+      assert_equal "etag1",                    @s3.api_requests[0][:params][:multipart_upload][:parts][0][:etag]
+    end
+
+    it "returns presigned URL to the S3 object" do
+      result = @client.complete_multipart_upload(upload_id: "bar", key: "foo", parts: [])
+
+      uri = URI.parse(result.fetch(:location))
+
+      assert_match %r{^my-bucket\.},  uri.host
+      assert_match %r{^/foo$},        uri.path
+      assert_match %r{X-Amz-Expires}, uri.query
+    end
+
+    it "forwards additional options to the aws-sdk call" do
+      result = @client.complete_multipart_upload(upload_id: "bar", key: "foo", parts: [], request_payer: "bob")
+
+      assert_equal :complete_multipart_upload, @s3.api_requests[0][:operation_name]
+      assert_equal "bob",                      @s3.api_requests[0][:params][:request_payer]
+    end
+  end
+
+  describe "#abort_multipart_upload" do
+    it "aborts the multipart upload" do
+      @client.abort_multipart_upload(upload_id: "bar", key: "foo")
+
+      assert_equal :abort_multipart_upload, @s3.api_requests[0][:operation_name]
+      assert_equal "foo",                   @s3.api_requests[0][:params][:key]
+      assert_equal "bar",                   @s3.api_requests[0][:params][:upload_id]
+      assert_equal "my-bucket",             @s3.api_requests[0][:params][:bucket]
+    end
+
+    it "retries the abort if it failed" do
+      @s3.stub_responses(:list_parts, [
+        { parts: [{ part_number: 1, etag: "etag" }] }, # first call
+        { parts: [] },                                 # second call
+      ])
+
+      @client.abort_multipart_upload(upload_id: "bar", key: "foo")
+
+      assert_equal :abort_multipart_upload, @s3.api_requests[0][:operation_name]
+      assert_equal "foo",                   @s3.api_requests[0][:params][:key]
+      assert_equal "bar",                   @s3.api_requests[0][:params][:upload_id]
+      assert_equal "my-bucket",             @s3.api_requests[0][:params][:bucket]
+
+      assert_equal :list_parts,             @s3.api_requests[1][:operation_name]
+      assert_equal "foo",                   @s3.api_requests[1][:params][:key]
+      assert_equal "bar",                   @s3.api_requests[1][:params][:upload_id]
+      assert_equal "my-bucket",             @s3.api_requests[1][:params][:bucket]
+
+      assert_equal :abort_multipart_upload, @s3.api_requests[2][:operation_name]
+      assert_equal "foo",                   @s3.api_requests[2][:params][:key]
+      assert_equal "bar",                   @s3.api_requests[2][:params][:upload_id]
+      assert_equal "my-bucket",             @s3.api_requests[2][:params][:bucket]
+
+      assert_equal :list_parts,             @s3.api_requests[3][:operation_name]
+      assert_equal "foo",                   @s3.api_requests[3][:params][:key]
+      assert_equal "bar",                   @s3.api_requests[3][:params][:upload_id]
+      assert_equal "my-bucket",             @s3.api_requests[3][:params][:bucket]
+    end
+
+    it "returns empty result" do
+      result = @client.abort_multipart_upload(upload_id: "bar", key: "foo")
+
+      assert_equal Hash.new, result
+    end
+
+    it "forwards additional options to the aws-sdk call" do
+      @client.abort_multipart_upload(upload_id: "bar", key: "foo", request_payer: "bob")
+
+      assert_equal :abort_multipart_upload, @s3.api_requests[0][:operation_name]
+      assert_equal "bob",                   @s3.api_requests[0][:params][:request_payer]
+    end
+  end
+end

test/shrine_test.rb

+require "test_helper"
+require "shrine"
+require "shrine/plugins/uppy_s3_multipart"
+require "shrine/storage/s3"
+require "rack/test_app"
+
+describe Shrine::Plugins::UppyS3Multipart do
+  def s3(**options)
+    Shrine::Storage::S3.new(
+      bucket: "my-bucket",
+      stub_responses: true,
+      **options
+    )
+  end
+
+  def test_app(**options)
+    app = @shrine.uppy_s3_multipart(:s3, **options)
+    Rack::TestApp.wrap(Rack::Lint.new(app))
+  end
+
+  def client
+    @shrine.storages[:s3].client
+  end
+
+  before do
+    @shrine = Class.new(Shrine)
+    @shrine.storages[:s3] = s3
+    @shrine.plugin :uppy_s3_multipart
+  end
+
+  it "returns the app" do
+    assert_instance_of Uppy::S3Multipart::App, @shrine.uppy_s3_multipart(:s3)
+  end
+
+  it "passes the bucket" do
+    app = test_app
+    app.post "/multipart"
+
+    assert_equal :create_multipart_upload, client.api_requests[0][:operation_name]
+    assert_equal "my-bucket",              client.api_requests[0][:params][:bucket]
+  end
+
+  it "passes the prefix" do
+    @shrine.storages[:s3] = s3(prefix: "prefix")
+
+    app = test_app
+    app.post "/multipart"
+
+    assert_equal :create_multipart_upload, client.api_requests[0][:operation_name]
+    assert_match /^prefix\/\w+$/,          client.api_requests[0][:params][:key]
+  end
+
+  it "passes client options" do
+    @shrine.plugin :uppy_s3_multipart, options: {
+      create_multipart_upload: { acl: "public-read" },
+    }
+
+    app = test_app
+    app.post "/multipart"
+
+    assert_equal :create_multipart_upload, client.api_requests[0][:operation_name]
+    assert_equal "public-read",            client.api_requests[0][:params][:acl]
+  end
+
+  it "allows overriding app options" do
+    app = test_app(options: { create_multipart_upload: { acl: "public-read" } })
+    app.post "/multipart"
+
+    assert_equal :create_multipart_upload, client.api_requests[0][:operation_name]
+    assert_equal "public-read",            client.api_requests[0][:params][:acl]
+  end
+end

test/test_helper.rb

+require "bundler/setup"
+
+ENV["MT_NO_EXPECTATIONS"] = "1"
+
+require "minitest/autorun"
+require "minitest/pride"
+
+require "uppy/s3_multipart"

uppy-s3_multipart.gemspec

+Gem::Specification.new do |gem|
+  gem.name          = "uppy-s3_multipart"
+  gem.version       = "0.1.0"
+
+  gem.required_ruby_version = ">= 2.2"
+
+  gem.summary      = "Provides a Rack application that implements endpoints for the AwsS3Multipart Uppy plugin."
+  gem.homepage     = "https://github.com/janko-m/uppy-s3_multipart"
+  gem.authors      = ["Janko Marohnić"]
+  gem.email        = ["janko.marohnic@gmail.com"]
+  gem.license      = "MIT"
+
+  gem.files        = Dir["README.md", "LICENSE.txt", "lib/**/*.rb", "*.gemspec"]
+  gem.require_path = "lib"
+
+  gem.add_dependency "roda", ">= 2.27", "< 4"
+  gem.add_dependency "aws-sdk-s3", "~> 1.0"
+
+  gem.add_development_dependency "rake"
+  gem.add_development_dependency "minitest"
+  gem.add_development_dependency "rack-test_app"
+  gem.add_development_dependency "shrine", "~> 2.0"
+  gem.add_development_dependency "aws-sdk-core", "~> 3.23"
+end