howdy/storage/s3

Storage in an S3-compatible bucket: AWS S3, Cloudflare R2, Tigris, MinIO, Garage, SeaweedFS, Backblaze B2 and others.

let assert Ok(config) = s3.from_env()
let files = s3.adapter(config)

Or describe the bucket in code:

s3.new(
  bucket: "acme-uploads",
  region: "auto",
  access_key_id: key_id,
  secret_access_key: secret,
)
|> s3.endpoint("https://<account>.r2.cloudflarestorage.com")
|> s3.adapter

Without an endpoint the bucket is on AWS, addressed as <bucket>.s3.<region>.amazonaws.com. With one, the bucket goes in the path, which every S3-compatible service understands; path_style changes either default.

put_file sends a file of up to 16 MiB in one request and a larger one in 8 MiB parts, so no more than a part is in memory at a time. A failed upload in parts is aborted, so its parts are not left to be charged for. get reads the whole object into memory: send browsers to url instead.

put_cors sets which other origins’ pages may use the bucket from the browser, as direct uploads need.

Types

Config

opaque </>

pub opaque type Config

CorsRule

</>

Which pages may use the bucket from the browser, and how. A page on another origin needs a rule to upload with storage.upload_url, or to read a response with fetch; an <img> or a link needs none.

pub type CorsRule {
  CorsRule(
    origins: List(String),
    methods: List(http.Method),
    headers: List(String),
    expose_headers: List(String),
    max_age: option.Option(Int),
  )
}

Constructors

```
CorsRule(
  origins: List(String),
  methods: List(http.Method),
  headers: List(String),
  expose_headers: List(String),
  max_age: option.Option(Int),
)
```
Arguments

origins

Origins such as "https://app.example.com", with no path or trailing slash. One * may stand for any part, as in "https://*.example.com", or "*" for any origin.

methods

Get, Put, Head, Post or Delete.

headers

Request headers the page may send, such as "content-type".

expose_headers

Response headers the page’s script may read, such as "etag".

max_age

How long a browser may remember the answer to its preflight request, in seconds.

EnvError

</>

Why from_env could not build a config.

pub type EnvError {
  MissingVariable(String)
}

Constructors

```
MissingVariable(String)
```

Values

adapter

</>

pub fn adapter(config: Config) -> storage.Storage

cors_rule

</>

pub fn cors_rule(
  origins origins: List(String),
  methods methods: List(http.Method),
) -> CorsRule

A rule letting origins use these methods, sending a Content-Type, with the preflight remembered for an hour. A rule for direct uploads:

s3.cors_rule(origins: ["https://app.example.com"], methods: [http.Put])

Change the rest with a record update: s3.CorsRule(..rule, expose_headers: ["etag"]).

endpoint

</>

pub fn endpoint(config: Config, url: String) -> Config

The service’s URL, such as http://localhost:9000 for MinIO, for a service other than AWS.

from_env

</>

pub fn from_env() -> Result(Config, EnvError)

Read the bucket from the environment, as AWS’s own tools and hosts such as Fly.io’s Tigris set it:

AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY, and AWS_SESSION_TOKEN for temporary credentials;
S3_BUCKET, or BUCKET_NAME;
AWS_REGION or AWS_DEFAULT_REGION, else us-east-1;
AWS_ENDPOINT_URL_S3 or AWS_ENDPOINT_URL for a service other than AWS.

multipart

</>

pub fn multipart(
  config: Config,
  threshold threshold: Int,
  part_size part_size: Int,
) -> Config

Upload files larger than threshold bytes in parts of part_size bytes. S3 needs parts of at least 5 MiB and allows at most 10,000 of them. Default 16 MiB and 8 MiB.

new

</>

pub fn new(
  bucket bucket: String,
  region region: String,
  access_key_id access_key_id: String,
  secret_access_key secret_access_key: String,
) -> Config

path_style

</>

pub fn path_style(config: Config, path_style: Bool) -> Config

Whether the bucket goes in the path (True) or the host name (False). By default it goes in the host on AWS and in the path elsewhere. AWS bucket names with dots always go in the path, since TLS certificates do not cover them as host names.

public_url

</>

pub fn public_url(config: Config, base_url: String) -> Config

Where the objects are public, such as a CDN or a public bucket’s URL, without a trailing slash. storage.public_url gives this followed by the key.

put_cors

</>

pub fn put_cors(
  config: Config,
  rules: List(CorsRule),
) -> Result(Nil, storage.Error)

Replace the bucket’s CORS rules with these. The key needs permission to change the bucket’s settings, as the bucket’s owner has. A browser follows the first rule that matches its request.

Run it when the app starts or deploys, or once by hand: the rules stay on the bucket. Some services set CORS only in their console, and refuse this.

session_token

</>

pub fn session_token(
  config: Config,
  token: option.Option(String),
) -> Config

Temporary credentials’ session token.

timeout

</>

pub fn timeout(config: Config, milliseconds: Int) -> Config

How long to wait for each request. Default 60 seconds.

Constructors

Arguments

Constructors