Authentication¶

IOLake supports AWS Signature Version 4 (SigV4) authentication for S3 API compatibility.

Overview¶

┌─────────────────────────────────────────────────────────────────┐
│                        S3 Client                                 │
│  (aws-cli, boto3, aws-sdk-rust, etc.)                           │
└─────────────────────────┬───────────────────────────────────────┘
                          │ Authorization: AWS4-HMAC-SHA256 ...
                          ▼
┌─────────────────────────────────────────────────────────────────┐
│                    IOLake Gateway                              │
│  ┌───────────────────────────────────────────────────────────┐  │
│  │                  Auth Middleware                           │  │
│  │  1. Parse Authorization header                             │  │
│  │  2. Lookup credentials from Metadata Service (with cache)  │  │
│  │  3. Compute canonical request                              │  │
│  │  4. Verify signature                                       │  │
│  │  5. Evaluate bucket policy                                 │  │
│  └───────────────────────────────────────────────────────────┘  │
└─────────────────────────────────────────────────────────────────┘

Credential Storage¶

Credentials are centrally managed by the Metadata Service, not the gateway. This ensures:

Persistence: Credentials survive gateway restarts
Consistency: All gateways see the same credentials
Centralized management: Single source of truth for IAM

┌─────────────┐      gRPC       ┌─────────────────┐
│   Gateway   │ ◄────────────► │  Metadata Svc   │
│             │ GetAccessKey   │                 │
│  (caches    │ ForAuth        │  - Users        │
│   5 min)    │                │  - Access Keys  │
└─────────────┘                └─────────────────┘

Initial Admin User¶

On first startup, the metadata service creates a default admin user and writes its credentials to admin-creds.env in its data directory, readable by its owner only. Meta logs the access key and where the file is, never the secret:

# A host install
sudo cat /var/lib/objectio/meta/admin-creds.env
# Kubernetes
# (<name>: the release name if it contains "iolake", else <release>-iolake)
kubectl exec -n <namespace> <name>-meta-0 -- cat /var/lib/objectio/meta/admin-creds.env
# Docker
docker exec iolake-meta cat /var/lib/objectio/meta/admin-creds.env

The file holds AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY as shell exports. Create named admins of your own, then delete it.

AWS Signature Version 4¶

IOLake implements AWS SigV4, the standard authentication method for AWS services.

Authorization Header Format¶

Authorization: AWS4-HMAC-SHA256
Credential={access_key}/{date}/{region}/s3/aws4_request,
SignedHeaders={signed_headers},
Signature={signature}

Example:

Authorization: AWS4-HMAC-SHA256
Credential=AKIAIOSFODNN7EXAMPLE/20240115/us-east-1/s3/aws4_request,
SignedHeaders=host;x-amz-content-sha256;x-amz-date,
Signature=fe5f80f77d5fa3beca038a248ff027d0445342fe2855ddc963176630326f1024

Required Headers¶

Header	Description
`Host`	Bucket endpoint
`x-amz-date`	Request timestamp (ISO 8601)
`x-amz-content-sha256`	SHA-256 hash of request body; on the S3 API only (see below)

Presigned URLs (query-string SigV4) are accepted too; SigV2, in a header or a presigned URL, is refused with 400.

The signed body¶

The signature covers the body through x-amz-content-sha256, and the gateway holds the body to it. What each value means:

`x-amz-content-sha256`	What is checked
A SHA-256, hex	The body's hash, as it arrives. A body that doesn't match is refused with `400 XAmzContentSHA256Mismatch`, and nothing is stored
`UNSIGNED-PAYLOAD`	Nothing: the client signed no body (presigned URLs send this, and SDKs over HTTPS often do)
`STREAMING-AWS4-HMAC-SHA256-PAYLOAD`	Each chunk's signature, chained from the request's. A chunk whose signature doesn't match is refused with `403 SignatureDoesNotMatch`, and nothing is stored
`STREAMING-AWS4-HMAC-SHA256-PAYLOAD-TRAILER`	As above, and the signature over the trailing checksum
`STREAMING-UNSIGNED-PAYLOAD-TRAILER`	The trailing checksum (`400 BadDigest` when it doesn't match)

On the S3 API, a request without x-amz-content-sha256 signed no body, as if it sent UNSIGNED-PAYLOAD; every S3 SDK sends the header.

The other APIs on the gateway (the admin API, the Iceberg REST catalog, Unity Catalog) also take a request signed as standard SigV4 is for any service but S3: over the SHA-256 of its body, with no x-amz-content-sha256 header. That is how botocore's generic SigV4Auth signs, and so PyIceberg with rest.sigv4-enabled, and most SigV4 libraries that aren't S3 SDKs. The gateway reads the body and checks the signature over its hash, so a body changed after signing is refused (401 on the catalogs, 403 SignatureDoesNotMatch on the admin API). It reads at most 2 MiB that way: a larger body is refused with 413, and is signed by sending its hash in the header. A request signed over UNSIGNED-PAYLOAD without the header is accepted there as on the S3 API, and its body is not read. The IAM and STS query API (POST /) checks its form body's hash as well.

A trailer the request names in x-amz-trailer must come at the end of the body: one left off is refused with 400 MalformedTrailerError. Each streaming form is what some SDK sends by default for uploads (aws-cli, boto3, the Go and Java SDKs, minio-go); none needs setting up.

A refused signature is logged at debug only, with the canonical request the gateway computed, to tell a proxy that rewrites requests from a client that signs them wrong. Credentials, session tokens, cookies and SSE-C keys are left out of that log.

User Management¶

Using the CLI¶

iolakectl is a client of the admin API, signed with SigV4 like any request — see CLI for configuring it.

iolakectl user list
iolakectl user create alice --email alice@example.com
iolakectl user suspend <user_id>          # every key refused; nothing deleted
iolakectl user delete <user_id>

iolakectl key list <user_id>
iolakectl key create <user_id>                              # unscoped
iolakectl key create <user_id> --scope s3://logs/ --read-only
iolakectl key deactivate <access_key_id>
iolakectl key delete <access_key_id>

Admin API¶

The Admin API provides HTTP endpoints for user and access key management. It uses the same SigV4 authentication as the S3 API. The system admin can use all of it; a tenant admin can use it for its own tenant.

The full identity surface — suspending users, deactivating keys, IAM policies, groups, roles, per-tenant identity providers and STS — is in Identity and Access.

Admin API Endpoints¶

Endpoint	Method	Description
`/_admin/users`	GET	List all users
`/_admin/users`	POST	Create a new user
`/_admin/users/{user_id}`	DELETE	Delete a user
`/_admin/users/{user_id}/access-keys`	GET	List user's access keys
`/_admin/users/{user_id}/access-keys`	POST	Create access key for user
`/_admin/access-keys/{access_key_id}`	DELETE	Delete an access key

Authentication¶

The Admin API requires SigV4 authentication with admin credentials. Non-admin users receive a 403 Forbidden response.

# Configure admin credentials
export AWS_ACCESS_KEY_ID=AKIA...      # From meta service startup logs
export AWS_SECRET_ACCESS_KEY=...

List Users¶

curl -X GET https://s3.example.com/_admin/users \
  --aws-sigv4 "aws:amz:us-east-1:s3" \
  --user "$AWS_ACCESS_KEY_ID:$AWS_SECRET_ACCESS_KEY"

Response:

{
  "users": [
    {
      "user_id": "550e8400-e29b-41d4-a716-446655440000",
      "display_name": "admin",
      "arn": "arn:objectio:iam::user/admin",
      "status": "UserActive",
      "created_at": 1705320000,
      "email": ""
    }
  ],
  "is_truncated": false,
  "next_marker": null
}

Create User¶

curl -X POST https://s3.example.com/_admin/users \
  -H "Content-Type: application/json" \
  -d '{"display_name": "alice", "email": "alice@example.com"}' \
  --aws-sigv4 "aws:amz:us-east-1:s3" \
  --user "$AWS_ACCESS_KEY_ID:$AWS_SECRET_ACCESS_KEY"

Response:

{
  "user_id": "660e8400-e29b-41d4-a716-446655440001",
  "display_name": "alice",
  "arn": "arn:objectio:iam::user/alice",
  "status": "UserActive",
  "created_at": 1705320100,
  "email": "alice@example.com"
}

Create Access Key¶

curl -X POST https://s3.example.com/_admin/users/{user_id}/access-keys \
  --aws-sigv4 "aws:amz:us-east-1:s3" \
  --user "$AWS_ACCESS_KEY_ID:$AWS_SECRET_ACCESS_KEY"

Response:

{
  "access_key_id": "AKIAIOSFODNN7EXAMPLE",
  "secret_access_key": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
  "user_id": "660e8400-e29b-41d4-a716-446655440001",
  "status": "KeyActive",
  "created_at": 1705320200
}

Important: The secret_access_key is only returned when creating a key. Save it immediately!

Delete User¶

curl -X DELETE https://s3.example.com/_admin/users/{user_id} \
  --aws-sigv4 "aws:amz:us-east-1:s3" \
  --user "$AWS_ACCESS_KEY_ID:$AWS_SECRET_ACCESS_KEY"

Returns 204 No Content on success.

Delete Access Key¶

curl -X DELETE https://s3.example.com/_admin/access-keys/{access_key_id} \
  --aws-sigv4 "aws:amz:us-east-1:s3" \
  --user "$AWS_ACCESS_KEY_ID:$AWS_SECRET_ACCESS_KEY"

Returns 204 No Content on success.

Error Responses¶

Status	Description
`401 Unauthorized`	Authentication required (no-auth mode)
`403 Forbidden`	Non-admin user attempted access
`404 Not Found`	User or access key not found
`500 Internal Server Error`	Server error

Bucket Policies¶

Bucket policies control access to buckets and objects using IAM-style JSON policies.

Policy Structure¶

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "PublicRead",
            "Effect": "Allow",
            "Principal": "*",
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::my-bucket/*"
        }
    ]
}

Policy Elements¶

Element	Description
`Version`	Policy version (always "2012-10-17")
`Statement`	Array of permission statements
`Sid`	Statement ID (optional)
`Effect`	"Allow" or "Deny"
`Principal`	Who the statement applies to
`Action`	S3 actions (e.g., "s3:GetObject")
`Resource`	ARN of bucket/objects
`Condition`	Optional conditions

Supported Actions¶

S3 Object Actions: - s3:GetObject - s3:PutObject - s3:DeleteObject - s3:GetObjectVersion

S3 Bucket Actions: - s3:ListBucket - s3:GetBucketLocation - s3:GetBucketPolicy - s3:PutBucketPolicy - s3:DeleteBucketPolicy

Iceberg Namespace Actions: - iceberg:ListNamespaces - iceberg:CreateNamespace - iceberg:LoadNamespace - iceberg:DropNamespace - iceberg:UpdateNamespaceProperties

Iceberg Table Actions: - iceberg:ListTables - iceberg:CreateTable - iceberg:LoadTable - iceberg:UpdateTable - iceberg:DropTable - iceberg:RenameTable

Example Policies¶

Public Read Access:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": "*",
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::public-bucket/*"
        }
    ]
}

Cross-Account Access:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::123456789012:user/external-user"
            },
            "Action": [
                "s3:GetObject",
                "s3:PutObject"
            ],
            "Resource": "arn:aws:s3:::shared-bucket/*"
        }
    ]
}

IP-Based Restriction:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Deny",
            "Principal": "*",
            "Action": "s3:*",
            "Resource": [
                "arn:aws:s3:::secure-bucket",
                "arn:aws:s3:::secure-bucket/*"
            ],
            "Condition": {
                "NotIpAddress": {
                    "aws:SourceIp": "192.168.1.0/24"
                }
            }
        }
    ]
}

Setting Bucket Policy¶

# Using AWS CLI
aws s3api put-bucket-policy \
  --endpoint-url http://localhost:9000 \
  --bucket my-bucket \
  --policy file://policy.json

# Get current policy
aws s3api get-bucket-policy \
  --endpoint-url http://localhost:9000 \
  --bucket my-bucket

# Delete policy
aws s3api delete-bucket-policy \
  --endpoint-url http://localhost:9000 \
  --bucket my-bucket

Condition Keys¶

Supported condition keys for policy evaluation:

Key	Description
`aws:SourceIp`	The client's address: the connection's peer, or the forwarded client when the peer is a trusted proxy
`aws:SourceVpce`	The named gateway endpoint the request arrived on; absent on an unnamed one
`aws:SecureTransport`	Whether HTTPS was used (as reported by a TLS-terminating proxy)
`aws:CurrentTime`, `aws:EpochTime`	Request time
`aws:username`	The caller's ARN
`s3:prefix`	A listing's prefix
`s3:ExistingObjectTag/<key>`	A tag on the object being accessed
`s3:x-amz-server-side-encryption`, `s3:x-amz-server-side-encryption-aws-kms-key-id`	The encryption a write asks for
`obio:CredentialType`	`Permanent`, `Sts` (vended), `AssumedRole` or `Anonymous`
`<issuer>:<claim>`	In a role's trust policy, the OIDC token's claims (see STS)

Every IAM condition operator is supported, with IfExists, ForAnyValue:/ForAllValues: and Null. A policy using an unknown operator is refused when it's saved.

Anonymous Access¶

An unsigned request is anonymous. It reaches only what a bucket policy grants to everyone, and new buckets block that until their owner lifts the block. See Public Access.

--no-auth turns authentication off entirely, so every request is allowed. Use it only for local testing.

Client Configuration¶

AWS CLI¶

# Configure credentials
aws configure
# Enter: Access Key ID, Secret Access Key, Region, Output format

# Or use environment variables
export AWS_ACCESS_KEY_ID=AKIA...
export AWS_SECRET_ACCESS_KEY=wJalr...
export AWS_DEFAULT_REGION=us-east-1

# Use with IOLake
aws s3 --endpoint-url http://localhost:9000 ls

Boto3 (Python)¶

import boto3

s3 = boto3.client(
    's3',
    endpoint_url='http://localhost:9000',
    aws_access_key_id='AKIA...',
    aws_secret_access_key='wJalr...',
    region_name='us-east-1'
)

# List buckets
response = s3.list_buckets()

AWS SDK for Rust¶

use aws_config::BehaviorVersion;
use aws_sdk_s3::Client;

let config = aws_config::defaults(BehaviorVersion::latest())
    .endpoint_url("http://localhost:9000")
    .load()
    .await;

let client = Client::new(&config);

let buckets = client.list_buckets().send().await?;

Security Best Practices¶

Use HTTPS in Production
Configure TLS certificates
Redirect HTTP to HTTPS
Rotate Access Keys
Create new keys periodically
Delete old keys after transition
Use Least Privilege
Grant minimum required permissions
Use specific resource ARNs
Enable Logging
Track authentication failures
Monitor for suspicious activity
Secure Credential Storage
Never hardcode credentials
Use environment variables or secrets managers