Iceberg REST Catalog API¶

IOLake includes an Apache Iceberg REST Catalog API, enabling query engines like Spark, Trino, and Flink to discover and manage Iceberg tables.

Base URL¶

All Iceberg endpoints are served under /iceberg/v1/. Clients are given the catalog URI without /v1 and add it themselves:

http://gateway:9000/iceberg

Every catalog lives in a warehouse, named by the client's warehouse setting.

Authentication¶

Every request carries a credential, as on S3: a SigV4 signature with an access key (the same keys as S3), an OIDC bearer token from the configured identity provider (--oidc-issuer-url), or a console session. A request with none, or with one that doesn't check out (a bad signature, an unknown key, a token the provider doesn't accept), is refused with 401. A key scoped to a bucket or prefix is refused with 403: it is for that data, not for the catalog. Only under --no-auth are the endpoints open, and then nothing is checked.

Warehouses¶

Every request names a warehouse: ?warehouse=<name> on /v1/config, and from then on the ws/<name> prefix the configuration returns (/iceberg/v1/ws/<name>/namespaces/...). A warehouse belongs to a tenant; a caller sees only its own tenant's warehouses (a catalog admin sees all), and one of another tenant is answered as unknown.

Warehouses are managed on the admin API, not here: POST /_admin/warehouses ({"name": "..."}, created in the caller's tenant), GET /_admin/warehouses and DELETE /_admin/warehouses/{name}, by the system admin or an admin of the warehouse's tenant. Each has a backing bucket iceberg-<name>; a warehouse is deleted only once that bucket is empty (409 otherwise), since its bucket goes with it.

Warehouses¶

A warehouse is a catalog with its own namespaces and tables, and a bucket, iceberg-<name>, made with it for its data. Warehouses are managed on the admin API (iolakectl warehouse …):

Call	Who
`GET /_admin/warehouses`	system admin (every tenant's), or a tenant's admin (its tenant's)	`{"warehouses": [{"name", "bucket", "location", "tenant", "created_at", "properties"}]}`
`POST /_admin/warehouses`	system admin, or the tenant's admin	Body `{"name": "...", "tenant": "...", "properties": {...}}`; `tenant` defaults to the caller's. Answers the new warehouse; `400` if it or its bucket exists
`DELETE /_admin/warehouses/{name}`	system admin, or its tenant's admin

iolakectl warehouse create analytics      # also creates bucket iceberg-analytics
iolakectl warehouse list

Configuration¶

GetConfig¶

Get catalog configuration for a warehouse (the warehouse parameter is required).

GET /iceberg/v1/config?warehouse=analytics

Response:

{
  "defaults": {
    "warehouse": "s3://iceberg-analytics"
  },
  "overrides": {
    "prefix": "ws/analytics"
  }
}

The client then sends every call under /iceberg/v1/ws/analytics/….

Namespace Operations¶

ListNamespaces¶

List all namespaces, optionally filtered by parent.

GET /iceberg/v1/namespaces
GET /iceberg/v1/namespaces?parent=parent_ns

Response:

{
  "namespaces": [
    ["analytics"],
    ["production"]
  ]
}

CreateNamespace¶

Create a new namespace.

POST /iceberg/v1/namespaces

Request Body:

{
  "namespace": ["analytics"],
  "properties": {
    "owner": "alice",
    "description": "Analytics data"
  }
}

Response:

{
  "namespace": ["analytics"],
  "properties": {
    "owner": "alice",
    "description": "Analytics data"
  }
}

LoadNamespace¶

Get namespace details.

GET /iceberg/v1/namespaces/{namespace}

Response:

{
  "namespace": ["analytics"],
  "properties": {
    "owner": "alice"
  }
}

NamespaceExists¶

Check if a namespace exists.

HEAD /iceberg/v1/namespaces/{namespace}

Response: 204 No Content (exists) or 404 Not Found

DropNamespace¶

Delete a namespace. Must be empty (no tables).

DELETE /iceberg/v1/namespaces/{namespace}

Response: 204 No Content

UpdateNamespaceProperties¶

Update namespace properties.

POST /iceberg/v1/namespaces/{namespace}/properties

Request Body:

{
  "updates": {
    "description": "Updated description"
  },
  "removals": ["old_property"]
}

Response:

{
  "updated": ["description"],
  "removed": ["old_property"],
  "missing": []
}

Table Operations¶

ListTables¶

List tables in a namespace.

GET /iceberg/v1/namespaces/{namespace}/tables

Response:

{
  "identifiers": [
    {
      "namespace": ["analytics"],
      "name": "events"
    }
  ]
}

CreateTable¶

Create a new table.

POST /iceberg/v1/namespaces/{namespace}/tables

Request Body:

{
  "name": "events",
  "schema": {
    "type": "struct",
    "fields": [
      {"id": 1, "name": "id", "type": "long", "required": true},
      {"id": 2, "name": "data", "type": "string", "required": false}
    ]
  },
  "properties": {}
}

Response:

{
  "metadata": {
    "format-version": 2,
    "table-uuid": "...",
    "location": "s3://iceberg-analytics/analytics/events",
    "schema": { ... }
  },
  "metadata-location": "s3://iceberg-analytics/analytics/events/metadata/00000-....metadata.json"
}

A table lives inside its warehouse: a location in the request, or one set later by a commit (set-location), must be the warehouse's location or a path under it, or the request is refused with 400. The table's vended credentials are scoped to its location.

LoadTable¶

Get table metadata.

GET /iceberg/v1/namespaces/{namespace}/tables/{table}

Response: Same format as CreateTable response, with vended credentials for the table's data in config (s3.access-key-id, s3.secret-access-key, s3.session-token, s3.endpoint). They are scoped to the table's location, last an hour, and are read-write only for a caller who may also commit to the table (iceberg:UpdateTable); anyone else gets read-only ones. They stay inside the warehouse's tenant: they reach nothing in another tenant's buckets.

A caller that iceberg:LoadTable isn't granted to gets 403 and no credentials.

TableExists¶

Check if a table exists.

HEAD /iceberg/v1/namespaces/{namespace}/tables/{table}

Response: 204 No Content (exists) or 404 Not Found

UpdateTable¶

Update table metadata (schema evolution, property changes).

POST /iceberg/v1/namespaces/{namespace}/tables/{table}

DropTable¶

Delete a table.

DELETE /iceberg/v1/namespaces/{namespace}/tables/{table}

Response: 204 No Content

RenameTable¶

Rename a table (optionally across namespaces).

POST /iceberg/v1/tables/rename

Request Body:

{
  "source": {
    "namespace": ["analytics"],
    "name": "events"
  },
  "destination": {
    "namespace": ["analytics"],
    "name": "events_v2"
  }
}

Response: 204 No Content

Access Control¶

Policies, tags, quotas, encryption requirements and role bindings are set by the warehouse's administrators: the system admin, the principals named by --oidc-admin-roles, and the admins of the warehouse's tenant. The namespace properties that hold them (names starting __) can be changed only by them, through these endpoints or through UpdateNamespaceProperties. Data filters are managed by the system admin and the --oidc-admin-roles principals only.

Namespace Policy¶

Set a policy on a namespace (admin only).

PUT /iceberg/v1/namespaces/{namespace}/policy

Request Body:

{
  "policy": "{\"Version\":\"2012-10-17\",\"Statement\":[{\"Effect\":\"Allow\",\"Principal\":{\"OBIO\":[\"arn:objectio:iam::user/alice\"]},\"Action\":[\"iceberg:*\"],\"Resource\":[\"arn:obio:iceberg:::analytics/*\"]}]}"
}

Table Policy¶

Set a policy on a table (admin only).

PUT /iceberg/v1/namespaces/{namespace}/tables/{table}/policy

Request Body: Same format as namespace policy.

Policy Evaluation¶

A request is decided by the warehouse's catalog policy (PUT /v1/catalog/policy), the policy of each level of the namespace and, for a table, the table's policy, the way S3 decides on a bucket policy:

Explicit Deny anywhere refuses the request (403).
Explicit Allow anywhere grants it.
With neither (no policy, or none naming the caller), only the warehouse's administrators go ahead, as only a bucket's owner does on S3. Anyone else is refused: a tenant's users need a policy that allows them.
A stored policy that can't be read refuses the request rather than being skipped, and so does a policy meta can't be asked for (503, retry).
The system admin is not subject to policies, so no policy can lock the operator out.

Iceberg Actions¶

Action	Description
`iceberg:ListNamespaces`	List namespaces
`iceberg:CreateNamespace`	Create a namespace
`iceberg:LoadNamespace`	Load namespace details
`iceberg:DropNamespace`	Delete a namespace
`iceberg:UpdateNamespaceProperties`	Update namespace properties
`iceberg:ListTables`	List tables in a namespace
`iceberg:CreateTable`	Create a table
`iceberg:LoadTable`	Load table metadata
`iceberg:UpdateTable`	Update table metadata
`iceberg:DropTable`	Delete a table
`iceberg:RenameTable`	Rename a table

Resource ARNs¶

Namespace: arn:obio:iceberg:::{namespace}
Table: arn:obio:iceberg:::{namespace}/{table}

Client Configuration¶

The URI is .../iceberg: Iceberg's REST clients add /v1 themselves, so .../iceberg/v1 makes them call /iceberg/v1/v1/…. The warehouse is a warehouse's name, not an s3:// location. The pyiceberg example in the code repo (examples/pyiceberg) signs with SigV4 (rest.sigv4-enabled, rest.signing-name s3) and runs against iolake-aio. PyIceberg signs with botocore's generic SigV4 signer, over the hash of each request's body without sending it in x-amz-content-sha256; the gateway hashes the body to check the signature, up to 2 MiB a request (the signed body); no query engine is tested against the catalog in CI yet (iolake#8), so treat the Spark and Trino settings below as a starting point.

PyIceberg¶

from pyiceberg.catalog.rest import RestCatalog

catalog = RestCatalog(
    name="iolake",
    uri="http://gateway:9000/iceberg",
    warehouse="analytics",
    **{
        "rest.sigv4-enabled": "true",
        "rest.signing-name": "s3",
        "rest.signing-region": "us-east-1",
        "s3.endpoint": "http://gateway:9000",
        "s3.path-style-access": "true",
        "s3.access-key-id": "...",
        "s3.secret-access-key": "...",
    },
)

Apache Spark¶

spark = SparkSession.builder \
    .config("spark.sql.catalog.iolake", "org.apache.iceberg.spark.SparkCatalog") \
    .config("spark.sql.catalog.iolake.type", "rest") \
    .config("spark.sql.catalog.iolake.uri", "http://gateway:9000/iceberg") \
    .config("spark.sql.catalog.iolake.warehouse", "analytics") \
    .getOrCreate()

# Use the catalog
spark.sql("CREATE NAMESPACE iolake.analytics")
spark.sql("CREATE TABLE iolake.analytics.events (id BIGINT, data STRING)")

Trino¶

connector.name=iceberg
iceberg.catalog.type=rest
iceberg.rest-catalog.uri=http://gateway:9000/iceberg
iceberg.rest-catalog.warehouse=analytics

Error Responses¶

Iceberg errors follow the Iceberg REST API error format:

{
  "error": {
    "message": "Namespace does not exist: analytics",
    "type": "NoSuchNamespaceException",
    "code": 404
  }
}

Error Types¶

Type	HTTP Status	Description
`NoSuchNamespaceException`	404	Namespace not found
`NamespaceAlreadyExistsException`	409	Namespace already exists
`NoSuchTableException`	404	Table not found
`AlreadyExistsException`	409	Table already exists
`ForbiddenException`	403	Access denied by policy
`BadRequestException`	400	Invalid request