Sergei Grebnov

Spice v2.4.0-rc.1 is now available! 🔥

Spice v2.4.0-rc.1 is the first release candidate for v2.4.0. It adds performance improvements, S3 event-driven ingestion, SQL results-cache warmup, and adaptive HTTP rate controls. The release also upgrades to DataFusion v55, Ballista v55, Arrow v59, Vortex v0.86, Iceberg v0.11, and Turso v0.81.

Highlights in v2.4.0-rc.1 include:

• Performance & Query Engine: DataFusion v55, Vortex v0.86.1, Iceberg v0.11.0, and Ballista v55 update query planning, storage, and distributed execution.
• S3 Event-Driven Ingestion — ingest newly created objects through SQS notifications.
• Cache Warmup and Shared Fetches — warm the SQL results cache after the first refresh and coalesce concurrent source fetches.
• Adaptive HTTP Rate Controls — bound request waits and coordinate request limits across instances.
• ORC Support — query ORC files through listing connectors.
• Hugging Face Datasets: Query and accelerate files from the Hugging Face Hub.

What's New in v2.4.0-rc.1​

Performance & Query Engine​

This release upgrades Apache DataFusion to the v55.2.0 dependency line and Apache Arrow to v59.3.0. It also upgrades Vortex to v0.86.1, Apache Iceberg to the v0.11.0 fork, and Apache Ballista to v55.

Apache DataFusion v55​

The DataFusion v55 release adds the following improvements:

• Sort pushdown and TopK pruning: Parquet scans reevaluate each unread row group as the threshold for ORDER BY ... LIMIT tightens. They skip groups that cannot contribute to the result. TopK pruning also supports multiple sort columns.
• Join planning: The optimizer converts eligible inner joins to semi joins and removes redundant sides of outer joins. It also orders filter predicates by estimated cost.
• Aggregation and expressions: Multi-column GROUP BY uses column-oriented storage for all supported key types, such as fixed-size binary UUIDs. More string functions preserve dictionary encoding, and IN lists use specialized paths for small integer types.
• Parquet reads: Scans skip nested fields that the declared schema does not contain. They also skip page-index reads when a file has no page index.
• Spill handling: Sorts bound the number of streams in a merge. If memory is insufficient, they spill the largest stream again in smaller batches.
• SQL diagnostics and functions: EXPLAIN accepts PostgreSQL-style options and FORMAT pgjson. New array functions cover element-wise addition, subtraction, scaling, sums, and averages.

Spice carries these changes through its query plans and preserves statistics across plan wrappers. Cayenne keeps Vortex scans below 10 MiB unsplit to avoid repeated footer reads. See #14612.

Vortex v0.79.0 to v0.86.1​

Spice v2.3.2 used the Vortex v0.79.0 fork. This upgrade covers the full upstream range from v0.79.0 through v0.86.1, not only the v0.86 changes:

• Types and arithmetic: Vortex adds native Map arrays, Arrow map conversion, and operations and compression for maps. It also adds union arrays and decimal addition, subtraction, multiplication, and division.
• Row selection: Piecewise-sequence indices represent contiguous selections without an expanded index for every row. Specialized paths handle chunked arrays, fixed-size lists, variable-length lists, and binary values.
• Scans and expressions: Layout scans gain a physical plan and expression optimization. Filters can pass through scalar functions with multiple arguments. Filters on wide lists restrict child elements to the selected range, and constant masks can resolve from metadata.
• Compression: Binary arrays support FSST compression with variable-length offsets. OnPair becomes a stable encoding for reads and gains a storage-backed dictionary. Vortex can convert run-end arrays of lists and decimals.
• File metadata: Files can store custom metadata. Readers cache decoded type descriptions, and file-format editions define the supported encodings and types.
• Memory and execution: Builders append nested values in batches, preserve list views, and propagate buffer allocators. Expression rewrites retain unchanged nodes, and row functions support batch execution.
• Correctness and validation: NULL handling changes cover dictionary predicates, BETWEEN bounds, and empty arrays. Readers add validation for footer offsets, compression metadata, and array indices. Other corrections cover nested scalar hashes, decimal operations, and variable-length binary selections above 4 GiB.

See the complete Vortex v0.79.0 to v0.86.1 changelog for every upstream change. Changes to standalone Vortex bindings and GPU execution do not imply new Spice features.

Apache Iceberg v0.11.0​

Spice updates its Iceberg reader and catalog integrations from v0.10.1 to the v0.11.0 fork for DataFusion 55 and Arrow 59. The upstream v0.11 changes extend reads and catalog compatibility:

• Iceberg v3 reads: The reader applies deletion vectors from Puffin files and carries row identifiers and sequence numbers through scans.
• Metadata-only scans: Metadata-only projections do not require data-column reads. Manifest reads reuse partition types, and positional-delete processing buffers runs instead of allocating a key per row.
• REST catalogs: Clients negotiate server-advertised endpoints and use session-scoped OAuth2 authentication.

The upgrade also aligns Iceberg storage with OpenDAL v0.58. See #14771.

Apache Ballista v55​

Spice.ai Enterprise feature. See the Enterprise documentation.

The Ballista v55 upgrade adds virtual-core resource accounting for distributed tasks. A protocol-version handshake detects incompatible schedulers and executors. Cancellation identifies tasks by their task IDs, and task-state records use an append-only model.

Schedulers and executors must run the same version. Upgrade all cluster components together.

S3 Event-Driven Ingestion​

S3 listing datasets can use refresh_mode: changes with S3 event notifications delivered through SQS:

datasets:
  - from: s3://my-bucket/events/
    name: events
    params:
      file_format: parquet
      s3_region: us-east-1
      s3_auth: iam_role
      s3_changes_queue_url: ${ secrets:events_queue_url }
    acceleration:
      enabled: true
      engine: cayenne
      mode: file
      refresh_mode: changes

New object notifications append the object's rows. A periodic listing backfill covers missed or expired notifications. Each dataset needs its own queue and permissions to read and delete SQS messages, alongside its S3 read/list permissions.

This is object ingestion: removal notifications are ignored by default. Set s3_on_object_removed: rebuild to rebuild the entire prefix when an object is removed. An overwrite of an already applied object key is not ingested again; use new object keys for incoming data. See #14121.

SQL Results-Cache Warmup and Shared Fetches​

The SQL results cache can persist query plan shapes and replay them after a dataset's first full or append refresh:

runtime:
  caching:
    sql_results:
      enabled: true
      warmup: on_first_refresh

For example, run these queries against an accelerated orders dataset with warmup enabled:

SELECT id, status FROM orders WHERE id = 1;
SELECT id, status FROM orders WHERE id = 2;

Spice records one query shape because only the equality-filter value differs. After a restart and the dataset's first full or append refresh, warmup reruns that shape with distinct id values from the refreshed dataset, filling the cache before the dataset becomes ready. Keep the local .spice/data directory across restarts, or configure runtime.state.location, to retain recorded query shapes.

Warmup replays up to ten distinct recorded query shapes and tries up to 1,024 distinct filter-value combinations per shape, stopping when the cache is full. A dataset stays not ready until warmup completes. Later refreshes do not repeat warmup. The feature requires the default plan-based cache key; cache_key_type: sql is incompatible. See #14178.

Concurrent cache-miss fetches for the same request now share a source fetch. SQL results caching also includes changes for tables updated during query execution and for stale results served while revalidation runs. HTTP dataset caching supports RFC 5861 stale-if-error handling. See #14142, #14710, #14708, and #14134.

SQL, search, and embedding caches now use Spice's sharded cache backend. Existing engine: moka and engine: pingora values are accepted for configuration compatibility but no longer select a backend.

Adaptive HTTP Rate Controls​

HTTP rate controls now adapt admission to upstream failures while staying within configured request limits. rate_control_acquire_timeout bounds how long a request waits for capacity and defaults to the connector's client timeout. rate_control_failure_threshold and rate_control_window control the response to upstream failures.

In Spice.ai Enterprise, instances that share a runtime.state.location coordinate per-second and per-minute limits through shared state; OSS instances keep these limits in memory independently. Concurrency limits remain local to each instance. Components sharing an upstream origin must use matching rate-control settings. See HTTP rate-control documentation and #14143.

ORC Files and Object Metadata Queries​

Listing connectors support file_format: orc. Object-store listing also uses predicates on metadata columns, including _last_modified, to narrow eligible objects. Queries that select only partition or metadata columns can use those values without reading file contents. See #14075, #14265, #14303, and #14116.

Hugging Face Datasets​

The new Hugging Face data connector queries and accelerates datasets from the Hugging Face Hub. It supports Parquet, CSV, TSV, JSON, and ORC files. Public datasets need no credentials:

datasets:
  - from: hf://datasets/stanfordnlp/imdb/plain_text/
    name: imdb
    acceleration:
      enabled: true

The location format is hf://datasets/<owner>/<dataset>[@<revision>][/<path>]. A path can select a file, a folder, or a glob. A revision can name a branch, a tag, or a commit. Use @~parquet to select the Hub's automatic Parquet conversion.

Set hf_token for private or gated datasets. Set hf_endpoint for a Hub mirror or proxy. Each scan reads one commit. Refreshes follow the selected branch, but the dataset keeps its registered schema until reload. See #14877.

Automatic Primary-Key Handling​

Cayenne keeps one row per primary_key without requiring an on_conflict policy. When a dataset sets time_column, the row with the newest time wins; without it, the last arrival wins. This applies to full and append refreshes as well as writes. Existing explicit conflict policies remain accepted during the deprecation period.

datasets:
  - from: s3://my-bucket/orders/
    name: orders
    time_column: updated_at
    params:
      file_format: parquet
    acceleration:
      enabled: true
      engine: cayenne
      mode: file
      primary_key: id

For a read-write dataset whose writes should stay in its acceleration, set acceleration.write_mode: acceleration. Its source need not support writes. This mode cannot be combined with a dataset that refreshes by changes.

Cayenne secondary indexes also support dynamic join filters, and their write handling covers inserts, updates, deletes, and refreshes. See #14726, #14282, and #14593.

PostgreSQL and MySQL replication, and MongoDB change streams, rejected Cayenne datasets that omitted on_conflict. Their validation still required an explicit upsert policy. These sources now accept Cayenne datasets with primary_key alone. See #14880.

For file-mode Cayenne datasets, append refreshes failed with a configuration that combined primary_key, time_column, and retention_sql. The refresh selected a version-resolution path that did not support retention. The refresh now resolves each key's newest version before Cayenne applies retention. See #14878.

Cayenne maintained aggregates now share one compact index of per-key contributions across views. Rebuilds capture concurrent writes and apply them after the scan. The aggregate budget uses 10% of a bounded query pool without the former 512 MiB cap. An unbounded pool retains the 512 MiB budget. See #14762.

Read Datasets from Published Snapshots​

Spice.ai Enterprise feature. See the Enterprise documentation.

A dataset can now read published acceleration snapshots directly, without configuring the original source connector:

datasets:
  - from: s3://my-bucket/spice/snapshots/orders/
    name: orders
    params:
      file_format: snapshot
      s3_region: us-east-1

Spice reads the snapshot metadata to select the engine, restores the published data, and checks for newer snapshots. The dataset is read-only. Snapshot-mode readers can also use S3 notifications delivered through SQS, with periodic checks retained for missed notifications. Each reader process needs its own queue.

This release includes changes to snapshot retries, slow-connection bootstrap, publication metadata, and coordination between snapshot archiving and Cayenne maintenance. Cayenne datasets with a datalake tier cannot create acceleration snapshots. See snapshot documentation, #14529, and #14335.

Connector and Protocol Updates​

• Connector status: ADBC, Databricks Spark Connect and SQL Warehouse, FlightSQL, Glue, HTTP/HTTPS, Iceberg, Localpod, and MongoDB are now Stable data connectors.
• MCP: support for specification version 2026-07-28, alongside the earlier protocol era. See #14043.
• GitHub: nested GraphQL pagination and rate-limit pacing updates; the default concurrency limit is now four. See #14179 and #14431.
• GitHub nested pages: Scans could return incomplete reviews or comments because pagination accepted a short page as complete. The connector now rejects incomplete connections and repeated cursors. It retries a failed nested page without another fetch of the outer page. Datasets with the same token also share the REST quota. See #14862.
• Iceberg REST: Clients could read an empty dataset because the catalog synthesized metadata without snapshots. The catalog now returns the source metadata for Iceberg datasets that Spice reads unchanged. Other datasets return 400 BadRequestException. Clients need their own storage credentials. See #14588 and Breaking Changes.

Other Fixes​

The release includes fixes in the following areas; the linked PRs provide details of the changes:

• Cayenne queries and writes: NULL-aware NOT IN, maintained aggregates, dynamic filters, partition-filter forwarding, memory-mode DML and retention, and primary-key handling across CDC checkpoints. See #14429, #14761, #14370, #14047, and #14344.
• Startup and reloads: retry datasets with unavailable sources, serve existing accelerations during source outages, and invalidate cached plans and results on catalog replacement or dataset unload. See #14623, #14624, #13914, and #14365.
• Federation: local evaluation of casts and functions whose source semantics differ, plus filter pushdown changes for DynamoDB, Cosmos DB, and MongoDB. See #14484, #14601, and #14419.
• HTTP and GraphQL: response-status handling during refresh, retry-budget handling, non-JSON gateway responses, and URL redaction in HTTP errors. See #13538, #14313, #14781, and #14490.
• Search: deletion of obsolete Elasticsearch chunks, non-finite embedding handling, and source-scan coordination during full-text refresh. See #13960, #13902, and #14663.
• Models and tools: tool-call-only assistant turns, required tool choices, streaming tool-use completion, model-load diagnostics, and propagation of the API-key principal into MCP tool calls. See #14232, #14460, #14548, and #14828.
• CDC shutdown and reconnects: source-position recording before accelerations close, and MySQL shared-stream reconnect handling. See #14702 and #14751.
• SQL weekdays: date_part('dow') aligns with EXTRACT(dow), with Sunday represented as zero. See #14796.
• Cayenne schema statistics: Decimal bounds could retain an old scale after schema evolution because maintenance published statistics from the previous schema. Cayenne now rejects statistics from an obsolete schema and keeps row counts conservative. See #14856.
• Vector search: vector_search planning failed after the DataFusion 55 upgrade because a second optimization pass tried to reorder a join with a dynamic filter. The planner now preserves that join's input order. See #14857.

Default accelerator: Datasets and views that enable acceleration without engine now use Cayenne. Explicit engine settings keep their behavior. Storage still defaults to memory. Set mode: file for persistent acceleration. See Breaking Changes for migration guidance.

Dependency Updates​

Contributors​

• @bjchambers
• @ewgenius
• @Jeadie
• @krinart
• @lukekim
• @peasee
• @phillipleblanc
• @sgrebnov

Breaking Changes​

Cayenne is the default accelerator on supported platforms. A dataset or view that omits acceleration.engine switches from Arrow to Cayenne. To retain Arrow, set engine: arrow explicitly before upgrading. Windows keeps Arrow as its default. Persistent datasets should continue to name their engine and use mode: file.

on_conflict is deprecated and scheduled for removal in v3.0. Cayenne automatically keeps one row per primary key, choosing the newest time_column value when configured, or the last arrival otherwise. Existing explicit policies remain supported during the deprecation period. Review those policies before removing them, especially drop or policies that reject conflicting rows.

on_conflict no longer routes writes to the acceleration. For read-write datasets whose writes should stay in the acceleration, use:

acceleration:
  enabled: true
  engine: cayenne
  write_mode: acceleration

This setting cannot be used with refresh_mode: changes, including a connector's default change-stream mode. The default write_through and write_back modes require a writable source.

Cache engine selection is retired. engine: moka and engine: pingora remain accepted but are ignored. Remove the field and use caching_policy to select eviction behavior.

GitHub connector default concurrency is four. Review explicit concurrency settings if your deployment relied on the previous default.

HTTP rate-control waits are bounded by default. Requests waiting for rate-control capacity now time out after the connector's client timeout. Set rate_control_acquire_timeout to a suitable duration, or 0 to retain the previous unbounded wait behavior.

Cayenne acceleration snapshots are unavailable for datalake-tier datasets. Review snapshot settings on datasets using cayenne_datalake_location; this release disables snapshotting that configuration.

Iceberg REST no longer synthesizes metadata for unsupported datasets. GET /v1/namespaces/{namespace}/tables/{table} returns 400 BadRequestException for accelerated datasets, views, and other datasets that Spice does not read unchanged from Iceberg. If a client used this endpoint for schema discovery, use SQL DESCRIBE or information_schema.columns instead. Query these datasets through /v1/sql or Arrow Flight SQL. For eligible Iceberg datasets, clients read the source metadata and need their own storage access. See the Get a table API.

Cookbook Updates​

The Spice Cookbook provides recipes to help you get started with Spice.

Upgrading​

To upgrade to v2.4.0-rc.1 once the release artifacts are available, use one of the following methods:

CLI:

spice upgrade v2.4.0-rc.1

Docker:

Pull the spiceai/spiceai:2.4.0-rc.1 image:

docker pull spiceai/spiceai:2.4.0-rc.1

For available tags, see DockerHub.

Helm:

helm repo update
helm upgrade spiceai spiceai/spiceai --version 2.4.0-rc.1

AWS Marketplace:

Spice is available in the AWS Marketplace. Marketplace availability follows its published versions.

What's Changed​

Changelog​

• fix(runtime): discard cached logical plans when a hot reload replaces a catalog (fixes #13910) by @claudespice in #13914
• fix(acceleration): let a schema repair correct a checkpoint without resetting the freshness clock (fixes #13817) by @claudespice in #13894
• fix(search): filter a chunked Elasticsearch delete on a field that can match the key (fixes #13714) by @claudespice in #13926
• fix(search): classify a partially non-finite embedding as unindexable on every backend (fixes #13872) by @claudespice in #13902
• fix: stabilize GitHub tests and bound GraphQL registration (fixes #13762) by @lukekim in #13939
• fix(postgres): decode versioned JSONB binary replication values by @phillipleblanc in #13962
• docs: require a reviewed Enhancement before any user-facing surface changes by @lukekim in #13970
• ci: upgrade spiceio setup action to v0.9.0 by @lukekim in #13971
• fix(postgres): preserve microseconds in timestamp writeback by @phillipleblanc in #13963
• feat(hash-index): verify the bloom filter's block index with Verus by @lukekim in #13777
• build(deps-dev): bump js-yaml by @dependabot in #13989
• docs: release notes for v2.3.0 by @bjchambers in #13999
• fix(ci): drop the dangling substrait-compliance submodule pointer by @bjchambers in #14002
• fix(cayenne): release the keyset bytes an abandoned PK checkout accounted (fixes #13668) by @grokspice in #13925
• fix(caching): keep a declared key from disabling eviction and stranding stale rows (fixes #13976) by @bjchambers in #13992
• Add Substrait compliance harness (IBM TPC-H Mode A + FlightSQL Mode B stub) by @lukekim in #13879
• ci: skip DynamoDB TPC-H benches in OSS testoperator dispatch by @phillipleblanc in #14016
• docs: update security support and roadmap after v2.3.0 by @phillipleblanc in #14024
• chore: post v2.3.0 release housekeeping by @bjchambers in #13969
• test(adbc): guard BigQuery corpus offline and in release gate by @phillipleblanc in #14017
• fix(duckdb): deny the regexp built-ins DuckDB cannot answer faithfully (fixes #13809) by @claudespice in #13871
• fix: Update tpch benchmark snapshots for federated/adbc[bigquery].yaml by @app/github-actions in #13984
• fix(search): drop the chunks a shortened row no longer produces from a chunked index (refs #13717) by @claudespice in #13960
• Reduce Cayenne allocations during primary-key validation and filtering by @lukekim in #14009
• test(forks): guard seven fork patches that had no repo-side test by @krinart in #13996
• fix(deps): bump arrow-rs to correctly-rounded Decimal→Float cast (closes #13978) by @Jeadie in #14012
• perf(vortex): defer projection setup on filtered scans until the filter resolves by @bjchambers in #14035
• endgame: include spiceai/skills versioned release by @lukekim in #14031
• fix(caching): partition doomed entries at the survivor cutoff so eviction converges (closes #13994) by @Jeadie in #14021
• fix(deps): bump arrow-rs fork pin for Decimal->Float rounding fix by @Jeadie in #14049
• Fix subqueries with use_source acceleration by @phillipleblanc in #14022
• fix(cayenne): make DELETE, UPDATE and INSERT work on a mode: memory acceleration (fixes #12008) by @bjchambers in #14047
• fix: clarify OpenDAL S3 retry warnings by @lukekim in #14040
• test(s3): run the parquet-overwrite fixtures on RustFS by @bjchambers in #14067
• feat(cayenne): materialize multi-reference CTEs on the query path by @lukekim in #13918
• perf(vortex): answer a constant IN list by probing a set, and falsify it by interval by @bjchambers in #14061
• perf(vortex): skip a scan split whose zones cannot satisfy the filter by @peasee in #14064
• fix(arrow): report an exact row count from the indexed point-lookup scan by @krinart in #13972
• fix(cayenne): apply sort_columns with refresh_mode: full by @peasee in #14063
• fix(smb): pad an empty CREATE buffer so Samba lists the share root (fixes #13293) by @grokspice in #14050
• perf(cache): key the logical-plan cache on SQL text, not parameter values by @bjchambers in #14069
• fix: harden HuggingFace E2E chat against slow Metal generation by @lukekim in #14072
• fix(cayenne): move accelerator filesystem I/O off Tokio workers by @lukekim in #14073
• test(chbench): enable CTE materialization and IVM on mysql/postgres adaptive HTAP by @lukekim in #14070
• ci: run Substrait Mode A TPC-H on pull requests and the merge queue by @lukekim in #14071
• feat(mcp): support MCP specification 2026-07-28 (dual-era) by @lukekim in #14043
• fix(ci): call a linker that died of a signal an infrastructure failure, not a check failure (fixes #13614) by @grokspice in #14044
• fix: Provide temporary directory in docker images by @Jeadie in #14089
• fix: restore OSS installer, CLI and test workflow coverage by @phillipleblanc in #14025
• Delete v2.2.0.md by @Jeadie in #14095
• docs(release): add v2.3.1 release notes by @phillipleblanc in #14094
• fix(test): allow DELETE in the CORS allow-methods assertion by @claudespice in #14098
• fix(ci): stop install-protoc unzipping into a shared ~/.local by @lukekim in #14097
• feat(connectors): add ORC listing format via in-repo FileFormat by @lukekim in #14075
• fix(cayenne): run snapshot bootstrap check before opening the metastore by @Jeadie in #14093
• feat(cache): verify the results-cache namespace prefix with Verus by @lukekim in #14074
• feat(cloud-connect): add a GetDatasets command that answers the /v1/datasets document (refs #13369) by @grokspice in #14051
• Suppress Cayenne startup logs when no Cayenne dataset is configured by @Jeadie in #14042
• fix(cache): re-bind parameter values when revalidating a stale result (fixes #14099) by @bjchambers in #14100
• fix(cluster): support distributed HTTP scans by @phillipleblanc in #14108
• fix(bigquery): keep ILIKE evaluation local by @phillipleblanc in #14110
• docs: update security support for v2.3.1 by @phillipleblanc in #14117
• feat(cayenne): reuse ScanView until write, lag only for read-only CDC by @lukekim in #14055
• fix(deps): remediate open Dependabot alerts by @phillipleblanc in #14111
• fix(testoperator): validate results in every scale factor 1 TPC-H, TPC-DS and ClickBench benchmark by @lukekim in #14119
• fix(cayenne): reject ambiguous metastore paths by @phillipleblanc in #14130
• perf: serve results-cache hits where the request arrives and cut per-hit overhead by @lukekim in #14103
• test(runtime): record query previews in the management export test by @lukekim in #14155
• ci: upgrade spiceio setup action to v0.11.0 by @lukekim in #14152
• feat(caching): Make caching_stale_if_error RFC-5861 compliant (with stale-if-error header) by @Jeadie in #14134
• perf(runtime-table): defer cache-eviction key extraction to entries a delete actually names by @Jeadie in #14138
• fix(runtime): report the acceleration.ready_state deprecation once per component (fixes #13749) by @claudespice in #14006
• fix(runtime): write the inferred Arrow sort order under the prefixed key its validation accepts (fixes #14023) by @claudespice in #14032
• fix(connectors): Fix JSON/Orca files using metadata columns by @Jeadie in #14115
• feat(cayenne): build secondary indexes from indexes in file and memory mode by @phillipleblanc in #14149
• fix(cayenne): round-trip decimal, binary, and time stats and drop them on scale change by @lukekim in #14139
• feat(cayenne): cluster warm and datalake tiers, and write full refreshes as key-range files by @lukekim in #14124
• fix(cayenne): compile the cold-tier pruning test and backtick a doc literal by @lukekim in #14175
• fix(cayenne): make the crates own targets lint and compile by @phillipleblanc in #14200
• test(forks): guard five more fork patches, and drop a row that is not fork state by @krinart in #14015
• perf(cache): promote encoded SQL results to raw after the second decode by @lukekim in #14199
• fix(turso): build a dictionary column directly so a dictionary over a list, map or boolean value reads back (fixes #13033) by @grokspice in #14181
• fix(ci): probe the macOS toolchain before reaching for brew in the release builds by @grokspice in #14203
• fix(ci): skip Metal kernel precompilation in the macOS release build by @grokspice in #14204
• fix(github): paginate nested GraphQL connections and pace to GitHub's rate limits by @lukekim in #14179
• fix(vortex): stop an IN list holding a NULL from panicking the scan by @krinart in #14163
• bench(cayenne): use std::hint::black_box in the clustering bench by @lukekim in #14129
• fix(runtime-table): stop rebuilding SessionContext on every cache fetch by @Jeadie in #14141
• test(chbench): cluster order_line, oorder and customer on the adaptive HTAP arms by @lukekim in #14192
• fix(runtime): count a first load as still loading in the Dataset load summary (fixes #13974) by @claudespice in #14020
• fix(duckdb): push regexp_count down again at a rendering that counts as the kernel does (fixes #13870) by @claudespice in #14153
• build: lint and test the sign-off under the same profile as the merge queue by @lukekim in #14180
• ci: require the Verus proofs in the merge queue as one check by @lukekim in #14189
• ci: run the longest macOS jobs on their own runner pool by @lukekim in #14229
• fix(cayenne): build the DELETE sink inside the execution-time write lock (fixes #13828) by @claudespice in #14218
• build(deps): bump the github-actions-dependencies group across 1 directory with 7 updates by @dependabot in #14231
• fix(cluster): decide what a Flight message carries by its IPC header, not its body length (refs #13737) by @claudespice in #14212
• ci: run Mode A TPC-H on merge queue and trunk/release push only by @lukekim in #14247
• chore(deps): bump spiceai/duckdb-rs to 76655d2f by @lukekim in #14246
• ci: stop exporting empty AWS and DuckLake endpoints to the schema test by @phillipleblanc in #14194
• fix(runtime): reload a localpod dataset when the dataset it reads through is reloaded (fixes #3288) by @claudespice in #14208
• build(deps): bump the aws-sdk group with 3 updates by @dependabot in #14255
• fix(ci): resolve Homebrew prefix when brew is the spice flock wrapper by @lukekim in #14210
• chore(deps): raise the datafusion-table-providers pin to include the NUMERIC result-column fix by @phillipleblanc in #14254
• ci: align the DuckLake bootstrap with the embedded DuckDB, wait for Databricks startup, and stop dispatching legs that cannot pass by @phillipleblanc in #14250
• fix(runtime): install the Spice function deny-list on the PostgreSQL catalog connector (refs #13664) by @claudespice in #14225
• perf(cayenne): share inline-cache view entries by Arc instead of cloning them per scan by @krinart in #14191
• build(deps): bump aws-actions/configure-aws-credentials by @dependabot in #14256
• ci: stop dispatching the indexed turso TPC-H SF1 tests by @phillipleblanc in #14252
• fix(catalog): keep the tables registered under an existing schema by @phillipleblanc in #14193
• feat(cache): Spice sharded cache as the sole LruCache engine by @lukekim in #14206
• ci: lint GitHub Actions definitions with actionlint, and fix the 73 findings it surfaced by @grokspice in #14223
• perf(cache): tighten the Raw SQL results-cache serve path by @lukekim in #14205
• ci: stop triggering the CUDA build on pull requests by @lukekim in #14267
• ci: run CodeQL on pull requests and the merge queue by @lukekim in #14269
• Release 2.3.2 release notes by @krinart in #14271
• chore: make AGENTS.md the canonical agent instructions by @lukekim in #14237
• ci: run CodeQL Analyze on spiceai-dev-runners by @lukekim in #14281
• feat: TypeSafe Jev System One evaluation provider by @lukekim in #14215
• fix(cayenne): tag file statistics bounds as the column's Arrow type (fixes #14280) by @phillipleblanc in #14283
• ci: install spiceio when the runner has no gh (refs #14233) by @lukekim in #14288
• test(forks): 11 repo guards by @krinart in #14261
• docs: add Spice.ai in Action manuscript and companion labs by @lukekim in #13965
• test(duckdb): add an integration test for the index CTE materialization by @sgrebnov in #13885
• fix(cayenne): coalesce inline writes into one batch by @sgrebnov in #14279
• release: Update SECURITY.md and endgame template after 2.3.2 by @peasee in #14293
• fix(cayenne): count each Arrow allocation once in the inline-cache gauge by @krinart in #14272
• feat(s3): SQS event-driven changes for refresh_mode: changes by @lukekim in #14121
• feat(caching): single-flight coalesce concurrent cache-miss fetches by @Jeadie in #14142
• fix(caching): make caching_stale_if_error detect transient HTTP failures on real schemas by @krinart in #14161
• Use Cayenne secondary indexes for dynamic join filters by @phillipleblanc in #14282
• fix(ci): compare DynamoDB sets without relying on element order by @bjchambers in #14328
• docs(release): Remove QA analytics step from endgame by @peasee in #14329
• ci: run CodeQL on the merge queue and trunk, not on pull requests by @lukekim in #14290
• docs(cayenne): reposition the reference, add query serving, re-audit against trunk by @lukekim in #14338
• docs(endgame): update versioned docs release steps by @ewgenius in #14277
• test(forks): guard the ballista per-task file-scan restriction by @krinart in #14292
• fix(cli): surface the full error chain for spice chat connection failures by @krinart in #14289
• fix(ci): degrade the incomplete-sign-off handler when the runner has no gh (refs #14234) by @claudespice in #14304
• fix(runtime-table): serialize a direct write against acceleration snapshot creation (fixes #13548) by @claudespice in #14310
• fix(duckdb): screen regexp_like and regexp_replace as regexp_count is screened (fixes #14148) by @claudespice in #14321
• fix(http): honor retry budget without an extra origin request by @phillipleblanc in #14313
• Fix SchemaCastScanExec's schema conversion in fn partition_statistics by @Jeadie in #14258
• fix(cluster): recognise a Flight keepalive by its empty envelope, not by what its header declares (fixes #13737) by @claudespice in #14327
• perf(http): defer zero-TTL acceleration lookup until origin failure by @phillipleblanc in #14302
• fix(cayenne): keep a key visible when it is re-inserted over a stale-insert tombstone by @sgrebnov in #14312
• perf(cayenne): read only key and filter columns in filtered key deletes by @sgrebnov in #14374
• fix(cayenne): let small protected-snapshot merges run during a long large-tier merge by @sgrebnov in #14296
• perf(cayenne): serve primary-key lookups on a freshly loaded table in ~1 ms by @lukekim in #14314
• fix(cayenne): skip min/max statistics for nested columns (fixes #14368) by @sgrebnov in #14392
• test(caching): cover SchemaCastScanExec statistics projection by name, retype, and SQL filter by @Jeadie in #14146
• fix(turso): keep a quantified comparison out of Turso SQL (fixes #14041) by @grokspice in #14393
• fix(udfs): declare the local_embed dev-dependency the embed tests need (fixes #13092) by @claudespice in #14377
• perf(cayenne): batch metastore manifest rewrites, upsert in place, and run every write on one writer connection by @lukekim in #14369
• fix(runtime-table): ignore zero-row batches in stale fallback by @phillipleblanc in #14331
• Prune object-store file listing by _last_modified predicates by @Jeadie in #14265
• Reading only partition or metadata columns needlessly scans all file contents by @Jeadie in #14116
• fix(duckdb): keep a concat over a binary operand out of the federated plan (fixes #13915) by @claudespice in #14333
• fix(ci): keep the sign-off attribution inside GitHub's 140-character status cap (fixes #14076) by @grokspice in #14390
• fix(ci): expose a present-but-unlinked cc tool on macOS runners instead of routing it through brew install (fixes #13479) by @grokspice in #14387
• ci: re-measure integration.yml's job bounds after the archive consolidation (fixes #13429) by @grokspice in #14388
• fix(ci): start DuckLake's local MinIO from an image that is still published by @grokspice in #14399
• test(runtime-table): make the metric-scraping refresh tests pass under cargo test by @claudespice in #14381
• fix(federation): keep a correlated subquery predicate above a join of two sources (refs #8220) by @claudespice in #14372
• fix(caching): snapshot staleness before the origin fetch for stale_if_error by @Jeadie in #14263
• perf(snapshots): skip unchanged snapshot metadata with a conditional GET by @sgrebnov in #14409
• fix(search): prune the rest of a key group from an Elasticsearch chunked index (refs #13717) by @claudespice in #14320
• fix(bench): derive MySQL's empty-field NULL handling from the column type (refs #13152) by @claudespice in #14345
• fix(graphql): debit a LIMIT by the rows a page returned, not the declared page size (fixes #14308) by @claudespice in #14353
• fix(ci): fit retention_oom's retry budget inside its workflow step, and guard the coupling (fixes #13512) by @grokspice in #14389
• docs: say plainly what Spice is, refresh the README for v2.3, and promote connector statuses by @lukekim in #14410
• fix(ci): pin, checksum and retry the oha download in the E2E graceful-shutdown jobs by @grokspice in #14418
• fix(snapshots): resolve snapshot entries relative to the metadata location (#14425) by @sgrebnov in #14426
• Lower the GitHub connector default concurrency limit to 4 by @lukekim in #14431
• build(deps): bump nvidia/cuda in the docker-dependencies group by @dependabot in #14439
• build(deps): bump the aws-sdk group with 3 updates by @dependabot in #14440
• build(deps): bump the github-actions-dependencies group across 1 directory with 5 updates by @dependabot in #14441
• test(cayenne): bound the refused-build guard by builds, not by the host's speed by @grokspice in #14424
• fix(cache): don't report an invalidation cancelled by runtime shutdown as a failure by @grokspice in #14417
• ci: run remote sign-off on the spiceai-macos pool by @lukekim in #14449
• ci: run CodeQL Analyze on spiceai-macos by @lukekim in #14442
• fix(runtime): keep the built-in date_part so both weekday spellings agree (fixes #13920) by @claudespice in #14154
• fix(cayenne): include the in-memory CDC tier when an overwrite or a retention pass covers the whole table by @lukekim in #14428
• fix: keep NOT IN null-aware through the join reorder and the Cayenne sort-merge rewrite by @lukekim in #14429
• fix(cayenne): discard a compaction whose snapshot an overwrite replaced mid-pass by @lukekim in #14432
• fix(cayenne): stop maintained views, Vortex IN lists and dynamic-filter sharing from returning wrong rows by @lukekim in #14427
• fix(cayenne): hide spilled rows on CDC upsert fallback by @bjchambers in #14416
• fix(ci): skip the integration, ADBC, chDB and E2E gate jobs on pull requests instead of passing them (fixes #13841) by @grokspice in #14454
• fix(cayenne): judge a filtered key delete's captured sources by the index captured with them (refs #13913) by @claudespice in #14455
• fix(ci): clear the macos-15 image's openssl@1.1 symlink before installing MySQL by @lukekim in #14489
• ci: produce CodeQL SARIF in the Analyze job on spiceai-macos by @lukekim in #14500
• fix(cayenne): correctness fixes for the goal-driven adaptive controller, with a closed-loop simulation harness by @lukekim in #14443
• fix(cdc): keep the newest source commit timestamp on a coalesced change batch by @lukekim in #14463
• fix(cayenne): draw a sequence for a current-snapshot append so per-key OCC can order it (fixes #13685) by @claudespice in #14360
• fix(http): name a configured model's load failure instead of reporting it not found (fixes #13303) by @claudespice in #14395
• fix(duckdb): keep inferred source indexes off change-stream accelerations so upserts commit under concurrent reads (refs #13929) by @claudespice in #14396
• fix(duckdb): keep a text cast over a binary operand out of the federated plan (fixes #14355) by @claudespice in #14448
• fix(llms): honor tool_choice required and allowed_tools on mistral.rs-hosted models instead of panicking (fixes #14230) by @claudespice in #14460
• test(runtime): run the load-error counter test in its own process (fixes #13085) by @claudespice in #14462
• fix(runtime): stop a replaced dataset configuration's load from registering over the new one (fixes #1458) by @claudespice in #14367
• fix(install): stop asking for sudo on a first install into a fresh HOME (fixes #14445) by @claudespice in #14495
• perf(cayenne): serve primary-key point lookups in half the time by @lukekim in #14433
• fix(deps): bump DataFusion for upstream fixes to wrong results from filter pushdown, simplification and planning by @lukekim in #14430
• fix(runtime): size every internal DataFusion session from the CPU budget by @bjchambers in #14412
• fix(runtime): serve nested, zoned and half-float columns from the Iceberg catalog API (fixes #4815) by @claudespice in #14480
• fix(acceleration): keep cached results when a snapshot refresh finds no newer snapshot by @sgrebnov in #14497
• fix(runtime): sleep cron tests to the next boundary, not one that already fired (fixes #13759) by @claudespice in #14474
• fix(ci): derive every lint-rust guard make runs, whatever its separator or recipe layout (fixes #13783) by @claudespice in #14475
• fix(cayenne): stop the small-file compaction of a position-mode PK table from deadlocking on its own write lock (fixes #14420) by @claudespice in #14481
• fix(runtime-table): report refresh bytes for the rows each batch holds by @Jeadie in #14469
• fix(spark,databricks): keep Spice-only functions out of SQL sent to Spark Connect and Databricks SQL Warehouse (refs #13664) by @claudespice in #14498
• fix(vortex): size a cached footer by what it retains, not its serialized bytes (fixes #12917) by @claudespice in #14502
• fix(cayenne,telemetry): Register accelerated sink dataset immediately from existing acceleration by @peasee in #13955
• Update spicepod.yml by @Jeadie in #14533
• fix(cayenne): keep a rewrite's count inexact when it retains a late protected snapshot (fixes #14383) by @claudespice in #14385
• fix: Update tpch benchmark snapshots for accelerated/on_zero_results/file[parquet]-cayenne[file]-on_zero_results.yaml by @app/github-actions in #14408
• build(rust): upgrade toolchain to 1.98.1 by @lukekim in #14560
• fix(postgres): release a shared slot's hold on a table its publication cannot drop (fixes #13032) by @claudespice in #14527
• fix(cayenne): size the build side before sort-merging a non-Cayenne outer join by @krinart in #14520
• Update DF Upgrade template by @krinart in #14526
• fix(ci): wait for the refresh to invalidate the results cache, not a fixed 3s by @grokspice in #14598
• fix(deps): move the DataFusion pin past the four unparser fixes, and guard each of them (fixes #13022) by @grokspice in #14570
• fix(dynamodb, cosmosdb, mongodb): push filters down only where the source evaluates them as SQL does by @lukekim in #14419
• feat(snapshots): serve a dataset from published snapshots with from: s3://… and file_format: snapshot by @lukekim in #14529
• perf(cayenne): keep the per-shard PK index across checkpoint flushes and back off futile bakes; fix q17 and two wrong-results races (fixes #14235) by @lukekim in #14555
• test(chbench): lower the MySQL adaptive pods' CDC linger to 500 ms by @lukekim in #14615
• fix(graphql): show the parse-failure bytes in JSON decode previews by @Jeadie in #14530
• Defer source-first cache fallback planning by @phillipleblanc in #14556
• perf(cayenne): order whole-table rewrites per scan partition, then merge by @bjchambers in #14488
• fix(ci): tolerate the parquet-rename race's third DuckDB error in the E2E log scan by @grokspice in #14541
• fix(graphql): retry an inferred credential refusal instead of failing the refresh by @Jeadie in #14539
• fix(cayenne): serve a widened table's files from their persisted statistics (refs #13829) by @claudespice in #14316
• fix(acceleration): checkpoint DuckDB's write-ahead log before a snapshot copies its file (fixes #13912) by @claudespice in #14325
• fix(ci): let E2E cleanup run when the job failed before creating its working directory by @grokspice in #14553
• Replace unmaintained backoff with a workspace crate by @phillipleblanc in #14621
• test: read Expect stand-ins with the installed shell by @phillipleblanc in #14634
• fix(models): apply a tool_choice that forces a call to one round of the tool-use loop (fixes #14459) by @claudespice in #14635
• fix(runtime): don't end a tool-use stream on a stale tool_calls finish (fixes #13309) by @claudespice in #14548
• fix(smb): name listed locations under the share so directory datasets load (fixes #14060) by @claudespice in #14549
• fix(models): name a configured model's load failure in ai() and streaming nsql (fixes #14394) by @claudespice in #14632
• test(chbench): remove the mysql-cayenne[file]-adaptive-split HTAP arm by @sgrebnov in #14620
• fix(cayenne): record sharded CDC keys in the table-wide PK index by @lukekim in #14603
• fix(sqlite): keep TRY_CAST and every cast SQLite answers differently out of the federated plan (fixes #14398) by @claudespice in #14496
• ci: move remaining Ubuntu 22.04 references to 24.04 by @lukekim in #14677
• fix(snapshots): retry a failed snapshot attempt with backoff by @sgrebnov in #14676
• fix(snapshots): large snapshots no longer fail bootstrap on slow connection by @sgrebnov in #14571
• fix(refresh): start a refresh's source scan only when the sink reads it, so the full-text index keeps its rows (fixes #14619) by @claudespice in #14663
• test(cayenne): compare suite answers cell by cell against SQLite and chDB by @lukekim in #14473
• Evaluate any chat model via /v1/evaluate by @lukekim in #14568
• ci: quarantine the management API integration schedule until its dev OAuth client is reactivated (refs #12376) by @grokspice in #14684
• ci: stop scheduled Testoperator Ballista benchmarks by @phillipleblanc in #14682
• fix(cayenne): stop snapshotting datasets with a datalake tier, whose restored copies deleted each other's files by @lukekim in #14583
• fix(cayenne): count each Arrow allocation once for the mem-tier limit and checkpoint write sizing by @krinart in #14675
• fix(ci): classify a sign-off that reached its step budget as signalled, so an expiry publishes no verdict (fixes #13843) by @grokspice in #14685
• fix(runtime): retry a dataset whose source is unreachable at startup (fixes #14609) by @bjchambers in #14623
• fix(cayenne): archive only the snapshot directories a dataset snapshot references (fixes #14605) by @sgrebnov in #14627
• ci: route trunk-push macOS builds to the standard pool by @phillipleblanc in #14645
• ci: install strip for the retention OOM regression test by @phillipleblanc in #14701
• fix(smb): serve every share on a host from the one store registered for it (fixes #14550) by @grokspice in #14689
• fix(runtime): discard cached plans and results when a dataset is unloaded (refs #14251) by @claudespice in #14365
• feat(snapshots): reload snapshot-mode datasets from S3 event notifications (SQS) by @lukekim in #14335
• fix(cayenne): report runs per size tier when protected-snapshot compaction declines (fixes #13622) by @claudespice in #14476
• fix(cayenne): re-run a post-write compaction pass a concurrent append asked for (fixes #13906) by @claudespice in #14479
• fix(http): keep the request URL out of HTTP connector errors (fixes #13534) by @claudespice in #14490
• fix(runtime): load every localpod dataset that reads from one parent at startup (fixes #13087) by @claudespice in #14532
• fix(build): share one cargo metadata helper across the lint guards, so a broken cargo is never a violation (fixes #13121) by @claudespice in #14534
• fix(duckdb): push regexp_replace down again for a one-digit group reference, and refresh the Postgres ClickBench q35 plan (refs #13966) by @claudespice in #14552
• fix(duckdb): keep a cast into binary out of the federated plan (refs #14397) by @claudespice in #14633
• fix(runtime): keep finished async query jobs finished, and delete expired ones by @lukekim in #14585
• fix(federation): keep DataFusion's cast built-ins out of every federated plan (fixes #14444) by @claudespice in #14484
• test(cayenne): cover a swapped join filter through the sort-merge rewrite end to end (refs #14235) by @claudespice in #14551
• fix(runtime): decide every Spark/built-in function collision by name, and refuse an undecided one (fixes #14361) by @grokspice in #14400
• fix(ci): run every bin target's unit tests in the sign-off gate, and give spice connect its own --cloud-region refusal (fixes #13426) by @grokspice in #14406
• test(data_components): compile the federation unparser guards in every scoped test run (fixes #13625) by @grokspice in #14447
• fix(runtime-component): skip an inferred Cayenne index on a floating-point column instead of failing the load (fixes #14590) by @grokspice in #14599
• fix(cache): serve stale cached results during frequent table updates (stale_while_revalidate_ttl) by @sgrebnov in #14708
• Update Turso to 0.8.1 and retry write conflicts a metastore statement raises by @lukekim in #14680
• fix(snapshots): stop a snapshot dataset's load when a reload replaces it by @lukekim in #14673
• fix(cayenne): hand non-partition filters to every partition scan (fixes #12959) by @claudespice in #14370
• fix(postgres-accel): resolve secret references in the sidecar connection parameters (fixes #13296) by @claudespice in #14544
• fix(cayenne): drain the in-memory CDC tier before a full rewrite scans it (fixes #14450) by @claudespice in #14486
• feat(caching): warm SQL results cache on first refresh from persisted plan shapes by @lukekim in #14178
• fix(ci): retry only the container startup in the zero-retry MySQL CDC tests by @grokspice in #14727
• feat(key-index): immutable secondary index runs over compound Arrow keys by @bjchambers in #14592
• fix(federation): keep a fractional-to-integer cast out of the plans pushed to DuckDB, PostgreSQL, MySQL and BigQuery (fixes #14482) by @grokspice in #14601
• fix(testoperator): give accelerated bench configs a 300s ready_wait and name unready datasets on timeout (fixes #13973) by @claudespice in #14728
• fix(federation): keep arrow_typeof and its plan-introspection siblings local on every backend (fixes #14334) by @grokspice in #14695
• test(bench): refresh the tpch_q16 explain snapshots for the null-aware NOT IN plan (fixes #13977) by @claudespice in #14731
• ci: build trunk-push macOS legs on spiceai-macos-large again, keeping one-at-a-time coalescing by @grokspice in #14735
• ci(verus): derive the verified crates from cargo metadata; pin vstd once by @bjchambers in #14709
• docs: raise the test and evidence bar: differential first, exact assertions, performance always measured by @lukekim in #14723
• test(vortex): wait for a dropped segment cache to be freed before asserting it is gone (fixes #13295) by @claudespice in #14470
• ci(integration): report each integration part's own result in its required check by @grokspice in #14743
• fix(cayenne): back off futile bakes under a violated query goal too by @lukekim in #14736
• feat(object_store_occ): add transactional WAL and MVCC snapshots by @lukekim in #14732
• fix(snapshots): bootstrap readers from late snapshot publications by @phillipleblanc in #14468
• fix(cayenne): apply retention_sql to a mode: memory acceleration (fixes #14045) by @claudespice in #14344
• fix(duckdb, sqlite): keep the first copy of a key a write repeats under on_conflict: drop (fixes #14629) by @claudespice in #14748
• fix(cache): release a removed dataset's memory when the plan cache discards its plans (fixes #14251) by @claudespice in #14760
• build(deps): bump the aws-sdk group with 2 updates by @dependabot in #14764
• fix(llms): explain an Anthropic model's refusal of a sampling control instead of passing the bare 400 through (fixes #13564) by @grokspice in #14700
• build(deps-dev): bump dompurify by @dependabot in #14613
• fix(snapshots): allow cayenne_file_path and cayenne_metadata_dir on file_format: snapshot datasets (fixes #14696) by @sgrebnov in #14698
• test(snapshots): make snapshot integration tests robust under parallel CI runs by @sgrebnov in #14765
• feat(cayenne): keep the secondary index current across every write by @bjchambers in #14593
• fix: Update tpch benchmark snapshots for accelerated/file[parquet]-duckdb[memory].yaml by @app/github-actions in #14674
• Isolate Docker integration fixtures across concurrent processes by @phillipleblanc in #14639
• ci: run Substrait Mode A TPC-H at SF 1 on spiceai-macos in every merge-queue entry by @lukekim in #14522
• ci: stop repeating scheduled benchmarks: one source per accelerator, hosted sources weekly, a release commit once by @lukekim in #14681
• build(deps): bump nvidia/cuda by @dependabot in #14763
• fix(http): stop a non-2xx response body replacing an accelerated table's rows (fixes #13515) by @claudespice in #13538
• fix(chat-api): answer a tool-call-only assistant turn instead of panicking (fixes #13207) by @grokspice in #14232
• fix(sqlite, duckdb): keep a decimal AVG, and on SQLite a decimal SUM, out of the federated plan (fixes #14492) by @claudespice in #14670
• fix(spiceai, duckdb, sharepoint): name the Spicepod key a missing-parameter error asks for (refs #14446) by @claudespice in #14769
• Add table-bound ChangeSink ownership and backends by @phillipleblanc in #14704
• fix(ci): serialize rustup installs on shared macOS runners by @lukekim in #14717
• fix(cayenne): keep the per-shard PK index when the table-wide index is discarded by @lukekim in #14604
• Upgrade to DataFusion 55.1 and Arrow 59.3 by @krinart in #14612
• Update openapi.json by @app/github-actions in #14742
• build(deps): bump rustls from 0.23.40 to 0.23.45 by @dependabot in #14773
• Upgrade to iceberg-rust to 0.11.0 and DF 55.2 by @sgrebnov in #14771
• fix(cayenne): write an append into an empty unkeyed table as one load by @phillipleblanc in #14784
• fix(mysql_replication): don't re-send a live member's delivered commits after a reconnect by @lukekim in #14751
• ci(codeql): don't fail the SARIF upload when the merge queue already deleted its ref by @grokspice in #14793
• fix(acceleration): accept time_format iso8601 when the time_column is already a timestamp by @lukekim in #14777
• fix(cache): store SQL results that go stale mid-query by @lukekim in #14710
• fix(cayenne): serve filtered and global maintained aggregates by @lukekim in #14761
• perf(cayenne): build the checkpoint's tombstone union after releasing the capture locks by @lukekim in #14759
• ci: keep no artifacts or caches from PR and merge-queue checks, and run gating macOS jobs on spiceai-macos-large by @lukekim in #14804
• fix(cayenne): keep maintenance from deleting files a snapshot is archiving by @sgrebnov in #14789
• fix(snapshots): never overwrite shared snapshot metadata, and publish more than once to a file:// location by @lukekim in #14582
• perf(cdc): build deferred change rows in the reader while the apply runs by @lukekim in #14746
• fix: Update tpch benchmark snapshots for accelerated/on_zero_results/file[parquet]-cayenne[file]-on_zero_results.yaml by @app/github-actions in #14786
• test(testoperator): add a cold-start time-to-ready regression test by @phillipleblanc in #14785
• test(testoperator): make append tests more robust by @sgrebnov in #14817
• fix(runtime,data_components): wait for change-data-capture sources to record their positions before shutdown closes the accelerations (fixes #14523) by @grokspice in #14702
• fix(graphql): detect non-JSON responses and treat gateway errors as retryable by @lukekim in #14781
• fix(runtime): require keys for s3_auth: key, honor gs:// state location params, and leave newer rate-control state alone by @lukekim in #14584
• build(deps): bump hickory-resolver from 0.26.1 to 0.26.2 by @dependabot in #14774
• fix(listing): skip zero-byte objects (S3 folder markers) in format-selected listings by @sgrebnov in #14822
• Avoid unnecessary repartitioning in indexed dynamic-filter joins by @bjchambers in #14799
• feat(cayenne): keep one row per primary key automatically and deprecate on_conflict by @bjchambers in #14726
• fix(object-store): stop reading a response body at its first error by @phillipleblanc in #14831
• fix(ci): use byte-order collation in the benchmark Postgres container (refs #14815) by @sgrebnov in #14836
• fix(listing): keep a partition predicate as a residual filter on the _location fast path by @grokspice in #14790
• fix(ci): search only the unixodbc keg, not all of Homebrew's lib, in the macOS release builds by @grokspice in #14846
• fix(federation): keep date, timestamp and interval values local on SQLite reached through ADBC or ODBC (fixes #14753) by @claudespice in #14840
• chore: pin the fork at spiceai/datafusion#249 and guard the DataFusion fixes backported from 54 by @krinart in #14800
• Revert "fix(sqlite, duckdb): keep a decimal AVG, and on SQLite a decimal SUM, out of the federated plan (fixes #14492) (#14670)" by @krinart in #14825
• perf(runtime): skip EnsureRequirements while planning a point lookup by @lukekim in #14807
• fix(testoperator): report CH-benCH queries with no rows on either side as vacuous, not as matches by @lukekim in #14810
• feat(acceleration): make Cayenne the default acceleration engine by @phillipleblanc in #14837
• fix(cayenne): keep a join's LIMIT when the oversized-join rewrite makes it a sort-merge join by @lukekim in #14805
• test(bigquery): exclude subtrees an empty join build side never runs from the corpus job count (fixes #14848) by @sgrebnov in #14849
• fix(cayenne): fail contradictory write settings once, and order NULL times below every time by @bjchambers in #14847
• fix(runtime): serve an existing acceleration while its source is unavailable (fixes #14610) by @bjchambers in #14624
• Prune object-store file listing by metadata columns (#14264) by @Jeadie in #14303
• fix(vortex): fold partition values into the file-pruning predicate; test null-equal joins under mode:file by @lukekim in #14797
• feat(rate-control): adaptive, bounded and cluster-coordinated HTTP rate controls by @Jeadie in #14143
• fix(sql): align date_part('dow') with EXTRACT(dow) Sunday=0 by @lukekim in #14796
• fix(runtime): carry API-key principal into MCP tools/call by @lukekim in #14828
• perf(cayenne): hold maintained-aggregate retraction state in a compact shared index by @lukekim in #14762
• fix(deps): keep a hash join's order once it has a dynamic filter, so vector_search plans again by @Jeadie in #14857
• fix(cayenne): fence schema statistics and control maintenance tests by @bjchambers in #14856
• fix(github): retry nested GraphQL pages in place and fail incomplete nested connections by @sgrebnov in #14862
• fix(cayenne): load an append dataset that has retention_sql and orders versions by time by @phillipleblanc in #14878
• fix(cdc): Cayenne replication needs only a primary key, not on_conflict by @phillipleblanc in #14880
• feat(connector-huggingface): Hugging Face datasets data connector (hf://datasets/...) by @lukekim in #14877
• fix(iceberg): Iceberg REST clients read the real table or get an error, never an empty one by @lukekim in #14588

Full Changelog: https://github.com/spiceai/spiceai/compare/v2.3.2...v2.4.0-rc.1

Spice v2.1.4 is now available! 🛠️

Spice v2.1.4 is a patch release that improves DuckDB acceleration and runtime stability: datasets with UTC timestamp columns, such as Iceberg tables, now load successfully when accelerated with DuckDB, retention policies run reliably, and the runtime is more resilient under heavy query load.

What's New in v2.1.4​

DuckDB Acceleration Works with Iceberg Timestamps​

A dataset with a UTC timestamp column — for example, any Iceberg timestamptz column — could previously fail to load when accelerated with DuckDB, leaving the dataset unhealthy and unqueryable. These datasets now load and become ready normally, with no configuration changes needed.

Reliable Retention Policies on DuckDB​

Retention policies now apply cleanly on DuckDB-accelerated datasets, including policies that combine a time window with an additional condition. Expired rows are evicted on every retention interval, keeping accelerated data fresh and storage bounded.

Improved Runtime Stability​

The runtime is now more robust when queries are cancelled — whether by a client disconnecting, a timeout, or a new refresh superseding an in-flight read. A rare crash in this path has been eliminated.

Contributors​

• @phillipleblanc
• @peasee

Breaking Changes​

No breaking changes.

Cookbook Updates​

No new cookbook recipes.

The Spice Cookbook includes more than 100 recipes to help you get started with Spice quickly and easily.

Upgrading​

To upgrade to v2.1.4, use one of the following methods:

CLI:

spice upgrade

Homebrew:

brew upgrade spiceai/spiceai/spice

Docker:

Pull the spiceai/spiceai:2.1.4 image:

docker pull spiceai/spiceai:2.1.4

For available tags, see DockerHub.

Helm:

helm repo update
helm upgrade spiceai spiceai/spiceai --version 2.1.4

AWS Marketplace:

Spice is available in the AWS Marketplace.

What's Changed​

Changelog​

• chore(deps): bump datafusion and table-providers for the DuckDB timezone and retention fixes (2.1 backport) by @phillipleblanc in #12546
• fix(deps): bump the Vortex pin to pick up the task-cancellation fix by @phillipleblanc in #12547

*Full Changelog: https://github.com/spiceai/spiceai/compare/v2.1.3...v2.1.4

Spice v2.1.3 is now available! 🛠️

Spice v2.1.3 is a patch release focused on resource efficiency: explicit CPU sizing with the new runtime.cpu.cores setting, more query memory for Cayenne deployments, and improved memory and crash diagnostics. It also fixes Cayenne acceleration of Iceberg datasets with timestamptz columns and restores WHERE filters on federated LEFT/RIGHT JOIN queries.

What's New in v2.1.3​

CPU Sizing with runtime.cpu.cores​

The new runtime.cpu.cores setting controls how many cores the runtime targets. Thread pools, query partitioning, and accelerator concurrency are all derived from it.

runtime:
  cpu:
    cores: 4 # `auto` (default) detects. Accepts 4, 3.5, 3500m

Also available as --cpu-cores and SPICE_CPU_CORES (precedence: flag > environment > Spicepod).

This is most useful on large, shared nodes. A pod that sets resources.requests.cpu without a CPU limit exposes no cgroup quota, so the runtime sizes itself for every core on the node rather than its allocated share. Setting the entitlement aligns parallelism and memory footprint with the CPU the pod actually receives.

The effective value, its source, and the derived sizing are logged at startup and exported as the spiced_cpu_budget_cores gauge.

More Query Memory for Cayenne Deployments​

The Cayenne compaction memory pool is now reserved only for accelerations that can compact into it: file mode with a small-write refresh profile. Other deployments, including refresh_mode: full, keep the full memory limit available to queries — up to 6.4 GiB on a 32 GiB limit, with no configuration change.

Memory budgets are now derived from the process's own cgroup limit rather than total host memory.

Diagnostics​

Three new gauges report memory in use: query_memory_pool_used_bytes, cayenne_compaction_memory_pool_used_bytes, and process_resident_memory_bytes.

Memory pool refusals now return ResourcesExhausted and HTTP 503, distinguishing them from query errors.

Fatal native signals (SIGSEGV, SIGBUS, SIGILL, SIGFPE) report the signal, faulting address, and thread before exit, so a crash can be diagnosed from logs.

Fixed a bug where setting runtime.task_history.enabled: false also disabled every query metric — query_duration_ms, query_execution_duration_ms, query_executions, query_failures, query_returned_rows, and query_returned_bytes. These are now reported regardless of the task history setting.

Cayenne Acceleration of Iceberg timestamptz Columns​

Accelerating an Iceberg dataset with a timestamptz column using the Cayenne engine previously failed during the refresh write with an error resolving the time zone +00:00. Iceberg maps every timestamptz column to the fixed-offset Arrow time zone +00:00, which the file writer could not resolve when building column statistics. Fixed-offset time zones (±HH:MM, ±HHMM, and ±HH) are now resolved wherever time zones are handled, so these datasets accelerate correctly.

Federated Outer Join Filters​

A federated query combining a LEFT JOIN with a WHERE filter on the left table previously returned all rows instead of the filtered rows: when the query was pushed down to the data source or accelerator, the filter was folded into the JOIN ON clause, where it no longer filters the left side (RIGHT JOIN was affected symmetrically). Filters now stay on the side of the join they came from, so these queries return the correct rows.

Contributors​

• @lukekim
• @sgrebnov
• @phillipleblanc
• @bjchambers
• @Jeadie

Breaking Changes​

No breaking changes.

Cookbook Updates​

No new cookbook recipes.

The Spice Cookbook includes more than 100 recipes to help you get started with Spice quickly and easily.

Upgrading​

To upgrade to v2.1.3, use one of the following methods:

CLI:

spice upgrade

Homebrew:

brew upgrade spiceai/spiceai/spice

Docker:

Pull the spiceai/spiceai:2.1.3 image:

docker pull spiceai/spiceai:2.1.3

For available tags, see DockerHub.

Helm:

helm repo update
helm upgrade spiceai spiceai/spiceai --version 2.1.3

AWS Marketplace:

Spice is available in the AWS Marketplace.

What's Changed​

Changelog​

• feat(runtime): size every CPU-derived pool from the CPU entitlement by @bjchambers in #12276
• fix(runtime): carve the Cayenne compaction memory pool only when a dataset can compact into it by @sgrebnov in #12326
• fix(runtime): size memory budgets from the process's own cgroup limit by @lukekim in #12263
• fix(telemetry): read the cgroup CPU quota along the whole cgroup path by @sgrebnov in #12300
• feat(runtime): expose the memory numbers that explain an OOM as gauges by @lukekim in #12195
• fix(runtime): report a memory-pool refusal as ResourcesExhausted and answer it with 503 by @sgrebnov in #12289
• fix(cayenne): the write-concurrency raise must respect the memory brake by @lukekim in #12317
• feat(spiced): report fatal signals before exit by @sgrebnov in #12334
• fix: report query metrics when task history is disabled by @sgrebnov in #12227
• chore(deps): repoint vortex at the 2.1 fixed-offset timezone fix by @phillipleblanc in #12455
• chore(deps): bump datafusion rev for outer-join unparser fix by @Jeadie in #12460

*Full Changelog: https://github.com/spiceai/spiceai/compare/v2.1.2...v2.1.3

Spice v2.1.2 is now available! 🛠️

Spice v2.1.2 is a patch release focused on improving the DuckDB data accelerator. It upgrades the DuckDB engine to v1.5.5 and introduces the on_full_refresh parameter, giving file-mode DuckDB accelerations compact, predictable disk usage across repeated full refreshes.

What's New in v2.1.2​

Bounded DuckDB Acceleration File Growth with on_full_refresh​

File-mode DuckDB accelerations using refresh_mode: full now reclaim disk space on every refresh, keeping the database file compact and disk usage predictable for long-running deployments. Each full refresh bulk-loads a fresh copy of the data, and the new on_full_refresh modes ensure the space held by prior copies is returned rather than accumulating in the file.

The new on_full_refresh acceleration parameter controls how disk space is reclaimed after each full refresh:

acceleration:
  engine: duckdb
  mode: file
  refresh_mode: full
  params:
    duckdb_file: /data/shared.duckdb
    on_full_refresh: replace_file # default: reuse_file

• reuse_file (default): Existing behavior — refresh into the existing database file.
• replace_file: Each full refresh streams data into a fresh staging database file, carries over every other object sharing the file (other datasets' tables, views, indexes, and Spice metadata), checkpoints it, and atomically replaces the configured file. Queries are never interrupted — in-flight queries drain against the old file while new queries see the new file — and space is fully reclaimed on every refresh.
• checkpoint_file: After each refresh, run a CHECKPOINT in place, escalating to FORCE CHECKPOINT when concurrent transactions block the plain attempt (waiting for in-flight transactions; never aborting them).

DuckDB 1.5.5​

The DuckDB engine is upgraded from v1.5.3 to v1.5.5, bringing the latest upstream stability fixes.

Contributors​

• @lukekim
• @sgrebnov
• @bjchambers
• @krinart

Breaking Changes​

No breaking changes.

Cookbook Updates​

No new cookbook recipes.

The Spice Cookbook includes more than 100 recipes to help you get started with Spice quickly and easily.

Upgrading​

To upgrade to v2.1.2, use one of the following methods:

CLI:

spice upgrade

Homebrew:

brew upgrade spiceai/spiceai/spice

Docker:

Pull the spiceai/spiceai:2.1.2 image:

docker pull spiceai/spiceai:2.1.2

For available tags, see DockerHub.

Helm:

helm repo update
helm upgrade spiceai spiceai/spiceai --version 2.1.2

AWS Marketplace:

Spice is available in the AWS Marketplace.

What's Changed​

Changelog​

• feat(duckdb): on_full_refresh: replace_file — full refresh into a new database file, atomically replaced by @lukekim in #12135
• feat(duckdb): 'on_full_refresh: checkpoint_file' to bound acceleration file growth by @sgrebnov in #12139

*Full Changelog: https://github.com/spiceai/spiceai/compare/v2.1.1...v2.1.2

Announcing the release of Spice v1.11.5! 🛠️

Spice v1.11.5 is a patch release improving on_zero_results: use_source fallback performance, Delta Lake timestamp predicate data skipping, S3 Parquet read performance, PostgreSQL partitioned table support, Cayenne target file size handling, and preparing the CLI for v2.0 runtime upgrades.

What's New in v1.11.5​

on_zero_results: use_source Fallback Performance Improvement​

Improved the on_zero_results: use_source fallback path to run DataFusion's physical optimizer on the federated scan plan (#9927). The fallback path now runs SessionState::physical_optimizers() rules on the federated scan plan before execution, enabling parallel file group scanning and other optimizations. This results in significantly faster fallback queries on multi-core machines, particularly for file-based data sources like Delta Lake.

Delta Lake: Improved Data Skipping for >= Timestamp Predicates​

Delta Lake table scans with >= timestamp filters now correctly prune files that do not match the predicate (#9932), improving query performance through more effective data skipping (file-level pruning).

PostgreSQL: Partitioned Tables Support​

The PostgreSQL data connector now supports partitioned tables (#9997) for both federated and accelerated queries.

S3 Parquet Read Performance Improvement​

Improved parquet read performance from S3 and other object stores (#10064), particularly for tables with many columns. Column data ranges are now coalesced into fewer, larger requests instead of being fetched individually, reducing the number of HTTP round-trips.

Cayenne: Ensure Target File Size is Respected​

The Cayenne accelerator now correctly respects the configured target file size (#10071). Previously, Cayenne could produce many small, fragmented Vortex files; with this fix, files are written at the expected target size, improving storage efficiency and query performance.

CLI: Support for v2.0 Runtime Upgrades​

The Spice CLI can now upgrade to v2.0 runtime versions. This enables upgrading to v2.0 release candidates and, once released, the v2.0 stable runtime.

spice upgrade v2.0.0-rc.1

Running spice upgrade without a version will upgrade to the latest stable version, including v2.0 once released.

Note: Native Windows runtime builds will no longer be provided in v2.0. Use WSL for local development instead.

Contributors​

• @sgrebnov
• @lukekim
• @krinart

Breaking Changes​

No breaking changes.

Cookbook Updates​

No new cookbook recipes.

The Spice Cookbook includes 86 recipes to help you get started with Spice quickly and easily.

Upgrading​

To upgrade to v1.11.5, use one of the following methods:

CLI:

spice upgrade

Homebrew:

brew upgrade spiceai/spiceai/spice

Docker:

Pull the spiceai/spiceai:1.11.5 image:

docker pull spiceai/spiceai:1.11.5

For available tags, see DockerHub.

Helm:

helm repo update
helm upgrade spiceai spiceai/spiceai --version 1.11.5

AWS Marketplace:

Spice is available in the AWS Marketplace.

What's Changed​

Changelog​

• fix(runtime): Run physical optimizer on FallbackOnZeroResultsScanExec fallback plan by @sgrebnov in #9927
• fix(delta_lake): Fix data skipping for >= timestamp predicates by @sgrebnov in #9932
• fix(PostgreSQL): Fix schema discovery for PostgreSQL partitioned tables by @sgrebnov in #9997
• fix(cli): Skip models variant download for v2+ in upgrade/install by @lukekim and @sgrebnov in #10052
• perf(s3): Improve Parquet read performance by @sgrebnov in #10064
• fix(cayenne): Ensure Cayenne respects target file size by @krinart in #10071

Full Changelog: https://github.com/spiceai/spiceai/compare/v1.11.4...v1.11.5

Announcing the release of Spice v1.11.4! ⚡

Spice v1.11.4 is a patch release improving S3 metadata column query robustness and enabling on_zero_results: use_source for accelerated views.

What's New in v1.11.4​

Accelerated Views: on_zero_results: use_source Support​

Accelerated views now support the on_zero_results: use_source configuration (#9699). Previously, accelerated views only supported on_zero_results: return_empty, which returned an empty result set when the accelerated data contained no matching rows. With this change, views can fall back to querying the source data when the accelerated query returns zero results, matching the behavior already available for accelerated datasets.

Example configuration:

views:
  - name: sales_summary
    sql: |
      SELECT region, SUM(amount) as total
      FROM sales
      GROUP BY region
    acceleration:
      enabled: true
      on_zero_results: use_source

How the Fallback Works​

When an accelerated view is configured with on_zero_results: use_source, the following happens at query time:

1. The accelerated store is queried first. The query runs against the view's accelerated data (e.g., Spice Cayenne, Arrow, DuckDB, or SQLite).

2. If the accelerated query returns zero rows, the runtime falls back to re-executing the view's SQL query against the datasets it references.

3. Referenced datasets are queried according to their own configuration. The view's SQL is re-executed against each referenced dataset as it is configured. This means:

   • If a referenced dataset is accelerated, the query hits that dataset's accelerated store — not the raw data source.
   • If a referenced dataset is accelerated with on_zero_results: use_source and its accelerated store also returns zero rows, it will independently fall back to its own federated data source (e.g., Postgres, S3, etc.).
   • If a referenced dataset is federated (not accelerated), the query goes directly to the data source.

This means the fallback can chain through multiple layers: first the view's acceleration, then each referenced dataset's acceleration, and finally the original data source — each layer independently applying its own on_zero_results behavior.

Example: Multi-layer fallback

datasets:
  - from: postgres:orders
    name: orders
    acceleration:
      enabled: true
      refresh_sql: "SELECT * FROM orders WHERE created_at > now() - interval '7 days'"
      on_zero_results: use_source  # Falls back to Postgres if accelerated data has no matches

views:
  - name: recent_orders_summary
    sql: |
      SELECT status, COUNT(*) as order_count
      FROM orders
      GROUP BY status
    acceleration:
      enabled: true
      on_zero_results: use_source  # Falls back to re-running the SQL against referenced datasets

In this example, a query like SELECT * FROM recent_orders_summary WHERE status = 'cancelled' follows this path:

1. Queries recent_orders_summary in the view's accelerated store (DuckDB/SQLite).
2. If zero rows are returned, re-executes SELECT status, COUNT(*) ... FROM orders GROUP BY status against the orders dataset.
3. Since orders is accelerated, the query hits the orders accelerated store.
4. If orders also returns zero rows (e.g., the refresh_sql excluded cancelled orders), it falls back to querying Postgres directly.

S3 Data Connector: More Robust Metadata Column Handling​

Improved the robustness of metadata column (location, last_modified, size) handling for S3 datasets. Building on the v1.11.3 release, this update addresses an additional edge case where the query optimizer's projection swap could cause an index out of bounds panic when metadata columns are referenced in projections with filters or scalar functions.

Contributors​

• @sgrebnov
• @krinart

Breaking Changes​

No breaking changes.

Cookbook Updates​

No new cookbook recipes.

The Spice Cookbook includes 86 recipes to help you get started with Spice quickly and easily.

Upgrading​

To upgrade to v1.11.4, use one of the following methods:

CLI:

spice upgrade

Homebrew:

brew upgrade spiceai/spiceai/spice

Docker:

Pull the spiceai/spiceai:1.11.4 image:

docker pull spiceai/spiceai:1.11.4

For available tags, see DockerHub.

Helm:

helm repo update
helm upgrade spiceai spiceai/spiceai --version 1.11.4

AWS Marketplace:

Spice is available in the AWS Marketplace.

What's Changed​

Changelog​

• fix(s3): Make metadata column handling more robust by @sgrebnov in #9714
• feat(views): Enable on_zero_results: use_source for accelerated views by @krinart in #9699

Full Changelog: https://github.com/spiceai/spiceai/compare/v1.11.3...v1.11.4

Announcing the release of Spice v2.0-rc.1! 🚀

v2.0.0-rc.1 is the first release candidate for early testing of v2.0.

Highlights in this release candidate include:

• Active-Active Highly-Available Distributed Query that is object-store-native and built on Apache Ballista, with dynamic cluster sizing, distributed ingestion, and cluster observability
• Spice Cayenne RC with staged append writes, file-based retention deletes, composite partitioning, and distributed ingestion
• DataFusion v52.2.0 Upgrade with sort pushdown, a new merge join, and dynamic filters
• DDL Support for CREATE TABLE and DROP TABLE via SQL for Iceberg and Cayenne catalogs
• DuckLake Catalog & Data Connector for lakehouse-style data management
• GCS Data Connector (Alpha) for Google Cloud Storage
• Rust CLI Rewrite for a unified single-binary experience
• Dependency upgrades including DuckDB v1.4.4, delta_kernel v0.18.2, and mistral.rs

Spice v2.0 includes several breaking changes. Review the breaking changes section before upgrading.

Distribution Changes​

AI/ML support including local LLM/ML model and hosted LLM inference is now included in the default Spice build and image. The separate models build variant has been removed.

With models now included by default, the data-only distribution (without AI/ML support) is only published in nightly builds. Official production-ready data-only distributions are available exclusively through Spice Cloud and the Enterprise release.

A new Network Attached Storage (NAS) distribution with built-in SMB and NFS data connector support is also now available in nightly builds and with Spice.ai Enterprise.

For more details, see the Distributions documentation.

What's New in v2.0.0-rc.1​

Active-Active HA Distributed Query​

Distributed Query exits Beta with active-active highly-available object-store-based distributed query.

Distributed query supports two execution modes:

• Synchronous: Queries for accelerated datasets are distributed across executors and results are streamed back in real-time. Non-accelerated datasets execute only on the scheduler. Best for interactive queries where low latency is critical.
• Asynchronous: Queries are submitted via the new HTTP-only /v1/queries API and results are materialized to object storage for later retrieval. Best for long-running analytical workloads, batch processing, and non-accelerated datasets in distributed mode.

Key improvements:

• Dynamic Cluster Sizing: The query planner automatically adjusts parallelism based on the number of active executors in the cluster, ensuring optimal resource utilization as nodes are added or removed.
• Distributed Ingestion: Data ingestion for partitioned accelerated tables is now distributed across executor nodes, enabling higher throughput and parallel data loading in cluster mode. Regular (non-partitioned) accelerated tables do not distribute ingestion loads.
• Synchronous Execution on Scheduler: /v1/sql and FlightSQL queries now execute synchronously on the scheduler when appropriate, reducing inter-node overhead for queries that don't benefit from distribution.
• Faster Failure Detection: Executor heartbeat timeout reduced from 180s to 30s, enabling the cluster to quickly detect and respond to executor failures.
• Cluster Observability: New metrics and Grafana dashboard for monitoring distributed query clusters.

Spice Cayenne Improvements​

The Spice Cayenne data accelerator exits Beta with significant reliability and performance improvements:

• Staged Append Writes: WAL-based staged append writes prevent partial writes and data loss on stream errors. Batches are written to a WAL file before being committed, ensuring atomicity.
• File-Based Retention Deletes: Time-based retention now supports file-level deletes for both position-based and primary-key tables, reducing I/O overhead compared to row-level deletion.
• Multiple Partition Expressions: Support for composite partitioning with partition_by: [col1, col2] using hierarchical path-like keys (e.g., 2025/10/15).
• Distributed Ingestion: Cayenne catalog now supports distributed ingestion across executor nodes in cluster mode, including UPDATE operations.
• Improved Robustness: Fixed CDC edge case where DELETE + UPSERT sequences could produce duplicate primary keys across protected snapshots. Improved upsert handling during runtime restarts.

DataFusion v52.2.0 Upgrade​

Apache DataFusion has been upgraded to v52.2.0, bringing significant performance improvements, new query features, and enhanced extensibility.

Performance Improvements:

• Faster CASE Expressions: Lookup-table-based evaluation for certain CASE expressions avoids repeated evaluation, accelerating common ETL patterns
• MIN/MAX Aggregate Dynamic Filters: Queries with MIN/MAX aggregates now create dynamic filters during scan to prune files and rows as tighter bounds are discovered during execution
• New Merge Join: Rewritten sort-merge join (SMJ) operator with speedups of three orders of magnitude in pathological cases (e.g., TPC-H Q21: minutes → milliseconds)
• Caching Improvements: New statistics cache for file metadata avoids repeatedly recalculating statistics, significantly improving planning time. A prefix-aware list-files cache accelerates evaluating partition predicates for Hive partitioned tables
• Improved Hash Join Filter Pushdown: Build-side hash map contents are now passed dynamically to probe-side scans for pruning files, row groups, and individual rows

Major Features:

• Sort Pushdown to Scans: Sorts are pushed into data sources, enabling ~30x performance improvement on pre-sorted data with top-K queries. Parquet scans now reverse row group order for DESC queries on ASC-sorted files
• TableProvider supports DELETE and UPDATE: New hooks for DELETE and UPDATE statements in the TableProvider trait, enabling Iceberg and Cayenne connectors to implement SQL DELETE and UPDATE operations
• More Extensible SQL Planning: New RelationPlanner API for extending SQL planning for FROM clauses, enabling support for vendor-specific SQL dialects

DDL Support for Iceberg and Cayenne​

SQL Schema Management: Spice now supports CREATE TABLE and DROP TABLE DDL operations for Iceberg and Cayenne catalogs via FlightSQL and the /v1/sql API. DML validation has been updated for catalog-level writability.

DuckLake Catalog & Data Connector​

Lakehouse-Style Data Management: New DuckLake catalog and data connector enable lakehouse-style data management with DuckDB as the metadata catalog and object storage for data files. DuckLake provides ACID transactions, time travel, and schema evolution on top of Parquet files.

GCS Data Connector (Alpha)​

Google Cloud Storage Support: New Google Cloud Storage data connector enables federated queries against data stored in GCS buckets, with Iceberg table support.

Rust CLI Rewrite​

Unified Single-Binary Experience: The Spice CLI has been completely rewritten from Go to Rust, eliminating the Go dependency and providing a single spice binary built from the same codebase as spiced. This improves startup performance, reduces distribution size, and ensures consistent behavior between CLI and runtime.

Key Features:

• Full Feature Parity: All 27+ CLI commands re-implemented in Rust with identical behavior
• New spice query Command: Interactive REPL for async queries via the /v1/queries API with multi-line SQL input, spinner progress indicator, Ctrl+C cancellation, and partial query ID matching
• --output=json Flag: Machine-readable JSON output for CLI commands, enabling scripting and automation
• spice login --output: New output modes (env, json, keychain) for flexible credential management
• spice cloud metrics: New command for Spice Cloud deployment metrics

Models Included by Default​

Local LLM/ML model inference (via mistral.rs) is now included in the default Spice build. The separate models build variant has been removed. This simplifies installation and ensures all users have access to local AI inference capabilities.

Error Propagation for Dataset and Model Status APIs​

The /v1/datasets and /v1/models APIs now return structured error information when a component is in an Error state. The ?status=true query parameter must be passed to retrieve the real-time component status, including the error state and details. Previously, the status field only indicated Error with no further detail. Now, two new fields are included when ?status=true is specified:

• error: A structured object with category, type, and code fields for programmatic error handling (e.g. { "category": "dataset", "type": "auth", "code": "dataset.auth" }).
• error_message: A human-readable description of why the component entered an error state.

These fields are only present when ?status=true is passed and the component is in an error state.

Example /v1/datasets?status=true response:

[
  {
    "from": "postgres:syncs",
    "name": "daily_journal",
    "replication_enabled": false,
    "acceleration_enabled": true,
    "status": "Ready"
  },
  {
    "from": "databricks:hive_metastore.default.messages",
    "name": "messages",
    "replication_enabled": false,
    "acceleration_enabled": true,
    "status": "Error",
    "error": {
      "category": "dataset",
      "type": "auth",
      "code": "dataset.auth"
    },
    "error_message": "Unable to authenticate with datasource credentials"
  }
]

The spice datasets and spice models CLI commands now include an ERROR column that displays the error message for any component in an error state.

Additional Dependency Upgrades​

Other Improvements​

• Spicepod v2 Support: Spicepods now support version v2, and spice init generates spicepod.yaml files with version: v2 by default while maintaining backward compatibility for existing v1 spicepods.
• x.ai Models: x.ai models now exclusively use the /v1/responses endpoint with rate limiting support.
• HuggingFace Chat Templates: Added support for chat templates in HuggingFace model configurations.
• Databricks SQL Dialect: Added Databricks SQL dialect for DataFusion unparser, improving federation query generation.
• Snowflake: Added snowflake_private_key parameter for key-pair authentication.
• Acceleration Metrics: New rows_written, bytes_written, and dataset_acceleration_size_bytes metrics for acceleration refresh ingestion.
• Refresh SQL UDFs: Core scalar UDFs are now enabled in refresh SQL expressions.
• FlightSQL: Fixed TLS connection handling for grpc+tls:// endpoints with custom CA certificate support.
• FlightSQL: Fixed schema consistency by expanding view types and verifying field names.
• Hash Index: Fixed query correctness when hash index is used with additional filters.
• Results Cache: Fixed schema preservation for empty query results.
• Query Nullability: Reconciled execution stream nullability with logical plan schema.
• Schema Evolution: Graceful handling of schema evolution mismatch errors during data refresh.
• Internal YAML Parser: Replaced deprecated serde_yaml with an internal YAML implementation.

Spicepod v1 to v2 Changes​

Spicepod v2 introduces configuration improvements while maintaining backward compatibility with v1. Existing v1 spicepods continue to work — deprecated fields are automatically migrated at load time.

Version support:

Configuration changes:

New v2 fields:

• runtime.ready_state — Controls when the runtime reports ready (on_load default, or on_registration).
• runtime.flight.do_put_rate_limit_enabled — Enable/disable FlightSQL DoPut rate limiting (default: true).
• runtime.query.spill_compression — Compression for query spill files (e.g., lz4_frame).
• runtime.scheduler.partition_management — Configure partition assignment interval, limits, and timeouts for distributed mode.
• runtime.caching.sql_results.stale_while_revalidate_ttl — Serve stale cached results while revalidating in the background.
• runtime.caching.sql_results.encoding — Cache entry compression (e.g., zstd).
• catalog.access: read_write_create — New access mode for catalogs that support DDL operations.

Migration note: When both the deprecated v1 field and its v2 equivalent are set, the v2 field takes priority.

Contributors​

• @avishafir
• @cluster2600
• @Jeadie
• @krinart
• @lukekim
• @peasee
• @phillipleblanc
• @sgrebnov

Breaking Changes​

• Cayenne and Distributed Query exit Beta: Beta warnings have been removed from documentation and code. Both features are now considered GA-ready.
• Models included by default: The separate models build variant has been removed. Local LLM inference is now always included.
• Spicepod version defaults to v2: New spicepods created with spice init now default to version: v2. Existing v1 spicepods remain supported, and v1beta1 is no longer accepted.
• Windows native builds removed: Native Windows builds are no longer provided. Use WSL for local development instead.
• Metric renames: accelerated_refresh metrics renamed to acceleration_refresh for consistency. last_refresh_time gauge renamed to include milliseconds unit.
• Caching config renamed: ResultsCache replaced with SQLResultsCacheConfig in configuration.
• DuckDB parameter rename: partitioned_write_flush_threshold renamed to partitioned_write_flush_threshold_rows.
• v1/search API: The /v1/search API now always returns an array in matches, even for single results.
• x.ai model endpoint: x.ai models now exclusively use the /v1/responses endpoint.
• Error messages: Error messages across S3 Vectors, ScyllaDB, Snowflake, ClickHouse, and other components have been refactored for clarity and consistency.

Cookbook Updates​

New and updated Spice Cookbook recipes:

• Async Queries: Submit long-running queries asynchronously and retrieve results later.
• DuckLake Catalog Connector: Use DuckLake for lakehouse-style data management with ACID transactions and time travel.

The Spice Cookbook includes 88 recipes to help you get started with Spice quickly and easily.

Upgrading​

To upgrade to v2.0.0-rc.1, use one of the following methods:

CLI:

spice upgrade

Homebrew:

brew upgrade spiceai/spiceai/spice

Docker:

Pull the spiceai/spiceai:2.0.0-rc.1 image:

docker pull spiceai/spiceai:2.0.0-rc.1

For available tags, see DockerHub.

Helm:

helm repo update
helm upgrade spiceai spiceai/spiceai --version 2.0.0-rc.1

AWS Marketplace:

Spice is available in the AWS Marketplace.

What's Changed​

Changelog​

• Add TPC-DS integration tests with S3 source and PostgreSQL acceleration by @phillipleblanc in #9006
• fix(tests): fix flaky/slow/failing unit tests by @phillipleblanc in #9009
• fix: Update benchmark snapshots for DF51 upgrade by @app/github-actions in #9008
• fix: add feature gate to rrf TEST_EMBEDDING_MODEL by @phillipleblanc in #9017
• fix: features check by @phillipleblanc in #9014
• fix: Enable Cayenne acceleration snapshots by @lukekim in #9020
• URL table support by @lukekim in #9018
• ScyllaDB key filter by @lukekim in #8997
• fix: Schema mismatch when using column projection with HTTP caching by @phillipleblanc in #9021
• Add more tests for HTTP caching with columns selection by @sgrebnov in #9025
• HTTP cache snapshots: default to time_interval and fix snapshots_creation_policy: on_change by @sgrebnov in #9026
• Fix duplicate snapshot creation on startup by @sgrebnov in #9029
• Add ScyllaDB and SMB to the README table by @krinart in #9034
• Remove waiting for runtime to be ready before creating snapshot by @krinart in #9033
• Fix snapshot on_change policy to skip when no writes occurred by @sgrebnov in #9028
• Release notes for release release/1.11.0-rc.2 by @krinart in #9016
• ci: use arduino/setup-protoc for official protobuf compiler by @phillipleblanc in #9036
• ci: install unzip on aarch64 runner for arduino/setup-protoc by @phillipleblanc in #9038
• fix: don't fail release if upload to minio fails by @phillipleblanc in #9039
• Add missing protoc step to setup-cc action by @krinart in #9041
• fix: Update Search integration test snapshots by @app/github-actions in #9013
• Fix formula_1 and codebase_community in bird-bench by @Jeadie in #9000
• Cayenne S3 Express One Zone improvements by @lukekim in #9015
• Add zlib1g-dev to CI by @lukekim in #9052
• Improve validation and logging for hash indexes by @lukekim in #9047
• Upgrade Vortex with CASE-WHEN by @lukekim in #9051
• x.ai models now exclusively use /v1/responses endpoint by @lukekim in #9400
• Improvements for snapshot schema comparison by @krinart in #9401
• v2.0 breaking changes by @lukekim in #9233
• Create PartitionManagementTask for scheduler to update accelerated table partition assignments by @Jeadie in #9378
• refactor(Cayenne): route all write orchestration through CayenneDataSink by @sgrebnov in #9402
• Refactor benchmark to use QueryExecutor trait by @Jeadie in #9418
• feat: Add spidapter build and release workflow by @peasee in #9427
• Testoperator: add support for api-key when connecting to external spice instance by @sgrebnov in #9421
• Initial implementation of Ducklake catalog & data connectors by @lukekim in #9083
• Require aws_lc_rs since jsonwebtoken upgrade by @Jeadie in #9426
• feat: Add spidapter tool by @peasee in #9425
• Add release notes for 1.11.2 patch release by @sgrebnov in #9430
• feat(spidapter): integrate system-adapter-protocol with SCP provisioning by @phillipleblanc in #9434
• Add DuckLake TPCH E2E workflow and federated Spicepod configuration by @lukekim in #9431
• fix(spidapter): use Flight handshake auth instead of x-api-key header by @phillipleblanc in #9435
• [spidapter] Keep only what sparks joy by @Jeadie in #9439
• Refactor binary operator balancing by @Jeadie in #9424
• feat: Add Iceberg DDL support (CREATE TABLE / DROP TABLE) for default catalog override by @phillipleblanc in #9440
• Fix Flight SQL schema consistency: expand view types and verify field names by @sgrebnov in #9438
• Update spidapter for new system-adapter-protocol by @sgrebnov in #9442
• docs: fix typos and syntax errors in style guide and error handling docs by @cluster2600 in #9445
• Add acceleration refresh ingestion metrics (rows_written, bytes_written) by @phillipleblanc in #9461
• Refactor(Cayenne): Replace CatalogError and string based errors with Snafu errors by @sgrebnov in #9403
• Replace deprecated claude-3-5-haiku-latest with claude-haiku-4-5 by @Jeadie in #9492
• Fix #9481: Preserve schema in results cache for empty query results by @phillipleblanc in #9485
• Fix partition by serializing by @Jeadie in #9474
• query: reconcile execution stream nullability with logical plan schema by @phillipleblanc in #9486
• initial spice-cloud-client crate and spice cloud metrics --app <app-name>. by @Jeadie in #9480
• feat: Return dataset error message in datasets API by @peasee in #9487
• Spicebench by @lukekim in #9447
• build(deps): consolidate dependabot dependency updates by @phillipleblanc in #9504
• fix(cluster): route non-partitioned accelerated tables in distributed mode by @phillipleblanc in #9508
• Enable core scalar UDFs in refresh SQL by @sgrebnov in #9502
• Fix metrics in Spidapter again by @Jeadie in #9497
• fix(cluster): tolerate Completed->status propagation race in distributed query handle by @phillipleblanc in #9510
• feat: Support distributed ingestion in cayenne catalog by @peasee in #9506
• Fix Cayenne duplicate primary keys after DELETE + UPSERT CDC sequences by @krinart in #9494
• fix(cluster): rewrite table scans inside subqueries for distributed execution by @phillipleblanc in #9518
• fix: Set catalog mode to readwritecreate in spidapter by @peasee in #9519
• Upgrade AWS SDK crates & set APN user-agent in AWS SDK credential bridge by @lukekim in #8328
• feat(runtime): add runtime ready_state on_registration semantics by @lukekim in #9522
• fix: Add spidapter post-setup retries by @peasee in #9526
• Make partition discovery more robust and make initialization non-blocking by @sgrebnov in #9499
• Make lint-rust-fix support targeted packages and features by @Jeadie in #9511
• Handle new Cloud SCP API by @Jeadie in #9532
• Refactor and simplify streaming benchmarks by @krinart in #9405
• fix: ensure spidapter only increments attempts on failures by @peasee in #9534
• feat: Support specifying app resources in spidapter by @peasee in #9536
• test(runtime): Spice Cayenne DDL integration test by @lukekim in #9535
• fix: Handle schema evolution mismatch errors during data refresh by @lukekim in #9527
• fix: resolve clippy lint warnings by @phillipleblanc in #9547
• pr-builds --tag <TAG> for build_and_release.yml by @Jeadie in #9507
• Add --output flag to spice login with env/json/keychain modes by @Jeadie in #9541
• Don't use 'PartitionedTableScanRewrite' in async distributed query by @Jeadie in #9548
• feat(spidapter): add local backend mode with single executor by @phillipleblanc in #9531
• support chat template in HF by @Jeadie in #9543
• fix(cayenne): stream PK retention deletes and run OOM regression in CI by @phillipleblanc in #9533
• cayenne: Staged append writes to prevent partial writes and data loss on stream error by @sgrebnov in #9491
• AcceleratedTable::scan use FederatedTable::scan when ClusterRole::Scheduler by @Jeadie in #9550
• Upgrade to delta-kernel-rs v0.18.2 by @lukekim in #9528
• Run cayenne tests as part of PR CI by @sgrebnov in #9554
• Upgrade to DataFusion v52.2.0 by @lukekim in #9419
• Remove Snapshot Compaction + Add snapshot existence check by @krinart in #9523
• Update dependencies by @lukekim in #9566
• fix: Update benchmark snapshots by @app/github-actions in #9565
• fix: Compare Cayenne table configuration on startup by @peasee in #9529
• Make Refresh::refresh_sql more robust to alterations over time. by @Jeadie in #9549
• fix: Update datafusion-table-providers dependency to latest revision by @lukekim in #9574
• Unset AWS_ENDPOINT_URL when empty by @krinart in #9575
• fix: allow BytesProcessedExec repartitioning for unordered input by @lukekim in #9540
• Sanitize DataFusion errors by @lukekim in #9530
• Add conditional logging for partition assignments by @Jeadie in #9577
• use 'properly early exit on SIGTERM' by @Jeadie in #9573
• Update datafusion to 52.2.0 by @phillipleblanc in #9582
• Ensure we query one and only one partition per request by @Jeadie in #9416
• feat: Add support for Spicepod version v2 by @lukekim in #9583
• [SpiceDQ] Improve error messages; Avoid race condition on allocate_initial_partitions. by @Jeadie in #9579
• Update ballista dependencies to latest 52.0.0 revision by @lukekim in #9581
• Fix Databricks spark_connect mode always disabled by @phillipleblanc in #9586
• Support partitioning in Arrow accelerator by @Jeadie in #9571
• Fix spice query CLI response deserialization by @phillipleblanc in #9588
• fix: Update benchmark snapshots by @app/github-actions in #9584
• fix: Share RuntimeEnv across Cayenne read/write/delete paths for targeted list_files_cache invalidation by @sgrebnov in #9589
• feat: Add file:// state_location support for async queries scheduler by @phillipleblanc in #9590
• Update endgame links by @krinart in #9598

Full Changelog: https://github.com/spiceai/spiceai/compare/v1.11.2...v2.0.0-rc.1

Announcing the release of Spice v1.10.2! 🔥

v1.10.2 introduces Tiered Caching Acceleration with Localpod for multi-layer acceleration architectures, Periodic Acceleration Snapshots with configurable intervals, DynamoDB JSON Nesting for column consolidation, and Kafka/Debezium Batching for faster data ingestion. This release also includes fixes for SQLite accelerator decimal/date handling and real-time status reporting for the /v1/datasets and /v1/models API endpoints.

What's New in v1.10.2​

Tiered Caching with Localpod​

Multi-Layer Acceleration Architecture: The Localpod connector now supports caching refresh mode, enabling tiered acceleration where a persistent cache (e.g., file-mode DuckDB) feeds a fast in-memory cache (e.g., Arrow, memory-mode DuckDB).

Key Features:

• Automatic Cache Propagation: New cache entries automatically propagate from parent to child accelerators
• Warm Startup: Child accelerators initialize from existing parent data on startup, eliminating cold-start latency
• Flexible Tiering: Combine any accelerator engines (DuckDB, SQLite, Cayenne) across tiers

Example spicepod.yaml configuration:

datasets:
  # Parent: persistent file-mode cache
  - from: https://api.example.com
    name: api_cache
    acceleration:
      enabled: true
      refresh_mode: caching
      engine: duckdb
      mode: file

  # Child: fast in-memory cache fed by parent
  - from: localpod:api_cache
    name: api_cache_memory
    acceleration:
      enabled: true
      refresh_mode: caching
      engine: arrow
      mode: memory

For more details, refer to the Localpod Data Connector Documentation.

Periodic Acceleration Snapshots​

Configurable Snapshot Intervals: A new snapshots_create_interval parameter enables periodic snapshot creation for accelerated datasets across all refresh modes. This provides better control over snapshot frequency and ensures consistent recovery points for accelerated data.

Example spicepod.yaml configuration:

datasets:
  - from: s3://my-bucket/data.parquet
    name: my_data
    acceleration:
      enabled: true
      engine: duckdb
      mode: file
      refresh_mode: caching
      snapshots: enabled
      params:
        snapshots_create_interval: 60s # Write a snapshot every 60 seconds

For more details, refer to the Data Acceleration Documentation.

DynamoDB JSON Nesting​

Consolidate Columns into JSON: The DynamoDB Data Connector now supports consolidating columns into a single JSON column using the json_object: "*" metadata option. This is useful when only a few columns are needed as discrete fields while the rest can be accessed as nested JSON.

Example spicepod.yaml configuration:

datasets:
  - from: dynamodb:my_table
    name: my_table
    columns:
      - name: PK
      - name: SK
      - name: data_json
        metadata:
          json_object: '*' # Captures all other columns as JSON

Example Output: Given a DynamoDB table with columns PK, SK, name, email, and status, the resulting table schema consolidates all non-specified columns into the data_json column:

For more details, refer to the DynamoDB JSON Nesting Documentation.

Kafka/Debezium Batching​

Faster Data Ingestion: Configure message batching for Kafka and Debezium connectors to improve data ingestion throughput. Batching reduces processing overhead by grouping multiple messages together before insertion.

Key Features:

• Configurable Batch Size: Control the maximum number of records per batch (default: 10,000)
• Configurable Batch Duration: Set the maximum wait time before flushing a partial batch (default: 1s)

Example spicepod.yaml configuration:

datasets:
  - from: debezium:kafka-server.public.my_table
    name: my_table
    params:
      batch_max_size: 10000 # Max records per batch (default: 10000)
      batch_max_duration: 1s # Max wait time per batch (default: 1s)

For more details, refer to the Kafka Data Connector Documentation and Debezium Data Connector Documentation.

Additional Improvements & Bug Fixes​

• Reliability: Fixed SQLite accelerator decimal and date type handling for improved data type accuracy.
• Reliability: Fixed real-time status reporting for /v1/datasets and /v1/models API endpoints.
• Reliability: Fixed Kafka warning when security.protocol is set to PLAINTEXT.

Contributors​

• @phillipleblanc
• @lukekim
• @krinart

Breaking Changes​

No breaking changes.

Cookbook Updates​

New Cayenne Data Accelerator Recipe: New recipe demonstrating how to accelerate a local copy of the taxi trips dataset using Cayenne as the data accelerator engine. See Cayenne Data Accelerator Recipe for details.

New Dataset Partitioning Recipe: New recipe demonstrating how to partition accelerated datasets to improve query performance. See Dataset Partitioning for details.

The Spice Cookbook includes 84 recipes to help you get started with Spice quickly and easily.

Upgrading​

To upgrade to v1.10.2, use one of the following methods:

CLI:

spice upgrade

Homebrew:

brew upgrade spiceai/spiceai/spice

Docker:

Pull the spiceai/spiceai:1.10.2 image:

docker pull spiceai/spiceai:1.10.2

For available tags, see DockerHub.

Helm:

helm repo update
helm upgrade spiceai spiceai/spiceai

AWS Marketplace:

🎉 Spice is now available in the AWS Marketplace!

What's Changed​

Changelog​

• Fix kafka warning when security.protocol is set to PLAINTEXT by @krinart in #8587
• fix: SQLite accelerator decimal/date handling by @phillipleblanc in #8606
• feat: Enable localpod with caching mode accelerator for tiered caching by @phillipleblanc in #8621
• Remove the clippy::too_many_lines lint by @phillipleblanc in #8549
• Add snapshot interval for acceleration snapshots by @phillipleblanc in #8627
• Json Nesting for DynamoDB by @krinart in #8623
• Implement batching for Kafka/Debezium + null Decimal handling by @krinart in #8622
• fix: Status field in /v1/datasets & /v1/models by @lukekim in #8633

Announcing the release of Spice v1.9.0-rc.2! 🌶

This is the second release candidate for v1.9.0, which introduces Spice Cayenne, a new high-performance data accelerator built on the Vortex columnar format that delivers better than DuckDB performance without single-file scaling limitations and a preview of Multi-Node Distributed Query based on Apache Ballista. v1.9.0-rc.2 also upgrades to DataFusion v50 and DuckDB v1.4.1 for even higher query performance, expands search capabilities with full-text search on views and multi-column embeddings, includes significant DynamoDB and DuckDB accelerator improvements, expands the HTTP data connector to support endpoints as tables, and delivers many security and reliability improvements.

What's New in v1.9.0-rc.2​

Cayenne Data Accelerator (Beta)​

Introducing Cayenne: SQL as an Acceleration Format: A new high-performance Data Accelerator that simplifies multi-file data acceleration by using an embedded database (SQLite) for metadata while storing data in the Vortex columnar format, a Linux Foundation project. Cayenne delivers query and ingestion performance better than DuckDB's file-based acceleration without DuckDB's memory overhead and the scaling challenges of single DuckDB files.

Cayenne uses SQLite to manage acceleration metadata (schemas, snapshots, statistics, file tracking) through simple SQL transactions, while storing data in Vortex's compressed columnar format. This architecture provides:

Key Features:

• SQLite + Vortex Architecture: All metadata is stored in SQLite tables with standard SQL transactions, while data lives in Vortex's compressed, chunked columnar format designed for zero-copy access and efficient scanning.
• Simplified Operations: No complex file hierarchies, no JSON/Avro metadata files, no separate catalog servers—just SQL tables and Vortex data files. The entire metadata schema is intentionally simple for maximum reliability.
• Fast Metadata Access: Single SQL query retrieves all metadata needed for query planning—no multiple round trips to storage, no S3 throttling, no reconstruction of metadata state from scattered files.
• Efficient Small Changes: Dramatically reduces small file proliferation. Snapshots are just rows in SQLite tables, not new files on disk. Supports millions of snapshots without performance degradation.
• High Concurrency: Changes consist of two steps: stage Vortex files (if any), then run a single SQL transaction. Much faster conflict resolution and support for many more concurrent updates than file-based formats.
• Advanced Data Lifecycle: Full ACID transactions, delete support, and retention SQL execution on refresh commit.

Example Spicepod.yml configuration:

datasets:
  - from: s3:my_table
    name: accelerated_data_30d
    acceleration:
      enabled: true
      engine: cayenne
      mode: file
      refresh_mode: append
      retention_sql: DELETE FROM accelerated_data WHERE created_at < NOW() - INTERVAL '30 days'

Note, the Cayenne Data Accelerator is in Beta with limitations.

For more details, refer to the Cayenne Documentation, the Vortex project, and the DuckLake announcement that partly inspired this design.

Multi-Node Distributed Query (Preview)​

Apache Ballista Integration: Spice now supports distributed query execution based on Apache Ballista, enabling distributed queries across multiple executor nodes for improved performance on large datasets. This feature is in preview in v1.9.0-rc.2.

Architecture:

A distributed Spice cluster consists of:

• Scheduler: Responsible for distributed query planning and work queue management for the executor fleet
• Executors: One or more nodes responsible for running physical query plans

Getting Started:

Start a scheduler instance using an existing Spicepod. The scheduler is the only spiced instance that needs to be configured:

# Start scheduler (note the flight bind address override if you want it reachable outside localhost)
spiced --cluster-mode scheduler --flight 0.0.0.0:50051

Start one or more executors configured with the scheduler's flight URI:

# Start executor (automatically selects a free port if 50051 is taken)
spiced --cluster-mode executor --scheduler-url spiced://localhost:50051

Query Execution:

Queries run through the scheduler will now show a distributed_plan in EXPLAIN output, demonstrating how the query is distributed across executor nodes:

EXPLAIN SELECT count(id) FROM my_dataset;

Current Limitations:

• Accelerated datasets are currently not supported. This feature is designed for querying partitioned data lake formats (Parquet, Delta Lake, Iceberg, etc.)
• The feature is in preview and may have stability or performance limitations
• Specific acceleration support is planned for future releases

DataFusion v50 Upgrade​

Spice.ai is built on the Apache DataFusion query engine. The v50 release brings significant performance improvements and enhanced reliability:

Performance Improvements 🚀:

• Dynamic Filter Pushdown: Enhanced dynamic filter pushdown for custom ExecutionPlans, ensuring filters propagate correctly through all physical operators for improved query performance.

• Partition Pruning: Expanded partition pruning support ensures that unnecessary partitions are skipped when filters are not used, reducing data scanning overhead and improving query execution times.

Apache Spark Compatible Functions: Added support for Spark-compatible functions including array, bit_get/bit_count, bitmap_count, crc32/sha1, date_add/date_sub, if, last_day, like/ilike, luhn_check, mod/pmod, next_day, parse_url, rint, and width_bucket.

Bug Fixes & Reliability: Resolved issues with partition name validation and empty execution plans when vector index lists are empty. Fixed timestamp support for partition expressions, enabling better partitioning for time-series data.

See the Apache DataFusion 50.0.0 Release for more details.

DuckDB v1.4.1 Upgrade and Accelerator Improvements​

DuckDB v1.4.1: DuckDB has been upgraded to v1.4.1, which includes several performance optimizations.

Composite ART Index Support: DuckDB in Spice now supports composite (multi-column) Adaptive Radix Tree (ART) indexes for accelerated table scans. When queries filter on multiple columns fully covered by a composite index, the optimizer automatically uses index scans instead of full table scans, delivering significant performance improvements for selective queries.

Example configuration:

datasets:
  - from: file://data.parquet
    name: sales
    acceleration:
      enabled: true
      engine: duckdb
      indexes:
        '(region, product_id)': enabled

Performance example with composite index on 7.5M rows:

SELECT * FROM sales WHERE region = 'US' AND product_id = 12345;

-- Without index: 0.282s
-- With composite index (region, product_id): 0.037s
-- Performance improvement: 7.6x faster with composite index

DuckDB Intermediate Materialization: Queries with indexes now use intermediate materialization (WITH ... AS MATERIALIZED) to leverage faster index scans. Currently supported for non-federated queries (query_federation: disabled) against a single table with indexes only. When predicates cover more columns than the index, the optimizer rewrites queries to first materialize index-filtered results, then apply remaining predicates. This optimization can deliver significant performance improvements for selective queries.

Example configuration:

datasets:
  - from: file://sales_data.parquet
    name: sales
    acceleration:
      enabled: true
      engine: duckdb
      mode: file
      params:
        query_federation: disabled # Required currently for intermediate materialization
      indexes:
        '(region, product_id)': enabled

Performance example:

-- Query with indexed columns (region, product_id) plus additional filter (amount)
SELECT * FROM sales
WHERE region = 'US' AND product_id = 12345 AND amount > 1000;

-- Optimized execution time: 0.031s (with intermediate materialization)
-- Standard execution time: 0.108s (without optimization)
-- Performance improvement: ~3.5x faster

The optimizer automatically rewrites the query to:

WITH _intermediate_materialize AS MATERIALIZED (
  SELECT * FROM sales WHERE region = 'US' AND product_id = 12345
)
SELECT * FROM _intermediate_materialize WHERE amount > 1000;

Parquet Buffering for Partitioned Writes: DuckDB partitioned writes in table mode now support Parquet buffering, reducing memory usage and improving write performance for large datasets.

Retention SQL on Refresh Commit: DuckDB accelerations now support running retention SQL on refresh commit, enabling automatic data cleanup and lifecycle management during refresh operations.

UTC Timezone for DuckDB: DuckDB now uses UTC as the default timezone, ensuring consistent behavior for time-based queries across different environments.

Example Spicepod.yml configuration:

datasets:
  - from: s3://my_bucket/large_table/
    name: partitioned_data
    acceleration:
      enabled: true
      engine: duckdb
      mode: file
      retention:
        sql: DELETE FROM partitioned_data WHERE event_time < NOW() - INTERVAL '7 days'

HTTP Data Connector​

• Querying endpoints as tables: The HTTP/HTTPS Data Connectors now supports querying HTTP endpoints directly as tables in SQL queries with dynamic filters. This feature transforms REST APIs into queryable data sources, making it easy to integrate external service data.

• Query HTTP endpoint that returns structured data (JSON, CSV, etc.) as if it were a database table

• Configurable retry logic, timeouts, and POST request support for more complex API interactions

Example Spicepod.yml configuration:

datasets:
  - from: https://api.tvmaze.com
    name: tvmaze
    params:
      file_format: json
      max_retries: 3
      client_timeout: 10s

Example SQL query:

SELECT request_path, request_query, content
FROM tvmaze
WHERE request_path = '/search/people' and request_query = 'q=michael'
LIMIT 10;

If a request_body is supplied it will be posted to the endpoint:

Example SQL query:

SELECT request_path, request_query, content
FROM tvmaze
WHERE request_path = '/search/people' and request_query = 'q=michael' and request_body = '{"name": "michael"}'
LIMIT 10;

HTTP endpoints can be accelerated using refresh_sql:

datasets:
  - from: https://api.tvmaze.com
    name: tvmaze
    acceleration:
      enabled: true
      refresh_mode: full
      refresh_sql: |
        SELECT request_path, request_query, content 
        FROM tvmaze
        request_path = '/search/people'
          AND request_query IN ('q=michael', 'q=luke')

DynamoDB Data Connector Improvements​

Improved Query Performance: The DynamoDB Data Connector now includes improved filter handling for edge cases, parallel scan support for faster data ingestion, and better error handling for misconfigured queries. These improvements enable more reliable and performant access to DynamoDB data.

Example Spicepod.yml configuration:

datasets:
  - from: dynamodb:my_table
    name: ddb_data
    params:
      scan_segments: 10 # Default `auto` which calculates optimal segments based on number of rows

S3 Versioning Support​

Atomic Range Reads for Versioned Files: Spice now supports S3 Versioning for all connectors using object-store (S3, Delta Lake, etc.), ensuring range reads over versioned files are atomically correct. When S3 versioning is enabled, Spice automatically tracks version IDs during file discovery and uses them for all subsequent range reads, preventing inconsistencies from concurrent file modifications.

Current limitations:

• Multi-file connections (e.g., partitioned datasets) do not yet support version tracking across all files
• Version tracking is automatic when S3 versioning is enabled on the bucket

Search & Embeddings Enhancements​

Full-Text Search on Views: Full-text search indexes are now supported on views, enabling advanced search scenarios over pre-aggregated or transformed data. This extends the power of Spice's search capabilities beyond base datasets.

Multi-Column Embeddings on Views: Views now support embedding columns, enabling vector search and semantic retrieval on view data. This is useful for search over aggregated or joined datasets.

Vector Engines on Views: Vector search engines are now available for views, enabling similarity search over complex queries and transformations.

Example Spicepod.yml configuration:

views:
  - name: aggregated_reviews
    sql: SELECT review_id, review_text FROM reviews WHERE rating > 4
    embeddings:
      - column: review_text
        model: openai:text-embedding-3-small

Dedicated Query Thread Pool (Now Enabled by Default)​

Dedicated Query Thread Pool: Query execution and accelerated refreshes now run on their own dedicated thread pool, separate from the HTTP server. This prevents heavy query workloads from slowing down API responses, keeping health checks fast and avoiding unnecessary Kubernetes pod restarts under load.

This feature was opt-in in previous releases and is now enabled by default in v1.9.0-rc.2. To disable it and revert to the previous behavior, add the following spicepod.yaml configuration:

runtime:
  params:
    dedicated_thread_pool: none

Query Performance Optimizations​

Stale-While-Revalidate Cache Control: Query results now support "stale-while-revalidate" cache control, allowing stale cached data to be served immediately while asynchronously refreshing the cache entry in the background. This improves response times for frequently-accessed queries while maintaining data freshness. Requires cache key type to be set to "sql (raw)" for proper operation.

Optimized Prepared Statements: Prepared statement handling has been optimized for better performance with parameterized queries, reducing planning overhead and improving execution time for repeated queries.

Large RecordBatch Chunking: Large Arrow RecordBatch objects are now automatically chunked to control memory usage during query execution, preventing memory exhaustion for queries returning large result sets.

Query Result Cache: Stale-While-Revalidate​

HTTP Cache-Control Support: The query result cache now supports the stale-while-revalidate Cache-Control directive, enabling faster response times by serving stale cached results immediately while asynchronously refreshing the cache in the background. This feature is particularly useful for applications that can tolerate slightly stale data in exchange for improved performance.

How it works:

When a cache entry is stale but within the stale-while-revalidate window, Spice will:

1. Immediately return the stale cached result to the client
2. Asynchronously re-execute the query in the background to refresh the cache
3. Future requests will use the refreshed data

Configuration:

Use the Cache-Control HTTP header with the stale-while-revalidate directive:

Cache-Control: max-age=300, stale-while-revalidate=60

This configuration caches results for 5 minutes (300 seconds), and allows serving stale results for an additional 60 seconds while refreshing in the background.

Requirements:

• Must use plan or raw SQL cache keys (set cache_key_type to sql or plan in results_caching configuration)
• Background revalidation re-executes queries through the normal query path
• Timestamp tracking automatically determines cache entry age for staleness checks

Example configuration via HTTP header:

GET /v1/sql
Cache-Control: max-age=600, stale-while-revalidate=120
X-Cache-Key-Type: sql

This feature improves application responsiveness while ensuring data freshness through background updates.

Security & Reliability Improvements​

Enhanced HTTP Client Security: HTTP client usage across the runtime has been hardened with improved TLS validation, certificate pinning for critical endpoints, and better error handling for network failures.

ODBC Connector Improvements: Removed unwrap calls from the ODBC connector, improving error handling and reliability. Fixed secret handling and Kubernetes secret integration.

CLI Permissions Hardening: Tightened file permissions for the CLI and install script, ensuring secure defaults for configuration files and credentials.

Oracle Instant Client Pinning: Oracle Instant Client downloads are now pinned to specific SHAs, ensuring reproducible builds and preventing supply chain attacks.

AWS Authentication Improvements​

Improved Credential Retry Logic: AWS SDK credential initialization has been significantly improved with more robust retry logic and better error handling. The system now automatically retries transient credential resolution failures using Fibonacci backoff, allowing Spice to tolerate extended AWS outages (up to ~48 hours) without manual intervention.

Key features:

• Automatic retry with backoff: Implements Fibonacci backoff for transient credential failures (network issues, temporary AWS service disruptions)
• Configurable retry limits: Supports up to 300 retry attempts with a maximum retry interval of 600 seconds
• Better error handling: Distinguishes between retryable errors (connector errors) and non-retryable errors (misconfiguration)
• Unauthenticated access support: Properly supports unauthenticated access to public S3 buckets without requiring credentials
• Improved error messages: Provides detailed logging with attempt numbers, retry intervals, and error context for better troubleshooting

The improvements ensure more reliable AWS service integration, particularly in environments with intermittent network connectivity or during AWS service degradations.

Observability & Tracing​

DataFusion Log Emission: The Spice runtime now emits DataFusion internal logs, providing deeper visibility into query planning and execution for debugging and performance analysis.

AI Completions Tracing: Fixed tracing so that ai_completions operations are correctly parented under sql_query traces, improving observability for AI-powered queries.

Git Data Connector (Alpha)​

Version-Controlled Data Access: The new Git Data Connector (Alpha) enables querying datasets stored in Git repositories. This connector is ideal for use cases involving configuration files, documentation, or any data tracked in version control.

Example Spicepod.yml configuration:

datasets:
  - from: git:https://github.com/myorg/myrepo
    name: git_metrics
    params:
      file_format: csv

For more details, refer to the Git Data Connector Documentation.

Spice Java SDK 0.4.0​

The Spice Java SDK have been upgraded with support configurable Arrow memory limit: spice-java v0.4.0

SpiceClient client = SpiceClient.builder()
    .withArrowMemoryLimitMB(1024) // 1GB limit
    .build();

CLI Improvements​

Install Specific Versions: The spice install command now supports installing specific versions of the Spice runtime and CLI. This enables easy version management, downgrading, or installation of specific releases for testing or compatibility requirements.

Usage:

# Install a specific version
spice install v1.8.3

# Install a specific version with AI flavor
spice install v1.8.3 ai

# Install latest version (existing behavior)
spice install
spice install ai

Note: Homebrew installations require manual version management via brew install spiceai/spiceai/spice@<version>.

Persistent Query History: The Spice CLI REPL (SQL, search, and chat interfaces) now persists command history to ~/.spice/query_history.txt, making your query history available across sessions. The history file is automatically created if it doesn't exist, with graceful fallback if the home directory cannot be determined.

New REPL Commands:

• .clear - Clear the screen using ANSI escape codes for a clean workspace
• .clear history - Clear and persist the query history, removing all stored commands

Tab Completion: Tab completion now includes suggestions based on your command history, making it faster to re-run or modify previous queries.

Example usage:

sql> SELECT * FROM my_table;
sql> .clear          # Clears the screen
sql> .clear history  # Clears command history
sql> # Use arrow keys or tab to access previous commands

Additional Improvements & Bug Fixes​

• Reliability: Fixed refresh worker panics with recovery handling to prevent runtime crashes during acceleration refreshes.
• Reliability: Improved error messages for missing or invalid spicepod.yaml files, providing actionable feedback for misconfiguration.
• Reliability: Fixed DuckDB metadata pointer loading issues for snapshots.
• Performance: Ensured ListingTable partitions are pruned correctly when filters are not used.
• Reliability: Fixed vector dimension determination for partitioned indexes.
• Search: Fixed casing issues in Reciprocal Rank Fusion (RRF) for hybrid search queries.
• Search: Fixed search field handling as metadata for chunked search indexes.
• Validation: Added timestamp support for partition expressions.
• Validation: Fixed regexp_match function for DuckDB datasets.
• Validation: Fixed partition name validation for improved reliability.

Contributors​

• @Jeadie
• @lukekim
• @peasee
• @phillipleblanc
• @sgrebnov
• @kczimm
• @krinart
• @mach-kernel

Breaking Changes​

No breaking changes.

Cookbook Updates​

New HTTP Data Connector Recipe: New recipe demonstrating how to query REST APIs and HTTP(s) endpoints. See HTTP Connector Recipe for details.

The Spice Cookbook includes 82 recipes to help you get started with Spice quickly and easily.

Upgrading​

To upgrade to v1.9.0-rc.2, use one of the following methods:

CLI:

spice upgrade

Homebrew:

brew upgrade spiceai/spiceai/spice

Docker:

Pull the spiceai/spiceai:1.9.0-rc.2 image:

docker pull spiceai/spiceai:1.9.0-rc.2

For available tags, see DockerHub.

Helm:

helm repo update
helm upgrade spiceai spiceai/spiceai

AWS Marketplace:

🎉 Spice is now available in the AWS Marketplace!

What's Changed​

Dependencies​

• DataFusion: Upgraded to v50
• Apache Arrow: Upgraded to v56
• DuckDB: Upgraded to v1.4.1
• Delta Kernel: Upgraded to v0.16.0

Changelog​

• Fix for search field as metadata for chunked search indexes by @Jeadie in #7429
• Bump object_store from 0.12.3 to 0.12.4 by @app/dependabot in #7433
• Properly respect disabling snapshots by @phillipleblanc in #7431
• Revert "Properly respect disabling snapshots" by @sgrebnov in #7439
• Revert "Disable snapshots by default" by @sgrebnov in #7438
• Add preview warning for write access mode by @sgrebnov in #7440
• fix: regexp_match for DuckDB datasets by @kczimm in #7443
• Add feature is currently in preview warning for snapshots by @sgrebnov in #7442
• [Logger] Also emit Datafusion logs by @mach-kernel in #7441
• add missing snapshot by @kczimm in #7446
• Fix tracing so that ai_completions are parented under sql_query by @lukekim in #7415
• Enable snapshot acceleration by default by @phillipleblanc in #7451
• Disable acceleration refresh metrics by @krinart in #7450
• Add v1.8 release notes by @phillipleblanc in #7430
• fix: partition name validation by @kczimm in #7452
• Fix lint error due to ignore without reasons by @krinart in #7454
• Add models and CUDA support to spiced install script by @lukekim in #7457
• Post-release 1.8 updates by @phillipleblanc in #7455
• Remove println in datafusion by @phillipleblanc in #7461
• Update end_game.md to notify once release is done by @sgrebnov in #7460
• Remove italics from snapshot logging by @phillipleblanc in #7463
• Update openapi.json by @app/github-actions in #7466
• Fix generate spicepod schema by @phillipleblanc in #7464
• Fix generate acknowledements by @phillipleblanc in #7465
• Update spicepod.schema.json by @app/github-actions in #7469
• fix: Ensure ListingTable partitions are pruned when filters are not used by @peasee in #7471
• Create runtime-secrets crate by @phillipleblanc in #7474
• Create runtime-parameters crate by @phillipleblanc in #7475
• Don't download the snapshot if the acceleration is present by @phillipleblanc in #7477
• Fix casing for keywords and additional columns by @Jeadie in #7770
• Bump actions/upload-artifact from 4 to 5 by @app/dependabot in #7750
• Bump criterion from 0.5.1 to 0.7.0 by @app/dependabot in #7740
• Bump rustls-native-certs from 0.8.1 to 0.8.2 by @app/dependabot in #7744
• Git Data Connector (Alpha) by @lukekim in #7772
• Pepper accelerator delete support by @lukekim in #7616
• Update Helm chart instructions for Helm in end_game.md by @sgrebnov in #7776
• Turso data accelerator by @lukekim in #7472
• Apply retention SQL filter to refresh fetch by @phillipleblanc in #7778
• Add Parquet buffering option for DuckDB partitioned writes (tables mode) by @sgrebnov in #7735
• fix: EmptyExec when list indexes is empty by @kczimm in #7784
• 1.8.3 post-release housekeeping by @mach-kernel in #7783
• feat: Upgrade to Datafusion v50 by @peasee in #7777
• fix: Replace vortex datafusion with public crate by @peasee in #7791
• Full-text search on views by @Jeadie in #7733
• Revert "Apply retention SQL filter to refresh fetch (#7778)" by @phillipleblanc in #7796
• fix: Add ingest duration and acceleration size metrics to testoperator by @peasee in #7792
• Set local timezone to UTC for DuckDB by @phillipleblanc in #7797
• add Timestamp support for partition expressions by @kczimm in #7803
• Fix trunk lint by @krinart in #7804
• Add missing mongodb params by @krinart in #7807
• Embedding columns on view components by @Jeadie in #7795
• Add Turso as a Pepper Catalog metastore by @lukekim in #7793
• Run retention_sql on refresh commit for DuckDB by @lukekim in #7785
• docs: Update datafusion upgrade checklist by @peasee in #7812
• Vector engines on views by @Jeadie in #7808
• Handle refresh worker panics and add recovery test by @phillipleblanc in #7815
• chunk large record batches to control memory usage by @kczimm in #7802
• fix: cannot determine vector dimension for partitioned indexes by @kczimm in #7818
• Upgrade to Turso v0.3 by @lukekim in #7821
• fix: Ensure custom *Exec ExecutionPlans push down dynamic filters by @peasee in #7811
• handle casing in RRF by @Jeadie in #7825
• Enable 'turso' for pepper acceleration by default by @sgrebnov in #7826
• Improved DynamoDB Data Connector by @krinart in #7715
• Initial support for llama.cpp as LLM inference backend by @lukekim in #7794
• Pepper: Implement retention SQL on refresh commit by @phillipleblanc in #7814
• Fix Dockerfiles for arm64 by @lukekim in #7834
• [DynamoDB] Handle filter edge-cases by @krinart in #7830
• [DynamoDB] Support parallelization for Scan request by @krinart in #7829
• Don't feature gate Pepper by @lukekim in #7832
• Fix llama.cpp static link by @lukekim in #7835
• fix: docker nightly builds by @kczimm in #7837
• Use GitHub-hosted macOS runner only for tag releases by @lukekim in #7836
• Fix Bug: DuckDB INTERNAL Error: Failed to load metadata pointer by @sgrebnov in #7839
• Fix docker arm64 build to use aegis in pure-rust mode by @lukekim in #7840
• Revert "Use GitHub-hosted macOS runner only for tag releases" by @lukekim in #7843
• Rename Pepper to Cayenne by @lukekim in #7844
• Tighten CLI permissions and install script by @lukekim in #7845
• Set mvcc for Cayenne Turso metastore by @lukekim in #7850
• Optimize Prepared Statements by @lukekim in #7859
• Remove unwrap from ODBC connector, fix secrets, and kuberenetes secre… by @lukekim in #7846
• Improve and secure HTTP client usage by @lukekim in #7847
• Pin Oracle Instant Client download to a SHA by @lukekim in #7851
• Improve experience for missing or invalid Spicepod.yaml by @lukekim in #7849
• chore: Fix PR linting by @peasee in #7865
• Revert FlightIPC issues by @Jeadie in #7870
• Improve error message by adding 'cayenne' to the list of valid accelerator engines by @sgrebnov in #7882
• fix: allow parameter index without dollar signs by @kczimm in #7887
• Temporary disable supports_limit_pushdown for SchemaCastScanExec by @sgrebnov in #7893
• Remove '.embeddings[].metadata' by @Jeadie in #7897
• Optimize macOS and Windows builds by @lukekim in #7863
• fix: Kafka message delivery failed by @kczimm in #7883
• docs: Update component criteria by @peasee in #7891
• fix: Make integration run with no relevant changes, disable makefile targets on PR by @peasee in #7896
• Add Cayenne benchmark and concurrency tests and remove indexes for Turso MVCC by @lukekim in #7879
• Revert llama.cpp engine by @lukekim in #7898
• Make Cayenne snapshotting more robust by @sgrebnov in #7899
• Release notes v1.9.0-rc1 by @Jeadie in #7902
• Fix dataset_acceleration_last_refresh_time_ms unit to milliseconds in description by @ewgenius in #7901
• Fix lint error in record_explain_plan functionality by @sgrebnov in #7906
• Cleanup old snapshots after full refresh by @lukekim in #7908
• Cayenne deletion vector caching support by @lukekim in #7903
• Split filters into partition filters (for pruning) and data filters by @lukekim in #7889
• fix: Update benchmark snapshots by @app/github-actions in #7911
• fix: Update benchmark snapshots by @app/github-actions in #7912
• fix: Update benchmark snapshots by @app/github-actions in #7913
• Update spicepod.schema.json by @app/github-actions in #7916
• fix: Update benchmark snapshots by @app/github-actions in #7917
• Add Cayenne & Turso accelerators to E2E CI test matrix by @lukekim in #7922
• Make preview warnings consistent by @lukekim in #7921
• Filter and write optimizations by @lukekim in #7918
• fix: Set sccache region explicitly by @peasee in #7928
• fix: Enable integration test merge group checks by @peasee in #7927
• Update testoperator release branch from 1.8 to 1.9 by @peasee in #7926
• Update DuckDB to 1.4.1 with composite ART scans by @mach-kernel in #7884
• Don't build Windows on trunk pushes by @lukekim in #7931
• fix: Use correct minio secret in build binary push by @peasee in #7934
• Update test-framework workers to use dedicated Flight client by @sgrebnov in #7938
• Fix financebench, configure s3vectors for appropriate snapshotting by @Jeadie in #7935
• Don't try to initialize accelerator if it is disabled by @lukekim in #7932
• Add spark UDFs to Spice by @Jeadie in #7936
• Fix extra async_trait in cayenne metadata catalog by @phillipleblanc in #7942
• deps: Upgrade to Rust 1.90 by @peasee in #7941
• Add cayenne accelerator to README.md by @ewgenius in #7905
• fix: Update benchmark snapshots by @app/github-actions in #7948
• Run integration tests with AWS_EC2_METADATA_DISABLED flag by @sgrebnov in #7952
• Only retry credentials on ConnectorError by @kczimm in #7944
• fix: Improve join reordering by ensuring JoinSelection is applied by @peasee in #7828
• fix: Remove unused deps, consolidate workspace deps by @peasee in #7953
• bump async-openai commit by @kczimm in #7929
• deps: Use vortex fork by @peasee in #7954
• Enable tracing in delta lake integration tests by @sgrebnov in #7951
• Update datasets in S3 vectors test case by @Jeadie in #7956
• Add spiced metrics scraping to test operator by @lukekim in #7937
• Memoize S3 vectors ListIndex API call with configurable TTL by @kczimm in #7910
• Cayenne performance optimizations by @lukekim in #7907
• Setup HotFix issue template by @ewgenius in #7957
• Fix AWS SDK credential cache retry handling by @phillipleblanc in #7943
• Infer RRF join_key from TableProvider::constraints and implement SearchQueryProvider::constraints. by @Jeadie in #7959
• [Optimizer]: DuckDB intermediate materialization (non-federated) by @mach-kernel in #7964
• 1.7.3 post-release housekeeping by @ewgenius in #7962
• Fix digest_many UDF for ColumnarValue::Array. by @Jeadie in #7960
• Fix spiced metrics reporting as part of benchmark tests by @sgrebnov in #7967
• Avoid pushing down Spice specific UDFs to accelerators during federation by @Jeadie in #7909
• CLI file persisted history with .clear and .clear history commands by @lukekim in #7970
• ResultsCache Cache-Control stale-while-revalidate by @lukekim in #7963
• Use GetVectors API instead of returnData by @kczimm in #7083
• Make DuckDB intermediate materialization logic more robust by @sgrebnov in #7971
• [Cayenne] Configurable target Vortex file size by @lukekim in #7972
• fix: Update benchmark snapshots by @app/github-actions in #7974
• Bump github.com/klauspost/compress from 1.17.11 to 1.18.1 by @app/dependabot in #7872
• fix: Update benchmark snapshots by @app/github-actions in #7978
• fix: Update benchmark snapshots by @app/github-actions in #7982
• Run Integration tests on spiceai-dev-runners by @sgrebnov in #7985
• [CLI] Fix cursor issue due to flush by @lukekim in #7981
• fix: Support S3 versioning, Vortex dynamic filter pushdown by @peasee in #7984
• Make cluster a default feature by @lukekim in #7994
• Optimize DuckDB Intermediate Index Materialization for No-Index Case by @sgrebnov in #7998
• HTTP connector with dynamic filter support by @lukekim in #7969
• Revert federation 'can_execute_plan' by @Jeadie in #7999
• Fix stale caching by @lukekim in #7995
• Fix count(*) for http connector by @krinart in #8001
• [CLI] Install specific version by @lukekim in #8006
• Fix stale with revalidate request/response by @lukekim in #8005
• Fallback RequestContext for cluster queries by @Jeadie in #8007
• Use use_rustls_tls for Spice Cloud /connect by @lukekim in #8008
• Use delta-kernel-rs 0.16x + Parquet reader with object meta API changes by @mach-kernel in #8011
• fix: Update datafusion & arrow-rs with S3 versioning fix by @lukekim in #8012

Announcing the release of Spice v1.7.0! ⚡

Spice v1.7.0 upgrades to DataFusion v49 for improved performance and query optimization, introduces real-time full-text search indexing for CDC streams, EmbeddingGemma support for high-quality embeddings, new search table functions powering the /v1/search API, embedding request caching for faster and cost-efficient search and indexing, and OpenAI Responses API tool calls with streaming. This release also includes numerous bug fixes across CDC streams, vector search, the Kafka Data Connector, and error reporting.

What's New in v1.7.0​

DataFusion v49 Highlights​

Source: DataFusion 49.0.0 Release Blog.

Performance Improvements 🚀

• Equivalence System Upgrade: Faster planning for queries with many columns, enabling more sophisticated sort-based optimizations.
• Dynamic Filters & TopK Pushdown: Queries with ORDER BY and LIMIT now use dynamic filters and physical filter pushdown, skipping unnecessary data reads for much faster top-k queries.
• Compressed Spill Files: Intermediate files written during sort/group spill to disk are now compressed, reducing disk usage and improving performance.
• WITHIN GROUP for Ordered-Set Aggregates: Support for ordered-set aggregate functions (e.g., percentile_disc) with WITHIN GROUP.
• REGEXP_INSTR Function: Find regex match positions in strings.

Spice Runtime Highlights​

EmbeddingGemma Support: Spice now supports EmbeddingGemma, Google's state-of-the-art embedding model for text and documents. EmbeddingGemma provides high-quality, efficient embeddings for semantic search, retrieval, and recommendation tasks. You can use EmbeddingGemma via HuggingFace in your Spicepod configuration:

Example spicepod.yml snippet:

embeddings:
  - from: huggingface:huggingface.co/google/embeddinggemma-300m
    name: embeddinggemma
    params:
      hf_token: ${secrets:HUGGINGFACE_TOKEN}

Learn more about EmbeddingGemma in the official documentation.

POST /v1/search API Use Search Table Functions: The /v1/search API now uses the new text_search and vector_search Table Functions for improved performance.

Embedding Request Caching: The runtime now supports caching embedding requests, reducing latency and cost for repeated content and search requests.

Example spicepod.yml snippet:

runtime:
  caching:
    embeddings:
      enabled: true
      max_size: 128mb
      item_ttl: 5s

See the Caching documentation for details.

Real-Time Indexing for Full Text Search: Full Text search indexing is now supported for connectors that enable real-time changes, such as Debezium CDC streams. Adding a full-text index on a column with refresh_mode: changes works as it does for full/append-mode refreshes, enabling instant search on new data.

Example spicepod.yml snippet:

datasets:
  - from: debezium:cdc.public.question
    name: questions
    acceleration:
      enabled: true
      engine: duckdb
      primary_key: id
      refresh_mode: changes # Use 'changes'
    params: *kafka_params
    columns:
      - name: title
        full_text_search:
          enabled: true # Enable full-text-search indexing
          row_id:
            - id

OpenAI Responses API Tool Calls with Streaming: The OpenAI Responses API now supports tool calls with streaming, enabling advanced model interactions such as web_search and code_interpreter with real-time response streaming. This allows you to invoke OpenAI-hosted tools and receive results as they are generated.

Learn more in the OpenAI Model Provider documentation.

Runtime Output Level Configuration: You can now set the output_level parameter in the Spicepod runtime configuration to control logging verbosity in addition to the existing CLI and environment variable support. Supported values are info, verbose, and very_verbose. The value is applied in the following priority: CLI, environment variables, then YAML configuration.

Example spicepod.yml snippet:

runtime:
  output_level: info # or verbose, very_verbose

For more details on configuring output level, see the Troubleshooting documentation.

Bug Fixes​

Several bugs and issues have been resolved in this release, including:

• CDC Streams: Fixed issues where refresh_mode: changes could prevent the Spice runtime from becoming Ready, and improved support for full-text indexing on CDC streams.
• Vector Search: Fixed bugs where vector search HTTP pipeline could not find more than one IndexedTableProvider, and resolved errors with field mismatches in vector_search UDTF.
• Kafka Integration: Improved Kafka schema inference with configurable sample size, improved consumer group persistence for SQLite and Postgres accelerations, and added cooperative mode support.
• Perplexity Web Search: Fixed bug where Perplexity web search sometimes used incorrect query schema (limit).
• Databricks: Fixed issue with unparsing embedded columns.
• Error Reporting: ThrottlingException is now reported correctly instead of as InternalError.
• Iceberg Data Connector: Added support for LIMIT pushdown.
• Amazon S3 Vectors: Fixed ingestion issues with zero-vectors and improved handling when vector index is full.
• Tracing: Fixed vector search tracing to correctly report SQL status.

Contributors​

• @Jeadie
• @peasee
• @sgrebnov
• @kczimm
• @phillipleblanc
• @Advayp
• @lukekim
• @ewgenius
• @mach-kernel
• @krinart
• @ChrisTomAlxHitachi

New Contributors​

• @ChrisTomAlxHitachi made their first contribution in github.com/spiceai/spiceai/pull/6932 🎉

Breaking Changes​

No breaking changes.

Cookbook Updates​

• New Spice with Dotnet SDK Recipe - The recipe shows how to query Spice using the Dotnet SDK.

The Spice Cookbook includes 78 recipes to help you get started with Spice quickly and easily.

Upgrading​

To upgrade to v1.7.0, use one of the following methods:

CLI:

spice upgrade

Homebrew:

brew upgrade spiceai/spiceai/spice

Docker:

Pull the spiceai/spiceai:1.7.0 image:

docker pull spiceai/spiceai:1.7.0

For available tags, see DockerHub.

Helm:

helm repo update
helm upgrade spiceai spiceai/spiceai

AWS Marketplace:

🎉 Spice is now available in the AWS Marketplace!

What's Changed​

Dependencies​

• Rust: Upgraded from 1.88.0 to 1.89.0
• DataFusion: Upgraded from 48.0.1 to 49.0.0
• text-embeddings-inference: Upgraded from 1.7.3 to 1.8.2
• twox-hash: Upgraded from 1.6.3 to 2.1.0.

Changelog​

• Fix parameterised query planning in DataFusion by @Jeadie in #6942
• fix: Update benchmark snapshots by @app/github-actions in #6944
• refactor: Decouple full text search candidate from UDTF by @peasee in #6940
• fix: Re-enable search integration tests by @peasee in #6930
• Update acknowledgements and spicepod.schema.json by @sgrebnov in #6948
• Add enabling the responses API by @lukekim in #6949
• Post-release housekeeping by @sgrebnov in #6951
• Add missing param in release notes by @Advayp in #6959
• Create comprehensive S3vectors test by @Jeadie in #6903
• Update ROADMAP after v1.6 release by @sgrebnov in #6955
• Update openapi.json by @app/github-actions in #6961
• Add build step for new spiced images in end game template by @Jeadie in #6960
• refactor: Use text search UDTF in v1/search by @peasee in #6962
• Bump Jimver/cuda-toolkit from 0.2.26 to 0.2.27 by @app/dependabot in #6922
• Bump notify from 8.0.0 to 8.2.0 by @app/dependabot in #6924
• Use model2vec for search integration tests for speed by @Jeadie in #6971
• feat: Add initial DuckDB regexp pushdown support by @peasee in #6966
• Bump rustyline from 16.0.0 to 17.0.1 by @app/dependabot in #6976
• Upgrade delta_kernel to 0.14 by @phillipleblanc in #6977
• Consistent snapshots for mongodb by @krinart in #6974
• Bump indexmap from 2.10.0 to 2.11.0 by @app/dependabot in #6921
• Fix mongo tests: ignore container_registry() when building image name by @krinart in #6983
• Implement support for s3 tables for glue DataConnector by @krinart in #6981
• Bump serde_json from 1.0.142 to 1.0.143 by @app/dependabot in #6925
• Update build_and_release macOS pipeline to skip updating cmake if installed by @phillipleblanc in #6998
• Mark Kafka Data Connector Alpha quality by @sgrebnov in #6991
• Add v1.6.1 release notes by @lukekim in #7000
• Spice CLI trace: make error friendlier when task_history is disabled by @sgrebnov in #6996
• Warn when runtime or management is added in spicepod dependency by @Jeadie in #6953
• Enable .datasets[].vectors.params.s3_vectors_distance_metric for S3 Vectors by @Jeadie in #6982
• Add s3_vectors index support for CDC and Append streams by @sgrebnov in #6986
• Find all vector indexes in v1/search by @Jeadie in #7004
• Fix RRF; reorder by score by @Jeadie in #7007
• Fix for nested VectorScanTableProvider by @krinart in #7017
• Add --sql flag to output SQL query for spice trace by @Jeadie in #7002
• Make web search params engine-specific by @Advayp in #7022
• Add more MTEB benchmark spicepods by @peasee in #7026
• Improve error messaging in tools by @Jeadie in #6895
• Add retry for exporting task history records by @sgrebnov in #7049
• Increase DoPut write timeout for the next batch from 30 to 120 seconds by @sgrebnov in #7054
• Avoid redundant search embedding by @peasee in #7053
• Truncate text_embed task_history trace by @sgrebnov in #7050
• Use the UTC offset for the start_time and end_time fields in the task history by @ewgenius in #7056
• Update supported versions in SECURITY.md by @Jeadie in #7060
• Add integration test for Kafka S3 Vectors by @sgrebnov in #6988
• Enable parameters to enforce the value is one of several options by @Jeadie in #6984
• feat(iceberg): lakekeeper catalog - add warehouse param to spicepod by @ChrisTomAlxHitachi in #6932
• feat: Add HTTP query concurrency support to testoperator by @peasee in #7025
• Ensure no data does not throw error in v1/search by @Jeadie in #7033
• Bump github.com/spf13/cobra from 1.9.1 to 1.10.1 by @app/dependabot in #7013
• Add QA analytics for 1.6.x releases by @sgrebnov in #7082
• Use env variable for HF cache in model2vec by @Jeadie in #7076
• chore: upgrade to Rust 1.88 by @kczimm in #7077
• Kafka/Debezium: make common errors user-friendlier by @sgrebnov in #7084
• Create Apache Datafusion upgrade issue template by @kczimm in #6800
• No join predicate pushdown on empty results by @Jeadie in #7075
• Bump tract-onnx from 0.21.10 to 0.22.0 by @app/dependabot in #7071
• Bump mongodb from 3.2.4 to 3.3.0 by @app/dependabot in #7073
• Bump indicatif from 0.17.11 to 0.18.0 by @app/dependabot in #7070
• Bump actions/github-script from 7 to 8 by @app/dependabot in #7069
• Bump actions/setup-go from 5 to 6 by @app/dependabot in #7068
• Bump actions/download-artifact from 4 to 5 by @app/dependabot in #7066
• Bedrock: Tool use without inputs must empty Document by @Jeadie in #7036
• Bump github.com/stretchr/testify from 1.10.0 to 1.11.1 by @app/dependabot in #7015
• Bump actions/setup-python from 5 to 6 by @app/dependabot in #7067
• Upgrade dependabot dependencies by @phillipleblanc in #7061
• Bump tempfile from 3.20.0 to 3.21.0 by @app/dependabot in #7018
• Only call 'list_datasets' once, after initial system/user messages by @Jeadie in #7039
• Bump github.com/spf13/pflag from 1.0.7 to 1.0.10 by @app/dependabot in #7062
• Bump actions/checkout from 4 to 5 by @app/dependabot in #7065
• Bump golang.org/x/mod from 0.27.0 to 0.28.0 by @app/dependabot in #7064
• Bump github.com/AzureAD/microsoft-authentication-library-for-go from 1.4.1 to 1.5.0 by @app/dependabot in #7063
• Add friendly message for Kafka operation timeout error, improve code by @sgrebnov in #7088
• embed UDF by @mach-kernel in #6967
• fix: Update benchmark snapshots by @app/github-actions in #7097
• Fix SF100 benchmark tests dispatch by @sgrebnov in #7098
• chore(logging): add log when iceberg rest catalog fails with ssl cert error by @ChrisTomAlxHitachi in #6909
• Add xxhash support for search/sql results by @krinart in #6978
• Use proper federation in max_timestamp_df during acceleration refresh by @krinart in #7055
• Fix spiced_docker workflows for new actions/download-artifact@v5 behavior by @phillipleblanc in #7108
• Fix spiced_docker workflow by @phillipleblanc in #7111
• Add filter for zero vectors before writing to S3 Vectors by @phillipleblanc in #7110
• Ensure we find vector index when it also has text search by @Jeadie in #7120
• Enable unified traceparent override support for HTTP API by @sgrebnov in #7122
• Fix ORDER BY: (BytesProcessedExec to avoid pruning ordered execs during physical optimization) by @mach-kernel in #7105
• Fix spiced_docker_nightly workflow by @sgrebnov in #7125
• Add output_level to runtime config by @krinart in #7119
• Add tests for xxhash hashers by @krinart in #7124
• Add input option to update snapshots in Integration tests by @Jeadie in #7127
• Fix formatting to improve merges by @lukekim in #7128
• Add tests to nulling logic by @Jeadie in #7113
• Bump chrono from 0.4.41 to 0.4.42 by @app/dependabot in #7131
• Bump ctrlc from 3.4.7 to 3.5.0 by @app/dependabot in #7132
• Search: RRF UDTF by @mach-kernel in #7090
• Update openapi.json by @app/github-actions in #7141
• Bump packages to DF49; resolve incompatibilities by @Jeadie in #7101
• fix: Don't error for chunked columns when vectors are disabled by @peasee in #7150
• Allow bzip2-1.0.6 license in deny.toml by @Jeadie in #7148
• Tune retry settings for Kafka/Debezium connectors by @sgrebnov in #7142
• Update TEI by @Jeadie in #7152
• Use twox-hash version 2.1.2 by @krinart in #7165
• Revert "Use proper federation in max_timestamp_df during acceleration refresh (#7055)" by @phillipleblanc in #7156
• Bump octocrab from 0.44.1 to 0.45.0 by @app/dependabot in #7158
• Bump github.com/spf13/viper from 1.19.0 to 1.21.0 by @app/dependabot in #7130
• Bump keyring from 3.6.2 to 3.6.3 by @app/dependabot in #7157
• fix: Remove keywords from AI document search by @peasee in #7052
• Bump tract-core from 0.21.10 to 0.22.0 by @app/dependabot in #7134
• Update TEI by @Jeadie in #7171
• Update openapi.json by @app/github-actions in #7172
• fix: Ensure vector search UDTF respects the supplied projection by @peasee in #7155
• Bump clap from 4.5.45 to 4.5.47 by @app/dependabot in #7135
• Bump golang.org/x/sys from 0.35.0 to 0.36.0 by @app/dependabot in #7129
• Include 'catalog_id' in Glue catalog parameters by @Jeadie in #7151
• fix: Use head ref from merge group event in pulls-with-spice concurrency group by @peasee in #7175
• Fix lint for xxhash feature by @phillipleblanc in #7176
• Add Kafka-specific metrics for consumer lag and consumed records by @sgrebnov in #7146
• Kafka: persist consumer between restarts with SQLite and PG acceleration by @sgrebnov in #7177
• Kafka: support specifying a target consumer group ID by @sgrebnov in #7178
• Fix timestamp parsing for spice trace by @krinart in #7173
• Support full-text indexing on CDC/append streams by @phillipleblanc in #7180
• Bump iceberg-rust version to include limit push down by @krinart in #7191
• Make full text stream connector more robust by @phillipleblanc in #7193
• fix: Update benchmark snapshots by @app/github-actions in #7179
• Initial changes for SearchIndex by @Jeadie in #7103
• Robustly handle indexing FTS for CDC streams by @phillipleblanc in #7197
• Proper handling/mapping for ThrottlingException during embedding calls by @krinart in #7170
• Add spicepod.yml by @lukekim in #7202
• Delta Lake: Support read pruning on timestamp columns using maxValues stats by @sgrebnov in #7203
• feat: Add initial embeddings cache by @peasee in #7194
• Make S3vector a FixedSizeListArray by @Jeadie in #7201
• Fix projection mismatch issues with RRF calling vector search / text search by @mach-kernel in #7200
• feat: Add embeddings cache to all embeddings by @peasee in #7204
• Revert "Make S3vector a FixedSizeListArray (#7201)" by @kczimm in #7210
• Update duckdb version to make ICU statically linked by default by @krinart in #7215
• Change DataType list nullability from true to false by @Jeadie in #7216
• Use Instant + saturating_sub to handle time drift by @krinart in #7212
• Flatten 'IndexedTableProvider' when adding full-text support by @Jeadie in #7219
• Include comments in pulls by @lukekim in #7224
• Add github_max_concurrent_connections = 5 by @lukekim in #7225
• RRF: Fix scoring by @mach-kernel in #7226
• Update RRF search integration snapshots after scoring change by @mach-kernel in #7227
• Make S3vector a FixedSizeListArray by @Jeadie in #7230
• Proper federation during acceleration refresh + datafusion version bump + integration tests by @krinart in #7228
• Use DuckDBDialect for DuckDB non-federated queries by @krinart in #7232
• Move chunking out of llms and into new crate chunking by @Jeadie in #7229
• Remove duplicate pg_port configuration in test by @lukekim in #7233
• Upgrade to Rust 1.89 by @phillipleblanc in #7235
• Catalog connection error: fix connector name from 'iceberg' to 'spice.ai' by @sgrebnov in #7240
• Create PutVectorsSink by @kczimm in #7199
• Benchmark tests: fix API key reference in spicecloud catalog by @sgrebnov in #7239
• Add Dotnet SDK sample to end game template by @sgrebnov in #7238
• Update spicepod.schema.json by @app/github-actions in #7254
• Postgres: Improve Decimals read performance and add Name type support by @sgrebnov in #7255
• Add tests for hybrid search on a vector engine by @Jeadie in #7220