DuckDB 2.0 Alpha: What's New for CSV and JSON Processing

DuckDB v2.0 "Cyanoptera" entered alpha on September 2, 2026, with a full release expected in October. The preview post lists ten headline features, and the alpha announcement invites testing on CLI and Python clients.

You do not need to install it to benefit from the ideas. Two of the v2.0 themes matter directly for everyday CSV and JSON work: fast semi-structured JSON via VARIANT and new JSON reconcile functions for diffing, merging, and deduping. This post explains both, then shows the no-install browser equivalent on RividTech. For background, start with Run SQL on CSV in Your Browser and Flatten Nested JSON for Spreadsheets and CSV.

Why v2.0 matters for file people, not just database people

DuckDB made its name as an in-process analytical engine that can SELECT * FROM 'orders.csv' without a server. v2.0 pushes that further:

• DuckDB as a server (Quack + CONNECT) — any DuckDB can serve over the network. Interesting for teams, but you still do not need it for a one-off file.
• Asynchronous I/O — Parquet first, then CSV and native storage. Remote scans from S3 get much more parallel. Local CSV parsing also benefits.
• Faster queries — aggregates pushed below joins, rewritten recursive CTEs (40x faster on a 1M-edge graph benchmark), partition-aware planning for lakehouse formats, stronger row-group pruning.
• New PEG SQL parser, new storage format, native timezones without ICU — faster, smaller, better errors.

If you only process CSV, TSV, and JSON files, the two JSON items below are the ones to learn.

VARIANT: JSON on steroids

VARIANT shipped in v1.5 and becomes first-class in v2.0. Think of it as fast JSON: each row can have different shape like JSON, but DuckDB detects shared structure and "shreds" it for compression and vectorized execution — no schema declaration required.

In v2.0 the pipeline works end to end: shredded execution from storage, extraction pushdown into scans, shredded VARIANT read/write for Parquet, and variant_* functions:

CREATE TABLE events (payload VARIANT);
INSERT INTO events
VALUES ('{"user": {"id": 42, "tags": ["a", "b"]}}'::JSON::VARIANT);

SELECT variant_type(payload), variant_keys(payload)
FROM events;

SELECT *
FROM events
WHERE variant_contains(payload, {'user': {'id': 42}}::VARIANT);

Longer term the regular JSON type will likely be backed by VARIANT, so existing JSON queries get the speedup for free.

Browser takeaway: you already hit the same problem when an API returns uneven records — sparse columns, optional nested fields. The RividTech equivalent is Flatten JSON into a table, then JSON Preview to inspect shape. Background in Flatten Nested JSON and JSONL / NDJSON Explained.

New in v2.0: four JSON reconcile functions

The JSON patch post (guest post by Atlan's Mustafa Khan) adds four scalars you can try in the v2.0-dev preview with LOAD json;. They solve pipeline problems that previously forced you out of SQL:

1. json_merge_patch_diff(orig, modified) — compute the minimal patch

Inverse of RFC 7396 json_merge_patch. Unchanged keys are omitted, removed keys become null, changed/added keys carry new values:

SELECT json_merge_patch_diff('{"a":1,"b":2,"c":3}', '{"a":1,"b":99,"d":4}');
-- {"c":null,"b":99,"d":4}

It recurses, so only changed paths appear:

SELECT json_merge_patch_diff(
  '{"user":{"name":"Alice","age":30}}',
  '{"user":{"name":"Alice","age":31}}'
);
-- {"user":{"age":31}}

Use it for CDC-style pipelines: ship diff(prev, next) downstream instead of the full document, reconstruct with json_merge_patch.

2. json_deep_merge(doc, patch, ...) — null means "skip"

Same recursion as json_merge_patch, except null in the patch keeps the original instead of deleting the key. That matches how upstreams emit fragments with unknown fields set to null:

SELECT json_merge_patch('{"a":1,"b":2}', '{"b":null}');
-- {"a":1}  (RFC 7396: null deletes)

SELECT json_deep_merge('{"a":1,"b":2}', '{"b":null}');
-- {"a":1,"b":2}  (skip-on-null: keeps b)

Variadic, left to right:

SELECT json_deep_merge(
  '{"columnName":"user_id","parentColumn":null}',
  '{"columnName":null,"parentColumn":"accounts.id"}'
);
-- {"columnName":"user_id","parentColumn":"accounts.id"}

Note: SQL NULL argument vs JSON null value behave differently — SQL NULL patch yields NULL, SQL NULL original is ignored.

3. json_normalize(doc) — canonical form for hashing

Recursively sorts object keys (arrays keep order) so semantically equal documents hash equal:

SELECT json_normalize('{"z":1,"a":2,"m":3}');
-- {"a":2,"m":3,"z":1}

SELECT
  md5(json_normalize('{"z":1,"a":2}')) =
  md5(json_normalize('{"a":2,"z":1}')) AS same_hash;
-- true

GROUP BY md5(json_normalize(payload)) collapses duplicates that differ only in key order — common when two services describe the same entity.

4. json_strip_nulls(doc) — recursively drop null-valued keys

Removes null-valued keys from objects (array elements untouched):

SELECT json_strip_nulls('{"a":1,"b":null,"d":2}');
-- {"a":1,"d":2}

Clean patches before storing: json_strip_nulls(json_merge_patch_diff(orig, modified)) keeps adds/updates but never deletes. Same idea trims verbose API responses before hashing.

v2.0 also adds SQL/JSON edit primitives json_set, json_insert, json_replace, json_remove.

Benchmarks in the post on 500,000 synthetic CDC events (Apple M3 Pro): 10–123x faster than equivalent Python json.loads + transform + json.dumps — json_strip_nulls 123.4x, json_normalize 46.8x, full composed chain ~10.8x and ~1 second total.

Do the same steps in your browser — no install

Installing the alpha is worth it if you live in DuckDB. For a one-off file, especially a sensitive one, the browser equivalent is faster and private:

• Preview shape first. JSON Preview / CSV Preview plus JSON Formatter catches the killers: multi-line JSONL, trailing commas, one bad line in 100k.
• Flatten. Flatten JSON pulls user.address.city and arrays into columns, then JSON to CSV or JSON to Excel for tables. Reverse with CSV to JSON — safely, see How to Convert CSV to JSON Safely.
• Normalize and dedupe. After flattening, null vs missing vs "" shows up as sparse columns. Normalize with Column Tools and Find & Replace, dedupe on ID with Remove Duplicates — the browser version of json_normalize + hash dedupe.
• Validate and count. Validate CSV for ragged rows and bad emails/URLs/numbers, Data Stats for uniques and null rates.
• SQL-style filter/join/pivot. Data Process for WHERE, Pivot Table for GROUP BY, Merge CSV for JOIN, Diff CSV to compare snapshots — full mapping in SQL on CSV in Browser.
• Generate load scripts. CSV to SQL / JSON to SQL / TSV to SQL for Postgres, MySQL, SQLite.

Everything runs on your device — see Why Browser-Only CSV Tools Are Safer and our Privacy Policy.

Getting started

Try the mental model on your next API dump: preview, flatten, normalize nulls, dedupe on hash/ID, validate. Open Flatten JSON, convert with JSON to CSV, and check the result in Data Stats. If the file is JSONL, follow JSONL / NDJSON Explained; if you need shell automation, compare with jq for JSON Processing and Tools to Process CSV Files in the Terminal.