Skip to content

Conversation

@chenjian2664
Copy link
Contributor

@chenjian2664 chenjian2664 commented Jan 17, 2026

Description

Fixes the issue #27886 (comment) :

CREATE TABLE test (x int, y varchar) WITH (checkpoint_interval = 2, location = 's3://test-bucket/test', deletion_vectors_enabled = true);
INSERT INTO test VALUES (1, 'aa');
INSERT INTO test VALUES (1, 'bb');

CREATE OR REPLACE TABLE test (x int, y varchar) WITH (checkpoint_interval = 2, location = 's3://test-bucket/test');

SELECT * FROM default.test; -- Spark fails 

Follow the #27886 (comment) :

I find https://github.com/delta-io/delta/blob/master/PROTOCOL.md#table-features, it means we do wrote the illegal protocol entry.

I am not able to find the behavior of create or replace in the protocol, but I find https://github.com/delta-io/delta/blob/master/PROTOCOL.md#table-features-for-new-and-existing-tables, it says:

When upgrading an existing table to Reader Version 3 and/or Writer Version 7, the client should, on a best effort basis,
 determine which features supported by the original protocol version are used in any historical version of the table, and
 add only used features to reader and/or writer feature sets. The client must assume a feature has been used, unless it 
can prove that the feature is definitely not used in any historical version of the table that is reachable by time travel.
combined with https://github.com/delta-io/delta/blob/master/PROTOCOL.md#supported-features, my understanding is that no matter what's the version it is, if the procotol has readFeatures or writeFeatures, we should inherit it from original version, unless we pretty sure that all the historic data not use the feature at all.

which means we should keep the original read features and write features if we know they were used.

Additional context and related issues

Release notes

( ) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
(x) Release notes are required, with the following suggested text:

## Delta Lake
* Preserve table features on `CREATE OR REPLACE TABLE`. ({issue}`issuenumber`)

@cla-bot cla-bot bot added the cla-signed label Jan 17, 2026
@github-actions github-actions bot added the delta-lake Delta Lake connector label Jan 17, 2026
@ebyhr
Copy link
Member

ebyhr commented Jan 17, 2026

The CI failure is related to this change.

@chenjian2664 chenjian2664 force-pushed the jack/fix-protocol-and-create-or-replace branch from 3358586 to ae75b5e Compare January 18, 2026 01:05
Following the spec in the
https://github.com/delta-io/delta/blob/master/PROTOCOL.md#table-features,
we should write the `readerFeatures` when reader version is 3,
write the `writeFeatures` when writer version is 7.
@chenjian2664 chenjian2664 force-pushed the jack/fix-protocol-and-create-or-replace branch from ae75b5e to d24cf0f Compare January 18, 2026 01:48
@chenjian2664 chenjian2664 force-pushed the jack/fix-protocol-and-create-or-replace branch from 6915d7d to 47b0e39 Compare January 19, 2026 01:07
}

@Test(groups = {DELTA_LAKE_OSS, PROFILE_SPECIFIC_TESTS})
public void testCreateOrReplaceTableOnlyDiscardFeaturesOnTrino()
Copy link
Member

@ebyhr ebyhr Jan 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel that the test name doesn't match what the test actually verifies. Which assertion ensures "OnlyDiscardFeaturesOnTrino"?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

renamed to "testCreateOrReplaceTableWithOnlyFeaturesChangedOnTrino", I want to say: it tests the create or replace and with only discard a feature on Trino, do you have suggestions on the name?

@chenjian2664 chenjian2664 force-pushed the jack/fix-protocol-and-create-or-replace branch from 47b0e39 to bf52914 Compare January 19, 2026 07:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla-signed delta-lake Delta Lake connector

Development

Successfully merging this pull request may close these issues.

3 participants