Skip to main content

STATE_STORE_INVALID_AVRO_SCHEMA error condition

SQLSTATE: 3F000

Detected an invalid schema for Avro encoding.

ADDED_AND_REMOVED_COLUMN_SIMULTANEOUSLY

Added columns <addedColsStr> and removed columns <removedColsStr> in column family <colFamilyName>, in the same query evolution. To allow this schema evolution, set the spark.sql.streaming.join.schemaEvolution.disallowAddDrop to false, but be aware that this will drop accumulated state. If trying to rename a column without losing accumulated state, consider keeping the same query and adding a projection.

ADDED_NON_NULLABLE_FIELDS

Added non nullable fields after a schema evolution: <nonNullableFields>, in column family: <colFamilyName>. Note that newly added fields must be nullable to allow schema evolution with Avro encoding, consider adding nullable fields only.

DUPLICATE_NAME_IN_SCHEMA

Detected a duplicate name: "<invalidName>". Note that field names with Avro encoding must all be unique.

GENERIC

Please review the field names used in the query, and the following Avro encoding exception thrown.

ILLEGAL_CHAR_IN_NAME

Detected an illegal name: "<invalidName>". Note that field names with Avro encoding must start with either a letter or underscore, and can only be followed by alphanumeric characters or underscores.