STATE_STORE_INVALID_AVRO_SCHEMA error condition
Detected an invalid schema for Avro encoding.
ADDED_AND_REMOVED_COLUMN_SIMULTANEOUSLY
Added columns <addedColsStr>
and removed columns <removedColsStr>
in column family <colFamilyName>
, in the same query evolution. To allow this schema evolution, set the spark.sql.streaming.join.schemaEvolution.disallowAddDrop to false, but be aware that this will drop accumulated state. If trying to rename a column without losing accumulated state, consider keeping the same query and adding a projection.
ADDED_NON_NULLABLE_FIELDS
Added non nullable fields after a schema evolution: <nonNullableFields>
, in column family: <colFamilyName>
. Note that newly added fields must be nullable to allow schema evolution with Avro encoding, consider adding nullable fields only.
DUPLICATE_NAME_IN_SCHEMA
Detected a duplicate name: "<invalidName>
". Note that field names with Avro encoding must all be unique.
GENERIC
Please review the field names used in the query, and the following Avro encoding exception thrown.
ILLEGAL_CHAR_IN_NAME
Detected an illegal name: "<invalidName>
". Note that field names with Avro encoding must start with either a letter or underscore, and can only be followed by alphanumeric characters or underscores.