What's New in v21.1.0-alpha.2

February 1, 2021

Get future release notes emailed to you:

Downloads

Docker image

icon/buttons/copy 
$ docker pull cockroachdb/cockroach-unstable:v21.1.0-alpha.2

Backward-incompatible changes

  • The payload fields for certain event types in system.eventlog have been changed and/or renamed. Note that the payloads in system.eventlog were an undocumented, reserved feature so no guarantee was made about cross-version compatibility to this point. The list of changes includes (but is not limited to):
    • TargetID has been renamed to NodeID for node_join.
    • TargetID has been renamed to TargetNodeID for node_decommissioning / node_decommissioned / node_recommissioned.
    • NewDatabaseName has been renamed to NewDatabaseParent for convert_to_schema.
    • grant_privilege and revoke_privilege have been removed; they are replaced by change_database_privilege, change_schema_privilege, change_type_privilege, and change_table_privilege. Each event only reports a change for one user/role, so the Grantees field was renamed to Grantee.
    • Each drop_role event now pertains to a single user/role. #57737
  • The connection and authentication logging enabled by the cluster settings server.auth_log.sql_connections.enabled and server.auth_log.sql_sessions.enabled was previously using a text format which was hard to parse and integrate with external monitoring tools. This has been changed to use the standard notable event mechanism, with standardized payloads. The output format is now structured; see the generated reference documentation for details about the supported event types and payloads. #57839
  • The logging format for SQL audit, execution, and query logs has changed from a crude space-delimited format to JSON. To opt out of this new behavior and restore the pre-v21.1 logging format, you can set the cluster setting sql.log.unstructured_entries.enabled to true. #59110

Security updates

  • The entropy of the auto-generated user account password for cockroach demo shell has been lowered, to ensure that the passwords are short and easy to copy over. This makes the password easy to brute-force and thus removes any remaining protection there may have been to the confidentiality of demo sessions. (The reason why a password remains, as opposed to no password whatsoever, is to prevent accidental misuse of the HTTP service by other applications running on the same machine.) Since the required shortness erases any pretense of security, the algorithm has been simplified to remove usage of a cryptographic PRNG altogether. #58305
  • When using a SQL proxy, by default CockroachDB only knows about the network address of the proxy. That peer address is then used for logging, authentication rules, etc. This is undesirable, as security logging and authentication rules need to operate on the actual (final) client address instead. CockroachDB can now be configured to solve this problem (conf mechanism detailed below). When so configured, a SQL proxy can inform the CockroachDB server of the real address of the client via a server status parameter called crdb:remote_addr. The value must be the IP address of the client, followed by a colon, followed by the port number, using the standard Go syntax (e.g., 11.22.33.44:5566 for IPv4, [11:22::33]:4455 for IPv6). When provided, this value overrides the SQL proxy's address for logging and authentication purposes. In any case, the original peer address is also logged alongside the client address (overridden or not), via the new logging tag peer. Security considerations:
    • Enabling this feature allows the peer to spoof its address with regard to authentication and thus bypass authentication rules that would otherwise apply to its address, which can introduce a serious security vulnerability if the peer is not trusted. This is why this feature is not enabled by default, and must only be enabled when using a trusted SQL proxy.
    • This feature should only be used with SQL proxies which actively scrub a crdb:remote_addr parameter received by a remote client, and replaces it by its own. If the proxy mistakenly forwards the parameter as provided by the client, it opens the door to the aforementioned security vulnerability.
    • Care must be taken in host-based authentication (HBA) rules:
    • TLS client cert validation, if requested by a rule, is still performed using the certificate presented by the proxy, not that presented by the client. This means that this new feature is not sufficient to forward TLS client cert authentication through a proxy. (If TLS client cert authentication is required, it must be performed by the proxy directly.)
    • The protocol field (first column) continues to apply to the connection type between CockroachDB and the proxy, not between the proxy and the client. Only the 4th column (the CIDR pattern) is matched against the proxy-provided remote address override. Therefore, it is not possible to apply different rules to different client address when proxying TCP connections via a unix socket, because HBA rules for unix connections don't use the address column. Also when proxying client SSL connections via a non-SSL proxy connection, or proxying client non-SSL connections via a SSL proxy connection, care must be taken to configure address-based rule matching using the proper connection type. A reliable way to bypass this complexity is to only use the host connection type which applies equally to SSL and non-SSL connections. As of this implementation, the feature is enabled using the non-documented environment variable COCKROACH_TRUST_CLIENT_PROVIDED_SQL_REMOTE_ADDR. The use of an environment variable is a stop-gap so that this feature can be used in CockroachCloud SQL pods, which do not have access to cluster settings. The environment variable will be eventually removed and replaced by another mechanism. #58381
  • Added ability to set region-specific callback URLs in the OIDC config. The server.oidc_authentication.redirect_url cluster setting can now accept JSON as an alternative to the basic URL string setting. If a JSON value is set, it must contain a redirect_url key that maps to an object with key-value pairs where the key is a region matching an existing locality setting and the value is a callback URL. #57519

General changes

  • CockroachDB now runs fewer threads in parallel if running inside a container with a CPU limit. #57390
  • Upgraded the CockroachDB binary to Go 1.15.6. #57713
  • The new cluster setting server.eventlog.enabled controls whether notable events are also written to the table system.eventlog. Its default value is true. Changing this cluster setting can help recovering partial cluster availability when the system.eventlog table becomes unavailable. Note that even when false, notable events are still propagated to the logging system, where they can be redirected to files or other log sinks. #57879
  • Moved RBAC resources in Kubernetes manifests to rbac.authorization.k8s.io/v1. #55065
  • Cluster version upgrades, as initiated by SET CLUSTER SETTING version = <major>-<minor>, now perform internal maintenance duties that will increase the time that it takes for the command to complete. This delay is proportional to the amount of data currently stored in the cluster. The cluster will also experience a small amount of additional load during this period while the upgrade is being finalized. These changes expand upon our original prototype in #57445. #58088
  • Upgraded to v3.9.2 of protobuf to consume new changes for Bazel. #58891
  • Increased the default value of sql.tablecache.lease.refresh_limit to 500 in order to accomodate larger schemas without encountering RETRY_COMMIT_DEADLINE_EXCEEDED errors and generally higher latency. #59319

Enterprise edition changes

  • CockroachDB now permits partitioning by user-defined types such as enums. #57327

SQL language changes

  • Added a new builtin method to calculate voronoi polygons. #56496
  • Added a new builtin method to calculate voronoi lines. #56496
  • Previously, timezones had to be entered in the same case as they were stored on the system. Now, timezone names can be case-insensitive provided they match well-known zone names according to Go's time/tzdata package. #57250
  • Added LOCALITY REGIONAL BY TABLE IN PRIMARY REGION syntax for configuring table locality. #57275
  • The default value for vectorize_row_count_threshold setting has been decreased from 1000 to 0, meaning that, by default, CockroachDB will always use the vectorized engine for all supported queries regardless of the row estimate (unless vectorize=off is set). #55713
  • Added the ability to CREATE TABLE with LOCALITY set. #57419
  • EXPLAIN and EXPLAIN ANALYZE now show counts and durations in a more user-friendly way. #57420
  • EXPORT can now be used by non-admin users with SELECT privilege on the table being exported, unless the destination URI requires admin privileges. #57380
  • CREATE DATABASE ... WITH [PRIMARY] REGIONS now modifies the zone configurations with the settings as set by the multiregion configuration. #57244
  • SHOW [BACKUP] is no longer admin-only. It depends on the URI construct and the credentials specified in the SHOW [BACKUP] query. #57412
  • builtin aggregate function regr_avgx is now supported. #57295
  • Added support for running IMPORT in a mixed-version cluster. #57382
  • Introduced SHOW REGIONS FROM ALL DATABASES, which shows region metadata for each database in the table. #57500
  • Added collations table to the information_schema. The collations table contains the collations available in the current database. #57609
  • Added collation_character_set_applicability table to the information_schema. #57609
  • LOCALITY directives on CREATE TABLE are now persisted onto the database. #57513
  • Multi-region defined table localities now display on the SHOW CREATE TABLE command. #57513
  • Arrays in pg_catalog tables now are diaplayed in a format that matches PostgreSQL. #56980
  • The SET TRACING local option (to only trace messages issued by the local node) was removed. #57567
  • Several SHOW commands have been adjusted to enforce a particular ordering of the output rows. #57644
  • Implemented regr_avgy builtin aggregation function. #57583
  • crdb_internal.tables and SHOW TABLES now show locality data on the tables. #57653
  • CREATE TABLE ... LOCALITY ... statements now modify the zone configurations to be in line with the desired locality. #57654
  • Added sql.trace.stmt.enable_thresold. Similar to sql.trace.txn.enable_threshold, this logs a trace of any statements that take longer than the specified duration. This new setting allows for more granularity than the transaction threshold since it applies to individual statements and does not include overhead such as communication with a SQL client. #57733
  • Added a session setting experimental_enable_unique_without_index_constraints and cluster default sql.defaults.experimental_enable_unique_without_index_constraints.enabled to enable the use of UNIQUE WITHOUT INDEX syntax. The default value of both settings is false since this feature is not yet fully supported. Use of UNIQUE WITHOUT INDEX also depends on all nodes being upgraded to the cluster version UniqueWithoutIndexConstraints. #57666
  • Implemented builtin function levenshtein. #56843
  • Added table-view dependency information in pg_depend to improve compatibility with PostgreSQL. #57240
  • The cluster event logging system has been modernized. In particular, the schema of the entries for the info column in system.eventlog has been stabilized. #57737
  • The targetID and reportingID columns in system.eventlog are now deprecated. Their values, for relevant event types, can be found as fields inside the info column instead. #57737
  • Added the soundex() and difference() builtin functions. #57615
  • New EXPLAIN ANALYZE (DISTSQL) output format. EXPLAIN ANALYZE (DISTSQL) now only works as a top-level statement (it can no longer be part of a bigger query). #57804
  • Added builtin function ST_OrientedEnvelope to calculate the mimimum-area rotated rectangle. #57697
  • A new default_transaction_use_follower_reads session variable is now supported, which configures SQL transactions to perform stale reads from follower replicas. #57851
  • Both ALTER TABLE OWNER and REASSIGN OWNED BY now report structured notable events about the ownership changes. Note that REASSIGN OWNED BY currently also reports an alter_table_owner for views and sequences that were implicitly reassigned, even though CockroachDB does not yet support the ALTER VIEW OWNER / ALTER SEQUENCE OWNER statements. #57969
  • EXPLAIN (DISTSQL) has a new output schema and format. #57954
  • Added an overload to crdb_internal.pb_to_json to suppress populating default values in fields. #58087
  • IMPORT INTO for CSV now supports nextval as a default expression of a non-targeted column. #56473
  • The EXPLAIN output for foreign key checks is now labeled constraint-check rather than fk-check. This change is in preparation for adding support for unique constraint checks, which will use the same label. #58053
  • The error message for unique constraint violations now matches the error used by PostgreSQL. For example, the new error message looks like this: ERROR: duplicate key value violates unique constraint "primary" DETAIL: Key (k)=(1) already exists. #58053
  • EXPLAIN ANALYZE diagrams now contain "network hops" information on streams. #58078
  • Added support for the IF NOT EXISTS prefix to CREATE TYPE statements. #58173
  • Added a session_variables table to the information_schema. The session_variables table exposes the session variables. #57837
  • Indexing into scalar JSON values using an integer index is now properly supported. #58359
  • The crdb_internal.cluster_id method now returns the ID of the underlying KV cluster in multi-tenant scenarios rather than the Nil UUID. The ClusterID is needed for logging and metrics for SQL tenants. #58317
  • SHOW STATEMENTS is now an alias of SHOW QUERIES. The new syntax is preferred by the SQL parser. #58072
  • Implemented the gateway_region builtin, which returns the region of the connection's current node. #58423
  • BACKUP and RESTORE now use the block-level checksums embedded in their data files instead of collecting / verifying more expensive file-level SHA512 checksums. #58487
  • Multi-tenant clusters will now send anonymous usage information to the central CockroachDB registration server. #58399
  • Added support for ALTER DATABASE ... ADD REGION. #58233
  • Multiple SHOW ZONE CONFIGURATION statements that previously used FOR can now also use FROM (e.g., SHOW ZONE CONFIGURATION FOR RANGE). This change standardizes the use of FROM for SHOW ZONE CONFIGURATION. #58740
  • Implemented SHOW REGIONS, which shows a list of regions along with the databases associated with them. #57618
  • Added ALTER DATABASE ... PRIMARY REGION. #58725
  • Columns implicitly added from hash-sharded indexes will no longer display on pg_index and pg_indexes. #58749
  • Added a new implicit column to crdb_internal.index_columns, which signifies whether the column is implicitly added onto the index through PARTITION BY with an implicit column or a hash-sharded index. #58749
  • CockroachDB now omits implicit columns or hash-sharded index columns from automatically generated index names. #58898
  • Implemented PARTITION ALL BY syntax for CREATE TABLE, which automatically partitions the table, and all indexes, with the same partitioning scheme. #58748
  • PARTITION ALL BY tables now display correctly on SHOW CREATE TABLE. #58928
  • Creating a table and changing its schema within a transaction no longer schedules a schema change job. #58888
  • Implemented geo builtin function ST_GeneratePoints. #58288
  • Hash aggregation can now spill to disk when it exhausts its memory limit when executed via the vectorized engine. #57817
  • Implemented ALTER DATABASE ... SURVIVE REGION/ZONE FAILURE. #58937
  • CREATE INDEX will now inherit PARTITION BY clauses from PARTITION ALL BY tables. #58988
  • Implemented the geometry-based builtin functions ST_LineSubstring({geometry,float8,float8}). #58125
  • Implemented REGIONAL BY ROW logic on CREATE TABLE. This is gated behind the experimental session variable experimental_enable_implicit_column_partitioning. #58987
  • Added support for ALTER VIEW / SEQUENCE OWNER TO commands. #59049
  • Casts to type unknown[] are no longer accepted in CockroachDB. Any such casts will fail to parse and return the error ERROR: type "unknown[]" does not exist. This is consistent with PostgreSQL's behavior. #59136
  • Implemented REGIONAL BY ROW AS ..., which allows a column of type crdb_internal_region to be used as a column for REGIONAL BY ROW multi-region properties. #59121
  • Enabled altering of table locality for tables that are REGIONAL BY TABLE to REGIONAL BY TABLE (but in a different region). #59144
  • PARTITION ALL BY statements will now apply for tables when using ALTER PRIMARY KEY. #59178
  • Implemented the geometry builtin function ST_ShiftLongitude, which is useful for plotting data on a Pacific-centred map. #59234
  • Implemented ALTER TABLE ... SET LOCALITY GLOBAL for tables starting as REGIONAL BY ROW. #59256
  • Improved the error message when people use set search_path incorrectly, or with a schema that legitimately has a comma in its name. #53974
  • Creation of interleaved tables and indexes is now disabled by default. They can be re-enabled temporarily by running SET CLUSTER SETTING sql.defaults.interleaved_tables.enabled=true. #59248
  • CockroachDB now uses a structured logging format for the SQL audit, execution, and query logs. See the generated reference documentation for details. Of note, audit and execution logs now also include information about whether a query plan contains full index scans. Previously, this information was only included in the slow query log. #59110
  • CREATE INDEX on REGIONAL BY ROW tables will now correctly include the implicit partitioning and inherit the correct zone configurations. #59223
  • Made a minor improvement to EXPLAIN output for "render" node. #59313
  • EXPLAIN ANALYZE now defaults to the new EXPLAIN ANALYZE (PLAN), which shows a text representation of the logical plan, annotated with execution statistics. #57337

Command-line changes

  • The logging configuration can now be specified via the --log parameter. See the documentation for details. The flags --log-dir, --log-file-max-size, --log-file-verbosity, --log-group-max-size are now deprecated. #57134
  • A new command cockroch debug check-log-config prints out the logging configuration that results from the provided combination of --store, --log, and other logging-related flags on the command line. #57134
  • The events that were previously only stored in system.eventlog are now also directed unconditionally to an external logging channel using a JSON format. Refer to the configuration to see how to customize how events are directed to external sinks. Note that the exact external output format (and thus how to detect/parse the events from e.g. log files) is not yet stabilized and remains subject to change. #57737
  • Notable events that pertain to SQL schema, user, and privilege changes are now sent on the new SQL_SCHEMA, USER_ADMIN, and PRIVILEGES logging channels. These can now be redirected to different sinks from the other log entries.
    • The SQL_SCHEMA channel is used to report changes to the SQL logical schema, excluding privilege and ownership changes (which are reported on the separate channel PRIVILEGES) and zone config changes (which go to OPS). This includes:
      • Database, schema, table, sequence, view, and type creation.
      • Adding, removing, and changing table columns.
      • Changing sequence parameters.
      • More generally, changes to the schema that affect the functional behavior of client apps using stored objects.
    • The USER_ADMIN channel is typically configured in "audit" mode, with event numbering and synchronous writes. It is used to report changes in users and roles, including:
      • Users added and dropped.
      • Changes to authentication credentials, including passwords, validity, etc.
      • Role grants and revocations.
      • Role option grants and revocations.
    • The PRIVILEGES channel is typically configured in "audit" mode, with event numbering and synchronous writes. It is used to report data authorization changes, including:
      • Privilege grants and revocations on database, objects, etc.
      • Object ownership changes. #51987
  • Logging events that are relevant to cluster operators are now categorized under the new OPS and HEALTH logging channels. These can now be redirected separately from other logging events.
    • The OPS channel is used to report "point" operational events, initiated by user operators or automation:
      • Operator or system actions on server processes: process starts, stops, shutdowns, and crashes (if they can be logged). Each event includes any command-line parameters and the CockroachDB version.
      • Actions that impact the topology of a cluster: node additions, removals, decommissions, etc.
      • Cluster setting changes
      • Zone configuration changes.
    • The HEALTH channel is the channel used to report "background" operational events, initiated by CockroachDB for reporting on automatic processes:
      • Current resource usage, including critical resource usage.
      • Node-node connection events, including connection errors and gossip details.
      • Range and table leasing events.
      • Up- and down-replication.
      • Range unavailability. #57171
  • Server terminations that are triggered when a node encounters an internal fatal error are now reported on the OPS logging channel. The exact text of the error is not reported on the OPS channel, however, as it may be complex (e.g., when there is a replica inconsistency); and the OPS channel is typically monitored by tools that just detect irregularities. The text of the message refers instead to the channel where the additional details can be found. #57171
  • The notable events set_zone_config and remove_zone_config are now sent to the OPS logging channel. #57171
  • Added a flag to cockroach debug decode-proto to suppress populating default values in fields. #58087
  • When a SQL notable (structured) event is logged, the payload now attempts to include the session's application_name as field ApplicationName. This is intended for use when setting up event routing and filtering in external tools. #58130
  • It is now possible to set the format parameter of any log sink, including file sinks, to json, json-compact, json-fluent, or json-fluent-compact to write entries as structured JSON. Refer to the generated reference documentation for details. #58126
  • The DB Console URL printed by cockroach demo now automatically logs in the user when pasted into a web browser. #56740
  • The URLs generated when cockroach demo starts have been made shorter and easier to copy/paste by shortening the generated password. #58305
  • When using the JSON output formats for log entries, the server identifiers are now reported as part of each payload once known (either cluster ID + node ID in single-tenant or KV servers, or tenant ID + SQL instance ID in multi-tenant SQL servers). #58128
  • Fixed cockroach demo --global from crashing with didn't get expected magic bytes header. #58466
  • Previously, for certain log files, CockroachDB would both flush individual writes (i.e. propagate them from within the cockroach process to the OS) and also synchronize writes (i.e. ask the OS to confirm the log data was written to disk). Per-write synchronization was found to be unnecessary and possibly detrimental to performance and operating cost, so it was removed. Now, the log data continues to be flushed, as before, and CockroachDB only requests synchronization periodically (every 30s). #58995
  • The parameter sync-writes for file sink configurations has been removed. (This is not a backward-incompatible change because the configuration feature is new in v21.1.) #58995
  • The parameter buffered-writes for file sink configurations has been added. It is set to true (writes are buffered) by default and set to false (i.e. avoid buffering and flush every log entry) when the auditable flag is requested. #58995
  • The default output format for file-group and stderr sinks has been changed to crdb-v2. This new format is non-ambiguous and makes it possible to reliably parse log files. Refer to the format's documentation for details. Additionally, it prevents single log lines from exceeding a large size; this problem is inherent to the crdb-v1 format and can prevent cockroach debug zip from retrieving v1 log files. The new format has also been designed so that existing log file analyzers for the crdb-v1 format can read entries written the new format. However, this conversion may be imperfect. Refer to the new format's documentation for details. In case of incompatibility, users can force the previous format by using format: crdb-v1 in their logging configuration. #59087

API endpoint changes

  • The notable event create_statistics is only reported when the cluster setting sql.stats.post_events.enabled is enabled. This fact is now also reported in the event log reference documentation. #57877
  • The Timestamp field of structured notable events is now numeric, and encodes a number of nanoseconds since the Unix epoch. #58070
  • The Health API now checks that the SQL server is ready to accept clients when a readiness check is requested. #59350

DB Console changes

  • Minor style changes to represent new branding palette. #57130
  • Changed the default per-page value on the Transactions page to 20; minor style updates. #57824
  • Added a timeseries graph indicating statement denials due to feature flags on the SQL Dashboard of DB Console. #57533
  • Updated labels on the Hardware Dashboard to be more accurate. #57224
  • Updated the link back to Statements from Statement Details so that it will always link to the Statements list instead of invoking the "back" action on the browser. #57975
  • On the Sessions page, every age label has been replaced with duration and every txn label has been replaced with transaction. The actual metrics remain unchanged. #58616
  • Renamed the CPUs metric column to vCPUs in the Node List on the Cluster Overview page. #58495
  • Added Open SQL Transactions to the SQL Dashboard and renamed SQL Queries to SQL Statements. Removed references to "Distributed SQL Queries" when metrics really are discussing all SQL queries. #57477

Bug fixes

  • Fixed a bug in RESTORE where some unusual range boundaries in interleaved tables caused an error. #58219
  • Previously, CockroachDB would encounter an internal error when performing a JSONB - String operation via the vectorized execution engine. This has been fixed. The bug was introduced in v20.2.0. #57349
  • Previously, in rare situations, an automated replication change could result in a loss of quorum. This was possible when a cluster had down nodes and a simultaneous change in replication factor. Note that a change in the replication factor can occur automatically if the cluster is comprised of fewer than five available nodes. Experimentally the likelihood of encountering this issue, even under contrived conditions, was small. #56735
  • Fixed an internal error when using aggregates and window functions in an ORDER BY for a UNION or VALUES clause. #57498
  • DROP TYPE and certain other statements that work over SQL scalar types now properly support type names containing special characters. #57354
  • Fixed a performance regression introduced in v20.2 to reading virtual tables which introspect the schema. #57542
  • Removed system.jobs full table scan that is expensive in the face of many completed jobs. #57587
  • In v20.2.0 we mistakenly permitted users with the admin role to drop tables in the system database. This commit revokes that privilege. #57568
  • Previously, users could not perform a cluster restore from old backup chains (incrementals on top of fulls) when using the BACKUP INTO syntax. #57656
  • Fixed a bug that could cause IMPORT to incorrectly read files stored on Google Cloud if uploaded using its compression option (gsutil -Z). #57745
  • Fixed a bug where ST_MakeLine and ST_Collect did not respect ordering when used over a window clause. #57724
  • Fixed a bug where schema change jobs to add foreign keys to existing tables, via ALTER TABLE, could sometimes not be successfully reverted (either due to being canceled or having failed). #57598
  • Fixed a bug where concurrent addition of a foreign key constraint and drop of a unique index could cause the foreign key constraint to be added with no unique constraint on the referenced columns. #57598
  • Adding a primary key constraint to a table without a primary key no longer ignores the name specified for the primary key. #57146
  • Previously, when displaying replica counts during the decommissioning process, we were overcounting the replicas displayed for r1. This is no longer the case. #57812
  • Fixed a bug whereby tables in schemas other than public would not be displayed when running SHOW TABLES FROM <db>. #57749
  • Fixed a bug where canceled queries reading from virtual tables could cause a crashing panic. #57828
  • Fixed an assertion error caused by some DDL statements used in conjunction with common table expressions (WITH). #57927
  • Previously, SHOW GRANTS ON DATABASE did not include privileges that were granted on a database. Now it does. The SHOW GRANTS ON DATABASE output no longer includes a column for schema_name, as these grants are not specific to any schema. #56866
  • The information_schema.schema_privileges table now includes the correct schema-level privileges for non-user-defined schemas. Previously, all of these schemas were ommitted from the table. #56866
  • The has_schema_privilege builtin function now works on user-defined schemas when checking for the USAGE privilege. #56866
  • The ST_FrechetDistance builtin function no longer causes a SIGFPE panic for very small values of densifyFrac (e.g., 1e-20), and returns an error instead. #57966
  • CockroachDB now removes a node's status entry when the node is decommissioned, to prevent it from appearing in API calls and UIs, and prevent it from affecting node constraints such as localities and attributes for various operations. #56529
  • Fixed a bug which caused type information to be omitted when decoding descriptors using either crdb_internal.pb_to_json or cockroach debug decode-proto. #58087
  • Fixed a bug from v21.1.0-alpha.1 where the binary could crash if a running node lost its claim to a job while updating. #58161
  • Fixed a bug where multiple invocations of AddGeometryColumn affecting the same table would result in only the last invocation applying. #56663
  • Previously, CockroachDB could return non-deterministic output when querying the information_schema.statistics virtual table (internally used by SHOW INDEXES)—namely, the implicit columns of the secondary indexes could be in arbitrary order. This is now fixed, and the columns will be in the same order as they are in the primary index. #58191
  • Fixed a column family 0 not found crash caused by explaining or gathering statement diagnostics on certain queries involving virtual tables. #58208
  • Added a safeguard against crashes while running SHOW STATISTICS USING JSON, which is used internally for statement diagnostics and EXPLAIN ANALYZE (DEBUG). #58221
  • Fixed a bug where prior schema changes on a table that failed and could not be fully reverted could prevent the table from being dropped. #57836
  • Previously, CockroachDB could crash when performing a DELETE operation after an alteration of the primary key when in some cases. This is now fixed. The bug was introduced in v20.1. #58153
  • Fixed a bug that could cause incremental backups to a backup in a collection (i.e., BACKUP INTO ... IN ...) on some cloud storage providers to ignore existing incremental backups previously appended to that destination and instead backup incrementally from the base backup in that destination. #58292
  • Fixed an internal panic when using the SHOW STATISTICS USING JSON statement on a table containing ENUM types. #58251
  • A memory leak in the optimizer has been fixed. The leak could have caused unbounded growth of memory usage for a session when planning queries on tables with partial indexes. #58306
  • Fixed a bug where primary key changes on tables being watched by changefeeds would be silently ignored. #58140
  • The pg_catalog metadata tables were using the CHAR data type for single-byte character columns. Now they use the char data type, to match PostgreSQL. This resolves errors that would occur when using some drivers (like tokio-postgres for Rust) to access pg_catalog tables in the binary query format. #58084
  • Prepared statements that include enums and use the binary format will no longer result in an error. #58043
  • Fixed an internal error that could occur when executing prepared statements with placeholders that delete data from columns referenced by a foreign key with ON DELETE CASCADE. #58431
  • Fixed a bug which caused errors when querying a table with a disjunctive filter (an OR expression) that is the same or similar to the predicate of one of the table's partial indexes. #58434
  • Previously, in event log entries for ALTER TYPE and DROP TYPE statements, the TypeName field did not contain fully qualified names. #58257
  • A CREATE TABLE statement with indexes with duplicate names will no longer result in an assertion failure. This bug was present since v20.2. #58433
  • Previously, event logs were not capturing the qualified table names for COMMENT ON INDEX and COMMENT ON COLUMN commands. This PR changes the event logs to use the qualified table name. Tests were also added for these changes. #58472
  • The has_${OBJECT}_privilege built-in methods such as has_schema_privilege now additionally check whether the roles of which a user is a direct or indirect member also have privileges on the object. Previously only one user was checked, which was incorrect. This bug has been present since v2.0 but became more prominent in v20.2 when role-based access control was made available in the core version of CockroachDB. #58254
  • CockroachDB previously would return an internal error when attempting to execute a hash join on a JSON column via the vectorized engine. Now a more user-friendly error is returned. #57994
  • Fixed a panic in protobuf decoding. #58716
  • The user authentication flow no longer performs extraneous name lookups. This performance regression was present since v20.2. #58671
  • CockroachDB could previously return an internal error when evaluating a binary expression between a Decimal and an Interval that required a cast to a Float when the value is out of range. A more user-friendly error is now returned instead. #58743
  • Qualified table name for alter_table_owner event log. #58504
  • Fixed a bug that caused errors when accessing a tuple column (tuple.column syntax) of a tuple that could be statically determined to be null. #58747
  • The indoption column in pg_catalog.index is now populated correctly. #58947
  • Previously, CockroachDB could encounter an internal error when executing queries with tuples containing null values and enums in a distributed setting. This is now fixed. #58894
  • Fixed a nil pointer panic edge case in query setup code. #59002
  • Non-ASCII characters in NAME results in cockroach sql / demo (e.g., in the results SHOW TABLES, SHOW CONSTRAINTS) are now displayed without being escaped to octal codes. #56630
  • Garbage collection (GC) jobs now populate the running_status column for SHOW JOBS. This bug has been present since v20.1. #58612
  • Previously, CockroachDB could encounter an internal error when executing queries with BYTES or STRING types via the vectorized engine in rare circumstances. This is now fixed. #59028
  • Previously, the substring function on byte arrays would treat its input as unicode code points, which would cause the wrong bytes to be returned. Now it only operates on the raw bytes. #58265
  • Previously, the substring(byte[]) functions were not able to interpret bytes that had the \ character since it was being treated as the beginning of an escape sequence. This is now fixed. #58265
  • Fixed a bug in which some non-conflicting rows provided as input to an INSERT ... ON CONFLICT DO NOTHING statement could be discarded, and not inserted. This could happen in cases where the table had one or more unique indexes in addition to the primary index, and some of the rows in the input conflicted with existing values in one or more unique index. This scenario could cause the rows that did not conflict to be erroneously discarded. This has now been fixed. #59147
  • Fixed an internal error that could occur when ARRAY[NULL] was used in a query due to incorrect typing. ARRAY[NULL] is now typed as string[] if the type cannot be otherwise inferred from the context. This is the same logic that PostgreSQL uses, thus improving compatibility in addition to fixing the internal error. #59136
  • Fix a slow / hanging query that can be caused by using large max_decimal_digits on ST_AsGeoJSON. #59165
  • Queries that attempt to retrieve just the key columns of a single system table row will no longer return erroneous values. #58659
  • Fixed a bug in URL handling of HTTP external storage paths on Windows. #59216
  • Previously, CockroachDB could crash when executing ALTER INDEX ... SPLIT/UNSPLIT AT when more values were provided than are explicitly specified in the index. This has been fixed. #59213
  • Fixed a bug where multi-tenancy SQL pods would not successfully initialize the GEOS library. #59259
  • Placeholder values are now included alongside statements in structured events. #59110
  • Added qualification prefix for user-defined schema names in event logs. #58617
  • Added qualification prefix for dropped views in event logs. #59058
  • Parsing errors are no longer thrown when importing a pgdump file with array data. #58244
  • Fixed a crash when creating backup schedules writing to GCS buckets. #57617
  • CockroachDB now correctly exports schedules_BACKUP_* metrics as well as backup RPO metrics. #57488

Performance improvements

  • Previously, CockroachDB when performing an unordered DISTINCT operation via the vectorized execution engine would buffer up all tuples from the input, which is a suboptimal behavior when the query has a LIMIT clause. This has now been fixed. This behavior was introduced in v20.1. Note that the row-by-row engine doesn't have this issue. #57579
  • The query optimizer can use filters that constrain columns to multiple constant values to generate lookup joins. For example, a join filter x.a = y.a AND y.b IN (1, 2) can be used to generate a lookup join on table y assuming that it has an index on (a, b) or (b, a). #57690
  • The query optimizer now explores plans with lookup joins on partitioned indexes, resulting in more efficient query plans in some cases. #57690
  • Potentially improved performance for UPDATE statements where the table has computed columns that don't depend on updated columns. #58188
  • CockroachDB now allows the storage engine to compact sstables based on reads, on read-heavy workloads. #58247
  • Partial indexes with IS NOT NULL predicates can be used in cases where JOIN filters implicitly imply the predicate. This results in more efficient query plans for JOINs and foreign checks. #58204
  • Queries that use a geospatial inverted index can now take advantage of vectorized execution for some parts of the query plan, resulting in improved performance. #58241
  • Previously, indexed columns of partial indexes were always fetched for UPDATEs and UPSERTs. Now they are only fetched if they are required for maintaining the state of the index. If an UPDATE or UPSERT mutates columns that are neither indexed by a partial index nor referenced in a partial index predicate, they will no longer be fetched (assuming that they are not needed to maintain the state of other indexes, including the primary index). #58358
  • UPDATE operations on tables with partial indexes no longer evaluate partial index predicate expressions when it is guaranteed that the operation will not alter the state of the partial index. In some cases, this can eliminate fetching the existing value of columns that are referenced in partial index predicates. #58358
  • The sum_int aggregate function is now evaluated more efficiently in a distributed setting. #58345
  • INSERT ... ON CONFLICT ... DO NOTHING statements now use anti-joins for detecting conflicts. This simplifies the query plan for these statements, which may result in more efficient execution. #58679
  • Improved the accuracy of histogram calculations for the following types: string/uuid/inet family. Additionally, support for time/timetz histogram calculations was also added. This improves optimizer's estimates and results in better query plans in certain instances. #55797
  • The optimizer now uses collected histogram statistics to better estimate the cost of JSON and ARRAY inverted index scans, which may lead to more efficient query plans. #59326

Build changes

  • CockroachDB now builds on Ubuntu 20.10 and other distros using gcc-10. #58895

Doc updates

  • The types of logging sinks available through configuration are now automatically documented. #59083
  • The various output formats available for logging configurations are now documented. See the generated reference documentation for details. #58075
  • The cluster event logging system has been standardized. Reference documentation is now available (auto-generated from source code); changes to non-reserved payloads will now be announced at least one release version in advance. The event types are organized into broad categories: SQL Logical Schema Changes, SQL Privilege Changes, SQL User Management, CLuster-level events and SQL Miscellaneous operations. #57737
  • A report of the possible logging severities and channels is now automatically generated. #57134

Contributors

This release includes 615 merged PRs by 85 authors. We would like to thank the following contributors from the CockroachDB community:

  • Alan Acosta (first-time contributor)
  • ArjunM98
  • Azdim Zul Fahmi
  • Cheng Jing (first-time contributor)
  • Cyrus Javan
  • Erik Grinaker
  • Javier Fernandez-Ivern (first-time contributor)
  • Kumar Akshay (first-time contributor)
  • Maciej Rzasa (first-time contributor)
  • Marcin Knychała
  • Max Neverov
  • Miguel Novelo (first-time contributor)
  • Omar Bahareth (first-time contributor)
  • Petr Jediný
  • Ruixin Bao
  • Saif Al-Harthi (first-time contributor)
  • Vaibhav
  • Yanxin (first-time contributor)
  • b41sh (first-time contributor)
  • mosquito2333 (first-time contributor)
  • neeral
YesYes NoNo

Product

Resources

Learn

Support Channels

Company

Get developer news