What's New in v20.2.0-alpha.2

July 27, 2020

Get future release notes emailed to you:

Thanks!

Downloads

Docker image

$ docker pull cockroachdb/cockroach-unstable:v20.2.0-alpha.2

Backward-incompatible changes

• The SHOW RANGE FOR ROW statement now takes a tuple of the row's index columns instead of the full column set of the row. #50479
• Previously, issuing a SIGTERM signal twice or after another signal initiated a hard shutdown for a node. Now the first SIGTERM signal initiates a graceful shutdown and further occurrences of SIGTERM are ignored. To initiate a hard shutdown, issue SIGINT two times (or issue a SIGINT signal once after a SIGTERM signal). #50539
• Specifying the same option multiple times in the WITH clause of the BACKUP statement now results in an error message. Additionally, quoted option names are not allowed anymore. #50723
• For expression typing involving only operations on constant literals, each constant literal is now assigned a type before calculation. Previously, a type was assigned only to the final result. #50254

Security updates

• HTTP endpoints beginning with /debug/ now require a valid admin login session. #50483

General changes

• Go 1.14 is now the minimum required version necessary to build CockroachDB. #50671
• The docker image available with CockroachDB now includes the GEOS files required to work with Geospatial types in CockroachDB. #50362
• Schema changes are now logged in greater detail. #50372
• Added new options to YCSB to mirror OpenSource YCSB options. #50546
• Links returned in error messages that point to unimplemented issues now use the CockroachLabs redirect/short-link server. #49836

Enterprise edition changes

• The default flush interval for changefeeds that do not specify a RESOLVED option is now 5s instead of 200ms to more gracefully handle higher-latency sinks. #49770
• Full cluster restore is now more resilient to transient transaction retry errors while it restores. #49950
• RESTORE now remaps user-defined types within the backup to existing types in the cluster. #50349
• BACKUP can now run in a new detached mode, which means the Job ID is now returned immediately without waiting for the BACKUP job to finish. #50775

SQL language changes

Geospatial changes

• Implemented the following builtins: ST_Transform, ST_Intersection, ST_PointOnSurface, and ST_Union, ST_Segmentize({geometry, float8}), ST_Project({geography,float8,float8}), ST_LongestLine({geometry, geometry}), ST_Azimuth({geometry,geometry}), ST_ShortestLine({geometry, geometry}), ST_Centroid ({geometry,bool}) functions. #49783, #49833, #49827, #49949, #50018, #50188, #50292, #50484
• Implemented the following builtins: GeometryType, PostGIS_AddBBox, PostGIS_DropBBox, PostGIS_Extensions_Upgrade, PostGIS_Full_Version, PostGIS_GEOS_Version, PostGIS_HasBBox, PostGIS_LibXML_Version, PostGIS_Lib_Build_Date, PostGIS_Lib_Version, PostGIS_Liblwgeom_Version, PostGIS_PROJ_Version, PostGIS_Scripts_Build_Date, PostGIS_Scripts_Installed, PostGIS_Scripts_Released, PostGIS_Version, PostGIS_Wagyu_Version. #50589
• Added support for casting GEOGRAPHY and GEOMETRY types to and from Bytes/JSONB. #50577
• Implemented ST_AsGeoJSON with options to show bbox and CRS information. #49888
• Introduced maxDecimalDigits arguments for ST_AsText and ST_AsEWKT, which allow rounding of the decimal digits output in the WKT representation. #49875
• Populated the spatial_ref_sys table with support SRID entries for geospatial data types. #49887
• Implemented the ST_Azimuth functionality for the geography operator. #50708
• GEOMETRY now coerce invalid geography coordinates into correct geometry. #50457
• Added support for AddGeometryColumn and other functions that mutate schema in IMPORT PGDUMP. #50850

General SQL updates

• For Postgres compatibility, CockroachDB now supports the ANALYZE <tablename> statement that collects statistics on the given table for use by the optimizer. The statement is functionally equivalent to the existing CREATE STATISTICS statement. #49816
• The vectorized execution engine now supports the JSONFetchVal(->) operator, the Pow operator, the Bitand (&), Bitor (|), Bitxor (^), FloorDiv (//), and Mod (%) binary operators, and the binary shift (>> and <<) operators. #49818, #49761, #50143, #50417
• The vectorized execution engine now supports AVG and SUM aggregate functions on Ints and Intervals. #49900
• The vectorized execution engine now fully supports the SUM aggregate function (previously, the summation of integers was incomplete). #49900
• The vectorized execution engine now supports the MIN and MAX aggregate functions on columns of INT2 and INT4 types. #49900
• The optimizer can now remove an unnecessary join from the input of a Project operator. #49788
• The optimizer can now fold two Limit operators together. #49931
• Improved performance for queries with a limit on a join that is guaranteed to preserve input rows. #49802
• Added the crdb_internal.databases virtual table. #49522
• Referencing types across databases has been disabled. #49841
• Views now only create a dependency on a table's column if the column is referenced in the view definition. Previously, all columns in a table referenced in a view were added to the view's dependencies. #49872
• The pgdate util can now parse db2 formatted timestamps. This can be used to import db2 dumps. #50011
• For some EXPLAIN variants, the property distributed for the output variable Field has been renamed as distribution and can have local, partial, or full values. The partial state is currently only possible with the experimental setting. #49857
• Added support for the ALTER COLUMN TYPE ... USING EXPRESSION statement to allow users to optionally provide an expression to convert a column to a new type. #49452
• Enhanced the optimizer's ability to reorder joins. #49948
• Enabled aggregate queries which collect key/value pairs into a JSON object. #48306
• Removed the optimizer_foreign_keys and experimental_optimizer_foreign_key_cascades session settings (and their corresponding cluster settings). #50185
• RegClass expressions are now tracked as dependencies in views. For example, CREATE VIEW v AS SELECT 't'::regclass; will now add a dependency on table t for view v. #50213
• Sequences passed as a string argument into views are now tracked as a dependency. For example, CREATE VIEW v AS SELECT nextval('s') will now add a dependency on sequence s. #50103
• Fixed an error for the ALTER COLUMN TYPE statement for situations where the data matches byte for byte but CockroachDB needs to validate the INT8 -> INT4 conversion. #50278
• Previously, infinity evaluated to a negative value, -292277022365-05-08T08:17:07Z. Now infinity is the maximum supported timestamp in Postgres that is not infinity. Likewise, -infinity is the smallest supported value. Note that this work-in-progress feature currently does not behave exactly like infinity in Postgres. #50311
• The CHAR column type will now truncate the trailing space characters in line with Postgres. Existing stored CHAR entries with spaces at the end of the CHAR column type will no longer return rows with trailing space characters. Use the LIKE query to find and modify these rows. #50492
• Disabled arrays in non-inverted indexes. #50662
• Enabled dropping indexes used in outbound foreign key relationships. #50769
• CockroachDB no longer writes to the slow query log unless explicitly enabled. #50898
• IMPORT INTO now supports columns with constant default expressions, and non-targeted columns with constant default expressions are no longer required to be nullable. #50295

Command-line changes

• Advanced troubleshooting information (such as internal details from the Go runtime) is now saved in a separate log file named <program>-stderr.xxx.log in the configured log directory. #48051
• The server CLI commands (start, start-single-node, demo) now support a new flag named --redactable-logs, which introduces markers in generated log entries to facilitate redaction of sensitive information by cockroach debug zip and other CockroachDB APIs. This is an experimental feature currently not enabled by default, but is recommended for all deployments and will be enabled by default in a later version. Note: the flag is advertised (in --help) by cockroach start and cockroach start-single-node, where log files are expected as a matter of course; it is also supported, yet hidden (from --help), by every other CLI command because every command can also produce log entries when passed --log-dir or --logtostderr. #48051
• The commands cockroach debug zip and cockroach debug merge-logs support a new flag --redact-logs. When specified, it causes log messages that potentially contain confidential data or PII to be redacted away. This feature is experimental. Note: Currently, this redaction is extremely aggressive and may hinder the ability of Cockroach Labs to investigate issues from log files, especially if --redactable-logs is not enabled server-side. Other potentially-confidential data may still be retrieved by cockroach debug zip; the new flag only applies to retrieved log files inside the output zip file. #48051
• cockroach node decommission --wait=live is no longer supported. It was deprecated in an earlier release. #49666
• cockroach quit is now deprecated. Use the subcommands for cockroach node instead, and use termination signals to quit the cockroach process. #49668
• Allowed cockroach dump to dump tables and databases that contain user-defined types. #49808
• The commands cockroach node decommission and cockroach node recommission now recognize a new flag named --self. This can be passed instead of an explicit list of node IDs, and indicates that the operation targets the node the command is connected to (either via --host if specified, or localhost). #50532
• The command-line flag --log-dir-max-size is now deprecated and has been replaced with a new flag named --log-group-max-size. The flags limit the combined size of all files generated by one logging group inside CockroachDB. #50599

DB Console changes

• CockroachDB now preserves the selected tab on the Statements page after reload. #48090
• Fixed Statements page crash on database names with /. #47423
• The Statements page now shows if the vectorized execution engine is used or not. #50255
• Updated the empty state design for the Statement diagnostics history page. #46921
• Updated the tooltip designs on the Overview. #46919
• Updated the Storage Dashboard. #47523
• Fixed icon alignments for the Timeframe navigation buttons. #50010
• The Queues dashboard in the DB Console no longer shows a Compaction Queue graph, and the Queue Processing Failures and Queue Processing Times graphs no longer include the Compaction queue metrics. #50245
• Updated the Login page design. #46783
• Added a tooltip for the Stats by Node metric. #50348
• Changed the label for std deviation from < to > for positive values. #50275
• Updated the message for the Node logs page when remote debugging is restricted for users by using the server.remote_debugging.mode cluster setting. #49553
• The sidebar navigation panel no longer displays the Network Latency menu for a single-node cluster. #50269
• Aligned the Units selector and Remove Chart buttons on the same line. Search input for metric name now properly shows the search term. #50334
• Added a collection of new metrics to track the method of requests sent as RPCs between cluster nodes. These metrics are named distsender.rpc.<method>.sent, with examples of <method> being scan and conditionalput. #46747
• The time range selector has been extended with a 30 minutes option for all Metrics. #50810

Bug fixes

• Previously, when doing a series of table creations and drops, subsequent incremental backups ran into an error. This is now fixed. #49776
• The description provided with --help on the CLI for --socket-dir and --socket has been fixed. They were invalid since v20.1.0. #49905
• Previously, in some cases, CockroachDB didn't check whether the right argument of the Div (/), FloorDiv (//), or Mod (%) operations was zero, so instead of correctly returning a division by zero error, CockroachDB returned NaN, and this is now fixed. Additionally, the error message of modulus by zero has been changed to division by zero to be inline with Postgres. #49761
• Fixed an internal planning error that occurred for recursive CTEs (WITH RECURSIVE expressions) in which the left side of the UNION ALL query used in the CTE definition produced zero rows. #49961
• CockroachDB now correctly handles import cancellation and errors. #49979
• CockroachDB now correctly links producer/consumer during data import in order to handle errors correctly. #49995
• extract(epoch from timestamptz) from a session time zone not in UTC would previously return a value incorrectly offset by the session time zone. This has now been rectified. #50071
• Previously, when streaming values from a column declared of type char(n), the length of the value could be <= n. Now all values streamed have length exactly n by padding spaces to the right if necessary. #49886
• Casting to width-limited strings now works correctly for strings containing Unicode characters. #50156
• Fixed some cases where casting a string to a width-limited string array was not truncating the string. #50153
• Previously, if a full cluster restore failed while restoring the system table data, it would not clean up properly and leave some temporary tables public and not dropped. This is now fixed. #50003
• The Statements page of the DB Console previously displayed incorrect information about whether or not the statement execution was distributed. #50256
• Fixed a bug where an unexpected EOF error would be returned at startup with the Pebble storage engine, if the last instance of cockroach crashed in a rare case of the write-ahead log being written to. #50280
• Fixed a recently introduced no volatility for cast unknown::tuple error. #50436
• CockroachDB now correctly populates the rolvaliduntil value for roles in pg_roles and pg_authid. #50307
• Fixed a bug affecting some DROP DATABASE schema changes where multiple GC jobs are created, causing the GC job for the database to fail. GC jobs will no longer fail upon failing to find a table descriptor already deleted by a different GC job. #50411
• Fixed an internal error when referencing a type that does not exist by ID. #50286
• Lookup join predicate now shows proper column names in EXPLAIN. #50441
• The result of interpreting relative timestamps (like now, today) is no longer stored in cached plans. #50320
• Renaming a temporary table no longer converts it to a persistent table. The table continues to remain temporary after a rename. Moreover, users can no longer move a temp table into the public schema by providing a fully qualified name while renaming the table. #50500
• A server node now properly dumps Go stacks to its log file for troubleshooting upon receiving SIGQUIT during node shutdown. Previously, SIGQUIT was only recognized for this purpose while the node had not started shutting down already. #50539
• Previously performing ALTER TABLE..ADD COLUMN with a computed expression allowed users to provide a column in a different table or database if the column name was the same. Similarly, users could provide an expression referring to another table's column when executing ALTER COLUMN TYPE USING EXPRESSION. This is no longer the case and all columns must refer to the column in the table being altered. #50582
• Fixed a bug where creating a sequence owned by a table's column and dropping the sequence would make it impossible to drop the table after. #50665
• Fixed a bug where Pebble iteration would skip over blocks that failed checksum verification rather than propagating the error to the caller. #50697
• Some pg_catalog queries that previously returned an error like crdb_internal_vtable_pk column not allowed now work again. #50682
• Fixed internal errors in some queries involving CASE expressions with NULL branches. #50626
• Fixed a rare bug where a multi-Range SELECT FOR UPDATE statement containing an IN clause could fail to observe a consistent snapshot and violate serializability. #50633
• Fixed a bug where a cluster restore would fail when the largest descriptor in the backup was a database. This was typically seen when the last action in backing up cluster was a database creation. #50759
• Fixed an internal error during planning for some queries with aggregate functions embedded in complex scalar expressions. #50815
• Fixed a bug where a badly timed power outage or system crash could result in an error upon process restart. #50831
• Fixed a bug where a cluster backup would appear as BACKUP TABLE TO rather than BACKUP TO in the jobs table. #50760
• Fixed the column not in input internal error for some corner cases. #50845
• DROP DATABASE CASCADE now works as expected even when the database has a sequence with an owner in it. #50744

Performance improvements

• Introduced a new flag named server.consistency_check.max_rate expressed in bytes/second to throttle the rate at which CockroachDB scans through the disk to perform a consistency check. This control is necessary to ensure smooth performance on a cluster with large node sizes (i.e. in the 10TB+ range) #49763
• Fixed a performance regression in the SHOW DATABASES command introduced in 20.1. #49522
• Limited SELECT statements now do a better job avoiding unnecessary contention with UPDATE and SELECT FOR UPDATE statements. #49980
• PGCOPY import is now ~3.5x faster by parallelizing the conversion of raw input data to Datums. #50016
• Allowed the optimizer to use enum information to generate better query plans. #49284
• Range merges are now delayed for a short time after load-based splitting to prevent load-based split points from being merged away immediately after load is removed. #50151
• Transaction liveness pushes are now delayed by 50ms, up from 10ms. This allows high contention workloads to sustain high throughput up to much larger concurrency levels. #50161
• Improved sstable files organization in the Pebble storage engine, which should significantly improve import performance. #50371
• Improved the optimizer's estimation of the selectivity of some filters involving a disjunction (OR) of predicates over multiple columns. This results in more accurate cardinality estimation and enables the optimizer to choose better query plans in some cases. #50456
• Many queries that involve now() are executed more efficiently; in particular, the result of now() can be used with an index. #50320
• Better execution plans in some cases involving EXISTS. #50846
• Smoothed out disk writes when transferring Range snapshots to avoid latency spikes for other concurrent operations. #50866

Contributors

This release includes 446 merged PRs by 53 authors. We would like to thank the following contributors from the CockroachDB community:

• Andrii Vorobiov
• Artem Barger
• Arun Ranganathan (first-time contributor)
• George Papadrosou
• Jaewan Park
• John Kendall (first-time contributor, CockroachDB team member)
• Jordan Ryan Reuter (first-time contributor, CockroachDB team member)
• Madeline Liao (first-time contributor, CockroachDB team member)
• Robert S Lee (first-time contributor)
• Ryan Min (first-time contributor, CockroachDB team member)
• Tancredo Souza (first-time contributor)
• Yongyang Lai
• abhishek20123g
• gorjunov (first-time contributor)
• jieniu$ (first-time contributor)

Was this page helpful?

Yes No