What's New in v2.2.0-alpha.20190211

February 11, 2019

In addition to SQL language enhancements, general usability improvements, performance improvements, and bug fixes, this release includes several major highlights:

• Follower Reads: Enterprise users can now reduce read latencies by allowing queries to perform historical reads of the closest replica of a given piece of data rather than reading from the more distant "leaseholder" replica. To enable follower reads on a query, use the experimental_follower_read_timestamp() built-in function in conjunction with the AS OF SYSTEM TIME clause.
  
• Cost-Based Optimizer: The cost-based optimizer now supports almost all read-only queries (except window functions) and almost all mutations (e.g., CREATE TABLE AS, INSERT, UPDATE, UPSERT, DELETE). In addition, the cost-based optimizer now reorders up to 4 joins in a query to attempt to find the most performant ordering (via the new experimental_reorder_joins_limit session variable) and takes advantage of automatic statistics without impacting foreground traffic. Note that statistics are created by default on all indexed columns when a user upgrades to this version. Finally, a query plan cache now saves a portion of the planning time for frequent queries used in the cost-based optimizer.
• Change Data Capture: Enterprise users can now create CHANGEFEEDs that deliver table updates as JSON files to cloud storage endpoints like Google Storage or AWS S3. In addition, all CockroachDB users can now use the core implementation of change data capture, via the new EXPERIMENTAL CHANGEFEED FOR statement, to consumes table updates over a streaming Postgres connection. Finally, all CHANGEFEEDS now default to using a new "push" mechanism called rangefeeds to deliver data with increased reliability and lower latency.

Get future release notes emailed to you:

Thanks!

Downloads

Docker image

$ docker pull cockroachdb/cockroach-unstable:v2.2.0-alpha.20190211

Backward-incompatible changes

• The CHANGEFEED experimental_avro format is now backward- and forward-compatible with adjacent schemas for the same table. #34095

General changes

• Go 1.11.4 is now the minimum required version necessary to build CockroachDB. #33668
• Increased the maximum length of queries in crash reports, to make debugging easier. #34479

Enterprise edition changes

• CHANGEFEEDs now experimentally support writing to cloud storage, for easy use with analytics databases. #33647 #34193
• The CHANGEFEED envelope=row option is now deprecated and will be removed in the Fall 2019 release. The default envelope for new changefeeds is now wrapped. #34309
• CHANGEFEEDs now operate on an end-to-end "push" model, reducing latency of row changes. Some workloads will also see fewer transaction restarts on tables being watched by CHANGEFEEDs. #34457
• Added support for standard HTTP proxy environment variables in HTTP and S3 storage. #34067
• Added support for performing sufficiently old historical reads against the closest replicas rather than leaseholders as well as a new experimental_follower_read_timestamp() built-in function, which can be used with AS OF SYSTEM TIME clauses to generate a timestamp that is likely to be safe for reads from a follower. #33478

SQL language changes

• VALIDATE CONSTRAINT is now compatible with the new MATCH FULL and MATCH SIMPLE foreign key semantics and is more performant. #34365
• Table data is now validated against a newly added CHECK constraint asynchronously after the transaction commits. #32504
• NULL values are now supported in int and text arrays in the driver protocol. #33675
• CockroachDB now supports transmitting bit array values using the decimal encoding in the low-level client protocol. #34050
• It is now possible to force a reverse scan of a specific index using table@{FORCE_INDEX=index,DESC}. #34075
• Improved the output of EXPLAIN for index-join and lookup-join. #34138
• FILTER expressions are now supported by the cost-based optimizer. #34077
• EXPLAIN (OPT) now has a much shorter output. EXPLAIN (OPT,VERBOSE) and EXPLAIN (OPT,TYPES) can be used for more verbose output. #34128
• Using a sequence as a SELECT target is now supported by the cost-based optimizer. #33196
• Removed the 2.0-off and 2.0-auto modes for the sql.defaults.distsql cluster setting. All queries are now run via the newer, distributed SQL engine; queries are still only distributed if appropriate. #34163
• The experimental_force_lookup_join session variable has been removed. #34142
• Added the experimental_reorder_joins_limit session variable, which defaults to 4 and causes the cost-based optimizer to reorder up to 4 joins in a query to attempt to find the most performant ordering. This behavior can be disabled per-session by setting the experimental_reorder_joins_limit session variable to 0. #34549
• Formatting of timestamps as JSON strings has been changed to always use the RFC3339 format instead of Cockroach's customary format. Users can now expect to see a T separator instead of a space between the date and time components. #34412
• Introduced a new top-level statement for an experimental version of CHANGEFEED that doesn't require an enterprise license and that returns results as a stream over the sql connection. #34386
• The result buffer size can now be controlled on a per-connection basis with the results_buffer_size connection string parameter. #34385
• CREATE STATISTICS now runs as a job instead of as a regular SQL statement. #34279
• INT values are now stored with microsecond precision instead of nanoseconds. Existing intervals with nanoseconds are no longer able to return their nanosecond part. An existing table t with nanoseconds in intervals of column s can round them to the nearest microsecond with UPDATE t SET s = s + '0s'. Note that this could potentially cause uniqueness problems if the interval is a primary key. #34202
• Added support for AS OF SYSTEM TIME clauses in BEGIN TRANSACTION and SET TRANSACTION statements, which enables entire read-only transactions to be run against a historical timestamp. This functionality simplifies performing complex analytics against a consistent snapshot of historical data and eases the burden to use historical reads with ORMs which generally make modifying the syntax of generated SELECT statements difficult. #34305
• The behavior of the now() built-in function inside of historical SELECT ... AS OF SYSTEM TIME queries now reflects the historical timestamp at which the query is being run rather than the current clock time when the statement is executed. #34305
• The ORDER BY clause can no longer be used with a DELETE statement when there is no LIMIT clause present. Sorting the output should instead be done using SELECT ... FROM [DELETE ...] ORDER BY .... #34303
• Enabled automatic statistics collection. #34529
• DELETE, UPDATE, and UPSERT statements are now planned by the cost-based optimizer. #34522
• The value of information_schema.columns.character_maximum_column is set to NULL for all integer types, for compatibility with PostgreSQL. #34182

Command-line changes

• The modified time is now set for entries in cockroach debug zip output. #33714
• Clarified the informational message printed upon running cockroach start --join. #33435

Admin UI changes

• Added a debug endpoint listing the hottest ranges by QPS on each node/store. #33336
• Improved performance of graph detail tooltips when viewing long timespans (e.g., 1 month) #34032
• CHANGEFEED metrics are now exposed in the UI. #34427

Bug fixes

• Fixed a bug in RESTORE that prevented restoring some BACKUPs containing previously dropped or truncated interleaved tables. #34413
• Fixed a bug in cockroach node status that prevented it from displaying down nodes in the cluster in some circumstances. #34448
• Fixed several related panics in the optimizer related to plan exploration. #34667
• Resolved a cluster degradation scenario that could occur during IMPORT/RESTORE operations, manifested through a high number of pending Raft snapshots. #33582
• Fixed a bug where some comparison operations with constant inputs were not getting folded during query optimization, causing the optimizer to produce sub-optimal plans. #33597
• Window functions with non-empty PARTITION BY and ORDER BY clauses are now handled correctly when invoked via the low-level client protocol. #33591
• Fixed a memory leak around DEALLOCATE and DISCARD statements that could result in panics with the unexpected <amount> leftover bytes message. #33423
• Lookup joins now properly preserve their input order even if more than one row of the input corresponds to the same row of the lookup table. #33536
• Fixed a panic that occurred when performing an INSERT ON CONFLICT with a SET UPDATE that uses values from a subquery. #33553
• Preparing queries with missing placeholders (e.g., SELECT $2::int) now results in an error. #33716
• Fixed a goroutine leak that would occur while a cluster was unavailable (or a subset of nodes partitioned away from the cluster) and would cause a resource spike to resolve. #33282
• Fixed panics or incorrect results in some cases when grouping on constant columns (either with GROUP BY or DISTINCT ON). #34123
• Prevented down-replicating widely replicated ranges when nodes in the cluster are temporarily down. #34126
• Fixed a panic when an internal implementation error prevents proper handling of placeholders (query parameters). #34134
• CockroachDB now enables re-starting a node at an address previously allocated for another node. #34155
• The values reported in information_schema.columns for integer columns created prior to CockroachDB v2.1 as BIT are now fixed and consistent with other integer types. #34182
• CockroachDB 2.2-alpha releases can once again be built from source on FreeBSD (unsupported platform). #34244
• Fixed a back up in flow creation observed by "no inbound stream connection" caused by not releasing a lock before attempting a possibly blocking operation. #34218
• CHANGEFEEDs now can be started on tables that have been backfilled by schema changes. #34317
• Fixed a possible panic in crdb_internal.pretty_key(). #34480
• CHANGEFEEDs with changefeed.push.enabled set to true now resolve timestamps in the presence of inactive ranges. #34550
• Fixed a panic when updating a job that doesn't exist. #34574
• Fixed a panic due to incorrect statistics calculations when all values of a column are NULL. #34578
• Fixed a bug where lease transfers passed through Snapshots could forget to update in-memory state on the new leaseholder, allowing write-skew between read-modify-write operations. #34548

Performance improvements

• Reduced the network and storage overhead of multi-range transactions. #33566
• A query plan cache now saves a portion of the planning time for frequent queries. #34454
• Transaction record garbage collection requests are now batched on a per range basis to reduce the number of Raft entries in a high-throughput, write-heavy, transactional workload. #34242

Doc updates

• The new Life of a Distributed Transaction details the path that a query takes through CockroachDB's architecture, starting with a SQL client and progressing all the way to RocksDB (and then back out again). #4281
• Added a warning about cross-store rebalancing not working as expected in 3-node clusters with multiple stores per node. #4320
• Updated the INT documentation to include examples of actual min/max integers supported by each type for easier reference. Also added a description of possible compatibility issues caused by 64-bit integers vs., for example, JavaScript runtimes. #4317
• Documented the sql.metrics.statement_details.plan_collection.period cluster setting, which controls how often the logical plan for a fingerprint is sampled (5 minutes by default) on the Statements page of the Admin UI. #4316
• Added guidance on removing UNIQUE constraints. #4276
• Added a note that when a table that was previously split is truncated, the table must be pre-split again. #4274
• Updated the SQL Performance Best Practices with caveats around interleaving tables. #4273

Contributors

This release includes 258 merged PRs by 29 authors. We would like to thank the following contributors from the CockroachDB community:

• George Utsin (first-time contributor)
• Txiaozhe (first-time contributor)
• Vijay Karthik

Was this page helpful?

Yes No