1.50.7

Merged PRs

dolt

go-mysql-server

• 2898: [memo] variable to disable merge join
  Use SET @@GLOBAL.disable_merge_join = 1 to prevent join planning from every selecting a merge join. There will always be HASH_JOIN and INNER_JOIN alternatives.
• 2896: support for INSERT .. RETURNING, a postgres extension
• 2895: Bug fix: resolve column default expressions for ALTER COLUMN nodes
  When building plan nodes to handle altering a column's nullability or type, without respecifying the full column definition, the built node should ensure any column default expressions are resolved.
  This originally showed up as a panic in Doltgres, because Doltgres' TypeSanitzer found the UnresolvedColumnDefault instance and tried to invoke operations on it.
  Note that it's not currently possible to trigger this issue with Dolt or GMS because MySQL has a more limited syntax than Postgres for altering a column without respecfiying its full column definition, and Dolt/GMS does not support that syntax currently (e.g. ALTER TABLE t ALTER COLUMN SET VISIBLE).

Closed Issues

1.50.6

Merged PRs

dolt

• 9005: go/store/nbs: Fix perf regression in taking backups.
  1.50.0 introduced a perf regression which mainly effects taking backups to AWS or GCS. This is a partial fix which makes backups perform as before when they are going to a completely new destination. Incremental backups to an existing store are still slower. We continue to work on addressing the regression.
• 9003: Up server config default max conns from 100 to 1000
• 8994: Bug fix for TypeCompatibilityChecker to ignore ExtenedTypes
  The following query caused a panic in Doltgres, because DoltTable was using TypeCompatibilityChecker to test if the column type change was compatible. TypeCompatibilityChecker wasn't able to handle the ExtendedType instance and panic'ed. This change makes TypeCompatibilityChecker ignore extended types, since it doesn't know enough about them to determine if a type change is compatible or not.
• 8993: Bump golang.org/x/net in /go/gen/proto/dolt/services/eventsapi
  Bumps golang.org/x/net from 0.23.0 to 0.36.0.
  • Commits

  ---

  updated-dependencies:
  • dependency-name: golang.org/x/net dependency-type: indirect ...
• 8992: Remove unnecessary hashicorp/go-uuid dependency
  We are already using github.com/google/uuid for UUIDs. This dependency was accidentally introduced.
• 8990: Remove unnecessary hashicorp/go-uuid dependency
  We are already using github.com/google/uuid for UUIDs. I think this dependency was accidentally introduced.
  https://github.com/dolthub/dolt/blob/b7c0ddb4aac3702114c4e060651007d590a8be8c/go/go.mod#L24
  https://github.com/dolthub/dolt/blob/b7c0ddb4aac3702114c4e060651007d590a8be8c/go/go.mod#L71
• 8988: Test for INSERT ... ON DUPLICATE UPDATE with dolt_docs table
  Adds a test for the bug fix in dolthub/go-mysql-server#2892
• 8979: Bump golang.org/x/net from 0.23.0 to 0.36.0 in /go/gen/proto/dolt/services/eventsapi
• 8850: refactor: using clear to simplify the code
  This is a new built-in function added in the go1.21 standard library, which can make the code more concise and easy to read.

go-mysql-server

• 2895: Bug fix: resolve column default expressions for ALTER COLUMN nodes
  When building plan nodes to handle altering a column's nullability or type, without respecifying the full column definition, the built node should ensure any column default expressions are resolved.
  This originally showed up as a panic in Doltgres, because Doltgres' TypeSanitzer found the UnresolvedColumnDefault instance and tried to invoke operations on it.
  Note that it's not currently possible to trigger this issue with Dolt or GMS because MySQL has a more limited syntax than Postgres for altering a column without respecfiying its full column definition, and Dolt/GMS does not support that syntax currently (e.g. ALTER TABLE t ALTER COLUMN SET VISIBLE).
• 2892: Bug fix: Call StatementBegin for all TableEditors used in insertIter
insertIter can be given a sql.RowInserter and a sql.RowUpdater, and when the ON DUPLICATE UPDATE clause is present, it can use both to update a table. StatementBegin was only being called on the sql.RowInserter and not for the sql.RowUpdater, which caused a problem for some implementations (e.g. docsWriter) that depend on StatementBegin being called.
  Test for this added to the Dolt package (#8988), using the docWriter implementation where this bug was originally discovered.
• 2891: look through table aliases for table names when applying triggers
  Table aliases on update statements break triggers, because we are unable to find any trigger matches on the alias.
  The fix is to find the real table name when searching for triggers.
  fixes: #8926
• 2890: Sort tables by FK dependencies for DROP TABLES
  When multiple tables are supplied to DROP TABLES, MySQL (and also Postgres) will sort the tables by foreign key dependencies, so that the tables can be dropped cleanly.
  Resolves dolthub/doltgresql#1272
• 2889: don't erase dual table projection
  The rule eraseProjection erases the projection from select '', which causes validateSchemaSource to error.
  Adding a case in validateSchemaSource resulted in dummy rows being outputted, so I think this is the best solution for now.
  fixes: #8977
• 2888: Allow type conversions for foreign key checks
  This PR introduces a new method for the ExtendedType interface and uses it to allow converting between compatible column types during a foreign key check.
• 2887: Correct JSON_LENGTH with an object parameter
  When JSON_LENGTH is passed a JSON object, it should always return the number of fields in the object. However, due to a typo, it currently only returns 1 for objects. This PR corrects that behavior.
• 2886: reintroduce last_insert_id discrepancy
  Apparently the last_insert_id in ok result, is not the same as select last_insert_id().
  When there is an auto_increment column, the insert_id in the OKResult is the first value that increments the auto_increment value.
  Reverts some changes from: dolthub/go-mysql-server#2616
  fixes: #8914

Closed Issues

• 8942: Log when max_connections is reached
• 7428: Feature request for supporting health and readiness checks in Dolt DB
• 5997: Pro-active dolt gc warning
• 5785: dolt table import -u inserts and updates duplicate keys instead of failing. Unclear what the desired behavior is.
• 5535: dolt pull from DoltHub does not complete (gets stuck) in hours when a clone takes minutes
• 5371: dolt clone loads all dolt databases in working directory into memory
• 8926: BUG: BEFORE UPDATE Trigger Not Working in Dolt
• 3629: dolt cli commands take a long time for large databases
• 2322: dolt persisted globals --global are not initialized during sql engine creation
• 1987: Automatically GC during SQL server execution
• 864: Postgres compatible mode for dolt diff -r sql and dolt patch
• 8975: Cannot merge branches when two indexes covers same column
• 2847: BUG: Element types don't match MySQL: query-results have inconsistent types

1.50.5

Merged PRs

dolt

• 8984: Fix schema merge bug when schema is identical with multiple indexes on the same columns
  FIxes: #8975

Closed Issues

• 8975: Cannot merge branches when two indexes covers same column
• 8907: Why DOLT_DIFF_SUMMARY shows record without data change and schema change?
• 8977: Can't select ''

Performance

Read Tests	MySQL	Dolt	Multiple
covering_index_scan	1.93	0.67	0.35
groupby_scan	13.22	17.95	1.36
index_join	1.47	2.43	1.65
index_join_scan	1.47	1.44	0.98
index_scan	34.33	30.26	0.88
oltp_point_select	0.18	0.26	1.44
oltp_read_only	3.49	5.28	1.51
select_random_points	0.33	0.6	1.82
select_random_ranges	0.37	0.62	1.68
table_scan	34.95	31.37	0.9
types_table_scan	75.82	114.72	1.51
reads_mean_multiplier			1.28

Write Tests	MySQL	Dolt	Multiple
oltp_delete_insert	8.9	6.32	0.71
oltp_insert	4.1	3.07	0.75
oltp_read_write	8.9	11.45	1.29
oltp_update_index	4.18	3.13	0.75
oltp_update_non_index	4.18	3.07	0.73
oltp_write_only	5.67	6.32	1.11
types_delete_insert	8.43	6.67	0.79
writes_mean_multiplier			0.88

TPC-C TPS Tests	MySQL	Dolt	Multiple
tpcc-scale-factor-1	95.47	39.83	2.4
tpcc_tps_multiplier			2.4

Overall Mean Multiple	1.52

1.50.4

Merged PRs

dolt

• 8976: go: sqle/dsess: session.go: Fix transaction savepoint creation when cluster replication is enabled.
  Cluster replication creates a system database, dolt_cluster, which does not support transactions. There are also UserSpaceDatabases which do not support CreateTransaction. CreateTransaction has special code to avoid visiting them when recording the start transaction state. CreateSavepoint needed to be updated with that logic as well.
• 8974: Add support for setting notices in the session
  Adds the ability to queue notice messages in the session, for them to be sent to the client during the execution phase. Related to dolthub/doltgresql#1255.
• 8970: Fix InsertID discrepancy tests
  Fixes tests involving OKPacket's InsertID not aligning with last_insert_id.
  related: #8914
  bump: dolthub/go-mysql-server#2886

go-mysql-server

• 2886: reintroduce last_insert_id discrepancy
  Apparently the last_insert_id in ok result, is not the same as select last_insert_id().
  When there is an auto_increment column, the insert_id in the OKResult is the first value that increments the auto_increment value.
  Reverts some changes from: dolthub/go-mysql-server#2616
  fixes: #8914
• 2885: [planbuilder] sort expression aliases always referenced
  This fixes a bug where a sort expression alias computed in the lower projection fails to index the nested expression.
  Below, the first plan's sort searches for c5:6 in the child, but only finds a:7. The second plan fixes the correctness issue. Obviously there are more desirable projection organizations that version two, but this is small enough of an edge case that I think rewriting projection management with proper expression interning would be overkill right now. The rest of the plan tests look OK/improvements.

  select distinct abs(c5) as a from one_pk where c2 in (1,11,31) order by a
  before:
  Sort(abs(one_pk.c5:6)->a:7 ASC nullsFirst)
  └─ Distinct
  └─ Project
  ├─ columns: [abs(one_pk.c5:1)->a:0]
  └─ Filter
  ├─ HashIn
  │   ├─ one_pk.c2:0
  │   └─ TUPLE(1 (tinyint), 11 (tinyint), 31 (tinyint))
  └─ ProcessTable
  └─ Table
  ├─ name: one_pk
  └─ columns: [c2 c5]
  after:
  Distinct
  └─ Project
  ├─ columns: [abs(one_pk.c5:5)->a:0]
  └─ Sort(a:6 ASC nullsFirst)
  └─ Project
  ├─ columns: [one_pk.pk:0!null, one_pk.c1:1, one_pk.c2:2, one_pk.c3:3, one_pk.c4:4, one_pk.c5:5, abs(one_pk.c5:5)->a:0]
  └─ Filter
  ├─ HashIn
  │   ├─ one_pk.c2:2
  │   └─ TUPLE(1 (tinyint), 11 (tinyint), 31 (tinyint))
  └─ ProcessTable
  └─ Table
  ├─ name: one_pk
  └─ columns: [pk c1 c2 c3 c4 c5]

Closed Issues

• 8964: Recurrent Panic with querying geometry types after some queries
• 8926: BUG: BEFORE UPDATE Trigger Not Working in Dolt
• 8914: MySQL is not consistent regarding last_insert_id; go-mysql-server ceased to emulate this inconsistency in v0.19.0

Performance

Read Tests	MySQL	Dolt	Multiple
covering_index_scan	1.96	0.67	0.34
groupby_scan	13.46	17.63	1.31
index_join	1.5	2.43	1.62
index_join_scan	1.44	1.42	0.99
index_scan	36.89	29.72	0.81
oltp_point_select	0.18	0.26	1.44
oltp_read_only	3.49	5.09	1.46
select_random_points	0.34	0.59	1.74
select_random_ranges	0.37	0.62	1.68
table_scan	36.89	30.81	0.84
types_table_scan	80.03	112.67	1.41
reads_mean_multiplier			1.24

Write Tests	MySQL	Dolt	Multiple
oltp_delete_insert	8.9	6.21	0.7
oltp_insert	4.1	3.07	0.75
oltp_read_write	9.06	11.24	1.24
oltp_update_index	4.18	3.13	0.75
oltp_update_non_index	4.18	3.07	0.73
oltp_write_only	5.77	6.21	1.08
types_delete_insert	8.43	6.55	0.78
writes_mean_multiplier			0.86

TPC-C TPS Tests	MySQL	Dolt	Multiple
tpcc-scale-factor-1	96.5	40.13	2.4
tpcc_tps_multiplier			2.4

Overall Mean Multiple	1.50

1.50.3

Merged PRs

dolt

• 8969: [statspro] stats ignore non prolly indexes
  Stats collection skips non-prolly indexes. A more complete/safe refactoring of index use is in the new stats PR.
  fixes: #8964
• 8963: store/nbs: s3_object_reader: Fix a regression which prevented loading large table files with the AWS NomsBlockStore backend.
  Affects 1.50.0, 1.50.1 and 1.50.2.
• 8962: Backup restore procedure can load aws config
• 8950: go: cmd/dolt: sqlserver: Create a Config struct to encapsulate arguments that control sqlserver.ConfigureServices behavior.
  Allow configuring the ProtocolListenerFactory through ConfigureServices.
• 8948: go: cmd/dolt: sqlserver: When stopping the svcs.Controller for the sql-server, drain client connections so that the handler is guaranteed to no longer be handling any connections when the server is stopped.
• 8944: Added documentation for auto_gc_behavior
• 8941: chore: added log_format for auto_generated yaml
  Added log_format in sqlserver.go to automate addition of log_format in config
• 8934: /integration-tests/MySQLDockerfile: run the rustup install shit
• 8931: fix use info schema panic
  The active database being information_schema causes some issues with branch state/working sets.
  gms side: dolthub/go-mysql-server#2873
  fixes: #8930 (comment)
• 8920: Sql debug cmd
  use:

  > dolt debug --output debug --time 10 -q "select * from table"
  starting cpu profile...
  starting mem profile...
  starting trace profile...
  debug results in: /var/folders/f7/0hsvdb9j0q71y3742k64jfn80000gn/T/dolt-debug-1522503989
  zipped results in: debug.tar.gz
  > ls /var/folders/f7/0hsvdb9j0q71y3742k64jfn80000gn/T/dolt-debug-1522503989
  analysis.txt        exec.txt       mem.pprof               trace.out
  cpu.pprof           input.sql     plan.txt
  

  • cpu.pprof = CPU profile, run for at least time seconds (query loops if necessary)
  • mem.pprof = memory profile, run for at least time seconds (query loops if necessary)
  • trace.out = trace profile, run for at least time seconds (query loops if necessary)
  • input.sql = input query (select * from table)
  • analysis.txt = debug analysis output, includes memo and index cost estimations
  • plan.txt = simple and verbose plan output
  • exec.txt = accumulated stdout/stderr for all queries/background work
    note:
  • fails if server mode is running, having this be a stored procedure would be quite a bit more work
  • lightly tested, queries with errors, larger scripts with DML/DDL probably do not profile smoothly
• 8902: go/store/nbs: Move to golang maps instead of dolthub/swiss.
• 8890: feat:(logging) Added Log Format provision
  Added log_format feature for logger.

go-mysql-server

• 2885: [planbuilder] sort expression aliases always referenced
  This fixes a bug where a sort expression alias computed in the lower projection fails to index the nested expression.
  Below, the first plan's sort searches for c5:6 in the child, but only finds a:7. The second plan fixes the correctness issue. Obviously there are more desirable projection organizations that version two, but this is small enough of an edge case that I think rewriting projection management with proper expression interning would be overkill right now. The rest of the plan tests look OK/improvements.

  select distinct abs(c5) as a from one_pk where c2 in (1,11,31) order by a
  before:
  Sort(abs(one_pk.c5:6)->a:7 ASC nullsFirst)
  └─ Distinct
  └─ Project
  ├─ columns: [abs(one_pk.c5:1)->a:0]
  └─ Filter
  ├─ HashIn
  │   ├─ one_pk.c2:0
  │   └─ TUPLE(1 (tinyint), 11 (tinyint), 31 (tinyint))
  └─ ProcessTable
  └─ Table
  ├─ name: one_pk
  └─ columns: [c2 c5]
  after:
  Distinct
  └─ Project
  ├─ columns: [abs(one_pk.c5:5)->a:0]
  └─ Sort(a:6 ASC nullsFirst)
  └─ Project
  ├─ columns: [one_pk.pk:0!null, one_pk.c1:1, one_pk.c2:2, one_pk.c3:3, one_pk.c4:4, one_pk.c5:5, abs(one_pk.c5:5)->a:0]
  └─ Filter
  ├─ HashIn
  │   ├─ one_pk.c2:2
  │   └─ TUPLE(1 (tinyint), 11 (tinyint), 31 (tinyint))
  └─ ProcessTable
  └─ Table
  ├─ name: one_pk
  └─ columns: [pk c1 c2 c3 c4 c5]

• 2883: allow before insert trigger to specify missing column, but better
  It is possible that a BEFORE INSERT TRIGGER fills in the values for a column when it is otherwise missing. This should not trigger a missing default value error.
  Fixes: #8926
  This PR is an improvement on: dolthub/go-mysql-server#2876
• 2881: server/handler.go: Improve some edge cases and error handling in resultForDefaultIter.
• 2879: server: Get rid of globals for setting a protocol listener factory. Get rid of unused, global-ridden and complicated Interceptor and Option functionality.
• 2878: server/context.go: Add a method on SessionManager which allows an integrator to wait for all client connections to drain.
• 2870: fix stored procedures in triggers
  When prepending nodes for trigger execution, we don't prepend to the *plan.Procedure member in the *plan.Call nodes.
• 2868: Don't use non-unique indexes as functional dependencies in AND expressions.
  This fixes a customer issue that was causing Error 1105 (HY000): result max1Row iterator returned more than one row
  We can probably do better than this: we could tweak it so that adding a FD key on a non-unique index instead adds an implication that the non-unique index key determines the primary key.
  But this is a simpler fix in the meantime that makes us more correct.

vitess

• 401: Revert moving call to recycleReadPacket()
  Dolt binlog tests started failing, due to the change in location of calling recycleReadPacket(). This change moves those two calls back to their original location.
• 400: Cleanup: fixing buffer reuse issues and log statements
• 399: go/mysql: conn.go: Fix read-after-recycle bug of the packet byte buffer in COM_{STMT_SEND_LONG_DATA,REGISTER_REPLICA,BINLOG_DUMP_GTID}.
  Back ports vitessio/vitess@24820d8

Closed Issues

• 8964: Recurrent Panic with querying geometry types after some queries
• 8926: BUG: BEFORE UPDATE Trigger Not Working in Dolt
• 8952: Update Roadmap
• 8930: Panic when browsing database tables after connecting to INFORMATION_SCHEMA via SQLTools extension
• 8888: Feature Request: Json Logging
• 8906: Feature Request: Add support for REGEXP_SUBSTR() and REGEXP_INSTR() functions

1.50.2

Merged PRs

dolt

go-mysql-server

• 2868: Don't use non-unique indexes as functional dependencies in AND expressions.
  This fixes a customer issue that was causing Error 1105 (HY000): result max1Row iterator returned more than one row
  We can probably do better than this: we could tweak it so that adding a FD key on a non-unique index instead adds an implication that the non-unique index key determines the primary key.
  But this is a simpler fix in the meantime that makes us more correct.
• 2866: Fix alter table panics
  Multi-alters are executed with Block iterators. Block error handling always assumed the calling context was a stored procedure, which doesn't appear to be true in this case. Rather than create a new iterator, error handling noops to the default error if a proc reference is not found.
  fixes: #8917
• 2865: fix declare cursors in triggers
  fixes: #8913
• 2864: Allow SHOW CREATE to display auto increment for tables that store AutoIncrement but don't support writing it.
  This is the GMS side of an issue to properly display diffs when the autoincrement counter on a table changes.
  We add a new interface, AutoIncrementGetter for tables that store AutoIncrement info, even if they can't generate autoincremented ids (because they're immutable snapshots of a table with an autoincrement column)
• 2863: Added regexp_instr and regexp_substr
  This adds the functions regexp_instr and regexp_substr. This also references the new interface functions in:
  • dolthub/go-icu-regex#4
    This is for the following issue:
  • #8906

Closed Issues

• 8917: SIGSEGV panic when modifying the schema for foreign keys
• 8913: declare variables does not work within triggers
• 2872: Join planner chooses suboptimal plan for lookup join

Performance

Read Tests	MySQL	Dolt	Multiple
covering_index_scan	1.86	0.64	0.34
groupby_scan	13.22	17.95	1.36
index_join	1.5	2.61	1.74
index_join_scan	1.47	1.44	0.98
index_scan	34.33	30.26	0.88
oltp_point_select	0.18	0.26	1.44
oltp_read_only	3.49	5.28	1.51
select_random_points	0.34	0.59	1.74
select_random_ranges	0.37	0.61	1.65
table_scan	34.95	31.94	0.91
types_table_scan	75.82	112.67	1.49
reads_mean_multiplier			1.28

Write Tests	MySQL	Dolt	Multiple
oltp_delete_insert	8.9	6.32	0.71
oltp_insert	4.1	3.07	0.75
oltp_read_write	8.9	11.45	1.29
oltp_update_index	4.18	3.19	0.76
oltp_update_non_index	4.18	3.07	0.73
oltp_write_only	5.67	6.32	1.11
types_delete_insert	8.43	6.67	0.79
writes_mean_multiplier			0.88

TPC-C TPS Tests	MySQL	Dolt	Multiple
tpcc-scale-factor-1	96.23	40.29	2.39
tpcc_tps_multiplier			2.39

Overall Mean Multiple	1.52

1.50.1

Merged PRs

dolt

• 8910: Report auto increment changes in dolt diff and dolt_diff_summary()
  Potential fix for #8907
  Previously, diffs would only look at the schema hash when determining whether or not a table had a schema change. However, the auto increment counter, which is considered part of the schema, is not stored in the schema flatbuffer and thus does not affect the schema hash.
  This led to situations where a diff would detect that a table had changed (because the table hash changed), but it reported neither a data change or a schema change (because the data hash and schema hash were the same.) This was confusing.
  Now, dolt_diff_summary will correctly report a schema change when the auto increment counter has changed, and dolt diff will show the autoincrement value in the SHOW CREATE TABLE output.
• 8905: add pager support to sql shell
  Adds the \pager [on|off] capability to dolt sql.
  Fixes: #8883
  Currently no tests. Would require expect, and is probably beyond the investment in time for this feature.
• 8897: go: Bump to go 1.24. Fix some vet errors related to non-constant format strings.

go-mysql-server

• 2864: Allow SHOW CREATE to display auto increment for tables that store AutoIncrement but don't support writing it.
  This is the GMS side of an issue to properly display diffs when the autoincrement counter on a table changes.
  We add a new interface, AutoIncrementGetter for tables that store AutoIncrement info, even if they can't generate autoincremented ids (because they're immutable snapshots of a table with an autoincrement column)
• 2861: Added configurable options for how to quote identifiers in column defaults
• 2859: server: Add a ContextFactory parameter to the handler, giving integrators control over the *sql.Context creation.

vitess

• 399: go/mysql: conn.go: Fix read-after-recycle bug of the packet byte buffer in COM_{STMT_SEND_LONG_DATA,REGISTER_REPLICA,BINLOG_DUMP_GTID}.
  Back ports vitessio/vitess@24820d8
• 398: Add select expressions for the Postgres INSERT ... RETURNING support
  Postgres allows INSERT statements (and also UPDATE and DELETE) to specify a RETURNING clause with select expressions and will then evaluate those expressions against the affected rows and return the results. This change adds initial support for the RETURNING select expressions so that they can be represented in the AST. (PostgreSQL docs)

Closed Issues

• 8769: last_insert_id gives 0 when 0 is explicitly specified for an auto_increment primary key in an insertion
• 8883: Feature Request: Add pager support for dolt sql terminal output

Performance

Read Tests	MySQL	Dolt	Multiple
covering_index_scan	1.86	0.62	0.33
groupby_scan	13.22	17.63	1.33
index_join	1.47	2.57	1.75
index_join_scan	1.44	1.42	0.99
index_scan	34.33	29.72	0.87
oltp_point_select	0.18	0.26	1.44
oltp_read_only	3.43	5.09	1.48
select_random_points	0.33	0.58	1.76
select_random_ranges	0.37	0.6	1.62
table_scan	34.95	30.81	0.88
types_table_scan	75.82	108.68	1.43
reads_mean_multiplier			1.26

Write Tests	MySQL	Dolt	Multiple
oltp_delete_insert	8.9	6.21	0.7
oltp_insert	4.1	3.07	0.75
oltp_read_write	8.9	11.24	1.26
oltp_update_index	4.18	3.13	0.75
oltp_update_non_index	4.18	3.07	0.73
oltp_write_only	5.67	6.21	1.1
types_delete_insert	8.28	6.55	0.79
writes_mean_multiplier			0.87

TPC-C TPS Tests	MySQL	Dolt	Multiple
tpcc-scale-factor-1	96.54	40.01	2.41
tpcc_tps_multiplier			2.41

Overall Mean Multiple	1.51

1.50.0

Merged PRs

dolt

• 8896: go: sqle: remotesrv: Fix SqlContext interceptors to always forward along the sql.Context as the request Context as well.
• 8880: go: publishrelease: buildpgobinaries.sh: Manually specify the SDK version of our toolchain so it shows up in our release binaries.
• 8870: Allow primary keys to contain address columns sorted by their resolved values.
  This is the Dolt part of this change.
  GMS PR: dolthub/go-mysql-server#2854
  Doltgres PR: dolthub/doltgresql#1214
  The goal of the change is to allow for indexes to use an out-of-line variable-length type (like TEXT or BLOB) as a primary key while still storing just the address in the index (instead of being forced to store a prefix of the value).
  As a result of this change, any tuple comparison operation may need to resolve a hash in the NodeStore. This poses two complications:
  1. The tuple logic exists at a much lower level than the node store and can't depend on it without creating a dependency cycle. We get around this with a new ValueStore interface that can store and retrieve variable-length bytestrings by their content hash. NodeStore is the only implementation of this interface, but decoupling the interface from the implementation allows us to not depend on NodeStore's internals when passing it to lower-level code.
  2. Tuple comparison operations can now end up doing disk IO, which means they need a context parameter.
• 8865: Don't rewrite table data when changing enum/varchar/varbinary type to make it more permissive.
  Sometimes a schema change can widen the set of valid column values without affecting the representation of any existing columns. Examples include:
  • Adding a possible value to the end of an enum
  • Increasing the length of a varchar/varbinary column.
    In these cases, there's no need to rewrite the table, since every tuple is guaranteed to have the same representation it had before.
• 8849: sql-server: Add behavior: auto_gc_behavior: enable.
  When Auto GC is enabled, the running sql-server will periodically collect a Dolt database that is growing in size. This behavior is currently experimental. Tuning the behavior around how often to collect is ongoing work.

go-mysql-server

• 2859: server: Add a ContextFactory parameter to the handler, giving integrators control over the *sql.Context creation.
• 2858: include new/old expressions when resolving projections in trigger scope
  When creating the trigger scope, we copy over the columns and rewrite with the new and old scope, but we don't copy over their expressions as well. This PR fixes that.
  fixes: #8886
• 2857: [cte] fix issue with invalid cte scope mapping
  fixes: #8871
  We were not setting subquery scope mapping, which produced a nil expression when trying to preform a filter inversion.
• 2854: Allow primary keys to contain address columns sorted by their resolved values.
  This is the GMS part of this change.
  Dolt PR: #8870
  Doltgres PR: dolthub/doltgresql#1214
  The goal of the change is to allow for indexes to use an out-of-line variable-length type (like TEXT or BLOB) as a primary key while still storing just the address in the index (instead of being forced to store a prefix of the value).
  As a result of this change, any tuple comparison operation may need to resolve a hash in the NodeStore. This poses two complications:
  1. The tuple logic exists at a much lower level than the node store and can't depend on it without creating a dependency cycle. We get around this with a new ValueStore interface that can store and retrieve variable-length bytestrings by their content hash. NodeStore is the only implementation of this interface, but decoupling the interface from the implementation allows us to not depend on NodeStore's internals when passing it to lower-level code.
  2. Tuple comparison operations can now end up doing disk IO, which means they need a context parameter.

Closed Issues

• 8886: missing old and new scopes in triggers for projections

Performance

Read Tests	MySQL	Dolt	Multiple
covering_index_scan	1.93	0.63	0.33
groupby_scan	13.7	17.95	1.31
index_join	1.47	2.52	1.71
index_join_scan	1.44	1.5	1.04
index_scan	34.95	30.26	0.87
oltp_point_select	0.18	0.27	1.5
oltp_read_only	3.49	5.28	1.51
select_random_points	0.34	0.6	1.76
select_random_ranges	0.37	0.61	1.65
table_scan	34.95	31.94	0.91
types_table_scan	75.82	110.66	1.46
reads_mean_multiplier			1.28

Write Tests	MySQL	Dolt	Multiple
oltp_delete_insert	8.9	6.32	0.71
oltp_insert	4.1	3.07	0.75
oltp_read_write	8.9	11.45	1.29
oltp_update_index	4.18	3.13	0.75
oltp_update_non_index	4.18	3.07	0.73
oltp_write_only	5.67	6.32	1.11
types_delete_insert	8.28	6.67	0.81
writes_mean_multiplier			0.88

TPC-C TPS Tests	MySQL	Dolt	Multiple
tpcc-scale-factor-1	95.85	39.7	2.41
tpcc_tps_multiplier			2.41

Overall Mean Multiple	1.52

1.49.3

Merged PRs

dolt

• 8872: Add the dolt_backups table
  Read only table which only lists the name and url of backups created by the dolt_backups procedure

go-mysql-server

• 2857: [cte] fix issue with invalid cte scope mapping
  fixes: #8871
  We were not setting subquery scope mapping, which produced a nil expression when trying to preform a filter inversion.
• 2856: have explain format=tree,debug,estimates print an explain tree
• 2855: fix insert for subquery aliases with on duplicate update expr
  The table not found error was actually a bad error message; it really wasn't finding the column due to some weird logic around resolving duplicate indexes.
  The logic assumes that n.Destination.Schema() and n.Source.Schema() must be equal length, and if it's not then we need to replace the Source columns' Source node with the provided OnDupValuesPrefix.
  Additionally, when assigning getfield indexes, we can't assume that if the schema lengths are mismatched that they must be using the OnDupValuesPrefix. Instead, we should always search for a matching column, and then substitute.
  fixes: #8862
• 2853: [expression] coalesce should cast return values
  fixes: dolthub/go-mysql-server#2847

Closed Issues

• 8871: panic (cause not yet known)
• 8835: dolt init fails if dolt config is run without global arguments
• 8862: derived table syntax not supported (in select with insert into?)
• 2847: BUG: Element types don't match MySQL: query-results have inconsistent types

Performance

Read Tests	MySQL	Dolt	Multiple
covering_index_scan	1.89	0.63	0.33
groupby_scan	13.22	18.28	1.38
index_join	1.47	2.57	1.75
index_join_scan	1.42	1.44	1.01
index_scan	34.33	30.26	0.88
oltp_point_select	0.18	0.27	1.5
oltp_read_only	3.49	5.28	1.51
select_random_points	0.34	0.6	1.76
select_random_ranges	0.37	0.61	1.65
table_scan	34.33	31.94	0.93
types_table_scan	75.82	114.72	1.51
reads_mean_multiplier			1.29

Write Tests	MySQL	Dolt	Multiple
oltp_delete_insert	8.9	6.32	0.71
oltp_insert	4.1	3.13	0.76
oltp_read_write	8.9	11.65	1.31
oltp_update_index	4.18	3.19	0.76
oltp_update_non_index	4.18	3.13	0.75
oltp_write_only	5.67	6.32	1.11
types_delete_insert	8.43	6.67	0.79
writes_mean_multiplier			0.88

TPC-C TPS Tests	MySQL	Dolt	Multiple
tpcc-scale-factor-1	96.48	40.2	2.4
tpcc_tps_multiplier			2.4

Overall Mean Multiple	1.52

1.49.2

Merged PRs

dolt

• 8860: Add pointer for checking super user perms for backup restore for doltgres
• 8859: go: remotestorage: Rework how dictionary fetching and dictionary cache is populated.
• 8847: Added an extension point for doltgres to lookup values in the dolt_ignore table
• 8842: move status messages from parquet to stderr
  fixes: #8839
• 8838: allow dolt init on dolt repo with just config.json
  This PR changes dolt init to allow the dolt repo to still be created when there are no global users, but there are local users in .dolt/config.json.
  In order to get to that state you must do

  mkdir .dolt
  dolt config --add user.email <email>
  dolt config --add user.name <name>

  related: #8835
• 8821: New skipped test for bug merging roots with overlapping index defs

go-mysql-server

• 2853: [expression] coalesce should cast return values
  fixes: dolthub/go-mysql-server#2847
• 2850: processlist: Allow for killing the context associated with non-query operations like SetDB and Prepare.
  The processlist maintains a Process struct for a given mysql.Conn, and registers a context.CancelFunc for the running operation when the handler dispatches it. This works for queries today. This PR makes it so we also register CancelFunc callbacks for operations which touch the database but do not put the connection into CommandQuery, such as Prepare and ComInit.
• 2849: PlanBuilder: Fix spelling in error message s/prodecure/procedure
  Fix spelling of word "procedure".
  Since this is trivial, if it makes it easier for maintainers to make a commit like this themselves, or include as part of something else -- I don't mind.
• 2845: Added external function provider
  This adds an external function provider, currently needed for Doltgres to get function creation working. This is intended to be a temporary measure until a more permanent solution is developed (which may involve modifying Dolt's DatabaseProvider).

vitess

• 398: Add select expressions for the Postgres INSERT ... RETURNING support
  Postgres allows INSERT statements (and also UPDATE and DELETE) to specify a RETURNING clause with select expressions and will then evaluate those expressions against the affected rows and return the results. This change adds initial support for the RETURNING select expressions so that they can be represented in the AST. (PostgreSQL docs)
• 397: /.github/workflows: bump hosted runners
• 396: Implemented WalkableSQLNode for InjectedExpr
  This is required to examine / manipulate an expression tree involving one of these expressions
• 395: syntax for drop temporary table ...

Closed Issues

• 8839: dolt sql --result-format=parquet --file path.sql prints garbage to stdout
• 8791: dolt clone allows cloning a repo with a journal, but only if that repo is in a child directory.
• 2847: BUG: Element types don't match MySQL: query-results have inconsistent types