Releases: dolthub/dolt
1.50.5
Merged PRs
dolt
- 8984: Fix schema merge bug when schema is identical with multiple indexes on the same columns
FIxes: #8975
Closed Issues
- 8975: Cannot merge branches when two indexes covers same column
- 8907: Why DOLT_DIFF_SUMMARY shows record without data change and schema change?
- 8977: Can't
select ''
Performance
Read Tests | MySQL | Dolt | Multiple |
---|---|---|---|
covering_index_scan | 1.93 | 0.67 | 0.35 |
groupby_scan | 13.22 | 17.95 | 1.36 |
index_join | 1.47 | 2.43 | 1.65 |
index_join_scan | 1.47 | 1.44 | 0.98 |
index_scan | 34.33 | 30.26 | 0.88 |
oltp_point_select | 0.18 | 0.26 | 1.44 |
oltp_read_only | 3.49 | 5.28 | 1.51 |
select_random_points | 0.33 | 0.6 | 1.82 |
select_random_ranges | 0.37 | 0.62 | 1.68 |
table_scan | 34.95 | 31.37 | 0.9 |
types_table_scan | 75.82 | 114.72 | 1.51 |
reads_mean_multiplier | 1.28 |
Write Tests | MySQL | Dolt | Multiple |
---|---|---|---|
oltp_delete_insert | 8.9 | 6.32 | 0.71 |
oltp_insert | 4.1 | 3.07 | 0.75 |
oltp_read_write | 8.9 | 11.45 | 1.29 |
oltp_update_index | 4.18 | 3.13 | 0.75 |
oltp_update_non_index | 4.18 | 3.07 | 0.73 |
oltp_write_only | 5.67 | 6.32 | 1.11 |
types_delete_insert | 8.43 | 6.67 | 0.79 |
writes_mean_multiplier | 0.88 |
TPC-C TPS Tests | MySQL | Dolt | Multiple |
---|---|---|---|
tpcc-scale-factor-1 | 95.47 | 39.83 | 2.4 |
tpcc_tps_multiplier | 2.4 |
Overall Mean Multiple | 1.52 |
---|
1.50.4
Merged PRs
dolt
- 8976: go: sqle/dsess: session.go: Fix transaction savepoint creation when cluster replication is enabled.
Cluster replication creates a system database,dolt_cluster
, which does not support transactions. There are also UserSpaceDatabases which do not support CreateTransaction. CreateTransaction has special code to avoid visiting them when recording the start transaction state. CreateSavepoint needed to be updated with that logic as well. - 8974: Add support for setting notices in the session
Adds the ability to queue notice messages in the session, for them to be sent to the client during the execution phase. Related to dolthub/doltgresql#1255. - 8970: Fix InsertID discrepancy tests
Fixes tests involving OKPacket's InsertID not aligning withlast_insert_id
.
related: #8914
bump: dolthub/go-mysql-server#2886
go-mysql-server
- 2886: reintroduce
last_insert_id
discrepancy
Apparently thelast_insert_id
in ok result, is not the same asselect last_insert_id()
.
When there is an auto_increment column, the insert_id in the OKResult is the first value that increments the auto_increment value.
Reverts some changes from: dolthub/go-mysql-server#2616
fixes: #8914 - 2885: [planbuilder] sort expression aliases always referenced
This fixes a bug where a sort expression alias computed in the lower projection fails to index the nested expression.
Below, the first plan's sort searches forc5:6
in the child, but only findsa:7
. The second plan fixes the correctness issue. Obviously there are more desirable projection organizations that version two, but this is small enough of an edge case that I think rewriting projection management with proper expression interning would be overkill right now. The rest of the plan tests look OK/improvements.select distinct abs(c5) as a from one_pk where c2 in (1,11,31) order by a before: Sort(abs(one_pk.c5:6)->a:7 ASC nullsFirst) └─ Distinct └─ Project ├─ columns: [abs(one_pk.c5:1)->a:0] └─ Filter ├─ HashIn │ ├─ one_pk.c2:0 │ └─ TUPLE(1 (tinyint), 11 (tinyint), 31 (tinyint)) └─ ProcessTable └─ Table ├─ name: one_pk └─ columns: [c2 c5] after: Distinct └─ Project ├─ columns: [abs(one_pk.c5:5)->a:0] └─ Sort(a:6 ASC nullsFirst) └─ Project ├─ columns: [one_pk.pk:0!null, one_pk.c1:1, one_pk.c2:2, one_pk.c3:3, one_pk.c4:4, one_pk.c5:5, abs(one_pk.c5:5)->a:0] └─ Filter ├─ HashIn │ ├─ one_pk.c2:2 │ └─ TUPLE(1 (tinyint), 11 (tinyint), 31 (tinyint)) └─ ProcessTable └─ Table ├─ name: one_pk └─ columns: [pk c1 c2 c3 c4 c5]
Closed Issues
- 8964: Recurrent Panic with querying geometry types after some queries
- 8926: BUG: BEFORE UPDATE Trigger Not Working in Dolt
- 8914: MySQL is not consistent regarding
last_insert_id
;go-mysql-server
ceased to emulate this inconsistency in v0.19.0
Performance
Read Tests | MySQL | Dolt | Multiple |
---|---|---|---|
covering_index_scan | 1.96 | 0.67 | 0.34 |
groupby_scan | 13.46 | 17.63 | 1.31 |
index_join | 1.5 | 2.43 | 1.62 |
index_join_scan | 1.44 | 1.42 | 0.99 |
index_scan | 36.89 | 29.72 | 0.81 |
oltp_point_select | 0.18 | 0.26 | 1.44 |
oltp_read_only | 3.49 | 5.09 | 1.46 |
select_random_points | 0.34 | 0.59 | 1.74 |
select_random_ranges | 0.37 | 0.62 | 1.68 |
table_scan | 36.89 | 30.81 | 0.84 |
types_table_scan | 80.03 | 112.67 | 1.41 |
reads_mean_multiplier | 1.24 |
Write Tests | MySQL | Dolt | Multiple |
---|---|---|---|
oltp_delete_insert | 8.9 | 6.21 | 0.7 |
oltp_insert | 4.1 | 3.07 | 0.75 |
oltp_read_write | 9.06 | 11.24 | 1.24 |
oltp_update_index | 4.18 | 3.13 | 0.75 |
oltp_update_non_index | 4.18 | 3.07 | 0.73 |
oltp_write_only | 5.77 | 6.21 | 1.08 |
types_delete_insert | 8.43 | 6.55 | 0.78 |
writes_mean_multiplier | 0.86 |
TPC-C TPS Tests | MySQL | Dolt | Multiple |
---|---|---|---|
tpcc-scale-factor-1 | 96.5 | 40.13 | 2.4 |
tpcc_tps_multiplier | 2.4 |
Overall Mean Multiple | 1.50 |
---|
1.50.3
Merged PRs
dolt
- 8969: [statspro] stats ignore non prolly indexes
Stats collection skips non-prolly indexes. A more complete/safe refactoring of index use is in the new stats PR.
fixes: #8964 - 8963: store/nbs: s3_object_reader: Fix a regression which prevented loading large table files with the AWS NomsBlockStore backend.
Affects 1.50.0, 1.50.1 and 1.50.2. - 8962: Backup restore procedure can load aws config
- 8950: go: cmd/dolt: sqlserver: Create a Config struct to encapsulate arguments that control sqlserver.ConfigureServices behavior.
Allow configuring the ProtocolListenerFactory through ConfigureServices. - 8948: go: cmd/dolt: sqlserver: When stopping the svcs.Controller for the sql-server, drain client connections so that the handler is guaranteed to no longer be handling any connections when the server is stopped.
- 8944: Added documentation for auto_gc_behavior
- 8941: chore: added log_format for auto_generated yaml
Added log_format in sqlserver.go to automate addition of log_format in config - 8934: /integration-tests/MySQLDockerfile: run the rustup install shit
- 8931: fix use info schema panic
The active database beinginformation_schema
causes some issues with branch state/working sets.
gms side: dolthub/go-mysql-server#2873
fixes: #8930 (comment) - 8920: Sql debug cmd
use:> dolt debug --output debug --time 10 -q "select * from table" starting cpu profile... starting mem profile... starting trace profile... debug results in: /var/folders/f7/0hsvdb9j0q71y3742k64jfn80000gn/T/dolt-debug-1522503989 zipped results in: debug.tar.gz > ls /var/folders/f7/0hsvdb9j0q71y3742k64jfn80000gn/T/dolt-debug-1522503989 analysis.txt exec.txt mem.pprof trace.out cpu.pprof input.sql plan.txt
cpu.pprof
= CPU profile, run for at leasttime
seconds (query loops if necessary)mem.pprof
= memory profile, run for at leasttime
seconds (query loops if necessary)trace.out
= trace profile, run for at leasttime
seconds (query loops if necessary)input.sql
= input query (select * from table
)analysis.txt
= debug analysis output, includes memo and index cost estimationsplan.txt
= simple and verbose plan outputexec.txt
= accumulated stdout/stderr for all queries/background work
note:- fails if server mode is running, having this be a stored procedure would be quite a bit more work
- lightly tested, queries with errors, larger scripts with DML/DDL probably do not profile smoothly
- 8902: go/store/nbs: Move to golang maps instead of dolthub/swiss.
- 8890: feat:(logging) Added Log Format provision
Addedlog_format
feature for logger.
go-mysql-server
- 2885: [planbuilder] sort expression aliases always referenced
This fixes a bug where a sort expression alias computed in the lower projection fails to index the nested expression.
Below, the first plan's sort searches forc5:6
in the child, but only findsa:7
. The second plan fixes the correctness issue. Obviously there are more desirable projection organizations that version two, but this is small enough of an edge case that I think rewriting projection management with proper expression interning would be overkill right now. The rest of the plan tests look OK/improvements.select distinct abs(c5) as a from one_pk where c2 in (1,11,31) order by a before: Sort(abs(one_pk.c5:6)->a:7 ASC nullsFirst) └─ Distinct └─ Project ├─ columns: [abs(one_pk.c5:1)->a:0] └─ Filter ├─ HashIn │ ├─ one_pk.c2:0 │ └─ TUPLE(1 (tinyint), 11 (tinyint), 31 (tinyint)) └─ ProcessTable └─ Table ├─ name: one_pk └─ columns: [c2 c5] after: Distinct └─ Project ├─ columns: [abs(one_pk.c5:5)->a:0] └─ Sort(a:6 ASC nullsFirst) └─ Project ├─ columns: [one_pk.pk:0!null, one_pk.c1:1, one_pk.c2:2, one_pk.c3:3, one_pk.c4:4, one_pk.c5:5, abs(one_pk.c5:5)->a:0] └─ Filter ├─ HashIn │ ├─ one_pk.c2:2 │ └─ TUPLE(1 (tinyint), 11 (tinyint), 31 (tinyint)) └─ ProcessTable └─ Table ├─ name: one_pk └─ columns: [pk c1 c2 c3 c4 c5]
- 2883: allow before insert trigger to specify missing column, but better
It is possible that aBEFORE INSERT TRIGGER
fills in the values for a column when it is otherwise missing. This should not trigger a missing default value error.
Fixes: #8926
This PR is an improvement on: dolthub/go-mysql-server#2876 - 2881: server/handler.go: Improve some edge cases and error handling in resultForDefaultIter.
- 2879: server: Get rid of globals for setting a protocol listener factory. Get rid of unused, global-ridden and complicated Interceptor and Option functionality.
- 2878: server/context.go: Add a method on SessionManager which allows an integrator to wait for all client connections to drain.
- 2870: fix stored procedures in triggers
When prepending nodes for trigger execution, we don't prepend to the*plan.Procedure
member in the*plan.Call
nodes. - 2868: Don't use non-unique indexes as functional dependencies in AND expressions.
This fixes a customer issue that was causingError 1105 (HY000): result max1Row iterator returned more than one row
We can probably do better than this: we could tweak it so that adding a FD key on a non-unique index instead adds an implication that the non-unique index key determines the primary key.
But this is a simpler fix in the meantime that makes us more correct.
vitess
- 401: Revert moving call to
recycleReadPacket()
Dolt binlog tests started failing, due to the change in location of callingrecycleReadPacket()
. This change moves those two calls back to their original location. - 400: Cleanup: fixing buffer reuse issues and log statements
- 399: go/mysql: conn.go: Fix read-after-recycle bug of the packet byte buffer in COM_{STMT_SEND_LONG_DATA,REGISTER_REPLICA,BINLOG_DUMP_GTID}.
Back ports vitessio/vitess@24820d8
Closed Issues
- 8964: Recurrent Panic with querying geometry types after some queries
- 8926: BUG: BEFORE UPDATE Trigger Not Working in Dolt
- 8952: Update Roadmap
- 8930: Panic when browsing database tables after connecting to INFORMATION_SCHEMA via SQLTools extension
- 8888: Feature Request: Json Logging
- 8906: Feature Request: Add support for REGEXP_SUBSTR() and REGEXP_INSTR() functions
1.50.2
Merged PRs
dolt
go-mysql-server
- 2868: Don't use non-unique indexes as functional dependencies in AND expressions.
This fixes a customer issue that was causingError 1105 (HY000): result max1Row iterator returned more than one row
We can probably do better than this: we could tweak it so that adding a FD key on a non-unique index instead adds an implication that the non-unique index key determines the primary key.
But this is a simpler fix in the meantime that makes us more correct. - 2866: Fix alter table panics
Multi-alters are executed withBlock
iterators. Block error handling always assumed the calling context was a stored procedure, which doesn't appear to be true in this case. Rather than create a new iterator, error handling noops to the default error if a proc reference is not found.
fixes: #8917 - 2865: fix declare cursors in triggers
fixes: #8913 - 2864: Allow SHOW CREATE to display auto increment for tables that store AutoIncrement but don't support writing it.
This is the GMS side of an issue to properly display diffs when the autoincrement counter on a table changes.
We add a new interface,AutoIncrementGetter
for tables that store AutoIncrement info, even if they can't generate autoincremented ids (because they're immutable snapshots of a table with an autoincrement column) - 2863: Added regexp_instr and regexp_substr
This adds the functionsregexp_instr
andregexp_substr
. This also references the new interface functions in:- dolthub/go-icu-regex#4
This is for the following issue: - #8906
- dolthub/go-icu-regex#4
Closed Issues
- 8917: SIGSEGV panic when modifying the schema for foreign keys
- 8913: declare variables does not work within triggers
- 2872: Join planner chooses suboptimal plan for lookup join
Performance
Read Tests | MySQL | Dolt | Multiple |
---|---|---|---|
covering_index_scan | 1.86 | 0.64 | 0.34 |
groupby_scan | 13.22 | 17.95 | 1.36 |
index_join | 1.5 | 2.61 | 1.74 |
index_join_scan | 1.47 | 1.44 | 0.98 |
index_scan | 34.33 | 30.26 | 0.88 |
oltp_point_select | 0.18 | 0.26 | 1.44 |
oltp_read_only | 3.49 | 5.28 | 1.51 |
select_random_points | 0.34 | 0.59 | 1.74 |
select_random_ranges | 0.37 | 0.61 | 1.65 |
table_scan | 34.95 | 31.94 | 0.91 |
types_table_scan | 75.82 | 112.67 | 1.49 |
reads_mean_multiplier | 1.28 |
Write Tests | MySQL | Dolt | Multiple |
---|---|---|---|
oltp_delete_insert | 8.9 | 6.32 | 0.71 |
oltp_insert | 4.1 | 3.07 | 0.75 |
oltp_read_write | 8.9 | 11.45 | 1.29 |
oltp_update_index | 4.18 | 3.19 | 0.76 |
oltp_update_non_index | 4.18 | 3.07 | 0.73 |
oltp_write_only | 5.67 | 6.32 | 1.11 |
types_delete_insert | 8.43 | 6.67 | 0.79 |
writes_mean_multiplier | 0.88 |
TPC-C TPS Tests | MySQL | Dolt | Multiple |
---|---|---|---|
tpcc-scale-factor-1 | 96.23 | 40.29 | 2.39 |
tpcc_tps_multiplier | 2.39 |
Overall Mean Multiple | 1.52 |
---|
1.50.1
Merged PRs
dolt
- 8910: Report auto increment changes in dolt diff and
dolt_diff_summary()
Potential fix for #8907
Previously, diffs would only look at the schema hash when determining whether or not a table had a schema change. However, the auto increment counter, which is considered part of the schema, is not stored in the schema flatbuffer and thus does not affect the schema hash.
This led to situations where a diff would detect that a table had changed (because the table hash changed), but it reported neither a data change or a schema change (because the data hash and schema hash were the same.) This was confusing.
Now,dolt_diff_summary
will correctly report a schema change when the auto increment counter has changed, anddolt diff
will show the autoincrement value in theSHOW CREATE TABLE
output. - 8905: add pager support to sql shell
Adds the\pager [on|off]
capability todolt sql
.
Fixes: #8883
Currently no tests. Would require expect, and is probably beyond the investment in time for this feature. - 8897: go: Bump to go 1.24. Fix some
vet
errors related to non-constant format strings.
go-mysql-server
- 2864: Allow SHOW CREATE to display auto increment for tables that store AutoIncrement but don't support writing it.
This is the GMS side of an issue to properly display diffs when the autoincrement counter on a table changes.
We add a new interface,AutoIncrementGetter
for tables that store AutoIncrement info, even if they can't generate autoincremented ids (because they're immutable snapshots of a table with an autoincrement column) - 2861: Added configurable options for how to quote identifiers in column defaults
- 2859: server: Add a ContextFactory parameter to the handler, giving integrators control over the *sql.Context creation.
vitess
- 399: go/mysql: conn.go: Fix read-after-recycle bug of the packet byte buffer in COM_{STMT_SEND_LONG_DATA,REGISTER_REPLICA,BINLOG_DUMP_GTID}.
Back ports vitessio/vitess@24820d8 - 398: Add select expressions for the Postgres
INSERT ... RETURNING
support
Postgres allowsINSERT
statements (and alsoUPDATE
andDELETE
) to specify aRETURNING
clause with select expressions and will then evaluate those expressions against the affected rows and return the results. This change adds initial support for theRETURNING
select expressions so that they can be represented in the AST. (PostgreSQL docs)
Closed Issues
- 8769:
last_insert_id
gives 0 when 0 is explicitly specified for anauto_increment
primary key in an insertion - 8883: Feature Request: Add pager support for dolt sql terminal output
Performance
Read Tests | MySQL | Dolt | Multiple |
---|---|---|---|
covering_index_scan | 1.86 | 0.62 | 0.33 |
groupby_scan | 13.22 | 17.63 | 1.33 |
index_join | 1.47 | 2.57 | 1.75 |
index_join_scan | 1.44 | 1.42 | 0.99 |
index_scan | 34.33 | 29.72 | 0.87 |
oltp_point_select | 0.18 | 0.26 | 1.44 |
oltp_read_only | 3.43 | 5.09 | 1.48 |
select_random_points | 0.33 | 0.58 | 1.76 |
select_random_ranges | 0.37 | 0.6 | 1.62 |
table_scan | 34.95 | 30.81 | 0.88 |
types_table_scan | 75.82 | 108.68 | 1.43 |
reads_mean_multiplier | 1.26 |
Write Tests | MySQL | Dolt | Multiple |
---|---|---|---|
oltp_delete_insert | 8.9 | 6.21 | 0.7 |
oltp_insert | 4.1 | 3.07 | 0.75 |
oltp_read_write | 8.9 | 11.24 | 1.26 |
oltp_update_index | 4.18 | 3.13 | 0.75 |
oltp_update_non_index | 4.18 | 3.07 | 0.73 |
oltp_write_only | 5.67 | 6.21 | 1.1 |
types_delete_insert | 8.28 | 6.55 | 0.79 |
writes_mean_multiplier | 0.87 |
TPC-C TPS Tests | MySQL | Dolt | Multiple |
---|---|---|---|
tpcc-scale-factor-1 | 96.54 | 40.01 | 2.41 |
tpcc_tps_multiplier | 2.41 |
Overall Mean Multiple | 1.51 |
---|
1.50.0
Merged PRs
dolt
- 8896: go: sqle: remotesrv: Fix SqlContext interceptors to always forward along the sql.Context as the request Context as well.
- 8880: go: publishrelease: buildpgobinaries.sh: Manually specify the SDK version of our toolchain so it shows up in our release binaries.
- 8870: Allow primary keys to contain address columns sorted by their resolved values.
This is the Dolt part of this change.
GMS PR: dolthub/go-mysql-server#2854
Doltgres PR: dolthub/doltgresql#1214
The goal of the change is to allow for indexes to use an out-of-line variable-length type (like TEXT or BLOB) as a primary key while still storing just the address in the index (instead of being forced to store a prefix of the value).
As a result of this change, any tuple comparison operation may need to resolve a hash in the NodeStore. This poses two complications:- The tuple logic exists at a much lower level than the node store and can't depend on it without creating a dependency cycle. We get around this with a new ValueStore interface that can store and retrieve variable-length bytestrings by their content hash. NodeStore is the only implementation of this interface, but decoupling the interface from the implementation allows us to not depend on NodeStore's internals when passing it to lower-level code.
- Tuple comparison operations can now end up doing disk IO, which means they need a context parameter.
- 8865: Don't rewrite table data when changing enum/varchar/varbinary type to make it more permissive.
Sometimes a schema change can widen the set of valid column values without affecting the representation of any existing columns. Examples include:- Adding a possible value to the end of an enum
- Increasing the length of a varchar/varbinary column.
In these cases, there's no need to rewrite the table, since every tuple is guaranteed to have the same representation it had before.
- 8849: sql-server: Add behavior: auto_gc_behavior: enable.
When Auto GC is enabled, the running sql-server will periodically collect a Dolt database that is growing in size. This behavior is currently experimental. Tuning the behavior around how often to collect is ongoing work.
go-mysql-server
- 2859: server: Add a ContextFactory parameter to the handler, giving integrators control over the *sql.Context creation.
- 2858: include
new
/old
expressions when resolving projections in trigger scope
When creating the trigger scope, we copy over the columns and rewrite with thenew
andold
scope, but we don't copy over their expressions as well. This PR fixes that.
fixes: #8886 - 2857: [cte] fix issue with invalid cte scope mapping
fixes: #8871
We were not setting subquery scope mapping, which produced a nil expression when trying to preform a filter inversion. - 2854: Allow primary keys to contain address columns sorted by their resolved values.
This is the GMS part of this change.
Dolt PR: #8870
Doltgres PR: dolthub/doltgresql#1214
The goal of the change is to allow for indexes to use an out-of-line variable-length type (like TEXT or BLOB) as a primary key while still storing just the address in the index (instead of being forced to store a prefix of the value).
As a result of this change, any tuple comparison operation may need to resolve a hash in the NodeStore. This poses two complications:- The tuple logic exists at a much lower level than the node store and can't depend on it without creating a dependency cycle. We get around this with a new
ValueStore
interface that can store and retrieve variable-length bytestrings by their content hash.NodeStore
is the only implementation of this interface, but decoupling the interface from the implementation allows us to not depend onNodeStore
's internals when passing it to lower-level code. - Tuple comparison operations can now end up doing disk IO, which means they need a context parameter.
- The tuple logic exists at a much lower level than the node store and can't depend on it without creating a dependency cycle. We get around this with a new
Closed Issues
- 8886: missing
old
andnew
scopes in triggers for projections
Performance
Read Tests | MySQL | Dolt | Multiple |
---|---|---|---|
covering_index_scan | 1.93 | 0.63 | 0.33 |
groupby_scan | 13.7 | 17.95 | 1.31 |
index_join | 1.47 | 2.52 | 1.71 |
index_join_scan | 1.44 | 1.5 | 1.04 |
index_scan | 34.95 | 30.26 | 0.87 |
oltp_point_select | 0.18 | 0.27 | 1.5 |
oltp_read_only | 3.49 | 5.28 | 1.51 |
select_random_points | 0.34 | 0.6 | 1.76 |
select_random_ranges | 0.37 | 0.61 | 1.65 |
table_scan | 34.95 | 31.94 | 0.91 |
types_table_scan | 75.82 | 110.66 | 1.46 |
reads_mean_multiplier | 1.28 |
Write Tests | MySQL | Dolt | Multiple |
---|---|---|---|
oltp_delete_insert | 8.9 | 6.32 | 0.71 |
oltp_insert | 4.1 | 3.07 | 0.75 |
oltp_read_write | 8.9 | 11.45 | 1.29 |
oltp_update_index | 4.18 | 3.13 | 0.75 |
oltp_update_non_index | 4.18 | 3.07 | 0.73 |
oltp_write_only | 5.67 | 6.32 | 1.11 |
types_delete_insert | 8.28 | 6.67 | 0.81 |
writes_mean_multiplier | 0.88 |
TPC-C TPS Tests | MySQL | Dolt | Multiple |
---|---|---|---|
tpcc-scale-factor-1 | 95.85 | 39.7 | 2.41 |
tpcc_tps_multiplier | 2.41 |
Overall Mean Multiple | 1.52 |
---|
1.49.3
Merged PRs
dolt
- 8872: Add the dolt_backups table
Read only table which only lists the name and url of backups created by the dolt_backups procedure
go-mysql-server
- 2857: [cte] fix issue with invalid cte scope mapping
fixes: #8871
We were not setting subquery scope mapping, which produced a nil expression when trying to preform a filter inversion. - 2856: have
explain format=tree,debug,estimates
print an explain tree - 2855: fix insert for subquery aliases with on duplicate update expr
Thetable not found
error was actually a bad error message; it really wasn't finding the column due to some weird logic around resolving duplicate indexes.
The logic assumes thatn.Destination.Schema()
andn.Source.Schema()
must be equal length, and if it's not then we need to replace the Source columns' Source node with the providedOnDupValuesPrefix
.
Additionally, when assigning getfield indexes, we can't assume that if the schema lengths are mismatched that they must be using theOnDupValuesPrefix
. Instead, we should always search for a matching column, and then substitute.
fixes: #8862 - 2853: [expression] coalesce should cast return values
fixes: dolthub/go-mysql-server#2847
Closed Issues
- 8871: panic (cause not yet known)
- 8835:
dolt init
fails ifdolt config
is run without global arguments - 8862: derived table syntax not supported (in select with insert into?)
- 2847: BUG: Element types don't match MySQL: query-results have inconsistent types
Performance
Read Tests | MySQL | Dolt | Multiple |
---|---|---|---|
covering_index_scan | 1.89 | 0.63 | 0.33 |
groupby_scan | 13.22 | 18.28 | 1.38 |
index_join | 1.47 | 2.57 | 1.75 |
index_join_scan | 1.42 | 1.44 | 1.01 |
index_scan | 34.33 | 30.26 | 0.88 |
oltp_point_select | 0.18 | 0.27 | 1.5 |
oltp_read_only | 3.49 | 5.28 | 1.51 |
select_random_points | 0.34 | 0.6 | 1.76 |
select_random_ranges | 0.37 | 0.61 | 1.65 |
table_scan | 34.33 | 31.94 | 0.93 |
types_table_scan | 75.82 | 114.72 | 1.51 |
reads_mean_multiplier | 1.29 |
Write Tests | MySQL | Dolt | Multiple |
---|---|---|---|
oltp_delete_insert | 8.9 | 6.32 | 0.71 |
oltp_insert | 4.1 | 3.13 | 0.76 |
oltp_read_write | 8.9 | 11.65 | 1.31 |
oltp_update_index | 4.18 | 3.19 | 0.76 |
oltp_update_non_index | 4.18 | 3.13 | 0.75 |
oltp_write_only | 5.67 | 6.32 | 1.11 |
types_delete_insert | 8.43 | 6.67 | 0.79 |
writes_mean_multiplier | 0.88 |
TPC-C TPS Tests | MySQL | Dolt | Multiple |
---|---|---|---|
tpcc-scale-factor-1 | 96.48 | 40.2 | 2.4 |
tpcc_tps_multiplier | 2.4 |
Overall Mean Multiple | 1.52 |
---|
1.49.2
Merged PRs
dolt
- 8860: Add pointer for checking super user perms for backup restore for doltgres
- 8859: go: remotestorage: Rework how dictionary fetching and dictionary cache is populated.
- 8847: Added an extension point for doltgres to lookup values in the dolt_ignore table
- 8842: move status messages from parquet to stderr
fixes: #8839 - 8838: allow
dolt init
on dolt repo with justconfig.json
This PR changesdolt init
to allow the dolt repo to still be created when there are no global users, but there are local users in.dolt/config.json
.
In order to get to that state you must dorelated: #8835mkdir .dolt dolt config --add user.email <email> dolt config --add user.name <name>
- 8821: New skipped test for bug merging roots with overlapping index defs
go-mysql-server
- 2853: [expression] coalesce should cast return values
fixes: dolthub/go-mysql-server#2847 - 2850: processlist: Allow for killing the context associated with non-query operations like SetDB and Prepare.
The processlist maintains a Process struct for a given mysql.Conn, and registers a context.CancelFunc for the running operation when the handler dispatches it. This works for queries today. This PR makes it so we also register CancelFunc callbacks for operations which touch the database but do not put the connection into CommandQuery, such as Prepare and ComInit. - 2849: PlanBuilder: Fix spelling in error message s/prodecure/procedure
Fix spelling of word "procedure".
Since this is trivial, if it makes it easier for maintainers to make a commit like this themselves, or include as part of something else -- I don't mind. - 2845: Added external function provider
This adds an external function provider, currently needed for Doltgres to get function creation working. This is intended to be a temporary measure until a more permanent solution is developed (which may involve modifying Dolt'sDatabaseProvider
).
vitess
- 398: Add select expressions for the Postgres
INSERT ... RETURNING
support
Postgres allowsINSERT
statements (and alsoUPDATE
andDELETE
) to specify aRETURNING
clause with select expressions and will then evaluate those expressions against the affected rows and return the results. This change adds initial support for theRETURNING
select expressions so that they can be represented in the AST. (PostgreSQL docs) - 397: /.github/workflows: bump hosted runners
- 396: Implemented WalkableSQLNode for InjectedExpr
This is required to examine / manipulate an expression tree involving one of these expressions - 395: syntax for
drop temporary table ...
Closed Issues
1.49.1
Merged PRs
dolt
- 8831: Expand
ItemAccess::itemWidth
to 32 bits
ItemAccess
is a class used to read data out of prolly tree nodes. Because theitemWidth
field was limited to 16 bits, reading any value larger than 2^16 bytes would result in silent truncation.
We don't usually store values this large, although it should be safe to do so.
This issue was discovered because the new JSON chunker (introduced in #7912) always stores embedded strings as a single chunk, so a document containing a string larger than 32KB would result in a node with a single value whose length didn't fit in 16 bits.
While we were investigating this issue, we created #8723 to disable the new JSON chunker in the presence of these long strings. This PR partially reverts that one, resuming the smart chunking of JSON even in the presence of large embedded strings. - 8817: Bug fix for encoding extended types in keys
This PR allows certain Doltgres extended types to be correctly serialized as part of keys
go-mysql-server
- 2845: Added external function provider
This adds an external function provider, currently needed for Doltgres to get function creation working. This is intended to be a temporary measure until a more permanent solution is developed (which may involve modifying Dolt'sDatabaseProvider
). - 2844: Correctly skip skipped assertions in transaction tests
- 2843: Replace
golang.org/x/exp
with stdlib
These experimental packages are now available in the Go standard library.golang.org/x/exp/slices
->slices
(https://go.dev/doc/go1.21#slices)golang.org/x/exp/maps
->maps
(https://go.dev/doc/go1.21#maps)golang.org/x/exp/constraints
->cmp
(https://go.dev/doc/go1.21#cmp)golang.org/x/exp/rand
->math/rand/v2
(https://go.dev/doc/go1.22#math_rand_v2)
- 2842: fix panic when comparing system set type against other types
Comparison betweensystemSetTypes
and other types is still not correct.
It appears that MySQL actually treats@@sql_mode
as just a string.
This PR only fixes the panic - 2839: fix case insensitivity and return type for
str_to_date
This PR fixes an issue with thestr_to_date
function where we wouldn't match string literals in the date with literals in the format, because we were improperly converting them to lowercase.
Additionally, this PR has it so thestr_to_date
function returns atime.Time
instead of a string. This gets us closer to MySQL behavior over the server.
fixes: #8807 - 2833: refactor create procedure and call procedure
This PR refactors a ton of the stored procedure behavior to more closely match MySQL.
Changes:- properly error when creating procedures/triggers/events in other stored routines
- allow create procedure statements that reference symbols (tables, views, procedures, etc) that don't exist (yet).
- remove unnecessary analysis and building work
- simplify building
plan.Call
nodes
Partially addresses: #8053
Closed Issues
- 8807: Issue with "T" when using str_to_date
Performance
Read Tests | MySQL | Dolt | Multiple |
---|---|---|---|
covering_index_scan | 1.89 | 0.65 | 0.34 |
groupby_scan | 13.22 | 18.61 | 1.41 |
index_join | 1.47 | 2.48 | 1.69 |
index_join_scan | 1.44 | 1.47 | 1.02 |
index_scan | 34.33 | 31.37 | 0.91 |
oltp_point_select | 0.18 | 0.27 | 1.5 |
oltp_read_only | 3.49 | 5.37 | 1.54 |
select_random_points | 0.33 | 0.61 | 1.85 |
select_random_ranges | 0.37 | 0.64 | 1.73 |
table_scan | 34.33 | 33.72 | 0.98 |
types_table_scan | 75.82 | 123.28 | 1.63 |
reads_mean_multiplier | 1.33 |
Write Tests | MySQL | Dolt | Multiple |
---|---|---|---|
oltp_delete_insert | 9.06 | 6.32 | 0.7 |
oltp_insert | 4.1 | 3.13 | 0.76 |
oltp_read_write | 9.06 | 11.65 | 1.29 |
oltp_update_index | 4.18 | 3.19 | 0.76 |
oltp_update_non_index | 4.18 | 3.07 | 0.73 |
oltp_write_only | 5.77 | 6.32 | 1.1 |
types_delete_insert | 8.43 | 6.67 | 0.79 |
writes_mean_multiplier | 0.88 |
TPC-C TPS Tests | MySQL | Dolt | Multiple |
---|---|---|---|
tpcc-scale-factor-1 | 96.45 | 39.6 | 2.44 |
tpcc_tps_multiplier | 2.44 |
Overall Mean Multiple | 1.55 |
---|
1.49.0
Backwards Incompatible Changes
The --user
and --password
arguments to dolt sql-server
have been deprecated. This change affects the command line arguments as well as the configuration file parameters for dolt sql-server
. These options previously created a temporary superuser with the specified name and optional password; however, since support for SQL user and privilege management was added, these arguments introduced edge cases that caused issues for customers. Instead, customers should use the standard SQL support for managing users and grants. For more information on this change, including instructions on how to manage users and grants with SQL, see our announcement blog post for this change.
Per Dolt’s versioning policy, this is a minor version bump because these changes may impact existing applications. Please reach out to us on GitHub or Discord if you have questions or need help with any of these changes.
Merged PRs
dolt
- 8812: [kvexec] fix left join nullability bug
We were using a stale variable for a nullability comparison.
test here: dolthub/go-mysql-server#2840 - 8809: go: sqle/binlogreplication: Add some waits for more reliable observations of IO_error results.
- 8808: [kvexec] literal types cast to lookup expr type
- 8806: replace cli printErrf with printf in server startup logging
- 8804: Include
deleteErr
message in returned error - 8801: go: store/types: Fix dolt_gc on databases that use vector indexes.
- 8800: Remove
user
andpassword
options for sql-server
The--user
and--password
options to sql-server (whether specified on the command line or in a configuration file) allow users to specify an ephemeral superuser to use for the life of the sql-server process. This is a convenient way to temporarily create a superuser, but has several edge cases that can cause unexpected behavior.
This change removes support for the--user
and--password
options and instead customers should now explicitly create that user using standard SQL statements for managing users, such as:IfCREATE USER myUser@'%' IDENTIFIED BY 'myPassw0rd'; GRANT ALL ON *.* TO myUser@'%';
user
orpassword
is specified in a config file, the sql-server will still startup, but will log a warning. If--user
or--password
is specified as a CLI argument the sql-server will error out with an error message explaining the change. The reason for the difference in handling is that it may be harder for customers to update config files. - 8783: Avoid loading DB for commands where it's not necessary.
This PR avoids loading dolt DBs at startup, instead waiting for eitherDoltEnv.ReloadDB
orMultiRepoEnv.ReloadDBs
to be called.
Based on the specific subcommand being run:MultiRepoEnv.ReloadDBs
is called prior to command execution if the command will always need DB access.- Commands that conditionally require a DB can choose whether to call
DoltEnv.ReloadDB
- Commands that never need the DB will never load it.
Commands that make use ofCliContext
don't need to load the DB, as the underlying query engine will load the DB if it needs database access. As a result, commands that are implemented purely in terms of theCliContext
will load the DB when Dolt is runnning an embedded SQL engine, and will skip loading the DB when Dolt is connecting to a running server.
- 8774: /go/libraries/doltcore/env/actions: make iter resolved tags paginated sort in lexicographical order
- 8732: Tidy up journal writer code comments and error handling
Before this change, the code that iterates over journal records would silently stop processing journal records once it encountered any invalid record. This change turns those invalid records into errors so that data isn't silently ignored. Customers can opt-in to the previous behavior by setting theDOLT_SKIP_INVALID_JOURNAL_RECORDS
environment variable to any value.
go-mysql-server
- 2835: Added the concept of an interpreter expression
This adds the concept of an interpreter expression, which is primarily used by integrators. - 2828: return error for ddl in create proc statements
This PR moves the logic forvalidateStoredProcedure
analyzer rule into the planbuilder, allowing for proper error messages.
Related: #8762
Closed Issues
Performance
Read Tests | MySQL | Dolt | Multiple |
---|---|---|---|
covering_index_scan | 1.89 | 0.64 | 0.34 |
groupby_scan | 13.22 | 17.32 | 1.31 |
index_join | 1.47 | 2.48 | 1.69 |
index_join_scan | 1.42 | 1.44 | 1.01 |
index_scan | 35.59 | 30.26 | 0.85 |
oltp_point_select | 0.18 | 0.26 | 1.44 |
oltp_read_only | 3.49 | 5.28 | 1.51 |
select_random_points | 0.34 | 0.58 | 1.71 |
select_random_ranges | 0.37 | 0.62 | 1.68 |
table_scan | 36.24 | 32.53 | 0.9 |
types_table_scan | 75.82 | 114.72 | 1.51 |
reads_mean_multiplier | 1.27 |
Write Tests | MySQL | Dolt | Multiple |
---|---|---|---|
oltp_delete_insert | 8.9 | 6.21 | 0.7 |
oltp_insert | 4.1 | 3.07 | 0.75 |
oltp_read_write | 8.9 | 11.45 | 1.29 |
oltp_update_index | 4.18 | 3.13 | 0.75 |
oltp_update_non_index | 4.18 | 3.07 | 0.73 |
oltp_write_only | 5.77 | 6.21 | 1.08 |
types_delete_insert | 8.43 | 6.55 | 0.78 |
writes_mean_multiplier | 0.87 |
TPC-C TPS Tests | MySQL | Dolt | Multiple |
---|---|---|---|
tpcc-scale-factor-1 | 95.73 | 40.13 | 2.39 |
tpcc_tps_multiplier | 2.39 |
Overall Mean Multiple | 1.51 |
---|