Add zone config troubleshooting guide #19283
Conversation
rmloveland
force-pushed
the
20241209-DOC-9210-zone-config-troubleshooting-guide
branch
7 times, most recently
from
e8df32a to
674915d
Compare
rmloveland
force-pushed
the
20241209-DOC-9210-zone-config-troubleshooting-guide
branch
6 times, most recently
from
e11a980 to
d460824
Compare
rmloveland
force-pushed
the
20241209-DOC-9210-zone-config-troubleshooting-guide
branch
3 times, most recently
from
fc2c697 to
ab3faec
Compare
rmloveland
force-pushed
the
20241209-DOC-9210-zone-config-troubleshooting-guide
branch
from
ab3faec to
fb31a32
Compare
rmloveland
force-pushed
the
20241209-DOC-9210-zone-config-troubleshooting-guide
branch
2 times, most recently
from
0f0bfde to
a2f820e
Compare
Fixes DOC-9210 Summary of changes: - Add a new page, 'Troubleshoot Replication Zones', to _The ZoneConfigonomicon (tm)_ - Update the 'Replication controls' page with more detailed info re: zone config inheritance hierarchy and behavior - Update the made-up examples in the 'Critical nodes endpoint' docs to point back to 'Troubleshoot Replication Zones' - Fix incorrect statements on the `ALTER RANGE` page since they're needed to map from range IDs returned by the critical nodes endpoint (mentioned in 'Troubleshoot Replication Zones') to actual schema objects - Add moar links (tm) from various zone config-related pages to the new troubleshooting guide and amongst themselves - Add a note to various zone config-related docs saying "most users should not do manual zone config changes, see Multi-region SQL and Zone Config Extensions instead"
rmloveland
force-pushed
the
20241209-DOC-9210-zone-config-troubleshooting-guide
branch
from
a2f820e to
a91c384
Compare
|
|
||
| The following example query uses the [`SHOW RANGES`]({% link {{ page.version.version }}/show-ranges.md %}) statement to show, for each range ID, which tables and indexes use that range for their underlying storage. The query assumes the [`movr` schema]({% link {{ page.version.version }}/movr.md %}#the-movr-database) that is loaded by [`cockroach demo`]({% link {{ page.version.version }}/cockroach-demo.md %}), so you'll need to modify it to work with your schema. | ||
|
|
||
| XXX: GET HELP WITH THIS QUERY, OUTPUT SEEMS NOT QUITE RIGHT??? (or is it ok?) |
There was a problem hiding this comment.
(as you can see the output is maybe not the best)
There was a problem hiding this comment.
this looks fine to me! maybe faizan has other thoughts...
There was a problem hiding this comment.
I think you can introduce array_aggregation here to make it more readable:
WITH movr_tables AS (SHOW RANGES FROM DATABASE movr WITH TABLES),
movr_indexes AS (SHOW RANGES FROM DATABASE movr WITH INDEXES)
SELECT array_agg(movr_indexes.range_id) as ranges,
movr_tables.table_name,
movr_indexes.index_name
FROM movr_tables, movr_indexes
WHERE movr_tables.range_id = movr_indexes.range_id
GROUP BY movr_tables.table_name, movr_indexes.index_name ORDER BY table_name, index_name;
WDYT? @annrpom @rmloveland
There was a problem hiding this comment.
i think it's great @fqazi , agree that it's easier to read, have updated to use this query - thank you! i have also updated the surrounding text to match the new table output
| 1. If there's no applicable table replication zone, CockroachDB uses the database replication zone. | ||
| 1. If there's no applicable database replication zone, CockroachDB uses the `default` cluster-wide replication zone. | ||
| 1. If there's a replication zone [for the row](#create-a-replication-zone-for-a-partition) (a.k.a. [partition]({% link {{ page.version.version }}/partitioning.md %})), CockroachDB uses it. | ||
| 1. If there's no applicable row replication zone and the row is from a secondary index, CockroachDB uses the [secondary index replication zone](#create-a-replication-zone-for-a-secondary-index). |
There was a problem hiding this comment.
i wonder why the docs before also just specified secondary index -- can we generalize "secondary indexes" to just "indexes" (or was this specification intentional)?
primary indexes can be configured as such:
demo@127.0.0.1:26257/demoapp/movr> CREATE TABLE foo(i INT PRIMARY
-> KEY, j INT) PARTITION BY LIST
-> (i) (
-> PARTITION p VALUES IN (1, 5)
-> );
demo@127.0.0.1:26257/demoapp/movr> ALTER INDEX foo@foo_pkey
-> CONFIGURE ZONE USING
-> num_replicas = 11;
There was a problem hiding this comment.
thanks for catching this, idk why it says "secondary index" and is so specific about it, i don't think there is a serious intent behind that, i think it's just how the person writing the docs did it at one time. looking at git it's a change from v1.2 so i think it's just historical accident
went through here and a few other places in this doc that said "secondary index" and changed to just say "index"
There was a problem hiding this comment.
i have a similar comment here as the outline-style diagram, but also:
i would expect a direct arrow from index (on the left) to the partitions -- right now, it looks like the inheritance has table as the parent and index, partition, and subpartitions as the direct children
There was a problem hiding this comment.
I think the intent was to show table / index level partitions? Probably naming would help the individual examples..I did a quick thing with PlantUML to split it off, but we need to call both of them Partition
There was a problem hiding this comment.
i would expect a direct arrow from index (on the left) to the partitions -- right now, it looks like the inheritance has table as the parent and index, partition, and subpartitions as the direct children
ah yes, oops, i added a direct arrow from the index to the first partition
this leaves table as the parent of the index (and the subpartitions per cockroachdb/cockroach#75862)
i hope it's a bit better, please let me know what you think!
I think the intent was to show table / index level partitions? Probably naming would help the individual examples
yes, sorry, the intent was to show the table/index level partitions and how the subpartitions are really inheriting from tables per the linked issue above
i also changed all of the names here to match the names added to the other diagram, i hope it helps but please let me know
| - sub-partition 1.1 (NB. Sub-partitions inherit from tables, *not* partitions - see cockroachdb/cockroach#75862) | ||
| - sub-partition 1.1.1 | ||
| - sub-partition 1.2 | ||
| - table |
There was a problem hiding this comment.
nit: i believe adding 2 tables to this diagram is trying to illustrate how inheritance comes in when multiple of the same type of object have their zone configured (which is great), but would we be able to make this distinction more clear?
right now, i think i'm reading "table" as "all tables" if that makes why i was confused a bit more clear
There was a problem hiding this comment.
The 2 tables here does make it a bit confusing, maybe naming the objects will help?
There was a problem hiding this comment.
ah sorry, ya i can see how that is confusing
took a shot at adding some names to the DBs/table/indexes , PTAL - i don't know if these are good names but they attempt to show the hierarchy a little. def open to feedback tho
- default
- database A
- table A.B
- index A.B.1
- partition (row) 1
- sub-partition 1.1 (NB. Sub-partitions inherit from tables, *not* partitions - see cockroachdb/cockroach#75862)
- sub-partition 1.1.1
- sub-partition 1.2
- table A.C
- index A.C.1
- database D
- ...
|
|
||
| As [discussed previously](#level-priorities), CockroachDB always uses the most granular replication zone available for each schema object (database, table, etc.). More-specific settings applied to schema objects at lower levels of the inheritance tree will always override the settings of objects above them in the tree. All configurations will therefore be modified versions of the [`default` range]({% link {{ page.version.version }}/configure-replication-zones.md %}#view-the-default-replication-zone), which acts as the root of the tree. This means that in practice, **any changes to a specific schema object's zone configurations are by definition user-initiated**, either via [Multi-region SQL abstractions]({% link {{ page.version.version }}/multiregion-overview.md %}) or [manual changes](#why-manual-zone-config-management-is-not-recommended). | ||
|
|
||
| Because each zone config inherits all of its initial values from its parent object, **it only stores the values that differ from its parent**. Any fields that are unset will be looked up in the parent object’s zone configuration. This continues recursively up the inheritance hierarchy all the way to the `default` zone config. In practice, most values are cached for performance. |
There was a problem hiding this comment.
In practice, most values are cached for performance.
I guess the intention here is to say there is no overhead from the inheritance right?
There was a problem hiding this comment.
yes but i'm also happy to remove it, seems like a level of detail that doesn't matter for conceptual understanding. wdyt?
| - sub-partition 1.1 (NB. Sub-partitions inherit from tables, *not* partitions - see cockroachdb/cockroach#75862) | ||
| - sub-partition 1.1.1 | ||
| - sub-partition 1.2 | ||
| - table |
There was a problem hiding this comment.
The 2 tables here does make it a bit confusing, maybe naming the objects will help?
|
|
||
| The following example query uses the [`SHOW RANGES`]({% link {{ page.version.version }}/show-ranges.md %}) statement to show, for each range ID, which tables and indexes use that range for their underlying storage. The query assumes the [`movr` schema]({% link {{ page.version.version }}/movr.md %}#the-movr-database) that is loaded by [`cockroach demo`]({% link {{ page.version.version }}/cockroach-demo.md %}), so you'll need to modify it to work with your schema. | ||
|
|
||
| XXX: GET HELP WITH THIS QUERY, OUTPUT SEEMS NOT QUITE RIGHT??? (or is it ok?) |
There was a problem hiding this comment.
I think you can introduce array_aggregation here to make it more readable:
WITH movr_tables AS (SHOW RANGES FROM DATABASE movr WITH TABLES),
movr_indexes AS (SHOW RANGES FROM DATABASE movr WITH INDEXES)
SELECT array_agg(movr_indexes.range_id) as ranges,
movr_tables.table_name,
movr_indexes.index_name
FROM movr_tables, movr_indexes
WHERE movr_tables.range_id = movr_indexes.range_id
GROUP BY movr_tables.table_name, movr_indexes.index_name ORDER BY table_name, index_name;
WDYT? @annrpom @rmloveland
There was a problem hiding this comment.
I think the intent was to show table / index level partitions? Probably naming would help the individual examples..I did a quick thing with PlantUML to split it off, but we need to call both of them Partition
Fixes DOC-9210
Summary of changes:
Add a new page, 'Troubleshoot Replication Zones', to The
ZoneConfigonomicon (tm)
Update the 'Replication controls' page with more detailed info re:
zone config inheritance hierarchy and behavior
Update the made-up examples in the 'Critical nodes endpoint' docs to
point back to 'Troubleshoot Replication Zones'
Fix incorrect statements on the
ALTER RANGEpage since they'reneeded to map from range IDs returned by the critical nodes
endpoint (mentioned in 'Troubleshoot Replication Zones') to actual
schema objects
Add moar links (tm) from various zone config-related pages to the new
troubleshooting guide and amongst themselves
Add a note to various zone config-related docs saying "most users
should not do manual zone config changes, see Multi-region SQL and
Zone Config Extensions instead"
For reviewers, the main changes are:
SHOW RANGESqueries to get them to actually work (they were missing the... WITH DETAILSclause, and/or usingcrdb_internal.ranges- this was a bit of a driveby change but seemed good to do since I was touchingALTER RANGE ... CONFIGURE ZONEto point to the troubleshooting guide