Update docs for LDR v25.1 #19353

kathancox · 2025-02-06T18:59:42Z

Fixes DOC-11948, DOC-12060

This PR adds the new LDR syntax CREATE LOGICALLY REPLICATED. We're keeping the original syntax from 24.2 in place as well, which has resulted in restructuring the Set Up Logical Data Replication tutorial. Both the new syntaxes, plus unidirectional/bidirectional intersect, so the updates are intended to pave a logical way through the tutorial no matter if the user setting up unidirectional or bidirectional.

Rendered Previews

Setup Guide
New SQL Page

github-actions · 2025-02-06T19:00:10Z

Files changed:

src/current/_includes/releases/v25.1/v25.1.0-alpha.1.md:
- releases/v25.1.md
src/current/_includes/releases/v25.1/v25.1.0-beta.1.md:
- releases/v25.1.md
src/current/_includes/v25.1/ldr/create_logically_replicated_stmt.html:

src/current/v25.1/create-logically-replicated.md

src/current/_includes/v25.1/ldr/note-manage-ldr.md:

src/current/v25.1/create-logically-replicated.md

src/current/_includes/v25.1/ldr/show-logical-replication-responses.md:

src/current/_includes/v25.1/ldr/use-create-logically-replicated.md:

src/current/v25.1/manage-logical-data-replication.md

src/current/_includes/v25.1/sidebar-data/sql.json
src/current/v25.1/create-logical-replication-stream.md
src/current/v25.1/create-logically-replicated.md
src/current/v25.1/logical-data-replication-monitoring.md
src/current/v25.1/manage-logical-data-replication.md
src/current/v25.1/set-up-logical-data-replication.md
src/current/v25.1/show-logical-replication-jobs.md

netlify · 2025-02-06T19:00:22Z

✅ Deploy Preview for cockroachdb-interactivetutorials-docs canceled.

Name	Link
🔨 Latest commit	`bb79370`
🔍 Latest deploy log	https://app.netlify.com/sites/cockroachdb-interactivetutorials-docs/deploys/67b8a401896779000866d48b

netlify · 2025-02-06T19:00:23Z

✅ Deploy Preview for cockroachdb-api-docs canceled.

Name	Link
🔨 Latest commit	`bb79370`
🔍 Latest deploy log	https://app.netlify.com/sites/cockroachdb-api-docs/deploys/67b8a401d5d7b000081cad1a

netlify · 2025-02-06T19:06:39Z

✅ Netlify Preview

Name	Link
🔨 Latest commit	`bb79370`
🔍 Latest deploy log	https://app.netlify.com/sites/cockroachdb-docs/deploys/67b8a4017f53b60008c37166
😎 Deploy Preview	https://deploy-preview-19353--cockroachdb-docs.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

kathancox · 2025-02-07T21:36:06Z

src/current/v25.1/set-up-logical-data-replication.md

@@ -36,12 +42,6 @@ You'll need:
    - All nodes in each cluster will need access to the Certificate Authority for the other cluster. Refer to [Step 2. Connect from the destination to the source](#step-2-connect-from-the-destination-to-the-source).
 - LDR replicates at the table level, which means clusters can contain other tables that are not part of the LDR job. If both clusters are empty, create the tables that you need to replicate with **identical** schema definitions (excluding indexes) on both clusters. If one cluster already has an existing table that you'll replicate, ensure the other cluster's table definition matches. For more details on the supported schemas, refer to [Schema Validation](#schema-validation).


@msbutler @alicia-l2 Am I right in thinking that the main point of this prerequisite bullet applies only to CREATE LOGICAL REPLICATION STREAM?

create the tables that you need to replicate with identical schema definitions (excluding indexes) on both clusters.

correct, this only applies to the original syntax.

kathancox · 2025-02-07T21:36:47Z

src/current/v25.1/set-up-logical-data-replication.md

-{{site.data.alerts.end}}{% endcomment %}
-
-To create bidirectional LDR, you can complete the [optional step](#step-4-optional-set-up-bidirectional-ldr) to start the second LDR job that sends writes from the table on cluster B to the table on cluster A.
-
 ### Schema validation


What about Schema validation? Is this just the old syntax since the new syntax will create the table?

both flavors undergo schema validation. In fact, the CREATE LOGICALLY REPLICATED is slightly more restricive. Unlike with the og syntax, a replicating table with the new syntax cannot have:

UDTs

foreign keys

alicia-l2

LGTM, have some comments

alicia-l2 · 2025-02-13T18:29:07Z

src/current/_includes/v25.1/ldr/use-create-logically-replicated.md

@@ -0,0 +1 @@
+If your table does not contain any user-defined types or [foregin key]({% link {{ page.version.version }}/foreign-key.md %}) dependencies, use the [`CREATE LOGICALLY REPLICATED`]({% link {{ page.version.version }}/create-logically-replicated.md %}) syntax to start the stream for a fast, offline initial scan and automatic destination table setup.


@msbutler Can you fact check that those are the only two limitations for the ofline initial scan?

Yes, i believe these are the only ones.

alicia-l2 · 2025-02-13T21:53:42Z

src/current/v25.1/create-logically-replicated.md

+{{site.data.alerts.callout_info}}
+{% include feature-phases/preview.md %}
+
+Logical data replication is only supported in CockroachDB {{ site.data.products.core }} clusters.


*currently only supported

We should avoid potential "forward-looking" language, and remain with what is factually available right now.

msbutler

Looks good! Left mostly nits

msbutler · 2025-02-14T14:52:15Z

src/current/_includes/v25.1/ldr/use-create-logically-replicated.md

@@ -0,0 +1 @@
+If your table does not contain any user-defined types or [foregin key]({% link {{ page.version.version }}/foreign-key.md %}) dependencies, use the [`CREATE LOGICALLY REPLICATED`]({% link {{ page.version.version }}/create-logically-replicated.md %}) syntax to start the stream for a fast, offline initial scan and automatic destination table setup.


Yes, i believe these are the only ones.

src/current/v25.1/create-logically-replicated.md

msbutler · 2025-02-14T15:21:45Z

src/current/v25.1/set-up-logical-data-replication.md


 - [Column families]({% link {{ page.version.version }}/column-families.md %})
 - [Partial indexes]({% link {{ page.version.version }}/partial-indexes.md %}) and [hash-sharded indexes]({% link {{ page.version.version }}/hash-sharded-indexes.md %})
 - Indexes with a [virtual computed column]({% link {{ page.version.version }}/computed-columns.md %})
 - Composite types in the [primary key]({% link {{ page.version.version }}/primary-key.md %})

+Additionally, for the `CREATE LOGICALLY REPLCATED` syntax, you cannot use LDR on a table with a schema that contains:


i find it a bit confusing that we have this list of limitations as well as a separate list of known limitations. LDR, with both syntaxes, currently do not support UDFs, Sequences etc. IMHO, it would be clearer to have just one list.

For this, I've created myself an issue that I'll tackle as a follow-up. I want to take a look at the KL sections across LDR and this Schema validation section to evaluate the best approach: https://cockroachlabs.atlassian.net/browse/DOC-12331

msbutler · 2025-02-14T15:23:15Z

src/current/v25.1/set-up-logical-data-replication.md


 You can use the `cockroach encode-uri` command to generate a connection string containing a cluster's certificate.

-1. On the **source** cluster in a new terminal window, generate a connection string, by passing the replication user, node IP, and port, along with the directory to the source cluster's CA certificate:
+1. On the **source** cluster in a new terminal window, generate a connection string, by passing the user, node IP, and port, along with the directory to the source cluster's CA certificate:


it would be good to clarify that the user in the external conn requires the REPLICATION priv.

I've added this back in, not sure why I took that out while testing 🤔

if you your user was root, then you had these privs automatically

msbutler · 2025-02-14T15:32:14Z

src/current/v25.1/set-up-logical-data-replication.md


-## Step 4. (Optional) Set up bidirectional LDR
+{% include_cached copy-clipboard.html %}


I just realized I did something bad: I should have added a release note to this SHOW LOGICAL REPLICATION JOBS change I made for 25.1, which means we'll need to update the examples in the docs. Should I open a separate docs issue?

This is another SHOW LDR JOBS change that should have had a release note as well.

No worries! I've updated both of these column name changes and added the release notes to the 25.1 testing releases they released with.

kathancox · 2025-02-14T21:18:02Z

I think I've addressed the comments / opened an issue to take care of the KLs in a follow-up. PTAL when you have a chance, @msbutler @alicia-l2

rmloveland

LGTM, had non-blocking questions/comments to take or leave as you desire

src/current/_includes/v25.1/ldr/use-create-logically-replicated.md

src/current/v25.1/create-logically-replicated.md

src/current/v25.1/set-up-logical-data-replication.md

kathancox force-pushed the ldr-syntax-bidirectional branch 2 times, most recently from f2364da to 366c902 Compare February 7, 2025 21:15

kathancox commented Feb 7, 2025

View reviewed changes

kathancox requested review from msbutler and alicia-l2 February 7, 2025 21:37

kathancox force-pushed the ldr-syntax-bidirectional branch from 366c902 to 2e3493f Compare February 10, 2025 15:04

kathancox marked this pull request as ready for review February 10, 2025 16:48

alicia-l2 reviewed Feb 13, 2025

View reviewed changes

msbutler reviewed Feb 14, 2025

View reviewed changes

msbutler approved these changes Feb 14, 2025

View reviewed changes

kathancox requested a review from rmloveland February 18, 2025 13:44

rmloveland approved these changes Feb 19, 2025

View reviewed changes

kathancox added 4 commits February 21, 2025 11:04

Update docs for LDR v25.1

820b6c3

Feedback

263f9a4

Add missing s on table

3a5e279

Rich's doc feedback

bb79370

kathancox force-pushed the ldr-syntax-bidirectional branch from 5d3eee9 to bb79370 Compare February 21, 2025 16:04

kathancox merged commit c0b88c3 into main Feb 21, 2025
6 checks passed

kathancox deleted the ldr-syntax-bidirectional branch February 21, 2025 16:14

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update docs for LDR v25.1 #19353

Update docs for LDR v25.1 #19353

kathancox commented Feb 6, 2025 •

edited

Loading

github-actions bot commented Feb 6, 2025 •

edited

Loading

netlify bot commented Feb 6, 2025 •

edited

Loading

netlify bot commented Feb 6, 2025 •

edited

Loading

netlify bot commented Feb 6, 2025 •

edited

Loading

kathancox Feb 7, 2025

msbutler Feb 7, 2025

kathancox Feb 7, 2025

msbutler Feb 7, 2025

alicia-l2 left a comment

alicia-l2 Feb 13, 2025

msbutler Feb 14, 2025

alicia-l2 Feb 13, 2025

kathancox Feb 14, 2025

msbutler left a comment

msbutler Feb 14, 2025

msbutler Feb 14, 2025

kathancox Feb 14, 2025

msbutler Feb 14, 2025

kathancox Feb 14, 2025

msbutler Feb 14, 2025 •

edited

Loading

msbutler Feb 14, 2025

kathancox Feb 14, 2025

msbutler Feb 14, 2025

kathancox commented Feb 14, 2025

rmloveland left a comment

		@@ -36,12 +42,6 @@ You'll need:
		- All nodes in each cluster will need access to the Certificate Authority for the other cluster. Refer to [Step 2. Connect from the destination to the source](#step-2-connect-from-the-destination-to-the-source).
		- LDR replicates at the table level, which means clusters can contain other tables that are not part of the LDR job. If both clusters are empty, create the tables that you need to replicate with identical schema definitions (excluding indexes) on both clusters. If one cluster already has an existing table that you'll replicate, ensure the other cluster's table definition matches. For more details on the supported schemas, refer to [Schema Validation](#schema-validation).

		@@ -0,0 +1 @@
		If your table does not contain any user-defined types or [foregin key]({% link {{ page.version.version }}/foreign-key.md %}) dependencies, use the [`CREATE LOGICALLY REPLICATED`]({% link {{ page.version.version }}/create-logically-replicated.md %}) syntax to start the stream for a fast, offline initial scan and automatic destination table setup.


		## Step 4. (Optional) Set up bidirectional LDR
		{% include_cached copy-clipboard.html %}

Update docs for LDR v25.1 #19353

Update docs for LDR v25.1 #19353

Conversation

kathancox commented Feb 6, 2025 • edited Loading

Rendered Previews

github-actions bot commented Feb 6, 2025 • edited Loading

Files changed:

netlify bot commented Feb 6, 2025 • edited Loading

✅ Deploy Preview for cockroachdb-interactivetutorials-docs canceled.

netlify bot commented Feb 6, 2025 • edited Loading

✅ Deploy Preview for cockroachdb-api-docs canceled.

netlify bot commented Feb 6, 2025 • edited Loading

✅ Netlify Preview

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alicia-l2 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

msbutler left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

msbutler Feb 14, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kathancox commented Feb 14, 2025

rmloveland left a comment

Choose a reason for hiding this comment

kathancox commented Feb 6, 2025 •

edited

Loading

github-actions bot commented Feb 6, 2025 •

edited

Loading

netlify bot commented Feb 6, 2025 •

edited

Loading

netlify bot commented Feb 6, 2025 •

edited

Loading

netlify bot commented Feb 6, 2025 •

edited

Loading

msbutler Feb 14, 2025 •

edited

Loading