Skip to content

Commit

Permalink
Updates to the titles in the documentation
Browse files Browse the repository at this point in the history
  • Loading branch information
piotrczarnas committed Dec 13, 2024
1 parent 6d2a9ec commit b56331d
Show file tree
Hide file tree
Showing 329 changed files with 766 additions and 698 deletions.
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
---
title: How to detect data accuracy issues
title: How to Detect Data Accuracy Issues? Examples and Best Practices
---
# How to detect data accuracy issues
Data accuracy checks in DQOps compare an aggregated value in a tested table to the same aggregated value in a reference table.
# How to Detect Data Accuracy Issues? Examples and Best Practices
Data accuracy checks compare an aggregated value in a tested table to the same aggregated value in a reference table to detect differences.

The accuracy checks in DQOps are configured in the `accuracy` category of data quality checks.

Expand Down
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
---
title: How to detect anomalies in numeric data
title: How to Detect Anomalies in Numeric Data? Examples and Best Practices
---
# How to detect anomalies in numeric data
Read this guide to learn how to detect anomaly data quality issues in numeric data using DQOps.
# How to Detect Anomalies in Numeric Data? Examples and Best Practices
Read this guide to learn how to detect data anomalies (outliers) in numeric data that has timestamp columns to identify time-series with historic values.

The data quality checks are configured in the `anomaly` category in DQOps.

## What is an anomaly in data
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: How to detect values not matching patterns
title: How to Detect Values not Matching Patterns? Examples
---
# How to detect values not matching patterns
# How to Detect Values not Matching Patterns? Examples
Read this guide to learn how to validate column values if they match patterns, such as phone numbers, emails, or any regular expression.

The pattern match checks are configured in the `patterns` category in DQOps.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: How to detect whitespace and null value placeholders
title: How to Detect Whitespace and NULL Value Placeholders? Examples
---
# How to detect whitespace and null value placeholders
# How to Detect Whitespace and NULL Value Placeholders? Examples
Read this guide to learn how to detect whitespaces, such as spaces, tabs, or special texts equivalent to a null value in text columns using SQL checks.

The data quality checks for detecting whitespace and empty value placeholders are configured in the `whitespace` category in DQOps.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: How to measure the percentage of true and false values
title: How to Measure Percentage of TRUE and FALSE Values? Examples
---
# How to measure the percentage of true and false values
# How to Measure Percentage of TRUE and FALSE Values? Examples
Read this guide to learn how to measure the percentage of true and false boolean values and how to set up data quality checks that assert the valid range.

The data quality checks for bool columns are configured in the `bool` category in DQOps.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: How to detect out-of-range numbers in data
title: How to detect out-of-range numbers? Examples and Best Practices
---
# How to detect out-of-range numbers in data
# How to detect out-of-range numbers? Examples and Best Practices
Read this guide to learn how to detect numeric values that are out of an accepted range and how to raise a data quality issue.

The data quality checks responsible for numeric values are configured in the `numeric` category in DQOps.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: How to detect out-of-range text values
title: How to detect out-of-range text values? Examples
---
# How to detect out-of-range text values
# How to detect out-of-range text values? Examples
Read this guide to learn how to find text values that are too short or too long, which are most likely invalid values stored in a database.

The data quality checks that detect issues with too short or too long texts are configured in the `text` category in DQOps.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: How to detect referential integrity issues and missing keys
title: How to detect referential integrity issues and missing keys, examples
---
# How to detect referential integrity issues and missing keys
# How to detect referential integrity issues and missing keys, examples
Read this guide to learn how to detect referential integrity issues, such as missing keys in dictionary tables or wrong foreign keys.

The data quality checks that detect missing keys are configured in the `integrity` category in DQOps.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: How to detect data type changes in text columns
title: How to detect data type changes in text columns, with examples
---
# How to detect data type changes in text columns
# How to detect data type changes in text columns, with examples
Read this guide to learn how DQOps detect the effective data type in text columns. Data type detection verifies that values are castable to a given data type.

The data quality checks that detect data types in text columns are configured in the `datatype` category in DQOps.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: How to detect duplicate data and measure data uniqueness
title: How to detect duplicate data and measure uniqueness, examples
---
# How to detect duplicate data and measure data uniqueness
# How to detect duplicate data and measure uniqueness, examples
Read this guide to learn how to detect duplicate data and how distinct values, data uniqueness, and duplicate data are related to each other.

The data uniqueness and duplicate detection checks are configured in the `uniqueness` category in DQOps.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: How to monitor data volume and detect empty tables
title: How to monitor data volume and detect empty tables, examples
---
# How to monitor data volume and detect empty tables
# How to monitor data volume and detect empty tables, examples
Read this guide to learn how to monitor the number of rows in tables, detect empty tables, and detect unexpected increases or decreases in the volume.

The table volume monitoring checks are configured in the `volume` category in DQOps.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: How to detect empty and incomplete columns
title: How to detect empty and incomplete columns with examples
---
# How to detect empty and incomplete columns
# How to detect empty and incomplete columns with examples
Read this guide to learn how to detect empty columns or incomplete columns containing too many null values in a dataset.

The data quality checks that detect empty and incomplete columns are configured in the `nulls` category in DQOps.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: How to detect invalid dates
title: How to Detect Invalid Dates? Examples and Best Practices
---
# How to detect invalid dates
# How to Detect Invalid Dates? Examples and Best Practices
Read this guide to learn how to detect invalid dates in data, such as dates in the future or out of reasonable range and dates in a wrong format.

The data quality checks that detect invalid dates are configured in the `datetime` category in DQOps.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: How to detect PII (Personal Identifiable Information) data
title: How to Detect PII Data? Examples and Best Practices
---
# How to detect PII (Personal Identifiable Information) data
# How to Detect PII Data? Examples and Best Practices
Read this guide to learn how to detect the presence of Personal Identifiable Information such as emails or phone numbers in tables.

The data quality checks that detect PII values are configured in the `pii` category in DQOps.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
---
title: How to detect table schema changes
title: How to Detect Table Schema Changes? Examples and Best Practices
---
# How to detect table schema changes
Read this guide to learn how DQOps detects table schema changes, such as missing columns, data type changes, or reordering columns.
# How to Detect Table Schema Changes? Examples and Best Practices
Read this guide to learn how to detect table schema changes, such as missing columns, missing tables, data type changes, or reordering the columns.

The table schema change detection checks are configured in the `schema` category in DQOps.

Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: How to measure data timeliness, freshness and staleness metrics
title: How to Measure Data Timeliness, Freshness and Staleness Metrics
---
# How to measure data timeliness, freshness and staleness metrics
# How to Measure Data Timeliness, Freshness and Staleness Metrics
Read this guide to learn how to measure data timeliness metrics, such as freshness (the most recent data) or staleness (when the data was loaded).

The timeliness data quality checks are configured in the `timeliness` category in DQOps.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: How to reconcile data with table comparison checks
title: How to Reconcile Data with Table Comparison Checks, Examples
---
# How to reconcile data with table comparison checks
# How to Reconcile Data with Table Comparison Checks, Examples
Read this guide to learn how to reconcile data across data sources to find discrepancies using table comparison data quality checks.

Data reconciliation checks are defined in the `comparisons` category in DQOps.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
---
title: How to monitor table availability
title: How to Monitor Table Availability? Examples and Best Practices
---
# How to monitor table availability
Read this guide to learn how to enable table availability monitoring. DQOps will detect when the table is not available for use.
# How to Monitor Table Availability? Examples and Best Practices
Read this guide to learn how to enable table availability monitoring. You can detect when the table is not available, or a table is physically corrupted.

The table availability monitoring checks are configured in the `availability` category in DQOps.

Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: How to validate accepted values in columns
title: How to Validate Columns to Data Dictionaries? Examples
---
# How to validate accepted values in columns
# How to Validate Columns to Data Dictionaries? Examples
Read this guide to learn how to verify that text and numeric columns contain accepted values. Assert that all expected values are used in tested columns.

## Accepted values category
Expand Down
2 changes: 1 addition & 1 deletion docs/categories-of-data-quality-checks/index.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Categories of data quality checks
# Categories of Data Quality Checks - Full List
Read this guide to learn how the types of popular data quality checks are divided into categories, and what categories of checks are supported by DQOps.


Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: expected numbers in use count data quality checks
title: Expected numbers in use count data quality checks, SQL examples
---
# expected numbers in use count data quality checks
# Expected numbers in use count data quality checks, SQL examples

A column-level check that counts unique values in a numeric column and counts how many values out of a list of expected numeric values were found in the column.
The check raises a data quality issue when the threshold for the maximum number of missing has been exceeded (too many expected values were not found in the column).
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: expected text values in use count data quality checks
title: Expected text values in use count data quality checks, SQL examples
---
# expected text values in use count data quality checks
# Expected text values in use count data quality checks, SQL examples

A column-level check that counts unique values in a text column and counts how many values out of a list of expected string values were found in the column.
The check raises a data quality issue when the threshold for the maximum number of missing has been exceeded (too many expected values were not found in the column).
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: expected texts in top values count data quality checks
title: Expected texts in top values count data quality checks, SQL examples
---
# expected texts in top values count data quality checks
# Expected texts in top values count data quality checks, SQL examples

A column-level check that counts how many expected text values are among the TOP most popular values in the column.
The check will first count the number of occurrences of each column's value and will pick the TOP X most popular values (configurable by the 'top' parameter).
Expand Down
4 changes: 2 additions & 2 deletions docs/checks/column/accepted_values/index.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: column level accepted values data quality checks
title: List of column level accepted values data quality checks
---
# column level accepted values data quality checks
# List of column level accepted values data quality checks

This is a list of accepted_values column data quality checks supported by DQOps and a brief description of what data quality issued they detect.

Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: number found in set percent data quality checks
title: Number found in set percent data quality checks, SQL examples
---
# number found in set percent data quality checks
# Number found in set percent data quality checks, SQL examples

A column-level check that calculates the percentage of rows for which the tested numeric column contains a value from a set of expected values.
Columns with null values are also counted as a passing value (the sensor assumes that a 'null' is also an expected and accepted value).
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: text found in set percent data quality checks
title: Text found in set percent data quality checks, SQL examples
---
# text found in set percent data quality checks
# Text found in set percent data quality checks, SQL examples

A column-level check that calculates the percentage of rows for which the tested text column contains a value from a set of expected values.
Columns with null values are also counted as a passing value (the sensor assumes that a 'null' is also an expected and accepted value).
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: text valid country code percent data quality checks
title: Text valid country code percent data quality checks, SQL examples
---
# text valid country code percent data quality checks
# Text valid country code percent data quality checks, SQL examples

This check measures the percentage of text values that are valid two-letter country codes.
It raises a data quality issue when the percentage of valid country codes (excluding null values) falls below a minimum accepted rate.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: text valid currency code percent data quality checks
title: Text valid currency code percent data quality checks, SQL examples
---
# text valid currency code percent data quality checks
# Text valid currency code percent data quality checks, SQL examples

This check measures the percentage of text values that are valid currency names. It raises a data quality issue when the percentage of valid currency names (excluding null values) falls below a minimum accepted rate.

Expand Down
4 changes: 2 additions & 2 deletions docs/checks/column/accuracy/index.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: column level accuracy data quality checks
title: List of column level accuracy data quality checks
---
# column level accuracy data quality checks
# List of column level accuracy data quality checks

This is a list of accuracy column data quality checks supported by DQOps and a brief description of what data quality issued they detect.

Expand Down
4 changes: 2 additions & 2 deletions docs/checks/column/accuracy/total-average-match-percent.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: total average match percent data quality checks
title: Total average match percent data quality checks, SQL examples
---
# total average match percent data quality checks
# Total average match percent data quality checks, SQL examples

A column-level check that ensures that the difference between the average value in the tested column and the average value of another column in the referenced table is below the maximum accepted percentage of difference.
This check runs an SQL query with an INNER JOIN clause to join another (referenced) table that must be defined in the same database.
Expand Down
4 changes: 2 additions & 2 deletions docs/checks/column/accuracy/total-max-match-percent.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: total max match percent data quality checks
title: Total max match percent data quality checks, SQL examples
---
# total max match percent data quality checks
# Total max match percent data quality checks, SQL examples

A column-level check that ensures that the difference between the maximum value in the tested column and the maximum value in another column in a referenced table is below a maximum accepted percentage of difference.
This check runs an SQL query with an INNER JOIN clause to join another (referenced) table that must be defined in the same database.
Expand Down
4 changes: 2 additions & 2 deletions docs/checks/column/accuracy/total-min-match-percent.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: total min match percent data quality checks
title: Total min match percent data quality checks, SQL examples
---
# total min match percent data quality checks
# Total min match percent data quality checks, SQL examples

A column-level check that ensures that the difference between the minimum value in the tested column and the minimum value in another column in a referenced table is below a maximum accepted percentage of difference.
This check runs an SQL query with an INNER JOIN clause to join another (referenced) table that must be defined in the same database.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: total not null count match percent data quality checks
title: Total not null count match percent data quality checks, SQL examples
---
# total not null count match percent data quality checks
# Total not null count match percent data quality checks, SQL examples

A column-level check that ensures that the difference between the count of null values in the tested column and the count of null values in another column in a referenced table is below a maximum accepted percentage of difference.
This check runs an SQL query with an INNER JOIN clause to join another (referenced) table that must be defined in the same database.
Expand Down
4 changes: 2 additions & 2 deletions docs/checks/column/accuracy/total-sum-match-percent.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: total sum match percent data quality checks
title: Total sum match percent data quality checks, SQL examples
---
# total sum match percent data quality checks
# Total sum match percent data quality checks, SQL examples

A column-level check that ensures that the difference between the sum of all values in the tested column and the sum of values in another column in a referenced table is below a maximum accepted percentage of difference.
This check runs an SQL query with an INNER JOIN clause to join another (referenced) table that must be defined in the same database.
Expand Down
4 changes: 2 additions & 2 deletions docs/checks/column/anomaly/index.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: column level anomaly data quality checks
title: List of column level anomaly data quality checks
---
# column level anomaly data quality checks
# List of column level anomaly data quality checks

This is a list of anomaly column data quality checks supported by DQOps and a brief description of what data quality issued they detect.

Expand Down
4 changes: 2 additions & 2 deletions docs/checks/column/anomaly/max-anomaly.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: max anomaly data quality checks
title: Max anomaly data quality checks, SQL examples
---
# max anomaly data quality checks
# Max anomaly data quality checks, SQL examples

This check finds a maximum value in a numeric column and detects anomalies in a time series of previous maximum values.
It raises a data quality issue when the current maximum value is in the top *anomaly_percent* percentage of the most outstanding
Expand Down
4 changes: 2 additions & 2 deletions docs/checks/column/anomaly/mean-anomaly.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: mean anomaly data quality checks
title: Mean anomaly data quality checks, SQL examples
---
# mean anomaly data quality checks
# Mean anomaly data quality checks, SQL examples

This check calculates a mean (average) of values in a numeric column and detects anomalies in a time series of previous averages.
It raises a data quality issue when the mean is in the top *anomaly_percent* percentage of the most outstanding values in the time series.
Expand Down
4 changes: 2 additions & 2 deletions docs/checks/column/anomaly/mean-change-1-day.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: mean change 1 day data quality checks
title: Mean change 1 day data quality checks, SQL examples
---
# mean change 1 day data quality checks
# Mean change 1 day data quality checks, SQL examples

This check detects that the mean (average) of numeric values has changed more than *max_percent* from the mean value measured one day ago (yesterday).

Expand Down
4 changes: 2 additions & 2 deletions docs/checks/column/anomaly/mean-change-30-days.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: mean change 30 days data quality checks
title: Mean change 30 days data quality checks, SQL examples
---
# mean change 30 days data quality checks
# Mean change 30 days data quality checks, SQL examples

This check detects that the mean (average) of numeric values has changed more than *max_percent* from the mean value measured thirty days ago.
This check aims to overcome a monthly seasonability and compare a value to a similar value a month ago.
Expand Down
Loading

0 comments on commit b56331d

Please sign in to comment.