1.4.0#26

Merged

dqops merged 273 commits intomainfrom 1.4.0

May 29, 2024

This pull request is big! We're only showing the most recent 250 commits

Commits on May 13, 2024

Merge branch 'develop' of https://dev.azure.com/imagetemplates/documati.io/_git/dqoado into develop
piotrczarnas
committed
One compare operation fixed to protect from nulls on the data type.
piotrczarnas
committed
using dependency injection in TablesListerProvider
Patryk Jatczak
committed
Add symbolic link to the certficate store in Dockerfile
piotrczarnas
committed
merge
Patryk Jatczak
committed
Merged PR 2636: using dependency injection in TablesListerProvider

Patryk Jatczak
authored and
piotrczarnas
committed
Most recent first seen calculated for incidents on each connection.
piotrczarnas
committed
Default check patterns support disabling.
piotrczarnas
committed
Check patterns support using multiple filters, separated by a comma.
piotrczarnas
committed
Load only one instance of DuckDB jdbc connection and duplicate the connection when new connections are required.
piotrczarnas
committed
Configure additional daily partitioned default checks (nulls, distinct, anomalies, data type changes).
piotrczarnas
committed

Commits on May 14, 2024

fixed backgroud color on tableList
psychologianauki
committed
Merged PR 2637: 11803bfixed backgroud color on tableList

Aleksy Lisowski
authored and
piotrczarnas
committed
azure listTables more readable, throwing exception when directories on connection are used as one to many (multiple paths for single schema)
Patryk Jatczak
committed
Profiling and monitoring checks do not render time periods.
piotrczarnas
committed
Change log updated for 1.3.1 release
piotrczarnas
committed
Query grouping updated to support queries without a time period.
piotrczarnas
committed
added icons to header
psychologianauki
committed
made data sources svg line bolder
psychologianauki
committed
accepted values sensor time_period rendering on time_series presence
Patryk Jatczak
committed
removed bold font from column table quality status
psychologianauki
committed
accepted values sensor simplified
Patryk Jatczak
committed
made font smaller and setting since date = now - 1 month;
psychologianauki
committed
test found in set sensor no grouping fix
Patryk Jatczak
committed
Merged PR 2638: 11775 added icons to header

Aleksy Lisowski
authored and
piotrczarnas
committed
Merged PR 2641: 11739 removed bold font from column table quality status & made font smaller and setting since date = now - 1 month;

Aleksy Lisowski
authored and
piotrczarnas
committed
added disabled and description field to default pattern check
psychologianauki
committed
deleted console.logs
psychologianauki
committed
color of text default check pattern
psychologianauki
committed
Support creating a check pattern with a disabled flag and a description.
piotrczarnas
committed
Merged PR 2643: 11767 added disabled and description field to default pattern check

Aleksy Lisowski
authored and
piotrczarnas
committed
keeping clicked incidentConnections in local storage to detect if last incident was opened
psychologianauki
committed
sending checkTypes to request status on TableColumn page correctly
psychologianauki
committed
Merged PR 2644: 11770 keeping clicked incidentConnections in local storage to detect if last incide...

Aleksy Lisowski
authored and
piotrczarnas
committed
Merged PR 2645: 11752 sending checkTypes to request status on TableColumn page correctly

Aleksy Lisowski
authored and
piotrczarnas
committed

Commits on May 15, 2024

expected text in top values sensor grouping for mysql
Patryk Jatczak
committed
expected text in top values sensor grouping for next databases
Patryk Jatczak
committed
expected text in top values sensor grouping for sqlserver
Patryk Jatczak
committed
expected text in top values sensor grouping for trino, bigquery
Patryk Jatczak
committed
number found in set sensor: MAX(0.0)
Patryk Jatczak
committed
percentile sensor for bigquery grouping fix
Patryk Jatczak
committed
MAX(0.0) at one more bigquery sensor
Patryk Jatczak
committed
MAX(0.0) at text found in set percent, oracle grouping fix
Patryk Jatczak
committed
percentile sensors grouping fix
Patryk Jatczak
committed
File change detection uses a read-write user home context, in case that index files must be written.
piotrczarnas
committed
Loading issues related to a time period applies time window filter before selecting the partitions to load.
piotrczarnas
committed
fixed opening the incident directly from url
psychologianauki
committed
Protect from nulls when listening incidents.
piotrczarnas
committed
Merged PR 2646: 11828 fixed opening the incident directly from url

Aleksy Lisowski
authored and
piotrczarnas
committed
Merge branch 'develop' of https://dev.azure.com/imagetemplates/documati.io/_git/dqoado into develop
piotrczarnas
committed
added horizontal scrollbar to table list
psychologianauki
committed
refetching data 5 seconds after first request and fixed keeping homePage tab
psychologianauki
committed
Merged PR 2647: 11733 added horizontal scrollbar to table list

Aleksy Lisowski
authored and
piotrczarnas
committed
Merged PR 2648: refetching data 5 seconds after first request and fixed keeping homePage tab

Aleksy Lisowski
authored and
piotrczarnas
committed
made filePaths editable on tableDetails screen
psychologianauki
committed
Merged PR 2649: 11758 made filePaths editable on tableDetails screen

Aleksy Lisowski
authored and
piotrczarnas
committed

Commits on May 16, 2024

sqlserver percentile sensor grouping fix
Patryk Jatczak
committed
sqlserver contains ipv4 sensor reverted
Patryk Jatczak
committed
fixed two sqlserver sensors
Patryk Jatczak
committed
Merged PR 2650: sensor grouping fixed

Patryk Jatczak
authored and
piotrczarnas
committed
Connection settings for DuckDB describes azure auth mode fields
Patryk Jatczak
committed
Merge branch 'feature/11647_Consider_disabling_the_time_period_columns_for_monitoring_profiling_checks' into feature/11760_Docs_for_DuckDB_about_Azure_authentication_modes
Patryk Jatczak
committed
azure default credentials file in userhome use the service principal, hide sql server service principal authentication (not implemented) and added active_directory_default
Patryk Jatczak
committed
duckdb cli authentication modes
Patryk Jatczak
committed
fetching data correctly is data quality status is undefined
psychologianauki
committed
Change the label in the configuration section for the data dictionaries.
piotrczarnas
committed
Merge branch 'develop' of https://dev.azure.com/imagetemplates/documati.io/_git/dqoado into develop
piotrczarnas
committed
Merged PR 2651: 11844 fetching data correctly is data quality status is undefined

Aleksy Lisowski
authored and
piotrczarnas
committed
Update the documentation after changes to rendering profiling and monitoring checks.
piotrczarnas
committed
fixed layout on default check configuration
psychologianauki
committed
duckdb azure default credentials
Patryk Jatczak
committed
made description field as textarea
psychologianauki
committed
Azure default credentials docs for DuckDB
Patryk Jatczak
committed
setting correct label when editing shared credential
psychologianauki
committed
Merged PR 2654: Azure default credentials changes

Patryk Jatczak
authored and
piotrczarnas
committed
Merged PR 2652: 11849 fixed layout on default check configuration

Aleksy Lisowski
authored and
piotrczarnas
committed
Merged PR 2655: 11831 setting correct label when editing shared credential

Aleksy Lisowski
authored and
piotrczarnas
committed
improvements notification panel
psychologianauki
committed
Merged PR 2656: 11756 improvements notification panel

Aleksy Lisowski
authored and
piotrczarnas
committed
Swagger files regenerated.
piotrczarnas
committed
Merge branch 'develop' of https://dev.azure.com/imagetemplates/documati.io/_git/dqoado into develop
piotrczarnas
committed
added action icons to tablelist page
psychologianauki
committed
added onlick navigation to action buttons
psychologianauki
committed
rotate functionality init
psychologianauki
committed
Merged PR 2657: 11851 added action icons to tablelist page

Aleksy Lisowski
authored and
piotrczarnas
committed
added space between icons and labels and displaying just icons if width < 1600 px
psychologianauki
committed
Rest api for returning columns also returns the column DQ status (when loaded).
piotrczarnas
committed
Merge branch 'develop' of https://dev.azure.com/imagetemplates/documati.io/_git/dqoado into develop
piotrczarnas
committed
Merged PR 2658: 11857 added space between icons and labels and displaying just icons if width < 160...

Aleksy Lisowski
authored and
piotrczarnas
committed
Merge branch 'develop' of https://dev.azure.com/imagetemplates/documati.io/_git/dqoado into develop
piotrczarnas
committed
Rest api for global search for columns.
piotrczarnas
committed
Searching columns globally by labels filters only by labels assigned to the column.
piotrczarnas
committed

Commits on May 17, 2024

duckdb zstd compression extension
Patryk Jatczak
committed
Merge branch 'develop' of https://dev.azure.com/imagetemplates/documati.io/_git/dqoado into feature/11801_DuckDB_connector_should_automatically_add__snappy___zstd______file_extension_when_compressi…
Patryk Jatczak
committed
Incidents shown for the histogram are filtered by the incident start date, not just the time window.
piotrczarnas
committed
unit tests for sensor formatting updated
Patryk Jatczak
committed
percentile fixed, unit tests updated
Patryk Jatczak
committed
removed margin top on tablelist
psychologianauki
committed
Merged PR 2659: 11870 removed margin top on tablelist

Aleksy Lisowski
authored and
piotrczarnas
committed
expected texts in top values sensor fixed, unit tests updated - bigquery
Patryk Jatczak
committed
collapsable data quality dashboard section in user profile
psychologianauki
committed
expected texts in top values sensor updated - spark, databricks
Patryk Jatczak
committed
expected texts in top values sensor updated - presto, trino, snowflake, duckdb, mysql
Patryk Jatczak
committed
expected texts in top values sensor updated - postgresql, redshift
Patryk Jatczak
committed
Merged PR 2660: collapsable data quality dashboard section in user profile

Aleksy Lisowski
authored and
piotrczarnas
committed
Merged PR 2661: unit tests fixed - some sensors still need work

Patryk Jatczak
authored and
piotrczarnas
committed
global columns page init
psychologianauki
committed

Commits on May 20, 2024

expected texts in top values sensor updated - oracle
Patryk Jatczak
committed
expected texts in top values sensor updated - sqlserver
Patryk Jatczak
committed
expected texts in top values sensor updated - sqlserver
Patryk Jatczak
committed
percentile sensor updated - databricks, spark, presto, trino
Patryk Jatczak
committed
percentile sensor updated - sqlserver
Patryk Jatczak
committed
Merged PR 2662: Grouping solved in sensors: percentile and expected texts in top values

Patryk Jatczak
authored and
piotrczarnas
committed
Merge branch 'develop' into feature/11799_We_need_to_retest_accessing_S3_and_Azure_in_DuckDB_after_the_refactoring_to_use_a_single_duck_db_instance
Patryk Jatczak
committed
removed the use of memory tables in duckdb
Patryk Jatczak
committed
Allow searching for columns by data type using patterns (star).
piotrczarnas
committed
developed columnList on schema and connection level
psychologianauki
committed
Merge branch 'develop' of https://dev.azure.com/imagetemplates/documati.io/_git/dqoado into feature/global_list_of_columns
psychologianauki
committed
fixed sending findColumn request (column dataType)
psychologianauki
committed
added schema input
psychologianauki
committed
Merge branch 'develop' of https://dev.azure.com/imagetemplates/documati.io/_git/dqoado into feature/global_list_of_columns
psychologianauki
committed
Search columns by table patters.
piotrczarnas
committed
Merged PR 2663: feature global columnList

Aleksy Lisowski
authored and
piotrczarnas
committed
Return an empty column DQ status model when the status for the table is missing.
piotrczarnas
committed
labels attachment screen
psychologianauki
committed

Commits on May 21, 2024

Commits on May 22, 2024

Commits on May 23, 2024

JinjaTemplateRenderParametersProvider use dependency injection, SampleTableMetadata creation does not use a random names for schema and table
Patryk Jatczak
committed
duckdb accuracy sensors can be run on physical files
Patryk Jatczak
committed
added margin and disabled sorting on table and column list
psychologianauki
committed
added spinner if results are loading
psychologianauki
committed
integrity sensors use referenced table interface
Patryk Jatczak
committed
spinner
psychologianauki
committed
wrong table spec fixed on referenced table
Patryk Jatczak
committed
Moved TableListView to 1 component
psychologianauki
committed
Using 1 ColumnListView component
psychologianauki
committed
duckdb parsable to boolean trims whitespaces, tests asserts adjusted to present whitespaces
Patryk Jatczak
committed
date format integration test asset adjustment
Patryk Jatczak
committed
Merge branch 'develop' of https://dev.azure.com/imagetemplates/documati.io/_git/dqoado into feature/11799_We_need_to_retest_accessing_S3_and_Azure_in_DuckDB_after_the_refactoring_to_use_a_single_du…
Patryk Jatczak
committed
azure default credetnial is set in duckdb credential manager
Patryk Jatczak
committed
Merged PR 2670: Duckdb works now

Patryk Jatczak
authored and
piotrczarnas
committed
Update to the python client and documentation with updated DuckDB sensors.
piotrczarnas
committed
refresh button
psychologianauki
committed
decreased size of dimesion overview
psychologianauki
committed
aligned spinner correctly
psychologianauki
committed
not displaying actions if data_quality_status is undefined
psychologianauki
committed
decreased size of font
psychologianauki
committed
tooltip
psychologianauki
committed
Merged PR 2671: global_tables_columns_spinner_sorting_margins_fonts

Aleksy Lisowski
authored and
piotrczarnas
committed

Commits on May 24, 2024

string_test_data csv test file got quoting on surrounded_by_whitespace column
Patryk Jatczak
committed
spark whitespace tests adjusted
Patryk Jatczak
committed
changing width of logo if user is not on homePage
psychologianauki
committed
Merged PR 2672: 11896 changing width of logo if user is not on homePage

Aleksy Lisowski
authored and
piotrczarnas
committed
whitespace tests adjusted for: mysql, bigquery, postgres, presto, trino, athena, snowflake, databricks, redshift, singlestore
Patryk Jatczak
committed
oracle whitespace sensors fixed
Patryk Jatczak
committed
added action buttons to incident page
psychologianauki
committed
decreased size of icons
psychologianauki
committed
Merged PR 2673: 11933 added action buttons to incident page

Aleksy Lisowski
authored and
piotrczarnas
committed
sqlserver whitespace sensors fixed
Patryk Jatczak
committed
Merged PR 2674: Whitespaces are tested properly now, fixed invalid whitespace sensors

Patryk Jatczak
authored and
piotrczarnas
committed
Merge branch 'develop' of https://dev.azure.com/imagetemplates/documati.io/_git/dqoado into feature/11801_DuckDB_connector_should_automatically_add__snappy___zstd______file_extension_when_compressi…
Patryk Jatczak
committed
opening incident tab correctly
psychologianauki
committed
getting IncidentsHistograms correctly
psychologianauki
committed
Merged PR 2675: incident tabs

Aleksy Lisowski
authored and
piotrczarnas
committed

Commits on May 25, 2024

Commits on May 27, 2024

new icons for profiling, monitoring, partitioned checks and data sources
Patryk Jatczak
committed
sync picture refresh button on columnsListView
Patryk Jatczak
committed
dqops logo size does not move another header buttons
Patryk Jatczak
committed
Merged PR 2676: new icons for profiling, monitoring, partitioned checks and data sources

Patryk Jatczak
authored and
piotrczarnas
committed
Merged PR 2677: sync picture refresh button on columnsListView

Patryk Jatczak
authored and
piotrczarnas
committed
Merged PR 2678: dqops logo size does not move another header buttons

Patryk Jatczak
authored and
piotrczarnas
committed
labels section smaller
Patryk Jatczak
committed
Merged PR 2679: labels section smaller

Patryk Jatczak
authored and
piotrczarnas
committed
column and table dashboard layout fixes, pagination on right, dimensions text lighter, dashboard icon, table names wider
Patryk Jatczak
committed
merge
Patryk Jatczak
committed
Updated changelog.
piotrczarnas
committed
Rule calibration for float values applies rounding to generate a user friendly value.
piotrczarnas
committed

Commits on May 28, 2024

bigger logo, label size fix
Patryk Jatczak
committed
Data Quality KPI is rotated
Patryk Jatczak
committed
fixed tooltips
Patryk Jatczak
committed
tooltip mistaken name fix, font size fix
Patryk Jatczak
committed
fix: vertical dimension names disappear on loading data, page margins
Patryk Jatczak
committed
dimensions letter spacing, paging only when data are available, labels slicing, labels wrapping
Patryk Jatczak
committed
Merged PR 2680: Dashboard view fixes

Patryk Jatczak
authored and
piotrczarnas
committed
refresh button on columnsView
psychologianauki
committed
linking to column page correctly
psychologianauki
committed
Merge branch 'develop' of https://dev.azure.com/imagetemplates/documati.io/_git/dqoado into bug/global_tables_layout_changes
psychologianauki
committed
Merged PR 2681: global tables & columns layout changes

Aleksy Lisowski
authored and
piotrczarnas
committed
duckdb integration tests do not rely on date format of csv sniffer
Patryk Jatczak
committed
Merged PR 2682: duckdb updated, integration tests do not rely on date format of csv sniffer

Patryk Jatczak
authored and
piotrczarnas
committed
code comments
Patryk Jatczak
committed
Merge branch 'develop' of https://dev.azure.com/imagetemplates/documati.io/_git/dqoado into develop
Patryk Jatczak
committed
duckdb compression got no_ext options
Patryk Jatczak
committed

Commits on May 29, 2024

decreased font size & margin of dimension cell & showing dqkpi in tooltip
psychologianauki
committed
Merge branch 'develop' of https://dev.azure.com/imagetemplates/documati.io/_git/dqoado into feature/11801_DuckDB_connector_should_automatically_add__snappy___zstd______file_extension_when_compressi…
Patryk Jatczak
committed
compression combo-box component, noCompressionExtension moved to csv,json,parquet file formats
Patryk Jatczak
committed
json and parquet compressionEnumOptions fixed
Patryk Jatczak
committed
Small improvements on Home page - changed eBook image and button text alignment
Andrzej Nagalski
committed
Small improvements on Home page
Andrzej Nagalski
committed
Fixes to support recalibration and disabling checks.
piotrczarnas
committed
tooltips fixed for column view
Patryk Jatczak
committed
pagination is shown over 10 items, 5 items removed from pagination
Patryk Jatczak
committed
pagination is shown over 10 items, 5 items removed from pagination
Patryk Jatczak
committed
text length limited
Patryk Jatczak
committed
removed duplicated function
Patryk Jatczak
committed
added missing imports
Patryk Jatczak
committed
Do not create empty columns in a parquet table if they were not present.
piotrczarnas
committed
table columns screen improvemetns
psychologianauki
committed
Merged PR 2683: 11950 decreased font size & margin of dimension cell & showing dqkpi in tooltip

Aleksy Lisowski
authored and
piotrczarnas
committed
Merged PR 2685: no compression extension feature

Patryk Jatczak
authored and
piotrczarnas
committed
Merged PR 2686: bunch of ui improvements

Patryk Jatczak
authored and
piotrczarnas
committed
Merged PR 2687: tables columns screen layout fix

Aleksy Lisowski
authored and
piotrczarnas
committed
Merge branch 'develop' of https://dev.azure.com/imagetemplates/documati.io/_git/dqoado into develop
piotrczarnas
committed
Version upgrade to 1.4.0
piotrczarnas
committed
Update python client and documentation.
piotrczarnas
committed