From 64f8e69dd4afc48f247b576d7b69b6ae7c101c0b Mon Sep 17 00:00:00 2001 From: martonvago Date: Thu, 27 Feb 2025 18:29:59 +0000 Subject: [PATCH 1/5] docs: :memo: add docs for data types --- docs/design/interface/data-types.qmd | 219 +++++++++++++++++++++++++++ 1 file changed, 219 insertions(+) create mode 100644 docs/design/interface/data-types.qmd diff --git a/docs/design/interface/data-types.qmd b/docs/design/interface/data-types.qmd new file mode 100644 index 00000000..f27836fc --- /dev/null +++ b/docs/design/interface/data-types.qmd @@ -0,0 +1,219 @@ +--- +title: "Data types supported by Sprout" +--- + +### Overview + +Sprout implements the Frictionless Data Package standard and aims to support the [data types](https://datapackage.org/standard/table-schema/#field-types) it defines. However, Sprout not only describes data with metadata but also transforms it into a tidy Parquet file, ready for querying (see [Outputs](/docs/design/interface/outputs.qmd#files) and [Why Parquet](https://decisions.seedcase-project.org/why-parquet/) for more details). As a result, Sprout supports only data types that are compatible (or can be made compatible) with Parquet storage. + +Below, we list Frictionless data types as used in Sprout and give a precise definition for each. Any differences from the Frictionless specification are noted. + +| Frictionless | Polars | Arrow | Parquet | +|--------------|---------------------|------------------------------|------------------------| +| `string` | `String` | `string` | `BYTE_ARRAY` (String) | +| `number` | `Float64` | `float64` | `DOUBLE` | +| `integer` | `Int64` | `int64` | `INT64` | +| `boolean` | `Boolean` | `bool_` | `BOOLEAN` | +| `datetime` | `Datetime` | `timestamp[ms, tz]` | `INT64` (Timestamp) | +| `date` | `Date` | `date32[day]` | `INT32` (Date) | +| `time` | `Time` | `time64[ns]` | `INT64` (Time) | +| `year` | `Int32` | `int32` | `INT32` | +| `yearmonth` | `Date` | `date32[day]` | `INT32` (Date) | +| `duration` | `String` | `string` | `BYTE_ARRAY` (String) | +| `geopoint` | `Array[Float64, 2]` | `fixed_size_list[double, 2]` | `FIXED_LEN_BYTE_ARRAY` | +| `geojson` | `String` | `string` | `BYTE_ARRAY` (String) | +| `object` | `String` | `string` | `BYTE_ARRAY` (String) | +| `array` | `String` | `string` | `BYTE_ARRAY` (String) | +| `any` | `String` | `string` | `BYTE_ARRAY` (String) | + +: Mappings of Frictionless data types in Sprout + +### String + +A sequence of UTF-8 encoded characters. + +Supported formats: `default`, `email`, `uri`, `binary`, `uuid`. + +### Number + +A number with or without a decimal part. Precision is not reflected in the number of digits specified in the decimal part (`2.0` is not distinct from `2.00` ). + +Precision of up to 16 significant digits. + +| Name | Allowed values | Examples | +|----------------------------|-----------------------------------------|----------------------------| +| decimal indicator | `.` | `12.56`, `50.000` | +| leading sign | `+`(default) or `-` | `12.56`, `+5.00`, `-12` | +| leading and trailing zeros | `0` (optional) | `12.56`, `0012`, `12.5600` | +| exponent | `E` | `5E10`, `12.56E-3` | +| special values | `NaN`, `inf`, `-inf` (case-insensitive) | | + +: `number` format options + +The configuration options `decimalChar`, `groupChar` and `bareNumber` are not yet supported. + +### Integer + +A whole number with no decimal part. + +| Name | Allowed values | Examples | +|---------------|----------------|------------| +| leading sign | no sign or `-` | `5`, `-12` | +| leading zeros | `0` (optional) | `005` | + +: `integer` format options + +The configuration options `groupChar` and `bareNumber` are not yet supported. + +### Boolean + +One of two possible values: true or false. + +Sprout supports the default notation for truth values in Frictionless: + +- All values in `["true", "True", "TRUE", "1"]` are interpreted as true. +- All values in `["false", "False", "FALSE", "0"]` are interpreted as false. + +Setting custom `trueValues` and `falseValues` is not yet supported. + +### Datetime + +A date with a time and optional timezone. + +| Name | Allowed values | Examples | +|----------|----------------------------------------------|-----------------------------------------------| +| without milliseconds or timezone | `YYYY-MM-DDTHH:MM:SS` | `2002-10-12T12:04:15`, `0202-10-10T02:30:00` | +| with milliseconds | `YYYY-MM-DDTHH:MM:SS.sss` | `2002-10-12T12:04:15.3`, `0202-10-10T02:30:00.345` | +| with timezone | `YYYY-MM-DDTHH:MM:SSHH:MM` | `2002-10-12T12:04:15+05:00`, `0202-10-10T02:30:00-01:00` | +| with milliseconds and timezone | `YYYY-MM-DDTHH:MM:SS.sssHH:MM` | `2002-10-12T12:04:15.3+05:00`, `0202-10-10T02:30:00.345-01:00` | +| shorthand for UTC | `YYYY-MM-DDTHH:MM:SS(.sss)Z` | `2002-10-12T12:04:15Z`, `0202-10-10T02:30:00.345Z` | + +: `datetime` format options + +**Restrictions:** + +- Setting a custom `datetime` pattern in the `format` property is not yet supported. The `any` format is not supported. +- Negative `datetime` values are not supported. +- Years with more than 4 digits are not supported. +- Mixing `datetime` values with and without a timezone in one column is not allowed. +- When `datetime` values in a column have a timezone, they must all have the same timezone. + +### Date + +A date without a time. + +Expected format: `YYYY-MM-DD`. + +Example: `2022-12-09`. + +**Restrictions:** + +- Setting a custom `date` pattern in the `format` property is not yet supported. The `any` format is not supported. +- Negative `date` values are not supported. +- Years with more than 4 digits are not supported. +- `date` values with a timezone are not supported. + +### Time + +A time without a date. + +| Name | Allowed values | Examples | +|----------------------|-------------------|---------------------------------| +| without microseconds | `HH:MM:SS` | `12:04:15` | +| with microseconds | `HH:MM:SS.ssssss` | `12:04:15.3`, `02:30:00.345345` | + +: `time` format options + +**Restrictions:** + +- Setting a custom `time` pattern in the `format` property is not yet supported. The `any` format is not supported. +- `time` values with a timezone are not supported. + +### Year + +A calendar year without month or day. + +Expected format: `YYYY`. Negative `year` values are allowed. + +Examples: `2022`, `-1000`, `0005`. + +**Restrictions:** + +- `year` values with a timezone are not supported. + +### Yearmonth + +A specific month in a specific year. + +Expected format: `YYYY-MM`. + +Example: `2022-12`. + +The underlying representation of `yearmonth` is `date`. + +**Restrictions:** + +- Negative `yearmonth` values are not supported. +- Years with more than 4 digits are not supported. +- `yearmonth` values with a timezone are not supported. + +### Duration + +A duration of time. + +Expected pattern: `PnYnMnDTnHnMnS`. See the [definition of the Frictionless type](https://datapackage.org/standard/table-schema/#duration) for more information. + +Example: `P1Y2M3DT10H30M45.343S`. + +The number of seconds may include decimal digits to arbitrary precision. + +**Restrictions:** + +- The underlying representation of `duration` is `string`. Sprout does not attempt to parse `duration` values into a data type that is aware of the various time units contained within a `duration` value. +- As a consequence, constraints relying on the numeric comparison of `duration` values are not supported. These constraints are: `minimum`, `maximum`, `exclusiveMinimum`, `exclusiveMaximum`. + +### Geopoint + +A geographic point. + +Expected format: `LAT, LONG`. The space is optional. + +Examples: `45.50, 90.50`, `45.50,90.50`, `-45.50, -90.50`. + +The underlying representation is an array of two `number`s. + +**Restrictions:** + +- Other `geopoint` formats are not yet supported. + +### Array + +A JSON array. Must be well-formed [JSON](http://json.org/). + +The underlying representation is `string`. + +### Object + +A JSON object. Must be well-formed [JSON](http://json.org/). + +The underlying representation is `string`. + +### Geojson + +A JSON object compliant with the [GeoJSON](http://geojson.org/) or [TopoJSON](https://github.com/topojson/topojson-specification/blob/master/README.md) specification. + +The underlying representation is `string`. + +**Restrictions:** + +- `geojson` values are treated as plain `object`s. They are not checked against the GeoJSON or TopoJSON specification. + +### Any + +Unspecified or mixed values. + +The underlying representation is `string`. + +### List + +Not supported. From 45bfaaaa8761381eb35e735fa18acaef0230b7cc Mon Sep 17 00:00:00 2001 From: martonvago <57952344+martonvago@users.noreply.github.com> Date: Fri, 28 Feb 2025 11:38:00 +0000 Subject: [PATCH 2/5] docs: :memo: improve formatting Co-authored-by: Luke W. Johnston --- docs/design/interface/data-types.qmd | 61 +++++++++++++--------------- 1 file changed, 28 insertions(+), 33 deletions(-) diff --git a/docs/design/interface/data-types.qmd b/docs/design/interface/data-types.qmd index f27836fc..95e245df 100644 --- a/docs/design/interface/data-types.qmd +++ b/docs/design/interface/data-types.qmd @@ -1,8 +1,8 @@ --- -title: "Data types supported by Sprout" +title: "Data types" +description: "The data types Sprout supports" --- -### Overview Sprout implements the Frictionless Data Package standard and aims to support the [data types](https://datapackage.org/standard/table-schema/#field-types) it defines. However, Sprout not only describes data with metadata but also transforms it into a tidy Parquet file, ready for querying (see [Outputs](/docs/design/interface/outputs.qmd#files) and [Why Parquet](https://decisions.seedcase-project.org/why-parquet/) for more details). As a result, Sprout supports only data types that are compatible (or can be made compatible) with Parquet storage. @@ -28,13 +28,13 @@ Below, we list Frictionless data types as used in Sprout and give a precise defi : Mappings of Frictionless data types in Sprout -### String +## String A sequence of UTF-8 encoded characters. Supported formats: `default`, `email`, `uri`, `binary`, `uuid`. -### Number +## Number A number with or without a decimal part. Precision is not reflected in the number of digits specified in the decimal part (`2.0` is not distinct from `2.00` ). @@ -52,7 +52,7 @@ Precision of up to 16 significant digits. The configuration options `decimalChar`, `groupChar` and `bareNumber` are not yet supported. -### Integer +## Integer A whole number with no decimal part. @@ -65,7 +65,7 @@ A whole number with no decimal part. The configuration options `groupChar` and `bareNumber` are not yet supported. -### Boolean +## Boolean One of two possible values: true or false. @@ -76,7 +76,7 @@ Sprout supports the default notation for truth values in Frictionless: Setting custom `trueValues` and `falseValues` is not yet supported. -### Datetime +## Datetime A date with a time and optional timezone. @@ -98,13 +98,12 @@ A date with a time and optional timezone. - Mixing `datetime` values with and without a timezone in one column is not allowed. - When `datetime` values in a column have a timezone, they must all have the same timezone. -### Date +## Date A date without a time. -Expected format: `YYYY-MM-DD`. - -Example: `2022-12-09`. +- Expected format: `YYYY-MM-DD`. +- Example: `2022-12-09`. **Restrictions:** @@ -113,7 +112,7 @@ Example: `2022-12-09`. - Years with more than 4 digits are not supported. - `date` values with a timezone are not supported. -### Time +## Time A time without a date. @@ -129,25 +128,23 @@ A time without a date. - Setting a custom `time` pattern in the `format` property is not yet supported. The `any` format is not supported. - `time` values with a timezone are not supported. -### Year +## Year A calendar year without month or day. -Expected format: `YYYY`. Negative `year` values are allowed. - -Examples: `2022`, `-1000`, `0005`. +- Expected format: `YYYY`. Negative `year` values are allowed. +- Examples: `2022`, `-1000`, `0005`. **Restrictions:** - `year` values with a timezone are not supported. -### Yearmonth +## Yearmonth A specific month in a specific year. -Expected format: `YYYY-MM`. - -Example: `2022-12`. +- Expected format: `YYYY-MM`. +- Example: `2022-12`. The underlying representation of `yearmonth` is `date`. @@ -157,13 +154,12 @@ The underlying representation of `yearmonth` is `date`. - Years with more than 4 digits are not supported. - `yearmonth` values with a timezone are not supported. -### Duration +## Duration A duration of time. -Expected pattern: `PnYnMnDTnHnMnS`. See the [definition of the Frictionless type](https://datapackage.org/standard/table-schema/#duration) for more information. - -Example: `P1Y2M3DT10H30M45.343S`. +- Expected pattern: `PnYnMnDTnHnMnS`. See the [definition of the Frictionless type](https://datapackage.org/standard/table-schema/#duration) for more information. +- Example: `P1Y2M3DT10H30M45.343S`. The number of seconds may include decimal digits to arbitrary precision. @@ -172,13 +168,12 @@ The number of seconds may include decimal digits to arbitrary precision. - The underlying representation of `duration` is `string`. Sprout does not attempt to parse `duration` values into a data type that is aware of the various time units contained within a `duration` value. - As a consequence, constraints relying on the numeric comparison of `duration` values are not supported. These constraints are: `minimum`, `maximum`, `exclusiveMinimum`, `exclusiveMaximum`. -### Geopoint +## Geopoint A geographic point. -Expected format: `LAT, LONG`. The space is optional. - -Examples: `45.50, 90.50`, `45.50,90.50`, `-45.50, -90.50`. +- Expected format: `LAT, LONG`. The space is optional. +- Examples: `45.50, 90.50`, `45.50,90.50`, `-45.50, -90.50`. The underlying representation is an array of two `number`s. @@ -186,19 +181,19 @@ The underlying representation is an array of two `number`s. - Other `geopoint` formats are not yet supported. -### Array +## Array A JSON array. Must be well-formed [JSON](http://json.org/). The underlying representation is `string`. -### Object +## Object A JSON object. Must be well-formed [JSON](http://json.org/). The underlying representation is `string`. -### Geojson +## Geojson A JSON object compliant with the [GeoJSON](http://geojson.org/) or [TopoJSON](https://github.com/topojson/topojson-specification/blob/master/README.md) specification. @@ -208,12 +203,12 @@ The underlying representation is `string`. - `geojson` values are treated as plain `object`s. They are not checked against the GeoJSON or TopoJSON specification. -### Any +## Any Unspecified or mixed values. The underlying representation is `string`. -### List +## List Not supported. From 3ba7826eff7e4520cefc06cfb97e0da0272ef4c3 Mon Sep 17 00:00:00 2001 From: martonvago <57952344+martonvago@users.noreply.github.com> Date: Mon, 3 Mar 2025 13:02:41 +0000 Subject: [PATCH 3/5] docs: :memo: apply suggestions from code review Co-authored-by: Luke W. Johnston --- docs/design/interface/data-types.qmd | 38 +++++++++------------------- 1 file changed, 12 insertions(+), 26 deletions(-) diff --git a/docs/design/interface/data-types.qmd b/docs/design/interface/data-types.qmd index 95e245df..b13b07bb 100644 --- a/docs/design/interface/data-types.qmd +++ b/docs/design/interface/data-types.qmd @@ -26,19 +26,15 @@ Below, we list Frictionless data types as used in Sprout and give a precise defi | `array` | `String` | `string` | `BYTE_ARRAY` (String) | | `any` | `String` | `string` | `BYTE_ARRAY` (String) | -: Mappings of Frictionless data types in Sprout +: Mappings of Frictionless data types in Sprout. ## String -A sequence of UTF-8 encoded characters. - -Supported formats: `default`, `email`, `uri`, `binary`, `uuid`. +A sequence of UTF-8 encoded characters. Supported formats: `default`, `email`, `uri`, `binary`, `uuid`. ## Number -A number with or without a decimal part. Precision is not reflected in the number of digits specified in the decimal part (`2.0` is not distinct from `2.00` ). - -Precision of up to 16 significant digits. +A number with or without a decimal part. Precision is not reflected in the number of digits specified in the decimal part (`2.0` is not distinct from `2.00` ). Precision goes up to 16 significant digits. | Name | Allowed values | Examples | |----------------------------|-----------------------------------------|----------------------------| @@ -48,7 +44,7 @@ Precision of up to 16 significant digits. | exponent | `E` | `5E10`, `12.56E-3` | | special values | `NaN`, `inf`, `-inf` (case-insensitive) | | -: `number` format options +: `number` format options. The configuration options `decimalChar`, `groupChar` and `bareNumber` are not yet supported. @@ -61,15 +57,13 @@ A whole number with no decimal part. | leading sign | no sign or `-` | `5`, `-12` | | leading zeros | `0` (optional) | `005` | -: `integer` format options +: `integer` format options. The configuration options `groupChar` and `bareNumber` are not yet supported. ## Boolean -One of two possible values: true or false. - -Sprout supports the default notation for truth values in Frictionless: +One of two possible values: true or false. Sprout supports the default notation for truth values in Frictionless: - All values in `["true", "True", "TRUE", "1"]` are interpreted as true. - All values in `["false", "False", "FALSE", "0"]` are interpreted as false. @@ -88,7 +82,7 @@ A date with a time and optional timezone. | with milliseconds and timezone | `YYYY-MM-DDTHH:MM:SS.sssHH:MM` | `2002-10-12T12:04:15.3+05:00`, `0202-10-10T02:30:00.345-01:00` | | shorthand for UTC | `YYYY-MM-DDTHH:MM:SS(.sss)Z` | `2002-10-12T12:04:15Z`, `0202-10-10T02:30:00.345Z` | -: `datetime` format options +: `datetime` format options. **Restrictions:** @@ -121,7 +115,7 @@ A time without a date. | without microseconds | `HH:MM:SS` | `12:04:15` | | with microseconds | `HH:MM:SS.ssssss` | `12:04:15.3`, `02:30:00.345345` | -: `time` format options +: `time` format options. **Restrictions:** @@ -183,21 +177,15 @@ The underlying representation is an array of two `number`s. ## Array -A JSON array. Must be well-formed [JSON](http://json.org/). - -The underlying representation is `string`. +A JSON array. Must be well-formed [JSON](http://json.org/). The underlying representation is `string`. ## Object -A JSON object. Must be well-formed [JSON](http://json.org/). - -The underlying representation is `string`. +A JSON object. Must be well-formed [JSON](http://json.org/). The underlying representation is `string`. ## Geojson -A JSON object compliant with the [GeoJSON](http://geojson.org/) or [TopoJSON](https://github.com/topojson/topojson-specification/blob/master/README.md) specification. - -The underlying representation is `string`. +A JSON object compliant with the [GeoJSON](http://geojson.org/) or [TopoJSON](https://github.com/topojson/topojson-specification/blob/master/README.md) specification. The underlying representation is `string`. **Restrictions:** @@ -205,9 +193,7 @@ The underlying representation is `string`. ## Any -Unspecified or mixed values. - -The underlying representation is `string`. +Unspecified or mixed values. The underlying representation is `string`. ## List From 26e6dc69814da4c42690c1920cec8c2edce0e7d3 Mon Sep 17 00:00:00 2001 From: martonvago Date: Mon, 3 Mar 2025 13:30:16 +0000 Subject: [PATCH 4/5] docs: :memo: add tip for representing durations --- docs/design/interface/data-types.qmd | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/docs/design/interface/data-types.qmd b/docs/design/interface/data-types.qmd index b13b07bb..d0bcd00e 100644 --- a/docs/design/interface/data-types.qmd +++ b/docs/design/interface/data-types.qmd @@ -26,7 +26,7 @@ Below, we list Frictionless data types as used in Sprout and give a precise defi | `array` | `String` | `string` | `BYTE_ARRAY` (String) | | `any` | `String` | `string` | `BYTE_ARRAY` (String) | -: Mappings of Frictionless data types in Sprout. +: Mappings of Frictionless data types in Sprout. In the Parquet column, first the [primitive type](https://parquet.apache.org/docs/file-format/types/) is given, then the [logical type](https://github.com/apache/parquet-format/blob/master/LogicalTypes.md) in brackets, if relevant. ## String @@ -162,6 +162,13 @@ The number of seconds may include decimal digits to arbitrary precision. - The underlying representation of `duration` is `string`. Sprout does not attempt to parse `duration` values into a data type that is aware of the various time units contained within a `duration` value. - As a consequence, constraints relying on the numeric comparison of `duration` values are not supported. These constraints are: `minimum`, `maximum`, `exclusiveMinimum`, `exclusiveMaximum`. +::: callout-tip +If you are working with duration or interval values, you could consider converting them to a form that Sprout can parse and compare numerically. Here are some suggestions: + +- If your intervals have start and end points, you could represent them using two `date`, `time` or `datetime` columns. For example, a column called `_start` for the beginning of the interval and a column called `_end` for the end of the interval. +- If it doesn't make sense to represent your duration values as intervals between start and end points, you could represent them as plain `integer`s or `number`s. For example, by calculating the number of days, hours, seconds, milliseconds, etc. that make up your durations. +::: + ## Geopoint A geographic point. From 5a9bdc234fcad7855d33c4fe4511afb1808416bc Mon Sep 17 00:00:00 2001 From: martonvago <57952344+martonvago@users.noreply.github.com> Date: Fri, 7 Mar 2025 12:28:00 +0000 Subject: [PATCH 5/5] docs: :memo: apply suggestions from code review MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Signe Kirk Brødbæk <40836345+signekb@users.noreply.github.com> --- docs/design/interface/data-types.qmd | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/docs/design/interface/data-types.qmd b/docs/design/interface/data-types.qmd index d0bcd00e..dc2e20b5 100644 --- a/docs/design/interface/data-types.qmd +++ b/docs/design/interface/data-types.qmd @@ -4,7 +4,7 @@ description: "The data types Sprout supports" --- -Sprout implements the Frictionless Data Package standard and aims to support the [data types](https://datapackage.org/standard/table-schema/#field-types) it defines. However, Sprout not only describes data with metadata but also transforms it into a tidy Parquet file, ready for querying (see [Outputs](/docs/design/interface/outputs.qmd#files) and [Why Parquet](https://decisions.seedcase-project.org/why-parquet/) for more details). As a result, Sprout supports only data types that are compatible (or can be made compatible) with Parquet storage. +Sprout implements the Frictionless Data Package standard and aims to support the [data types](https://datapackage.org/standard/table-schema/#field-types) it defines. However, Sprout not only describes data with its properties (i.e., metadata) but also transforms it into a tidy Parquet file, ready for querying (see [Outputs](/docs/design/interface/outputs.qmd#files) and [Why Parquet](https://decisions.seedcase-project.org/why-parquet/) for more details). As a result, Sprout supports only data types that are compatible (or can be made compatible) with Parquet storage. Below, we list Frictionless data types as used in Sprout and give a precise definition for each. Any differences from the Frictionless specification are noted. @@ -30,7 +30,7 @@ Below, we list Frictionless data types as used in Sprout and give a precise defi ## String -A sequence of UTF-8 encoded characters. Supported formats: `default`, `email`, `uri`, `binary`, `uuid`. +A sequence of UTF-8 encoded characters. Sprout supports all Frictionless Data Package string formats: `default`, `email`, `uri`, `binary`, `uuid`. ## Number @@ -59,7 +59,7 @@ A whole number with no decimal part. : `integer` format options. -The configuration options `groupChar` and `bareNumber` are not yet supported. +The Frictionless Data Package configuration options `groupChar` and `bareNumber` are not yet supported by Sprout. ## Boolean @@ -68,7 +68,7 @@ One of two possible values: true or false. Sprout supports the default notation - All values in `["true", "True", "TRUE", "1"]` are interpreted as true. - All values in `["false", "False", "FALSE", "0"]` are interpreted as false. -Setting custom `trueValues` and `falseValues` is not yet supported. +Setting custom `trueValues` and `falseValues` is not yet supported by Sprout. ## Datetime @@ -80,13 +80,13 @@ A date with a time and optional timezone. | with milliseconds | `YYYY-MM-DDTHH:MM:SS.sss` | `2002-10-12T12:04:15.3`, `0202-10-10T02:30:00.345` | | with timezone | `YYYY-MM-DDTHH:MM:SSHH:MM` | `2002-10-12T12:04:15+05:00`, `0202-10-10T02:30:00-01:00` | | with milliseconds and timezone | `YYYY-MM-DDTHH:MM:SS.sssHH:MM` | `2002-10-12T12:04:15.3+05:00`, `0202-10-10T02:30:00.345-01:00` | -| shorthand for UTC | `YYYY-MM-DDTHH:MM:SS(.sss)Z` | `2002-10-12T12:04:15Z`, `0202-10-10T02:30:00.345Z` | +| with shorthand for UTC | `YYYY-MM-DDTHH:MM:SS(.sss)Z` | `2002-10-12T12:04:15Z`, `0202-10-10T02:30:00.345Z` | : `datetime` format options. **Restrictions:** -- Setting a custom `datetime` pattern in the `format` property is not yet supported. The `any` format is not supported. +- Setting a custom `datetime` pattern in the `format` property in `datapackage.json` is not yet supported. The `any` format is not supported. - Negative `datetime` values are not supported. - Years with more than 4 digits are not supported. - Mixing `datetime` values with and without a timezone in one column is not allowed. @@ -166,7 +166,7 @@ The number of seconds may include decimal digits to arbitrary precision. If you are working with duration or interval values, you could consider converting them to a form that Sprout can parse and compare numerically. Here are some suggestions: - If your intervals have start and end points, you could represent them using two `date`, `time` or `datetime` columns. For example, a column called `_start` for the beginning of the interval and a column called `_end` for the end of the interval. -- If it doesn't make sense to represent your duration values as intervals between start and end points, you could represent them as plain `integer`s or `number`s. For example, by calculating the number of days, hours, seconds, milliseconds, etc. that make up your durations. +- If it doesn't make sense to represent your duration values as intervals between start and end points, you could represent them as plain `integer`s or `number`s. For example, by calculating the number of days, hours, seconds, milliseconds, etc. (depending on the level of precision you need) that make up your durations. ::: ## Geopoint