Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add a keyword for custom formats #53

Merged
merged 2 commits into from
May 16, 2024
Merged

add a keyword for custom formats #53

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
60ba9d3
fixed
willow-ahrens May 13, 2024
a0d4313
custom format spec
willow-ahrens May 13, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,10 @@ Here is a table listing the current tensor frameworks that support the format:

| Language | Framework | Status | Notes |
| -------- | ------ | ------ | ----- |
| C | [binsparse-reference-c](https://github.com/GraphBLAS/binsparse-reference-c) | under development | converts between binsparse V1.0 and custom in-memory sparse matrices |
| C++ | [binsparse-reference-impl](https://github.com/GraphBLAS/binsparse-reference-impl) | under development | converts between binsparse V1.0 and custom in-memory sparse matrices |
| Julia | [Finch.jl](https://willowahrens.io/Finch.jl/dev/fileio/) | under development | converts between binsparse V1.0 and V2.0 and Finch matrices and tensors |
| Python | [binsparse-python](https://github.com/ivirshup/binsparse-python) | under development | converts between binsparse V1.0 and scipy.sparse matrices |
| C | [binsparse-reference-c](https://github.com/GraphBLAS/binsparse-reference-c) | under development | converts between binsparse and in-memory sparse matrices |
| C++ | [binsparse-reference-impl](https://github.com/GraphBLAS/binsparse-reference-impl) | under development | converts between binsparse and in-memory sparse matrices |
| Julia | [Finch.jl](https://willowahrens.io/Finch.jl/dev/fileio/) | under development | converts between binsparse and Finch matrices and tensors, supports custom format spec |
| Python | [binsparse-python](https://github.com/ivirshup/binsparse-python) | under development | converts between binsparse and scipy.sparse matrices |

### Editing

Expand Down
149 changes: 75 additions & 74 deletions spec/latest/index.bs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -274,10 +274,10 @@ Pairs must not be duplicated.

Coordinate format is an alias for [[#coor_format]] format.

### Version 2.0 only: Custom Formats ### {#custom_formats}
### Custom Formats ### {#custom_formats}

The contents of this section will be finalized with the release of Binsparse
V2.0, and are subject to change until then.
The contents of this section are optional for all parsers, but enable
customizable sparse formats for matrices and tensors.

Binsparse describes custom multidimensional formats hierarchically. We can
understand these formats as arrays of arrays, where the parent array and
Expand Down Expand Up @@ -305,7 +305,7 @@ and tensors. For example, a dense vector of sparse vectors is equivalent to the
CSR matrix format, and a sparse vector of sparse vectors is equivalent to the
hypersparse DCSR matrix format.

When defining a custom format, the outermost `subformat` key is defined as the
When defining a custom format, the outermost `level` key is defined as the
root level descriptor (a level which will only hold one array). If a level holds
many different arrays, we refer to the `p`th array as the array in position `p`.

Expand All @@ -316,24 +316,25 @@ to represent is `A` and the tensor described by the format descriptor is `B`,
then `A[i_1, ..., i_n] = B[i_(transpose[1]), ..., i_(transpose[n])]`. `transpose` must
be a permutation.

If the format key is a dictionary, the `level` key must be present and shall
describe the storage format of the level used to represent the sparse array.
If the `custom` key is present, it holds a dictionary for custom formats. The
root level is stored under the `level` key. Each level mush have a `level_desc`
attribute which describe the storage format of the level.

The level descriptors are dictionaries defined as follows:

#### Element #### {#element_level}

If the level key is "element", the level represents zero or more scalars.
If the level descriptor is "element", the level represents zero or more scalars.

: values
:: Array of size `number_of_positions` whose `p`th element holds the value of the scalar at position `p`.

#### Dense #### {#dense_level}

If the level key is "dense", the `subformat` key must be present. The `rank`
If the level descriptor is "dense", the `level` key must be present. The `rank`
key must be present, and set to an integer `r` greater than or equal to 1. The
dense level represents zero or more r-dimensional dense arrays whose elements
are themselves arrays specified by `subformat`. For example, a dense level
are themselves arrays specified by `level`. For example, a dense level
of
rank 2 represents a collection of dense matrices of subarrays.

Expand All @@ -350,10 +351,10 @@ of the sublevel.

#### Sparse #### {#sparse_level}

If the level key is "sparse", the `subformat` key must be present. The
If the level descriptor is "sparse", the `level` key must be present. The
`rank` key must be present, and set to an integer `r` greater than or equal to
`1`. The sparse level represents zero or more `r`-dimensional sparse arrays
whose non-implicit elements are themselves arrays specified by `subformat`. For
whose non-implicit elements are themselves arrays specified by `level`. For
example, a sparse level of rank 1 represents a collection of sparse vectors of
subarrays.

Expand All @@ -374,17 +375,17 @@ length of any of the `indices` arrays in this level.

### Equivalent Formats ### {#equivalent_formats}

The following formats are equivalent
The following formats are equivalent. Parsers which support custom formats should also write `format` aliases when appropriate.

#### DVEC #### {#dvec_format_equiv}

```json
"format": {
"subformat": {
"level": "dense",
"custom": {
"level": {
"level_desc": "dense",
"rank": 1,
"subformat": {
"level": "element",
"level": {
"level_desc": "element",
}
}
}
Expand All @@ -393,15 +394,15 @@ The following formats are equivalent
#### DMATR #### {#dmatr_format_equiv}

```json
"format": {
"subformat": {
"level": "dense",
"custom": {
"level": {
"level_desc": "dense",
"rank": 1,
"subformat": {
"level": "dense",
"level": {
"level_desc": "dense",
"rank": 1,
"subformat": {
"level": "element",
"level": {
"level_desc": "element",
}
}
}
Expand All @@ -411,16 +412,16 @@ The following formats are equivalent
#### DMATC #### {#dmatr_format_equiv}

```json
"format": {
"custom": {
"transpose": [1, 0],
"subformat": {
"level": "dense",
"level": {
"level_desc": "dense",
"rank": 1,
"subformat": {
"level": "dense",
"level": {
"level_desc": "dense",
"rank": 1,
"subformat": {
"level": "element",
"level": {
"level_desc": "element",
}
}
}
Expand All @@ -430,12 +431,12 @@ The following formats are equivalent
#### CVEC #### {#cvec_format_equiv}

```json
"format": {
"subformat": {
"level": "sparse",
"custom": {
"level": {
"level_desc": "sparse",
"rank": 1,
"subformat": {
"level": "element",
"level": {
"level_desc": "element",
}
}
}
Expand All @@ -444,15 +445,15 @@ The following formats are equivalent
#### CSR #### {#csr_format_equiv}

```json
"format": {
"subformat": {
"level": "dense",
"custom": {
"level": {
"level_desc": "dense",
"rank": 1,
"subformat": {
"level": "sparse",
"level": {
"level_desc": "sparse",
"rank": 1,
"subformat": {
"level": "element",
"level": {
"level_desc": "element",
}
}
}
Expand All @@ -462,16 +463,16 @@ The following formats are equivalent
#### CSC #### {#csc_format_equiv}

```json
"format": {
"custom": {
"transpose": [1, 0],
"subformat": {
"level": "dense",
"level": {
"level_desc": "dense",
"rank": 1,
"subformat": {
"level": "sparse",
"level": {
"level_desc": "sparse",
"rank": 1,
"subformat": {
"level": "element",
"level": {
"level_desc": "element",
}
}
}
Expand All @@ -481,15 +482,15 @@ The following formats are equivalent
#### DCSR #### {#dcsr_format_equiv}

```json
"format": {
"subformat": {
"level": "sparse",
"custom": {
"level": {
"level_desc": "sparse",
"rank": 1,
"subformat": {
"level": "sparse",
"level": {
"level_desc": "sparse",
"rank": 1,
"subformat": {
"level": "element",
"level": {
"level_desc": "element",
}
}
}
Expand All @@ -499,16 +500,16 @@ The following formats are equivalent
#### DCSC #### {#dcsc_format_equiv}

```json
"format": {
"custom": {
"transpose": [1, 0],
"subformat": {
"level": "sparse",
"level": {
"level_desc": "sparse",
"rank": 1,
"subformat": {
"level": "sparse",
"level": {
"level_desc": "sparse",
"rank": 1,
"subformat": {
"level": "element",
"level": {
"level_desc": "element",
}
}
}
Expand All @@ -518,12 +519,12 @@ The following formats are equivalent
#### COOR #### {#coor_format_equiv}

```json
"format": {
"subformat": {
"level": "sparse",
"custom": {
"level": {
"level_desc": "sparse",
"rank": 2,
"subformat": {
"level": "element",
"level": {
"level_desc": "element",
}
}
}
Expand All @@ -534,13 +535,13 @@ The following formats are equivalent
Column-wise Coordinate format

```json
"format": {
"custom": {
"transpose": [1, 0],
"subformat": {
"level": "sparse",
"level": {
"level_desc": "sparse",
"rank": 2,
"subformat": {
"level": "element",
"level": {
"level_desc": "element",
}
}
}
Expand Down
Loading