Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

docs: add standalone partition pages in user guide #3192

Open
wants to merge 25 commits into
base: main
Choose a base branch
from
Open

docs: add standalone partition pages in user guide #3192

wants to merge 25 commits into from

Conversation

braaannigan
Copy link
Contributor

Description

This page is a standalone user guide page that focuses on working with partitioned datasets.

Related Issue(s)

closes #3191

@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 4, 2025

ACTION NEEDED

delta-rs follows the Conventional Commits specification for release automation.

The PR title and description are used as the merge commit message. Please update your PR title and description to match the specification.

@braaannigan braaannigan changed the title docs: Add standalone partition pages in user guide docs: add standalone partition pages in user guide Feb 4, 2025
ion-elgreco
ion-elgreco requested changes Feb 5, 2025
View reviewed changes
@github-actions github-actions bot added binding/python Issues for the Python package and removed binding/python Issues for the Python package labels Feb 6, 2025
@braaannigan
Copy link
Contributor Author

@ion-elgreco This should be good to go

@github-actions github-actions bot added binding/python Issues for the Python package binding/rust Issues for the Rust crate and removed binding/python Issues for the Python package binding/rust Issues for the Rust crate labels Feb 11, 2025
ion-elgreco
ion-elgreco previously approved these changes Feb 14, 2025
View reviewed changes
@ion-elgreco
Copy link
Collaborator

@braaannigan can you rebase and squash the commits in to one please?

@braaannigan braaannigan force-pushed the github-pages branch 2 times, most recently from 3736dba to 3f2e4b5 Compare February 17, 2025 16:39
@github-actions github-actions bot added binding/python Issues for the Python package binding/rust Issues for the Rust crate documentation Improvements or additions to documentation labels Feb 18, 2025
Liam Brannigan and others added 4 commits February 19, 2025 14:45
Add standalone page on working with partitions
b864a15 
Signed-off-by: Liam Brannigan <liambrannigan@Liams-MacBook-Pro.local>
@HawaiianSpork
feat: return metrics about the commit
4ec52f2 
Post commit now contains the number of retries it took to do the commit and FinalizeCommit now contains number of retries, whether there was a checkpoint created and number of log files cleaned-up.

Signed-off-by: Michael Maletich <michael.maletich@relativity.com>
Signed-off-by: Liam Brannigan <liambrannigan@Liams-MacBook-Pro.local>
@HawaiianSpork
format and added asserts for new attributes on existing unit tests
7763906 
Signed-off-by: Michael Maletich <michael.maletich@relativity.com>
Signed-off-by: Liam Brannigan <liambrannigan@Liams-MacBook-Pro.local>
@ion-elgreco
fix: adhere write stats configuration
bdf5312 
Signed-off-by: Ion Koutsouris <15728914+ion-elgreco@users.noreply.github.com>
Signed-off-by: Liam Brannigan <liambrannigan@Liams-MacBook-Pro.local>
ion-elgreco and others added 20 commits February 19, 2025 14:45
@ion-elgreco
feat: cdf tableprovider
cb6e4fc 
Signed-off-by: Ion Koutsouris <15728914+ion-elgreco@users.noreply.github.com>
Signed-off-by: Liam Brannigan <liambrannigan@Liams-MacBook-Pro.local>
@ion-elgreco
feat: predicate pushdown in parquet scan
463d56c 
Signed-off-by: Ion Koutsouris <15728914+ion-elgreco@users.noreply.github.com>
Signed-off-by: Liam Brannigan <liambrannigan@Liams-MacBook-Pro.local>
@liamphmurphy
serialize fsck files_removed metric as a string for compatibility wit…
16755e6 
…h Spark

Signed-off-by: Liam Murphy <liam@phmurphy.com>
Signed-off-by: Liam Brannigan <liambrannigan@Liams-MacBook-Pro.local>
@rtyler
chore: switch to cache@v4
dcf41fa 
We got hit by a brownout :D

<https://github.blog/changelog/2024-12-05-notice-of-upcoming-releases-and-breaking-changes-for-github-actions/#actions-cache-v1-v2-and-actions-toolkit-cache-package-closing-down>

Signed-off-by: R. Tyler Croy <rtyler@brokenco.de>
Signed-off-by: Liam Brannigan <liambrannigan@Liams-MacBook-Pro.local>
@ion-elgreco
fix: advance state after conflict check
39d187d 
Signed-off-by: Ion Koutsouris <15728914+ion-elgreco@users.noreply.github.com>
Signed-off-by: Liam Brannigan <liambrannigan@Liams-MacBook-Pro.local>
@akesling
fix(typo): s/implemtnation/implementation/
cdf3d9f 
Signed-off-by: Liam Brannigan <liambrannigan@Liams-MacBook-Pro.local>
@omkar-foss
feat(python): add capability to read unity catalog (uc://) uris
0bb67f4 
This adds capability to read directly from uc:// uris using the
local catalog-unity crate. This also exposes the UC temporary
credentials in storage_options of the `DeltaTable` instance so
polars or similar readers can use it.

Signed-off-by: Omkar P <45419097+omkar-foss@users.noreply.github.com>
Signed-off-by: Liam Brannigan <liambrannigan@Liams-MacBook-Pro.local>
@omkar-foss
test(python): add tests with mock APIs for Unity Catalog variants
9bf1627 
Signed-off-by: Omkar P <45419097+omkar-foss@users.noreply.github.com>
Signed-off-by: Liam Brannigan <liambrannigan@Liams-MacBook-Pro.local>
@omkar-foss
fix: update mockoon datafiles path for github actions
b90cac1 
Signed-off-by: Omkar P <45419097+omkar-foss@users.noreply.github.com>
Signed-off-by: Liam Brannigan <liambrannigan@Liams-MacBook-Pro.local>
@omkar-foss
fix: replace panics with proper errors
6fbcc3f 
Signed-off-by: Omkar P <45419097+omkar-foss@users.noreply.github.com>
Signed-off-by: Liam Brannigan <liambrannigan@Liams-MacBook-Pro.local>
@omkar-foss
chore: reorganize imports
ad6dadb 
Signed-off-by: Omkar P <45419097+omkar-foss@users.noreply.github.com>
Signed-off-by: Liam Brannigan <liambrannigan@Liams-MacBook-Pro.local>
@omkar-foss
chore: reformat file, handle get_credentials() error
2389cfa 
Signed-off-by: Omkar P <45419097+omkar-foss@users.noreply.github.com>
Signed-off-by: Liam Brannigan <liambrannigan@Liams-MacBook-Pro.local>
@omkar-foss
docs: add doc loading delta table from UC
917102d 
Signed-off-by: Omkar P <45419097+omkar-foss@users.noreply.github.com>
Signed-off-by: Liam Brannigan <liambrannigan@Liams-MacBook-Pro.local>
@omkar-foss
fix: add dedicated function to execute futures
c412540 
Signed-off-by: Omkar P <45419097+omkar-foss@users.noreply.github.com>
Signed-off-by: Liam Brannigan <liambrannigan@Liams-MacBook-Pro.local>
@omkar-foss
chore: re-run cargo fmt
e5c95b7 
Signed-off-by: Omkar P <45419097+omkar-foss@users.noreply.github.com>
Signed-off-by: Liam Brannigan <liambrannigan@Liams-MacBook-Pro.local>
@ion-elgreco
refactor: simplify writer, schema evolution and generated columns
e0077ff 
Signed-off-by: Ion Koutsouris <15728914+ion-elgreco@users.noreply.github.com>
Signed-off-by: Liam Brannigan <liambrannigan@Liams-MacBook-Pro.local>
@ion-elgreco
chore: clippy pyo3 deprecations
ea1d290 
Signed-off-by: Ion Koutsouris <15728914+ion-elgreco@users.noreply.github.com>
Signed-off-by: Liam Brannigan <liambrannigan@Liams-MacBook-Pro.local>
@ion-elgreco
feat: streamed write execution except cdf
09f05fd 
Signed-off-by: Ion Koutsouris <15728914+ion-elgreco@users.noreply.github.com>
Signed-off-by: Liam Brannigan <liambrannigan@Liams-MacBook-Pro.local>
@ion-elgreco
feat: write metrics extension planner
382d9e5 
Signed-off-by: Ion Koutsouris <15728914+ion-elgreco@users.noreply.github.com>
Signed-off-by: Liam Brannigan <liambrannigan@Liams-MacBook-Pro.local>
@ion-elgreco
feat: support streamed execution with cdf
1cdb154 
Signed-off-by: Ion Koutsouris <15728914+ion-elgreco@users.noreply.github.com>
Signed-off-by: Liam Brannigan <liambrannigan@Liams-MacBook-Pro.local>
@github-actions github-actions bot removed documentation Improvements or additions to documentation binding/rust Issues for the Rust crate binding/python Issues for the Python package labels Feb 19, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ion-elgreco ion-elgreco ion-elgreco left review comments

@wjones127 wjones127 Awaiting requested review from wjones127

@fvaleye fvaleye Awaiting requested review from fvaleye

@roeap roeap Awaiting requested review from roeap

@rtyler rtyler Awaiting requested review from rtyler

@hntd187 hntd187 Awaiting requested review from hntd187

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

User guide: Add standalone page for working with partitions
7 participants
@braaannigan @ion-elgreco @HawaiianSpork @liamphmurphy @rtyler @akesling @omkar-foss