-
Notifications
You must be signed in to change notification settings - Fork 12
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add support for exporting to duckdb (via parquet) #157
Conversation
This patch adds functionality to export directly to a DuckDB database via the --duckdb flag or using a ".duckdb" or ".db" file. Optionally one can change the table name in which data will be imported. Documentation was mostly copied from existing functions but doctests were updated and checked for consistency with the results. Closes kraina-ai#94
As some of the runners fail and I am not too familiar with your CD requirements, @RaczeQ, maybe you can give me some hints on what to address first? This is not to pressure, rather I thought I'd ask how I can be most helpful with regards to your workflows. Just let me know |
Hi @mwip, sorry for the late reply. From what I can see, the tests aren't working and there are some If you want to run tests locally, there are two commands you can use:
I have to admit that I didn't prepare any instructions for collaborating, so sorry for that 😅 |
@RaczeQ awesome. Thanks for the hints. This will help me figuring it out. I'll work on this this weekend. I just thought this might be a good indication to create a "Contributing" section. Only if you're expecting many community contributions, of course. If you think this warrants the effort, I'd be happy to help with that. As I don't want to force you as the maintainer, feel free to open a dedicated issue and tag me. |
This patch fixes remaining doctest errors that occured during kraina-ai#157. Meanwhile, a remaining bug was discovered around the typing of filter_osm_ids. It was solved, too.
I think this should've fixed all issues introduced by me. |
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #157 +/- ##
==========================================
- Coverage 92.77% 92.42% -0.35%
==========================================
Files 23 23
Lines 1978 2034 +56
==========================================
+ Hits 1835 1880 +45
- Misses 143 154 +11
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
Hi @mwip Regarding contribution manual - you're totally right, I should implement it, I already have it in another repository I'm maintaining (https://github.com/kraina-ai/srai/blob/main/CONTRIBUTING.md) but I just didn't have time to prepare it. Maybe I'll do it in nearest future because the library is gaining some momentum 😛 Anyway, thank you very much for this contribution, I'll wait for the tests to pass and then release it shortly. |
This patch adds functionality to export directly to a DuckDB database via the --duckdb flag or using a ".duckdb" or ".db" file. Optionally one can change the table name in which data will be imported.
Documentation was mostly copied from existing functions but doctests were updated and checked for consistency with the results.
Closes #94