adding build using binary downloads #8
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
phughk
reviewed
Jan 4, 2024
modules/utils/build.rs
Outdated
|
|
||
| let (prebuilt_url, prebuilt_hash) = match target.as_str() { | ||
| "aarch64-apple-darwin" => ( | ||
| "https://parcel.pyke.io/v2/delivery/ortrs/packages/msort-binary/1.16.3/ortrs-msort_static-v1.16.3-aarch64-apple-darwin.tgz", |
There was a problem hiding this comment.
The versions and shared prefixes can be extracted. This guarantees that we don't have version typos then.
There was a problem hiding this comment.
I've now moved over to just using the download binaries feature in the ort for the dev-dependencies and then using the binary for the include bytes, all builds are now passing
| output/ | ||
| downloaded_onnx_package/ |
There was a problem hiding this comment.
Tbh we could probably move the package to output/downloaded_onnx instead. Because it is a byproduct of building, even if its required for the build.
There was a problem hiding this comment.
I've now removed the reliance on downloads and it's all now in thetarget directory
maxwellflitton
added a commit
that referenced
this pull request
Jan 5, 2024
Due to the new release of rust, how some of the files were stored in different directories which broke the embedding of the ONNX. This pull request fixes this by getting `ort` to download and compile ONNX for the specific platform in `dev-dependencies` and this compiled ONNX library is then embedded.
maxwellflitton
added a commit
that referenced
this pull request
Feb 24, 2024
* adding build using binary downloads (#8) * adding build using binary downloads * sorting out the build.rs * updating build.rs for surrealml package * prepping version for release * now has target tracking (#10) * adding check in build.rs for docs.rs * removing build.rs for main surrealml to ensure that libraries using the core do not need to do anything in their build.rs * Kings college london integration (#23) * adding machine learning pipelines for bioengineering projects at Kings College London * Remove integrated_training_runner/run_env/ from tracking * adding machine learning pipelines for bioengineering projects at Kings College London
maxwellflitton
added a commit
that referenced
this pull request
Feb 24, 2024
* merging bioengineering code for ML pipelines (#24) * adding build using binary downloads (#8) * adding build using binary downloads * sorting out the build.rs * updating build.rs for surrealml package * prepping version for release * now has target tracking (#10) * adding check in build.rs for docs.rs * removing build.rs for main surrealml to ensure that libraries using the core do not need to do anything in their build.rs * Kings college london integration (#23) * adding machine learning pipelines for bioengineering projects at Kings College London * Remove integrated_training_runner/run_env/ from tracking * adding machine learning pipelines for bioengineering projects at Kings College London * adding python bindings to data access for ML training
maxwellflitton
added a commit
that referenced
this pull request
Feb 24, 2024
* adding build using binary downloads (#8) * adding build using binary downloads * sorting out the build.rs * updating build.rs for surrealml package * prepping version for release * now has target tracking (#10) * adding check in build.rs for docs.rs * removing build.rs for main surrealml to ensure that libraries using the core do not need to do anything in their build.rs * Kings college london integration (#23) * adding machine learning pipelines for bioengineering projects at Kings College London * Remove integrated_training_runner/run_env/ from tracking * adding machine learning pipelines for bioengineering projects at Kings College London * adding python bindings to data access for ML training
maxwellflitton
added a commit
that referenced
this pull request
Mar 14, 2024
* adding build using binary downloads (#8) * adding build using binary downloads * sorting out the build.rs * updating build.rs for surrealml package * prepping version for release * now has target tracking (#10) * adding check in build.rs for docs.rs * removing build.rs for main surrealml to ensure that libraries using the core do not need to do anything in their build.rs * adding machine learning pipelines for bioengineering projects at Kings College London * Remove integrated_training_runner/run_env/ from tracking * adding machine learning pipelines for bioengineering projects at Kings College London * Update FFmpeg data access module and README (#29) * adding run_env to the gitignore --------- Co-authored-by: Yang Li <oliverlee2018@163.com>
maxwellflitton
added a commit
that referenced
this pull request
Mar 14, 2024
* adding glue module to enable error handling (#27) * added batching runner (#28) * Kings college london integration (#30) * adding build using binary downloads (#8) * adding build using binary downloads * sorting out the build.rs * updating build.rs for surrealml package * prepping version for release * now has target tracking (#10) * adding check in build.rs for docs.rs * removing build.rs for main surrealml to ensure that libraries using the core do not need to do anything in their build.rs * adding machine learning pipelines for bioengineering projects at Kings College London * Remove integrated_training_runner/run_env/ from tracking * adding machine learning pipelines for bioengineering projects at Kings College London * Update FFmpeg data access module and README (#29) * adding run_env to the gitignore --------- Co-authored-by: Yang Li <oliverlee2018@163.com> * bumping the version --------- Co-authored-by: Sam Hillman <116303632+SHillman836@users.noreply.github.com> Co-authored-by: Yang Li <oliverlee2018@163.com>
maxwellflitton
added a commit
that referenced
this pull request
Mar 14, 2024
* adding glue module to enable error handling (#27) * added batching runner (#28) * Kings college london integration (#30) * adding build using binary downloads (#8) * adding build using binary downloads * sorting out the build.rs * updating build.rs for surrealml package * prepping version for release * now has target tracking (#10) * adding check in build.rs for docs.rs * removing build.rs for main surrealml to ensure that libraries using the core do not need to do anything in their build.rs * adding machine learning pipelines for bioengineering projects at Kings College London * Remove integrated_training_runner/run_env/ from tracking * adding machine learning pipelines for bioengineering projects at Kings College London * Update FFmpeg data access module and README (#29) * adding run_env to the gitignore --------- Co-authored-by: Yang Li <oliverlee2018@163.com> * bumping the version * updating the README and module --------- Co-authored-by: Sam Hillman <116303632+SHillman836@users.noreply.github.com> Co-authored-by: Yang Li <oliverlee2018@163.com>
maxwellflitton
added a commit
that referenced
this pull request
Mar 14, 2024
* adding glue module to enable error handling (#27) * added batching runner (#28) * Kings college london integration (#30) * adding build using binary downloads (#8) * adding build using binary downloads * sorting out the build.rs * updating build.rs for surrealml package * prepping version for release * now has target tracking (#10) * adding check in build.rs for docs.rs * removing build.rs for main surrealml to ensure that libraries using the core do not need to do anything in their build.rs * adding machine learning pipelines for bioengineering projects at Kings College London * Remove integrated_training_runner/run_env/ from tracking * adding machine learning pipelines for bioengineering projects at Kings College London * Update FFmpeg data access module and README (#29) * adding run_env to the gitignore --------- Co-authored-by: Yang Li <oliverlee2018@163.com> * bumping the version * updating the README and module * updating the surrealml-core deployment workflow * updating the surrealml-core deployment workflow --------- Co-authored-by: Sam Hillman <116303632+SHillman836@users.noreply.github.com> Co-authored-by: Yang Li <oliverlee2018@163.com>
maxwellflitton
added a commit
that referenced
this pull request
Mar 14, 2024
* adding glue module to enable error handling (#27) * added batching runner (#28) * Kings college london integration (#30) * adding build using binary downloads (#8) * adding build using binary downloads * sorting out the build.rs * updating build.rs for surrealml package * prepping version for release * now has target tracking (#10) * adding check in build.rs for docs.rs * removing build.rs for main surrealml to ensure that libraries using the core do not need to do anything in their build.rs * adding machine learning pipelines for bioengineering projects at Kings College London * Remove integrated_training_runner/run_env/ from tracking * adding machine learning pipelines for bioengineering projects at Kings College London * Update FFmpeg data access module and README (#29) * adding run_env to the gitignore --------- Co-authored-by: Yang Li <oliverlee2018@163.com> * bumping the version * updating the README and module * updating the surrealml-core deployment workflow * updating the surrealml-core deployment workflow * updating cargo --------- Co-authored-by: Sam Hillman <116303632+SHillman836@users.noreply.github.com> Co-authored-by: Yang Li <oliverlee2018@163.com>
maxwellflitton
added a commit
that referenced
this pull request
Mar 14, 2024
* Develop (#35) * adding glue module to enable error handling (#27) * added batching runner (#28) * Kings college london integration (#30) * adding build using binary downloads (#8) * adding build using binary downloads * sorting out the build.rs * updating build.rs for surrealml package * prepping version for release * now has target tracking (#10) * adding check in build.rs for docs.rs * removing build.rs for main surrealml to ensure that libraries using the core do not need to do anything in their build.rs * adding machine learning pipelines for bioengineering projects at Kings College London * Remove integrated_training_runner/run_env/ from tracking * adding machine learning pipelines for bioengineering projects at Kings College London * Update FFmpeg data access module and README (#29) * adding run_env to the gitignore --------- Co-authored-by: Yang Li <oliverlee2018@163.com> * bumping the version * updating the README and module * updating the surrealml-core deployment workflow * updating the surrealml-core deployment workflow * updating cargo --------- Co-authored-by: Sam Hillman <116303632+SHillman836@users.noreply.github.com> Co-authored-by: Yang Li <oliverlee2018@163.com> * merging error modules into the core * merging error modules into the core * merging error modules into the core --------- Co-authored-by: Sam Hillman <116303632+SHillman836@users.noreply.github.com> Co-authored-by: Yang Li <oliverlee2018@163.com>
maxwellflitton
added a commit
that referenced
this pull request
Mar 14, 2024
* adding glue module to enable error handling (#27) * added batching runner (#28) * Kings college london integration (#30) * adding build using binary downloads (#8) * adding build using binary downloads * sorting out the build.rs * updating build.rs for surrealml package * prepping version for release * now has target tracking (#10) * adding check in build.rs for docs.rs * removing build.rs for main surrealml to ensure that libraries using the core do not need to do anything in their build.rs * adding machine learning pipelines for bioengineering projects at Kings College London * Remove integrated_training_runner/run_env/ from tracking * adding machine learning pipelines for bioengineering projects at Kings College London * Update FFmpeg data access module and README (#29) * adding run_env to the gitignore --------- Co-authored-by: Yang Li <oliverlee2018@163.com> * bumping the version * updating the README and module * updating the surrealml-core deployment workflow * updating the surrealml-core deployment workflow * updating cargo * Error modules (#36) * Develop (#35) * adding glue module to enable error handling (#27) * added batching runner (#28) * Kings college london integration (#30) * adding build using binary downloads (#8) * adding build using binary downloads * sorting out the build.rs * updating build.rs for surrealml package * prepping version for release * now has target tracking (#10) * adding check in build.rs for docs.rs * removing build.rs for main surrealml to ensure that libraries using the core do not need to do anything in their build.rs * adding machine learning pipelines for bioengineering projects at Kings College London * Remove integrated_training_runner/run_env/ from tracking * adding machine learning pipelines for bioengineering projects at Kings College London * Update FFmpeg data access module and README (#29) * adding run_env to the gitignore --------- Co-authored-by: Yang Li <oliverlee2018@163.com> * bumping the version * updating the README and module * updating the surrealml-core deployment workflow * updating the surrealml-core deployment workflow * updating cargo --------- Co-authored-by: Sam Hillman <116303632+SHillman836@users.noreply.github.com> Co-authored-by: Yang Li <oliverlee2018@163.com> * merging error modules into the core * merging error modules into the core * merging error modules into the core --------- Co-authored-by: Sam Hillman <116303632+SHillman836@users.noreply.github.com> Co-authored-by: Yang Li <oliverlee2018@163.com> --------- Co-authored-by: Sam Hillman <116303632+SHillman836@users.noreply.github.com> Co-authored-by: Yang Li <oliverlee2018@163.com>
maxwellflitton
added a commit
that referenced
this pull request
Mar 25, 2024
* adding glue module to enable error handling (#27) * added batching runner (#28) * Kings college london integration (#30) * adding build using binary downloads (#8) * adding build using binary downloads * sorting out the build.rs * updating build.rs for surrealml package * prepping version for release * now has target tracking (#10) * adding check in build.rs for docs.rs * removing build.rs for main surrealml to ensure that libraries using the core do not need to do anything in their build.rs * adding machine learning pipelines for bioengineering projects at Kings College London * Remove integrated_training_runner/run_env/ from tracking * adding machine learning pipelines for bioengineering projects at Kings College London * Update FFmpeg data access module and README (#29) * adding run_env to the gitignore --------- Co-authored-by: Yang Li <oliverlee2018@163.com> * bumping the version * updating the README and module * updating the surrealml-core deployment workflow * updating the surrealml-core deployment workflow * updating cargo * Error modules (#36) * Develop (#35) * adding glue module to enable error handling (#27) * added batching runner (#28) * Kings college london integration (#30) * adding build using binary downloads (#8) * adding build using binary downloads * sorting out the build.rs * updating build.rs for surrealml package * prepping version for release * now has target tracking (#10) * adding check in build.rs for docs.rs * removing build.rs for main surrealml to ensure that libraries using the core do not need to do anything in their build.rs * adding machine learning pipelines for bioengineering projects at Kings College London * Remove integrated_training_runner/run_env/ from tracking * adding machine learning pipelines for bioengineering projects at Kings College London * Update FFmpeg data access module and README (#29) * adding run_env to the gitignore --------- Co-authored-by: Yang Li <oliverlee2018@163.com> * bumping the version * updating the README and module * updating the surrealml-core deployment workflow * updating the surrealml-core deployment workflow * updating cargo --------- Co-authored-by: Sam Hillman <116303632+SHillman836@users.noreply.github.com> Co-authored-by: Yang Li <oliverlee2018@163.com> * merging error modules into the core * merging error modules into the core * merging error modules into the core --------- Co-authored-by: Sam Hillman <116303632+SHillman836@users.noreply.github.com> Co-authored-by: Yang Li <oliverlee2018@163.com> * Index overflow (#40) * adding buffer out of index check * adding buffer out of index check --------- Co-authored-by: Sam Hillman <116303632+SHillman836@users.noreply.github.com> Co-authored-by: Yang Li <oliverlee2018@163.com>
maxwellflitton
added a commit
that referenced
this pull request
Mar 25, 2024
* adding glue module to enable error handling (#27) * added batching runner (#28) * Kings college london integration (#30) * adding build using binary downloads (#8) * adding build using binary downloads * sorting out the build.rs * updating build.rs for surrealml package * prepping version for release * now has target tracking (#10) * adding check in build.rs for docs.rs * removing build.rs for main surrealml to ensure that libraries using the core do not need to do anything in their build.rs * adding machine learning pipelines for bioengineering projects at Kings College London * Remove integrated_training_runner/run_env/ from tracking * adding machine learning pipelines for bioengineering projects at Kings College London * Update FFmpeg data access module and README (#29) * adding run_env to the gitignore --------- Co-authored-by: Yang Li <oliverlee2018@163.com> * bumping the version * updating the README and module * updating the surrealml-core deployment workflow * updating the surrealml-core deployment workflow * updating cargo * Error modules (#36) * Develop (#35) * adding glue module to enable error handling (#27) * added batching runner (#28) * Kings college london integration (#30) * adding build using binary downloads (#8) * adding build using binary downloads * sorting out the build.rs * updating build.rs for surrealml package * prepping version for release * now has target tracking (#10) * adding check in build.rs for docs.rs * removing build.rs for main surrealml to ensure that libraries using the core do not need to do anything in their build.rs * adding machine learning pipelines for bioengineering projects at Kings College London * Remove integrated_training_runner/run_env/ from tracking * adding machine learning pipelines for bioengineering projects at Kings College London * Update FFmpeg data access module and README (#29) * adding run_env to the gitignore --------- Co-authored-by: Yang Li <oliverlee2018@163.com> * bumping the version * updating the README and module * updating the surrealml-core deployment workflow * updating the surrealml-core deployment workflow * updating cargo --------- Co-authored-by: Sam Hillman <116303632+SHillman836@users.noreply.github.com> Co-authored-by: Yang Li <oliverlee2018@163.com> * merging error modules into the core * merging error modules into the core * merging error modules into the core --------- Co-authored-by: Sam Hillman <116303632+SHillman836@users.noreply.github.com> Co-authored-by: Yang Li <oliverlee2018@163.com> * Index overflow (#40) * adding buffer out of index check * adding buffer out of index check * updating testing around meta data (#42) * updating the naming and increasing tests around the meta data of the stored ML models * updating the naming and increasing tests around the meta data of the stored ML models * updating the naming and increasing tests around the meta data of the stored ML models --------- Co-authored-by: Sam Hillman <116303632+SHillman836@users.noreply.github.com> Co-authored-by: Yang Li <oliverlee2018@163.com>
maxwellflitton
added a commit
that referenced
this pull request
Apr 2, 2024
* adding glue module to enable error handling (#27) * added batching runner (#28) * Kings college london integration (#30) * adding build using binary downloads (#8) * adding build using binary downloads * sorting out the build.rs * updating build.rs for surrealml package * prepping version for release * now has target tracking (#10) * adding check in build.rs for docs.rs * removing build.rs for main surrealml to ensure that libraries using the core do not need to do anything in their build.rs * adding machine learning pipelines for bioengineering projects at Kings College London * Remove integrated_training_runner/run_env/ from tracking * adding machine learning pipelines for bioengineering projects at Kings College London * Update FFmpeg data access module and README (#29) * adding run_env to the gitignore --------- Co-authored-by: Yang Li <oliverlee2018@163.com> * bumping the version * updating the README and module * updating the surrealml-core deployment workflow * updating the surrealml-core deployment workflow * updating cargo * Error modules (#36) * Develop (#35) * adding glue module to enable error handling (#27) * added batching runner (#28) * Kings college london integration (#30) * adding build using binary downloads (#8) * adding build using binary downloads * sorting out the build.rs * updating build.rs for surrealml package * prepping version for release * now has target tracking (#10) * adding check in build.rs for docs.rs * removing build.rs for main surrealml to ensure that libraries using the core do not need to do anything in their build.rs * adding machine learning pipelines for bioengineering projects at Kings College London * Remove integrated_training_runner/run_env/ from tracking * adding machine learning pipelines for bioengineering projects at Kings College London * Update FFmpeg data access module and README (#29) * adding run_env to the gitignore --------- Co-authored-by: Yang Li <oliverlee2018@163.com> * bumping the version * updating the README and module * updating the surrealml-core deployment workflow * updating the surrealml-core deployment workflow * updating cargo --------- Co-authored-by: Sam Hillman <116303632+SHillman836@users.noreply.github.com> Co-authored-by: Yang Li <oliverlee2018@163.com> * merging error modules into the core * merging error modules into the core * merging error modules into the core --------- Co-authored-by: Sam Hillman <116303632+SHillman836@users.noreply.github.com> Co-authored-by: Yang Li <oliverlee2018@163.com> * Index overflow (#40) * adding buffer out of index check * adding buffer out of index check * updating testing around meta data (#42) * updating the naming and increasing tests around the meta data of the stored ML models * updating the naming and increasing tests around the meta data of the stored ML models * updating the naming and increasing tests around the meta data of the stored ML models * Tensorflow support (#44) * stashing for branch switch * adding tests for tensorflow * adding tests for tensorflow * fixing requirement conflicts * fixing requirement conflicts * fixing requirement conflicts * fixing requirement conflicts * fixing requirement conflicts --------- Co-authored-by: Sam Hillman <116303632+SHillman836@users.noreply.github.com> Co-authored-by: Yang Li <oliverlee2018@163.com>
maxwellflitton
added a commit
that referenced
this pull request
Apr 16, 2024
* adding glue module to enable error handling (#27) * added batching runner (#28) * Kings college london integration (#30) * adding build using binary downloads (#8) * adding build using binary downloads * sorting out the build.rs * updating build.rs for surrealml package * prepping version for release * now has target tracking (#10) * adding check in build.rs for docs.rs * removing build.rs for main surrealml to ensure that libraries using the core do not need to do anything in their build.rs * adding machine learning pipelines for bioengineering projects at Kings College London * Remove integrated_training_runner/run_env/ from tracking * adding machine learning pipelines for bioengineering projects at Kings College London * Update FFmpeg data access module and README (#29) * adding run_env to the gitignore --------- Co-authored-by: Yang Li <oliverlee2018@163.com> * bumping the version * updating the README and module * updating the surrealml-core deployment workflow * updating the surrealml-core deployment workflow * updating cargo * Error modules (#36) * Develop (#35) * adding glue module to enable error handling (#27) * added batching runner (#28) * Kings college london integration (#30) * adding build using binary downloads (#8) * adding build using binary downloads * sorting out the build.rs * updating build.rs for surrealml package * prepping version for release * now has target tracking (#10) * adding check in build.rs for docs.rs * removing build.rs for main surrealml to ensure that libraries using the core do not need to do anything in their build.rs * adding machine learning pipelines for bioengineering projects at Kings College London * Remove integrated_training_runner/run_env/ from tracking * adding machine learning pipelines for bioengineering projects at Kings College London * Update FFmpeg data access module and README (#29) * adding run_env to the gitignore --------- Co-authored-by: Yang Li <oliverlee2018@163.com> * bumping the version * updating the README and module * updating the surrealml-core deployment workflow * updating the surrealml-core deployment workflow * updating cargo --------- Co-authored-by: Sam Hillman <116303632+SHillman836@users.noreply.github.com> Co-authored-by: Yang Li <oliverlee2018@163.com> * merging error modules into the core * merging error modules into the core * merging error modules into the core --------- Co-authored-by: Sam Hillman <116303632+SHillman836@users.noreply.github.com> Co-authored-by: Yang Li <oliverlee2018@163.com> * Index overflow (#40) * adding buffer out of index check * adding buffer out of index check * updating testing around meta data (#42) * updating the naming and increasing tests around the meta data of the stored ML models * updating the naming and increasing tests around the meta data of the stored ML models * updating the naming and increasing tests around the meta data of the stored ML models * Tensorflow support (#44) * stashing for branch switch * adding tests for tensorflow * adding tests for tensorflow * fixing requirement conflicts * fixing requirement conflicts * fixing requirement conflicts * fixing requirement conflicts * fixing requirement conflicts * adding support for raw ONNX models * adding support for raw ONNX models (#46) adding support for raw ONNX models * updating * fixing conflicts --------- Co-authored-by: Sam Hillman <116303632+SHillman836@users.noreply.github.com> Co-authored-by: Yang Li <oliverlee2018@163.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Instead of doing a
cargo buildon theortlibrary, thebuild.rsfile now directly downloads the onnx runtime and links it. The new update to rust automatically linked workspaces breaking the previous approach