prop(os): Evaluate adding soar to Rhino Linux #40
Comments
Azathothas
added
Feature
|
I like the prop |
|
I also like this idea. It would be cool to have this alongside the "GUI App Store." |
|
Hello @Azathothas, Thank you for contacting us and opening a proposal for integrating Soar into Rhino Linux. Myself and the rest of the team appreciate the work and time taken to open this proposal, and acknowledge the features Soar could bring to Rhino Linux. We have decided not to include Soar as an option within Rhino Linux by default, or as an option within our setup wizard for the following reasons: Rhino Linux already includes Pacstall by default which serves the needs for package management on Rhino Linux. Additionally we support additional formats such as Flatpaks, Snaps, AppImages and Nixpkgs. We have concerns regarding the way Soar sources some of its packages. We have identified that Soar scrapes packages from Nix and the AUR. This would raise large concerns for us, if integrating into Rhino Linux, about upstream compliance, ensuring package integrity and long-term maintainability. Finally, I see a large amount of overlap between the use-cases of Soar and Nix in providing universal packaging. Given Nix's existing capabilities, I see no advantage in using Soar over Nix. Many thanks, |
Pacstall has way too few packages and the GitHub issues are all about new pkg request which have gone unresolved for quite a while.
Flatpaks don't allow for CLI apps (Our Nix:
$ nix profile install nixpkgs#bash
$ nix path-info -rSh nixpkgs#bash
/nix/store/78qyrv70jf0pz76awkh9pd5lkcr325z5-xgcc-14-20241116-libgcc 197.1 KiB
/nix/store/bpdm3x8i7qwg8b7wq9h7y78qw1390ni5-libunistring-1.3 2.0 MiB
/nix/store/wm87zrqsi07as7jvr3lyzawhlql577l7-libidn2-2.3.7 2.3 MiB
/nix/store/6q2mknq81cyscjmkv72fpcsvan56qhmg-glibc-2.40-66 31.4 MiB
/nix/store/x8nrg8zjns1zc5nvw0sf576ak50b5gzp-ncurses-6.5 35.0 MiB
/nix/store/l7ah0gkkfrq31iy8aliqrby81js4rh81-readline-8.2p13 35.5 MiB
/nix/store/4k90qpzh1a4sldhnf7cxwkm9c0agq4fp-bash-interactive-5.2p37 42.4 MiB
/nix/store/r2n2ss9i7hjvqs3slixm7361c4h2myq2-bash-interactive-5.2p37-man 97.3 KiB
#We use nixpkgs to build bash statically and then only add what's needed
#The build happens on CI, and the end user only has to download a 1.20 MB statically Linked binary
[-] bash#nixpkgs.bash:bincache (static-5.2p37) - GNU Bourne-Again Shell, the de facto standard shell on Linux (1.20 MiB)
Soar doesn't scrape anything, it fetches packages from our prebuilt cache.
We don't violate any Licensing or TOS (because we use the src distribution itself), and we also redistribute
Everything is recorded as very, very verbose log. Along with checksums: https://docs.pkgforge.dev/repositories/soarpkgs/security
Soar/Soarpkgs are "new" only in the rebrand, previously it was called toolpacks, we have been at it since early
This is false. Nix requires installation as a daemon and will take a path, usually at
Soar is a 7 MB static binary, doesn't need root, doesn't need daemon, doesn't run build scripts on host, doesn't pull in 100 MB of dependencies to install a 1 MB package. With all of that said, |
Pacstall intends to provide packages that are outside of standard APT repositories already existing on the system, as an addition, not a replacement. While we would love for maintainers to submit more packages to the repo and resolve those old requests, we as the developers are students with limited time, and can only ourselves maintain so many packages. If you have recommendations about automation, we are happy to hear your suggestions, as your project clearly has a substantial amount. At the same time, Pacstall is a key integration of Rhino Linux that handles the majority of the core components that distinguish a Rhino install from an Ubuntu install. By and large, Rhino Linux is nothing without Pacstall, it is our most essential utility.
That is commendable, but most of those CLI apps appear to already be in standard Ubuntu repositories. We don't have a need for those. As for AppImages, that is also spectacular, but I think there is a bit of a marketing issue going on. It is not quite clear exactly what kind of package manager Soar is trying to be (more on that below), so if you told us that it was a huge additional AppImage repository, I think our initial reaction would have been different.
This is why in particular it is not clear what Soar is trying to be. First and foremost, Soar's "repackaging" is the textbook definition of scraping. It is fine if that is the case, and indeed makes Soar a powerful project, but in its current state you are falsely advertising what Soar really is and does. The information from the metadata link you provided, and an explanation on how Soar's "repackaging" works, should be plastered front and center on the documentation. You should be honest about what your project provides, as:
Actually tells me very little about what Soar is, how it works, or where its packages come from, or how they are built. That is what we mean by package integrity and long-term maintainability. One of the issues that holds Pacstall back from simply automating updates of pacscripts is the potential for a package's build method to change between versions, and the pacscript needing a manual update to remedy this. You appear to be the only person maintaining the packages for Soar, so how can you possibly ensure all 12000+ of your packages are properly up to date and non-broken upon build, unless you are purely scraping pre-existing builds from a large swath of other repositories? It is hard to be comfortable with Soar when these vulnerabilities, even if just in communication, are apparent.
Soar's minimalism can be a weakness as much as it can a strength. Can it bootstrap a Linux system from scratch? What exactly is the "user install"/sandbox that it does, if not like Nix? |
Soar isn't meant to replace existing core system package managers, as it can't manage libraries or root stuff as already said in the original prop.
We ingest repology data and ubuntu (even the bleeding edge ubuntu) mostly lags behind the versions. (Soar has the latest/bleeding edge version of all packages, but also provides old/stable versions at the same time)
The core people involved (including me) aren't native speakers, we are from Asia (Nepal).
The very first sentence on Soar's Readme: https://github.com/pkgforge/soar/blob/main/README.md?plain=1#L37
This is also on the readme: https://github.com/pkgforge/soar/blob/main/README.md?plain=1#L70
Because most of it is automated.
I believe, by now, I have already pointed out that soar and the related ecosystem is hardened & sanity checked by default.
https://github.com/pkgforge/soarpkgs/blob/main/MANIFESTO.md?plain=1#L48
Yes, there are already packages like
By user install, it is implied packages are installed in ~/.local/share or similar rather than |
I've read through your docs, and they still don't explain the questions I just asked in plain detail, and most of them missing entirely. I have had to learn them through you telling me through this correspondence. More importantly, you do not say anywhere plain and simple that you are just pulling prebuilt packages from other main Linux repos. You should be saying on your readme where you are getting your packages from, and not purely claiming them as your own; Soar is a coagulator of other package managers.
That is a huge security vulnerability. Automation at such a large scale is an incredibly unsafe idea, it's exactly why packages have so many different maintainers on different repos in the first place. You also don't seem to have a test suite for the packages, you just assume that they work if they build. Dependency conflicts or missing libraries on systems are sure to pop up, and cannot be purely tested on CI - hundreds to thousands of people each work to ensure small sets of these packages work across distributions, particularly in repositories of such a massive size.
Again, you are saying Soar is something that it is not, and are immediately backtracking. if "the SBUILD will simply fail", then it does not "forever work," and "broken packages" are very much possible. If you or other maintainers need to fix failing SBUILDs, then that is not guaranteeing that you write the SBUILD only once.
See above for some security vulnerabilities, but I also again want to point out that you have a vulnerability in communicating what your project is. You should be saying more clearly that "Soar pulls prebuilt packages from the repositories of Alpine, Arch, Debian, Nix, Flatpak, Homebrew & more" and "repackages them into stripped down versions", where it then "installs these binaries to users' local directories."
When I say sandbox, I mean I would assume Soar would have some kind of local sandbox to ensure all libraries needed by packages are up to date and installed on the system, as Nix does - i.e. how are you handling dependencies? |
oklopfer
added
Application Development
|
(re-opening because we are still ongoing in discussion) |
|
Hmm, I see. Okay let me try to re-explain:
How it works?
Once it is queued there, it will again go through some validation and if it passes, it is queued to be build shortly on Github Actions. From here, soar will detect that a new pkg was added, and then it will fetch fresh version of the databases(json files)
Because this is not accurate.
We offer two version because the statically linked one only has minimal features (due to a limitation of static linking itself), and in the first case, we git clone and build. Another example: Chromium
In either cases, do you believe it is right to call these as "just pulling prebuilt packages from other main Linux repos"?
From the above explanation, I hope it is clear that soar doesn't "steal" packages and then calls it as our "own"
Nope. My professional career is in cybersecurity, I do this packaging things as a hobby. If you or someone else has an actual technical demonstration of how this is a security vulnerability, please share it with me.
Once again, the way SBUILD is designed and the way our CI works, if a pkg built, it mostly means it will work.
There seems to be some misunderstanding, probably due to my own communication skills.
Yes, builds will fail. Soar won't receive new pkg updates, but soar itself and all our previously built packages will continue working forever (or as long as Github doesn't ban or delete our packages)
They are, and I hope I have already explained what happens if this is the case.
Again, all previous versions will continue working, only newer versions won't.
Can't push a patch for my social skills...
Let's find out how many of these we "pull" from the repositories of other distros? $ curl -qfsSL "https://meta.pkgforge.dev/soarpkgs/INDEX.json" | jq -r '.[] | .build_script' | grep -Ei 'alpine|aur|cachyos|debian|nix|flatpak|brew|pypi|npm' | wc -l
219As you might have noticed, I also used Anyway, what's the total pkgs (sbuilds)? $ curl -qfsSL "https://meta.pkgforge.dev/soarpkgs/INDEX.json" | jq -r '.[] | .build_script' | wc -l
1088So what do you suggest we say now?
I hope by now it is clear, but once again I had like to say |
Yes, as a large portion of the time, this is where your packages are initially coming from. You need to be more transparent about this.
You are calling them your own, though, rather than saying that you are reusing pre-existing build scripts and prebuilt binaries from other upstream sources.
Pacstall is "designed" the same way, with the same kind of build checks, but that philosophy fundamentally conflicts with how packages actually work in the real world. There are always issues that pop up, which is why we have a "quality assurance" command in Pacstall, so users can ensure that packages work correctly on top of the CI. The issue you linked perfectly highlights this vulnerability, so there is your technical demonstration. The lack of a testing suite, and just having a building suite, is a major oversight for automation of this magnitude. It doesn't really matter what your professional career is, and just saying "nope it's not" does not make your case.
And how have you been automatically verifying that? Or is there a possibility that a lot of your packages are actually not
Again, this is overselling the project for something that it is not. If that is your logic, then Pacstall is exactly the same, older versions will continue to build and maintainers don't need to ever alter their pacscripts (except to bump versions), and the same goes for the AUR and many other repositories. But if there are problems that will inevitably arise, as you have admitted they will sometimes, you can't be calling Soar packages immutable as you currently are. Even if it is just 1%, which, given your repository size, is an enormous number of packages that would falter here.
Then what's with the massive lists in https://github.com/pkgforge/metadata/tree/main/misc/data? Where are all of the packages coming from? Because reading the SBUILDs, they are evidently not just coming directly from their individual, original sources.
I think you are taking my criticism the wrong way. I have not accused you of stealing, but that you are not being transparent about where your packages are initially coming from. From the way that Soar has been built, particularly looking at the git history, there had to have been massive imports from third parties, even if these were eventually stripped down. Again, this is by no means inherently wrong, but to mask this is a disservice to both the project and your users.
I'd say I'm not really sure where you get your 12000+ number then on the link you provided earlier.
You're right, it does say all over the docs that you provide static/portable binaries, but do not explain how. How are you bundling in all of the necessary libraries? How are you verifying that all of these packages are completely portable? And how is what you are doing with static libs different than what Nix does? |
As shown in the previous response, only ~ 220 sbuilds are like that, this is ~ 20%
It is not. You are forgetting that Soar is full
Again, the misconception is about our pkg repository and soar.
End users will never even interact with our pkg repositories, aka the repo where we store build scripts.
This data is used for comparisons + statistics.
How many sbuilds have you actually looked through?
This again is a lack of reading stuff properly. On the very same page, you would see
Where is the docs where you describe how to use cmake when writing a pacscript?
Almost all compilers and build tools nowdays will fail and error out if they can't link libraries etc the way you tell them to.
Nix bundles everything in the nix/root dir, it doesn't try static linking unless you tell it to. I appreciate you reopening the discussion. |
|
Hello, I am the creator of Pacstall and I just saw this issue form. A couple things I noticed: I think pkgforge/soar#26 is a sign that either you are doing something very suspicious or you are throwing bandwidth everywhere, both of which are not signs of good programming.
That's not a good sign that your skills have not/cannot be transferable to others. Some of your commits are very, very, very suspicious. pkgforge/metadata@01622c2 I find this problematic, why are you running commands in a yaml file? This should ideally be a stateless file that only describes package declarations and dependencies, not download commands.
We have 5 open (https://github.com/pacstall/pacstall-programs/issues?q=sort%3Aupdated-desc+is%3Aissue+state%3Aopen+label%3ABroken). That is not "so many".
That will segfault because of missing Overall, I think this concept is great, but the execution so far seems to be very sketchy and poorly planned. This reminds me a lot about Vlang and its controversies: a hyped up tool that was not all it promised, but with time and effort, you will have a very cool tool! |
Rhino has issues with mirrors on distributing it's ISOs.
If you had looked just a few seconds more, you would have realized these "binary" file uploads are the compressed json/db files in xz, zstd format If you want to accuse us of "suspicious" activities, please provide more concrete proof
Because it's the most flexible approach.
Please re-read what I said there.
Again, please provide proof of "sketchy" and "suspicious" behaviour/code
Sure if you say so. And as you have already accused of doing sketchy things and bad programming, I feel like we aren't worthy of being included in such a pristine and well-thought project like PacStall/Rhino. |
I'm on your side here, I don't want to dog on your project without offering ways to help you. I don't understand why your first thought when I mentioned that cloudflare is shutting you down is to go and attack Rhino. That has nothing to do with anything here. About the compressed files, we all know that that's how Jia Tan infiltrated a well respected library, and people now should be more wary of this. It also slows down git in general and is harder to diff. Your comment about the commands running is fair, I didn't read enough before making that particular comment, but my point still stands somewhat: I think your build format should be more declarative. About the so files, that's basically the point of a package manager. I find it odd that a package manager won't make certain packages that it provides functional out of the box.
We can have serious conversations without being petty about it 🙂. |
Note
I am the lead maintainer for Soarpkgs, the repo Soar uses.
I am certainly biased, so I encourage independent verification of any claims I have made in this thread
Hi,
I had like to discuss what assurances, features or maturity must Soar meet before the devs can consider adding it as a core component in terms of packaging.
If anyone is unaware and would like a sales pitch: https://docs.pkgforge.dev/soar/readme
About
Soar is a package manager that works on all distros. (Yes, really, even on musl or something as old as cent os)
It is written in rust (we have a basic library that others can integrate), and is licensed under MIT.
The people involved (including me) have a long history in FOSS along with previous projects: https://docs.pkgforge.dev/orgs/readme/people
Features
I highly recommend reading the comparison page: https://docs.pkgforge.dev/soar/readme for details, here I will simply copy paste the Github Readme
Rhino would get access to a very, very large number of packages, all of which have transparent CI build logs, checksums + signing
Rhino would also be able to ship a GUI store as we provide json metadata and all our packages are prebuilt & stored on ghcr. (See: prop(rhino-pkg): [Rhino GUI App Store] #39)
Rhino has also been using AM for a while to support AppImage, if Rhino uses or completely switches to Soar, there would be these additional benefits:
What about pacstall?
Pacstall is a system package manager and can handle libraries. Pacstall is also suited better to installing system components & drivers.
Soar can't handle libraries, neither can it manage anything that requires root access.
Does anyone else use Soar?
Not as system component as of now.
But the creator of https://glaucuslinux.org/ uses & can vouch for Soar.
So do the people over at thc
You can reach them by following the chat links on their project page and asking them about soar
The text was updated successfully, but these errors were encountered: