Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable packages metadata to be float when int is expected #9726

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

m-czernek
Copy link
Contributor

@m-czernek m-czernek commented Feb 4, 2025

What does this PR change?

When we process package metadata, packages can mistakenly have float where we expect integer, for example Installed Size, Size, etc.

Though this is technically an invalid package, Uyuni can round such values to integer instead of throwing an exception.

In this PR, if a package contains a float value in metadata where we expect integer, we first round the value to the nearest integer.

In case where such value is neither float nor int, I improve the error that is displayed to the user.

Documentation

  • No documentation needed: only internal and user invisible changes

Links

Issue(s): #9613
Port(s): TODO

  • DONE

Changelogs

Make sure the changelogs entries you are adding are compliant with https://github.com/uyuni-project/uyuni/wiki/Contributing#changelogs and https://github.com/uyuni-project/uyuni/wiki/Contributing#uyuni-projectuyuni-repository

If you don't need a changelog check, please mark this checkbox:

  • No changelog needed

If you uncheck the checkbox after the PR is created, you will need to re-run changelog_test (see below)

Re-run a test

If you need to re-run a test, please mark the related checkbox, it will be unchecked automatically once it has re-run:

  • Re-run test "changelog_test"
  • Re-run test "backend_unittests_pgsql"
  • Re-run test "java_pgsql_tests"
  • Re-run test "schema_migration_test_pgsql"
  • Re-run test "susemanager_unittests"
  • Re-run test "javascript_lint"
  • Re-run test "spacecmd_unittests"

Before you merge

Check How to branch and merge properly!

@m-czernek m-czernek requested a review from a team as a code owner February 4, 2025 14:00
@m-czernek m-czernek requested review from agraul and removed request for a team February 4, 2025 14:00
value = value[: datatype.limit]
# ignore incomplete characters created after truncating
return value
return _sanitize_dbstring(value, datatype)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I refactored this into a separate function so that sonarcloud won't fail with method complexity being too high.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder why we use classes, but then don't do this with the classes. Would be too big to change that here, but it's something we could think about changing.

Copy link
Member

@agraul agraul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have any examples we can use for testing this change?

@@ -0,0 +1 @@
- Handle pkg metadata containing float instead of an int (GH#9613)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we using "GH" without any project/repo in changelogs? If we don't like gh#uyuni-project/uyuni#9613, maybe we could request to add uyuni#9613 to known trackers in OBS?

value = value[: datatype.limit]
# ignore incomplete characters created after truncating
return value
return _sanitize_dbstring(value, datatype)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder why we use classes, but then don't do this with the classes. Would be too big to change that here, but it's something we could think about changing.

@m-czernek
Copy link
Contributor Author

m-czernek commented Feb 7, 2025

@agraul for testing, see [0]. I personally tested it with the wolframscript pkg from [1], which uses Installed-Size: 6635.2451171875.

To not sync the whole repo, I created a local deb repo (using the file:// protocol). If it'd help, let me know, I can send you the repo.

[0] http://archive.raspberrypi.org/debian/dists/bookworm/main/binary-arm64/Packages
[1] https://archive.raspberrypi.org/debian/pool/main/w/wolframscript/wolframscript_14.1.0+20240822262_arm64.deb

@agraul
Copy link
Member

agraul commented Feb 10, 2025

Thanks for the pointer, I'll try to sync this to see if we can solve the issue higher up in the stack. Just to have a second approach we can compare to this one.

@agraul
Copy link
Member

agraul commented Feb 10, 2025

The alternative that immediately came to my mind was to fix this at the outside layer instead of deep down close to the DB insertion. The downside is that this approach only works for known keys, instead of anything. But casting anything to float then int is maybe not needed either.

python/spacewalk/server/importlib/debPackage.py | 5 +++++

modified   python/spacewalk/server/importlib/debPackage.py
@@ -59,6 +59,11 @@ class debBinaryPackage(headerSource.rpmBinaryPackage):
             value = header[field]
             if key == "build_time" and isinstance(value, int):
                 value = gmtime(value)  # unix timestamp
+            elif key == "payload_size" and value:
+                # Payload size / Installed-Size is an int, but some packages have
+                # bad metadata (float). Since Installed-Size is an estimation, truncating to int is ok
+                # https://www.debian.org/doc/debian-policy/ch-controlfields.html#installed-size
+                value = int(float(value))
             elif value == []:
                 value = None
             elif value:

I'm not sure why we currently keep every value as a string until it hits the DB and only casts it to the expected DB type. Maybe your approach fits better with the current approach.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants