Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(ubuntu): use iterator #291

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

knqyf263
Copy link
Contributor

@knqyf263 knqyf263 commented Feb 6, 2025

What did you implement:

Use iterator to improve memory usage.

Before

Max RSS: 3.98GiB

        Command being timed: "./gost fetch ubuntu"
        User time (seconds): 217.06
        System time (seconds): 24.51
        Percent of CPU this job got: 96%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 4:10.60
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 4173456
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 4849
        Minor (reclaiming a frame) page faults: 2797255
        Voluntary context switches: 323585
        Involuntary context switches: 1587913
        Swaps: 0
        File system inputs: 0
        File system outputs: 0
        Socket messages sent: 15
        Socket messages received: 31
        Signals delivered: 15960
        Page size (bytes): 16384
        Exit status: 0

After

Max RSS: 184MiB

        Command being timed: "./gost fetch ubuntu"
        User time (seconds): 181.94
        System time (seconds): 22.09
        Percent of CPU this job got: 87%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 3:52.91
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 188544
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 3942
        Minor (reclaiming a frame) page faults: 2263614
        Voluntary context switches: 376996
        Involuntary context switches: 1585121
        Swaps: 0
        File system inputs: 0
        File system outputs: 0
        Socket messages sent: 15
        Socket messages received: 31
        Signals delivered: 46630
        Page size (bytes): 16384
        Exit status: 0

Type of change

Refactoring

How Has This Been Tested?

Run gost fetch ubuntu before and after the changes and see if there are differences in SQLite and Redis.

SQLite

Before

sqlite> select count(1) from ubuntu_bugs;
18981
sqlite> select count(1) from ubuntu_cves;
53955
sqlite> select count(1) from ubuntu_notes;
37440
sqlite> select count(1) from ubuntu_patches;
898432
sqlite> select count(1) from ubuntu_references;
203626
sqlite> select count(1) from ubuntu_release_patches;
11634859
sqlite> select count(1) from ubuntu_upstream_links;
16813
sqlite> select count(1) from ubuntu_upstreams;
11180

After

sqlite> select count(1) from ubuntu_bugs;
18981
sqlite> select count(1) from ubuntu_cves;
53955
sqlite> select count(1) from ubuntu_notes;
37440
sqlite> select count(1) from ubuntu_patches;
898432
sqlite> select count(1) from ubuntu_references;
203626
sqlite> select count(1) from ubuntu_references;
203626
sqlite> select count(1) from ubuntu_release_patches;
11634859
sqlite> select count(1) from ubuntu_upstream_links;
16813
sqlite> select count(1) from ubuntu_upstreams;
11180

Redis

Before

127.0.0.1:6379> DBSIZE
(integer) 5364
127.0.0.1:6379> HLEN GOST#UBU#ADV
(integer) 8379
127.0.0.1:6379> HLEN GOST#UBU#CVE
(integer) 53955

After

127.0.0.1:6379> DBSIZE
(integer) 5364
127.0.0.1:6379> HLEN GOST#UBU#ADV
(integer) 8379
127.0.0.1:6379> HLEN GOST#UBU#CVE
(integer) 53955

Checklist:

You don't have to satisfy all of the following.

  • Write tests
  • Write documentation
  • Check that there aren't other open pull requests for the same issue/feature
  • Format your source code by make fmt
  • Pass the test by make test
  • Provide verification config / commands
  • Enable "Allow edits from maintainers" for this PR
  • Update the messages below

Signed-off-by: knqyf263 <knqyf263@gmail.com>
@knqyf263 knqyf263 marked this pull request as ready for review February 6, 2025 09:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant