Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI: wait and retry for diffile to exist #301

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

siddharth-krishna
Copy link
Collaborator

An attempt at fixing #265

Copy link

Regression test results on commit 4f0635b

       Benchmark    Time (s)                 GDX Diff     Accuracy       Correct    Additional
----------------  ----------  -----------------------  -----------  ------------  ------------
   DemoS_001-all   3.1   3.9            OK         OK  100.0 100.0    118    118      3      3
   DemoS_002-all   3.3   3.5            OK         OK  100.0 100.0    344    344      3      3
   DemoS_003-all   3.7   4.0            OK         OK  100.0 100.0    633    633      6      6
       DemoS_004   3.8   4.9            OK         OK  100.0 100.0    662    662     12     12
      DemoS_004a   3.9   4.5            OK         OK  100.0 100.0    665    665     12     12
      DemoS_004b   3.8   4.9            OK         OK  100.0 100.0    665    665     12     12
   DemoS_004-all   4.0   4.7            OK         OK  100.0 100.0    667    667     12     12
   DemoS_005-all   4.4   5.5            13         13   99.7  99.7   1160   1160     12     12
   DemoS_006-all   5.0   6.0            13         13   99.7  99.7   1258   1258     12     12
   DemoS_007-all   5.9   6.9            13         13   99.8  99.8   2155   2155     12     12
DemoS_007-all-1r   5.1   6.4            11         11   99.8  99.8   1179   1179     12     12
   DemoS_008-all   7.9   9.0            13         13   99.9  99.9   5333   5333     18     18
   DemoS_009-all   8.6  10.2            26         26   99.9  99.9   5807   5807     29     29
   DemoS_010-all   8.8  10.5           822        822   88.4  88.4   6139   6139     29     29
   DemoS_011-all   8.9  10.2           854        854   88.0  88.0   6150   6150     29     29
   DemoS_012-all   9.2  10.7           901        901   88.2  88.2   6306   6306     53     53
DemoS_special-t1   4.8   6.0  Error runn… Error runn…   92.5  92.5   2110   2110     48     48
    TIMES-IE-all  25.6  27.2          3131       3131   97.5  97.5  42460  42460   2826   2826
    TIMES-IE-NoM  23.5  23.7           363        363   99.3  99.3  40528  40528    632    632
    TIMES-IE-MCB  23.9  24.4          1106       1106   97.6  97.6  42067  42067    632    632
    TIMES-NZ-KEA  34.6  34.7          3932       3932   98.3  98.3  76416  76416    458    458
    TIMES-NZ-TUI  24.8  24.9          3966       3966   98.3  98.3  76042  76042    474    474
 (__main__:MainThread:pid-5938 "/home/runner/work/xl2times/xl2times/xl2times/utils/run_benchmarks.py:394")
2025-01-27 18:53:14.477 | INFO     : Total runtime: 246.55s (main: 226.65s) (__main__:MainThread:pid-5938 "/home/runner/work/xl2times/xl2times/xl2times/utils/run_benchmarks.py:418")
2025-01-27 18:53:14.477 | INFO     : Change in runtime (negative == faster): +19.90s (+8.8%) (__main__:MainThread:pid-5938 "/home/runner/work/xl2times/xl2times/xl2times/utils/run_benchmarks.py:419")
2025-01-27 18:53:14.477 | INFO     : Change in correct rows (higher == better): +0 (+0.0%) (__main__:MainThread:pid-5938 "/home/runner/work/xl2times/xl2times/xl2times/utils/run_benchmarks.py:426")
2025-01-27 18:53:14.477 | INFO     : Change in additional rows: +0 (+0.0%) (__main__:MainThread:pid-5938 "/home/runner/work/xl2times/xl2times/xl2times/utils/run_benchmarks.py:433")
2025-01-27 18:53:14.477 | SUCCESS  : No regressions. You're awesome! (__main__:MainThread:pid-5938 "/home/runner/work/xl2times/xl2times/xl2times/utils/run_benchmarks.py:447")

This comment will be updated when new commits are added to the PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant