Add a CI script to test the decoder on three different sets of SIF problems #20

amontoison · 2024-08-15T19:27:18Z

This PR adds a CI script to test that the new decoder, sifdecoder_standalone, works on all problems in the sifcollection, maros-meszaros and netlib-lp collections.

The problems are decoded in single, double, and quadruple precision, and then compiled.
For now, I am only compiling the decoded problems with gfortran on Linux.

amontoison · 2024-08-15T19:43:02Z

For the sifcollection, AKIVA.SIF can't be decoded in single and quadruple precision.
But all problems can be decoded in double precision. 🎉
If we can decode some problems only in double precision, it should be documented with a note at the end of the README.md.

For the SIF files in maros-meszaros and netlib, all problems can be decoded in the three precision 🎉 🎉

jfowkes · 2024-08-16T08:24:45Z

@nimgould in AKIVA.SIF you're using DMAX = DABS(DX(1)) and clearly DABS is not portable to other precisions.

nimgould · 2024-08-16T09:32:17Z

Well I can't see any DABS in AKIVA ... must be my old eyes :

jfowkes · 2024-08-16T09:36:22Z

Ah sorry, it's in the generated EXTERN.f:

EXTER_s.f:4282:18:

 4282 |       DMAX = DABS(DX(1))
      |                  1
Error: ‘a’ argument of ‘dabs’ intrinsic at (1) must be double precision

nimgould · 2024-08-16T09:39:40Z

Hmmm ... EXTERN_s.f is only 541 lines long ... and no mention of DABS. Are you sure you decoded the right example?

nimgould · 2024-08-16T09:43:37Z

It seems to be from BLEACHNG.SIF. That example should definitely only be
used in double as it relies on an ODE solvers for which accuracy is imperative.
Really anything that has an EXTER.f should be treated with caution!

jfowkes · 2024-08-16T09:44:55Z

I guess we need a way to flag which examples cannot be run in single and quadruple precision?

nimgould · 2024-08-16T09:49:02Z

I've updated DABS -> ABS in BLEACHING, so maybe it will now compile. I wouldn't trust the answer though!

jfowkes · 2024-08-16T09:50:04Z

Let me re-run the tests on here and we'll see!

jfowkes · 2024-08-16T09:56:08Z

@nimgould still getting some quad precision errors with AKIVA, see the failing quadruple test below for details. Also still quite a few errors in the single precision decoding attempts, see the single precision test log below.

nimgould · 2024-08-16T10:07:29Z

Yep, as I said before the external part of any SIF file really needs an interpreter to do a proper translation, akin to writing a compiler. As neither you nor I presumably have the inclination to write such a thing, these errors will persist. The poor algorithm tester will loose 10 or so examples out of 1750 if they want to explore single or quad.

jfowkes · 2024-08-16T10:09:17Z

Okay in that case @amontoison can we exclude all the SIF problems with EXTERN's from the single/quad tests?

amontoison · 2024-08-16T12:21:20Z

@jfowkes @nimgould
I also have an issue with DABS in CORE1.SIF.
The error is in ELFUN_s.f / ELFUN_q.f.

amontoison · 2024-08-16T12:43:05Z

The following 33 problems are excluded (in single and quadruple precision) because they contain an EXTER.f:

AKIVA.SIF
BA-L1.SIF
BA-L16.SIF
BA-L16LS.SIF
BA-L1LS.SIF
BA-L21.SIF
BA-L21LS.SIF
BA-L49.SIF
BA-L49LS.SIF
BA-L52.SIF
BA-L52LS.SIF
BA-L73.SIF
BA-L73LS.SIF
BLEACHNG.SIF
CAR2.SIF
HIELOW.SIF
HS67.SIF
HS68.SIF
HS69.SIF
HS85.SIF
HS88.SIF
HS89.SIF
HS90.SIF
HS91.SIF
HS92.SIF
JIMACK.SIF
ORBIT2.SIF
PARKCH.SIF
RAYBENDS.SIF
SARO.SIF
SAROMM.SIF
STRATEC.SIF
VANDANIUMS.SIF
VANDANMSLS.SIF

jfowkes · 2024-08-16T12:46:21Z

@jfowkes @nimgould I also have an issue with DABS in CORE1.SIF. The error is in ELFUN_s.f / ELFUN_q.f.

Thanks @amontoison, CORE1.SIF should definitely not have DABS in its ELFUN_s.f / ELFUN_q.f.

nimgould · 2024-08-16T12:54:14Z

Fixed

@jfowkes @nimgould I also have an issue with DABS in CORE1.SIF. The error is in ELFUN_s.f / ELFUN_q.f.

Thanks @amontoison, CORE1.SIF should definitely not have DABS in its ELFUN_s.f / ELFUN_q.f.

amontoison · 2024-08-16T13:08:46Z

Fixed

@jfowkes @nimgould I also have an issue with DABS in CORE1.SIF. The error is in ELFUN_s.f / ELFUN_q.f.

Thanks @amontoison, CORE1.SIF should definitely not have DABS in its ELFUN_s.f / ELFUN_q.f.

Same issue in CORE2.SIF 😅
Your binary sifdecoder_standalone is quite useful, Nick, for running some CI tests. 😉

jfowkes · 2024-08-16T13:09:35Z

Fixed

@jfowkes @nimgould I also have an issue with DABS in CORE1.SIF. The error is in ELFUN_s.f / ELFUN_q.f.

Thanks @amontoison, CORE1.SIF should definitely not have DABS in its ELFUN_s.f / ELFUN_q.f.

Same issue in CORE2.SIF 😅

Probably worth grepping globally for DABS in all the SIF files?

nimgould · 2024-08-16T13:17:53Z

The HS*, BA*, JIMACK, ORBIT2, IELOW, PARKCH, RAYBENDS, STRATEC and VANDAN* examples should be ok in single precision. I think I know how to make these accessible in quad as well, but it will need me to add an extra (commented) line that uses ISO_FORTRAN_ENV, ONLY : real128 and that is processed with the -qp flag. No time to try right now

amontoison · 2024-08-16T13:28:52Z

alexis@HP-Spectre:~/Téléchargements/optrove-sif-7a01e1efd8de$ grep -nr DABS
CORE2.SIF:939: F                      X * DABS(X)

nimgould · 2024-08-16T13:45:35Z

I modified the sif decoder so that it is more likely that it will translate any external parts to 128 bit reals correctly. It won't catch everything. some examples have hard-wired constants smaller than 10^-6 so the single versions may be in trouble there (but that is why we have double precision!)

jfowkes · 2024-08-16T13:54:01Z

Okay so we just need to fix the DABS in CORE2.SIF?

nimgould · 2024-08-16T14:24:18Z

Didn't I do that? Sent from Outlook for Android<https://aka.ms/AAb9ysg>

…

________________________________ From: Jari ***@***.***> Sent: Friday, August 16, 2024 2:54:22 PM To: ralna/SIFDecode ***@***.***> Cc: Gould, Nick (STFC,RAL,SC) ***@***.***>; Mention ***@***.***> Subject: Re: [ralna/SIFDecode] Add a CI script to test the decoder on three different sets of SIF problems (PR #20) Okay so we just need to fix the DABS in CORE2.SIF? — Reply to this email directly, view it on GitHub<#20 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ACW4A6TTJABNFXNVZEIWP4LZRYAA5AVCNFSM6AAAAABMSYRWGGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOJTGU2TKOJVGY>. You are receiving this because you were mentioned.Message ID: ***@***.***>

amontoison · 2024-08-16T18:19:21Z

I modified the sif decoder so that it is more likely that it will translate any external parts to 128 bit reals correctly. It won't catch everything. some examples have hard-wired constants smaller than 10^-6 so the single versions may be in trouble there (but that is why we have double precision!)

Thanks Nick, all tests passed if we don't use the SIF problems with EXTER.f.
I isolated to following resistance of 5 problems for single precision.
Except for HS68 and HS69, the error with NOEUD seems to be an issue not related single precision.

rm -f *.f *.o *.d
./sifdecoder_standalone -sp HIELOW.SIF
gfortran -shared -fPIC *.f

EXTER_s.f:10546:36:
10546 |             DO 1200 NOEUD = 1,NNOEUE-1
      |                                    1
Error: Symbol ‘nnoeue’ at (1) has no IMPLICIT type; did you mean ‘nnoeud’?

rm -f *.f *.o *.d
./sifdecoder_standalone -sp HS68.SIF
gfortran -shared -fPIC *.f

EXTER_s.f:147:51:

  147 |      1     XBIG,XHUGE,XMAX/26.543D0,6.71D7,2.53D307/
      |                                                   1
Error: Arithmetic overflow converting REAL(8) to REAL(4) at (1). This check can be disabled with the option ‘-fno-range-check’
EXTER_s.f:146:36:

  146 |       DATA XINF,XNEG,XSMALL/1.79D308,-26.628D0,1.11E-16/,
      |                                    1
Error: Arithmetic overflow converting REAL(8) to REAL(4) at (1). This check can be disabled with the option ‘-fno-range-check’

rm -f *.f *.o *.d
./sifdecoder_standalone -sp HS69.SIF
gfortran -shared -fPIC *.f

EXTER_s.f:147:51:

  147 |      1     XBIG,XHUGE,XMAX/26.543D0,6.71D7,2.53D307/
      |                                                   1
Error: Arithmetic overflow converting REAL(8) to REAL(4) at (1). This check can be disabled with the option ‘-fno-range-check’
EXTER_s.f:146:36:

  146 |       DATA XINF,XNEG,XSMALL/1.79D308,-26.628D0,1.11E-16/,
      |                                    1
Error: Arithmetic overflow converting REAL(8) to REAL(4) at (1). This check can be disabled with the option ‘-fno-range-check’

rm -f *.f *.o *.d
./sifdecoder_standalone -sp PARKCH.SIF
gfortran -shared -fPIC *.f

EXTER_s.f:34125:36:

34125 |             DO 1200 NOEUD = 1,NNOEUE-1
      |                                    1
Error: Symbol ‘nnoeue’ at (1) has no IMPLICIT type; did you mean ‘nnoeud’?

rm -f *.f *.o *.d
./sifdecoder_standalone -sp STRATEC.SIF
gfortran -shared -fPIC *.f

EXTER_s.f:34125:36:

34125 |             DO 1200 NOEUD = 1,NNOEUE-1
      |                                    1
Error: Symbol ‘nnoeue’ at (1) has no IMPLICIT type; did you mean ‘nnoeud’?

amontoison · 2024-08-16T18:39:30Z

For quadruple precision, I have 15 problems that are not working:

rm -f *.f *.o *.d
./sifdecoder_standalone -qp BLEACHNG.SIF
gfortran -shared -fPIC *.f

rm -f *.f *.o *.d
./sifdecoder_standalone -qp CAR2.SIF
gfortran -shared -fPIC *.f

rm -f *.f *.o *.d
./sifdecoder_standalone -qp HIELOW.SIF
gfortran -shared -fPIC *.f

rm -f *.f *.o *.d
./sifdecoder_standalone -qp HS88.SIF
gfortran -shared -fPIC *.f

rm -f *.f *.o *.d
./sifdecoder_standalone -qp HS89.SIF
gfortran -shared -fPIC *.f

rm -f *.f *.o *.d
./sifdecoder_standalone -qp HS90.SIF
gfortran -shared -fPIC *.f

rm -f *.f *.o *.d
./sifdecoder_standalone -qp HS91.SIF
gfortran -shared -fPIC *.f

rm -f *.f *.o *.d
./sifdecoder_standalone -qp HS92.SIF
gfortran -shared -fPIC *.f

rm -f *.f *.o *.d
./sifdecoder_standalone -qp JIMACK.SIF
gfortran -shared -fPIC *.f

rm -f *.f *.o *.d
./sifdecoder_standalone -qp ORBIT2.SIF
gfortran -shared -fPIC *.f

rm -f *.f *.o *.d
./sifdecoder_standalone -qp PARKCH.SIF
gfortran -shared -fPIC *.f

rm -f *.f *.o *.d
./sifdecoder_standalone -qp RAYBENDS.SIF
gfortran -shared -fPIC *.f

rm -f *.f *.o *.d
./sifdecoder_standalone -qp SARO.SIF
gfortran -shared -fPIC *.f

rm -f *.f *.o *.d
./sifdecoder_standalone -qp SAROMM.SIF
gfortran -shared -fPIC *.f

rm -f *.f *.o *.d
./sifdecoder_standalone -qp STRATEC.SIF
gfortran -shared -fPIC *.f

I have this kind of error:

EXTER_q.f:8:11:

    8 |       REAL(REAL128)    FUNCTION SAROFN(
      |           1
Error: Parameter ‘real128’ at (1) has not been declared or is a variable, which does not reduce to a constant expression
EXTER_q.f:10:18:

   10 |      1   RK, RIDX, SYSEQ,
      |                  1
Error: Syntax error in USE statement at (1)
EXTER_q.f:16:11:

   16 |       REAL(REAL128)    RK, RIDX
      |           1
Error: Parameter ‘real128’ at (1) has not been declared or is a variable, which does not reduce to a constant expression
EXTER_q.f:18:11:

   18 |       REAL(REAL128)    X1, X2, X3, X4, X5, X6, X7, X8, X9
      |           1
Error: Parameter ‘real128’ at (1) has not been declared or is a variable, which does not reduce to a constant expression
EXTER_q.f:19:11:

   19 |       REAL(REAL128)    U1, U2
      |           1
Error: Parameter ‘real128’ at (1) has not been declared or is a variable, which does not reduce to a constant expression
EXTER_q.f:21:11:

   21 |       REAL(REAL128)    X(NX), U(NU), F(NX), G(NG)
      |           1
Error: Parameter ‘real128’ at (1) has not been declared or is a variable, which does not reduce to a constant expression
EXTER_q.f:22:11:

   22 |       REAL(REAL128)    RKZBAK
      |           1
Error: Parameter ‘real128’ at (1) has not been declared or is a variable, which does not reduce to a constant expression
EXTER_q.f:75:11:

   75 |       REAL(REAL128)    FUNCTION SAROGN(
      |           1
Error: Parameter ‘real128’ at (1) has not been declared or is a variable, which does not reduce to a constant expression
EXTER_q.f:77:18:

   77 |      1   RK, RIDX, SYSEQ,
      |                  1
Error: Syntax error in USE statement at (1)
EXTER_q.f:14:72:

   14 |       INTEGER NX, NU, NG
      |                                                                        1
......
   83 |       INTEGER NX, NU, NG, PMAX
      |                                                                        2
Error: Two main PROGRAMs at (1) and (2)

…oblems

amontoison requested review from jfowkes and nimgould August 15, 2024 19:27

amontoison changed the title ~~Add a CI script to test the decoder on all problems of the sifcollection~~ Add a CI script to test the decoder on three different sets of SIF problems Aug 15, 2024

amontoison force-pushed the sifcollection branch 2 times, most recently from ee1b5dd to 70d4b0e Compare August 15, 2024 20:41

amontoison mentioned this pull request Aug 15, 2024

Update Artifacts.toml JuliaSmoothOptimizers/CUTEst.jl#338

Merged

amontoison force-pushed the sifcollection branch from 9136f7f to fb59979 Compare August 16, 2024 11:55

amontoison mentioned this pull request Aug 16, 2024

Provide the list of problems that can be only decoded in double precision #23

Closed

amontoison force-pushed the sifcollection branch 3 times, most recently from c1064fc to 8f3648b Compare August 17, 2024 16:31

Add a CI script to test the decoder on three different sets of SIF pr…

63aa4dd

…oblems

amontoison force-pushed the sifcollection branch from 8f3648b to 63aa4dd Compare August 17, 2024 16:48

amontoison merged commit 67d8dfb into master Aug 17, 2024
19 checks passed

amontoison deleted the sifcollection branch August 17, 2024 17:09

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add a CI script to test the decoder on three different sets of SIF problems #20

Add a CI script to test the decoder on three different sets of SIF problems #20

amontoison commented Aug 15, 2024 •

edited

Loading

amontoison commented Aug 15, 2024 •

edited

Loading

jfowkes commented Aug 16, 2024

nimgould commented Aug 16, 2024

jfowkes commented Aug 16, 2024

nimgould commented Aug 16, 2024

nimgould commented Aug 16, 2024

jfowkes commented Aug 16, 2024

nimgould commented Aug 16, 2024

jfowkes commented Aug 16, 2024

jfowkes commented Aug 16, 2024 •

edited

Loading

nimgould commented Aug 16, 2024

jfowkes commented Aug 16, 2024

amontoison commented Aug 16, 2024 •

edited

Loading

amontoison commented Aug 16, 2024 •

edited

Loading

jfowkes commented Aug 16, 2024

nimgould commented Aug 16, 2024

amontoison commented Aug 16, 2024 •

edited

Loading

jfowkes commented Aug 16, 2024

nimgould commented Aug 16, 2024

amontoison commented Aug 16, 2024

nimgould commented Aug 16, 2024

jfowkes commented Aug 16, 2024

nimgould commented Aug 16, 2024 via email

amontoison commented Aug 16, 2024 •

edited

Loading

amontoison commented Aug 16, 2024 •

edited

Loading

Add a CI script to test the decoder on three different sets of SIF problems #20

Add a CI script to test the decoder on three different sets of SIF problems #20

Conversation

amontoison commented Aug 15, 2024 • edited Loading

amontoison commented Aug 15, 2024 • edited Loading

jfowkes commented Aug 16, 2024

nimgould commented Aug 16, 2024

jfowkes commented Aug 16, 2024

nimgould commented Aug 16, 2024

nimgould commented Aug 16, 2024

jfowkes commented Aug 16, 2024

nimgould commented Aug 16, 2024

jfowkes commented Aug 16, 2024

jfowkes commented Aug 16, 2024 • edited Loading

nimgould commented Aug 16, 2024

jfowkes commented Aug 16, 2024

amontoison commented Aug 16, 2024 • edited Loading

amontoison commented Aug 16, 2024 • edited Loading

jfowkes commented Aug 16, 2024

nimgould commented Aug 16, 2024

amontoison commented Aug 16, 2024 • edited Loading

jfowkes commented Aug 16, 2024

nimgould commented Aug 16, 2024

amontoison commented Aug 16, 2024

nimgould commented Aug 16, 2024

jfowkes commented Aug 16, 2024

nimgould commented Aug 16, 2024 via email

amontoison commented Aug 16, 2024 • edited Loading

amontoison commented Aug 16, 2024 • edited Loading

amontoison commented Aug 15, 2024 •

edited

Loading

amontoison commented Aug 15, 2024 •

edited

Loading

jfowkes commented Aug 16, 2024 •

edited

Loading

amontoison commented Aug 16, 2024 •

edited

Loading

amontoison commented Aug 16, 2024 •

edited

Loading

amontoison commented Aug 16, 2024 •

edited

Loading

amontoison commented Aug 16, 2024 •

edited

Loading

amontoison commented Aug 16, 2024 •

edited

Loading