[PRE REVIEW]: gpu-ISTL - Extending OPM Flow with GPU Linear Solvers

[editorialbot]

Submitting author: @multitalentloes (Tobias Meyer Andersen)
Repository: https://github.com/OPM/opm-simulators
Branch with paper.md (empty if default branch): joss_gpuistl_paper
Version: 2024.10
Editor: @prashjha
Reviewers: @pratikvn, @berenger-eu
Managing EiC: Daniel S. Katz

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/8c35a29762fc508215904c0e1dd16013"><img src="https://joss.theoj.org/papers/8c35a29762fc508215904c0e1dd16013/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/8c35a29762fc508215904c0e1dd16013/status.svg)](https://joss.theoj.org/papers/8c35a29762fc508215904c0e1dd16013)

Author instructions

Thanks for submitting your paper to JOSS @multitalentloes. Currently, there isn't a JOSS editor assigned to your paper.

@multitalentloes if you have any suggestions for potential reviewers then please mention them here in this thread (without tagging them with an @). You can search the list of people that have already agreed to review and may be suitable for this submission.

Editor instructions

The JOSS submission bot @editorialbot is here to help you find and assign reviewers and start the main review. To find out what @editorialbot can do for you type:

@editorialbot commands

[editorialbot]

Hello human, I'm @editorialbot, a robot that can help you with some common editorial tasks.

For a list of things I can do to help you, just type:

@editorialbot commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@editorialbot generate pdf

[editorialbot]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

✅ OK DOIs

- 10.1137/15M1026419 is OK
- 10.1016/j.parco.2021.102831 is OK
- 10.1016/j.camwa.2020.05.014 is OK
- 10.1016/j.camwa.2020.06.007 is OK
- 10.1016/j.camwa.2020.02.012 is OK
- 10.2118/218015-pa is OK
- 10.3390/mca24030070 is OK

🟡 SKIP DOIs

- No DOI given, and none found for title: Computer Architecture, Sixth Edition: A Quantitati...
- No DOI given, and none found for title: An Introduction to Reservoir Simulation Using MATL...
- No DOI given, and none found for title: CUDA, release: 12.2.r12.2
- No DOI given, and none found for title: hipify-perl
- No DOI given, and none found for title: Heterogeneous-computing Interface for Portability
- No DOI given, and none found for title: Bandicoot: C++ Library for GPU Linear Algebra and ...

❌ MISSING DOIs

- 10.1007/978-3-540-75755-9_82 may be a valid DOI for title: The Iterative Solver Template Library

❌ INVALID DOIs

- --- is INVALID

[editorialbot]

Software report:

github.com/AlDanial/cloc v 1.90  T=0.70 s (1694.9 files/s, 400720.8 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
C/C++ Header                   617          24852          39348          92073
C++                            437          16384          12267          73855
SVG                              5              4             23           4704
CMake                           16            483            285           4298
TeX                             27            617            130           4024
CUDA                            12            375            286           2202
Bourne Shell                    25            261            119           1377
OpenCL                          23            124             97           1011
Python                           9             53            112            344
Markdown                         5            101              0            340
JSON                             3              0              0            124
YAML                             2              4              0             40
MATLAB                           1              8              0             20
Dockerfile                       1              2              3             13
awk                              1              1             24             10
TOML                             1              0              0              7
INI                              1              0              0              3
-------------------------------------------------------------------------------
SUM:                          1186          43269          52694         184445
-------------------------------------------------------------------------------

Commit count by author:

  3007	Arne Morten Kvarving
  2216	Atgeirr Flø Rasmussen
  1444	Andreas Lauser
  1255	Kai Bao
  1206	Tor Harald Sandve
  1024	Markus Blatt
   894	Bård Skaflestad
   796	Joakim Hove
   320	Liu Ming
   226	Robert Kloefkorn
   185	Roland Kaufmann
   184	Kjetil Olsen Lye
   176	Xavier Raynaud
   174	hnil
   170	Tong Dong Qiu
   146	babrodtk
   143	Håkon Hægland
   102	Tobias Meyer Andersen
   100	Vegard Kippe
    89	Robert K
    82	Jørgen Kvalsvik
    74	osae
    73	Paul Egberts
    69	T.D. (Tongdong) Qiu
    67	Alf Birger Rustad
    67	Bernd Flemisch
    63	Stein Krogstad
    52	Lisa Julia Nebel
    45	Kristian Flikka
    36	Antonella Ritorto
    36	goncalvesmachadoc
    34	OPMUSER
    34	Pål Grønås Drange
    33	Benjamin Faigle
    31	Steinar Foss
    30	Philipp Nuske
    30	Razvan Nane
    29	Markus Wolff
    28	Cintia Goncalves Machado
    28	Halvor Møll Nilsen
    28	jakobtorben
    26	Klaus Mosthaf
    26	kel85uk
    25	Melanie Darcis
    24	Jose Eduardo Bueno
    24	Ove Sævareid
    24	Svenn Tveit
    23	David Landa Marban
    23	Rohith Nair
    22	Elyes Ahmed
    20	Torbjørn Skille
    19	André R. Brodtkorb
    19	David Werner
    19	Franz G. Fuchs
    19	Paul
    19	chflo
    17	Christoph Grueninger
    17	tqiu
    16	josh bowden
    12	Josh Bowden
    11	Alf B. Rustad
    10	Halvor M. Nilsen
    10	Jostein Alvestad
    10	Jostein R. Natvig
    10	andrthu
     9	Kjell W. Kongsvik
     8	Katherina Baber
     7	jilocode
     6	Andrea T. Lonn
     6	Michael Sargado
     6	Trine Mykkeltvedt
     6	WesselDeZeeuw
     5	Edvin Brudevoll
     5	Franz Georg Fuchs
     4	Anders Matheson
     4	Erik Hide Sæternes
     4	Fredrik Gundersen
     4	Júlio Hoffimann
     4	Nicolas Schwenck
     4	Peter Verveer
     4	Trine S. Mykkeltvedt
     4	dependabot[bot]
     3	Alexander Kissinger
     3	Jens Olav Nygaard
     3	Justin Chang
     3	Jørgen Kvalsvk
     3	Karin Erbertseder
     3	rube051
     2	Andreas Thune
     2	Erik Hide Saeternes
     2	Giacomo Marchiori
     2	Joshua Bowden
     2	Kristian Hole-Drabløs
     2	Lars Petter Øren Hauge
     2	Tom Hogervorst
     2	Tor Harald Sanve
     2	Trine S Mykkeltvedt
     2	daavid00
     2	locture
     1	Andrea Tronstad Lønn (ST MSU DYN)
     1	Baurzhan Muftakhidinov
     1	Bjørn Spjelkavik
     1	Eduardo Bueno
     1	Georg Zitzlsberger
     1	Holger Class
     1	Jakob Torben
     1	Jean-Paul Balabanian
     1	Jochen Fritz
     1	Matthew Goodfield
     1	Timo Koch
     1	Viktor Szukitsch
     1	Vishal Jambhekar
     1	WesselZ
     1	Zeeuw
     1	kristinf
     1	nairr
     1	paean
     1	steink

[editorialbot]

Paper file info:

📄 Wordcount for paper.md is 1521

✅ The paper includes a Statement of need section

[editorialbot]

License info:

🟡 License found: GNU General Public License v3.0 (Check here for OSI approval)

[editorialbot]

👉📄 Download article proof 📄 View article proof on GitHub 📄 👈

[editorialbot]

Five most similar historical JOSS papers:

Ginkgo: A high performance numerical linear algebra library
Submitting author: @pratikvn
Handling editor: @mjsottile (Retired)
Reviewers: @adam-m-jcbs, @zeroset
Similarity score: 0.6914

SPbLA: The Library of GPGPU-powered Sparse Boolean Linear Algebra Operations
Submitting author: @EgorOrachyov
Handling editor: @Nikoleta-v3 (Active)
Reviewers: @mlxd, @bencardoen
Similarity score: 0.6856

libCEED: Fast algebra for high-order element-based discretizations
Submitting author: @jedbrown
Handling editor: @diehlpk (Active)
Reviewers: @thelfer, @FreddieWitherden,
Similarity score: 0.6804

iharm3D: Vectorized General Relativistic Magnetohydrodynamics
Submitting author: @bprather
Handling editor: @eloisabentivegna (Retired)
Reviewers: @bgiacoma, @cpalenzuela
Similarity score: 0.6789

The 2DECOMP&FFT library: an update with new CPU/GPU capabilities
Submitting author: @rfj82982
Handling editor: @diehlpk (Active)
Reviewers: @CeciliaCoelho, @p-costa
Similarity score: 0.6763

⚠️ Note to editors: If these papers look like they might be a good match, click through to the review issue for that paper and invite one or more of the authors before considering asking the reviewers of these papers to review again for JOSS.

[danielskatz]

@multitalentloes - Our apologies for the delay in getting this started. Something went wrong internally, but it's now fixed.

[danielskatz]

I note that this is a very large repo. It appears that you are submitted a part of this for review, as an element of the full repo, but this is not completely clear.

Can you identify the branch/files/etc that is the submitted contribution?

Additionally, the paper is about 50% longer than recommended for a JOSS submission. I wonder if there's part of this that could be place in the documentation or repo, then pointed to there, rather than being in the paper? The performance case study perhaps?

[multitalentloes]

Hello Daniel, thank you very much for initiating the pre-review.

The contributions described in this article predominantly reside in the gpuistl folder. Redoing the work of the bot with cloc on that folder yields something like this:

$ cloc opm-simulators/opm/simulators/linalg/gpuistl/
      59 text files.
      59 unique files.                              
       0 files ignored.

github.com/AlDanial/cloc v 1.90  T=0.02 s (2756.2 files/s, 507055.6 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
C/C++ Header                    42            804           2616           3193
C++                             12            316            270           2136
CUDA                             5            154            133           1232
-------------------------------------------------------------------------------
SUM:                            59           1274           3019           6561
-------------------------------------------------------------------------------

There are other minor parts of the repository which have been added to support the gpu-ISTL framework, such as adjustments in:

• GraphColoring.hpp
• FlexibleSolver_impl.hpp
• PreconditionerFactory_impl.hpp
• CMakeLists.txt
• CMakeLists_files.txt

Regarding the length of the article; I think the section demonstrating that the current framework can yield significant speedup over conventional processors when running OPM Flow makes an important point, so I would like to keep it in some shape or form. We can maybe move Appendix A, and maybe into the repository somewhere, or just remove it? Additionally we can replace the section about the build system supporting both AMD and Nvidia with a few sentences in another section. These two changes would probably remove a few hundred words. Would that be properly in line with the typical JOSS format?

Edit: I forgot to mention that we also have a folder with tests specifically for gpu-ISTL functionality located in tests/gpuistl, all files in that folder (a few thousand lines) are also a part of this submission.

[danielskatz]

Thanks. And your suggestions for reducing the paper's word count seem reasonable.

Please feel free to make changes, then use the command @editorialbot generate pdf to make a new PDF.

editorialbot commands need to be the first entry in a new comment.

You can also use editorialbot check repository to check a bunch of things, including the word count for the paper.

[multitalentloes]

@editorialbot check repository

[editorialbot]

Software report:

github.com/AlDanial/cloc v 1.90  T=0.71 s (1664.8 files/s, 393679.4 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
C/C++ Header                   617          24852          39348          92073
C++                            437          16384          12267          73855
SVG                              5              4             23           4704
CMake                           16            483            285           4298
TeX                             27            617            130           4024
CUDA                            12            375            286           2202
Bourne Shell                    25            261            119           1377
OpenCL                          23            124             97           1011
XML                              1              0              0            520
Python                           9             53            112            344
Markdown                         6            100              0            339
JSON                             3              0              0            124
YAML                             2              4              0             40
MATLAB                           1              8              0             20
Dockerfile                       1              2              3             13
awk                              1              1             24             10
TOML                             1              0              0              7
INI                              1              0              0              3
-------------------------------------------------------------------------------
SUM:                          1188          43268          52694         184964
-------------------------------------------------------------------------------

Commit count by author:

  3007	Arne Morten Kvarving
  2216	Atgeirr Flø Rasmussen
  1444	Andreas Lauser
  1255	Kai Bao
  1206	Tor Harald Sandve
  1024	Markus Blatt
   894	Bård Skaflestad
   796	Joakim Hove
   320	Liu Ming
   226	Robert Kloefkorn
   185	Roland Kaufmann
   184	Kjetil Olsen Lye
   176	Xavier Raynaud
   174	hnil
   170	Tong Dong Qiu
   146	babrodtk
   143	Håkon Hægland
   103	Tobias Meyer Andersen
   100	Vegard Kippe
    89	Robert K
    82	Jørgen Kvalsvik
    74	osae
    73	Paul Egberts
    69	T.D. (Tongdong) Qiu
    67	Alf Birger Rustad
    67	Bernd Flemisch
    63	Stein Krogstad
    52	Lisa Julia Nebel
    45	Kristian Flikka
    36	Antonella Ritorto
    36	goncalvesmachadoc
    34	OPMUSER
    34	Pål Grønås Drange
    33	Benjamin Faigle
    31	Steinar Foss
    30	Philipp Nuske
    30	Razvan Nane
    29	Markus Wolff
    28	Cintia Goncalves Machado
    28	Halvor Møll Nilsen
    28	jakobtorben
    26	Klaus Mosthaf
    26	kel85uk
    25	Melanie Darcis
    24	Jose Eduardo Bueno
    24	Ove Sævareid
    24	Svenn Tveit
    23	David Landa Marban
    23	Rohith Nair
    22	Elyes Ahmed
    20	Torbjørn Skille
    19	André R. Brodtkorb
    19	David Werner
    19	Franz G. Fuchs
    19	Paul
    19	chflo
    17	Christoph Grueninger
    17	tqiu
    16	josh bowden
    12	Josh Bowden
    11	Alf B. Rustad
    10	Halvor M. Nilsen
    10	Jostein Alvestad
    10	Jostein R. Natvig
    10	andrthu
     9	Kjell W. Kongsvik
     8	Katherina Baber
     7	jilocode
     6	Andrea T. Lonn
     6	Michael Sargado
     6	Trine Mykkeltvedt
     6	WesselDeZeeuw
     5	Edvin Brudevoll
     5	Franz Georg Fuchs
     4	Anders Matheson
     4	Erik Hide Sæternes
     4	Fredrik Gundersen
     4	Júlio Hoffimann
     4	Nicolas Schwenck
     4	Peter Verveer
     4	Trine S. Mykkeltvedt
     4	dependabot[bot]
     3	Alexander Kissinger
     3	Jens Olav Nygaard
     3	Justin Chang
     3	Jørgen Kvalsvk
     3	Karin Erbertseder
     3	rube051
     2	Andreas Thune
     2	Erik Hide Saeternes
     2	Giacomo Marchiori
     2	Joshua Bowden
     2	Kristian Hole-Drabløs
     2	Lars Petter Øren Hauge
     2	Tom Hogervorst
     2	Tor Harald Sanve
     2	Trine S Mykkeltvedt
     2	daavid00
     2	locture
     1	Andrea Tronstad Lønn (ST MSU DYN)
     1	Baurzhan Muftakhidinov
     1	Bjørn Spjelkavik
     1	Eduardo Bueno
     1	Georg Zitzlsberger
     1	Holger Class
     1	Jakob Torben
     1	Jean-Paul Balabanian
     1	Jochen Fritz
     1	Matthew Goodfield
     1	Timo Koch
     1	Viktor Szukitsch
     1	Vishal Jambhekar
     1	WesselZ
     1	Zeeuw
     1	kristinf
     1	nairr
     1	paean
     1	steink

[editorialbot]

Paper file info:

📄 Wordcount for paper.md is 1228

✅ The paper includes a Statement of need section

[editorialbot]

License info:

🟡 License found: GNU General Public License v3.0 (Check here for OSI approval)

[multitalentloes]

Run configurations for the case study have been moved to a separate .md file living next to the paper in the branch for the paper. The paragraph on the build system has also been updated, is the current length acceptable?

[danielskatz]

Yes, thanks.

But we're currently managing a large backlog of submissions and the editors most appropriate for your area is already rather busy. For now, we will need to waitlist this paper and process it as the queue reduces. Thanks for your patience!

[danielskatz]

👋 @prashjha - given that another submission you have is almost done, would you be willing to edit this one?

[danielskatz]

@editorialbot invite @prashjha as editor

[editorialbot]

Invitation to edit this submission sent!

[prashjha]

@danielskatz, happy to edit this!

[prashjha]

@editorialbot assign me as editor

[editorialbot]

Assigned! @prashjha is now the editor

[multitalentloes]

Thank you @prashjha for accepting to be the editor of this submission, we are looking forward to the review process.

[multitalentloes]

What is the next stage of the pre-review @prashjha, finding reviewers?

[danielskatz]

👋 @prashjha - can you please get this one started?

[prashjha]

@danielskatz will do!

[prashjha]

@editorialbot generate pdf

[editorialbot]

👉📄 Download article proof 📄 View article proof on GitHub 📄 👈

[prashjha]

@editorialbot help

[editorialbot]

Hello @prashjha, here are the things you can ask me to do:

# List all available commands
@editorialbot commands

# Add to this issue's reviewers list
@editorialbot add @username as reviewer

# Remove from this issue's reviewers list
@editorialbot remove @username from reviewers

# Get a list of all editors's GitHub handles
@editorialbot list editors

# Assign a user as the editor of this submission
@editorialbot assign @username as editor

# Remove the editor assigned to this submission
@editorialbot remove editor

# Remind an author, a reviewer or the editor to return to a review after a 
# certain period of time (supported units days and weeks)
@editorialbot remind @reviewer in 2 weeks

# Adds a checklist for the reviewer using this command
@editorialbot generate my checklist

# Set a value for version
@editorialbot set v1.0.0 as version

# Set a value for branch
@editorialbot set joss-paper as branch

# Set a value for repository
@editorialbot set https://github.com/organization/repo as repository

# Set a value for the archive DOI
@editorialbot set 10.5281/zenodo.6861996 as archive

# Mention the EiCs for the correct track
@editorialbot ping track-eic

# Run checks and provide information on the repository and the paper file
@editorialbot check repository

# Check the references of the paper for missing DOIs
@editorialbot check references

# Generates the pdf paper
@editorialbot generate pdf

# Recommends the submission for acceptance
@editorialbot recommend-accept

# Generates a LaTeX preprint file
@editorialbot generate preprint

# Flag submission with questionable scope
@editorialbot query scope

# Get a link to the complete list of reviewers
@editorialbot list reviewers

# Creates a post-review checklist with editor and authors tasks
@editorialbot create post-review checklist

# Open the review issue
@editorialbot start review

[editorialbot]

Five most similar historical JOSS papers:

iharm3D: Vectorized General Relativistic Magnetohydrodynamics
Submitting author: @bprather
Handling editor: @eloisabentivegna (Retired)
Reviewers: @bgiacoma, @cpalenzuela
Similarity score: 0.6911

ACHR.cu: GPU-accelerated sampling of metabolic networks
Submitting author: @marouenbg
Handling editor: @lpantano (Active)
Reviewers: @wmegchel, @prasunanand
Similarity score: 0.6813

vSmartMOM.jl: an Open-Source Julia Package for Atmospheric Radiative Transfer and Remote Sensing Tools
Submitting author: @RupeshJey
Handling editor: @pdebuyl (Active)
Reviewers: @jimmielin, @arjunsavel
Similarity score: 0.6766

Ginkgo: A high performance numerical linear algebra library
Submitting author: @pratikvn
Handling editor: @mjsottile (Retired)
Reviewers: @adam-m-jcbs, @zeroset
Similarity score: 0.6720

SPbLA: The Library of GPGPU-powered Sparse Boolean Linear Algebra Operations
Submitting author: @EgorOrachyov
Handling editor: @Nikoleta-v3 (Active)
Reviewers: @mlxd, @bencardoen
Similarity score: 0.6664

⚠️ Note to editors: If these papers look like they might be a good match, click through to the review issue for that paper and invite one or more of the authors before considering asking the reviewers of these papers to review again for JOSS.

[prashjha]

Hello @bprather, @marouenbg, @RupeshJey, @pratikvn, and @EgorOrachyov, I am looking for two reviewers for this JOSS submission. Please let me know if you will be interested in reviewing this submission. If you have questions, feel free to ping me here.

You can access the pdf from this link and the corresponding repository is here.

[marouenbg]

Apologies @prashjha, it is not directly linked to my area of research and so I won't be able to evaluate the use case effectively. Thanks!

[pratikvn]

@prashjha, I am interested in reviewing. I am currently a bit busy with other commitments, so it might take more time than usual for me to finish my review. If that is okay, then you can count me in.

[bprather]

@prashjha I'm afraid this is pretty far from my expertise as well, sorry!

[prashjha]

@marouenbg and @bprather thank you for letting me know.

[prashjha]

@pratikvn, thank you for your positive response. I am adding you as a reviewer; hopefully, it will work out fine.

[prashjha]

@editorialbot add @pratikvn as reviewer

[editorialbot]

@pratikvn added to the reviewers list!

[prashjha]

Howdy @yhtang, @berenger-eu, @kris-rowe, @leios, I am looking for a reviewer for this JOSS submission. Please let me know if you will be interested in reviewing this submission. If you have questions, feel free to ping me here.

You can access the pdf from this link and the corresponding repository is here.

[berenger-eu]

Hello @prashjha yes I can be a reviewer.

[prashjha]

Hi @berenger-eu, great. Thank you for volunteering!

[prashjha]

@editorialbot add @berenger-eu as reviewer

[editorialbot]

@berenger-eu added to the reviewers list!

[prashjha]

Hello @yhtang, @kris-rowe, @leios, @RupeshJey, and @EgorOrachyov, we have two reviewers. Please ignore my review invitation.

[prashjha]

@multitalentloes, we have two reviewers, so I will start the review. It will take place in a new GitHub issue.

[prashjha]

@editorialbot start review

[editorialbot]

OK, I've started the review over in #7740.