Merge pull request #2632 from stfc/2116_practical_update

(Closes #2116) 'existing-code' practical update

changelog

@@ -144,6 +144,8 @@
 	51) PR #2575 towards #2201. Allow FileContainer to have symbols of their
 	inner subroutines. It also solves an issue with missing PURE, ELEMENTAL, ...
 
+	52) PR #2632 for #2116. Update the code-transformation tutorial.
+
 release 2.5.0 14th of February 2024
 
 	1) PR #2199 for #2189. Fix bugs with missing maps in enter data

tutorial/practicals/README.md

@@ -1,8 +1,7 @@
 # PSyclone Hands-On Practicals
 
-Welcome to the PSyclone hands-on practicals. Unlike those parts of the
-tutorial that use Jupyter notebooks, the sections in the directories
-below this one work through using PSyclone in a 'normal' Linux environment.
+Welcome to the PSyclone hands-on practicals. The sections in the directories
+below this one work through using PSyclone in a 'normal' Linux-type environment.
 There are two sections to the practicals:
 
 1. Transforming [existing code](nemo/README.md)
@@ -29,54 +28,15 @@ need access to a terminal (command-line) on a computer running a Linux
 recommend using the Windows Subsystem for Linux (WSL). Alternatively,
 you can run a Linux virtual machine (e.g. using VirtualBox or
 VMWare). Whichever of these you use, we recommend that you install
-Ubuntu 20.04 LTS.
+the latest LTS release of Ubuntu.
 
-* Please see
-  https://docs.microsoft.com/en-us/windows/wsl/install-win10 for
-  instructions on getting started with WSL. Note that WSL requires
-  Windows 10 version 1607 (the Anniversary update) or better.
-
-   * In order to have multiple windows open simultaneously you will
-     need to install an X-server such as VcXsrv
-     (https://sourceforge.net/projects/vcxsrv/). Once you have that
-     running, set DISPLAY=localhost:0.0 in your linux terminal and you
-     will then be able to launch GUI applications.
+* Please see https://learn.microsoft.com/en-us/windows/wsl/install
+  for instructions on getting started with WSL.
 
 * Please see https://www.oracle.com/virtualization/technologies/vm/virtualbox.html
   for instructions on getting started with VirtualBox and the system requirements.
-  Here are the basic steps:
-
-   * Download the ubuntu 20.04.1 ISO from
-     [here](https://ubuntu.com/download/desktop);
-   * Install Virtual Box from
-     https://www.oracle.com/virtualization/technologies/vm/downloads/virtualbox-downloads.html
-     (using the Windows hosts link for Windows);
-   * Select new in the Virtual Box manager page;
-   * Choose a name for the VM and linux and ubuntu (64-bit) as the options;
-   * Click start;
-   * Find the ubuntu iso from the search box and add it, then follow the
-     install instructions;
-   * If your ubuntu screen does not resize run Devices -> Insert Guest CD.
-     After doing this and after reboot you should be able to resize the VM
-     screen. If not, try clicking on View -> Auto-resize guest display.
-   * In order to enable copy-n-paste between the host and virtual machines
-     please follow the instructions at
-     https://www.howtogeek.com/187535/how-to-copy-and-paste-between-a-virtualbox-host-machine-and-a-guest-machine/
-
-* To install VMWare:
-
-   * As above, download the ubuntu 20.04.01 ISO;
-   * Install VMware player from
-     https://www.vmware.com/uk/products/workstation-player/workstation-player-evaluation.html;
-   * You may be told to restart your system and re-run the installer;
-   * Confirm you are using it for non-commercial use and skip the update
-     to pro;
-   * Click on create a new VM, find the ubuntu ISO and install;
-   * If you get an error message beginning with "Your host does not meet 
-     minimum requirements to run VMware Player with Hyper-V or Device/Credential 
-     Guard enabled...." then it is probably best to install Virtual Box instead. 
-     If not, google how to fix the problem.
 
+* For VMWare see: https://knowledge.broadcom.com/external/article?articleNumber=309355
 
 In the instructions below we assume that you are using Ubuntu. If you
 are using some other Linux distribution you will have to change the
@@ -89,9 +49,7 @@ installed. Additionally, we recommend working with a Python virtual
 environment which virtualenvwrapper makes quite straightforward (see
 e.g. https://docs.python-guide.org/dev/virtualenvs/). Some of the
 commands in the remainder of this document assume a virtual
-environment so would need to be changed if one is not used (e.g. use
-the `--user` flag to any pip commands so as to install packages as a
-local user).
+environment so would need to be changed if one is not used.
 
 We recommend using pip to install Python packages. To get pip (for Python 3):
 
@@ -172,11 +130,12 @@ $ sudo apt install gfortran
 ```
 
 For the session on adding OpenACC to NEMO, you may wish to be able to
-compile the generated OpenACC code. This requires version 8 or higher
-of gfortran. Alternatively, the NVIDIA HPC SDK is also freely
-available (https://developer.nvidia.com/hpc-sdk). This has the
-advantage that the NVIDIA Fortran compiler is also capable of
-compiling OpenACC code to target multi-core CPUs as well as GPUs.
+compile the generated OpenACC code. This requires version 11 or higher
+of gfortran (compiled with nvpvtx support). Alternatively, the NVIDIA
+HPC SDK is also freely available
+(https://developer.nvidia.com/hpc-sdk). This has the advantage that
+the NVIDIA Fortran compiler is also capable of compiling OpenACC code
+to target multi-core CPUs as well as GPUs.
 
 LFRic code is routinely built with the Gnu Fortran compiler 6.1.0. and
 later, and it is proven to work with the latest GCC 10.2.0.

tutorial/practicals/nemo/1_nemo_psyir/Makefile

@@ -48,8 +48,9 @@ compile: transform
 run: compile
 	@echo "No run targets for example tutorial/1_nemo_psyir"
 
+# (output.f90 is created when following the tutorial instructions)
 clean:
-	rm -f *.o *.mod *~
+	rm -f output.f90 *.o *.mod *~
 
 allclean: clean
 	rm -f *.exe output.dat

tutorial/practicals/nemo/1_nemo_psyir/README.md

@@ -1,8 +1,8 @@
 # PSyclone for existing code - Tutorial 1 #
 
 Welcome to the first part of the tutorial on using PSyclone with existing
-Fortran code via its code-transformation capabilites. For this tutorial
-we will be using a standalone, single-source-file mini-app (tra_adv.F90)
+Fortran code via its code-transformation capabilities. For this tutorial
+we will be using a standalone, single-file mini-app (tra_adv.F90)
 based on a tracer-advection routine that has been extracted from the full
 source of the NEMO ocean model (www.nemo-ocean.eu). The original version of
 this mini-app was kindly provided by Silvia Mocavero of CMCC.
@@ -12,12 +12,10 @@ this mini-app was kindly provided by Silvia Mocavero of CMCC.
 The requirements for this section are as described in the practicals
 [README.md](../../README.md#Requirements).
 
-Check that PSyclone is installed and configured correctly by doing
-(assuming that your current working directory is the one containing
-this file):
+Check that PSyclone is installed and configured correctly by doing:
 
 ```bash
-    psyclone -h
+psyclone -h
 ```
 
 When examining the PSyIR (the intermediate representation that PSyclone
@@ -26,13 +24,35 @@ scripts, it may be useful to see the documentation of the various node
 types. The best way to do this is to use the [PSyclone Reference Guide
 ](https://psyclone-ref.readthedocs.io/en/latest/).
 
+## A note on importing PSyclone modules ##
+
+Throughout these tutorials you will be working with PSyclone transformation
+scripts written in Python. If you are adding functionality that requires
+class definitions not already imported into the script then you will have
+to add suitable import statements. e.g. to make the `Directive` node class
+available you would add:
+
+```python
+from psyclone.psyir.nodes import Directive
+```
+
+All PSyIR node definitions are within the `psyclone.psyir.nodes` module
+and all symbols and datatypes are within `psyclone.psyir.symbols`.
+
+Unfortunately, the various transformation classes are not yet as well
+organised. Many are now in the 'proper' location of
+`psyclone.psyir.transformations`, but some are still in
+`psyclone.transformations`. If in doubt, consult the
+[Reference Guide](https://psyclone-ref.readthedocs.io/en/latest/_static/html/annotated.html).
+
+
 ## 1. Processing NEMO Fortran code with PSyclone ##
 
 By default (i.e. without the additional arguments), the `psyclone` command
 parses the provided input Fortran file:
 
 ```bash
-    psyclone tra_adv.F90
+psyclone tra_adv.F90
 ```
 
 This command should result in PSyclone processing the supplied Fortran
@@ -57,7 +77,7 @@ to be written to a file instead of stdout. This is achieved with the
 `-o` flag.
 
 ```bash
-    psyclone tra_adv.F90 -o output.f90
+psyclone tra_adv.F90 -o output.f90
 ```
 
 will create a new file, `output.f90`, containing the generated Fortran
@@ -71,7 +91,7 @@ instruct PSyclone to limit the line lengths in the output Fortran via
 the `-l output` flag:
 
 ```bash
-    psyclone tra_adv.F90 -o output.f90 -l output
+psyclone tra_adv.F90 -o output.f90 -l output
 ```
 
 (Note that if we also wanted PSyclone to validate that the *incoming*
@@ -81,7 +101,7 @@ Then we can provide the generated file to a Fortran compiler. For example,
 if we use `gfortran`, we can do:
 
 ```bash
-    gfortran output.f90 -o tra_adv.exe
+gfortran output.f90 -o tra_adv.exe
 ```
 
 The mini-app picks-up the domain size and number of iterations from
@@ -91,18 +111,18 @@ using csh or tcsh. You can either cut-n-paste the commands into your
 shell or do:
 
 ```bash
-    source ../domain_setup.csh
+source ../domain_setup.sh
 ```
 
 Once the environment variables are set, you are ready to execute the
 mini-app:
 
 ```bash
-    ./tra_adv.exe 
-    Tracer-advection Mini-app:
-    Domain is  100x 100 grid points
-    Performing   10 iterations
-    Mini-app finished.
+./tra_adv.exe 
+Tracer-advection Mini-app:
+Domain is  100x 100 grid points
+Performing   10 iterations
+Mini-app finished.
 ```
 
 At this point we have succeeded in processing some Fortran code
@@ -120,11 +140,11 @@ with a transformation script, `view_trans.py`. This is done
 via the `-s` flag to PSyclone.
 
 ```bash
-    $ psyclone tra_adv.f90 -s view_trans.py -o /dev/null | less
+psyclone tra_adv.F90 -s view_trans.py -o /dev/null | less
 ```
 
 This should display a text representation of the tree of PSyIR nodes
-that represent the `tra_adv.f90` file:
+that represent the `tra_adv.F90` file:
 
 ```bash
 FileContainer[]
@@ -146,8 +166,7 @@ content of the program (the sequence of executable statements) as children.
 ## 3. Interpreting the PSyIR ##
 
 The basic structure and means of navigating the PSyIR are covered in the
-[PSyIR - Tree naviation documentation]
-(https://psyclone.readthedocs.io/en/latest/psyir.html#tree-navigation).
+[PSyIR - Tree navigation documentation](https://psyclone.readthedocs.io/en/latest/psyir.html#tree-navigation).
 In summary, all nodes in the PSyIR have `parent` and `children`
 properties and a `walk` method which may be used to find all nodes of
 a given type (or types) below the current node. Various sub-classes of
@@ -161,9 +180,9 @@ This is an important node type since it makes it possible
 for the PSyIR to represent arbitrary Fortran code without requiring
 that it be fully understood. Since PSyclone uses fparser2 to parse
 Fortran, a `CodeBlock` stores the nodes of the underlying fparser2
-parse tree that cannot be represented in the PSyIR. For more
-information on fparser2 see the
-[fparser User Guide](https://fparser.readthedocs.io/en/latest/).
+parse tree that cannot be represented in the PSyIR. PSyclone treats
+the CodeBlock as a blackbox that cannot be transformed, while still
+allowing the transformation of any code outside the CodeBlock.
 
 We can see from the fparser2 node type printed in the description of
 the `CodeBlock` that this particular node represents a Fortran `READ`
@@ -173,24 +192,25 @@ interesting from a performance point of view.
 To familiarise yourself with PSyIR navigation, you can:
 
 1. Modify the transformation script so that it breaks-out into the Python
-   debugger once it has obtained the PSyIR:
+   debugger, pdb, as soon as it is called (from PSyclone):
 
    ```python
     def trans(psyir):
        import pdb; pdb.set_trace()
    ```
 
    Re-running PSyclone:
-
-       $ psyclone -s ./view_trans.py tra_adv.F90
 
+   ```bash
+   psyclone -s ./view_trans.py tra_adv.F90
+   ```
    will now launch the Python debugger at that point:
 
        -> print(psyir.view())
        (Pdb) 
 
-   You can now interactively explore the Schedule and try the `walk`
-   method, e.g.:
+   You can now interactively explore the PSyIR (passed to the routine via the
+   `psyir` argument) and try the `walk` method, e.g.:
 
    ```python
     (Pdb) psyir
@@ -212,7 +232,7 @@ To familiarise yourself with PSyIR navigation, you can:
    by entering the `quit()` command.)
 
 2. Modify the transformation script so that it uses `walk` to search
-   for all of the CodeBlocks in the file and prints information
+   for all of CodeBlocks in the file and prints information
    about each of them. Work out which lines of Fortran in the
    mini-app each corresponds to. (All nodes have a `debug_string()`
    method to quickly print the node and its children.)
@@ -236,22 +256,33 @@ are [`Loop`](https://psyclone-ref.readthedocs.io/en/latest/_static/html/classpsy
    `Assignment` nodes:
 
    ```python
-   form psyclone.psyir.nodes import Assignment
+   from psyclone.psyir.nodes import Assignment
    assignments = sched.walk(Assignment)
    ```
 
-4. Use the `view()` method of one of these `Assignments` nodes to
+4. Use the `view()` method of one of these `Assignment` nodes to
    examine its children. Check that you are able to work out
    which Fortran assignment this corresponds to, e.g.:
 
    ```python
    print(assignments[0].view())
    ```
 
+   As mentioned earlier, the `debug_string` method can also be very
+   useful:
+
+   ```python
+   print(assignments[0].debug_string())
+   ```
+
 ## 4. Conclusion
 
 Congratulations, you have now completed this section of the tutorial.
 At this point you should be able to run PSyclone on a Fortran source
 file, use a transformation script to access the PSyIR of the code and
 be able to understand the structure of the PSyIR and how it relates to
 the original Fortran.
+
+You may now proceed to the [second section](../2_nemo_profiling/README.md)
+of the tutorial where you will use PSyclone to add profiling instrumentation
+to the mini-app.

tutorial/practicals/nemo/1_nemo_psyir/tra_adv.F90

@@ -7,7 +7,7 @@
 ! -----------------------------------------------------------------------------
 ! BSD 3-Clause License
 !
-! Modifications copyright (c) 2020, Science and Technology Facilities
+! Modifications copyright (c) 2020-2024, Science and Technology Facilities
 ! Council.
 ! All rights reserved.
 !
@@ -40,9 +40,7 @@
 
 PROGRAM tra_adv
    USE iso_c_binding, only: C_INT64_T
-   ! The below should be e.g. wp = KIND(1.0d0) but PSyclone does not support
-   ! the KIND intrinsic yet: TODO #585.
-   INTEGER, PARAMETER :: wp = 8
+   INTEGER, PARAMETER :: wp = KIND(1.0d0)
    REAL(wp), ALLOCATABLE, SAVE, DIMENSION(:,:,:,:) :: t3sn, t3ns, t3ew, t3we
    REAL(wp), ALLOCATABLE, SAVE, DIMENSION(:,:,:)   :: tsn 
    REAL(wp), ALLOCATABLE, SAVE, DIMENSION(:,:,:)   :: pun, pvn, pwn
@@ -67,11 +65,13 @@ PROGRAM tra_adv
    CALL get_environment_variable("IT", env)
    READ ( env, '(i10)' ) it
 
-   IF(jpi < 1) STOP 'Width of grid, JPI, must be > 0'
-   IF(jpj < 1) STOP 'Height of grid, JPJ, must be > 0'
-   IF(jpk < 1) STOP 'Depth of grid, JPK, must be > 0'
-   IF(it < 1) STOP  'Number of iterations, IT, must be > 0'
-
+   IF(jpi < 1 .or. jpj < 1 .or. jpk < 1 .or. it < 1)THEN
+      WRITE (*, "('Domain size (JPI * JPJ * JPK) and number of iterations (IT) " &
+           //"must be set ',/'using environment variables, e.g.:',/'   "         &
+           //"JPI=100 JPJ=100 JPK=30 IT=10 ./tra_adv.exe')")
+      STOP
+   END IF
+
    WRITE (*, "('Tracer-advection Mini-app:')")
    WRITE (*, "('Domain is ', I4, 'x', I4, ' grid points')") jpi, jpj
    WRITE (*, "('Performing ', I4, ' iterations')") it