From 2ddd9dc93ba34d5f207bb982d9283432e5653a46 Mon Sep 17 00:00:00 2001 From: dahlo Date: Tue, 29 Oct 2024 13:02:28 +0100 Subject: [PATCH] Renamed quarto variables to groupable prefixes in the name --- _quarto.yml | 13 ++-- topics/linux/lab_linux_intro.qmd | 107 ++++++++++++++++++++----------- 2 files changed, 79 insertions(+), 41 deletions(-) diff --git a/_quarto.yml b/_quarto.yml index 2a3fa5f4..17bf64c8 100644 --- a/_quarto.yml +++ b/_quarto.yml @@ -113,10 +113,15 @@ filters: - assets/custom.lua - reveal-logo -# used site-wide wherever project ids are used -project_id: "edu24.uppmax" -backup_project_id: "naiss2024-22-1375" -workspace_path: "~/ngsintro" +# used site-wide +# project id on active cluster (dardel) +id_project: "edu24.uppmax" +# project id on fallback cluster (rackham) +id_project_backup: "naiss2024-22-1375" +# path to course working directory +path_workspace: "~/ngsintro" +# path to course resources on cluster +path_resources: "/sw/courses/ngsintro" # location options are linkoping, lund, umea, uppsala or online. For rendering the info page. # one or more separated by commas or semicolon. online doesn't display any location info. diff --git a/topics/linux/lab_linux_intro.qmd b/topics/linux/lab_linux_intro.qmd index 6863dfd3..1ddf1f21 100644 --- a/topics/linux/lab_linux_intro.qmd +++ b/topics/linux/lab_linux_intro.qmd @@ -8,10 +8,11 @@ format: html ```{r,eval=TRUE,include=FALSE} library(yaml) library(here) -project_id <- yaml::read_yaml(here("_quarto.yml"))$project_id -workspace_path <- yaml::read_yaml(here("_quarto.yml"))$workspace_path -site_url <- yaml::read_yaml(here("_quarto.yml"))$website$`site-url` -output_dir <- yaml::read_yaml(here("_quarto.yml"))$project$`output-dir` +id_project <- yaml::read_yaml(here("_quarto.yml"))$id_project +path_resources <- yaml::read_yaml(here("_quarto.yml"))$path_resources +path_workspace <- yaml::read_yaml(here("_quarto.yml"))$path_workspace +site_url <- yaml::read_yaml(here("_quarto.yml"))$website$`site-url` +output_dir <- yaml::read_yaml(here("_quarto.yml"))$project$`output-dir` ``` ::: {.callout-note} @@ -27,7 +28,7 @@ The first step of this lab is to open a ssh connection to PDC. Please refer to [ Usually you would do most of the work in this lab directly on one of the login nodes at PDC, but we have arranged for you to have one core each for better performance. This was covered briefly in the lecture notes. ```bash -salloc -A `r project_id` -t 07:00:00 -p shared -n 4 +salloc -A `r id_project` -t 07:00:00 -p shared -n 4 ``` check which node you got (replace `username` with your PDC username) @@ -39,10 +40,10 @@ squeue -u username should look something like this ```bash -dahlo@login1 ~ $ squeue -u dahlo +user@login1 ~ $ squeue -u user JOBID PARTITION NAME USER ST TIME NODES NODELIST(REASON) - 5583899 shared interact dahlo R 2:22 1 nid001009 -dahlo@login1 ~ $ + 5583899 shared interact user R 2:22 1 nid001009 +user@login1 ~ $ ``` where `nid001009` is the name of the node I got (yours will probably be different). @@ -56,7 +57,17 @@ ssh -Y nid001009 It is good to know how to move around in the file system. I'm sure you all have experienced this using a graphical user interface (**GUI**) before, Windows Explorer in Windows and Finder in OSX. Using the command line can be confusing at first, but the more your do it, the easier it gets. -When you connect to PDC, you will start out in your home folder. The absolute path to your home folder is `/cfs/klemming/home//`, e.g. `/cfs/klemming/home/u/username` or `/cfs/klemming/home/d/dahlo` +When you connect to PDC, you will start out in your home folder. The absolute path to your home folder is + +```bash +# structure +/cfs/klemming/home// + +# e.g. +/cfs/klemming/home/u/username +# or +/cfs/klemming/home/a/anderson +``` Start with looking at what you have in your home folder. The command for this `ls`, and it stand for **L**i**S**t (list). @@ -67,34 +78,43 @@ ls -l This is how my home folder looks like, and yours should look somewhat similar: ``` -[12:56:32] dahlo@login1 ~ $ ls -l +[12:56:32] user@login1 ~ $ ls -l total 4384 -drwxr-xr-x 3 dahlo b2014068 2048 Apr 27 2017 archive --rw-rw-r-- 1 dahlo dahlo 49042 Sep 23 2016 bad.png -drwxr-xr-x 2 dahlo dahlo 2048 Mar 18 2016 data --rw-rw-r-- 1 dahlo dahlo 60944 Sep 23 2016 good.png -drwxr-xr-x 4 dahlo b2014209 2048 Oct 30 2014 igv -drwxrwxr-x 5 dahlo dahlo 2048 Sep 20 2016 ngsintro -drwx--S--- 2 dahlo dahlo 2048 May 4 2010 private -drwxr-xr-x 26 dahlo dahlo 4096 May 18 10:43 scripts -drwxrwxr-x 5 dahlo dahlo 2201600 May 14 14:02 work -[12:57:36] dahlo@login1 ~ $ +drwxr-xr-x 3 user b2014068 2048 Apr 27 2017 archive +-rw-rw-r-- 1 user user 49042 Sep 23 2016 bad.png +drwxr-xr-x 2 user user 2048 Mar 18 2016 data +-rw-rw-r-- 1 user user 60944 Sep 23 2016 good.png +drwxr-xr-x 4 user b2014209 2048 Oct 30 2014 igv +drwxrwxr-x 5 user user 2048 Sep 20 2016 ngsintro +drwx--S--- 2 user user 2048 May 4 2010 private +drwxr-xr-x 26 user user 4096 May 18 10:43 scripts +drwxrwxr-x 5 user user 2201600 May 14 14:02 work +[12:57:36] user@login1 ~ $ ``` Usually when you are running analysis for real, you would not keep the data or analysis result files in your own home folder. You would store all files belonging to a project in a project folder that is shared among all the project members. Since this is a course project, we have not gotten a project folder so we will be storing all the files in a subfolder in our own home folder. We start off with creating this folder, our workspace, so that we have a place to organize our course files. ```bash # create the course folder -mkdir `r workspace_path` +mkdir `r path_workspace` ``` As seen in the lecture, the command for moving around is `cd`. The command stands for **C**hange **D**irectory and does exactly that. It is the equivalent of double clicking a folder in a GUI. -To enter the course folder, simply type ``r paste0("cd ", basename(workspace_path))`` +To enter the course folder, simply type +```bash +cd `r basename(path_workspace)` +``` -We can easily see that this is a relative path, since it does not start with a `/`. That means that this command will only work when you are standing in your home folder. If you are standing somewhere else and say that you want to enter a folder named ``r basename(workspace_path)``, the computer will tell you that there is no folder named ``r basename(workspace_path)`` where you are located at the moment. +We can easily see that this is a relative path, since it does not start with a `/`. That means that this command will only work when you are standing in your home folder. If you are standing somewhere else and say that you want to enter a folder named ``r basename(path_workspace)``, the computer will tell you that there is no folder named ``r basename(path_workspace)`` where you are located at the moment. -The absolute path to the workspace folder would be ``r workspace_path`` , or ``r paste0("/cfs/klemming/home/u/username/", basename(workspace_path))`` if you write out the whole path to your home folder. +The absolute path to the workspace folder would be + +``r path_workspace`` +or +``r paste0("/cfs/klemming/home/u/username/", basename(path_workspace))`` + +if you write out the whole path to your home folder. It is the exact same thing as if you are using a GUI. If you are standing on your desktop, you can double click a folder which is located on your desktop. But if you are standing in another folder, you can't double click on that same folder, because it is just not located there. You have to move to your desktop first, and then double click it. @@ -118,13 +138,22 @@ The alias will now be available in all new terminals you open. Now we have practised moving around and looking at what we have in folders. The next step will show you how to do the same thing, but without the moving around part. -If we want to look at what we have in our home folder, while standing in the course's workspace folder, we type `ll /cfs/klemming/home///` and remember to substitute `` with the first letter of your username, and `` with your own user name. +If we want to look at what we have in our home folder, while standing in the course's workspace folder, we type + +```bash +ll /cfs/klemming/home/// +``` +and remember to substitute `` with the first letter of your username, and `` with your own user name. ```bash -ll /cfs/klemming/u/username +ll /cfs/klemming/home/u/username/ ``` -Since most programmers are lazy (efficient), there is a shortcut to your home folder so that you don't have to write it all the time. If you write `~/` it means the same as if you would write `/cfs/klemming/home/u/username/` +Since most programmers are lazy (efficient), there is a shortcut to your home folder so that you don't have to write it all the time. If you write `~/` it means the same as if you would write + +```bash +/cfs/klemming/home/u/username/ +``` Try using it with `ls`: @@ -134,25 +163,29 @@ ll ~/ ## Copy lab files -Now you will need some files. The files are located in the folder `/sw/courses/ngsintro/linux/linux_tutorial` +Now you will need some files. The files are located in the folder + +```bash +`r path_resources`/linux/linux_tutorial +``` or they can be downloaded if you are not on UPPMAX at the moment, [files.tar.gz](assets/files.tar.gz) (instruction on how to download further down) For structures sake, first create a folder called `linux_tutorial` inside the workspace folder, where you can put all your files belonging to this lab. ```bash -mkdir `r workspace_path`/linux_tutorial +mkdir `r path_workspace`/linux_tutorial ``` Next, copy the lab files to this folder. -``` +```bash # syntax cp -r ``` ```bash -cp -r /sw/courses/ngsintro/linux/linux_tutorial/* `r workspace_path`/linux_tutorial +cp -r `r path_resources`/linux/linux_tutorial/* `r path_workspace`/linux_tutorial ``` `-r` denotes recursively, which means all the files including sub-folders of the source folder. Without it, only files directly in the source folder would be copied, **NOT** sub-folders and files in sub-folders. @@ -178,7 +211,7 @@ Go to the folder you just copied and see what is in it. {{< fa lightbulb >}} Remember to tab-complete to avoid typos and too much writing. ```bash -cd `r workspace_path`/linux_tutorial +cd `r path_workspace`/linux_tutorial ll ``` @@ -425,7 +458,7 @@ First, open the file with `nano`. # syntax nano -nano /sw/courses/ngsintro/linux/linux_additional-files/large_file +nano `r path_resources`/linux/linux_additional-files/large_file ``` {{< fa lightbulb >}} Press `Ctrl`+`X` to close it down, use arrows to scroll up/down). @@ -440,7 +473,7 @@ To view the first rows of the large file, use `head`. # syntax head -head /sw/courses/ngsintro/linux/linux_additional-files/large_file +head `r path_resources`/linux/linux_additional-files/large_file ``` Remember how to view an arbitrary number of first rows in a file? @@ -449,7 +482,7 @@ Remember how to view an arbitrary number of first rows in a file? # syntax head -n -head -n 23 /sw/courses/ngsintro/linux/linux_additional-files/large_file +head -n 23 `r path_resources`/linux/linux_additional-files/large_file ``` The same syntax for viewing the last rows of a file with tail: @@ -458,13 +491,13 @@ The same syntax for viewing the last rows of a file with tail: # syntax tail -tail /sw/courses/ngsintro/linux/linux_additional-files/large_file +tail `r path_resources`/linux/linux_additional-files/large_file # syntax tail -n -tail -n 23 /sw/courses/ngsintro/linux/linux_additional-files/large_file +tail -n 23 `r path_resources`/linux/linux_additional-files/large_file ``` ## Wildcards