Clarify documentation according to feedback (infra) (#1740)

* Clarify manage validate, jq on base, builtin categories and mk-venv typo * Clarify how to work on ubuntu core further Removed very outdated howto as it serves no purpose anymore * Remember the user to disable the checkbox agent * Clarify mixed namespacing * Fixed weird styling, typos and clarified a few phrases * Apply suggestions from code review Co-authored-by: Pierre Equoy <pierre.equoy@canonical.com> --------- Co-authored-by: Pierre Equoy <pierre.equoy@canonical.com>
canonical · Feb 28, 2025 · 53e0d01 · 53e0d01
1 parent 858bc40
commit 53e0d01
Show file tree

Hide file tree

Showing 15 changed files with 131 additions and 162 deletions.
diff --git a/docs/_images/checkbox-snappy-1-test-plan.png b/docs/_images/checkbox-snappy-1-test-plan.png
diff --git a/docs/_images/checkbox-snappy-2-resume-session.png b/docs/_images/checkbox-snappy-2-resume-session.png
diff --git a/docs/_images/checkbox-snappy-3-select-jobs.png b/docs/_images/checkbox-snappy-3-select-jobs.png
diff --git a/docs/_images/checkbox-snappy-4-user-interact-job.png b/docs/_images/checkbox-snappy-4-user-interact-job.png
diff --git a/docs/_images/checkbox-snappy-5-rerun-jobs.png b/docs/_images/checkbox-snappy-5-rerun-jobs.png
diff --git a/docs/_images/checkbox-snappy-6-test-results.png b/docs/_images/checkbox-snappy-6-test-results.png
diff --git a/docs/how-to/index.rst b/docs/how-to/index.rst
@@ -7,7 +7,6 @@ These how-to guides cover key operations and processes in Checkbox.
    :maxdepth: 1
    :glob:
 
-   testing-ubuntu-core
    custom-app
    nested-test-plan
    side-loading

diff --git a/docs/how-to/testing-ubuntu-core.rst b/docs/how-to/testing-ubuntu-core.rst
diff --git a/docs/reference/units/category.rst b/docs/reference/units/category.rst
@@ -1,3 +1,5 @@
+.. _category_unit:
+
 =============
 Category Unit
 =============

diff --git a/docs/tutorial/index.rst b/docs/tutorial/index.rst
@@ -46,7 +46,7 @@ Once you have completed the core tutorial, the extended optional tutorial
 sections can be followed in any order - they don’t depend on each other.
 
 .. toctree::
-   :maxdepth: 1
+  :maxdepth: 1
 
    writing-tests/test-case
    writing-tests/test-plan
diff --git a/docs/tutorial/using-checkbox/installing-checkbox.rst b/docs/tutorial/using-checkbox/installing-checkbox.rst
@@ -23,12 +23,27 @@ run the following::
   [...]
   checkbox (22.04/stable) X.Y.Z from Canonical Certification Team (ce-certification-qa) installed
 
+If you are on Ubuntu Core, then you can't install the classic frontend.
+You should use the following strict frontend instead::
+
+  $ sudo snap install checkbox --channel=uc22/stable --devmode
+  [...]
+  checkbox (22.04/stable) X.Y.Z from Canonical Certification Team (ce-certification-qa) installed
+
 .. note::
   There are multiple frontends as you may discover by typing ``snap info checkbox``.
   If you are unsure about what ``frontend`` you should use, consider
   reading this page: :ref:`ref_which_snap`, but for the scope of this tutorial the one
   installed in this snipped is enough.
 
+.. warning::
+   Checkbox will automatically start a agent service on your device when you
+   install it. To follow this tutorial you will need this service but you
+   should consider stopping it whenever you don't need it as it allows full
+   unauthenticated root level remote control of your machine! To disable it run
+   ``snap stop --disable checkbox``. To start it once you need it run
+   ``snap start checkbox``.
+
 Now that we have installed both we can launch Checkbox running:
 
 .. code-block:: none

diff --git a/docs/tutorial/using-checkbox/remote.rst b/docs/tutorial/using-checkbox/remote.rst
@@ -20,33 +20,29 @@ Run the following command:
 
 .. code-block:: none
 
-    systemctl status snap.checkbox.agent.service
+    snap services checkbox
 
 You should see something like this:
 
 .. code-block:: none
 
-    ● snap.checkbox.agent.service - Service for snap application checkbox.service
-         Loaded: loaded (/etc/systemd/system/snap.checkbox.service.service; enabled; vendor preset: enabled)
-         Active: active (running) since Fri 2023-07-21 13:38:48 CST; 1h 29min ago
-       Main PID: 1411 (python3)
-          Tasks: 1 (limit: 19014)
-         Memory: 69.7M
-            CPU: 2.537s
-         CGroup: /system.slice/snap.checkbox.agent.service
-                 └─1411 python3 /snap/checkbox22/current/bin/checkbox-cli run-agent
-
-    Jul 21 13:38:48 coltrane systemd[1]: Started Service for snap application checkbox.service.
-    (...)
+    Service         Startup   Current   Notes
+    checkbox.agent  disabled  inactive  -
 
 When you install Checkbox on a device, a Systemd service is started to turn
-this device into a Checkbox agent.
-
-For the sake of this tutorial, let's stop this service for the moment:
+the device into a Checkbox agent. We have instructed you to disable this
+service in :ref:`installing_checkbox` but in case you didn't do so, run the
+following:
 
 .. code-block:: none
 
-    sudo systemctl stop snap.checkbox.agent.service
+   snap stop --disable checkbox
+
+.. warning::
+   Remember that while the agent is running, the device is fully controllable
+   via the network at a root level without any authentication! This is the
+   purpose of Checkbox, but a bit too scary to keep this enabled on your
+   personal device.
 
 Now, open two terminal windows using ``Ctrl+Alt+T``. In the first one,
 start the Checkbox agent:

diff --git a/docs/tutorial/writing-tests/test-case.rst b/docs/tutorial/writing-tests/test-case.rst
@@ -1,4 +1,4 @@
-.. _test_case:
+.. _adv_test_case:
 
 =================
 Writing Test Jobs
@@ -15,7 +15,7 @@ ideal for prototyping. To provision Checkbox from source do the following:
 .. code-block:: shell
 
     # first install python3 and python3-venv
-    $ sudo apt install python3 python3-venv python3-pip
+    $ sudo apt install python3 python3-virtualenv python3-pip
     # clone the Checkbox repository
     $ git clone https://github.com/canonical/checkbox.git
     # call the mk-venv script with the location of your virtualenv
@@ -37,6 +37,12 @@ ideal for prototyping. To provision Checkbox from source do the following:
     Remember to activate the virtual environment! You can also create an alias
     in your ``~/.bashrc`` to enable it when you need it.
 
+.. note::
+  All of the commands in this tutorial are using the
+  ``com.canonical.certification`` namespace. If you want to continue the one you
+  have started before, remember to change the namespace
+  (i.e. ``2024.com.tutorial::tutorial``) in the commands as well!
+
 Creating a new provider
 =======================
 
@@ -45,7 +51,7 @@ Checkbox organizes and manages all jobs, test plans and other test units in vari
 Let's create a new Checkbox provider by using the Checkbox sub-command
 ``startprovider``.
 
-.. code-block:: shell
+.. code-block:: none
 
    (checkbox_venv) $ checkbox-cli startprovider 2024.com.tutorial:tutorial
 
@@ -105,6 +111,13 @@ Now to run our test we can use the ``run`` sub-command. Try the following:
      ☑ : A job that always passes
 
 
+.. important::
+   You should always run ``python3 manage.py validate`` before running your
+   tests. This ensures that your unit is valid and Checkbox will interpret it
+   correctly. When you don't do it, Checkbox may ignore some errors in your
+   unit and, for example, fail to load some jobs leaving you wondering why the
+   ``run`` command doesn't work!
+
 First concrete test example
 ===========================
 
@@ -151,12 +164,44 @@ like this:
     ==================================[ Results ]===================================
      ☑ : Test that the internet is reachable
 
+Similarly to ``summary`` and ``id``, consider also providing a ``category`` for
+your tests. It makes the output easier to read and clearer (you can see that we
+are getting an ``uncategorised`` category right now). Additionally it is used
+in the Test Selection screen to group your tests, so when you have many of
+them, it makes sifting through them that much easier. You can create your own
+category (See: :ref:`category_unit`), but consider using the built-in ones.
+
+To get a list of them, try the following:
+
+.. code-block:: none
+
+   (checkbox_venv) $ checkbox-cli list "category" | grep "com.canonical.plainbox"
+
 .. note::
-   Similarly to ``summary`` and ``id``, consider also providing a ``category``
-   for your tests. It makes the output easier to read and clearer (you can see
-   that we are getting an ``uncategorised`` category right now). Additionally
-   it is used in the Test Selection screen to group your tests, so when you
-   have many of them, it makes sifting through them that much easier.
+    We grep for ``com.canonical.plainbox`` categories because those are part of
+    Checkbox. If you use a category id that is not builtin, remember that doing
+    so adds a new dependency between your provider and the one that defines the
+    category unit you are using!
+
+We can use the ``com.canonical.plainbox::networking`` category for our tests by
+modifying the unit as follows:
+
+.. code-block:: none
+
+    id: network_available
+    flags: simple
+    _summary: Test that the internet is reachable
+    category_id: com.canonical.plainbox::networking
+    command:
+      ping -c 1 1.1.1.1
+
+As any other unit that is not defined in the same namespace you are using, when
+referring to it you have to use the full name, including the namespace!
+
+.. note::
+   We will omit the ``category_id`` from all units in this tutorial to make the
+   snippets shorter but you should always use it for your production unit.
+
 
 Dependencies
 ============
@@ -337,7 +382,8 @@ Create a new job with the following content:
           "\nlink_info_kind: " + .linkinfo.info_kind +
           "\nlink_type: " + .link_type + "\n"'
 
-We are using ``jq`` to parse the output of the ``ip`` command, which means we need to make sure ``jq`` is available. We need to declare this in
+We are using ``jq`` to parse the output of the ``ip`` command, which means we
+need to make sure ``jq`` is available. We need to declare this in
 the correct spot, otherwise this will not work in a reproducible manner. Let's add
 a packaging meta-data unit to our ``units/extended_tutorial.pxu`` file:
 
@@ -349,7 +395,10 @@ a packaging meta-data unit to our ``units/extended_tutorial.pxu`` file:
     Depends:
       jq
 
-If you now run the following command you will notice a validation error.
+If you have ``developed`` the other providers that Checkbox comes with, by
+running the following command you will notice a validation error. If you don't
+see this error, don't worry, it means you don't have the base provider
+``installed`` or ``developed`` yet.
 
 .. code-block:: none