Add LED manifest entries for interactive LED checks (BugFix) (#1385)

* Add manifest for interactive LED jobs

Instead of asking the tester to manually skip tests when a device does
not have a given LED (for instance no Microphone Mute LED), this commit
introduces manifest entries for all the related LED jobs.

This way, the tester defines what LEDs the device has before running the
tests, and tests that are not required due to a lack of specific LED are
automatically skipped.

* Modify LED manifest used for GPIO/sysfs controlled LEDs jobs

The generic "has_led_indicator" manifest entry, which originally came
from the CE-OEM provider for IoT testing, is renamed as
"has_led_gpio_sysfs" to better reflect the need for IoT projects.

It is grouped with other LED-related manifest entries by using the same
prompt.

* Add led/wireless to test plans and remove redundant LED jobs

led/wlan, led/wlan-disabled and led/bluetooth are all testing a subset
of what led/wireless is testing. This commit

- removes these jobs
- replaces their calls with led/wireless in related test plans

* Add manifest for suspend LED

Based on feedback from QA team, add a manifest for the suspend LED and
use it in the led/suspend job.

* Make sure device has a power LED before running the led/power-blink test

providers/base/units/camera/jobs.pxu

@@ -54,16 +54,18 @@ id: camera/led_{name}
 template-id: camera/led_name
 _summary: Webcam LED test for {product_slug}
 estimated_duration: 120.0
+imports: from com.canonical.plainbox import manifest
+requires:
+ manifest.has_led_camera == 'True'
 depends: camera/detect
 command:
   camera_test.py led -d /dev/{name}
-_description:
- PURPOSE:
+_purpose:
      This test will check that the {product_slug} camera LED works
- STEPS:
+_steps:
      1. Select Test to activate camera
      2. Camera LED should light for a few seconds
- VERIFICATION:
+_verification:
      Did the camera LED light?
 flags: also-after-suspend
 
@@ -235,4 +237,4 @@ _description:
   a gstreamer pipeline is used to capture an image of the display. An attempt
   to decode a QRcode in the image is then made and data compared against the
   random string.
-user: root
+user: root

providers/base/units/camera/manifest.pxu

@@ -7,3 +7,9 @@ unit: manifest entry
 id: has_camera
 _name: Camera/Capture Device
 value-type: bool
+
+unit: manifest entry
+id: has_led_camera
+_name: Camera
+_prompt: Does this machine have the following LED indicators?
+value-type: bool

providers/base/units/led/jobs.pxu

@@ -10,6 +10,9 @@ _verification:
 plugin: manual
 category_id: led
 estimated_duration: 15.0
+imports: from com.canonical.plainbox import manifest
+requires:
+ manifest.has_led_power == 'True'
 
 id: led/power-blink-suspend
 _summary: Power LED behavior when suspended
@@ -23,16 +26,21 @@ _verification:
 plugin: manual
 category_id: led
 estimated_duration: 180
+imports: from com.canonical.plainbox import manifest
+requires:
+ manifest.has_led_power == 'True'
 
 plugin: manual
 category_id: led
 id: led/suspend
 estimated_duration: 30.0
+imports: from com.canonical.plainbox import manifest
+requires:
+ manifest.has_led_suspend == 'True'
 depends: suspend/suspend_advanced_auto
 _purpose:
  Suspend LED verification.
 _steps:
- Skip this test if your system does not have a dedicated Suspend LED.
  1. The Suspend LED should blink or change color while the system is suspended
 _verification:
  Did the Suspend LED blink or change color while the system was suspended?
@@ -97,6 +105,9 @@ category_id: led
 id: led/numeric-keypad
 flags: also-after-suspend
 estimated_duration: 30.0
+imports: from com.canonical.plainbox import manifest
+requires:
+ manifest.has_led_numeric_keypad == 'True'
 command: keyboard_test.py
 _description:
 _purpose:
@@ -114,6 +125,9 @@ plugin: manual
 category_id: led
 id: led/caps-lock
 flags: also-after-suspend
+imports: from com.canonical.plainbox import manifest
+requires:
+ manifest.has_led_caps_lock == 'True'
 estimated_duration: 30.0
 _description:
 _purpose:
@@ -125,59 +139,12 @@ _verification:
  Did the Caps Lock LED light as expected?
 _summary: Verify the functionality of the Caps Lock LED by pressing the Caps Lock key.
 
-plugin: manual
-category_id: led
-id: led/wlan
-depends: keys/wireless
-estimated_duration: 60.0
-_purpose:
-    WLAN LED verification
-_steps:
-    1. During the keys/wireless test you should have observed the
-    wireless LED while turning wireless back on.
-    2. WLAN LED should light or change color when wireless is turned on
-_verification:
-    Did the WLAN LED turn on or change color as expected?
-_summary: Verify the functionality of the WLAN LED during the keys/wireless test.
-
-plugin: manual
-category_id: led
-id: led/wlan-disabled
-estimated_duration: 30.0
-depends: keys/wireless
-_purpose:
-     Validate that WLAN LED shuts off when disabled
-_steps:
-     1. During the keys/wireless test you should have observed the WLAN LED
-     while performing that test after turning wireless off.
-     2. WLAN LED should turn off or change color when wireless is turned off
-_verification:
-     Did the WLAN LED turn off or change color as expected?
-_summary: Verify the behavior of the WLAN LED when wireless is disabled.
-
-id: led/bluetooth
-_summary: Bluetooth LED behavior
-_purpose:
- Check bluetooth LED behavior is correct
-_steps:
- 1. Run following command to start bluetoothctl console:
-  sudo bluetoothctl
- ***Following steps are run in bluetoothctl console***
- 2. Power on the device:
-  power on
- 3. Power off the device:
-  power off
- 4. Quit bluetoothctl console
-  quit
-_verification:
- Bluetooth LED is turned on in step 2, and turned off in step 3.
-plugin: manual
-category_id: led
-estimated_duration: 180
-
 plugin: manual
 category_id: led
 id: led/touchpad
+imports: from com.canonical.plainbox import manifest
+requires:
+ manifest.has_led_touchpad == 'True'
 estimated_duration: 30.0
 _description:
 _purpose:
@@ -195,6 +162,9 @@ plugin: manual
 category_id: led
 id: led/wireless
 estimated_duration: 120.0
+imports: from com.canonical.plainbox import manifest
+requires:
+ manifest.has_led_wireless == 'True'
 _description:
 _purpose:
     Wireless (WLAN + Bluetooth) LED verification
@@ -214,11 +184,13 @@ category_id: led
 id: led/mute
 depends: keys/mute
 estimated_duration: 30.0
+imports: from com.canonical.plainbox import manifest
+requires:
+ manifest.has_led_audio_mute == 'True'
 _description:
 _purpose:
     Audio Mute LED verification.
 _steps:
-    Skip this test if your system does not have a special Audio Mute LED.
     1. Press the Mute key twice and observe the Audio LED to determine if it
     either turned off and on or changed colors.
 _verification:
@@ -230,12 +202,14 @@ plugin: manual
 category_id: led
 id: led/microphone-mute
 estimated_duration: 30.0
+imports: from com.canonical.plainbox import manifest
+requires:
+ manifest.has_led_microphone_mute == 'True'
 depends: keys/microphone-mute
 _description:
 _purpose:
     Microphone Mute LED verification.
 _steps:
-    Skip this test if your system does not have a special Microphone Mute LED.
     1. Press the Microphone Mute key twice and observe the Microphone LED to
     determine if it either turned off and on or changed colors.
 _verification:
@@ -256,16 +230,21 @@ command: echo {0..1024} | tee /dev/ttyS{0..100} &> /dev/null
 user: root
 category_id: led
 estimated_duration: 15
+imports: from com.canonical.plainbox import manifest
+requires:
+ manifest.has_led_serial == 'True'
 
 plugin: manual
 category_id: led
 id: led/fn
 estimated_duration: 30.0
+imports: from com.canonical.plainbox import manifest
+requires:
+ manifest.has_led_fn_lock == 'True'
 flags: also-after-suspend
 _purpose:
      This test will test the Fn key LED.
 _steps:
-     Skip this test if your system does not have an Fn Lock key.
      1. Press the Fn+Fn Lock key to activate/deactivate Fn keys locking.
      2. The Fn key LED should be switched on/off every time the key is pressed.
 _verification:
@@ -302,7 +281,7 @@ estimated_duration: 10
 plugin: user-interact-verify
 user: root
 imports: from com.canonical.plainbox import manifest
-requires: manifest.has_led_indicator == 'True'
+requires: manifest.has_led_gpio_sysfs == 'True'
 flags: also-after-suspend
 _summary: Check control of {name} LED indicator.
 _purpose:
@@ -355,7 +334,7 @@ user: root
 flags: also-after-suspend
 estimated_duration: 10
 imports: from com.canonical.plainbox import manifest
-requires: manifest.has_led_indicator == 'True'
+requires: manifest.has_led_gpio_sysfs == 'True'
 command:
     led_control_test.py -n {path} -d 10
 
@@ -373,7 +352,7 @@ user: root
 flags: also-after-suspend
 estimated_duration: 10
 imports: from com.canonical.plainbox import manifest
-requires: manifest.has_led_indicator == 'True'
+requires: manifest.has_led_gpio_sysfs == 'True'
 command:
     led_control_test.py -n {path} -d 10
 
@@ -400,7 +379,7 @@ estimated_duration: 10
 plugin: user-interact-verify
 user: root
 imports: from com.canonical.plainbox import manifest
-requires: manifest.has_led_indicator == 'True'
+requires: manifest.has_led_gpio_sysfs == 'True'
 flags: also-after-suspend
 _summary: Check control of {name} LED indicator.
 _purpose:

providers/base/units/led/manifest.pxu

@@ -1,4 +1,65 @@
 unit: manifest entry
-id: has_led_indicator
-_name: Does device supported LED indicator?
-value-type: bool
+id: has_led_gpio_sysfs
+_name: LEDs controlled via GPIO or sysfs
+_prompt: Does this machine have the following LED indicators?
+value-type: bool
+
+unit: manifest entry
+id: has_led_power
+_name: Power
+_prompt: Does this machine have the following LED indicators?
+value-type: bool
+
+unit: manifest entry
+id: has_led_suspend
+_name: Suspend
+_prompt: Does this machine have the following LED indicators?
+value-type: bool
+
+unit: manifest entry
+id: has_led_caps_lock
+_name: Caps lock
+_prompt: Does this machine have the following LED indicators?
+value-type: bool
+
+unit: manifest entry
+id: has_led_touchpad
+_name: Touchpad
+_prompt: Does this machine have the following LED indicators?
+value-type: bool
+
+unit: manifest entry
+id: has_led_wireless
+_name: Wireless
+_prompt: Does this machine have the following LED indicators?
+value-type: bool
+
+unit: manifest entry
+id: has_led_audio_mute
+_name: Audio mute
+_prompt: Does this machine have the following LED indicators?
+value-type: bool
+
+unit: manifest entry
+id: has_led_microphone_mute
+_name: Microphone mute
+_prompt: Does this machine have the following LED indicators?
+value-type: bool
+
+unit: manifest entry
+id: has_led_serial
+_name: Serial transfer
+_prompt: Does this machine have the following LED indicators?
+value-type: bool
+
+unit: manifest entry
+id: has_led_fn_lock
+_name: Function key lock (Fn lock)
+_prompt: Does this machine have the following LED indicators?
+value-type: bool
+
+unit: manifest entry
+id: has_led_numeric_keypad
+_name: Numeric keypad
+_prompt: Does this machine have the following LED indicators?
+value-type: bool

providers/base/units/led/test-plan.pxu

@@ -5,8 +5,6 @@ _description:
  LED tests
  Notes: - led/power-blink-suspend and led/suspend are used later in the
           power management testplan just after a number of suspend tests.
-        - led/wireless is redundant given that we have led/wlan and
-          led/bluetooth already.
 include:
 nested_part:
  com.canonical.certification::led-cert-manual
@@ -20,8 +18,6 @@ _description:
     LED tests
     Notes: - led/power-blink-suspend and led/suspend are used later in the
              power management testplan just after a number of suspend tests.
-           - led/wireless is redundant given that we have led/wlan and
-             led/bluetooth already. (Manual)
 include:
     camera/led_.*                      certification-status=blocker
     led/caps-lock                      certification-status=blocker
@@ -30,6 +26,7 @@ include:
     led/fn                             certification-status=non-blocker
     led/mute                           certification-status=blocker
     led/microphone-mute                certification-status=blocker
+    led/wireless                       certification-status=blocker
 bootstrap_include:
     device
 
@@ -41,8 +38,6 @@ _description:
     LED tests
     Notes: - led/power-blink-suspend and led/suspend are used later in the
              power management testplan just after a number of suspend tests.
-           - led/wireless is redundant given that we have led/wlan and
-             led/bluetooth already. (Automated)
 include:
 
 
@@ -58,6 +53,7 @@ include:
     after-suspend-led/mute                      certification-status=blocker
     after-suspend-led/microphone-mute           certification-status=blocker
     after-suspend-led/fn                        certification-status=non-blocker
+    after-suspend-led/wireless                  certification-status=blocker
 bootstrap_include:
     device
 
@@ -73,6 +69,7 @@ include:
     led/power                          certification-status=blocker
     led/mute                           certification-status=blocker
     led/microphone-mute                certification-status=blocker
+    led/wireless                       certification-status=blocker
 bootstrap_include:
     device
 
@@ -88,6 +85,7 @@ include:
     suspend/led_after_suspend/power             certification-status=blocker
     after-suspend-led/mute                      certification-status=blocker
     after-suspend-led/microphone-mute           certification-status=blocker
+    after-suspend-led/wireless                  certification-status=blocker
 bootstrap_include:
     device
 
@@ -108,7 +106,7 @@ _description: Manual LED tests for IoT devices
 include:
     led/power
     led/power-blink-suspend
-    led/bluetooth
+    led/wireless
     led/serial
     led/fn
     led/sysfs_led_brightness_on_.*