Skip to content

Commit

Permalink
Add cert-blocker status based on coverage guide
Browse files Browse the repository at this point in the history
The "Internet of Things Certified Hardware Coverage for Ubuntu Core 22 /
Ubuntu 22.04" document specifies what jobs should be considered as
blockers.

For IoT devices, the after-suspend jobs are not considered blockers, so
they have been left out.

Fix CER-2593
  • Loading branch information
pieqq committed Apr 24, 2024
1 parent 9318a5c commit dad92ea
Show file tree
Hide file tree
Showing 15 changed files with 61 additions and 57 deletions.
6 changes: 3 additions & 3 deletions providers/base/units/audio/test-plan.pxu
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,7 @@ unit: test plan
_name: Manual audio tests
_description: Manual audio tests for Snappy Ubuntu Core devices
include:
audio/alsa-playback
audio/alsa-playback certification-status=blocker

id: audio-pa-manual
unit: test plan
Expand All @@ -126,7 +126,7 @@ _description: Automated audio tests for Snappy Ubuntu Core devices
include:
audio/detect-playback-devices
audio/detect-capture-devices
audio/alsa-loopback-automated
audio/alsa-loopback-automated certification-status=blocker

id: after-suspend-audio-full
unit: test plan
Expand All @@ -142,7 +142,7 @@ unit: test plan
_name: Manual audio tests (after suspend)
_description: Manual audio tests for Snappy Ubuntu Core devices
include:
after-suspend-audio/alsa-playback
after-suspend-audio/alsa-playback certification-status=blocker

id: after-suspend-audio-pa-manual
unit: test plan
Expand Down
6 changes: 3 additions & 3 deletions providers/base/units/bluetooth/test-plan.pxu
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ _name: Manual Bluetooth tests
_description: Manual QA tests for Bluetooth
estimated_duration: 5m
include:
bluetooth/keyboard-manual
bluetooth/keyboard-manual certification-status=blocker

id: bluez-automated
unit: test plan
Expand All @@ -77,7 +77,7 @@ include:
# bluetooth/bluez-internal-mgmt-tests_.*
bluetooth/bluez-internal-uc-tests_.*
bluetooth/bluez-internal-bnep-tests_.*
bluetooth4/beacon_eddystone_url_.*
bluetooth4/beacon_eddystone_url_.* certification-status=blocker
bluetooth/bluetooth_obex_send
bootstrap_include:
device
Expand Down Expand Up @@ -188,4 +188,4 @@ include:
after-suspend-bluetooth/bluetooth_obex_send certification-status=blocker
after-suspend-bluetooth4/beacon_eddystone_url_.* certification-status=blocker
bootstrap_include:
device
device
8 changes: 4 additions & 4 deletions providers/base/units/cpu/test-plan.pxu
Original file line number Diff line number Diff line change
Expand Up @@ -78,16 +78,16 @@ unit: test plan
_name: Automated CPU tests
_description: Automated CPU tests for Snappy Ubuntu Core devices
include:
cpu/scaling_test
cpu/scaling_test certification-status=blocker
cpu/scaling_test-log-attach
cpu/maxfreq_test
cpu/maxfreq_test certification-status=blocker
cpu/maxfreq_test-log-attach
cpu/clocktest
cpu/offlining_test
cpu/topology
cpu/topology certification-status=blocker
cpu/armhf_vfp_support_.*
cpu/arm64_vfp_support_.*
cpu/cstates
cpu/cstates certification-status=blocker
cpu/cstates_results.log

id: server-cpu
Expand Down
6 changes: 3 additions & 3 deletions providers/base/units/ethernet/test-plan.pxu
Original file line number Diff line number Diff line change
Expand Up @@ -69,8 +69,8 @@ _name: Automated ethernet tests
_description: Automated ethernet tests for Ubuntu Core devices
estimated_duration: 1m
include:
ethernet/detect
ethernet/ping_.*
ethernet/detect certification-status=blocker
ethernet/ping_.* certification-status=blocker
bootstrap_include:
device

Expand Down Expand Up @@ -145,4 +145,4 @@ include:
ethernet/multi_iperf3_nic_underspeed_device.* certification-status=non-blocker
bootstrap_include:
device
executable
executable
4 changes: 2 additions & 2 deletions providers/base/units/i2c/test-plan.pxu
Original file line number Diff line number Diff line change
Expand Up @@ -18,5 +18,5 @@ unit: test plan
_name: Automated I²C tests
_description: Automated I²C tests for Ubuntu Core devices
include:
i2c/i2c-bus-detect
i2c/i2c-device-detect
i2c/i2c-bus-detect certification-status=blocker
i2c/i2c-device-detect certification-status=blocker
4 changes: 2 additions & 2 deletions providers/base/units/led/test-plan.pxu
Original file line number Diff line number Diff line change
Expand Up @@ -106,10 +106,10 @@ unit: test plan
_name: Manual LED tests for IoT
_description: Manual LED tests for IoT devices
include:
led/power
led/power certification-status=blocker
led/power-blink-suspend
led/bluetooth
led/serial
led/serial certification-status=blocker
led/fn
led/sysfs_led_brightness_on_.*
led/sysfs_led_brightness_off_.*
Expand Down
2 changes: 1 addition & 1 deletion providers/base/units/memory/test-plan.pxu
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ unit: test plan
_name: Automated memory tests
_description: Automated memory tests for Ubuntu Core devices
include:
memory/info
memory/info certification-status=blocker

id: server-memory
unit: test plan
Expand Down
6 changes: 3 additions & 3 deletions providers/base/units/monitor/test-plan.pxu
Original file line number Diff line number Diff line change
Expand Up @@ -357,11 +357,11 @@ unit: test plan
_name: Manual monitor tests
_description: Manual monitor tests for Snappy Ubuntu Core devices
include:
monitor/dvi
monitor/hdmi
monitor/dvi certification-status=blocker
monitor/hdmi certification-status=blocker
monitor/dvi-to-vga
monitor/hdmi-to-vga
monitor/displayport_hotplug
monitor/displayport_hotplug certification-status=blocker
monitor/vga

id: after-suspend-monitor-full
Expand Down
10 changes: 5 additions & 5 deletions providers/base/units/power-management/test-plan.pxu
Original file line number Diff line number Diff line change
Expand Up @@ -98,10 +98,10 @@ unit: test plan
_name: Automated power tests
_description: Automated power tests for Snappy Ubuntu Core devices
include:
power-management/warm-reboot
power-management/post-warm-reboot
power-management/cold-reboot
power-management/post-cold-reboot
power-management/warm-reboot certification-status=blocker
power-management/post-warm-reboot certification-status=blocker
power-management/cold-reboot certification-status=blocker
power-management/post-cold-reboot certification-status=blocker

id: power-manual
unit: test plan
Expand All @@ -118,4 +118,4 @@ _description: CPU package cstate tests for Ubuntu PC devices
include:
power-management/idle-screen-on-residency-check
power-management/cpu-low-power-idle
power-management/system-low-power-idle
power-management/system-low-power-idle
2 changes: 1 addition & 1 deletion providers/base/units/rtc/test-plan.pxu
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ unit: test plan
_name: Manual RTC tests
_description: Manual RTC tests for Snappy Ubuntu Core devices
include:
rtc/battery
rtc/battery certification-status=blocker

id: rtc-automated
unit: test plan
Expand Down
4 changes: 2 additions & 2 deletions providers/base/units/serial/test-plan.pxu
Original file line number Diff line number Diff line change
Expand Up @@ -12,14 +12,14 @@ unit: test plan
_name: Manual serial tests
_description: Manual serial tests for Snappy Ubuntu Core devices
include:
serial/rs232-console
serial/rs232-console certification-status=blocker

id: serial-automated
unit: test plan
_name: Automated serial tests
_description: Automated serial tests for Snappy Ubuntu Core devices
include:
serial/loopback-.*
serial/loopback-.* certification-status=blocker
bootstrap_include:
serial_ports_static

Expand Down
10 changes: 5 additions & 5 deletions providers/base/units/usb/test-plan.pxu
Original file line number Diff line number Diff line change
Expand Up @@ -220,10 +220,10 @@ unit: test plan
_name: Manual USB tests
_description: Manual USB tests for Ubuntu Core devices
include:
usb/hid
usb/insert
usb/storage-automated # depends on manual one, so not automated
usb/remove
usb/hid certification-status=blocker
usb/insert certification-status=blocker
usb/storage-automated certification-status=blocker # depends on manual one, so not automated
usb/remove certification-status=blocker

id: usb-automated
unit: test plan
Expand Down Expand Up @@ -385,4 +385,4 @@ _description: USB tests for Server Cert
include:
usb/detect
usb/storage-server certification-status=blocker
usb3/storage-server certification-status=blocker
usb3/storage-server certification-status=blocker
38 changes: 19 additions & 19 deletions providers/base/units/wireless/test-plan.pxu
Original file line number Diff line number Diff line change
Expand Up @@ -128,26 +128,26 @@ _description:
Automated connection tests for unencrypted or WPA-encrypted 802.11 bg, n, ac, ax
networks.
include:
wireless/detect
wireless/detect certification-status=blocker
wireless/wireless_scanning_.*
wireless/wireless_connection_open_ax_nm_.*
wireless/wireless_connection_open_ac_nm_.*
wireless/wireless_connection_open_bg_nm_.*
wireless/wireless_connection_open_n_nm_.*
wireless/wireless_connection_wpa_ax_nm_.*
wireless/wireless_connection_wpa3_ax_nm_.*
wireless/wireless_connection_wpa_ac_nm_.*
wireless/wireless_connection_wpa_bg_nm_.*
wireless/wireless_connection_wpa_n_nm_.*
wireless/wireless_connection_open_ax_np_.*
wireless/wireless_connection_open_ac_np_.*
wireless/wireless_connection_open_bg_np_.*
wireless/wireless_connection_open_n_np_.*
wireless/wireless_connection_wpa_ax_np_.*
wireless/wireless_connection_wpa3_ax_np_.*
wireless/wireless_connection_wpa_ac_np_.*
wireless/wireless_connection_wpa_bg_np_.*
wireless/wireless_connection_wpa_n_np_.*
wireless/wireless_connection_open_ax_nm_.* certification-status=blocker
wireless/wireless_connection_open_ac_nm_.* certification-status=blocker
wireless/wireless_connection_open_bg_nm_.* certification-status=blocker
wireless/wireless_connection_open_n_nm_.* certification-status=blocker
wireless/wireless_connection_wpa_ax_nm_.* certification-status=blocker
wireless/wireless_connection_wpa3_ax_nm_.* certification-status=blocker
wireless/wireless_connection_wpa_ac_nm_.* certification-status=blocker
wireless/wireless_connection_wpa_bg_nm_.* certification-status=blocker
wireless/wireless_connection_wpa_n_nm_.* certification-status=blocker
wireless/wireless_connection_open_ax_np_.* certification-status=blocker
wireless/wireless_connection_open_ac_np_.* certification-status=blocker
wireless/wireless_connection_open_bg_np_.* certification-status=blocker
wireless/wireless_connection_open_n_np_.* certification-status=blocker
wireless/wireless_connection_wpa_ax_np_.* certification-status=blocker
wireless/wireless_connection_wpa3_ax_np_.* certification-status=blocker
wireless/wireless_connection_wpa_ac_np_.* certification-status=blocker
wireless/wireless_connection_wpa_bg_np_.* certification-status=blocker
wireless/wireless_connection_wpa_n_np_.* certification-status=blocker
wireless/check_iwlwifi_microcode_crash_.*
bootstrap_include:
device
Expand Down
6 changes: 3 additions & 3 deletions providers/base/units/wwan/test-plan.pxu
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@ _name: Automated wwan tests
_description: Automated wwan tests for Snappy Ubuntu Core devices
include:
# Note these tests require snap calling snap support
wwan/detect
wwan/gsm-connection-.*-auto
wwan/detect certification-status=blocker
wwan/gsm-connection-.*-auto certification-status=blocker
wwan/check-sim-present-.*-auto
bootstrap_include:
wwan_resource
Expand Down Expand Up @@ -58,4 +58,4 @@ include:
after-suspend-wwan/detect-manual
after-suspend-wwan/scan-networks-manual
after-suspend-wwan/check-sim-present-manual
after-suspend-wwan/gsm-connection-interrupted-manual
after-suspend-wwan/gsm-connection-interrupted-manual
6 changes: 5 additions & 1 deletion providers/tpm2/units/test-plan.pxu
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,11 @@ _description:
Clevis encryption tests
estimated_duration: 1m
include:
clevis.*
clevis-encrypt-tpm2/precheck certification-status=blocker
clevis-encrypt-tpm2/detect-rsa-capabilities certification-status=blocker
clevis-encrypt-tpm2/rsa certification-status=blocker
clevis-encrypt-tpm2/detect-ecc-capabilities certification-status=blocker
clevis-encrypt-tpm2/ecc certification-status=blocker
mandatory_include:
com.canonical.plainbox::manifest

Expand Down

0 comments on commit dad92ea

Please sign in to comment.