Console logging for statuschecker

.flake8

@@ -0,0 +1,9 @@
+[flake8]
+exclude =
+   __pycache__,
+   .venv,
+   .git,
+   dist,
+   build
+max-line-length = 100
+ignore = E203

.github/workflows/push.yaml

@@ -0,0 +1,38 @@
+name: Robot Framework statuschecker CI
+
+on: [push, pull_request]
+
+jobs:
+  build:
+
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        python-version: [3.7, 3.9,  pypy3]
+        rf-version: [4.0.3, 4.1.2]
+
+    steps:
+    - uses: actions/checkout@v2
+    - name: Set up Python ${{ matrix.python-version }} with Robot Framework ${{ matrix.rf-version }}
+      uses: actions/setup-python@v1
+      with:
+        python-version: ${{ matrix.python-version }}
+    - name: Install dependencies on Python
+      if: matrix.python-version != 'pypy3'
+      run: |
+        python -m pip install --upgrade pip
+        pip install -r requirements-dev.txt
+        pip install robotframework==${{ matrix.rf-version }}
+    - name: Install dependencies on pypy3
+      if: matrix.python-version == 'pypy3'
+      run: |
+        python -m pip install --upgrade pip
+        pip install robotframework==${{ matrix.rf-version }}
+    - name: Run lint
+      if: matrix.python-version != 'pypy3'
+      run: |
+        inv lint
+    - name: Run tests
+      run: |
+        python --version
+        python test/run.py

.gitignore

@@ -10,3 +10,4 @@ dist/
 build/
 *egg-info
 *~
+.venv

BUILD.rst

@@ -1,40 +1,173 @@
 Releasing StatusChecker
 =======================
 
-1. Execute tests using different Python implementations and versions.
-   See `<test/README.rst>`_ for instructions.
 
-2. Set ``$VERSION`` shell variable to ease copy-pasting further commands::
+Using Invoke
+~~~~~~~~~~~~
 
-      VERSION=x.y
+Invoke tasks are defined in the `<tasks.py>`_ file and they are executed from
+the command line like::
 
-3. Update ``__version__`` in `<robotstatuschecker.py>`_::
+    inv[oke] task [options]
 
-      sed -i "s/__version__ = .*/__version__ = '$VERSION'/" robotstatuschecker.py
-      git diff  # verify changes
-      git commit -m "Updated __version__ to $VERSION" robotstatuschecker.py && git push
+Run ``invoke`` without arguments for help. All tasks can be listed using
+``invoke --list`` and each task's usage with ``invoke --help task``.
 
-4. Tag::
+Preparation
+-----------
 
-      git tag -a $VERSION -m "Release $VERSION" && git push --tags
+1. Check that you are on the master branch and have nothing left to commit,
+   pull, or push::
 
-5. Create distribution::
+      git branch
+      git status
+      git pull --rebase
+      git push
 
-      python setup.py sdist register upload
+2. Clean up::
 
-6. Verify that `PyPI pages <https://pypi.python.org/pypi/robotstatuschecker>`_
-   look good.
+      invoke clean
 
-7. Test that installation works::
+3. Execute tests using different Python implementations and versions.
+   See `<README.rst>`_ for instructions.
 
-      pip install robotstatuschecker --upgrade
+4. Set version information to a shell variable to ease copy-pasting further
+   commands. Add ``aN``, ``bN`` or ``rcN`` postfix if creating a pre-release::
 
-8. ``__version__`` back to ``devel``::
+      VERSION=<version>
 
-      sed -i "s/__version__ = .*/__version__ = 'devel'/" robotstatuschecker.py
-      git diff  # verify changes
-      git commit -m "__version__ back to devel" robotstatuschecker.py && git push
+   For example, ``VERSION=3.0.1`` or ``VERSION=3.1a2``.
 
-9. Advertise on mailing lists, `Twitter <https://twitter.com/robotframework>`_,
+Release notes
+-------------
+
+1. Set GitHub user information into shell variables to ease copy-pasting the
+   following command::
+
+      GITHUB_USERNAME=<username>
+      GITHUB_PASSWORD=<password>
+
+   Alternatively, supply the credentials when running that command.
+
+2. Generate a template for the release notes::
+
+      invoke release-notes -w -v $VERSION -u $GITHUB_USERNAME -p $GITHUB_PASSWORD
+
+   The ``-v $VERSION`` option can be omitted if `version is already set
+   <Set version_>`__. Omit the ``-w`` option if you just want to get release
+   notes printed to the console, not written to a file.
+
+   When generating release notes for a preview release like ``3.0.2rc1``,
+   the list of issues is only going to contain issues with that label
+   (e.g. ``rc1``) or with a label of an earlier preview release (e.g.
+   ``alpha1``, ``beta2``).
+
+2. Fill the missing details in the generated release notes template.
+
+3. Make sure that issues have correct information:
+
+   - All issues should have type (bug, enhancement or task) and priority set.
+     Notice that issues with the task type are automatically excluded from
+     the release notes.
+   - Issue priorities should be consistent.
+   - Issue titles should be informative. Consistency is good here too, but
+     no need to overdo it.
+
+   If information needs to be added or edited, its better to edit it in the
+   issue tracker than in the generated release notes. This allows re-generating
+   the list of issues later if more issues are added.
+
+4. Add, commit and push::
+
+      git add docs/releasenotes/robotstatuschecker-$VERSION.rst
+      git commit -m "Release notes for $VERSION" docs/releasenotes/robotstatuschecker-$VERSION.rst
+      git push
+
+5. Update later if necessary. Writing release notes is typically the biggest
+   task when generating releases, and getting everything done in one go is
+   often impossible.
+
+
+Set version
+-----------
+
+1. Set version information in `<robotstatuschecker.py>`_::
+
+      invoke set-version $VERSION
+
+2. Commit and push changes::
+
+      git commit -m "Updated version to $VERSION" robotstatuschecker.py
+      git push
+
+
+
+Tagging
+-------
+
+1. Create an annotated tag and push it::
+
+      git tag -a v$VERSION -m "Release $VERSION"
+      git push --tags
+
+2. Add short release notes to GitHub's `releases page
+   <https://github.com/robotframework/statuschecker/releases>`_
+   with a link to the full release notes.
+
+Creating distributions
+----------------------
+
+1. Checkout the earlier created tag if necessary::
+
+      git checkout v$VERSION
+
+   This isn't necessary if continuing right after tagging_.
+
+2. Cleanup (again). This removes temporary files as well as ``build`` and
+   ``dist`` directories::
+
+      invoke clean
+
+3. Create source distribution and universal (i.e. Python 2 and 3 compatible)
+   `wheel <http://pythonwheels.com>`_::
+
+      python setup.py sdist bdist_wheel --universal
+      ls -l dist
+
+   Distributions can be tested locally if needed.
+
+4. Upload distributions to PyPI::
+
+      twine upload dist/*
+
+5. Verify that project the page at `PyPI
+   <https://pypi.org/project/robotstatuschecker/>`_
+   looks good.
+
+6. Test installation (add ``--pre`` with pre-releases)::
+
+      pip install --upgrade robotstatuschecker
+
+Post actions
+------------
+
+1. Back to master if needed::
+
+      git checkout master
+
+2. Set dev version based on the previous version::
+
+      invoke set-version dev
+      git commit -m "Back to dev version" robotstatuschecker.py
+      git push
+
+   For example, ``1.2.3`` is changed to ``1.2.4.dev1`` and ``2.0.1a1``
+   to ``2.0.1a2.dev1``.
+
+3. Close the `issue tracker milestone
+   <https://github.com/robotframework/statuschecker/milestones>`_.
+   Create also new milestone for the next release unless one exists already.
+
+4. Advertise on mailing lists, `Twitter <https://twitter.com/robotframework>`_,
    `LinkedIn <https://www.linkedin.com/groups/3710899>`_, and elsewhere as
    needed.

README.rst

@@ -60,10 +60,19 @@ the word ``FAIL`` (in uppercase) somewhere in the test case
 documentation. The expected error message must then follow
 the ``FAIL`` marker.
 
+For robotframework version 4 you can also change the expected status
+to *SKIP* by adding the word ``SKIP`` in the test case documentation.
+Like Fail, the expected skip message must follow the word ``SKIP``.
+If a test documentation contains the words ``FAIL`` and ``SKIP``, ``SKIP``
+will be ignored and the expected status will be *FAIL*.
+
 If a test is expected to *PASS* with a certain message, the word
 ``PASS`` must be added to its documentation explicitly and the
 expected message given after that.
 
+If a message check should happen in test setup or teardown, that check
+must be prefixed with ``SETUP`` or ``TEARDOWN`` word.
+
 The expected message can also be specified as a regular expression by
 prefixing it with ``REGEXP:``. The specified regular expression
 must match the error message fully. Having spaces between the status,
@@ -87,6 +96,11 @@ statuses and messages:
         [Documentation]    FAIL Expected error message
         Steps
 
+    Check in test setup is done by SETUP marker
+        [Documentation]    LOG SETUP    This first log message in test setup
+        [Setup]    Test specific setup
+        Steps
+
     Exclude documentation before marker
         [Documentation]    This text is ignored FAIL Expected error message
         Steps
@@ -126,11 +140,14 @@ The part after the colon species the message. For example, ``1:2``
 means the second message of the first keyword and ``1.2:3`` is
 the third message of the second child keyword of the first keyword.
 The message index is optional and defaults to ``1``.
+The message index also supports wildcard ``*``. For example ``1:*``
+matches any message of the first keyword.
 
 Message level is specified before the actual message, and it can be
 any of the valid log levels in capital letters. If the level is not
 given it defaults to ``INFO``. Starting from 1.4 release also
-``ERROR`` level is supported.
+``ERROR`` level is supported. The message level also supports wildcard
+``ANY`` which will match all log levels.
 
 Possible leading and trailing whitespace is ignored both in the expected
 and in the actual log message.

docs/releasenotes/robotstatuschecker-1.5.0.rst

@@ -0,0 +1,48 @@
+========================
+robotstatuschecker 1.5.0
+========================
+
+
+.. default-role:: code
+
+
+StatusChecker is a tool for validating that executed `Robot Framework`_ test cases
+have expected statuses and log messages. It is mainly useful for Robot Framework
+test library developers who want to use Robot Framework to also test their libraries.
+StatusChecker 1.4 and newer are compatible both with Python 2 and Python 3.
+
+StatusChecker project is hosted at GitHub and downloads are at PyPI_
+.. _Robot Framework: http://robotframework.org
+.. _PyPI: https://github.com/robotframework/statuschecker
+.. _issue tracker: https://github.com/robotframework/SeleniumLibrary/issues?q=milestone%3Av1.5.0
+
+
+.. contents::
+   :depth: 2
+   :local:
+
+Most important enhancements
+===========================
+
+Broken with RF4 (`#11`_)
+------------------------
+Fixes deprecation warning with RF 4.0
+
+Full list of fixes and enhancements
+===================================
+
+.. list-table::
+    :header-rows: 1
+
+    * - ID
+      - Type
+      - Priority
+      - Summary
+    * - `#11`_
+      - bug
+      - critical
+      - Broken with RF4
+
+Altogether 1 issue. View on the `issue tracker <https://github.com/robotframework/statuschecker/issues?q=milestone%3Av1.5.0>`__.
+
+.. _#11: https://github.com/robotframework/statuschecker/issues/11

docs/releasenotes/robotstatuschecker-1.5.1.rst

@@ -0,0 +1,43 @@
+========================
+robotstatuschecker 1.5.1
+========================
+
+
+.. default-role:: code
+
+
+StatusChecker is a tool for validating that executed `Robot Framework`_ test cases
+have expected statuses and log messages. It is mainly useful for Robot Framework
+test library developers who want to use Robot Framework to also test their libraries.
+StatusChecker 1.4 and newer are compatible both with Python 2 and Python 3.
+
+This reverts previous release because it does not work.
+
+StatusChecker project is hosted at GitHub and downloads are at PyPI_
+.. _Robot Framework: http://robotframework.org
+.. _PyPI: https://github.com/robotframework/statuschecker
+.. _issue tracker: https://github.com/robotframework/SeleniumLibrary/issues?q=milestone%3Av1.5.1
+
+
+.. contents::
+   :depth: 2
+   :local:
+
+Full list of fixes and enhancements
+===================================
+
+.. list-table::
+    :header-rows: 1
+
+    * - ID
+      - Type
+      - Priority
+      - Summary
+    * - `#17`_
+      - ---
+      - ---
+      - Revert RF 4.0 support because it does not work
+
+Altogether 1 issue. View on the `issue tracker <https://github.com/robotframework/statuschecker/issues?q=milestone%3Av1.5.1>`__.
+
+.. _#17: https://github.com/robotframework/statuschecker/issues/17