update README, add httpobs-local-scan script to poetry

README.md

@@ -1,4 +1,4 @@
-# Mozilla HTTP Observatory - [![Build Status](https://travis-ci.org/april/http-observatory.svg?branch=master)](https://travis-ci.org/april/http-observatory) [![Requirements Status](https://requires.io/github/mozilla/http-observatory/requirements.svg?branch=master)](https://requires.io/github/mozilla/http-observatory/requirements/?branch=master)
+# Mozilla HTTP Observatory
 
 The Mozilla HTTP Observatory is a set of tools to analyze your website and inform you if you are utilizing the many available methods to secure it.
 
@@ -19,28 +19,40 @@ Sites can be scanned using:
 ## Contributing
 
 ### Prerequisites
-* Python 3.7
+* Python 3.11
 * Git
-* pip3
+* pip
 
 #### Notes
 
-These instructions assume that you have a working Python3.7 development environment with `pip3` installed and capable of building requirements, which may require installing an additional python OS package (`-dev`, `-devel`).
+These instructions assume that you have a working Python3.11 development environment with `pip` installed and capable of building requirements, which may require installing an additional python OS package (`-dev`, `-devel`).
 
-If this is not appropriate for your environment, you may install the appropriate requirements using your OS package manager (or other means) and skip the `pip3 -r requirements` command.
+```bash
+# Clone the code
+$ git clone https://github.com/mozilla/http-observatory.git
+$ cd http-observatory
+# Install poetry
+$ pip install poetry
+# Install the project dependencies and scripts
+$ poetry install
+# Activate the virtual environment
+$ poetry shell
+# Install the pre-commit hooks
+$ pre-commit install
+```
 
 ## Running a scan from the local codebase, without DB, for continuous integration
 ```bash
 # Install the HTTP Observatory
 $ git clone https://github.com/mozilla/http-observatory.git
 $ cd http-observatory
-$ pip3 install --upgrade .
-$ pip3 install --upgrade -r requirements.txt
+$ pip install poetry
+$ poetry install
 ```
 
-### Using the local scanner function calls
+### Using the scanner function calls
 ```python
->>> from httpobs.scanner.local import scan
+>>> from httpobs.scanner import scan
 >>> scan('observatory.mozilla.org')  # a scan with default options
 >>> scan('observatory.mozilla.org',  # all the custom options
          http_port=8080,             # http server runs on port 8080
@@ -53,75 +65,11 @@ $ pip3 install --upgrade -r requirements.txt
 
 ### The same, but with the local CLI
 ```bash
+$ poetry shell
 $ httpobs-local-scan --http-port 8080 --https-port 8443 --path '/foo/bar' \
     --cookies '{"foo": "bar"}' --headers '{"X-Foo": "bar"}' --no-verify mozilla.org
 ```
 
-## Running a local scanner with Docker
-* Install [Docker Toolbox](https://docs.docker.com/toolbox/overview/) and [VirtualBox](https://www.virtualbox.org/wiki/Downloads)
-
-```bash
-# Install the HTTP Observatory client and requests library
-$ git clone https://github.com/mozilla/http-observatory.git
-$ cd http-observatory
-$ pip3 install .
-$ pip3 install --upgrade requests
-
-# Create docker machine
-$ docker-machine create --driver virtualbox --virtualbox-disk-size "40000" http-observatory
-
-# Save the URL to the API in your .profile, .bash_profile, or whatever
-$ echo export HTTPOBS_API_URL=http://$(docker-machine ip http-observatory):57001/api/v1 >> ~/.profile
-$ . ~/.profile
-
-# Start up the docker instance and install all the pieces
-$ eval $(docker-machine env http-observatory)
-$ docker-compose up -d
-```
-
-## Creating a local installation (tested on Ubuntu 15)
-```
-# Install git, postgresql, and redis
-# sudo -s
-# apt-get install -y git libpq-dev postgresql redis-server
-
-# Clone the repo
-# cd /opt
-# git clone https://github.com/mozilla/http-observatory.git
-# cd http-observatory
-
-# Install the observatory and scanner
-# pip install .
-# pip3 install -r requirements.txt
-
-# Install the database
-# su - postgres
-$ createdb http_observatory
-$ psql http_observatory < httpobs/database/schema.sql
-$ psql http_observatory
-http_observatory=# \password httpobsapi
-http_observatory=# \password httpobsscanner
-# vi /etc/postgresql/9.4/main/postgresql.conf (set max_connections = 512, shared_buffers = 256MB)
-# service postgresql restart
-
-# Create the httpobs user, and log/pid directories
-# useradd -m httpobs
-# install -m 750 -o httpobs -g httpobs -d /var/run/httpobs /var/log/httpobs
-
-# Update the environmental variables
-# su - httpobs
-$ echo export HTTPOBS_API_URL="http://localhost:57001/api/v1" >> ~/.profile
-
-# Start the scanner
-$ cd /opt/http-observatory
-$ HTTPOBS_DATABASE_USER="httpobsscanner" HTTPOBS_DATABASE_PASS="....." \
-    /opt/http-observatory/httpobs/scripts/httpobs-scan-worker
-
-# Start the API (in another terminal)
-# HTTPOBS_DATABASE_USER="httpobsapi" HTTPOBS_DATABASE_PASS="....." \
-    uwsgi --http :57001 --wsgi-file httpobs/website/main.py --processes 8 --callable app --master
-```
-
 ## Authors
 
 * April King

httpobs/scripts/httpobs-local-scan → httpobs/scripts/scan.py

@@ -5,9 +5,10 @@
 from operator import itemgetter
 from urllib.parse import urlparse
 
-import httpobs.scanner.local
+import httpobs.scanner
 
-if __name__ == "__main__":
+
+def main():
     parser = argparse.ArgumentParser()
 
     # Add the various arguments
@@ -56,7 +57,7 @@
         args['verify'] = False
 
     # Get the scan results
-    r = httpobs.scanner.local.scan(**args)
+    r = httpobs.scanner.scan(**args)
 
     # print out the results to the command line
     if output_format == 'json':
@@ -76,3 +77,7 @@
             if score[1] > 0:
                 score[1] = '+' + str(score[1])  # display 5 as +5
             print('  {test:<30} [{modifier:>3}]  {reason}'.format(test=score[0], modifier=score[1], reason=score[2]))
+
+
+if __name__ == "__main__":
+    main()

pyproject.toml

@@ -6,6 +6,9 @@ license = "MPL-2.0"
 authors = ["April King <april@mozilla.com>"]
 maintainers = ["Leo McArdle <leo@mozilla.com>"]
 
+[tool.poetry.scripts]
+httpobs-local-scan = 'httpobs.scripts.scan:main'
+
 [tool.poetry.dependencies]
 python = "^3.11"
 beautifulsoup4 = "^4.12.2"