* WIP commit for single job execution
* Convert exception to string
* Pass more arguments
* Move python modules lsit into requirements.txt
* Document single site spidering
* Remove debugging
* Fix detection of redirects to bad domains
* Fix bad domain check
* Add --url flag to spider for faster debugging
* Pass args to make spider
* Add spidering of a single URL for debugging purposes
* Fix tests
* Fix test in CI
* Remove pip upgrade
* Add webapp deployment script
* Add some docs for webapp
* Some fixes in run-job.sh
* Update webapp deployment script
* Add some kubernetes job manifests
* Create index.yaml
* Remove local creation of the docker image from targets
* Update README.md
* Fix the certificate check to support SNI
* Better tests for the certificate check
* Activate verbose output when running make test
* Add commenting on the spider test