* CLI: remove 'jobs' command, add 'manager'
* Add job definition
* Move jobs to manage folder
* Rename jobs to manager
* Add rq and redis dependencies
* Add docker-compose YAML
* Downgrade to alpine 3.8
* Adjust paths in Dockerfile, remove entrypoint
* Rename 'make spiderjobs' to 'make jobs'
* Fix docker exectution
* Adapt 'make jobs'
* Fix metadata scheme
* Add docker dependency
* Rendomize queue (a bit)
* Use latest image, remove debug output
* Make docker-compose file downwards-compatible
* Use latest instead of dev image tag
* Update docker-compose.yaml
* Adapt job start script
* Fix redis connection in manager
* Add support for increasing timeout via environment variable
* Adapt load_in_browser to cookies table schema change
* Fix execution
* Mitigate yaml warning
* Bump some dependency versions
* Report resource usage stats for each job
* checks/load_in_browser: Return DOM size, prevent multiple page loads
* Update .dockerignore
* Code update
* Script update
* Update README.md
* WIP
* WIP commit
* Update Dockerfile to alpine:edge and chromium v90
* Update TestCertificateChecker
* Set defaults for __init__ function
* Detect sunflower theme
* Update unit test for new datetime (zero-basing)
* Set logging prefs from Chromium in a new way
* Move datastore client instantiation
As it is not needed for all commands
* Change green-directory repository URL
* Add git settings for cloning green-directory
* Pin alpine version 3.14, fix py3-cryptography
* Use plain docker build progress output
* Add volumes to 'make test' docker run command
* Fix bug
* Update example command in README
* Update dependencies
* Add creation of Kubernetes jobs
* WIP commit for single job execution
* Convert exception to string
* Pass more arguments
* Move python modules lsit into requirements.txt
* Document single site spidering
* Remove debugging
* Fix detection of redirects to bad domains
* Fix bad domain check
* Add --url flag to spider for faster debugging
* Pass args to make spider
* Add spidering of a single URL for debugging purposes
* Fix tests
* Fix test in CI
* Remove pip upgrade
* Add webapp deployment script
* Add some docs for webapp
* Some fixes in run-job.sh
* Update webapp deployment script
* Add some kubernetes job manifests
* Create index.yaml
* Remove local creation of the docker image from targets
* Update README.md
Marian Steinbach
Marian Steinbach