5e183dc34a
Update dependencies ( #356 )
...
* Update Google modules
* Update pyasn1 modules
* Update cffi
* Update gitdb and GitPython
* Update various modules
* Update feedparser
* Update w3lib
2024-04-03 22:29:33 +02:00
99d0980686
Bump google-auth from 2.28.1 to 2.29.0 ( #351 )
2024-04-03 14:54:53 +00:00
ee167db02f
Bump certifi from 2023.7.22 to 2024.2.2 ( #352 )
2024-04-03 14:31:46 +00:00
a84c2244bd
Bump pyopenssl from 23.0.0 to 24.1.0 ( #349 )
2024-04-03 14:27:10 +00:00
e2bc62e51a
Bump redis from 4.1.0 to 5.0.3 ( #350 )
2024-04-03 14:18:30 +00:00
f2a4be1b9d
Bump urllib3 from 1.26.7 to 1.26.18 ( #355 )
2024-04-03 14:17:35 +00:00
1136637ed8
Bump rq from 1.16.0 to 1.16.1 ( #344 )
2024-04-03 13:58:04 +00:00
ab58152b8e
Set chromium version to 123.0.6312.86-r0 ( #354 )
2024-04-03 15:45:02 +02:00
0c0bcbf54e
Mehrere Fixes und Verbesserungen ( #343 )
...
* Use UTC for feed item age calculation
* Improvements in run-job.sh script
* Prevent output buffering in job creation
* Remove unused environment variable references
* Print more detailed results count
* Bring back function to execute a single spider job
* Fix 'make spider' command
* Upgrade docker to 5.0.3
2024-03-07 11:31:16 +01:00
c59db691a0
Reparatur und Aufräumen an der job execution ( #340 )
...
* Update jq URL
* Improve docker compose setup
* Script makeover: only one spider job, debian 11, add git clone
* Update image name
* Add some docs
* Pin click to v7 due to problems with rq
* Newline
* Improve manager code
* Add make tarket venv
* Remove obsolete 'spider' command from cli
* Remove git clone from manager code
* Remove worker functions from spider code
* Let 'make jobs' execute git clone and use docker compose
* Add 'spider' make target
* Update .dockerignore
* Add dryrun target to spider a URL without storing results
* Remove unused config entry
2024-03-04 17:18:37 +01:00
2927acde84
Bump kubernetes from 20.13.0 to 29.0.0 ( #334 )
2024-03-01 17:42:22 +00:00
27705c83cb
Bump idna from 2.10 to 3.6 ( #331 )
2024-03-01 17:29:18 +00:00
b17896cdb5
Bump cssselect from 1.1.0 to 1.2.0 ( #335 )
2024-03-01 17:19:48 +00:00
a0e0dae2b9
Bump google-auth from 2.13.0 to 2.28.1 ( #330 )
2024-03-01 17:14:42 +00:00
e48f80cd50
Bump rq from 1.8.0 to 1.16.0 ( #333 )
2024-03-01 17:10:31 +00:00
bfd5ebc25e
Bump smmap2 from 2.0.5 to 3.0.1 ( #338 )
2024-03-01 17:05:04 +00:00
724708217d
Bump cachetools from 5.3.2 to 5.3.3 ( #332 )
2024-03-01 16:59:50 +00:00
7d43d75b93
Update auf Chromium v122; Verbesserung am container image build ( #339 )
...
* Use Chromium v122
* Pin base image to checksum
* Change package sources, do not use edge/main
2024-03-01 17:51:56 +01:00
894711a2d8
Entfernen obsoleter Inhalte aus Devops script und README ( #329 )
...
* Remove screenshotter, fix job name argument
* Remove Kubernetes info from README
2024-02-22 09:43:22 +01:00
f0db5aad9b
Change quay.io to ghcr.io ( #328 )
2024-02-22 09:32:49 +01:00
b1762da7ef
Workflow zum Erzeugen und Veröffentlichen eines Container Image auf ghcr.io ( #327 )
...
* Add workflow
* Update image name in Makefile
* Add step to create VERSION file
* Remove debugging trigger
2024-02-22 09:16:44 +01:00
c86794b687
Update README content ( #326 )
2024-02-22 09:08:45 +01:00
42bc89f251
Bump chardet from 3.0.4 to 5.2.0 ( #324 )
2024-02-21 11:55:34 +00:00
fceda4f642
Bump beautifulsoup4 from 4.10.0 to 4.12.3 ( #314 )
2024-02-21 10:09:44 +00:00
fd8947cbf1
Bump rsa from 4.7.2 to 4.9 ( #291 )
2024-02-21 10:00:11 +00:00
e7a8511650
Bump smmap from 3.0.4 to 5.0.1 ( #323 )
2024-02-21 09:59:59 +00:00
a5b67c31ff
Bump dnspython from 2.1.0 to 2.6.1 ( #318 )
2024-02-21 09:55:26 +00:00
fe747e5e14
Bump cachetools from 4.2.4 to 5.3.2 ( #316 )
...
Bumps [cachetools](https://github.com/tkem/cachetools ) from 4.2.4 to 5.3.2.
- [Changelog](https://github.com/tkem/cachetools/blob/master/CHANGELOG.rst )
- [Commits](https://github.com/tkem/cachetools/compare/v4.2.4...v5.3.2 )
---
updated-dependencies:
- dependency-name: cachetools
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-21 10:54:46 +01:00
335e4d3af4
Bump certifi from 2021.10.8 to 2023.7.22 ( #321 )
...
Bumps [certifi](https://github.com/certifi/python-certifi ) from 2021.10.8 to 2023.7.22.
- [Commits](https://github.com/certifi/python-certifi/compare/2021.10.08...2023.07.22 )
---
updated-dependencies:
- dependency-name: certifi
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-21 10:51:00 +01:00
e8fae03f49
Bump gitpython from 3.1.24 to 3.1.41 ( #320 )
...
Bumps [gitpython](https://github.com/gitpython-developers/GitPython ) from 3.1.24 to 3.1.41.
- [Release notes](https://github.com/gitpython-developers/GitPython/releases )
- [Changelog](https://github.com/gitpython-developers/GitPython/blob/main/CHANGES )
- [Commits](https://github.com/gitpython-developers/GitPython/compare/3.1.24...3.1.41 )
---
updated-dependencies:
- dependency-name: gitpython
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-21 10:50:47 +01:00
a839597168
Bump requests from 2.26.0 to 2.31.0 ( #322 )
...
Bumps [requests](https://github.com/psf/requests ) from 2.26.0 to 2.31.0.
- [Release notes](https://github.com/psf/requests/releases )
- [Changelog](https://github.com/psf/requests/blob/main/HISTORY.md )
- [Commits](https://github.com/psf/requests/compare/v2.26.0...v2.31.0 )
---
updated-dependencies:
- dependency-name: requests
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-21 10:50:30 +01:00
a4c822ebfb
Bump websocket-client from 0.59.0 to 1.7.0 ( #313 )
...
Bumps [websocket-client](https://github.com/websocket-client/websocket-client ) from 0.59.0 to 1.7.0.
- [Release notes](https://github.com/websocket-client/websocket-client/releases )
- [Changelog](https://github.com/websocket-client/websocket-client/blob/master/ChangeLog )
- [Commits](https://github.com/websocket-client/websocket-client/compare/v0.59.0...v1.7.0 )
---
updated-dependencies:
- dependency-name: websocket-client
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-21 09:27:17 +01:00
ced7fbd9b9
Bump protobuf from 4.21.8 to 4.25.3 ( #311 )
...
Bumps [protobuf](https://github.com/protocolbuffers/protobuf ) from 4.21.8 to 4.25.3.
- [Release notes](https://github.com/protocolbuffers/protobuf/releases )
- [Changelog](https://github.com/protocolbuffers/protobuf/blob/main/protobuf_release.bzl )
- [Commits](https://github.com/protocolbuffers/protobuf/commits/v4.25.3 )
---
updated-dependencies:
- dependency-name: protobuf
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-21 09:26:42 +01:00
0efab3295c
Chromium und Alpine update ( #319 )
...
* Update base image to alpine v3.19
* Update Chromium to 121
* Remove libssl1.1 (no longer available in alpine 3.19)
* Add --break-system-packages to pip install commands
* Print debugging info
* Upgrade actions/checkout to v4
* Fix entrypoints and commands
* Upgrade pyOpenSSL to v24.0.0
* Upgrade tenacity to v8
2024-02-21 09:19:33 +01:00
a6af33772e
Use latest release in spider jobs
2022-10-25 09:41:01 +02:00
5e723c94db
Make and use a versioned docker image ( #279 )
...
* Revert redis module to 4.1.0
* Revert dnspython to 2.1.0
* Revert click to 8.0.3
* Specify alpine 3.16.2, reorganize into multiple steps
* Replace 'latest' with 'main' everywhere
* Fix deprecation warnings
* Add Google root certificates
* Re-order APK packages, write list after installing
* Create VERSION file during docker image build
* Pin chromium version
2022-10-24 21:35:15 +02:00
024ef118dd
Bump dnspython from 1.16.0 to 2.2.1 ( #253 )
...
Bumps [dnspython](https://github.com/rthalley/dnspython ) from 1.16.0 to 2.2.1.
- [Release notes](https://github.com/rthalley/dnspython/releases )
- [Changelog](https://github.com/rthalley/dnspython/blob/master/doc/whatsnew.rst )
- [Commits](https://github.com/rthalley/dnspython/compare/v1.16.0...v2.2.1 )
---
updated-dependencies:
- dependency-name: dnspython
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-10-24 14:36:31 +02:00
47f327290d
Bump cffi from 1.14.5 to 1.15.1 ( #263 )
...
Bumps [cffi](http://cffi.readthedocs.org ) from 1.14.5 to 1.15.1.
---
updated-dependencies:
- dependency-name: cffi
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-10-24 14:35:58 +02:00
f3f08f68df
Bump httpretty from 0.9.7 to 1.1.4 ( #222 )
...
Bumps [httpretty](https://github.com/gabrielfalcao/HTTPretty ) from 0.9.7 to 1.1.4.
- [Release notes](https://github.com/gabrielfalcao/HTTPretty/releases )
- [Changelog](https://github.com/gabrielfalcao/HTTPretty/blob/master/.release )
- [Commits](https://github.com/gabrielfalcao/HTTPretty/compare/0.9.7...1.1.4 )
---
updated-dependencies:
- dependency-name: httpretty
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-10-24 14:35:26 +02:00
643dd4c6c9
Bump click from 7.1.2 to 8.1.3 ( #257 )
...
Bumps [click](https://github.com/pallets/click ) from 7.1.2 to 8.1.3.
- [Release notes](https://github.com/pallets/click/releases )
- [Changelog](https://github.com/pallets/click/blob/main/CHANGES.rst )
- [Commits](https://github.com/pallets/click/compare/7.1.2...8.1.3 )
---
updated-dependencies:
- dependency-name: click
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-10-24 14:19:36 +02:00
2a8921b1fa
Bump protobuf from 3.19.1 to 4.21.8 ( #276 )
...
Bumps [protobuf](https://github.com/protocolbuffers/protobuf ) from 3.19.1 to 4.21.8.
- [Release notes](https://github.com/protocolbuffers/protobuf/releases )
- [Changelog](https://github.com/protocolbuffers/protobuf/blob/main/generate_changelog.py )
- [Commits](https://github.com/protocolbuffers/protobuf/commits )
---
updated-dependencies:
- dependency-name: protobuf
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-10-24 14:03:31 +02:00
c5a1feac5a
Bump redis from 3.5.3 to 4.3.4 ( #264 )
...
Bumps [redis](https://github.com/redis/redis-py ) from 3.5.3 to 4.3.4.
- [Release notes](https://github.com/redis/redis-py/releases )
- [Changelog](https://github.com/redis/redis-py/blob/master/CHANGES )
- [Commits](https://github.com/redis/redis-py/compare/3.5.3...v4.3.4 )
---
updated-dependencies:
- dependency-name: redis
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-10-21 16:57:00 +02:00
8310e9760d
Bump responses from 0.13.3 to 0.22.0 ( #275 )
...
Bumps [responses](https://github.com/getsentry/responses ) from 0.13.3 to 0.22.0.
- [Release notes](https://github.com/getsentry/responses/releases )
- [Changelog](https://github.com/getsentry/responses/blob/master/CHANGES )
- [Commits](https://github.com/getsentry/responses/compare/0.13.3...0.22.0 )
---
updated-dependencies:
- dependency-name: responses
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-10-21 16:46:50 +02:00
51052ebac1
Add tests action ( #273 )
...
* Add tests action
* Change test execution command
* Change action towards building a docker image
* Use pyyaml from alpine package
* Add test execution
* Enable running without terminal in CI
* Add creation of empty secret file
* Fill service account file
* Fix JSON
* Fix quiting
* Add more fields to fake secret
* Wrap execution in try/except
* Fix: local variable 'result' referenced before assignment
2022-10-21 16:27:15 +02:00
a864675d85
Upgrade PyOpenSSL, remove unit tests using TLS 1.0 and 1.1 ( #274 )
...
* Upgrade pyOpenSSL to v22.1.0
* Remove unit test using TLS 1.0 and 1.1
* Upgrade base image to alpine 3.15.6
2022-10-18 09:48:25 +02:00
2da7c81b43
Automatisierung für Job-Erzeugung und -Ausführung in Kubernetes ( #232 )
...
* Add TODO
* Add link to example for in-cluster config
* Ignore .env files
* Mount green-directory clone as a volume
* Update kubectl context name
* Add a bit of instructions
* Let clone_data_directory return early if clone exists
2022-10-18 09:00:31 +02:00
fa2299c92d
Update .gitignore
2022-10-17 21:26:41 +02:00
394b902fa6
check for gruene.social instead of Google Plus ( #267 )
2022-09-16 10:13:49 +02:00
Marian Steinbach
ba40bcf5b4
Update kubernetes ( #242 )
...
* Add kubernetes module
* Update context name
* Remove pod anti-affinity rule
* Update image tag
2021-12-06 22:40:43 +01:00
Marian Steinbach
357ecb6fd3
Update some dependencies ( #237 )
2021-12-06 22:38:38 +01:00
Marian Steinbach
Marian Steinbach