NB-Public/green-spider
14
0
Fork 1
mirror of https://github.com/netzbegruenung/green-spider.git synced 2024-05-04 10:03:40 +02:00
Code Issues 2 Projects Releases Wiki Activity

Commit graph

  • 6129525026
         Bump tenacity from 5.0.2 to 6.0.0 dependabot-preview[bot] 2019-11-26 15:29:35 +0000
  • 33f007ed64
         Bump httpretty from 0.9.4 to 0.9.7 dependabot-preview[bot] 2019-11-26 15:29:06 +0000
  • a5d37cacea
         Bump smmap2 from 2.0.3 to 2.0.5 (#146) dependabot-preview[bot] 2019-11-26 15:28:25 +0000
  • e3165f25dc
         Bump requests from 2.20.0 to 2.22.0 (#143) dependabot-preview[bot] 2019-11-26 15:27:55 +0000
  • 6fef5ba3ba
         Bump smmap2 from 2.0.3 to 2.0.5 dependabot-preview[bot] 2019-11-26 15:14:31 +0000
  • cfccbbfbd1
         Bump requests from 2.20.0 to 2.22.0 dependabot-preview[bot] 2019-11-26 15:14:30 +0000
  • e24c2507e6
         Bump responses from 0.9.0 to 0.10.7 (#147) dependabot-preview[bot] 2019-11-26 15:13:17 +0000
  • e3d537cef7
         Bump responses from 0.9.0 to 0.10.7 dependabot-preview[bot] 2019-11-26 07:07:59 +0000
  • 3ef838663c Rename 'make spiderjobs' to 'make jobs' Marian Steinbach 2019-11-25 22:59:48 +0100
  • 74d36a895f Adjust paths in Dockerfile, remove entrypoint Marian Steinbach 2019-11-25 22:59:21 +0100
  • 1d75633bc2 Downgrade to alpine 3.8 Marian Steinbach 2019-11-25 22:58:50 +0100
  • ab14950311 Add docker-compose YAML Marian Steinbach 2019-11-25 22:58:30 +0100
  • 647182a0d3 Add rq and redis dependencies Marian Steinbach 2019-11-25 22:52:37 +0100
  • 29284f702b Rename jobs to manager Marian Steinbach 2019-11-25 22:52:22 +0100
  • 64faa47e5b Move jobs to manage folder Marian Steinbach 2019-11-25 22:52:04 +0100
  • 12d3ff4731 Add job definition Marian Steinbach 2019-11-25 22:51:39 +0100
  • de2142c031 CLI: remove 'jobs' command, add 'manager' Marian Steinbach 2019-11-25 22:50:54 +0100
  • 74c7a07981
         Bump selenium from 3.8.0 to 3.141.0 dependabot-preview[bot] 2019-11-25 11:07:54 +0000
  • 5f5c8b0210
         Bump responses from 0.9.0 to 0.10.6 dependabot-preview[bot] 2019-11-25 11:07:48 +0000
  • 9789af1688
         Bump urllib3 from 1.24.2 to 1.25.7 (#140) dependabot-preview[bot] 2019-11-25 11:06:34 +0000
  • b88390a93e
         Bump beautifulsoup4 from 4.6.0 to 4.8.1 dependabot-preview[bot] 2019-11-25 07:18:51 +0000
  • 56af8a45d4
         Bump urllib3 from 1.24.2 to 1.25.7 dependabot-preview[bot] 2019-11-24 23:15:15 +0000
  • 36de884837 Upgrade beautifulsoup Marian Steinbach 2019-11-25 00:09:34 +0100
  • ff6eae3955
         Upgrade to alpine 3.9 (#135) Marian Steinbach 2019-11-24 23:54:26 +0100
  • 80d4f15271
         Set Python version to 3.7 (#136) Marian Steinbach 2019-11-24 23:54:15 +0100
  • 76a52f3cd1 Set Python version to 3.7 Marian Steinbach 2019-11-24 23:53:44 +0100
  • 847091bd62 Upgrade to alpine 3.9 Marian Steinbach 2019-11-24 23:33:34 +0100
  • 7c75e1194c
         Bump urllib3 from 1.22 to 1.24.2 (#133) dependabot[bot] 2019-11-22 22:55:15 +0000
  • 01a0000316
         Bump requests from 2.18.4 to 2.20.0 (#134) dependabot[bot] 2019-11-22 22:26:40 +0000
  • fca55496ba
         Bump requests from 2.18.4 to 2.20.0 dependabot[bot] 2019-11-22 22:14:25 +0000
  • 26018e5a35
         Bump urllib3 from 1.22 to 1.24.2 dependabot[bot] 2019-11-22 22:14:23 +0000
  • b3bb8f34c3
         Problembehebung mit Exception-Daten im Ergbebnis, die nicht geschrieben werden können, und Spidern einzelner Sites (#132) Marian Steinbach 2019-11-22 23:13:57 +0100
  • 2d15996591 Remove debugging Marian Steinbach 2019-11-22 23:02:21 +0100
  • 6cf115d4b8 Document single site spidering Marian Steinbach 2019-11-22 22:59:25 +0100
  • c82f977775 Move python modules lsit into requirements.txt Marian Steinbach 2019-11-22 17:29:16 +0100
  • 5adbcff56e Pass more arguments Marian Steinbach 2019-11-22 17:29:01 +0100
  • c4c1b3f65d Convert exception to string Marian Steinbach 2019-11-22 17:28:31 +0100
  • 3b4f2098f1 WIP commit for single job execution Marian Steinbach 2019-11-22 10:22:51 +0100
  • 725ed5439d Set share memory size Marian Steinbach 2019-11-22 08:39:56 +0100
  • 802a977fe5 Add required secrets for screenshots job Marian Steinbach 2019-11-22 08:38:49 +0100
  • 7788166b0c Fix logging output in dns_resolution Marian Steinbach 2019-11-22 08:38:26 +0100
  • 4f626947d2 Add the green power Rater class Chris Adams 2019-08-31 12:58:04 +0200
  • 27d5661dba Tidy up and remove useless assertion Chris Adams 2019-08-31 12:49:07 +0200
  • ec45dc68a1 Add test for the green and a grey powered site Chris Adams 2019-08-31 12:47:12 +0200
  • 3452fbf8e1 Add initial check with basic test Chris Adams 2019-08-31 12:36:10 +0200
  • 437ccfce17 Add env variable to test run Marian Steinbach 2019-08-16 00:21:54 +0200
  • 9c33c1250c Adapt rater Marian Steinbach 2019-08-15 23:48:35 +0200
  • 1b49020911 Rewrite mobile friendliness check Marian Steinbach 2019-08-15 23:39:57 +0200
  • b266960aad Add API key environment variable Marian Steinbach 2019-08-15 23:39:34 +0200
  • e71b74352c Gather more page dimension data Marian Steinbach 2019-08-06 23:52:36 +0200
  • 06f983d2f6 Emulate mobile device for responsiveness check Marian Steinbach 2019-08-06 23:06:04 +0200
  • 68f2288617
         Prüfe DNS auf IPv6 AAAA Record (#124) Marian Steinbach 2019-07-15 22:59:33 +0200
  • 165bf544b1 Adapt rating/resolvable Marian Steinbach 2019-07-14 23:59:51 +0200
  • d996adca15 Add check for IPv6 AAAA record Marian Steinbach 2019-07-14 23:52:11 +0200
  • 4401683133 Add shared memory to docker container Marian Steinbach 2019-07-14 22:51:05 +0200
  • 1087cc25ec Little devops/README update Marian Steinbach 2019-07-14 22:50:44 +0200
  • 0c59111044
         Alpine downgrade to 3.8 (#118) Marian Steinbach 2019-06-04 08:08:53 +0200
  • c6be4b9ad4 Alpine downgrade to 3.8 Marian Steinbach 2019-06-03 21:44:21 +0200
  • 576050d3cd Update alpine repository URLs Marian Steinbach 2019-06-03 08:12:30 +0200
  • aac5fb3192 Some Makefile updates Marian Steinbach 2019-06-03 08:08:55 +0200
  • 2f6cd9a304 Add extra docker run flags Marian Steinbach 2019-06-03 08:08:37 +0200
  • c41e717d3d Update devops stuff Marian Steinbach 2019-05-05 22:42:48 +0200
  • 9f7efddc61 Merge branch 'master' of https://github.com/netzbegruenung/green-spider Marian Steinbach 2019-05-05 22:41:34 +0200
  • 35fa87a04c Remove broken cookies test Marian Steinbach 2019-05-05 20:10:49 +0200
  • 04a1e98b79
         Prüfe Existenz von /favicon.ico und werte dies ebenso wie ein Icon, das im HTML Head verlinkt ist (#115) Marian Steinbach 2019-05-05 22:26:41 +0200
  • 1f06dde5f5 Remove broken cookies test Marian Steinbach 2019-05-05 20:10:49 +0200
  • 231ffd5a68 Check for /favicon.ico and rate it as icon available Marian Steinbach 2019-05-05 16:18:28 +0200
  • 8e83b71d21 Update .dockerignore Marian Steinbach 2019-05-05 15:44:14 +0200
  • 47d5c4f511 Move python modules into requirements.txt Marian Steinbach 2019-05-05 15:44:02 +0200
  • a0c8e66789 Fix package source URL Marian Steinbach 2019-05-05 14:03:34 +0200
  • cc6a52bf45 Update README Marian Steinbach 2019-05-04 23:01:04 +0200
  • 620610b48e Update ignore list Marian Steinbach 2019-05-04 23:00:56 +0200
  • ab942ca91d Fix full JSON export Marian Steinbach 2019-05-04 23:00:00 +0200
  • 9e5426ccde Use alpine 3.9 base image Marian Steinbach 2019-05-03 23:20:08 +0200
  • cff4d55f17 Fix problem where pageload was not counted Marian Steinbach 2019-05-03 22:54:25 +0200
  • 7621b7ef75 Remove debugging output Marian Steinbach 2019-05-03 22:54:05 +0200
  • 56f9f1ba86 Check third party cookies Marian Steinbach 2019-04-29 10:09:25 +0200
  • 5e8347916c
         Fehlerbehebung im url_reachability check (#108) Marian Steinbach 2019-04-19 00:35:28 +0200
  • f25dd87963 Remove pip upgrade Marian Steinbach 2019-04-19 00:25:47 +0200
  • 7ba11a32a1 Fix test in CI Marian Steinbach 2019-04-19 00:24:43 +0200
  • 73318abf09 Fix tests Marian Steinbach 2019-04-19 00:05:27 +0200
  • 685a67951a Add spidering of a single URL for debugging purposes Marian Steinbach 2019-04-18 22:50:45 +0200
  • 0ebd5af0b3 Pass args to make spider Marian Steinbach 2019-04-18 22:50:08 +0200
  • 3a69d0e0ca Add --url flag to spider for faster debugging Marian Steinbach 2019-04-18 22:49:22 +0200
  • 45309f1681 Fix bad domain check Marian Steinbach 2019-04-18 22:49:06 +0200
  • 272d4fa01d Fix detection of redirects to bad domains Marian Steinbach 2019-04-17 23:14:06 +0200
  • 2dfcf61cc0
         Add netzbegruenung/green-spider-indexer to README Marian Steinbach 2019-04-11 22:43:16 +0200
  • 16a05b751b Several fixes for edge cases Marian Steinbach 2018-12-17 23:54:09 +0100
  • 3b8328d804 Fixing several bugs in spider code Marian Steinbach 2018-12-17 17:31:09 +0100
  • 3b9ead330d
         Load feeds and gather info (#103) Marian Steinbach 2018-12-07 16:32:42 +0100
  • 3063a4488d
         Detect frameset (#102) Marian Steinbach 2018-12-07 16:31:56 +0100
  • 59a1e9078c Load feeds and gather info Marian Steinbach 2018-12-06 21:43:05 +0100
  • e1d7c88e71 Remove unused variable (unrelated) Marian Steinbach 2018-12-06 17:39:48 +0100
  • 5c0672dabe Add frameset checker Marian Steinbach 2018-12-06 17:39:35 +0100
  • deff95306b
         Extend CMS detection for Urwahl3000 theme (#96) Marian Steinbach 2018-12-05 21:27:45 +0100
  • 346b2a54a2 Remove unused import Marian Steinbach 2018-12-05 21:25:15 +0100
  • 7ecaa7587b Extend check for Urwahl3000 theme Marian Steinbach 2018-12-05 21:25:03 +0100
  • d0e3a4210f
         Fix link raters (social media links, contact link) (#95) Marian Steinbach 2018-11-28 23:46:40 +0100
  • bc71dedeb8 Skip checks when dependencies not met Marian Steinbach 2018-11-28 23:40:44 +0100
  • e2f0f2c507 Fix rating for contact_link and social_media_link Marian Steinbach 2018-11-28 23:40:18 +0100