NB-Public/green-spider
14
0
Fork 1
mirror of https://github.com/netzbegruenung/green-spider.git synced 2024-04-28 07:14:51 +02:00
Code Issues 2 Projects Releases Wiki Activity
258 commits 7 branches 7 tags 4.1 MiB
Commit graph

3 commits

Author SHA1 Message Date
Marian Steinbach 04a1e98b79
Prüfe Existenz von /favicon.ico und werte dies ebenso wie ein Icon, das im HTML Head verlinkt ist (#115) 
* Fix full JSON export

* Update ignore list

* Update README

* Check for /favicon.ico and rate it as icon available

* Remove broken cookies test
 2019-05-05 22:26:41 +02:00
Marian Steinbach 5e8347916c
Fehlerbehebung im url_reachability check (#108) 
* Fix detection of redirects to bad domains

* Fix bad domain check

* Add --url flag to spider for faster debugging

* Pass args to make spider

* Add spidering of a single URL for debugging purposes

* Fix tests

* Fix test in CI

* Remove pip upgrade
 2019-04-19 00:35:28 +02:00
Marian Steinbach fd4a29da8e
Collect cookies in load_in_browser check (#74) 2018-10-04 21:21:30 +02:00