5 Commits

Author	SHA1	Message	Date
Marian Steinbach	5e8347916c	Fehlerbehebung im url_reachability check (#108) ... * Fix detection of redirects to bad domains * Fix bad domain check * Add --url flag to spider for faster debugging * Pass args to make spider * Add spidering of a single URL for debugging purposes * Fix tests * Fix test in CI * Remove pip upgrade	2019-04-19 00:35:28 +02:00
Marian Steinbach	ae6a2e83e9	Refactor and modularize spider (#70) ... See PR description for details	2018-10-03 11:05:42 +02:00
Marian Steinbach	c2758652f1	Modify travis config to run docker	2018-05-03 12:04:32 +02:00
Marian Steinbach	15a7315886	Remove "make venv"	2018-05-03 12:02:48 +02:00
Marian Steinbach	857aa55f0b	Add travis config	2018-05-03 11:43:36 +02:00