green-spider

Commit Graph

Select branches

Hide Pull Requests

dependabot/pip/click-gte-7-and-lt-9

dependabot/pip/docker-7.0.0

dependabot/pip/idna-3.7

dependabot/pip/protobuf-5.26.1

font-detection

main

selenium-v4

#1

#102

#103

#108

#114

#115

#118

#124

#128

#129

#129

#132

#133

#134

#135

#136

#137

#138

#139

#14

#140

#141

#142

#143

#144

#145

#146

#147

#149

#150

#151

#152

#153

#155

#156

#157

#158

#159

#160

#161

#164

#165

#167

#168

#169

#17

#170

#172

#173

#175

#176

#177

#178

#179

#180

#181

#182

#183

#184

#185

#186

#187

#188

#189

#190

#191

#192

#194

#195

#196

#197

#199

#2

#200

#201

#202

#203

#204

#205

#206

#207

#208

#209

#21

#210

#212

#213

#214

#215

#216

#217

#218

#219

#220

#221

#222

#223

#224

#225

#226

#227

#228

#229

#230

#231

#232

#233

#234

#235

#236

#237

#238

#239

#24

#240

#241

#242

#243

#245

#246

#247

#248

#249

#25

#250

#251

#252

#253

#254

#255

#256

#257

#258

#259

#260

#261

#262

#263

#264

#265

#266

#267

#268

#269

#269

#27

#270

#271

#272

#273

#274

#275

#276

#279

#284

#285

#286

#287

#288

#289

#29

#290

#291

#292

#293

#294

#295

#296

#297

#298

#299

#3

#300

#301

#302

#303

#304

#305

#306

#307

#308

#309

#310

#311

#312

#313

#314

#315

#316

#317

#318

#319

#320

#321

#322

#323

#324

#326

#327

#328

#329

#33

#330

#331

#332

#333

#334

#335

#336

#337

#338

#339

#34

#340

#342

#342

#343

#344

#345

#345

#346

#346

#347

#348

#348

#349

#35

#350

#351

#352

#353

#354

#355

#356

#357

#357

#358

#358

#36

#39

#40

#41

#42

#43

#44

#45

#46

#47

#5

#51

#52

#54

#55

#56

#58

#59

#6

#60

#62

#63

#64

#65

#67

#70

#71

#72

#73

#74

#75

#78

#80

#81

#87

#88

#89

#90

#93

#94

#95

#96

v1.0.0

v1.0.0-rc1

v2024.2.21

v2024.2.22

v2024.3.3

v2024.3.7

v2024.4.3

ea00f30557 Update a comment Marian Steinbach 2018-11-20 22:43:03 +0100
3cb7bbbec6 Add rating for contact and social media links Marian Steinbach 2018-11-20 22:38:28 +0100
f555781735 Add hyperlink checker Marian Steinbach 2018-11-20 22:38:14 +0100
4524cb5714

Consider site reachable only with status code < 400 (#89) Marian Steinbach 2018-11-20 20:14:52 +0100
caa0489deb Consider site reachable only with status code < 400 Marian Steinbach 2018-11-20 20:11:21 +0100
c03ff21a9c

Simplify export (#88) Marian Steinbach 2018-11-20 20:00:47 +0100
57b92418a5 Merge branch 'master' into simplify-export Marian Steinbach 2018-11-20 19:55:43 +0100
899b58f017 Create file output in current working directory Marian Steinbach 2018-11-20 19:55:28 +0100
a18eb80455 Simplify exports Marian Steinbach 2018-11-20 19:55:05 +0100
38481236ca

Add webapp deployment (#87) Marian Steinbach 2018-11-20 19:54:23 +0100
5ae29175ae Update README.md Marian Steinbach 2018-11-20 19:45:34 +0100
79483b94c8 Remove local creation of the docker image from targets Marian Steinbach 2018-11-20 19:42:06 +0100
1709566c18 Create index.yaml Marian Steinbach 2018-11-20 19:41:41 +0100
ef006c667e Add some kubernetes job manifests Marian Steinbach 2018-11-20 19:25:36 +0100
f3183d45fc Update webapp deployment script Marian Steinbach 2018-11-20 19:25:20 +0100
5a73fb248c Some fixes in run-job.sh Marian Steinbach 2018-11-12 23:49:52 +0100
6399274764 Add some docs for webapp Marian Steinbach 2018-11-12 23:48:41 +0100
c74ee5f886 Add webapp deployment script Marian Steinbach 2018-11-12 23:48:28 +0100
924981659b

Allow Titillium together with Arvo (#78) Marian Steinbach 2018-11-05 23:18:11 +0100
325caee2bb

Detect generator jimdo (#81) Marian Steinbach 2018-11-05 23:00:01 +0100
df1f0bb452

Detect Drupal (#80) Marian Steinbach 2018-11-05 22:32:06 +0100
6ddc3868e1

Detect generator jimdo Marian Steinbach 2018-11-05 22:24:56 +0100
196cdb7d8d

Detect Drupal Marian Steinbach 2018-11-05 22:21:20 +0100
10b5db2ce8

Allow Titillium together with Arvo Marian Steinbach 2018-11-05 14:21:04 +0100
8ce8768465 Improve error messages in export Marian Steinbach 2018-10-08 08:42:29 +0200
0538e437ea Fix NoneType error in rater responsive_layout Marian Steinbach 2018-10-07 21:14:29 +0200
18be6e7adf Fix rater no_network_errors Marian Steinbach 2018-10-07 20:55:57 +0200
4251df6b06

Fixes for two problems found during spidering (#75) Marian Steinbach 2018-10-05 10:25:05 +0200
323205a333 Fixes for two problems found during spidering Marian Steinbach 2018-10-05 07:22:44 +0200
fd4a29da8e

Collect cookies in load_in_browser check (#74) Marian Steinbach 2018-10-04 21:21:30 +0200
2945372aaf

Fix README and Makefile (#72) Marian Steinbach 2018-10-04 21:20:53 +0200
c50b6ae93d Remove wait time, as webdrive automatically waits Marian Steinbach 2018-10-04 09:35:20 +0200
621ead17b6 Add cookie collection Marian Steinbach 2018-10-04 09:34:47 +0200
c065da4957

More unittests for checks (#73) Marian Steinbach 2018-10-03 22:43:22 +0200
7f4424b4cb Move previous result assertion Marian Steinbach 2018-10-03 21:41:50 +0200
02cb5e43d9 Rename test function for clarity Marian Steinbach 2018-10-03 21:40:45 +0200
b6ed615e08 Add test for duplicate_content check Marian Steinbach 2018-10-03 21:39:42 +0200
4b7537fd17 Add test for domain_variations Marian Steinbach 2018-10-03 21:21:25 +0200
f6edfd2ce6 Add test for dns_resolution Marian Steinbach 2018-10-03 21:21:14 +0200
764a4647c8 Sort urls property for stable testing Marian Steinbach 2018-10-03 21:20:58 +0200
57f8dea4e0

Improve certificate check to support SNI (#71) Marian Steinbach 2018-10-03 21:01:52 +0200
bcfbac7198 Fix README and Makefile Marian Steinbach 2018-10-03 20:55:36 +0200
0ce57a026e Add commenting on the spider test Marian Steinbach 2018-10-03 20:48:20 +0200
ed4e73adbf Activate verbose output when running make test Marian Steinbach 2018-10-03 20:48:06 +0200
49e9f9ee46 Better tests for the certificate check Marian Steinbach 2018-10-03 20:47:45 +0200
ae6a2e83e9

Refactor and modularize spider (#70) Marian Steinbach 2018-10-03 11:05:42 +0200
580c6256a2 Set db entoty/kind back to production Marian Steinbach 2018-10-02 23:47:41 +0200
041ecaaefd Add simple unit test for spider.check_and_rate_site Marian Steinbach 2018-10-02 23:47:27 +0200
078a092b31 Change chrome_options to options due to deprecation warning Marian Steinbach 2018-10-02 23:40:47 +0200
d4f561045d Set python version in Travis config Marian Steinbach 2018-10-02 23:15:27 +0200
91af26f85b Fix Travis config Marian Steinbach 2018-10-02 23:04:44 +0200
af3e07bc60 Add codecov support Marian Steinbach 2018-10-02 22:52:59 +0200
021a2b6d20 Update make test target to work with new unit tests Marian Steinbach 2018-10-02 22:48:25 +0200
fd9584aac3 Move image name into variable Marian Steinbach 2018-10-02 22:47:59 +0200
024546968d Remove file (actually has been renamed) Marian Steinbach 2018-10-02 22:47:39 +0200
ce1cd1b99b Remove unused expand_hostname function from dns-resolution Marian Steinbach 2018-10-02 22:47:20 +0200
aa0273875d Modify Config to have default user agent for convenience Marian Steinbach 2018-10-02 22:47:02 +0200
c8b55e93ec Add unit test for charset checker Marian Steinbach 2018-10-02 22:46:41 +0200
7b2c3c15ff Modify charset check to assert page_content in results Marian Steinbach 2018-10-02 22:46:27 +0200
70290f0140 Add unit tests for certificate Marian Steinbach 2018-10-02 22:46:03 +0200
15aa722bfe Add unit tests for url_reachability Marian Steinbach 2018-10-02 22:45:37 +0200
2cebb3ff28 Mofivy url_reachabliity to remove URLs if status 404 Marian Steinbach 2018-10-02 22:45:18 +0200
cf9136b481 Add httpretty dependency Marian Steinbach 2018-10-02 22:33:12 +0200
dbbb6a24be Remove broken, outdated spider tests Marian Steinbach 2018-10-02 22:32:55 +0200
76f049ba4c Adapt Dockerfile, Makefile, script Marian Steinbach 2018-10-02 18:11:29 +0200
7b2c1fe7b5 Reorganize command line invocation Marian Steinbach 2018-10-02 18:10:10 +0200
b7b8e0feb4 New rater use_specific_fonts Marian Steinbach 2018-10-02 18:09:17 +0200
a4c155e16a Fix in rater no_network_errors Marian Steinbach 2018-10-02 18:09:02 +0200
0687414c9b Modify url_reachability to filter out domains like facebook.com, Marian Steinbach 2018-10-02 18:08:03 +0200
6497b54074 Modify load_in_browser to collect details on fonts used Marian Steinbach 2018-10-02 18:07:24 +0200
0539c4802c Add check 'certificate' to collect TLS cert details Marian Steinbach 2018-10-02 18:06:40 +0200
456272a2af Rename check responsive_layout to load_in_browser Marian Steinbach 2018-10-01 11:06:16 +0200
876e91bde6 Fix in datastore exception handling Marian Steinbach 2018-10-01 08:15:24 +0200
4d262859a5 Fix in response_duration rating Marian Steinbach 2018-10-01 08:15:03 +0200
afcb54374e New rating: no script errors Marian Steinbach 2018-10-01 08:14:42 +0200
1f9c764a66 New rating: no network errors Marian Steinbach 2018-10-01 08:14:32 +0200
92cfb4375c Make rating_modules a dict, to display that order is not important Marian Steinbach 2018-10-01 08:14:15 +0200
9612823b57 Fix in responsive_layout checker Marian Steinbach 2018-10-01 08:13:39 +0200
3e42e342a1 Fix in duplicate_content checker Marian Steinbach 2018-10-01 08:13:27 +0200
1c0c7539e2 Fix in charset checker Marian Steinbach 2018-10-01 08:13:12 +0200
93823577a5 Adapt data export Marian Steinbach 2018-09-29 01:38:34 +0200
cc20f48dad Fix in rating feeds Marian Steinbach 2018-09-29 01:38:28 +0200
57e1c20c08 Fix in rating canonical_url Marian Steinbach 2018-09-29 01:38:12 +0200
b8e54c576a responsive_layout: Add collection of browser logs Marian Steinbach 2018-09-29 01:37:50 +0200
8a5229861e Remove spider code no longer used Marian Steinbach 2018-09-28 22:49:28 +0200
d4b5695ae9 Introduce rating modules Marian Steinbach 2018-09-28 22:49:07 +0200
849aa1ad2a page_content check: lowercase headers, additional result attributes Marian Steinbach 2018-09-28 22:48:39 +0200
dd0406fd5f Add type field from directory entries to spider jobs Marian Steinbach 2018-09-28 20:30:41 +0200
2a6e02cf5d Change responsive_layout sizes and result format Marian Steinbach 2018-09-27 21:45:12 +0200
818e072756 Add check responsive_layout Marian Steinbach 2018-09-27 01:10:10 +0200
62033d587a Fetch viewport meta tag in html_head Marian Steinbach 2018-09-27 01:09:50 +0200
32e569a397 Little tweaks in generator check Marian Steinbach 2018-09-27 01:09:24 +0200
7f29091690 WIP commit Marian Steinbach 2018-09-27 00:34:54 +0200
d8f72f974e Add random index to spider jobs Marian Steinbach 2018-09-25 22:07:11 +0200
a2fd2ce4fb More commenting Marian Steinbach 2018-09-25 00:44:21 +0200
a2287388dd Start refactoring checks Marian Steinbach 2018-09-25 00:44:07 +0200
3e1c0ede73 Update Dockerfile Marian Steinbach 2018-09-24 21:19:54 +0200
2391207755 Move job handling into module Marian Steinbach 2018-09-24 21:19:47 +0200
73eff37110 Move configuration into module Marian Steinbach 2018-09-24 21:19:23 +0200
7514aeb542 Add date and time to spider result export Marian Steinbach 2018-09-19 23:04:09 +0200