Tom Parker-Shemilt
|
0bf87cf7be
|
Higher logging of download steps
|
2021-08-10 23:56:05 +01:00 |
|
Tom Parker-Shemilt
|
396b3b58b1
|
Fix no-username downloading
|
2021-08-10 23:53:32 +01:00 |
|
Tom Parker-Shemilt
|
a2d783ef80
|
Fix editorconfig issues
|
2021-08-10 23:51:34 +01:00 |
|
Tom Parker-Shemilt
|
e3116d0eac
|
Merge branch 'master' into popularity
|
2021-08-10 23:31:55 +01:00 |
|
Tom Parker-Shemilt
|
bbdc9eeb03
|
Remove all the unpopular items
|
2021-08-10 23:22:48 +01:00 |
|
William Desportes
|
8cc83c00b8
|
Editorconfig fixes
|
2021-08-08 20:57:44 +02:00 |
|
Tom Parker-Shemilt
|
cfaeb2ac88
|
Initial hacks towards popularity checks
|
2021-08-08 10:02:09 +01:00 |
|
Tom Parker-Shemilt
|
8cc705a14e
|
Merge branch 'master' into sorted-readme
|
2021-08-03 11:28:18 +01:00 |
|
Tom Parker-Shemilt
|
0513259ff9
|
Realised that the Github actions default to the default branch for a repo
(unlike Travis) and so the branch error isn't needed any more
|
2021-07-30 18:37:30 +01:00 |
|
Tom Parker-Shemilt
|
0491e2ce2d
|
Sort the README and enforce it with the checker
|
2021-07-25 20:44:10 +01:00 |
|
Tom Parker-Shemilt
|
90edcef35a
|
Ordering was in the wrong order...
|
2021-06-29 21:07:56 +01:00 |
|
Tom Parker-Shemilt
|
d3260a60d6
|
Reset some unneeded changes
|
2021-06-29 20:49:32 +01:00 |
|
Tom Parker-Shemilt
|
e3cd6c6a10
|
Check the oldest/unchecked ones first
|
2021-06-29 20:48:00 +01:00 |
|
Tom Parker-Shemilt
|
7bc2d0bb56
|
Don't retry 429's and don't worry about them if we've got a previous success
|
2021-06-24 16:28:08 +01:00 |
|
Tom Parker-Shemilt
|
64d605e723
|
Fix all the other issues with README
|
2021-06-13 16:07:34 +01:00 |
|
Tom Parker-Shemilt
|
80e35f47c3
|
Upgrade almost all the deps
|
2021-05-05 22:21:23 +01:00 |
|
Tom Parker-Shemilt
|
b49400cd8b
|
Kill all the HTML images in favour of Markdown
|
2020-10-09 09:37:09 +01:00 |
|
Tom Parker-Shemilt
|
722227efed
|
Update the minimum times between checks to reduce build issues
|
2020-07-24 08:20:26 +01:00 |
|
Tom Parker-Shemilt
|
3b592dae23
|
Simplified handles with tokio Semaphore
|
2020-06-03 18:05:12 +01:00 |
|
Tom Parker-Shemilt
|
4b7e6d1488
|
Get handle once per recursive sequence of get_url calls
|
2020-06-03 17:57:07 +01:00 |
|
Tom Parker-Shemilt
|
5e9357ed6f
|
Swap back to older reqwest API
|
2020-06-02 23:02:18 +01:00 |
|
Tom Parker-Shemilt
|
87ad03cb15
|
Remove handle debug because it overloads travis
|
2020-06-02 22:40:28 +01:00 |
|
Tom Parker-Shemilt
|
a1abde30b9
|
Upgrade packages and change over deprecated Reqwest function
|
2020-06-02 22:32:16 +01:00 |
|
Tom Parker-Shemilt
|
c680c63f8a
|
Disable connection pooling, as it seems to break stuff
|
2020-06-02 22:25:01 +01:00 |
|
Tom Parker-Shemilt
|
25084b928a
|
Various debug for checks
|
2020-06-02 22:24:49 +01:00 |
|
Tom Parker-Shemilt
|
1f6cb3749b
|
Panic earlier on 404'ed links
|
2020-05-01 00:22:48 +01:00 |
|
Tom Parker-Shemilt
|
6532f7681f
|
Only do Github API rewrite if have environment variables
|
2020-04-30 22:04:01 +01:00 |
|
Tom Parker-Shemilt
|
2dfb07854f
|
Merge pull request #867 from palfrey/add-github-secrets
Support optional use of Github secrets to workaround rate limits
|
2020-04-25 18:26:03 +01:00 |
|
Tom Parker-Shemilt
|
1efa0de042
|
Missed two other cases of needing to print url with link
|
2020-04-21 11:18:53 +01:00 |
|
Tom Parker-Shemilt
|
e953dbb269
|
Print url before Link
|
2020-04-21 11:04:59 +01:00 |
|
Tom Parker-Shemilt
|
0e3bada631
|
Support optional use of Github secrets to workaround rate limits (fixes #859)
|
2020-04-03 22:10:09 +01:00 |
|
Tom Parker-Shemilt
|
15dbb3602b
|
For most errors, intermittent failures are ok, not so much for redirects
|
2020-04-03 21:12:54 +01:00 |
|
Tom Parker-Shemilt
|
60e614f51e
|
Nicer "failure only occured X ago" message
|
2020-03-26 23:29:34 +00:00 |
|
Tom Parker-Shemilt
|
d4b44fd6d5
|
Don't remove last_working if a link fails
|
2020-03-26 23:22:40 +00:00 |
|
Tom Parker-Shemilt
|
12464c2844
|
Redo (not-)working with a enum, not a pair of variables
|
2020-03-10 21:17:59 +00:00 |
|
Tom Parker-Shemilt
|
592a4bed92
|
Output all the info on link failure
|
2020-03-08 16:49:14 +00:00 |
|
Tom Parker-Shemilt
|
edc17508e1
|
Only write every so often
|
2020-03-08 16:49:04 +00:00 |
|
Tom Parker-Shemilt
|
2901a666cb
|
Merge branch 'master' into cache-results
|
2020-03-03 23:10:02 +00:00 |
|
Tom Parker-Shemilt
|
f3ac7ba5d3
|
Merge pull request #840 from palfrey/move-cleanup
Move cleanup into bin folder
|
2020-03-03 23:08:37 +00:00 |
|
Tom Parker-Shemilt
|
0d798c3896
|
Github workflows have started to do 406, unless we add an explicit image to the Accept header
|
2020-02-26 21:26:18 +00:00 |
|
Tom Parker-Shemilt
|
43eab5fa9a
|
Merge branch 'master' into better-unknown-travis-detection
|
2020-02-25 21:26:29 +00:00 |
|
Tom Parker-Shemilt
|
9d41a10148
|
Cache the results of getting links across Travis build and only re-check
every so often
|
2020-02-24 23:41:45 +00:00 |
|
Tom Parker-Shemilt
|
d649b575a7
|
Update cleanup instructions
|
2020-02-24 22:38:19 +00:00 |
|
Tom Parker-Shemilt
|
e6520cbac0
|
Move cleanup to bin folder and run it on the repo
|
2020-02-24 22:36:50 +00:00 |
|
Tom Parker-Shemilt
|
ecd6658ec6
|
Improve detection of "unknown" travis builds
|
2020-02-16 20:44:45 +00:00 |
|
Tom Parker-Shemilt
|
0a527ff6e3
|
Catch no-branch cases for Travis
|
2020-02-12 23:38:26 +00:00 |
|
Tom Parker-Shemilt
|
5bceef3656
|
Catch github actions badges with no branch
|
2020-02-12 23:32:02 +00:00 |
|
Tom Parker-Shemilt
|
89efda92e3
|
Azure devops does 302 twice, so need to cope with that
|
2020-02-10 21:02:28 +00:00 |
|
Tom Parker-Shemilt
|
4d048eba27
|
Add workaround for Youtube 302s (fixes #814)
|
2020-01-30 15:23:42 +00:00 |
|
Tom Parker-Shemilt
|
dec99f4416
|
Also look for travis.org as well as .com
|
2020-01-20 18:05:20 +00:00 |
|
Tom Parker-Shemilt
|
f0af98f3fc
|
Detect unknown status travis builds
|
2020-01-20 17:53:24 +00:00 |
|
Tom Parker-Shemilt
|
7ea1063c32
|
Update user-agent so Reddit stops complaining
|
2020-01-20 13:50:02 +00:00 |
|
Tom Parker-Shemilt
|
6cb9433de0
|
Look for links/images in HTML content in the Markdown
|
2020-01-18 14:01:05 +00:00 |
|
Tom Parker-Shemilt
|
a37b1cfee3
|
Redo error checking with failure and less verbose formatting
|
2020-01-17 23:24:03 +00:00 |
|
Tom Parker-Shemilt
|
5733288c9a
|
Record actual real errors
|
2020-01-13 22:21:03 +00:00 |
|
Tom Parker-Shemilt
|
f799381f75
|
Add 20 second timeout
|
2020-01-13 22:13:13 +00:00 |
|
Tom Parker-Shemilt
|
c25072ec06
|
Add workaround for github actions
|
2020-01-13 22:13:00 +00:00 |
|
Tom Parker-Shemilt
|
ff703bac03
|
Fix accept header so we also get github svgs
|
2020-01-13 21:53:25 +00:00 |
|
Tom Parker-Shemilt
|
a6b79d73ec
|
Also pick up image tags
|
2020-01-13 17:04:46 +00:00 |
|
Tom Parker-Shemilt
|
fd843a5dd8
|
Add extra debug on redirects
|
2020-01-13 16:17:24 +00:00 |
|
Tom Parker-Shemilt
|
39b2e4b3b6
|
Add text/hmtl header to fix crates.io
|
2020-01-13 16:07:48 +00:00 |
|
Tom Parker-Shemilt
|
70361602dd
|
Detect redirects
|
2020-01-13 08:39:38 +00:00 |
|
Tom Parker-Shemilt
|
67bfa13da5
|
Retry up to 5 times just in case on failure
|
2020-01-11 14:45:49 +00:00 |
|
Tom Parker-Shemilt
|
f6e1302d17
|
Limit running gets to 20 to stop issues with file handles
|
2020-01-11 12:06:40 +00:00 |
|
Tom Parker-Shemilt
|
57a7f96ba5
|
Start writing a rust-based url checker
|
2020-01-10 23:07:36 +00:00 |
|
kud1ing
|
0d11f0b0c5
|
remove
|
2017-05-03 18:46:46 +02:00 |
|
kud1ing
|
4b22d82e49
|
merge the deployment sections
|
2017-04-29 10:00:25 +02:00 |
|
kud1ing
|
2978c44acc
|
add section 'Text search'
|
2016-12-28 17:32:40 +01:00 |
|
kud1ing
|
d0d937e00a
|
collect the footer Markdown events
|
2016-12-10 21:49:04 +01:00 |
|
kud1ing
|
daae790834
|
flesh out the model some more
|
2016-12-10 20:59:18 +01:00 |
|
kud1ing
|
76bbf8d6d2
|
refactor
|
2016-12-10 20:47:16 +01:00 |
|
kud1ing
|
b9996a01cd
|
cleanup
|
2016-12-10 20:38:31 +01:00 |
|
kud1ing
|
98d01abf50
|
more high-level parsing
|
2016-12-10 20:32:42 +01:00 |
|
kud1ing
|
21f8b53241
|
cleanup
|
2016-12-10 19:23:48 +01:00 |
|
kud1ing
|
eb15075946
|
add a parser stub for the README.md file
|
2016-12-10 18:07:07 +01:00 |
|