repo.hyron.dev crawl failure with empty error log

[Ilya Zlobintsev]

Hi, I have an issue with the repo.hyron.dev mirror (host id 3010). It is currently marked
as inactive due to consecutive crawl failures, but the crawl log is empty, and I'm not
sure what the problem is. There was a period of downtime 4 days ago, but it's been up
since then, and the content is available under the normal URLs:
https://repo.hyron.dev/fedora/linux/releases/40/Workstation/x86_64/iso/
https://repo.hyron.dev/fedora/linux/updates/40/Everything/x86_64/Packages/a/

[Jing Luo]

On 2024-06-03 17:55, Ilya Zlobintsev via Mirror-admin wrote:
>  Hi, I have an issue with the repo.hyron.dev mirror (host id 3010).
> It
>  is currently marked as inactive due to consecutive crawl failures, but
>  the crawl log is empty, and I'm not sure what the problem is. There
>  was a period of downtime 4 days ago, but it's been up since then, and
>  the content is available under the normal URLs:
>  https://repo.hyron.dev/fedora/linux/releases/40/Workstation/x86_64/iso/
>  https://repo.hyron.dev/fedora/linux/updates/40/Everything/x86_64/Packages/a/ 
Hi. I think the crawler log was just rotated. AFAIK you can just enable 
your site again.

On the other hand, I had a similar issue back in March: repo.jing.rocks 
suffered from a kernel lockup caused by NFS for half a day, and fedora's 
mirror manager just silently disabled the mirror, only then I found out 
nearly a month later. When I filled out the form in mirror manager, the 
"email" field says the email will be used for robot notifications, 
however I've never got any email from bots.

Can fedora please not *silently* disable mirror, not notifying admins? 
repo.jing.rocks mirrors so many distros that I can't check one by one if 
it was disabled or not.

-- 
Jing Luo
About me: https://jing.rocks/about/
PGP Fingerprint: 4E09 8D19 00AA 3F72 1899 2614 09B3 316E 13A1 1EFC

[Aurelien Bompard]

>  Can fedora please not *silently* disable mirror, not notifying
> admins? 
>  repo.jing.rocks mirrors so many distros that I can't check one by one if 
>  it was disabled or not. 
Agreed, I've opened an enhancement request on MirrorManager about this:
https://github.com/fedora-infra/mirrormanager2/issues/342

[Aurelien Bompard]

Hello! I retried the crawl on this mirror and saw this error:

--------
2024-06-03 13:05:05 INFO     repo.hyron.dev (3010): Server load exceeded on
https://repo.hyron.dev/fedora/linux/releases/38/Spins/x86_64/iso - trying again in 0.3s
(after 1 tries)
2024-06-03 13:05:16 INFO     repo.hyron.dev (3010): Server load exceeded on
https://repo.hyron.dev/fedora/linux/releases/38/Spins/x86_64/iso - trying again in 0.8s
(after 2 tries)
2024-06-03 13:05:31 INFO     repo.hyron.dev (3010): Server load exceeded on
https://repo.hyron.dev/fedora/linux/releases/38/Spins/x86_64/iso - trying again in 1.4s
(after 3 tries)
2024-06-03 13:05:47 INFO     repo.hyron.dev (3010): Server load exceeded on
https://repo.hyron.dev/fedora/linux/releases/38/Spins/x86_64/iso - trying again in 0.8s
(after 4 tries)
2024-06-03 13:05:58 INFO     repo.hyron.dev (3010): Server load exceeded on
https://repo.hyron.dev/fedora/linux/releases/38/Spins/x86_64/iso - trying again in 9.8s
(after 5 tries)
2024-06-03 13:06:18 INFO     repo.hyron.dev (3010): Server load exceeded on
https://repo.hyron.dev/fedora/linux/releases/38/Spins/x86_64/iso - trying again in 23.2s
(after 6 tries)
2024-06-03 13:07:10 INFO     repo.hyron.dev (3010): Server load exceeded on
https://repo.hyron.dev/fedora/linux/releases/38/Spins/x86_64/iso - trying again in 11.9s
(after 7 tries)
2024-06-03 13:07:32 INFO     repo.hyron.dev (3010): Server load exceeded on
https://repo.hyron.dev/fedora/linux/releases/38/Spins/x86_64/iso - trying again in 107.1s
(after 8 tries)
2024-06-03 13:09:30 INFO     repo.hyron.dev (3010): Server load exceeded on
https://repo.hyron.dev/fedora/linux/releases/38/Spins/x86_64/iso - trying again in 247.6s
(after 9 tries)
2024-06-03 13:13:49 INFO     repo.hyron.dev (3010): Server load exceeded on
https://repo.hyron.dev/fedora/linux/releases/38/Spins/x86_64/iso - giving up after 10
tries
--------

The "Server load exceeded" message is logged when there's a HTTP timeout.
There's been many successful requests before that, a few that had to be retried but
only once or twice. Do you see something relevant in your logs on those timestamps?

Thanks