For several days, my internal private mirror has failed to run report_mirror:
./report_mirror -c report_mirror.conf Error checking in: Service Temporarily Unavailable. Please try again later.
Known issue?
On Fri, May 31, 2024 at 11:42:26AM GMT, Chris Schanzle via Mirror-admin wrote:
For several days, my internal private mirror has failed to run report_mirror:
./report_mirror -c report_mirror.conf Error checking in: Service Temporarily Unavailable. Please try again later.
Known issue?
Yeah:
https://pagure.io/fedora-infrastructure/issue/11955
We just moved mirrormanager from rhel7/python2 to openshift, and this apparently broke along the way. ;(
Hopefully we can get it sorted soon... sorry for the trouble
kevin
Yeah, that's my bad, I moved the instance URL to be similar to other apps and setup a redirect from admin.fedoraproject.org/mirrormanager to mirrormanager.fedoraproject.org, but forgot about the XML-RPC endpoint that only listens to POST queries, and those don't follow redirects.
The solution is to edit /etc/mirrormanager-client/report_mirror.conf and change the server= line at the beginning to: server=https://mirrormanager.fedoraproject.org/xmlrpc
I'll update the RPM as soon as possible, but the config file will not be updated by the rpm update alone as you've most likely changed it from its default state by adding your mirror definition.
Sorry for the trouble!
There's been a very big mirrormanager update, if you're experiencing other troubles please report them here or on https://pagure.io/fedora-infrastructure/ and I'll have a look at them asap.
Thanks
Aurelien Bompard abompard@fedoraproject.org writes:
There's been a very big mirrormanager update, if you're experiencing other troubles please report them here or on https://pagure.io/fedora-infrastructure/ and I'll have a look at them asap.
So after reading about report_mirror still being expected to work, I wonder if the information I had previously received about it no longer being useful has been superseded.
quick-fedora-mirror used to have code that would duplicate the functionality of report_mirror, but it was disabled because I was told that there was no point in running and in fact it was harmful to do so due to additional server load, and that I should simply rely on the crawler to keep things updated.
Are things different now? Should I restore that code and start using it? It simply built a payload in the expected format and call curl.
- J<
On Mon, Jun 03, 2024 at 12:05:17PM GMT, Jason L Tibbitts III wrote:
Aurelien Bompard abompard@fedoraproject.org writes:
There's been a very big mirrormanager update, if you're experiencing other troubles please report them here or on https://pagure.io/fedora-infrastructure/ and I'll have a look at them asap.
So after reading about report_mirror still being expected to work, I wonder if the information I had previously received about it no longer being useful has been superseded.
quick-fedora-mirror used to have code that would duplicate the functionality of report_mirror, but it was disabled because I was told that there was no point in running and in fact it was harmful to do so due to additional server load, and that I should simply rely on the crawler to keep things updated.
Are things different now? Should I restore that code and start using it? It simply built a payload in the expected format and call curl.
My understanding was that it was not useful for public mirrors, but was still useful for private mirrors, since private mirrors cannot be crawled to see if they are up to date or not.
However, I am unsure what exactly we do with that information.
I'll defer to one of the folks working on the code here...
kevin
On Tue, Jun 04, 2024 at 12:40:35PM -0700, Kevin Fenzi wrote:
On Mon, Jun 03, 2024 at 12:05:17PM GMT, Jason L Tibbitts III wrote:
> Aurelien Bompard abompard@fedoraproject.org writes:
There's been a very big mirrormanager update, if you're experiencing other troubles please report them here or on https://pagure.io/fedora-infrastructure/ and I'll have a look at them asap.
So after reading about report_mirror still being expected to work, I wonder if the information I had previously received about it no longer being useful has been superseded.
quick-fedora-mirror used to have code that would duplicate the functionality of report_mirror, but it was disabled because I was told that there was no point in running and in fact it was harmful to do so due to additional server load, and that I should simply rely on the crawler to keep things updated.
Are things different now? Should I restore that code and start using it? It simply built a payload in the expected format and call curl.
My understanding was that it was not useful for public mirrors, but was still useful for private mirrors, since private mirrors cannot be crawled to see if they are up to date or not.
However, I am unsure what exactly we do with that information.
I'll defer to one of the folks working on the code here...
It was disabled for non-private mirrors as it does not provide really correct results. For private mirrors it is currently the only way to let MirrorManager know which directories the private mirror has.
Adrian
On Wed, Jun 05, 2024 at 11:04:19AM GMT, Adrian Reber via Mirror-admin wrote:
On Tue, Jun 04, 2024 at 12:40:35PM -0700, Kevin Fenzi wrote:
On Mon, Jun 03, 2024 at 12:05:17PM GMT, Jason L Tibbitts III wrote:
>> Aurelien Bompard abompard@fedoraproject.org writes:
There's been a very big mirrormanager update, if you're experiencing other troubles please report them here or on https://pagure.io/fedora-infrastructure/ and I'll have a look at them asap.
So after reading about report_mirror still being expected to work, I wonder if the information I had previously received about it no longer being useful has been superseded.
quick-fedora-mirror used to have code that would duplicate the functionality of report_mirror, but it was disabled because I was told that there was no point in running and in fact it was harmful to do so due to additional server load, and that I should simply rely on the crawler to keep things updated.
Are things different now? Should I restore that code and start using it? It simply built a payload in the expected format and call curl.
My understanding was that it was not useful for public mirrors, but was still useful for private mirrors, since private mirrors cannot be crawled to see if they are up to date or not.
However, I am unsure what exactly we do with that information.
I'll defer to one of the folks working on the code here...
It was disabled for non-private mirrors as it does not provide really correct results. For private mirrors it is currently the only way to let MirrorManager know which directories the private mirror has.
Ah yes, it does need to know which directories the private mirror has.
I wonder if we couldn't add this to the web interface? when you setup your private mirror, just add the directories there and adjust them over time if they change? Just a thought...
kevin
On Wed, Jun 05, 2024 at 07:57:34AM -0700, Kevin Fenzi wrote:
On Wed, Jun 05, 2024 at 11:04:19AM GMT, Adrian Reber via Mirror-admin wrote:
On Tue, Jun 04, 2024 at 12:40:35PM -0700, Kevin Fenzi wrote:
On Mon, Jun 03, 2024 at 12:05:17PM GMT, Jason L Tibbitts III wrote:
>>> Aurelien Bompard abompard@fedoraproject.org writes:
There's been a very big mirrormanager update, if you're experiencing other troubles please report them here or on https://pagure.io/fedora-infrastructure/ and I'll have a look at them asap.
So after reading about report_mirror still being expected to work, I wonder if the information I had previously received about it no longer being useful has been superseded.
quick-fedora-mirror used to have code that would duplicate the functionality of report_mirror, but it was disabled because I was told that there was no point in running and in fact it was harmful to do so due to additional server load, and that I should simply rely on the crawler to keep things updated.
Are things different now? Should I restore that code and start using it? It simply built a payload in the expected format and call curl.
My understanding was that it was not useful for public mirrors, but was still useful for private mirrors, since private mirrors cannot be crawled to see if they are up to date or not.
However, I am unsure what exactly we do with that information.
I'll defer to one of the folks working on the code here...
It was disabled for non-private mirrors as it does not provide really correct results. For private mirrors it is currently the only way to let MirrorManager know which directories the private mirror has.
Ah yes, it does need to know which directories the private mirror has.
I wonder if we couldn't add this to the web interface? when you setup your private mirror, just add the directories there and adjust them over time if they change? Just a thought...
For high level directories that sounds doable, but if you are excluding certain arches it might be a lot of work to add everything to the web interface. Especially with a new release every 6 months it would need to be updated. Supporting regex would be a way around this.
One private mirror using a caching system asked in a ticket to turn on "always up to date" for his mirror. That is something we could open to private mirrors, but also not useful if you want to exclude certain things.
Adrian
On 05.06.2024 17:05, Adrian Reber via Mirror-admin wrote:
On Wed, Jun 05, 2024 at 07:57:34AM -0700, Kevin Fenzi wrote:
On Wed, Jun 05, 2024 at 11:04:19AM GMT, Adrian Reber via Mirror-admin wrote:
On Tue, Jun 04, 2024 at 12:40:35PM -0700, Kevin Fenzi wrote:
On Mon, Jun 03, 2024 at 12:05:17PM GMT, Jason L Tibbitts III wrote:
>>>> Aurelien Bompard abompard@fedoraproject.org writes:
There's been a very big mirrormanager update, if you're experiencing other troubles please report them here or on https://pagure.io/fedora-infrastructure/ and I'll have a look at them asap.
So after reading about report_mirror still being expected to work, I wonder if the information I had previously received about it no longer being useful has been superseded.
quick-fedora-mirror used to have code that would duplicate the functionality of report_mirror, but it was disabled because I was told that there was no point in running and in fact it was harmful to do so due to additional server load, and that I should simply rely on the crawler to keep things updated.
Are things different now? Should I restore that code and start using it? It simply built a payload in the expected format and call curl.
My understanding was that it was not useful for public mirrors, but was still useful for private mirrors, since private mirrors cannot be crawled to see if they are up to date or not.
However, I am unsure what exactly we do with that information.
I'll defer to one of the folks working on the code here...
It was disabled for non-private mirrors as it does not provide really correct results. For private mirrors it is currently the only way to let MirrorManager know which directories the private mirror has.
Ah yes, it does need to know which directories the private mirror has.
I wonder if we couldn't add this to the web interface? when you setup your private mirror, just add the directories there and adjust them over time if they change? Just a thought...
For high level directories that sounds doable, but if you are excluding certain arches it might be a lot of work to add everything to the web interface. Especially with a new release every 6 months it would need to be updated. Supporting regex would be a way around this.
Is there perhaps an opportunity to amend mirrormanager to instead of just doing via web ui, to expose a more full featured API that allows mirrors to manage their mirror content/state remotely?
I've had an idea for a while to add a REST-ish API to manage mirror content, as it would be extraordinarily useful for several projects that need to be able to manipulate site/host metadata.
--Neil
One private mirror using a caching system asked in a ticket to turn on "always up to date" for his mirror. That is something we could open to private mirrors, but also not useful if you want to exclude certain things.
Adrian-- _______________________________________________ Mirror-admin mailing list -- mirror-admin@lists.fedoraproject.org To unsubscribe send an email to mirror-admin-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/mirror-admin@lists.fedoraproje... Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Adrian Reber via Mirror-admin mirror-admin@lists.fedoraproject.org writes:
It was disabled for non-private mirrors as it does not provide really correct results.
OK, that makes sense. Though I'll reiterate that if there was a way to provide correct results, I would be happy to at least attempt to do so. It makes a lot of sense to do it immediately after new content has been downloaded, but of course that might mean a lot of hosts hammering the server immediately after a new big content push. Surely it has to be more efficient to make the client do the work to create a nice compressed payload instead of parsing rsync directory listings.
- J<
On Wed, Jun 05, 2024 at 12:59:03PM GMT, Jason L Tibbitts III wrote:
Adrian Reber via Mirror-admin mirror-admin@lists.fedoraproject.org writes:
It was disabled for non-private mirrors as it does not provide really correct results.
OK, that makes sense. Though I'll reiterate that if there was a way to provide correct results, I would be happy to at least attempt to do so. It makes a lot of sense to do it immediately after new content has been downloaded, but of course that might mean a lot of hosts hammering the server immediately after a new big content push. Surely it has to be more efficient to make the client do the work to create a nice compressed payload instead of parsing rsync directory listings.
yeah, but... the problem was this: mirrors would run a script of some kind to sync content, it would fail for whatever reason, but they called report_mirror at the end unconditionally.
So, then the mirror would be marked up to date, the crawler would remove it for being out of date, then the script would mark it up to date again, repeat.
kevin
mirror-admin@lists.fedoraproject.org
-
Adrian Reber -
Aurelien Bompard -
Chris Schanzle -
Jason L Tibbitts III -
Kevin Fenzi -
Neil Hanlon