New EPEL Mirror

[Morten Stevens]

Hello,

My name is Morten Stevens and my company offers a new EPEL Mirror.

http://mirror.imt-systems.com/epel/
IP: 81.30.145.134
Bandwidth: 1000 Mbit
Location: Germany
Company: IMT-Systems GmbH

The Mirror Manager log shows:

Directory pub/epel/4/i386/debug/headers is not up-to-date on this host.
Directory pub/epel/testing/4/i386/headers is not up-to-date on this 
host.
Directory pub/epel/4/i386/headers is not up-to-date on this host.
Directory pub/epel/4/SRPMS/headers is not up-to-date on this host.
Directory pub/epel/testing/4/ppc/headers is not up-to-date on this 
host.
Directory pub/epel/4/x86_64/headers is not up-to-date on this host.
Directory pub/epel/testing/4/ppc/debug/headers is not up-to-date on 
this host.
Directory pub/epel/testing/4/x86_64/headers is not up-to-date on this 
host.
Directory pub/epel/4/x86_64/debug/headers is not up-to-date on this 
host.
Directory pub/epel/4/ppc/debug/headers is not up-to-date on this host.
Directory pub/epel/testing/4/x86_64/debug/headers is not up-to-date on 
this host.
Directory pub/epel/testing/4/i386/debug/headers is not up-to-date on 
this host.
Directory pub/epel/4/ppc/headers is not up-to-date on this host.
Directory pub/epel/testing/4/SRPMS/headers is not up-to-date on this 
host.

This is strange, because we are syncing from 
rsync://dl.fedoraproject.org/fedora-epel

Nevertheless, everything seems to be okay?

Best regards,

Morten

[Matt Domsch]

On Tue, Aug 21, 2012 at 06:25:20PM -0500, Morten Stevens wrote:
> Hello,
>
> My name is Morten Stevens and my company offers a new EPEL Mirror.
>
> http://mirror.imt-systems.com/epel/
> IP: 81.30.145.134
> Bandwidth: 1000 Mbit
> Location: Germany
> Company: IMT-Systems GmbH
>
> The Mirror Manager log shows:
>
> Directory pub/epel/4/i386/debug/headers is not up-to-date on this
> host.

Thanks for the report.  Here's the MirrorManager fix I'll need to
deploy to production.  It shouldn't have any effect on client
operation though, as up2date doesn't request header files from MM so
deep in the structure I believe.

commit 5d61bb4a1f90ae37efadff590b9eb407ac69ca6b
Author: Matt Domsch 
Date:   Wed Aug 22 09:46:45 2012 -0500

crawler: handle zero-length files

The crawler, when doing HTTP HEAD calls to retrieve info about a file,
    gets its content length from the HTTP header.  If a file is
    zero-length, the HTTP server may not return a Content-Length header.
    This was causing the crawler to believe the file (and thus its
    directory) was not up-to-date, even though it was up-to-date.

Fix is to check that content-length value is present, and if not
    present, don't rely upon it for determining up-to-date-edness.

diff --git a/server/crawler_perhost b/server/crawler_perhost
index b8027d2..81bdab0 100755
--- a/server/crawler_perhost
+++ b/server/crawler_perhost
@@ -275,7 +275,7 @@ def check_head(hoststate, url, filedata, recursion, readable, retry=0):

if status >= 200 and status < 300:
         # fixme should check last_modified too
-        if filedata['size'] == content_length:
+        if filedata['size'] == content_length or content_length is None: # handle no content-length header, streaming/chunked return or zero-length file
             return True
         else:
             return False