Is there any current work on yum-presto, deltarpm or some other method of getting the downloads for updates to be smaller and quicker? Can you compress rpm's to make the the downloads faster?
Currently with yum presto there is only one site that I have found that has the deltarpms.
https://fedorahosted.org/presto
Currently now on a fresh install of F9 && update
Transaction Summary ============================================================================= Install 14 Package(s) Update 185 Package(s) Remove 0 Package(s)
Total download size: 475 M
Half a gig is a lot (181/199): nautilus-2.22.3-1.fc9.x86_64.rpm | 5.0 MB 00:42 (182/199): rhythmbox-0.11.5-12.fc9.x86_64.rpm | 5.3 MB 00:42 (183/199): gtk2-2.12.10-2.fc9.x86_64.rpm | 7.1 MB 00:52 (184/199): libpurple-2.4.2-1.fc9.x86_64.rpm | 7.4 MB 00:53 (185/199): libgweather-2.22.2-1.fc9.x86_64.rpm | 7.5 MB 00:47 (186/199): dasher-4.9.0-1.fc9.x86_64.rpm | 7.7 MB 00:57 (187/199): openoffice.org-calc-2.4.1-17.3.fc9.x86_64.rpm | 8.3 MB 01:03 (188/199): samba-client-3.2.0-1.rc1.15.fc9.x86_64.rpm | 9.6 MB 01:43 (189/199): gnome-applets-2.22.2-1.fc9.x86_64.rpm | 9.6 MB 01:29 (190/199): evolution-2.22.2-2.fc9.x86_64.rpm | 13 MB 01:32 (191/199): perl-5.10.0-22.fc9.x86_64.rpm | 14 MB 02:12 (192/199): gnome-user-docs-2.22.1-1.fc9.noarch.rpm | 16 MB 01:55 (193/199): samba-common-3.2.0-1.rc1.15.fc9.x86_64.rpm | 16 MB 01:53 (194/199): kernel-2.6.25.4-30.fc9.x86_64.rpm | 18 MB 02:14 (195/199): gnome-games-2.22.2.1-1.fc9.x86_64.rpm | 19 MB 02:30 (196/199): java-1.6.0-openjdk-1.6.0.0-0.13.b09.fc9.x86_6 | 27 MB 04:12 (197/199): frysk-0.4-0.fc9.x86_64.rpm | 28 MB 03:29 (198/199): evolution-help-2.22.2-2.fc9.x86_64.rpm | 46 MB 03:44
(199/199): openoffice.org-core-2.4.1-17.3.fc9.x86_64.rpm | 84 MB 08:07
Large rpm's are very time consuming to update and/or install.
Guess I answered my own question earlier, I ran gzip on openopfice.org-core and it went from 84MB to 83MB.
Fairly loaded question, don't say get a better ISP, I have a very fast connection at home, and this d/l view was taken at work with an extremely fat pipe. The problem is size of rpm's being downloaded.
Debian seems to d/l much faster, not exact version but you can get some idea size wise.
http://packages.debian.org/etch/openoffice.org-core
wget http://security.debian.org/debian-security/pool/updates/main/o/openoffice.or...
There package is 34MB.
This is an example of one of the larger apps out there I realize that, I just grabbed the last one to use as an example. At 34MB it still takes time, but .deb is 50MB cheaper that the .rpm.
Is deltarpm's the solution? Has anyone talked to all the fedora mirrors to have a deltarpm repository?
On Thu, Jun 12, 2008 at 5:42 PM, Justin Conover justin.conover@gmail.com wrote:
Guess I answered my own question earlier, I ran gzip on openopfice.org-core and it went from 84MB to 83MB.
rpms are already compressed.
Fairly loaded question, don't say get a better ISP, I have a very fast connection at home, and this d/l view was taken at work with an extremely fat pipe. The problem is size of rpm's being downloaded.
Debian seems to d/l much faster, not exact version but you can get some idea size wise.
http://packages.debian.org/etch/openoffice.org-core
wget http://security.debian.org/debian-security/pool/updates/main/o/openoffice.or...
There package is 34MB.
This is an example of one of the larger apps out there I realize that, I just grabbed the last one to use as an example. At 34MB it still takes time, but .deb is 50MB cheaper that the .rpm.
you are comparing two different packages (2.0.4 and 2.4)
On Thu, Jun 12, 2008 at 10:54 AM, drago01 drago01@gmail.com wrote:
On Thu, Jun 12, 2008 at 5:42 PM, Justin Conover justin.conover@gmail.com wrote:
Guess I answered my own question earlier, I ran gzip on
openopfice.org-core
and it went from 84MB to 83MB.
rpms are already compressed.
Fairly loaded question, don't say get a better ISP, I have a very fast connection at home, and this d/l view was taken at work with an extremely fat pipe. The problem is size of rpm's being downloaded.
Debian seems to d/l much faster, not exact version but you can get some
idea
size wise.
http://packages.debian.org/etch/openoffice.org-core
wget
http://security.debian.org/debian-security/pool/updates/main/o/openoffice.or...
There package is 34MB.
This is an example of one of the larger apps out there I realize that, I just grabbed the last one to use as an example. At 34MB it still takes time, but .deb is 50MB cheaper that the .rpm.
you are comparing two different packages (2.0.4 and 2.4)
Here 38MB, still close to 50MB cheaper.
http://http.us.debian.org/debian/pool/main/o/openoffice.org/openoffice.org-c...
http://packages.debian.org/sid/amd64/openoffice.org-core/download
On Thu, 2008-06-12 at 17:54 +0200, drago01 wrote:
On Thu, Jun 12, 2008 at 5:42 PM, Justin Conover justin.conover@gmail.com wrote: At 34MB it still takes
time, but .deb is 50MB cheaper that the .rpm.
you are comparing two different packages (2.0.4 and 2.4)
There's a few other problems with a direct comparison of sizes because they are two packages called "core" with somewhat differing content. The Debian "core" package depends on a "common" package which is an additional 27megs in size. The content of both of these is included in the single Fedora "core" rpm. Additionally the default help content is included in the Fedora "core" rpm, which is available in the deb packages as help-en_US, which is another additional 11 megs.
Additionally displaying help itself requires the use of the core writer libraries to render the html help, in Debian this means that the "writer" package is a dependency of help, and that's an additional 6megs in size in its .deb. While in Fedora writer is split into the optional bits called "writer" and the core required for use by help. Shrinking the "writer" rpm by approx 3 megs, and inflating the "core" one by the same.
To manually extract the various contents for side-by-side comparison you can use
rpm2cpio something.rpm | cpio -ivd vs ar x something.rpm tar xzf data.tar.gz
C.
On Thu, Jun 12, 2008 at 12:40 PM, Caolan McNamara caolanm@redhat.com wrote:
On Thu, 2008-06-12 at 17:54 +0200, drago01 wrote:
On Thu, Jun 12, 2008 at 5:42 PM, Justin Conover justin.conover@gmail.com wrote: At 34MB it still takes
time, but .deb is 50MB cheaper that the .rpm.
you are comparing two different packages (2.0.4 and 2.4)
There's a few other problems with a direct comparison of sizes because they are two packages called "core" with somewhat differing content. The Debian "core" package depends on a "common" package which is an additional 27megs in size. The content of both of these is included in the single Fedora "core" rpm. Additionally the default help content is included in the Fedora "core" rpm, which is available in the deb packages as help-en_US, which is another additional 11 megs.
Additionally displaying help itself requires the use of the core writer libraries to render the html help, in Debian this means that the "writer" package is a dependency of help, and that's an additional 6megs in size in its .deb. While in Fedora writer is split into the optional bits called "writer" and the core required for use by help. Shrinking the "writer" rpm by approx 3 megs, and inflating the "core" one by the same.
To manually extract the various contents for side-by-side comparison you can use
rpm2cpio something.rpm | cpio -ivd vs ar x something.rpm tar xzf data.tar.gz
C.
-- fedora-test-list mailing list fedora-test-list@redhat.com To unsubscribe: https://www.redhat.com/mailman/listinfo/fedora-test-list
Ok, everyone in this thread is replying to OOO, that was not my intent. If you compare the speed of getting updates in debian and fedora, debian is much faster. Forget package size at this point. Is it parallel downloads maybe.
My main deal here is about the speed in which it takes to download all updates and install. I was merely trying to understand why debian seems to be much faster.
I've been a loyal Fedora user since RH 6.2 or some were in there :) so I'm not leaving, just trying to understand when i play with it once in awhile it just handles downloads differently.
parallel downloads is the answer imho. ( or at least one of the answers :) )
2008/6/12 Justin Conover justin.conover@gmail.com:
On Thu, Jun 12, 2008 at 12:40 PM, Caolan McNamara caolanm@redhat.com wrote:
On Thu, 2008-06-12 at 17:54 +0200, drago01 wrote:
On Thu, Jun 12, 2008 at 5:42 PM, Justin Conover justin.conover@gmail.com wrote: At 34MB it still takes
time, but .deb is 50MB cheaper that the .rpm.
you are comparing two different packages (2.0.4 and 2.4)
There's a few other problems with a direct comparison of sizes because they are two packages called "core" with somewhat differing content. The Debian "core" package depends on a "common" package which is an additional 27megs in size. The content of both of these is included in the single Fedora "core" rpm. Additionally the default help content is included in the Fedora "core" rpm, which is available in the deb packages as help-en_US, which is another additional 11 megs.
Additionally displaying help itself requires the use of the core writer libraries to render the html help, in Debian this means that the "writer" package is a dependency of help, and that's an additional 6megs in size in its .deb. While in Fedora writer is split into the optional bits called "writer" and the core required for use by help. Shrinking the "writer" rpm by approx 3 megs, and inflating the "core" one by the same.
To manually extract the various contents for side-by-side comparison you can use
rpm2cpio something.rpm | cpio -ivd vs ar x something.rpm tar xzf data.tar.gz
C.
-- fedora-test-list mailing list fedora-test-list@redhat.com To unsubscribe: https://www.redhat.com/mailman/listinfo/fedora-test-list
Ok, everyone in this thread is replying to OOO, that was not my intent. If you compare the speed of getting updates in debian and fedora, debian is much faster. Forget package size at this point. Is it parallel downloads maybe.
My main deal here is about the speed in which it takes to download all updates and install. I was merely trying to understand why debian seems to be much faster.
I've been a loyal Fedora user since RH 6.2 or some were in there :) so I'm not leaving, just trying to understand when i play with it once in awhile it just handles downloads differently.
-- fedora-test-list mailing list fedora-test-list@redhat.com To unsubscribe: https://www.redhat.com/mailman/listinfo/fedora-test-list
2008/6/13 seth vidal skvidal@fedoraproject.org:
On Thu, 2008-06-12 at 22:18 +0300, cornel panceac wrote:
parallel downloads is the answer imho. ( or at least one of the answers :) )
Parallel downloads crush a lot of network connections. Dial up is one in particular.
i see. this happens only on slow connections?
-- fedora-test-list mailing list fedora-test-list@redhat.com To unsubscribe: https://www.redhat.com/mailman/listinfo/fedora-test-list
On Fri, Jun 13, 2008 at 9:36 AM, cornel panceac cpanceac@gmail.com wrote:
2008/6/13 seth vidal skvidal@fedoraproject.org:
On Thu, 2008-06-12 at 22:18 +0300, cornel panceac wrote:
parallel downloads is the answer imho. ( or at least one of the answers :) )
Parallel downloads crush a lot of network connections. Dial up is one in particular.
i see. this happens only on slow connections?
Not necessarily. Even at high speed, I don't want dozens of rpm downloads taking all the bandwidth. I have mailgroups to read!
jerry
2008/6/13 Jerry Amundson jamundso@gmail.com:
On Fri, Jun 13, 2008 at 9:36 AM, cornel panceac cpanceac@gmail.com wrote:
2008/6/13 seth vidal skvidal@fedoraproject.org:
On Thu, 2008-06-12 at 22:18 +0300, cornel panceac wrote:
parallel downloads is the answer imho. ( or at least one of the answers :) )
Parallel downloads crush a lot of network connections. Dial up is one in particular.
i see. this happens only on slow connections?
Not necessarily. Even at high speed, I don't want dozens of rpm downloads taking all the bandwidth. I have mailgroups to read!
jerry
what if you download only one rpm per repo?
-- fedora-test-list mailing list fedora-test-list@redhat.com To unsubscribe: https://www.redhat.com/mailman/listinfo/fedora-test-list
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
cornel panceac wrote:
2008/6/13 Jerry Amundson <jamundso@gmail.com mailto:jamundso@gmail.com>: Not necessarily. Even at high speed, I don't want dozens of rpm downloads taking all the bandwidth. I have mailgroups to read!
jerrywhat if you download only one rpm per repo?
Doesn't really help if you're bandwidth constrained. Parallel downloads from different repos will still end up crushing your network connection.
Parallel downloading is good for some users but it probably shouldn't be the default.
Regards, Bryn.
On Fri, Jun 13, 2008 at 10:46 AM, Bryn M. Reeves bmr@redhat.com wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
cornel panceac wrote:
2008/6/13 Jerry Amundson <jamundso@gmail.com <mailto:jamundso@gmail.com : Not necessarily. Even at high speed, I don't want dozens of rpm downloads taking all the bandwidth. I have mailgroups to read!
jerrywhat if you download only one rpm per repo?
Doesn't really help if you're bandwidth constrained. Parallel downloads from different repos will still end up crushing your network connection.
Parallel downloading is good for some users but it probably shouldn't be the default.
Regards, Bryn.
So a yum-paralllel-download would be a good package to have ;)
On Fri, Jun 13, 2008 at 12:45 PM, Justin Conover justin.conover@gmail.com wrote:
On Fri, Jun 13, 2008 at 10:46 AM, Bryn M. Reeves bmr@redhat.com wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
cornel panceac wrote:
2008/6/13 Jerry Amundson <jamundso@gmail.com mailto:jamundso@gmail.com>: Not necessarily. Even at high speed, I don't want dozens of rpm downloads taking all the bandwidth. I have mailgroups to read!
jerrywhat if you download only one rpm per repo?
Doesn't really help if you're bandwidth constrained. Parallel downloads from different repos will still end up crushing your network connection.
Parallel downloading is good for some users but it probably shouldn't be the default.
Regards, Bryn.
So a yum-paralllel-download would be a good package to have ;)
Or preemptive throttled rpm downloader to download likely to be needed files into the yum cache
On Fri, 13 Jun 2008, Justin Conover wrote:
So a yum-paralllel-download would be a good package to have ;)
If you want to try writing a paralell downloader as a plugin that runs in the predownload plugin hook I think that would be a good proof of concept and if it worked and looked good could rapidly be merged.
-sv
cornel panceac wrote:
2008/6/13 seth vidal skvidal@fedoraproject.org:
On Thu, 2008-06-12 at 22:18 +0300, cornel panceac wrote:
parallel downloads is the answer imho. ( or at least one of the answers :) )
Parallel downloads crush a lot of network connections. Dial up is one in particular.
i see. this happens only on slow connections?
I can download stuff from my local IAP, one one connexion, at 1.2-1.7 Mbytes/sec.
I wouldn't expect two network connexions to download noticeably faster, but it would increase latency at my end: I can only receive one packet at a time.
I _might_ do better if I use parallel downloads from more servers, but I figure if I do that I make things worse at the server and on the network between. Besides, I _know_ some public ftp (and rsync) server operators object to parallel connexions from a single IP address. I don't need to be banned for 24 hours.
On Mon, 2008-06-16 at 08:14 +0800, John Summerfield wrote:
I _might_ do better if I use parallel downloads from more servers, but I figure if I do that I make things worse at the server and on the network between. Besides, I _know_ some public ftp (and rsync) server operators object to parallel connexions from a single IP address. I don't need to be banned for 24 hours.
AFAIK apt-get uses parallel connections on different servers.
poc
2008/6/16 Patrick O'Callaghan pocallaghan@gmail.com:
On Mon, 2008-06-16 at 08:14 +0800, John Summerfield wrote:
I _might_ do better if I use parallel downloads from more servers, but I figure if I do that I make things worse at the server and on the network between. Besides, I _know_ some public ftp (and rsync) server operators object to parallel connexions from a single IP address. I don't need to be banned for 24 hours.
AFAIK apt-get uses parallel connections on different servers.
which was the whole idea.
poc
-- fedora-test-list mailing list fedora-test-list@redhat.com To unsubscribe: https://www.redhat.com/mailman/listinfo/fedora-test-list
cornel panceac wrote:
2008/6/16 Patrick O'Callaghan pocallaghan@gmail.com:
On Mon, 2008-06-16 at 08:14 +0800, John Summerfield wrote:
I _might_ do better if I use parallel downloads from more servers, but I figure if I do that I make things worse at the server and on the network between. Besides, I _know_ some public ftp (and rsync) server operators object to parallel connexions from a single IP address. I don't need to be banned for 24 hours.
AFAIK apt-get uses parallel connections on different servers.
I can't tell that even that is true. My debian systems have one primary source of packages, and one of security updates. I don't have a ready way to test package downloads, but certainly getting metadata (updated package info) doesn't seem to run in parallel.
Most Debian users will have at least two mirrors enabled, their base repo plus security. They may also have contrib and (I think it still exists) non-free. Oh, and volatile (spam filters and other frequently updated data).
I don't think I would want apt-get hitting all these at once, and I've not net considered after-market repos such as backport.org and all those catalogued at http://apt-get.org/
which was the whole idea.
Debian doesn't use a mirror list, it asks users to choose a mirror at install/setup time and that's where is sources new packages into the future.
My two cents worth.
I only download once to burn a dvd. Thereafter, I download the updates. I only fetch the DVD about two days after the release. I also go to a site half way around the world, where the typical linux developer is asleep (local time 9pm, remote time, 3am), so that that mirror is not under heavy load. I get fairly good download times.
With updates I use the designated mirror list. Usually find that updates rarely take more then 10 minutes of download time. No, I do not have super high speed, only DSL speed. I am not concerned about a difference of 10 megabytes, I am concerned that the rpm is received without corruption due to transmission errors. This is caught by rpm or yum.
Leslie
--- On Fri, 6/13/08, seth vidal <skvidal@fedoraproject.org> wrote: From: seth vidal <skvidal@fedoraproject.org> Subject: Re: How can we speed up rpm downloads? To: "For testers of Fedora Core development releases" <fedora-test-list@redhat.com> Date: Friday, June 13, 2008, 9:29 AM
On Thu, 2008-06-12 at 22:18 +0300, cornel panceac wrote: > parallel downloads is the answer imho. ( or at least one of the > answers :) )
Parallel downloads crush a lot of network connections. Dial up is one in particular.
-sv
Leslie Satenstein wrote:
My two cents worth.
I only download once to burn a dvd. Thereafter, I download the updates. I only fetch the DVD about two days after the release. I also go to a site half way around the world, where the typical linux developer is asleep (local time 9pm, remote time, 3am), so that that mirror is not under heavy load. I get fairly good download times.
I find my own IAP hard to beat. In that timeframe I can easily download locally on one connexion just as fast as my ADSL2+ connexion will go, and it doesn't even count to quota.
With updates I use the designated mirror list. Usually find that updates rarely take more then 10 minutes of download time. No, I do not have super high speed, only DSL speed. I am not concerned about a difference of 10 megabytes, I am concerned that the rpm is received without corruption due to transmission errors. This is caught by rpm or yum.
08:17 [summer@mail ~]$ echo 1.5*60*10|bc 900.0 08:18 [summer@mail ~]$ 900 Mbytes in ten minutes. If you have a good ADSL2+ or better, it shouldn't.
--- On Thu, 6/12/08, Justin Conover justin.conover@gmail.com wrote:
From: Justin Conover justin.conover@gmail.com Subject: Re: How can we speed up rpm downloads? To: "For testers of Fedora Core development releases" fedora-test-list@redhat.com Cc: "For users of Fedora" fedora-list@redhat.com Date: Thursday, June 12, 2008, 10:54 AM On Thu, Jun 12, 2008 at 12:40 PM, Caolan McNamara caolanm@redhat.com wrote:
On Thu, 2008-06-12 at 17:54 +0200, drago01 wrote:
On Thu, Jun 12, 2008 at 5:42 PM, Justin Conover justin.conover@gmail.com wrote: At 34MB it still takes
time, but .deb is 50MB cheaper that the
.rpm.
you are comparing two different packages (2.0.4
and 2.4)
There's a few other problems with a direct
comparison of sizes because
they are two packages called "core" with
somewhat differing content. The
Debian "core" package depends on a
"common" package which is an
additional 27megs in size. The content of both of
these is included in
the single Fedora "core" rpm. Additionally
the default help content is
included in the Fedora "core" rpm, which is
available in the deb
packages as help-en_US, which is another additional 11
megs.
Additionally displaying help itself requires the use
of the core writer
libraries to render the html help, in Debian this
means that the
"writer" package is a dependency of help,
and that's an additional 6megs
in size in its .deb. While in Fedora writer is split
into the optional
bits called "writer" and the core required
for use by help. Shrinking
the "writer" rpm by approx 3 megs, and
inflating the "core" one by the
same.
To manually extract the various contents for
side-by-side comparison you
can use
rpm2cpio something.rpm | cpio -ivd vs ar x something.rpm tar xzf data.tar.gz
C.
-- fedora-test-list mailing list fedora-test-list@redhat.com To unsubscribe:
https://www.redhat.com/mailman/listinfo/fedora-test-list
Ok, everyone in this thread is replying to OOO, that was not my intent. If you compare the speed of getting updates in debian and fedora, debian is much faster. Forget package size at this point. Is it parallel downloads maybe.
My main deal here is about the speed in which it takes to download all updates and install. I was merely trying to understand why debian seems to be much faster.
I've been a loyal Fedora user since RH 6.2 or some were in there :) so I'm not leaving, just trying to understand when i play with it once in awhile it just handles downloads differently.--
As I understand, Fedora 9 was to have deltarpms by default, but it did not get through :(, OpenSuse uses them that is a + for them, As far as debian/ubuntu is concerned I cannot comment on them. But to update a rawhide machine/fedora 9 machine with dialup and that size of MB is pointless. I willa get back to testing when I have access to a higher speed connection last week in August. As of now I have to be selective as to which updates I pick up like kernel updates and such.
Regards,
Antonio
Justin Conover wrote:
On Thu, Jun 12, 2008 at 12:40 PM, Caolan McNamara caolanm@redhat.com wrote:
On Thu, 2008-06-12 at 17:54 +0200, drago01 wrote:
On Thu, Jun 12, 2008 at 5:42 PM, Justin Conover justin.conover@gmail.com wrote: At 34MB it still takes
time, but .deb is 50MB cheaper that the .rpm.
you are comparing two different packages (2.0.4 and 2.4)
There's a few other problems with a direct comparison of sizes because they are two packages called "core" with somewhat differing content. The Debian "core" package depends on a "common" package which is an additional 27megs in size. The content of both of these is included in the single Fedora "core" rpm. Additionally the default help content is included in the Fedora "core" rpm, which is available in the deb packages as help-en_US, which is another additional 11 megs.
Additionally displaying help itself requires the use of the core writer libraries to render the html help, in Debian this means that the "writer" package is a dependency of help, and that's an additional 6megs in size in its .deb. While in Fedora writer is split into the optional bits called "writer" and the core required for use by help. Shrinking the "writer" rpm by approx 3 megs, and inflating the "core" one by the same.
To manually extract the various contents for side-by-side comparison you can use
rpm2cpio something.rpm | cpio -ivd vs ar x something.rpm tar xzf data.tar.gz
C.
-- fedora-test-list mailing list fedora-test-list@redhat.com To unsubscribe: https://www.redhat.com/mailman/listinfo/fedora-test-list
Ok, everyone in this thread is replying to OOO, that was not my intent. If you compare the speed of getting updates in debian and fedora, debian is much faster. Forget package size at this point. Is it parallel downloads maybe.
My main deal here is about the speed in which it takes to download all updates and install. I was merely trying to understand why debian seems to be much faster.
To compare.you need to find like-sized packages; the example quoted was a bad example. and that's the point discussed.
You also need to find mirrors with equivalent capacity to give you; if one mirror uploads to you faster than the other, that's not a fair comparison.
yum is very slow to get itself organised, but once it gets underway it should download at the same rate.
I don't think either downloads in parallel, and if your internet is running at its rated speed, that is likely the bottleneck do running two, three or more downloads in parallel will serve only to choke your self. And waste server resources.
For a valid comparison, you need to set up a benchmark where you control the server, the intervening network, and the clients. And both servers need to be identical, and both clients need to be identical.
I've been a loyal Fedora user since RH 6.2 or some were in there :) so I'm not leaving, just trying to understand when i play with it once in awhile it just handles downloads differently.
If you download enough that i's an important issue for you, consider establishing your own mirror that you update at a time that is convenient to you, and/or run your own Squid proxy.
On Mon, 2008-06-16 at 08:08 +0800, John Summerfield wrote:
I don't think either downloads in parallel, and if your internet is running at its rated speed, that is likely the bottleneck do running two, three or more downloads in parallel will serve only to choke your self. And waste server resources.
apt-get does run several downloads in parallel. This makes sense when some servers can only server data at a rate lower than the connection bandwidth, which does happen particularly with high-traffic sites.
poc
Patrick O'Callaghan wrote:
On Mon, 2008-06-16 at 08:08 +0800, John Summerfield wrote:
I don't think either downloads in parallel, and if your internet is running at its rated speed, that is likely the bottleneck do running two, three or more downloads in parallel will serve only to choke your self. And waste server resources.
apt-get does run several downloads in parallel. This makes sense when some servers can only server data at a rate lower than the connection bandwidth, which does happen particularly with high-traffic sites.
I use apt-get regularly, I also have some Debian systems, and the progress meter doesn't reflect parallel downloads.
If a remote (free!) server is already overloaded, adding to its stress doesn't seem very sensible. It doesn't take a very large increase in requests for a service to go from "very busy but coping" to "thrashing." Just take a look at supermarket queues and think how well the are run, and how they might be run better (from the customer's POV). I've rairely seen an idle checkout operator.
If an operator can (on average) serve one customer per minute (and the times don't vary much), and customers arrive at one per minute, there won't be much of a queue. However, if customers arrive each 55 seconds, it won't take long for the queue to go out the door, so to speak.
If the bottleneck isn't the server, but the network, then IP is designed to discard packets when overloaded, and TCP manages this by detecting discarded packets and requesting they be resent. An IP network is quite resilient, but it can be flooded and this is what use of parallel downloads does.
On Tue, 2008-06-17 at 10:24 +0800, John Summerfield wrote:
Patrick O'Callaghan wrote:
On Mon, 2008-06-16 at 08:08 +0800, John Summerfield wrote:
I don't think either downloads in parallel, and if your internet is running at its rated speed, that is likely the bottleneck do running two, three or more downloads in parallel will serve only to choke your self. And waste server resources.
apt-get does run several downloads in parallel. This makes sense when some servers can only server data at a rate lower than the connection bandwidth, which does happen particularly with high-traffic sites.
I use apt-get regularly, I also have some Debian systems, and the progress meter doesn't reflect parallel downloads.
I have an Ubuntu system in the office and apt-get definitely does do parallel downloads when updating its database ('update'), but I'm less sure about downloading packages ('upgrade'). It's an Internet 2 connection which is very fast so maybe it can't be bothered :-). Adept and synaptic are the same. I've also used apt-get on Fedora (a few years back) and it definitely did parallel downloads of packages, probably from different servers.
poc
On Thu, Jun 12, 2008 at 10:42 AM, Justin Conover justin.conover@gmail.com wrote:
Is there any current work on yum-presto, deltarpm or some other method of getting the downloads for updates to be smaller and quicker? Can you compress rpm's to make the the downloads faster?
If you have multiple machines, would suggest you proxy it.
On Thu, Jun 12, 2008 at 5:42 PM, Justin Conover justin.conover@gmail.com wrote:
Is there any current work on yum-presto, deltarpm or some other method of getting the downloads for updates to be smaller and quicker?
IIRC The work on the client side was done, but it lacked the necessary bits on the infrastructure side to make it happen by default for all packages.
You may want to search in fedora-devel archives for more info
Can you compress rpm's to make the the downloads faster?
rpms _are_ compressed; we also had arguments about using alternate (possibly more efficient) compression algorithms but the outcome was it is not worth the trouble.
Is deltarpm's the solution?
I think so
Has anyone talked to all the fedora mirrors to have a deltarpm repository?
That's not the problem. The problem is that deltarpm info has to be prepared somewhere during the rpm build phase, so it will be available for all mirrors to pick up. IIRC the "somewhere" is in mock but please double check with fedora-devel archives.
-
Antonio Olivares -
Arthur Pemberton -
Bryn M. Reeves -
Caolán McNamara -
cornel panceac -
drago01 -
Gianluca Sforna -
Jerry Amundson -
John Summerfield -
Justin Conover -
Leslie S Satenstein -
Patrick O'Callaghan -
Seth Vidal