On Thu, Dec 02, 2004 at 12:40:06PM -0500, David Zeuthen wrote:
On Thu, 2004-12-02 at 11:58 -0500, Daniel Veillard wrote:
On Thu, Dec 02, 2004 at 11:48:56AM -0500, David Zeuthen wrote:
Not sure; the big boost really comes from reordering the files on the filesystem - running readahead (which takes 11 seconds) only gives me a
I'm looking into this. a mix of creating a new directory, copying the file there, then unlinking, linking and unlinking the new copies to the old name is more likely to help. The problem is that it's also bad for the current session where that change is done since you will end up with multiple in-memory copies of all system libraries... It is gonna be a bit tricky to set-up, either as a post-install step or very early on during boot. It's very easy to break too ...
To me it makes more sense to look at this on the block level than on the file level; remember, a lot the files in my readahead set stems from
block level is quite harder, fs specific, and I'm afraid impossible to do in a generic way, you would need specific kernel APIs for all filesystems supported.
stat'ing files - for example, there are more than 2200 distinct .png files in my readahead set. We probably only need one sector from most of those (the inode) rather than the entire file.
I'm almost positive it requires kernel changes to do this the right way; one naive idea is to have a userspace daemon, capturing what blocks are read when (kernel tells this daemon using the kernel events layer). This would run in the first three minutes on each and every boot. When the system is idle (and only when running on AC power!) another daemon rearranges blocks on the disk. What blocks to rearrange could be the result of a computation involving several three-minute result sets.
I suggest a more generic but possibly slightly less efficient way by working at the file level. Maybe it will not be optimal, inducing a bit too much data or too many seeks, but would be doable without as much risk than trying to push a block reallocation API though the full I/O stack down to every fs driver. You suppose you can rearranges blocks on the disk, and I'm really not sure it's doable realistically within a reasonnable time frame. Ideally someone well versed in kernel FS internals can talk about this. Something tells me it's not gonna fly...
Daniel