Re: HAMMER changes in HEAD, also needs testing
:Dennis Melentyev wrote:
:>> I also made a minor change to cpdup to not use temporary filenames
:>> when doing a fresh copy (when the target file does not exist). To test
:>> HAMMER with cpdup you need the new cpdup. The old cpdup always creates
:>> a temporary file and then rename()'s and that unfortunately breaks the
:>> optimization that HAMMER makes to order the inode numbers.
:>
:> Just not sure was it the right decision. AFAIU, tmpname->rename was
:> there because of atomicity of file operation, so the file with correct
:> name appear fully written.
It's a good point. I would argue that simultanious access is a
problem specific for ftp/www access, and in that case we can add
a new option to cpdup (say, -t) to force the use of .tmp files in
all cases instead of just in the overwrite case.
:I see two different possible optimization ways:
:
:- Determine inode number from the parent directory inode number, not from
:the filename hash. This won't give linearity within the directory scan,
:but will give rough spatial locality vs temporal locality (i.e. adding a
:file to a directory every day won't make hammer jump all over the place).
: This is very important in my eyes.
:
:- Implement "link data extents from other file" ioctl. Just write data to
:a tmp file, then create the dst file with the data extents of the temp
:file. Like a hardlink, just on the backend fs layer. That's cute, but
:not really necessary.
:
:cheers
: simon
Now that I have NCQ operational with AHCI I have been able to do more
specific performance tests with find, tar, ls -lR, etc.
Right now HAMMER's biggest issue with the likes of find or ls is
when it calls getdirentries() and when it lstat()'s the first
entry in the directory. If the information is not cached the
buffer cache both case take an excessive amount of time.
In both cases I think the mistake I made was putting directory
entries and inodes in different localization domains in the B-Tree.
Thus when a directory is pushed into and we want to read its directory
entries, significant seeks are required to access those entries relative
to the directory inode. Similarly when we lstat() the first entry
in a directory, significant seeks are required to look-up the inode(s)
because they are nowhere near the directory entries.
I believe I can improve this by moving directory entries into the inode's
localization domain in the B-Tree, at least for newly created directories.
Then the B-Tree elements for the directory entries will actually be
near the B-Tree elements for the related inodes, and the directory entry
data will be near the inode data. I'm testing this now.
A larger problem is simply the extra seeks required to traverse a
directory tree, and fixing it would require a major change to HAMMER's
media structures in order to embed directory entries and inode data
directly in the B-Tree. I'll have to save this for some future HAMMER2
because there is simply no way to maintain media compatibility with
older versions if I do it. And it's major surgery on top of that.
-Matt
Matthew Dillon
<dillon@backplane.com>
討論串 (同標題文章)
完整討論串 (本文為第 5 之 8 篇)：