Re: ZFS directory with a large number of files
--GPJrCs/72TxItFYR
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
On 2011-Aug-02 08:39:03 +0100, "seanrees@gmail.com" <seanrees@gmail.com> wr=
ote:
>On my FreeBSD 8.2-S machine (built circa 12th June), I created a
>directory and populated it over the course of 3 weeks with about 2
>million individual files. As you might imagine, a 'ls' of this
>directory took quite some time.
>
>The files were conveniently named with a timestamp in the filename
>(still images from a security camera, once per second) so I've since
>moved them all to timestamped directories (yyyy/MM/dd/hh/mm). What I
>found though was the original directory the images were in is still
>very slow to ls -- and it only has 1 file in it, another directory.
I've also seen this behaviour on Solaris 10 after cleaning out a
directory with a large number of files (though not as pathological as
your case). I tried creating and deleting entries in an unsuccessful
effort to trigger directory compaction. I wound up moving the
remaining contents into a new directory, deleting the original one
and renaming the new directory.
It would appear te be a garbage collection bug in ZFS.
On 2011-Aug-02 13:10:27 +0300, Daniel Kalchev <daniel@digsys.bg> wrote:
>On 02.08.11 12:46, Daniel O'Connor wrote:
>> I am pretty sure UFS does not have this problem. i.e. once you=20
>> delete/move the files out of the directory its performance would be=20
>> good again.=20
>
>UFS would be the classic example of poor performance if you do this.
Traditional UFS (including Solaris) behave badly in this scenario but
4.4BSD derivatives will release unused space at the end of a directory
and have smarts to more efficiently skip unused entries at the start
of a directory.
--=20
Peter Jeremy
--GPJrCs/72TxItFYR
Content-Type: application/pgp-signature
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.17 (FreeBSD)
iEYEARECAAYFAk448A8ACgkQ/opHv/APuIfmjQCgg4ijcCrG0q7oX4cLwKxDd9io
TWcAoLDGoEoEkURljItE768LbQddMILj
=ZKso
-----END PGP SIGNATURE-----
--GPJrCs/72TxItFYR--
討論串 (同標題文章)
完整討論串 (本文為第 14 之 24 篇)：