Re: Blogbench RAID benchmarks
Since Dragonfly has high res timers...
Would it make any sense to use two different metrics to help balance the
loading of the read and write pipelines? In some sense, we want to keep
the full pipeline (read and write), the disk I/O path utilized, but we
don't want
to starve one type of request for the other. We want all I/O requests to
complete in a reasonable amount of time, both on controller, and while waiting
to get onto a disk queue.
Can we use the timer to timestamp requests, both when they first get
constructed, and also when they hit the I/O controller queue, as well as when
they get completed, to help balance the I/O load and latency in an acceptable
fashion?
Note, if we have such timestamps, it would enable us to also make smarter
decisions in certain situations (which mirror disk do I issue a
read/write to first?),
and may allow us to keep summary statistics about the performance of a disk,
and complain if things get too far outta whack.
Naturally, if hammer (or anything) is generating a certain type of I/O
in the course
of servicing some benchmark, it's going to be rather hard to have the
I/O path be
able to be fair, being overwhelmed with one type of I/O, which may be necessary
for the upper layers (hamer, etc) to make forward progress.
-Toby.
On Mon, Jul 18, 2011 at 8:06 PM, Matthew Dillon
<dillon@apollo.backplane.com> wrote:
> ꀠ嘢k, well this is interesting. 脷asically it comes down to whether we
> ꀠ烀ant to starve read operations or whether we want to starve write
> ꀠ漑perations.
>
> ꀠ糍he FreeBSD results starve read operations, while the DragonFly results
> ꀠ澵tarve write operations. 糍hat's the entirety of the difference between
> ꀠ濳he two tests.
>
> ꀠ糍he final numbers don't do justice to this... if you look at the
> ꀠ澑aw numbers though it is apparent. 咗hen the blogbench test blows out
> ꀠ澵ystem caches the read activity on FreeBSD drops into the ~600 range
> ꀠ烀hile on DragonFly the read activity drops to the ~25000 range. 孭t
> ꀠ濳he same time FreeBSD's write activity stays in the ~4000 range while
> ꀠ餸ragonFly's write activity drops into the ~50's.
>
> ꀠ啱 tracked the reason for the DragonFly write activity dropping. 啱t
> ꀠ毪asically comes down to the backlog of inodes in HAMMER needing
> ꀠ澑eclamation. 餸ue to the heavy concurrent read load the HAMMER flusher
> ꀠ湶s constantly stuck in B-Tree locks and cannot flush inode meta-data
> ꀠ漑ut quickly enough to keep up with blogbench. 嘢nce it hits the inode
> ꀠ毪acklog limit (25000) write throughput goes down drastically.
>
> ꀠ咗hile one can increase the limit (vfs.hammer.limit_reclaim), all that
> ꀠ温appens is that HAMMER takes a little longer before it hits it, at
> ꀠ溻east in the blogbench test. 啫or more bursty bulk write operations
> ꀠ湶ncreasing the limit would be a good tuning parameter.
>
> ꀠ啫rankly both FreeBSDs and DragonFlys results are incorrect. 啫reeBSD is
> ꀠ溆illing read performance way way way too much while DragonFly is killing
> ꀠ烀rite performance way way way too much.
>
> ꀠ啱'm not sure how it could be fixed, though. 啱 can definitely reduce
> ꀠ脷-Tree deadlocks in HAMMER by unlocking b-tree nodes during synchronous
> ꀠ澑ead I/O (for meta-data), but the result that we really want is more
> ꀠ毪alanced read vs write performance, not these extreme tilts that we see.
>
> ꀠ孭lso note that blogbench's 'final' results are worthless. 糍he read
> ꀠ漤erformance is mostly counting the pre-cache-blowout numbers. 餸ragonFly's
> ꀠ澑ead performance is 41x FreeBSD's once the caches are blown out,
> ꀠ烀hile FreeBSD's write performance is 80x DragonFly's write performance
> ꀠ漑nce the caches are blown out. 嘞eads tend to be less localized than
> ꀠ烀rites so, generally speaking, the disk bandwidth *IS* being used fairly
> ꀠ汢fficiently in both cases. 脷ut neither result is really acceptable
> ꀠ啱MHO.
>
> ꀠ糍his is all with swapcache turned off. 糍he only way to test in a
> ꀠ沲air manner with swapcache turned on (with a SSD) is if the FreeBSD
> ꀠ濳est used a similar setup w/ZFS.
>
> ꀠꀠꀠꀠꀠꀠꀠꀠꀠꀠꀠꀠꀠꀠꀠꀠꀠꀠꀠꀭMatt
> ꀠꀠꀠꀠꀠꀠꀠꀠꀠꀠꀠꀠꀠꀠꀠꀠꀠꀠꀠ嗰atthew Dillon
> ꀠꀠꀠꀠꀠꀠꀠꀠꀠꀠꀠꀠꀠꀠꀠꀠꀠꀠꀠꀼdillon@backplane.com>
>
討論串 (同標題文章)
完整討論串 (本文為第 15 之 23 篇)：