Allow separation of sequential_cutoff

[Hannes Tismer]

Dear bcache-developers,

This is a feature request for a very specific use case:

Having a caching device in front of an md-raid on a machine which =20
serves as a low power media server.

The last writes could be cached on the caching device to reduce the =20
non-sleep time of the raid (drives in spindown state) while playing =20
media, together with a huge writeback delay of, let's say, 3 hours,
newest media would always be read from the caching device.

It's not likely that a recently read (watched) media file will be read =
=20
again after having been read before (very recently), but it's very =20
likely that it'll be read once after it was written.

I could disable sequential_cutoff or set it rediculously high. That =20
would result in every sequential read being cached into the caching =20
device, too, which in this use case is not wanted.

Could you split up the sequential_cutoff setting to =20
sequential_read_cutoff and sequential_write_cutoff?

That way writes including sequential ones can be cached for later, but =
=20
sequential reads could be ignored by the cache.


Thank you in advance

Hannes
--=20
Hannes Tismer
Herzogstr. 4
41238 M=C3=B6nchengladbach

[Larkin Lowrey]

I am also interested in this use case and also would like to see
separate sequential cutoff values for read and write.

In addition to that I would like to offer another idea for
consideration, high and low watermarks for writeback_percent.

The documentation at http://bcache.evilpiepirate.org/Design/ is either
out of date or incomplete... or I don't understand it. Specifically the
meaning of writeback_percent is not clear to me. That documentation
states that write-back will not begin until the writeback_percent
threshold has been reached. The definition in bcache.txt states that
bcache tries to maintain that percent of the cache dirty. The latter
seems to match my observations so I'll go with that.

My idea is to have high and low watermarks for writeback_percent where
writeback would not begin until reaching the high watermark and writing
would continue until the low watermark was reached. To match the present
behavior, as I understand it, both values would be set the same.
However, in the use case described here the spread between the two would
define how often the backing store would need to be spun up for writing.

If one writes 10GB a day and the spread between high and low
writeback_percent watermarks is set to 10GB then the backing store would
spin up only once per day. Ideally the writeback process would be able
to write at a high rate in order to limit the amount of time the drives
would need to be spinning.

I see two competing priorities for writeback. One is to smooth writes so
they are less bursty and less disruptive and the other is to consolidate
writes so they can be done in batches (bursty). It would be great if
both cases could be supported.

--Larkin

Post by Hannes Tismer
Dear bcache-developers,
Having a caching device in front of an md-raid on a machine which
serves as a low power media server.
The last writes could be cached on the caching device to reduce the
non-sleep time of the raid (drives in spindown state) while playing
media, together with a huge writeback delay of, let's say, 3 hours,
newest media would always be read from the caching device.
It's not likely that a recently read (watched) media file will be read
again after having been read before (very recently), but it's very
likely that it'll be read once after it was written.
I could disable sequential_cutoff or set it rediculously high. That
would result in every sequential read being cached into the caching
device, too, which in this use case is not wanted.
Could you split up the sequential_cutoff setting to
sequential_read_cutoff and sequential_write_cutoff?
That way writes including sequential ones can be cached for later, but
sequential reads could be ignored by the cache.
Thank you in advance
Hannes

[Hannes Tismer]

I second that.

This would make the whole process even more tunable.

Also a "simple" feature to limit caching to only-write-operations or =20
only-read-operations
would be very nice to have also.


Hannes




On Wed, 16 Jul 2014 11:19:16 -0500, Larkin Lowrey =20

Post by Larkin Lowrey
I am also interested in this use case and also would like to see
separate sequential cutoff values for read and write.
In addition to that I would like to offer another idea for
consideration, high and low watermarks for writeback_percent.
The documentation at http://bcache.evilpiepirate.org/Design/ is eithe=

r

Post by Larkin Lowrey
out of date or incomplete... or I don't understand it. Specifically t=

he

Post by Larkin Lowrey
meaning of writeback_percent is not clear to me. That documentation
states that write-back will not begin until the writeback_percent
threshold has been reached. The definition in bcache.txt states that
bcache tries to maintain that percent of the cache dirty. The latter
seems to match my observations so I'll go with that.
My idea is to have high and low watermarks for writeback_percent wher=

e

Post by Larkin Lowrey
writeback would not begin until reaching the high watermark and writi=

ng

Post by Larkin Lowrey
would continue until the low watermark was reached. To match the pres=

ent

Post by Larkin Lowrey
behavior, as I understand it, both values would be set the same.
However, in the use case described here the spread between the two wo=

uld

Post by Larkin Lowrey
define how often the backing store would need to be spun up for writi=

ng.

Post by Larkin Lowrey
If one writes 10GB a day and the spread between high and low
writeback_percent watermarks is set to 10GB then the backing store wo=

uld

Post by Larkin Lowrey
spin up only once per day. Ideally the writeback process would be abl=

e

Post by Larkin Lowrey
to write at a high rate in order to limit the amount of time the driv=

es

Post by Larkin Lowrey
would need to be spinning.
I see two competing priorities for writeback. One is to smooth writes=

so

Post by Larkin Lowrey
they are less bursty and less disruptive and the other is to consolid=

ate

Post by Larkin Lowrey
writes so they can be done in batches (bursty). It would be great if
both cases could be supported.
--Larkin

Post by Hannes Tismer
Dear bcache-developers,
Having a caching device in front of an md-raid on a machine which
serves as a low power media server.
The last writes could be cached on the caching device to reduce the
non-sleep time of the raid (drives in spindown state) while playing
media, together with a huge writeback delay of, let's say, 3 hours,
newest media would always be read from the caching device.
It's not likely that a recently read (watched) media file will be re=

ad

Post by Larkin Lowrey

Post by Hannes Tismer
again after having been read before (very recently), but it's very
likely that it'll be read once after it was written.
I could disable sequential_cutoff or set it rediculously high. That
would result in every sequential read being cached into the caching
device, too, which in this use case is not wanted.
Could you split up the sequential_cutoff setting to
sequential_read_cutoff and sequential_write_cutoff?
That way writes including sequential ones can be cached for later, b=

ut

Post by Larkin Lowrey

Post by Hannes Tismer
sequential reads could be ignored by the cache.
Thank you in advance
Hannes

--=20
Hannes Tismer
Herzogstr. 4
41238 M=C3=B6nchengladbach

[Jianjian Huo]

I just posted the patch to add an option to disable read caching.
It works with all three cache polices, then you can get write-only.
If you want read-only-operations, you can just use writearound policy.

-Jianjian

Post by Hannes Tismer
I second that.
This would make the whole process even more tunable.
Also a "simple" feature to limit caching to only-write-operations or
only-read-operations
would be very nice to have also.
Hannes
On Wed, 16 Jul 2014 11:19:16 -0500, Larkin Lowrey

Post by Larkin Lowrey
I am also interested in this use case and also would like to see
separate sequential cutoff values for read and write.
In addition to that I would like to offer another idea for
consideration, high and low watermarks for writeback_percent.
The documentation at http://bcache.evilpiepirate.org/Design/ is eith=

er

Post by Hannes Tismer

Post by Larkin Lowrey
out of date or incomplete... or I don't understand it. Specifically =

the

Post by Hannes Tismer

Post by Larkin Lowrey
meaning of writeback_percent is not clear to me. That documentation
states that write-back will not begin until the writeback_percent
threshold has been reached. The definition in bcache.txt states that
bcache tries to maintain that percent of the cache dirty. The latter
seems to match my observations so I'll go with that.
My idea is to have high and low watermarks for writeback_percent whe=

re

Post by Hannes Tismer

Post by Larkin Lowrey
writeback would not begin until reaching the high watermark and writ=

ing

Post by Hannes Tismer

Post by Larkin Lowrey
would continue until the low watermark was reached. To match the pre=

sent

Post by Hannes Tismer

Post by Larkin Lowrey
behavior, as I understand it, both values would be set the same.
However, in the use case described here the spread between the two w=

ould

Post by Hannes Tismer

Post by Larkin Lowrey
define how often the backing store would need to be spun up for writ=

ing.

Post by Hannes Tismer

Post by Larkin Lowrey
If one writes 10GB a day and the spread between high and low
writeback_percent watermarks is set to 10GB then the backing store w=

ould

Post by Hannes Tismer

Post by Larkin Lowrey
spin up only once per day. Ideally the writeback process would be ab=

le

Post by Hannes Tismer

Post by Larkin Lowrey
to write at a high rate in order to limit the amount of time the dri=

ves

Post by Hannes Tismer

Post by Larkin Lowrey
would need to be spinning.
I see two competing priorities for writeback. One is to smooth write=

s so

Post by Hannes Tismer

Post by Larkin Lowrey
they are less bursty and less disruptive and the other is to consoli=

date

Post by Hannes Tismer

Post by Larkin Lowrey
writes so they can be done in batches (bursty). It would be great if
both cases could be supported.
--Larkin

Post by Hannes Tismer
Dear bcache-developers,
Having a caching device in front of an md-raid on a machine which
serves as a low power media server.
The last writes could be cached on the caching device to reduce the
non-sleep time of the raid (drives in spindown state) while playing
media, together with a huge writeback delay of, let's say, 3 hours,
newest media would always be read from the caching device.
It's not likely that a recently read (watched) media file will be r=

ead

Post by Hannes Tismer

Post by Larkin Lowrey

Post by Hannes Tismer
again after having been read before (very recently), but it's very
likely that it'll be read once after it was written.
I could disable sequential_cutoff or set it rediculously high. That
would result in every sequential read being cached into the caching
device, too, which in this use case is not wanted.
Could you split up the sequential_cutoff setting to
sequential_read_cutoff and sequential_write_cutoff?
That way writes including sequential ones can be cached for later, =

but

Post by Hannes Tismer

Post by Larkin Lowrey

Post by Hannes Tismer
sequential reads could be ignored by the cache.
Thank you in advance
Hannes

--
Hannes Tismer
Herzogstr. 4
41238 M=C3=B6nchengladbach
--
To unsubscribe from this list: send the line "unsubscribe linux-bcach=

e" in

Post by Hannes Tismer
More majordomo info at http://vger.kernel.org/majordomo-info.html