Where to set readahead: LVM, RAID devices, device-mapper, block devices?

You want to set readahead to tune the performance of you disk reads and you find that in your server there are several levels of devices, block devices, RAID devices, then LVM with device-mapper, etc.

You can set the readahead in any of them, which one is the right one?

I came up with this Stackoverflow question: https://serverfault.com/questions/418352/readahead-settings-for-lvm-device-mapper-software-raid-and-block-devices-wha

And i decided to do some tests to prove what wojciechz was saying, and he is right, let me show you:

My setup is a server with RAID10 and LVM with a /db partition mounted on the logical volume

# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md127 : active raid10 sdj[7] sdi[6] sdh[5] sdg[4] sdf[3] sde[2] sdd[1] sdc[0]
3906525184 blocks super 1.2 512K chunks 2 near-copies [8/8] [UUUUUUUU]

# pvdisplay
--- Physical volume ---
PV Name /dev/md127
VG Name vg1
PV Size 3.64 TiB / not usable 0
Allocatable yes
PE Size 4.00 MiB
Total PE 953741
Free PE 489746
Allocated PE 463995
PV UUID KH4RjS-lgAN-2OdI-hiYQ-HuR1-naDM-nSmc5S

# mount | grep db
/dev/mapper/vg1-db on /db type ext4 (rw,noatime,nodiratime,discard,stripe=512,data=ordered)

Continue reading