On 4/11/18 4:19 pm, Eyal Lebedinsky wrote:
While configuring a new raid I ran iostat to see that it is idle. It was, and there was no io showing at all.
I then mounted it on a new mount point which I have no process using. I started hearing knocks from the PC case, and touching the disks revealed that they all had activity 1-2 times a second, concurrently, leading to the louder than usual noise.
Here is what "iostat 60" is now showing on a totally idle system (this is a very typical entry):
avg-cpu: %user %nice %system %iowait %steal %idle 0.21 0.00 0.19 1.36 0.00 98.24
Device tps kB_read/s kB_wrtn/s kB_read kB_wrtn sda 0.73 0.93 4.07 56 244 sdb 5.87 8.53 20.78 512 1247 sdd 6.67 10.93 23.18 656 1391 sdf 8.50 40.27 52.52 2416 3151 sde 6.20 4.40 16.65 264 999 sdh 5.97 10.13 22.38 608 1343 sdg 7.77 38.00 50.25 2280 3015 sdc 5.87 8.53 20.78 512 1247 md127 1.80 0.00 40.27 0 2416
sda is the root fs (ext4). md127 (ext4) is a RAID6 of 7 disks sd[b-h]1.
What is this io, and can it be stopped? I want to allow the disks to enter low power mode (not spin down) when idle.
This is up-to-date recent install of f28 (this is a test system, so not customized).
TIA
A summary of what I learnt so far:
I was pointed at the lazy init feature of ext4 as the culprit (read the thread). I was not aware of this feature so there is a silver lining to this cloudy issue.
After some searching I now see in mkfs.ext4 man page these two options lazy_itable_init, lazy_journal_init
Furthermore, I read about it here https://ext4.wiki.kernel.org/index.php/Ext4_Disk_Layout#Lazy_Block_Group_Ini...
One can see the activity using iotop:
$ iotop -oP Total DISK READ : 0.00 B/s | Total DISK WRITE : 0.00 B/s Actual DISK READ: 0.00 B/s | Actual DISK WRITE: 103.76 K/s PID PRIO USER DISK READ DISK WRITE SWAPIN IO> COMMAND 1872 be/3 root 0.00 B/s 0.00 B/s 0.00 % 3.50 % [jbd2/md127-8] 1874 be/4 root 0.00 B/s 0.00 B/s 0.00 % 1.11 % [ext4lazyinit]
However it seems that there is no way to see how far the lazy init progressed or how much data needs to be written.
My other observation is that the RAID6 write amplification probably has a large effect if the init process is writing non sequential single blocks.
HTH