Poor write performance on Fusion MPT SAS RAID controllers
If you have a hardware RAID controller using mptsas module on your Linux server, you may notice your write performance is extremely poor – i.e. some database workloads can be 30x slower than on other servers (i.e. using Linux software RAID).
An example card below – lspci shows:
02:05.0 SCSI storage controller: LSI Logic / Symbios Logic SAS1068 PCI-X Fusion-MPT SAS (rev 01)
How can you fix it? See below.
The issue happens because write caching is by default disabled on drives behind the RAID – and there is no way to enable it in controller firmware / BIOS, or by running scsi-config command.
To enable write caching, you may use below commands in your startup scripts:
# create a /dev/sg0 node first mknod /dev/sg0 c 21 0 # this one should create /dev/sg1 and /dev/sg2 nodes, too (you'll find sg_map in sg3-utils package, or, sg3_utils, if you're using RedHat / CentOS) sg_map -i -x # finally, enable write cache - assuming you have two drives; /dev/sg2 would be the controller itself sdparm -s WCE=1 /dev/sg0 sdparm -s WCE=1 /dev/sg1
That’s it! Performance should be back to normal. Of course you have to be aware of the dangers of write-caching and potential data loss on unexpected power outage.
Other, useful commands:
# modprobe mptctl # mpt-status ioc0 vol_id 0 type IM, 2 phy, 231 GB, state OPTIMAL, flags ENABLED ioc0 phy 1 scsi_id 8 ATA ST3250310AS A , 232 GB, state ONLINE, flags NONE ioc0 phy 0 scsi_id 1 ATA ST3250310AS A , 232 GB, state ONLINE, flags NONE # lsscsi [4:0:0:0] disk ATA ST3250310AS A - [4:0:1:0] disk ATA ST3250310AS A - [4:1:0:0] disk LSILOGIC Logical Volume 3000 /dev/sda
You can also add /dev/sg0 and /dev/sg1 to monitoring by smartmontools (/etc/smartd.conf).