SSD/NVMe Tweaks (TRIM/Discard)

Lin­ux Desk­top

Lin­ux Serv­er

Prox­mox

Note, with­in the ex­am­ples in the fol­low­ing sec­tions, /​​​dev/​​​sda refers to a SSD de­vice while /​​​dev/​​​nvme0n1 refers to a NVMe de­vice.

Dis­card­ing Un­used Blocks

You can per­form or sched­ule dis­card op­er­a­tions on block de­vices that sup­port them. Block dis­card op­er­a­tions dis­card blocks that are no longer in use by a mount­ed file sys­tem. They are use­ful on:

• Sol­id-state dri­ves (SS­Ds)
• Thin­ly-pro­vi­sioned stor­age

You can run dis­card op­er­a­tions us­ing dif­fer­ent meth­ods:

• Batch dis­card: Are run ex­plic­it­ly by the user. They dis­card all un­used blocks in the se­lect­ed file sys­tems.
• On­line dis­card: Are spec­i­fied at mount time. They run in re­al time with­out user in­ter­ven­tion. On­line dis­card op­er­a­tions dis­card on­ly the blocks that are tran­si­tion­ing from used to free.
• Pe­ri­od­ic dis­card: Are batch op­er­a­tions that are run reg­u­lar­ly by a sys­temd ser­vice.

All types are sup­port­ed by the XFS and ext4 file sys­tems and by VDO.

Rec­om­men­da­tions. Red Hat rec­om­mends that you use batch or pe­ri­od­ic dis­card. Use on­line dis­card on­ly if:

• the system’s work­load is such that batch dis­card is not fea­si­ble, or
• on­line dis­card op­er­a­tions are nec­es­sary to main­tain per­for­mance.

Ref­er­ences

• Red Hat Docs: Red Hat > [8] > Man­ag­ing stor­age de­vices > Chap­ter 8. Dis­card­ing un­used blocks
• Red Hat Docs: Red Hat > [9] > Man­ag­ing stor­age de­vices > Chap­ter 10. En­abling mul­ti­pathing on NVMe de­vices
• Red Hat Docs: Red Hat > [9] > Man­ag­ing stor­age de­vices > Chap­ter 37. Dis­card­ing un­used blocks
• Red Hat Docs: Red Hat > [9] > Dedu­pli­cat­ing and com­press­ing LVs > Chap­ter 4. Trim op­tions on an LVM-VDO vol­ume
• Arch Lin­ux Wi­ki: Sol­id state dri­ve
• Unix and Lin­ux: Fstrim "the dis­card op­er­a­tion is not sup­port­ed" with ext4 on LVM
• Red Hat Cus­tomer Por­tal: Does fstrim work on a LVM filesys­tem that has a snap­shot or on a snap­shot it­self?
• Red Hat Bugzil­la: Bug 1390148 – Dis­card op­er­a­tion is not sup­port­ed on SSD disk
• Red Hat Bugzil­la: Bug 1400824 – Fstrim/​​​Discard op­er­a­tion is not sup­port­ed for LVMRAID1 in RHEL7.3

Test Whether a De­vise Sup­ports TRIM

To test whether the SSD de­vice sup­ports TRIM/​​​Discard op­tion you can use ei­ther of the fol­low­ing com­mands.

sudo hdparm -I /dev/sda | grep TRIM

* Data Set Management TRIM supported (limit 8 blocks)

sudo smartctl --all /dev/sda | grep TRIM

TRIM Command:     Available

To test does an NVMe sup­ports TRIM (phys­i­cal dis­card op­tion) the out­put of the fol­low­ing com­mand must be greater than 0. This ap­proach is ap­plic­a­ble al­so for SSD de­vices.

cat /sys/block/nvme0n1/queue/discard_max_bytes

cat /sys/block/sda/queue/discard_max_bytes

grep . /sys/block/sd*/queue/discard_*   # Inspect all devices

In ad­di­tion if the columns DISC-GRAN and DISC-MAX with­in the out­put of ls­blk ‑D has val­ues the rel­e­vant de­vices sup­ports TRIM.

lsblk -dD

NAME    DISC-ALN DISC-GRAN DISC-MAX DISC-ZERO
sda            0        4K       2G         0   # SSD
sdb            0        0B       0B         0   # HDD
sr0            0        0B       0B         0   # DVD ROM
nvme0n1        0      512B       2T         0   # NVMe

Get Full SSD/​​​NVMe/​​​HDD De­vice In­fo

For SSD/HDD de­vices use:

smartctl -x /dev/sda            # -x, --xall; -a, --all

hdparm -I /dev/sda              # doesn't support NVMe

For NVMe de­vices use:

smartctl -x /dev/nvme0n1        # -x, --xall; -a, --all

nvme smart-log -H /dev/nvme0n1  # apt install nvme-cli

Ref­er­ences

• Chia De­cen­tral at YouTube: Read­ing NVMe SMART in Win­dows and Lin­ux
• NVMexpress​.org: NVMe Cur­rent Spec­i­fi­ca­tions
• NVMexpress​.org: NVMe Com­mand Line In­ter­face (NVMe-CLI)
• Lin­ux NVMe at GitHub: NVMe-CLI
• Smart­Mon­Tools: NVMe sup­port
• Per­cona: Us­ing NVMe Com­mand Line Tools to Check NVMe Flash Health
• Unix and Lin­ux: How to eval­u­ate the wear lev­el of a NVMe SSD?

[Batch dis­card] TRIM from CLI

sudo fstrim -av             # trim/discardd all mounted devices, verbose output
sudo fstrim -v /            # trim/discardd the root fs, verbose output
sudo fstrim /mount-point    # trim/discardd a device mounted at specific mount point

[Pe­ri­od­ic Dis­card] Set the TRIM Job to Dai­ly

sudo systemctl enable fstrim.timer

sudo mkdir /etc/systemd/system/fstrim.timer.d
sudo nano /etc/systemd/system/fstrim.timer.d/override.conf

[Timer]
OnCalendar=
OnCalendar=daily

Re­boot the sys­tem or do dae­mon-re­load and check the up­dat­ed val­ue.

sudo systemctl daemon-reload

systemctl cat fstrim.timer

# /lib/systemd/system/fstrim.timer
[Unit]
Description=Discard unused blocks once a week
Documentation=man:fstrim
ConditionVirtualization=!container

[Timer]
OnCalendar=weekly
AccuracySec=1h
Persistent=true

[Install]
WantedBy=timers.target

# /etc/systemd/system/fstrim.timer.d/override.conf
[Timer]
OnCalendar=
OnCalendar=daily

sudo systemctl status fstrim.service
sudo systemctl status fstrim.timer

In­spect the logs.

journalctl -u fstri

#Out­put

Aug 25 01:18:25 example.com systemd[1]: Starting Discard unused blocks on filesystems from /etc/fstab...
Aug 25 01:18:56 example.com fstrim[2387712]: /mnt/500GB-HT: 108.2 GiB (116132655104 bytes) trimmed on /dev/disk/by-uuid/b6ec78e6-bf2e-4cf0-bb1e-0042645af13b
Aug 25 01:18:56 example.com fstrim[2387712]: /mnt/2TB-WD-2: 149.6 GiB (160665714688 bytes) trimmed on /dev/disk/by-uuid/5d0b4c70-8936-41ef-8ac5-eba6f3142d9d
Aug 25 01:18:56 example.com fstrim[2387712]: /mnt/2TB-WD-1: 36.9 GiB (39626182656 bytes) trimmed on /dev/disk/by-uuid/9aa5e7bb-a0ca-463c-856b-86a3b9b4944a
Aug 25 01:18:56 example.com fstrim[2387712]: /: 0 B (0 bytes) trimmed on /dev/disk/by-uuid/09e7c8ed-fb55-4a44-8be4-18b1696fc714
Aug 25 01:18:56 example.com systemd[1]: fstrim.service: Deactivated successfully.
Aug 25 01:18:56 example.com systemd[1]: Finished Discard unused blocks on filesystems from /etc/fstab.
Aug 25 01:18:56 example.com systemd[1]: fstrim.service: Consumed 1.213s CPU time.

Un­do the change if you need.

sudo rm -v /etc/systemd/system/fstrim.timer.d/override.conf

Ref­er­ences

• Easy Lin­ux Tips Project: SSD – How to op­ti­mize your Sol­id State Dri­ve for Lin­ux Mint and Ubun­tu

[On­line Dis­card] En­able TRIM in /​​​etc/​​​fstab

This ap­proach is not rec­om­mend­ed by the most man­u­als.

sudo nano /etc/fstab

#/dev/disk/by-uuid/09e7c--18b1696fc714 / ext4 defaults 0 0
#/dev/disk/by-uuid/09e7c--18b1696fc714 / ext4 discard,defaults 0 0
#/dev/disk/by-uuid/09e7c--18b1696fc714 / ext4 discard,async,noatime,nodiratime,errors=remount-ro 0 1
/dev/disk/by-uuid/09e7c--18b1696fc714 / ext4 discard,errors=remount-ro 0 1

Warn­ing: Users need to be cer­tain that their block de­vice sup­ports TRIM be­fore at­tempt­ing to use the dis­card mount­ing op­tion. Oth­er­wise da­ta loss can oc­cur!

Prox­moxVE Node

Batch dis­card of the guests

For unpriv­i­leged Lin­ux con­tain­ers use the fol­low­ing com­mand at the Prox­moxVE host side .

pct fstrim 151              # Where 151 is the Id of the container

For priv­i­leged con­tain­ers fstrim could be ex­e­cut­ed al­so from the guest's side. For the vir­tu­al ma­chines – en­able the dis­card and SSD em­u­la­tion op­tions in the stor­age de­vice con­fig­u­ra­tion, use Vir­tIo SC­SI as con­troller and trim at the guest side. Al­so we can use one of the fol­low­ing com­mands at he Prox­moxVE host side.

qm guest cmd 251 fstrim     # Where 251 is the Id of the container

qm agent 251 fstrim         # Where 251 is the Id of the container

Notes about the PVE Node (the PVE host)

1. Sched­ule fstrim to run week­ly on PE hosts.
2. Use the Vir­tio SC­SI dri­ver for VMs when pos­si­ble, en­abling the dis­card op­tion each time?!?
3. Sched­ule TRIM (eg fstrim in Lin­ux) to run week­ly in guest VMs.
4. Sched­ule fstrim to run week­ly in PE con­tain­ers.

Ref­er­ences

• Prox­mox Fo­rum: TRIM SS­Ds

Tweak the APM Lev­el of SSD

Most Lin­ux dis­tri­b­u­tions use Lin­ux Kernel’s “Ad­vanced Pow­er Man­age­ment (APM)” API to han­dle con­fig­u­ra­tion, op­ti­mize per­for­mance, and en­sure sta­bil­i­ty of stor­age de­vices. These de­vices are as­signed an APM val­ue be­tween 1 and 255 to con­trol their pow­er man­age­ment thresh­olds. A val­ue of 254 in­di­cates best per­for­mance, while a val­ue of 1 in­di­cates bet­ter pow­er man­age­ment. As­sign­ing a val­ue of 255 will dis­able APM al­to­geth­er. By de­fault, SS­Ds are as­signed an APM of 254 when the sys­tem is run­ning on ex­ter­nal pow­er. In bat­tery mode, the APM lev­el is set to 128, re­duc­ing the read and write speeds of SS­Ds. This ar­ti­cle ex­plains how to in­crease SSD APM lev­els to 254 when your Lin­ux lap­top is run­ning on bat­tery mode.

sudo hdparm -B254 /dev/sda

Get the cur­rent AMP val­ue.

sudo hdparm -B /dev/sda

Test the per­for­mance.

sudo hdparm -tT /dev/sda

Ref­er­ences

• Ask Ubun­tu: Ubun­tu SSD – Was fast, is now ex­treme­ly slow
• Lin­ux­Hint: How to Im­prove SSD Per­for­mance in Lin­ux Lap­tops
• Lin­ux Mag­a­zine: Tune Your Hard Disk with hd­parm
• Lin­run­ner: TLP FAQ: Disk Dri­ves

See al­so

• Lin­ux Swap and Swap­file
• Pre­load Tool for Bet­ter Sys­tem Per­for­mance
• Arch Lin­ux Wi­ki: Im­prov­ing per­for­mance (Need to read care­ful­ly!)
• Arch Lin­ux Wi­ki: Ext4 (Need to read care­ful­ly!)
• Web­site for Stu­dents: Im­prove Ng­inx Cache Per­for­mance with tmpfs on Ubun­tu
• Ask Ubun­tu: How do I op­ti­mize the OS for SS­Ds?
• How-To Geek: How to Tweak Your SSD in Ubun­tu for Bet­ter Per­for­mance
• How-To Geek: What Is the Lin­ux fstab File, and How Does It Work?
• SSD Sphere: Why you should not wor­ry about SSD’s TBW and MTBF?
• GBMB​.org: PB to TB Con­ver­sion

Reg­is­ter: Lin­ux Desk­top

Идентифициране

Разработване

Авторизиране

Утвърждаване

Spas

Spas

Spas

Spas

2.07.2022

18.08.2022

18.08.2022

26.09.2022

27114

30332

30333 30328

31951 30330