Tmpfs Tools Profile Sync Daemon (PSD): Difference between revisions

From WikiMLT
mNo edit summary
m (Text replacement - "I/O Monitoring and Analyze" to "Linux I/O Monitoring and Analyze")
 
(9 intermediate revisions by the same user not shown)
Line 1: Line 1:
<noinclude>{{ContentArticleHeader/Linux_Server|toc=off}}{{ContentArticleHeader/Linux_Desktop}}</noinclude>
<noinclude>{{ContentArticleHeader/Linux_Server|toc=off}}{{ContentArticleHeader/Linux_Desktop}}</noinclude>
''[https://github.com/graysky2/profile-sync-daemon Profile-sync-daemon] (psd) is a tiny pseudo-daemon designed to manage your browser's profile in <code>tmpfs</code> and to periodically sync it back to your physical disc (HDD/SSD). This is accomplished via a symlinking step and an innovative use of <code>rsync</code> to maintain back-up and synchronization between the two. One of the major design goals of <code>psd</code> is a completely transparent user experience.''


== Preparation ==
It is designed to work on desktop systems, but by creating custom fake profiles it could be used for non production servers :) Currently I'm using it to minimize [[Kali Linux Install Brave browser|Brave browser]] disk writings. More information about the PSD's Brave profile could be foun in the references section below.
Identify which directories has high [[I/O Monitoring and Analyze|I/O rate]] by the following command.<syntaxhighlight lang="shell" line="1" class="force-prompt">
 
sudo watch -d -n 1 "find /var -type f -size +80k -mmin -10 -printf '%-30s \t %t %p\n'"
{{collapse/begin}}
== Identify the Problem ==
{{collapse/div|#Preparation details}}
Identify which directories has high [[Linux I/O Monitoring and Analyze|I/O rate]] by the following command.<syntaxhighlight lang="shell" line="1" class="force-prompt">
sudo watch -d -n 1 "find ~/.cache ~/.config -type f -size +80k -mmin -10 -printf '%-30s \t %t %p\n'"
</syntaxhighlight>Check the current size of the identified directories.<syntaxhighlight lang="shell" line="1" class="force-prompt">
</syntaxhighlight>Check the current size of the identified directories.<syntaxhighlight lang="shell" line="1" class="force-prompt">
du -hs /var/log
du -hs ~/.cache ~/.config
</syntaxhighlight>Take a statistic for the average disk write per minute before the setup.<syntaxhighlight lang="shell" line="1" class="force-prompt">
</syntaxhighlight>Take a statistic for the average disk write per minute before the setup.<syntaxhighlight lang="shell" line="1" class="force-prompt">
iostat -h /dev/nvme0n1 -d 60 -t
iostat -h /dev/nvme0n1 -d 60 -t
</syntaxhighlight>
</syntaxhighlight>
{{collapse/end}}


== Install log2ram ==
== Install PSD ==
<syntaxhighlight lang="shell" line="1" class="force-prompt">
The PSD tool is available as [https://github.com/graysky2/profile-sync-daemon#installation-from-distro-packages package] for many Linux distributions. Here is how to install on Debian/Kali.<syntaxhighlight lang="shell" line="1" class="force-prompt">
echo "deb [signed-by=/usr/share/keyrings/azlux-archive-keyring.gpg] http://packages.azlux.fr/debian/ bullseye main" | sudo tee /etc/apt/sources.list.d/azlux.list
sudo apt install profile-sync-daemon
sudo wget -O /usr/share/keyrings/azlux-archive-keyring.gpg  https://azlux.fr/repo.gpg
</syntaxhighlight>However the Debian's package is little bit older but works stable. If you want the latest version [https://github.com/graysky2/profile-sync-daemon/blob/master/INSTALL install] it from its GitHub repository by performing the following steps.<syntaxhighlight lang="shell" line="1" class="force-prompt">
sudo apt update
cd ~/Git # or ~/Downloads or where you prefer
sudo apt install log2ram
git clone https://github.com/graysky2/profile-sync-daemon.git
cd profile-sync-daemon
make
sudo make install
</syntaxhighlight>
</syntaxhighlight>
== Setup PSD ==
Within the latest version on GitHub is available a [https://github.com/graysky2/profile-sync-daemon/blob/master/contrib/brave profile for Brave browser] also there is a [https://github.com/graysky2/profile-sync-daemon/pull/240 discussion] about it. However I prefer to create two custom profiles: 1) the first will handle <code>~/.config/BraveSoftware/Brave-Browser</code>, and 2) the second will handle <code>~/.cache/BraveSoftware/Brave-Browser</code>.<syntaxhighlight lang="shell" line="1">
echo -e 'DIRArr[0]="$XDG_CONFIG_HOME/BraveSoftware/Brave-Browser"\nPSNAME="brave"' | \
sudo tee /usr/share/psd/browsers/brave
echo -e 'DIRArr[0]="'"$HOME"'/.cache/BraveSoftware/Brave-Browser"\nPSNAME="brave-cache"' | \
sudo tee /usr/share/psd/browsers/brave-cache
</syntaxhighlight>The next step is to add brave and brave-cache to the PSD user's configuration file - <code>psd.conf</code>.<syntaxhighlight lang="shell" line="1">
echo -e '\nBROWSERS="brave brave-cache"' >> ~/.config/psd/psd.conf
</syntaxhighlight>I prefer also to enable the use of <code>overlayfs</code> mode to improve sync speed and to use a smaller memory footprint. The first thing according to that is to add entry in the <code>psd.conf</code> file, and also an entry into the <code>sudoers.d</code> directory, because this feature requires more permissions. <syntaxhighlight lang="shell" line="1">
echo -e '\nUSE_OVERLAYFS="yes"' >> ~/.config/psd/psd.conf
echo "$USER ALL=(ALL) NOPASSWD: /usr/bin/psd-overlay-helper" | \
sudo tee -a /etc/sudoers.d/psd-overlay-helper
</syntaxhighlight>
* Note it is better and save to use just <code>echo "$USER ALL=(ALL) NOPASSWD: /usr/bin/psd-overlay-helper"</code> to generate the line and the use <code>sudo visudo /etc/sudoers.d/psd-overlay-helper</code>.
In addition you can enable the suspend-sync feature.<syntaxhighlight lang="shell" line="1">
echo -e '\nUSE_SUSPSYNC="yes"' >> ~/.config/psd/psd.conf
</syntaxhighlight>This feature requires <code>glib2</code> library, it could be installed on nowadays Debian/Ubuntu/Kali/etc. via the following [https://askubuntu.com/a/58357/566421 command].<syntaxhighlight lang="shell" line="1">
sudo apt install libglib2.0-dev
</syntaxhighlight>
== Usage ==
You can use <code>man psd</code> to get detailed information. However here are few essential commands.
'''PREVIEW MODE.''' The preview option can be called to show users exactly what <code>psd</code> will do/is doing based on the entries in the config file. It will also provide useful information such as profile size, paths, and if any recovery snapshots have been created.<syntaxhighlight lang="shell" line="1">
psd p
</syntaxhighlight><syntaxhighlight lang="terraform">
Profile-sync-daemon v6.45
systemd service: active
resync-timer:    active
sync on sleep:  disabled
use overlayfs:  enabled


== Setup log2ram ==
Psd will manage the following per /home/<user>/.config/psd/.psd.conf:
<syntaxhighlight lang="shell" line="1" class="force-prompt">
sudo nano /etc/log2ram.conf
</syntaxhighlight><syntaxhighlight lang="bash" line="1">
SIZE=2G
MAIL=true
PATH_DISK="/var/log"
ZL2R=false
COMP_ALG=lz4
LOG_DISK_SIZE=100M


</syntaxhighlight>Modify the default systemd timer unit, if you want to log more or less frequently than one time per day.<syntaxhighlight lang="shell" line="1" class="force-prompt">
browser/psname:  brave/brave
sudo systemctl edit log2ram-daily.timer
owner/group id:  <user>/1000
</syntaxhighlight>Reboot the system. After the reboot check [https://github.com/azlux/log2ram#is-it-working does it working] and use the <code>iostat</code> command to take a new statistic.
sync target:    /home/<user>/.config/BraveSoftware/Brave-Browser
tmpfs dir:      /run/user/1000/psd/<user>-brave
profile size:    420M
overlayfs size:  85M
recovery dirs:  none
 
browser/psname:  brave-cache/brave-cache
owner/group id:  <user>/1000
sync target:    /home/<user>/.cache/BraveSoftware/Brave-Browser
tmpfs dir:      /run/user/1000/psd/<user>-brave-cache
profile size:    277M
overlayfs size:  21M
recovery dirs:  none
 
</syntaxhighlight>'''START AND STOP PSD.''' Psd  ships  with a systemd user service to start or stop it (<code>psd.service</code>). Additionally, a provided <code>resync-timer</code> will run an hourly <code>resync</code> from <code>tmpfs</code> back to the disk. The <code>resync-timer</code> is started automatically with <code>psd.service</code> so there is no need to start the timer; only start <code>psd.service</code>.<syntaxhighlight lang="shell" line="1">
systemctl --user status psd.service
systemctl --user start psd.service
systemctl --user stop psd.service
systemctl --user enable psd.service
systemctl --user disable psd.service
</syntaxhighlight>Tweak the <code>rsync-timer</code>.<syntaxhighlight lang="shell" line="1">
systemctl --user cat psd-resync.timer
systemctl --user edit psd-resync.timer
</syntaxhighlight>'''CLEAN MODE.''' The clean mode will delete ALL recovery snapshots that have accumulated. Only run it when sure these are no longer needed.<syntaxhighlight lang="shell" line="1">
psd c
</syntaxhighlight>


== References ==
== References ==
 
* GitHub: [https://github.com/graysky2/profile-sync-daemon Profile-sync-daemon (psd)] | [https://github.com/graysky2/profile-sync-daemon/pull/240 Add Brave Browser #240]
* GitHub: [https://github.com/azlux/log2ram log2ram]
* Arch Linux Wiki: [https://wiki.archlinux.org/title/Profile-sync-daemon Profile-sync-daemon]
* ArchLinux Wiki: [https://wiki.archlinux.org/title/Firefox/Profile_on_RAM Firefox/Profile on RAM]
* Arch Linux Wiki: [https://wiki.archlinux.org/title/Firefox/Profile_on_RAM Firefox/Profile on RAM]
* Linuxhowto.net: [https://www.linuxhowto.net/how-to-write-log-files-in-ram-using-log2ram-in-linux/ Write Log Files In RAM Using Log2ram In Linux]<br />
* Arch Linux Wiki: [https://wiki.archlinux.org/title/Chromium#Cache_in_tmpfs Chromium/Cache in tmpfs]
* Arch Linux Wiki: [https://wiki.archlinux.org/title/XDG_Base_Directory XDG Base Directory]<br />


<noinclude>
<noinclude>
Line 44: Line 102:
  | Прндл  = Linux Desktop
  | Прндл  = Linux Desktop
  | Прндл1 = Linux Server
  | Прндл1 = Linux Server
  | Стадий = 3
  | Стадий = 6
  | Фаза  = Разработване
  | Фаза  = Утвърждаване
  | Статус = Разработван
  | Статус = Утвърден
  | ИдтПт  = Spas
  | ИдтПт  = Spas
  | РзбПт  = {{REVISIONUSER}}
  | РзбПт = Spas
| АвтПт  = Spas
| УтвПт = {{REVISIONUSER}}
  | ИдтДт  = 31.08.2022
  | ИдтДт  = 31.08.2022
  | РзбДт  = {{Today}}
  | РзбДт = 31.08.2022
| АвтДт  = 31.08.2022
| УтвДт = {{Today}}
  | ИдтРв  = [[Special:Permalink/30872|30872]]
  | ИдтРв  = [[Special:Permalink/30872|30872]]
  | РзбРв  = {{REVISIONID}}
  | РзбРв = [[Special:Permalink/30883|30883]]
| АвтРв  = [[Special:Permalink/30884|30884]]
| УтвРв = {{REVISIONID}}
}}
}}
</div>
</div>
</noinclude>
</noinclude>

Latest revision as of 10:18, 3 September 2022

Pro­file-sync-dae­mon (psd) is a tiny pseu­do-dae­mon de­signed to man­age your browser's pro­file in tmpfs and to pe­ri­od­i­cal­ly sync it back to your phys­i­cal disc (HDD/SSD). This is ac­com­plished via a sym­link­ing step and an in­no­v­a­tive use of rsync to main­tain back-up and syn­chro­niza­tion be­tween the two. One of the ma­jor de­sign goals of psd is a com­plete­ly trans­par­ent user ex­pe­ri­ence.

It is de­signed to work on desk­top sys­tems, but by cre­at­ing cus­tom fake pro­files it could be used for non pro­duc­tion servers :) Cur­rent­ly I'm us­ing it to min­i­mize Brave brows­er disk writ­ings. More in­for­ma­tion about the PSD's Brave pro­file could be foun in the ref­er­ences sec­tion be­low.

Iden­ti­fy the Prob­lem

#Prepa­ra­tion de­tails
Iden­ti­fy which di­rec­to­ries has high I/O rate by the fol­low­ing com­mand.
sudo watch -d -n 1 "find ~/.cache ~/.config -type f -size +80k -mmin -10 -printf '%-30s \t %t %p\n'"
Check the cur­rent size of the iden­ti­fied di­rec­to­ries.
du -hs ~/.cache ~/.config
Take a sta­tis­tic for the av­er­age disk write per minute be­fore the set­up.
iostat -h /dev/nvme0n1 -d 60 -t

In­stall PSD

The PSD tool is avail­able as pack­age for many Lin­ux dis­tri­b­u­tions. Here is how to in­stall on Debian/​​​Kali.

sudo apt install profile-sync-daemon

How­ev­er the Debian's pack­age is lit­tle bit old­er but works sta­ble. If you want the lat­est ver­sion in­stall it from its GitHub repos­i­to­ry by per­form­ing the fol­low­ing steps.

cd ~/Git # or ~/Downloads or where you prefer
git clone https://github.com/graysky2/profile-sync-daemon.git
cd profile-sync-daemon
make
sudo make install

Set­up PSD

With­in the lat­est ver­sion on GitHub is avail­able a pro­file for Brave brows­er al­so there is a dis­cus­sion about it. How­ev­er I pre­fer to cre­ate two cus­tom pro­files: 1) the first will han­dle ~/.config/BraveSoftware/Brave-Browser, and 2) the sec­ond will han­dle ~/.cache/BraveSoftware/Brave-Browser.

echo -e 'DIRArr[0]="$XDG_CONFIG_HOME/BraveSoftware/Brave-Browser"\nPSNAME="brave"' | \
sudo tee /usr/share/psd/browsers/brave
echo -e 'DIRArr[0]="'"$HOME"'/.cache/BraveSoftware/Brave-Browser"\nPSNAME="brave-cache"' | \
sudo tee /usr/share/psd/browsers/brave-cache

The next step is to add brave and brave-cache to the PSD user's con­fig­u­ra­tion file – psd.conf.

echo -e '\nBROWSERS="brave brave-cache"' >> ~/.config/psd/psd.conf

I pre­fer al­so to en­able the use of over­layfs mode to im­prove sync speed and to use a small­er mem­o­ry foot­print. The first thing ac­cord­ing to that is to add en­try in the psd.conf file, and al­so an en­try in­to the sudoers.d di­rec­to­ry, be­cause this fea­ture re­quires more per­mis­sions.

echo -e '\nUSE_OVERLAYFS="yes"' >> ~/.config/psd/psd.conf
echo "$USER ALL=(ALL) NOPASSWD: /usr/bin/psd-overlay-helper" | \
sudo tee -a /etc/sudoers.d/psd-overlay-helper
  • Note it is bet­ter and save to use just echo "$USER ALL=(ALL) NOPASS­WD: /us­r/bin/psd-over­lay-helper" to gen­er­ate the line and the use su­do vi­su­do /etc/sudoers.d/psd-overlay-helper.

In ad­di­tion you can en­able the sus­pend-sync fea­ture.

echo -e '\nUSE_SUSPSYNC="yes"' >> ~/.config/psd/psd.conf

This fea­ture re­quires glib2 li­brary, it could be in­stalled on nowa­days Debian/​​​Ubuntu/​​​Kali/​​​etc. via the fol­low­ing com­mand.

sudo apt install libglib2.0-dev

Us­age

You can use man psd to get de­tailed in­for­ma­tion. How­ev­er here are few es­sen­tial com­mands.

PRE­VIEW MODE. The pre­view op­tion can be called to show users ex­act­ly what psd will do/​​​is do­ing based on the en­tries in the con­fig file. It will al­so pro­vide use­ful in­for­ma­tion such as pro­file size, paths, and if any re­cov­ery snap­shots have been cre­at­ed.

psd p
Profile-sync-daemon v6.45

 systemd service: active
 resync-timer:    active
 sync on sleep:   disabled
 use overlayfs:   enabled

Psd will manage the following per /home/<user>/.config/psd/.psd.conf:

 browser/psname:  brave/brave
 owner/group id:  <user>/1000
 sync target:     /home/<user>/.config/BraveSoftware/Brave-Browser
 tmpfs dir:       /run/user/1000/psd/<user>-brave
 profile size:    420M
 overlayfs size:  85M
 recovery dirs:   none

 browser/psname:  brave-cache/brave-cache
 owner/group id:  <user>/1000
 sync target:     /home/<user>/.cache/BraveSoftware/Brave-Browser
 tmpfs dir:       /run/user/1000/psd/<user>-brave-cache
 profile size:    277M
 overlayfs size:  21M
 recovery dirs:   none

START AND STOP PSD. Psd  ships  with a sys­temd user ser­vice to start or stop it (psd.service). Ad­di­tion­al­ly, a pro­vid­ed re­sync-timer will run an hourly re­sync from tmpfs back to the disk. The re­sync-timer is start­ed au­to­mat­i­cal­ly with psd.service so there is no need to start the timer; on­ly start psd.service.

systemctl --user status psd.service
systemctl --user start psd.service
systemctl --user stop psd.service
systemctl --user enable psd.service
systemctl --user disable psd.service

Tweak the rsync-timer.

systemctl --user cat psd-resync.timer
systemctl --user edit psd-resync.timer

CLEAN MODE. The clean mode will delete ALL re­cov­ery snap­shots that have ac­cu­mu­lat­ed. On­ly run it when sure these are no longer need­ed.

psd c

Ref­er­ences