[swapper] Virtual Memory

Black Hole

Black Hole

May contain traces of nut
This package doesn't seem to have an existing dedicated topic, so here it is! Note: introductory post updated to reflect later discussion.

swapper provides HDR-FOX with virtual memory to supplement the existing (limited) RAM. This is of general benefit for WebIF processing options, but particularly youtube-dl / qtube users, where the processing chain is likely to crash because of the RAM limitations. swapper invokes a swap file on the HDD (HDR-FOX). swapper is installed automatically, as an indirect dependency of web-if.

swapper is not currently compatible with HD-FOX (despite being installed automatically). However, it can be tweaked for HD-FOX – see post 2.

If you find a process is still running out of RAM, the swap space (128MB as standard) can be permanently increased by another tweak – see post 5.
 
Last edited:
The youtube-dl package runs into trouble if the download is too large, particularly on HD-FOX, because the ffmpeg clean-up runs out of memory. swapper is useful here, by making virtual memory available, but is not (in original form) compatible with the HD-FOX (due to any available disk space being external on USB). Instructions for manual set-up have been provided:
prpr said:
Code: 
humax# free -m
humax# dd if=/dev/zero of=/media/drive1/.swap0 bs=1M count=128
humax# mkswap /media/drive1/.swap0
humax# swapon /media/drive1/.swap0
humax# free -m
and to get rid of it again:
Code: 
humax# swapoff /media/drive1/.swap0
humax# rm /media/drive1/.swap0
Click to expand...

...but a modification to the swapper package itself has been proposed, which will make manual set-up on the HD-FOX (repeated after every reboot) unnecessary:
prpr said:
Permanent fix:
Code: 
--- /mod/etc/init.d/S00swapper~
+++ /mod/etc/init.d/S00swapper
@@ -1,8 +1,7 @@
#!/bin/sh

-[ "`cat /etc/model`" = "HDR" ] || exit 0
-
swapfile=/mnt/hd3/.swap0
+[ "`cat /etc/model`" = "HD" ] && swapfile=/media/drive1/.swap0

case "$1" in
     start)
Click to expand...

My question is: has this been incorporated into the package yet?
 
A recent indication that ffmpeg was running out of memory when patching up a youtube-dl download, even with swapper installed, led me to investigate increasing the size of the swap file:
Black Hole said:
The swapper package sets up swap space each boot (128MB), but equally the swapper thread demonstrates how to set up a 128MB swap manually, which can be adapted to to 256MB or whatever you like (but needs turning on again after boot). So, to make it double (I guess that will be enough, considering the crash is near the end of the processing):
Code: 
swapoff /mnt/hd3/.swap0
rm /mnt/hd3/.swap0
dd if=/dev/zero of=/mnt/hd3/.swap0 bs=1M count=256
mkswap /mnt/hd3/.swap0
swapon /mnt/hd3/.swap0

Google "linux help dd" for dd command explanation (like wot I just did), and you'll see that:
  • "if" means "Input File", so "=/dev/zero" means it is just sourcing a string of zero bytes;
  • "of" means "Output File", creating the .swap0 file;
  • "bs" is the number of bytes in a block;
  • "count" is the number of blocks
...in other words, it creates a 256MB empty .swap0 file on hd3 by copying that many bytes across from a null source.

The rm... command might not be strictly necessary, but I preferred to start with a clean slate.
Click to expand...

The swapper package instates swap space each boot, but uses whatever size /mnt/hd3/.swap0 happens to be, so setting the size (as above) is a one-off operation (the default is 128MB). Just change "count=" in the commands above to however many megabytes you want.

Without swapper installed, the above commands can be used to create swap space, but the swapon command will be needed to make the swap space available after each boot.
 
In the swapper update just published, you can now set the size of the swap file by writing the desired number of MB into /mod/boot/swapsize
e.g.
Code: 
humax# echo 512 >/mod/boot/swapsize
which will take effect the next time the machine is booted, or you can run /mod/etc/init.d/S00swapper stop followed by /mod/etc/init.d/S00swapper start.
If you want to go back to the default (128 256), then just delete the /mod/boot/swapsize file.
 
Last edited:
I wonder whether the larger swap size would help reduce some of the seemingly random crashes that humax is subject to,
 
bottletop said:
Will it be suitable to set the default size 256MB or 512MB for all? It won't take up much drive space.
Click to expand...
Probably.
Will it be detrimental to anyone using the standard 500GB (or greater) drive?
Click to expand...
Probably not, as it's on a partition that is virtually unused.
Will it help with the maintenance mode fix-disk as well?
Click to expand...
No. That uses its own separate swap file.
MymsMan said:
I wonder whether the larger swap size would help reduce some of the seemingly random crashes that humax is subject to
Click to expand...
I doubt it, seeing as it doesn't get much use according to the free command.
 
I've updated the package to default to 256.
If people want more for whatever reason, the method in post #9 tells you how.
 
I think my HDR is now taking longer to boot up - due to it creating new swapfile during boot.
I don't have the evidence, but I thought it use to just use a swap file if there was one there already and only create a new one during boot if necessary.
Eg I have an old and new modinit.log
Code: 
old modinit.log
=== Thu Apr 16 07:32:28 UTC 2020 - run init ===========================
Setting up swapspace version 1, size = 1073737728 bytes
UUID=35d6ed8a-3ca9-43b4-a27a-124e45b615e9
    Thu Apr 16 07:32:28 UTC 2020 - S00swapper (elapsed: 4s)
Disabling TCP window scaling
net.ipv4.tcp_window_scaling = 0
    Thu Apr 16 07:32:32 UTC 2020 - S00tcpfix (elapsed: 0s)

new modinit.log
=== Sat Jun 29 11:33:42 UTC 2024 - run init ===========================
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 28.9587 s, 37.1 MB/s
Setting up swapspace version 1, size = 1073737728 bytes
UUID=5d954ee8-d1b6-4dfe-ad21-35b06ef3ec27
Enabled swap file.
    Sat Jun 29 11:33:43 UTC 2024 - S00swapper (elapsed: 32s)
Disabling TCP window scaling
Alternatively - I can try installing the old version of swapper instead. Unfortunately I'm not sure how.
 
Last edited:
It should only recreate the swap file if the desired size has changed - at least that is the intention with the latest version of the package.
 
I think that's what it use to do, but no longer. Eg looking at ls -al /mnt/hd3 the swap file is freshly created during boot (also on the modinit.log).
 
You must log in or register to reply here.
Back
Top