Wednesday, October 13, 2021

Recent Questions - Unix & Linux Stack Exchange

Recent Questions - Unix & Linux Stack Exchange


Why doesn't this bash function work? (tmux and docker related)

Posted: 13 Oct 2021 11:03 AM PDT

I often reattach a docker container running weechat, so I thought I'd make this function to rename the tmux window if within tmux, but tmux hangs and then crashes when I run it.

docker() {      if [[ $TMUX ]] && [[ "$1" = "attach" ]]; then          tmux rename-window "$2"          docker "$@"      else          docker "$@"      fi  }  

Functions don't run within their own subshell, so I'm wondering what's going on here?

Internet tethering from Samsung to kali linux rpi3

Posted: 13 Oct 2021 10:59 AM PDT

I have a raspberry pi model 3 and I installed Kali Linux on it. I want to control it through my phone using SSH but I don't want to connect it to my phone's Internet using WiFi, instead I want to connect it using USB tethering.

I tried editing /etc/network/ interfaces by adding this code Auto usbo Iface usbo int static Address 192.168.42.42 Netmask 255.255.255.0 and it still doesn't work.

Whenever I connect my phone to it and use the ip address command it detects the usbo but it says DOWN. What's wrong?

FFmpeg is not giving the accurate audio duration

Posted: 13 Oct 2021 10:25 AM PDT

I have an audio file that I want to trim/cut from 00:00:10.840 to 00:00:15.520, but instead of output the audio for the giving specific duration, it output the audio five seconds more (trimmed to: 00:00:21.783)

ffmpeg -ss 00:00:10.840 -i out.wav -t 00:00:15.520 -c copy out.wav

What files should be deleted if i don't want anyone to know if the system has been once linked to the internet?

Posted: 13 Oct 2021 10:18 AM PDT

For certain reasons I have to delete all files that might indicate that the syatem has once been linked to the internet. I have deleted all the records in browser, and the internet ethernet address. What else I can do, such that no one is able to know that the system has linked to the internet? Can I delete all files in var/log? This is a new system with only the anaconda and relavant pachakges been installed.

How do you turn the exit status of a program into the signal name

Posted: 13 Oct 2021 11:11 AM PDT

pid_t waitpid(pid_t pid, int *status, int options);  

For example when the status=2, return the string "SIGINT" when the status=11, return the string "SIGSEGV". Is there a built_in c function that takes in the status and outputs the signal name?

Bash autocomplete: select random suggestion

Posted: 13 Oct 2021 10:33 AM PDT

Basically I want a way for figlet to choose a random font. It doesn't seem to have an inbuilt method, or at least I haven't found one in the manual or in the web.

I want the message to pop up every time I open up a new xterm, which is easy enough adding figlet -f <a font> Hello World to bash.rc

But I also want the font to be random. I'm wondering if there is a way for <a font> to be selected randomly from the autocomplete suggestions.

I see the sizes of the files on a disk incorrectly

Posted: 13 Oct 2021 10:05 AM PDT

Running services are filling up disk space. I reboot it frees up disk space. How can I find files created by programs in disk space but not visible with the "du -ch /.[!.]* /disk1" command.

$ df -Th

/dev/mapper/ubuntu--vg-ubuntu--lv        ext4       15G  9.5G  3.9G  71% /  /dev/xvdb1                               ext4      361G  333G   11G  97% /disk1  

$ du -sh /disk1

203G /disk1/

Ubuntu 18.04.3 LTS The server is a virtual on xenserver. Two virtual disks named root and disk1 are attached. /disk1 size is 366GB but the file size is 205GB. The remaining 161GB space appears to be full, only 10GB of free space is available. More than one apache and special services running on this disk are working by showing this disk. But after a while the disk space gets full.

SELF- TEST FAILED. Gnome disks

Posted: 13 Oct 2021 09:32 AM PDT

I tried to scan the disk health but got this error. Image: https://photos.app.goo.gl/oU7TxEnBjmuEcahh6 Help, please. Video: https://youtu.be/FEmR0C8Fhx8 I have back up to Google Drive so I am not worried at all. Thanks, Jonathan. The read error rate went from 1 to 2 fast. Does it need replacing?

How to verify if a given package is built in a reproducible way as an end user?

Posted: 13 Oct 2021 10:28 AM PDT

Let's I would like to verify if the package mksh can be built in a reproducible way. I am trying with

apt build-dep mksh  apt source mksh  cd mksh; dpkg-buildpackage -uc -us  cd ..; sha256sum <freshly build dep>  

If I now do

apt download mksh  

and compare the checksum of the downloaded deb with the debian package I created locally, the checksum differs (expected as I did not sign the deb)

How to make those checksums match ?

Detect USB FT232R USB UART and run script

Posted: 13 Oct 2021 09:26 AM PDT

I have ser2net working perfectly with a USB FT232R UART (Creality 3D printer) using setserial to work with the higher speed of 250000 baud. The problem is that when I turn off the printer and turn back on, I have to run again the setserial commands to make the port available for ser2net. Is there a way to automate this upon connection?

The commands i need to run:

setserial -a /dev/ttyUSB0 spd_cust

setserial -a /dev/ttyUSB0 divisor 96

Kind regards, Eric

Where are my lutris games saved?

Posted: 13 Oct 2021 09:20 AM PDT

Where do I need to look for the directory where games installed with lutris are saved? I didn't find anything in ~/.local/share/lutris or in ~/.wine.

Adding exectuable directory to path has no effect

Posted: 13 Oct 2021 09:22 AM PDT

I have a program called "spotbugs" that I would like to be able to run globally on my machine. It resides in a directory that I have, and i dependant on other files in the same directory. (it's the one called "spotbugs", here)

┌──(kali㉿kali)-[~/Downloads/spotbugs-4.4.2/bin]  └─$ ls  addMessages        copyBuggySource  experimental  filterBugs           mineBugHistory   rejarForAnalysis    spotbugs2     unionBugs  computeBugHistory  defectDensity    fb            findbugs-msv         printAppVersion  setBugDatabaseInfo  spotbugs.bat  xpathFind  convertXmlToText   deprecated       fbwrap        listBugDatabaseInfo  printClass       spotbugs            spotbugs.ico  

If you google around for this, the most common answer is that it should be added to my path.

So Go through the following steps:

┌──(kali㉿kali)-[~/Downloads/spotbugs-4.4.2/bin]  └─$ $PATH  zsh: no such file or directory: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/games:/usr/games                                                                                                                                            ┌──(kali㉿kali)-[~/Downloads/spotbugs-4.4.2/bin]  └─$ export PATH=$PATH:/home/kali/Downloads/spotbugs-4.4.2/bin                                                                     127 ⨯                                                                                                                                            ┌──(kali㉿kali)-[~/Downloads/spotbugs-4.4.2/bin]  └─$ $PATH                                                      zsh: no such file or directory: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/games:/usr/games:/home/kali/Downloads/spotbugs-4.4.2/bin                                                                                                                                            ┌──(kali㉿kali)-[~/Downloads/spotbugs-4.4.2/bin]  └─$ cd ..                                                                                                                         127 ⨯                                                                                                                                            ┌──(kali㉿kali)-[~/Downloads/spotbugs-4.4.2]  └─$ spotbugs    zsh: permission denied: spotbugs  ┌──(kali㉿kali)-[~/Downloads/spotbugs-4.4.2]  └─$ sudo spotbugs  sudo: spotbugs: command not found                                         

So this has no effect on anything, I am not able to run the executable anywhere other than when standing in it's directory.

HOw else can I make it glboally available?

Shell-script extract value pair with sed/awk?

Posted: 13 Oct 2021 09:06 AM PDT

Given a string like this:

NAME="bla" TYPE="test" SIZE="10"  

I want to get the values for each pair into into separate shell-script vars. So, for example, say we have three variables I want to load the values bla in the first variable, test in the second and 10 in the third.

name=bla  type=test  size=10  

What I tried so far:

echo \"NAME=\"bla\" TYPE=\"test\" SIZE=\"10\"" | awk '{print Var, $1}'  

But that would return the entire key-value pair, instead of just the value. Is there a nice way to do this with sed or awk?

Substract date in custom format

Posted: 13 Oct 2021 10:54 AM PDT

I need to substract some date from current date to determine the age of some connections. The date is in this format: YYMMDDHHMISS. The output can be pretty much anything that I can use to determine lets say oldest connection, number of connections older than 1 hour, 1 week, 1 month.

Example of input date: 211013134247

I got to the point of getting date in this format from date command (date '+%g%m%d%H%M%S'), but I don't know how to correctly substract this to get accurate result.


I tried this, but this get problematic as soon as one number from connection date (like seconds) is bigger than current date. Example: connection has 53 as seconds and current seconds are 47.

$line = connection date
$date = current date

  year=   $((${date:0:2} - ${line:0:2}))    month=  $((${date:2:2} - ${line:2:2}))    day=    $((${date:4:2} - ${line:4:2}))    hour=   $((${date:6:2} - ${line:6:2}))    minute= $((${date:8:2} - ${line:8:2}))    second= $((${date:10:2} - ${line:10:2}))       age=$year$month$day$hour$minute$second  

I also tried this (some code shamelessly stolen from stack exchange), but the output is wrong.. I would expect something like: 0607********, as in 6 years 7 months etc.. Input dates generally should not be years apart, so year calculation could be left out.

date="211013150325"  line="150325113419"  convert_date(){ printf `date '+%C'`'%s-%s-%s %s:%s:%s' ${1:0:2} ${1:2:2} ${1:4:2} ${1:6:2} ${1:8:2} ${1:10:2}; }  diff_dates(){      TS1=$(date -d "$1" +%s)      TS2=$(date -d "$2" +%s)      TZ=UTC date -d @$((TS1-TS2)) +%g%m%d%H%M%S      #+%Y.%m.%d.%H:%M:%S  }    date_converted=`convert_date $date`  echo $date_converted  line_converted=`convert_date $line`  echo $line_converted  diff_dates $date_converted $line_converted    
root@SomeHost:~# date="211013150325"  root@SomeHost:~# line="150325113419"  root@SomeHost:~# convert_date(){ printf `date '+%C'`'%s-%s-%s %s:%s:%s' ${1:0:2} ${1:2:2} ${1:4:2} ${1:6:2} ${1:8:2} ${1:10:2}; }  root@SomeHost:~# diff_dates(){  >     TS1=$(date -d "$1" +%s)  >     TS2=$(date -d "$2" +%s)  >     TZ=UTC date -d @$((TS1-TS2)) +%g%m%d%H%M%S  >     #+%Y.%m.%d.%H:%M:%S  > }  root@SomeHost:~#   root@SomeHost:~# date_converted=`convert_date $date`  root@SomeHost:~# echo $date_converted  2021-10-13 15:03:25  root@SomeHost:~# line_converted=`convert_date $line`  root@SomeHost:~# echo $line_converted  2015-03-25 11:34:19  root@SomeHost:~# diff_dates $date_converted $line_converted  701231085635  

Could I remove TZ=UTC date -d @$((TS1-TS2)) +%g%m%d%H%M%S and get better conversion with something else? With echo $(($TS2-$TS1)) I could get time in seconds, but I don't know how to convert that to a date that doesn't start with 1970...


Maybe (almost certainly) I'm doing this completely wrong and there is much less complicated way of doing this.
I pieced this question together over 2 hours so if there is something that doesn't make sense please point it out and I will try to fix it.

How can I see if a consecutive number name file is missing some file?

Posted: 13 Oct 2021 10:03 AM PDT

If I give ls -1 I get like this,

file_0001.jpeg  file_0002.jpeg  file_0003.jpeg  file_0004.jpeg  file_0005.jpeg  file_0006.jpeg  file_0007.jpeg  file_0008.jpeg  file_0009.jpeg  file_0010.jpeg  file_0011.jpeg  file_0012.jpeg  file_0013.jpeg  file_0014.jpeg  file_0015.jpeg  file_0016.jpeg  file_0017.jpeg  file_0018.jpeg  file_0019.jpeg  file_0020.jpeg  file_0021.jpeg  ...  file_0999.jpeg  

Is there a way using awk or other tool to see if some file is missing in this consecutive incremental way.

Ansible - looping over all items in a list inside a dictionary

Posted: 13 Oct 2021 09:28 AM PDT

I'm using Ansible to find files on Windows machines using the win_find module.

I'd like to do this in 2 steps:

  • Find any directory in a given location (ex: find all directories under C:/
  • Search each of those directories for a specific file (ex: blah.cfg)
  • If a directory has said file in it, mirror its structure on a remote location (using win_fetch).

I managed to find directories with the example file, as well as fetching and uploading them, but for some reason I cannot find the directories first and then loop over them to search for the file.

Here's what I'm trying to do:

   - ansible.windows.win_find:        file_type: directory        paths: 'C:\'        recurse: no        get_checksum: no      register: win_dirs      tags: find    - debug:        msg: "Found directory {{ item.path }}"      with_items: "{{ win_dirs.files }}"      tags: find    - ansible.windows.win_find:        patterns: [ 'blah.cfg' ]        file_type: file        paths: '{{ item.path }}'      with_items: "{{ win_dirs.files }}"      tags: find      register: win_apps    - debug:        msg: "found blah directory: {{ item.path }}"      with_items: "{{ win_apps.files }}"      tags: find  

When I try using files in the win_apps variable, I get the following error:

MSG:    'dict object' has no attribute 'files'    

To clarify, if I just search the entire root directory using recurse, this does work:

  - ansible.windows.win_find:        patterns: [ 'blah.cfg' ]        file_type: file        paths: 'C:\'        recurse: yes      tags: find      register: win_apps    - debug:        msg: "found blah directory: {{ item.path }}"      with_items: "{{ win_apps.files }}"      tags: find  

With this method, however, I cannot mirror the directories on the remote location as they are on the host, since win_find traverses the subdirectories.

I think something is wrong with this bit:

  - ansible.windows.win_find:        patterns: [ 'blah.cfg' ]        file_type: file        paths: '{{ item.path }}'      with_items: "{{ win_dirs.files }}"  

I've done my best to read the documentation and try other options and cannot figure this out.

Any help would be greatly appreciated.

Thanks!

UPDATE:

The results of the second loop (win_apps) are inside a results dictionary, as outlined in https://docs.ansible.com/ansible/latest/user_guide/playbooks_loops.html#registering-variables-with-a-loop.

So the file paths are in {{ win_apps.results.files.path }} - but I could not access them since win_apps.results.files is a list. This has been working:

 - debug:        msg: "Found file {{ item.files[1].path }}"      loop: "{{ win_files.results }}"  

While this has not:

 - debug:        msg: "Found file {{ item.files.path }}"      loop: "{{ win_files.results }}"  

returning The task includes an option with an undefined variable. The error was: 'list object' has no attribute 'path'

How would I be able to loop over the files and extract their paths in this scenario?

Thanks!

cron: send error messages to file, when no MTA is installed

Posted: 13 Oct 2021 11:00 AM PDT

I get this message when there is an error in my crontab:

cron: No MTA installed, discarding output  

I don't want to install a MTA on my system, but I also don't want to miss these error messages.

Where is it configured, that cron tries to send these by mail? Can I change that, so that these messages are send to a file instead? (perhaps via sysylog).

I don't want to log all cron messages, just the errors.

I have this in my rsyslog.conf:

cron.=info                    stop    *.*                          |/dev/xconsole  

Unfortunately, it seems that even error messages have the .info tag

How can I only log cron errors only? Or, in other words: how can I send to log file, what would otherwise be sent to MTA if it was installed?

My system is Debian 10, and i am using rsyslog for logging (no systemd)

UPDATE:

using redirection for each line individually, as suggested by @basin is the solution I was using up until now, and it has few problems:

First of all, as I stated, I would like a solution that redirects what would normally be sent to MTA by default to some other location, ie |/dev/xconsole, without having to specify for each lie individually.

Second, if there is a syntax error in my crontab line, the redirection does not work. Cron still tries to send the error via MTA, and I get the No MTA installed error in the log.

Is there some way to redirect what would have ben sent via MTA, so that it is sent (either directly or via sysylog) to /dev/xconsole ?

How to block specific desktop notifications? [closed]

Posted: 13 Oct 2021 11:00 AM PDT

I subscribe to the Boston Globe. I get unwanted news alerts from them as desktop notifications. I never asked for them. I don't want them. I contacted the Boston Globe and they won't help me. I logged on to bostonglobe.com and looked for a relevant setting - no luck. I get many desktop notifications. I only want to stop the ones from the Globe. I know how to programmatically throw up a notification, but I have no idea how to turn off or block specific notifications. And I would like to continue my Globe subscription. I'm running Pop!_OS 20.04 LTS and I login to i3 window manager. Thanks in advance.

Installing CUDA on Debian Machine 10.3

Posted: 13 Oct 2021 09:47 AM PDT

So, I am trying to install Cuda on Debian 10.3 on SageMaker studio instance ml.g4dn.xlarge.

I am trying the instructions from here:

https://developer.nvidia.com/cuda-downloads?target_os=Linux&target_arch=x86_64&target_distro=Debian&target_version=10&target_type=deblocal

So, I tried on Debian Machine:

  dpkg -i cuda-repo-debian10-11-1-local_11.1.0-455.23.05-1_amd64.deb    apt-get install gnupg    apt-key add /var/cuda-repo-debian10-11-1-local/7fa2af80.pub    apt-get install software-properties-common    add-apt-repository contrib    apt-get update    apt-get -y install cuda  

But, in the end, I get:

 /tmp/apt-dpkg-install-y7M08M/097-nvidia-persistenced_455.23.05-1_amd64.deb    E: Sub-process /usr/bin/dpkg returned an error code (1)  

Does any one have any clue, what is going on here?

enter image description here

More logs on error:

    dpkg: error processing archive /tmp/apt-dpkg-install-nkwkn4/097-nvidia-persistenced_455.23.05-1_amd64.deb (--unpack):       unable to make backup link of './usr/bin/nvidia-persistenced' before installing new version: Invalid cross-device link      Selecting previously unselected package libxnvctrl0:amd64.      Preparing to unpack .../098-libxnvctrl0_455.23.05-1_amd64.deb ...      Unpacking libxnvctrl0:amd64 (455.23.05-1) ...      Selecting previously unselected package libgdk-pixbuf2.0-common.      Preparing to unpack .../099-libgdk-pixbuf2.0-common_2.38.1+dfsg-1_all.deb ...      Unpacking libgdk-pixbuf2.0-common (2.38.1+dfsg-1) ...      Selecting previously unselected package libgdk-pixbuf2.0-0:amd64.      Preparing to unpack .../100-libgdk-pixbuf2.0-0_2.38.1+dfsg-1_amd64.deb ...      Unpacking libgdk-pixbuf2.0-0:amd64 (2.38.1+dfsg-1) ...      Selecting previously unselected package gtk-update-icon-cache.      Preparing to unpack .../101-gtk-update-icon-cache_3.24.5-1_amd64.deb ...      No diversion 'diversion of /usr/sbin/update-icon-caches to /usr/sbin/update-icon-caches.gtk2 by libgtk-3-bin', none removed.      No diversion 'diversion of /usr/share/man/man8/update-icon-caches.8.gz to /usr/share/man/man8/update-icon-caches.gtk2.8.gz by libgtk-3-bin', none removed.  

Incorrect partition size on new disk

Posted: 13 Oct 2021 09:49 AM PDT

I know this type of question has been asked frequently, but I cannot seem to figure out what is happening. tl;dr: I cloned an existing disk onto a larger disk, but df is only showing this at the size of the original disk, even though the partition table looks OK.

I have a 10TB backup drive on my Debian system at /dev/sda, and added a 12TB drive to serve as an additional backup at /dev/sdc. Eventually I will remove the first backup, to offsite storage. I used parted to create a new partition, using up the entire free space, and then mkfs.ext4 to create a filesystem on it. I then mounted this filesystem, and df -h showed me the expected result: The original disk was 9.1T, the new one was 11T.

I copied the original onto the new drive with pv < /dev/sda1 > /dev/sdc1. Since this was a clone, I then created a new UUID for this partition with uuidgen, and used this to mount the disk in /etc/fstab.

The new drive has the files I expect. However, df now shows the two drives as being identical:

# df -h  Filesystem      Size  Used Avail Use% Mounted on  [...]  /dev/sda1       9.1T  6.5T  2.6T  72% /mnt/Backup1  /dev/sdc1       9.1T  6.5T  2.6T  72% /mnt/Backup2  

This is the case when the disk is first mounted; it's not like any existing operation is holding a file open.

The output of fdisk shows that the partition is the expected size:

# fdisk -l /dev/sdc  Disk /dev/sdc: 10.9 TiB, 12000105070592 bytes, 23437705216 sectors  Disk model: Elements 25A3     Units: sectors of 1 * 512 = 512 bytes  Sector size (logical/physical): 512 bytes / 4096 bytes  I/O size (minimum/optimal): 4096 bytes / 4096 bytes  Disklabel type: gpt  Disk identifier: 96102C84-3B01-4361-A9C2-B44455AEC02E    Device     Start         End     Sectors  Size Type  /dev/sdc1   2048 23437703167 23437701120 10.9T Linux filesystem  

as does lsblk:

# lsblk  NAME        MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT  sda           8:0    0  9.1T  0 disk   └─sda1        8:1    0  9.1T  0 part /mnt/Backup1  sdc           8:32   0 10.9T  0 disk   └─sdc1        8:33   0 10.9T  0 part   

Running parted also seems to confirm that the partition is the correct size:

# parted /dev/sdc  GNU Parted 3.2  Using /dev/sdc  Welcome to GNU Parted! Type 'help' to view a list of commands.  (parted) print                                                              Model: WD Elements 25A3 (scsi)  Disk /dev/sdc: 12.0TB  Sector size (logical/physical): 512B/4096B  Partition Table: gpt  Disk Flags:     Number  Start   End     Size    File system  Name     Flags   1      1049kB  12.0TB  12.0TB  ext4         primary    

I then tried to let fsck have a go at it, and got this:

# fsck.ext4 /dev/sdc  e2fsck 1.44.5 (15-Dec-2018)  ext2fs_open2: Bad magic number in super-block  fsck.ext4: Superblock invalid, trying backup blocks...  fsck.ext4: Bad magic number in super-block while trying to open /dev/sdc    The superblock could not be read or does not describe a valid ext2/ext3/ext4  filesystem.  If the device is valid and it really contains an ext2/ext3/ext4  filesystem (and not swap or ufs or something else), then the superblock  is corrupt, and you might try running e2fsck with an alternate superblock:      e2fsck -b 8193 <device>   or      e2fsck -b 32768 <device>    Found a gpt partition table in /dev/sdc  

I tried the e2fsck options, but got the same result.

I have searched for discussions of this fsck issue, without finding anything useful, and I have looked at some of the many discussions of discrepancies between df output and other indications of disk size, also without much luck: usually the reason in this circumstance is that the new disk had an exact copy of the original partition. But my partition does seem to be the correct size.

I'd be grateful for any suggestions here. My files do seem to be on the new disk, so wiping it out and starting over again will take up many, many hours of recopying....

Edit: per request, output of gdisk:

# gdisk -l /dev/sdc  GPT fdisk (gdisk) version 1.0.3    Partition table scan:    MBR: protective    BSD: not present    APM: not present    GPT: present    Found valid GPT with protective MBR; using GPT.  Disk /dev/sdc: 23437705216 sectors, 10.9 TiB  Model: Elements 25A3     Sector size (logical/physical): 512/4096 bytes  Disk identifier (GUID): 96102C84-3B01-4361-A9C2-B44455AEC02E  Partition table holds up to 128 entries  Main partition table begins at sector 2 and ends at sector 33  First usable sector is 34, last usable sector is 23437705182  Partitions will be aligned on 2048-sector boundaries  Total free space is 4029 sectors (2.0 MiB)    Number  Start (sector)    End (sector)  Size       Code  Name     1            2048     23437703167   10.9 TiB    8300  primary  

Writeback cache (`dirty`) seems to be limited below the expected threshold where throttling starts. What is it being limited by?

Posted: 13 Oct 2021 09:15 AM PDT

In a previous question and answer, I showed an experiment about dirty_ratio.

Writeback cache (`dirty`) seems to be limited to even less than dirty_background_ratio. What is it being limited by? How is this limit calculated?

I thought I solved the question, by correcting my understanding of the dirty ratio calculation. But I repeated the experiment just now, and the write-back cache was limited to lower than I saw before. I can't work this out, what could be limiting it?

I have default values for the vm.dirty* sysctl's. dirty_background_ratio is 10, and dirty_ratio is 20. The "ratios" refer to the size of the dirty page cache aka write-back cache, as a percentage of MemFree + Cached. They are not a percentage of MemTotal - this was what confused me in the above question.

These ratios mean that reaching 10% causes background writeback to start, and 20% is the maximum size of the write-back cache. Additionally, I understand the write-back cache is limited by "I/O-less dirty throttling". When the write-back cache rises above 15%, processes which generate dirty pages e.g. with write() are "throttled". That is, the kernel causes the process to sleep inside the write() call. So the kernel can control the size of the write-back cache, by controlling the length of the sleeps. For references, see the answer to my previous question.

But my observed "ratio" seems to stay distinctly lower than the 15% throttling threshold. There must be some factor I am missing! Why is this happening?

In my previous test I saw values around 15-17.5% instead.

My kernel is Linux 4.18.16-200.fc28.x86_64.

The test is as follows: I ran dd if=/dev/zero of=~/test bs=1M status=progress. And at the same time, I monitored the achieved dirty ratio. I interrupted the dd command after 15GB.

$ while true; do grep -E '^(Dirty:|Writeback:|MemFree:|Cached:)' /proc/meminfo | tr '\n' ' '; echo; sleep 1; done  ...  MemFree:          139852 kB Cached:          3443460 kB Dirty:            300240 kB Writeback:        135280 kB  MemFree:          145932 kB Cached:          3437220 kB Dirty:            319588 kB Writeback:        112080 kB  MemFree:          134324 kB Cached:          3448776 kB Dirty:            237612 kB Writeback:        160528 kB  MemFree:          134012 kB Cached:          3449004 kB Dirty:            169064 kB Writeback:        143256 kB  MemFree:          133760 kB Cached:          3449024 kB Dirty:            105484 kB Writeback:        119968 kB  MemFree:          133584 kB Cached:          3449032 kB Dirty:             49068 kB Writeback:        104412 kB  MemFree:          134712 kB Cached:          3449116 kB Dirty:                80 kB Writeback:         78740 kB  MemFree:          135448 kB Cached:          3449116 kB Dirty:                 8 kB Writeback:             0 kB  

For example, the first line in the quoted output:

avail = 139852 + 3443460 = 3583312  dirty = 300240 + 135280 = 435520  ratio = 435520 / 3583312 = 0.122...  

I found one thing that was limiting it, but not enough to see these results. I had been experimenting with setting /sys/class/bdi/*max_ratio. The test results in the question are from running with max_ratio = 1.

Repeating the above test with max_ratio = 100, I can achieve a higher dirty ratio e.g. 0.142:

MemFree:   122936 kB Cached:   3012244 kB Dirty:     333224 kB Writeback: 13532 kB  

The write test needs to be quite long to observe this reliably, e.g. 8GB. This test takes about 100 seconds. I am using a spinning hard disk.

I tried testing with 4GB, and I only saw a dirty ratio of 0.129:

MemFree:   118388 kB Cached:   2982720 kB Dirty:     249020 kB Writeback: 151556 kB  

As I say, this surprises me. I have an expert source from 2013, saying that dd should have "free run" to generate dirty pages until the system hits a dirty ratio of 0.15. It is explicitly talking about max_ratio.

CentOS 7 - running FFMPEG as a service

Posted: 13 Oct 2021 11:03 AM PDT

I'm writing some tests for a video streaming project. In order to check that I stream video correctly, I've set up a VM that runs a video client (e.g ffmpeg / vlc) as a daemon. I want that, on command, the client will connect to the server, display the video and save the video to a file.

In order to do that, I've set a service. Here is my .service file:

[Unit]  Description=Video client for testing    [Service]  Type=simple  WorkingDirectory=/home/user/  ExecStart=client.sh  Restart=always    [Install]  WantedBy=default.target  

Ideally, I want this to be a user service. Sadly, my VM runs CentOS 7 which can not run user services. As a regular service, the $DISPLAY and $XAUTHORITY needs to be set as well, but I couldn't find any script that initializes them like in arch so I don't think it will be healthy to set them up myself. I'm also not sure how this is going to work if login is not automatic. The script itself does not work well even with just saving the video and not opening a graphical window. It does work well when launching it

I've been thinking of running the script from a .profile, but I don't like this idea too much, it seems awkward (or sure if that is even a good option). Another option would be to set up the script via SSH (for example, using paramiko - which requires additional dependencies / scripts.

I would really appreciate some suggestions / alternatives / ways of action, since I don't want it to hack this too much.

Thanks, Joey.

How to get only the name of the physical ethernet interface?

Posted: 13 Oct 2021 09:09 AM PDT

Is there a way to get only the name of physical ethernet interface(i.e not virtual ethernet interface)?
To give a bit of background, I'm trying to get a few SBCs(RPi 3) to write their IP addresses to a DataBase. But since the names of the physical ethernet interface on different SBCs is not usually same, I'm finding it hard to get their IP addresses.
One way I could think of solving this is to give all the SBCs ethernet interface a common name like eth0. But this method feels a bit clunky. So, is there any other alternative to get only the name of physical ethernet interface?

Sound via HDMI on Arch Linux

Posted: 13 Oct 2021 10:40 AM PDT

I've got my Toshiba laptop (Satellite A300) connected to my TV via HDMI. Using VLC 2.2.6, video works just fine.

Currently, I'm trying to output sound to the TV's speakers.

aplay -l shows the HDMI playback device as the third one:

**** List of PLAYBACK Hardware Devices ****  card 0: Intel [HDA Intel], device 0: ALC268 Analog [ALC268 Analog]    Subdevices: 1/1    Subdevice #0: subdevice #0  card 0: Intel [HDA Intel], device 1: ALC268 Digital [ALC268 Digital]    Subdevices: 1/1    Subdevice #0: subdevice #0  card 0: Intel [HDA Intel], device 3: HDMI 0 [HDMI 0]    Subdevices: 1/1    Subdevice #0: subdevice #0  

I was able to play a test sound on the TV using speaker-test -D plughw:0,3 -c 2 after I've unmuted S/PDIF in alsamixer.

Yet, when playing a file with VLC, the only option in Audio → Audio Device is "Built-in Audio Analog Stereo". At the moment, sound is played using the laptop's speakers.

How can I have VLC output the sound to the TV's speakers?

Blocking telegram using /etc/hosts

Posted: 13 Oct 2021 10:00 AM PDT

I'm using /etc/hosts to block domains that I don't want to be accessible, and it works for most domains. However it doesn't seem to be working for the Telegram webchat (web.telegram.org).

I have the following lines in my file:

0.0.0.0 web.telegram.org  0.0.0.0 www.telegram.org  0.0.0.0 telegram.org  

But I am still able to go on the site and use the full functionality of the web app. Am I doing something wrong?

Is there a shell command or utility for caching process output?

Posted: 13 Oct 2021 10:42 AM PDT

Hope this is the right place to ask.

Is there a unix tool that works similarly to this?

# invokes echo, saves to cache (using command with arguments as key), returns "hi"  cache 10 echo hi     # wait 2 seconds  sleep 2    # doesn't invoke echo, from cache returns "hi"  cache 10 echo hi     # wait 10 seconds  sleep 10    # with cache expired, invokes echo, returns "hi"  cache 10 echo hi   

Obviously echo wouldn't be the real use-case.

Basically cache STDOUT, STDERR and status for a given command+arguments, so the next call to the same process doesn't have to re-run it.

I could write a script that does it but I wondered if there was one in the unix toolset that I don't know about.

Ffmpeg problems on centOS 5.8 64 bit

Posted: 13 Oct 2021 10:07 AM PDT

I did install ffmpeg with winff, unsuccessfully when I try converting avi to x264, it says I don't have it. When I try to convert mp4 to Xvid I get unknown encoder 'libxvid'. It's like nothing really works except avi Msi compatible using winff.

Can anyone help?

How can I tell what version of Linux I'm using?

Posted: 13 Oct 2021 09:28 AM PDT

Often times I will ssh into a new client's box to make changes to their website configuration without knowing much about the server configuration. I have seen a few ways to get information about the system you're using, but are there some standard commands to tell me what version of Unix/Linux I'm on and basic system information (like if it is a 64-bit system or not), and that sort of thing?

Basically, if you just logged into a box and didn't know anything about it, what things would you check out and what commands would you use to do it?

Sunrise and other astronomical data from the command line

Posted: 13 Oct 2021 10:28 AM PDT

Is there a simple open-source, command-line program that can show the sunrise and sunset times at a given date and location, and perhaps moon and planet data as well?

Browsing the Debian package database and Google searches, I can't find anything relevant. This surprises me — considering the number of people who are both astronomy geeks and unix geeks, I'd have expected a de-facto-standard sunrise(1) (or perhaps sunrise(6)).

I am not interested in more complex programs that incidentally perform the calculations, such as sky maps (celestia, kstars, starplot, stellarium), earth maps (sunclock, xplanet), calendars/agendas (emacs, remind), clocks (glunarclock, wmsun), tide almanachs (xtide). There is perhaps aa (astronomical-almanac), but I don't see a simple way of asking it what time the sun will rise on this date at these longitude and latitude.

What did I miss?

No comments:

Post a Comment