Sunday, May 30, 2021

Recent Questions - Unix & Linux Stack Exchange

Recent Questions - Unix & Linux Stack Exchange


How to set the default username and password for LxQt Desktop in Debian

Posted: 30 May 2021 09:53 AM PDT

I am having a Debian system which is having:

pstree     sddm--->Xorg         |         | -->LxQt  

where sddm is service running as a combination of Xorg(server) and LxQt(Desktop application).

I want to log in using the default username and password from Lxqt Desktop. May I please know where I can set this default username and password?

how do I resume an scp download?

Posted: 30 May 2021 10:08 AM PDT

I started downloading about 1gb from a colleague, and received 750mb in about 8 hours.

The command I used was

$ scp dev@dev_server:/home/dev/trainingVM/Centos7VMa_app_zxTmw457.ova  

What command could I use next, so that the remainder would download in under 3 hours, not 10?

VNC Fedora 34 - headphone jack and usb camera not working

Posted: 30 May 2021 09:45 AM PDT

Always had vnc on Fedora boxes so I have one keyboard / mouse from my main Windoz workstation with lots of monitors. Everything is local gig LAN.

On the physical Fedora box I have a usb logitech camera and cheap speakers connected to the 3.5mm headphone jack. Camera / speakers worked great with zoom, skype, and I could have videos playing on the Fedora box and still do work stuff on the Windoz machine.

Did a fresh install of Fedora 34 and now when I VNC to the Fedora box, the sound output device is blank. If I am on the physical Fedora machine sound is normal. Researched a lot and pipewire had loads of issues so I switched back to pulseaudio. Now sound settings shows Dummy Output and still no sound. Again, on the physical Fedora machine - sound is great.

I only have onboard sound and has worked for the past several years through countless Fedora versions until 34. Any idea how to get the headphone jack and camera on the physical Fedora 34 box to work when I am connected via VNC to it. Also tried xRPD and got the same results.

Ubuntu 16.04 dropping IPv4-mapped IPv6 addresses

Posted: 30 May 2021 09:22 AM PDT

I have a Debian 11(kernel version 5.10.0-6) and Ubuntu 16.04(kernel version 4.4.0-201) in the same broadcast domain. Both systems have an iputils traceroute6 version s20121221 installed and firewall allows all the IPv6 traffic. Now the problem is, that Ubuntu installation does not pass the IPv4-mapped IPv6 addresses to traceroute6 utility. I can see that the packets reach the system, but traceroute6 does not see those. When I execute the strace -f ./traceroute6 -d -q1 -n fc00::10:2:34:30 command in the Debian 11 system, then the output related to fourth hop(::ffff:172.16.0.2) can be seen below:

setsockopt(4, SOL_IPV6, IPV6_UNICAST_HOPS, [4], 4) = 0  sendto(4, "\0\0'\335\0\0\0\4Y\266\263`\0\0\0\0K\267\6\0\0\0\0\0", 24, 0, {sa_family=AF_INET6, sin6_port=htons(33434), sin6_flowinfo=htonl(0), inet_pton(AF_INET6, "fc00::10:2:34:30", &sin6_addr), sin6_scope_id=0}, 28) = 24  select(4, [3], NULL, NULL, {tv_sec=5, tv_usec=0}) = 1 (in [3], left {tv_sec=4, tv_usec=988513})                                                                                                                                                                                                  recvmsg(3, {msg_name={sa_family=AF_INET6, sin6_port=htons(0), sin6_flowinfo=htonl(0), inet_pton(AF_INET6, "::ffff:172.16.0.2", &sin6_addr), sin6_scope_id=0}, msg_namelen=28, msg_iov=[{iov_base="\3\0R4\t\0\0\0`\v\207\265\0 \21\2\374\0\0\0\0\0\0\0\0\20\0\1\0\22\0\21"..., iov_len=512}], ms  g_iovlen=1, msg_control=[{cmsg_len=36, cmsg_level=SOL_IPV6, cmsg_type=0x32}, {cmsg_len=36, cmsg_level=SOL_IPV6, cmsg_type=0x2}], msg_controllen=80, msg_flags=0}, 0) = 152                                                                                                                     write(1, " 4  ::ffff:172.16.0.2  12.777 ms", 32 4  ::ffff:172.16.0.2  12.777 ms) = 32                                                                                                                                                                                                            write(1, "\n", 1                                                                                                                                 )                       = 1    

As seen above, after sending the packet to fc00::10:2:34:30 in the sendto(), the recvmsg() receives the packet from ::ffff:172.16.0.2. However, in case of Ubuntu 16.04 system, the recvmsg() never receives the packet:

setsockopt(4, SOL_IPV6, IPV6_UNICAST_HOPS, [4], 4) = 0  sendto(4, "\0\0Q\222\0\0\0\4\346\267\263`\0\0\0\0\365\241\1\0\0\0\0\0", 24, 0, {sa_family=AF_INET6, sin6_port=htons(33434), inet_pton(AF_INET6, "fc00::10:2:34:30", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, 28) = 24  select(4, [3], NULL, NULL, {5, 0})      = 0 (Timeout)  write(1, " 4  *", 5 4  *)                    = 5  write(1, "\n", 1  )                       = 1  

As seen above, the fourth hop is a * in the Ubuntu 16.04 system. Socket definition in iputils traceroute6 is following:

socket(PF_INET6, SOCK_DGRAM, IPPROTO_IP)

The problem is specifically with IPv4-mapped IPv6 addresses. Other types of IPv6 packets are processed fine in the Ubuntu system.

What might cause such behavior?

Unable to get Xorg configuration on Slackware

Posted: 30 May 2021 09:37 AM PDT

I'm setting up a Slackware docker image and I'm having issues to get Xorg going:

# Xorg -configure  _XSERVTransSocketUNIXCreateListener: ...SocketCreateListener() failed  _XSERVTransMakeAllCOTSServerListeners: server already running  (EE)   Fatal server error:  (EE) Cannot establish any listening sockets - Make sure an X server isn't already running(EE)   (EE)   Please consult the The X.Org Foundation support        at http://wiki.x.org   for help.   (EE) Please also check the log file at "/var/log/Xorg.0.log" for additional information.  (EE)   (EE) Server terminated with error (1). Closing log file.  

Where the log file doesn't reveal any additional details and my process list looks like:

# ps ax  PID TTY      STAT   TIME COMMAND    1 pts/0    Ss     0:00 /bin/bash  

4534 pts/0 R+ 0:00 ps ax

i.e. I sure do not have an Xserver running already. Some hardware details:

# lspci  00:00.0 Host bridge: Intel Corporation Device 5914 (rev 08)  00:02.0 VGA compatible controller: Intel Corporation Device 5917 (rev 07)  00:04.0 Signal processing controller: Intel Corporation Device 1903 (rev 08)  00:14.0 USB controller: Intel Corporation Device 9d2f (rev 21)  00:14.2 Signal processing controller: Intel Corporation Device 9d31 (rev 21)  00:15.0 Signal processing controller: Intel Corporation Device 9d60 (rev 21)  00:15.1 Signal processing controller: Intel Corporation Device 9d61 (rev 21)  00:15.2 Signal processing controller: Intel Corporation Device 9d62 (rev 21)  00:16.0 Communication controller: Intel Corporation Device 9d3a (rev 21)  00:17.0 SATA controller: Intel Corporation Device 9d03 (rev 21)  00:1c.0 PCI bridge: Intel Corporation Device 9d10 (rev f1)  00:1c.5 PCI bridge: Intel Corporation Device 9d15 (rev f1)  00:1f.0 ISA bridge: Intel Corporation Device 9d4e (rev 21)  00:1f.2 Memory controller: Intel Corporation Device 9d21 (rev 21)  00:1f.3 Audio device: Intel Corporation Device 9d71 (rev 21)  00:1f.4 SMBus: Intel Corporation Device 9d23 (rev 21)  02:00.0 Network controller: Intel Corporation Device 24fd (rev 78)  

and I've installed the following packages:

  • mesa
  • xf86-video-intel
  • intel-gpu-tools

I'm wondering what my problem might be that I can't even get Xorg to create an xorg.conf

Plank dock in improper position

Posted: 30 May 2021 07:29 AM PDT

I am upgrading my machine using

sudo apt-get upgrade  

after restarting, I got an improper position of plank dock which is shown below

Wallpaper with imroper dock

I've tried to customize it from plank preferences. but, I can't find any possibilities. I also have tried to search it. Yet, I don't find an article which fit this problem.

Anyone know how to fix issue?

awk to return percentage by day of the week

Posted: 30 May 2021 08:31 AM PDT

So I have a dataset of flights in a CSV file and I want to get a percentage of flights delayed (column 6) ordered by day. 0 is for not delayed and 1 is for delayed 15 or more minutes.

Input:

DAY_OF_MONTH,"DAY_OF_WEEK","ORIGIN","DEST","DEP_TIME","DEP_DEL15","CANCELLED","DIVERTED","DISTANCE"  1,Tuesday,ORD,GRB,1003,0.00,0.00,0.00,322.248  1,Tuesday,TUL,ORD,1027,0.00,0.00,0.00,1083.42  1,Tuesday,EWR,TYS,1848,0.00,0.00,0.00,1168.61  

Output:

Weekday, % delayed, delayed, total flights  Tuesday,0.00,0,3  

The dataset actually has a lot of rows and has every day of a month and all its flights, so that wouldn't be the actual output.

Someone helped me come up with:

#!/bin/awk  BEGIN { FS = OFS = "," }  FNR > 1 { total[$2]++; if ($6) delay[$2]++ }  END {      print "\"weekday\"", "\"percentage_delayed\"", "\"delayed\"", "\"total_flights\""      for ( day in total ) { print day, delay[day] / total[day] * 100, delay[day], total[day]}    }  

But when I save this to an awk file and try to execute it with: Terminal Output

why tell me illegal pid when kill process

Posted: 30 May 2021 07:27 AM PDT

Now I am using this script to restart my process:

PID=`ps -ef|grep -w ${APP_NAME}|grep -v grep|cut -c 9-15`  if [[ ${PID} -gt 1 ]]; then    kill -9 ${PID}  else    echo "Process not found"  fi  

but when I run this script in remote server from GitHub Actions,shows this error:

======CMD======  cd /opt/apps/dolphin-acientbay/libs  . /opt/apps/dolphin-acientbay/libs/upgrade.sh    ======END======  err: +/opt/apps/dolphin-acientbay/libs/upgrade.sh:13> JAVA_HOME=/***/.sdkman/candidates/java/11.0.11.hs-adpt   err: +/opt/apps/dolphin-acientbay/libs/upgrade.sh:14> APP_HOME=/opt/apps/dolphin-acientbay/libs   err: +/opt/apps/dolphin-acientbay/libs/upgrade.sh:15> APP_NAME=dolphin-acientbay-service-1.0.0-SNAPSHOT.jar   err: +/opt/apps/dolphin-acientbay/libs/upgrade.sh:17> PID=+/opt/apps/dolphin-acientbay/libs/upgrade.sh:17> ps -ef  err: +/opt/apps/dolphin-acientbay/libs/upgrade.sh:17> PID=+/opt/apps/dolphin-acientbay/libs/upgrade.sh:17> grep -w dolphin-acientbay-service-1.0.0-SNAPSHOT.jar  err: +/opt/apps/dolphin-acientbay/libs/upgrade.sh:17> PID=+/opt/apps/dolphin-acientbay/libs/upgrade.sh:17> grep -v grep  err: +/opt/apps/dolphin-acientbay/libs/upgrade.sh:17> PID=+/opt/apps/dolphin-acientbay/libs/upgrade.sh:17> cut -c 9-15  err: +/opt/apps/dolphin-acientbay/libs/upgrade.sh:17> PID=' 19882 '   2021/05/30 11:46:21 Process exited with status 1  err: +/opt/apps/dolphin-acientbay/libs/upgrade.sh:18> [[ ' 19882 ' -gt 1 ]]  err: +/opt/apps/dolphin-acientbay/libs/upgrade.sh:19> kill -9 ' 19882 '  err: /opt/apps/dolphin-acientbay/libs/upgrade.sh:kill:19: illegal pid:  19882   

this is the full script of upgrade.sh:

#!/usr/bin/env bash    set -u    set -e    set -x    JAVA_HOME="/root/.sdkman/candidates/java/11.0.11.hs-adpt"  APP_HOME="/opt/apps/dolphin-acientbay/libs"  APP_NAME="dolphin-acientbay-service-1.0.0-SNAPSHOT.jar"    PID=`ps -ef|grep -w ${APP_NAME}|grep -v grep|cut -c 9-15`  if [[ ${PID} -gt 1 ]]; then    kill -9 ${PID}  else    echo "Process not found"  fi    sleep 5    count=`ps -ef | grep ${APP_NAME} | grep -v "grep" | wc -l`  if [[ ${count} -lt 1 ]]; then    nohup ${JAVA_HOME}/bin/java -Xmx128M -Xms128M -jar \    -Xdebug -Xrunjdwp:transport=dt_socket,suspend=n,server=y,address=0.0.0.0:5021 \    -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/opt/apps/dolphin-acientbay/ \    ${APP_HOME}/${APP_NAME} >> ./acientbay.log &    sleep 5  else    echo "process aready exists!"    exit 1  fi  

I have tried to tweak the kill process command like this(the xargs trim the white space of pid) to remove the spaces of pid:

 echo "${PID}" | xargs | kill -9  

it tell me :

err: /opt/apps/dolphin-acientbay/libs/upgrade.sh:kill:19: not enough arguments  

String Replacement in Bash-Script "Bad substitution"

Posted: 30 May 2021 08:42 AM PDT

I try a basic string replacement in a bash script:

#!/bin/bash  x="I love Linux"  echo "${x/Linux/Unix}"  

It works fine on my mac, but does not work on my server. I tried different examples from various sites, but I always get the error: Bad substitution

My bash version: GNU bash, Version 4.4.12(1)-release (x86_64-pc-linux-gnu)

What am I doing wrong?

Synapse/matrix server container config permission/file denied

Posted: 30 May 2021 08:38 AM PDT

Answer posted by me below, can't accept my own answer yet

I am working on a chat client, for development I use a docker setup that uses a 'generate' command to create a config file that you can adjust and use to launch the actual service. On my desktop it works fine however when trying to run it on a respberry or a seperate hd on my laptop it throws a file handler error in the build process stating that permission has been denied.

I might be a total noob here but I did change folder mount point permission on my laptop and the raspberry has permissions set on the folder.

Docker also is set up to run w/o typing sudo for every command.

Here are the permission codes for working vs non working config file

# Auto generated, working local  -rwxrwxrwx 1 root root 103443 <date> <filename>  
# Auto generated not working pi and laptop  -rw-r--r-- 1 991 991 106030 <date> <filename>  

Am I just not correctly setting up my hard drives ? I get this error on my pi if it's on the default OS partition or not. I am able to start up and serve django with reverse proxy no issue.

The chat service I am building on is matrix/synapse. Here is the docker folder on the synapse server repo with the basic setup. The file giving issues is homeserver.yml that is generated from:

docker-compose run --rm synapse generate  

Bash : find all IP addresses that have been used for a non-autorised access attempt

Posted: 30 May 2021 06:46 AM PDT

I am working on my virtual machine on bash. I am asked to find all ip addresses that have been used for an attempt to access the machine without authorization, using the auth.log file. I do not seem to find any answer to my problem. Here is an example : enter image description here

all returned lines must be written in my shared file /mnt/analyse/traitement/cueillette.txt. Please help ! Thanks

How to reset left time of user systemd timer

Posted: 30 May 2021 09:34 AM PDT

How can I reset the left time of a user systemd timer?

I have a user systemd service:

[Unit]  Description=Remind to a take break    [Service]  ExecStart=/opt/scripts/break_reminder  Environment=PATH=/sbin:/bin:/usr/bin:/usr/sbin:/opt/scripts    [Install]  WantedBy=default.target  

and a user systemd timer

[Unit]  Description=Remind to a take break    [Timer]  OnStartupSec=0min  OnUnitActiveSec=30min    [Install]  WantedBy=timers.target  

mdadm not enough devices to start the array - recovery possible?

Posted: 30 May 2021 10:17 AM PDT

MD raid5 array appears to have stopped working suddenly. Symptoms are somewhat similar to this issue in that I'm getting errors talking about not enough devices to start the array, however in my case the event counts on all three drives are equal. It's a raid 5 array that should have 2 active drives and one parity, however mdadm --examine on each drive shows two of them having their role listed as spare and only one as an active drive.

I've tried mdadm --stop /dev/md1 followed by mdadm --assemble /dev/md1 (including attempts with the --force and --run flags).

SMART data doesn't indicate any issues with the drives (and current pending and reallocated sector counts are all zero), I've tried the raid.wiki.kernel.org guide linked below by frostschutz through the steps involving setting up mapped overlay devices.

I would have then assumed running the following command would create a raid array that I could then attempt to mount read-only and see if that resulted in a readable filesystem or just a garbled mess (i.e. to determine if my guess of sdf1 being the parity drive was correct or if I should try again with sde1) - but instead it gives the error show below (have also tried with the associated loop devices as per losetup --list, with the same result).

mdadm --create /dev/md2 --assume-clean --level=5 --chunk=64K --metadata=1.2 --data-offset=261888s --raid-devices=3 missing /dev/mapper/sdh1 /dev/mapper/sdf1

mdadm: super1.x cannot open /dev/mapper/sdh1: Device or resource busy  mdadm: /dev/mapper/sdh1 is not suitable for this array.  mdadm: super1.x cannot open /dev/mapper/sdf1: Device or resource busy  mdadm: /dev/mapper/sdf1 is not suitable for this array.  mdadm: create aborted  

Also, while mdadm --detail /dev/md1 previously gave the output below, it now gives:

/dev/md1:             Version : 1.2          Raid Level : raid0       Total Devices : 3         Persistence : Superblock is persistent                 State : inactive     Working Devices : 3                  Name : bob:1  (local to host bob)                UUID : 07ff9ba9:e8100e68:94c12c1a:3d7ad811              Events : 373364        Number   Major   Minor   RaidDevice           -     253       11        -        /dev/dm-11         -     253       10        -        /dev/dm-10         -     253        9        -        /dev/dm-9  

Also, I've noticed dmsetup status gives the same information for all three overlays, and has a number that looks suspiciously like it may refer to the size of original raid array (16TB) rather than an individual drive (8TB) - not sure if this is as it should be?

sde1: 0 15627528888 snapshot 16/16777216000 16  sdh1: 0 15627528888 snapshot 16/16777216000 16  sdf1: 0 15627528888 snapshot 16/16777216000 16  

Not sure how to progress from this point as far as attempting to create the device, mount and inspect the filesystem to confirm whether I guessed the correct parity device or not, using the overlay to prevent anything being written to the actual drives.

UPDATE: As per frostschutz's suggestion below, the array was somehow in some kind of state where --stop needed to be issued prior to being able to do anything with the underlying drives. I'd discounted that possibility previously as cat /proc/mdstat was showing the array as inactive, which I'd assumed meant it could not possibly be what was tying the drives up, but that was not in fact the case (I'd also previously ran --stop, but it would seem something was done afterwards triggering it's return to a non-stopped state). After getting the drive order correct (not on the first try, glad I was using overlays) the array passed a fsck check with no errors reported and is now up and running as if nothing ever happened.


The result of running other diagnostic commands:

cat /proc/mdstat:

Personalities : [raid1] [linear] [multipath] [raid0] [raid6] [raid5] [raid4] [raid10]   md1 : inactive sdh1[1](S) sde1[3](S) sdf1[0](S)        23440900500 blocks super 1.2    

mdadm --detail /dev/md1:

/dev/md1:             Version : 1.2          Raid Level : raid0       Total Devices : 3         Persistence : Superblock is persistent                 State : inactive     Working Devices : 3                  Name : bob:1  (local to host bob)                UUID : 07ff9ba9:e8100e68:94c12c1a:3d7ad811              Events : 373364        Number   Major   Minor   RaidDevice           -       8      113        -        /dev/sdh1         -       8       81        -        /dev/sdf1         -       8       65        -        /dev/sde1    

lines appearing in dmesg when trying to mdadm --assemble /dev/md1:

md/raid:md1: device sdh1 operational as raid disk 1  md/raid:md1: not enough operational devices (2/3 failed)  md/raid:md1: failed to run raid set.  md: pers->run() failed ..  

and the mdadm --examines

/dev/sde1:            Magic : a92b4efc          Version : 1.2      Feature Map : 0x1       Array UUID : 07ff9ba9:e8100e68:94c12c1a:3d7ad811             Name : bob:1  (local to host bob)    Creation Time : Mon Mar  4 22:10:29 2019       Raid Level : raid5     Raid Devices : 3     Avail Dev Size : 15627267000 (7451.66 GiB 8001.16 GB)       Array Size : 15627266688 (14903.32 GiB 16002.32 GB)    Used Dev Size : 15627266688 (7451.66 GiB 8001.16 GB)      Data Offset : 261888 sectors     Super Offset : 8 sectors     Unused Space : before=261808 sectors, after=312 sectors            State : clean      Device UUID : e856f539:6a1b5822:b3b8bfb7:4d0f4741    Internal Bitmap : 8 sectors from superblock      Update Time : Sun May 30 00:22:45 2021    Bad Block Log : 512 entries available at offset 40 sectors         Checksum : 9b5703bc - correct           Events : 373364             Layout : left-symmetric       Chunk Size : 64K       Device Role : spare     Array State : .AA ('A' == active, '.' == missing, 'R' == replacing)    /dev/sdf1:            Magic : a92b4efc          Version : 1.2      Feature Map : 0x1       Array UUID : 07ff9ba9:e8100e68:94c12c1a:3d7ad811             Name : bob:1  (local to host bob)    Creation Time : Mon Mar  4 22:10:29 2019       Raid Level : raid5     Raid Devices : 3     Avail Dev Size : 15627267000 (7451.66 GiB 8001.16 GB)       Array Size : 15627266688 (14903.32 GiB 16002.32 GB)    Used Dev Size : 15627266688 (7451.66 GiB 8001.16 GB)      Data Offset : 261888 sectors     Super Offset : 8 sectors     Unused Space : before=261800 sectors, after=312 sectors            State : clean      Device UUID : 7919e56f:2e08430e:95a4c4a6:1e64606a    Internal Bitmap : 8 sectors from superblock      Update Time : Sun May 30 00:22:45 2021    Bad Block Log : 512 entries available at offset 72 sectors         Checksum : d54ff3e1 - correct           Events : 373364             Layout : left-symmetric       Chunk Size : 64K       Device Role : spare     Array State : .AA ('A' == active, '.' == missing, 'R' == replacing)    /dev/sdh1:            Magic : a92b4efc          Version : 1.2      Feature Map : 0x1       Array UUID : 07ff9ba9:e8100e68:94c12c1a:3d7ad811             Name : bob:1  (local to host bob)    Creation Time : Mon Mar  4 22:10:29 2019       Raid Level : raid5     Raid Devices : 3     Avail Dev Size : 15627267000 (7451.66 GiB 8001.16 GB)       Array Size : 15627266688 (14903.32 GiB 16002.32 GB)    Used Dev Size : 15627266688 (7451.66 GiB 8001.16 GB)      Data Offset : 261888 sectors     Super Offset : 8 sectors     Unused Space : before=261800 sectors, after=312 sectors            State : clean      Device UUID : 0c9a8237:7e79a439:d4e35b31:659f3c86    Internal Bitmap : 8 sectors from superblock      Update Time : Sun May 30 00:22:45 2021    Bad Block Log : 512 entries available at offset 72 sectors         Checksum : 6ec2604b - correct           Events : 373364             Layout : left-symmetric       Chunk Size : 64K       Device Role : Active device 1     Array State : .AA ('A' == active, '.' == missing, 'R' == replacing)      

Adding Windows Boot Manager to GRUB2

Posted: 30 May 2021 06:52 AM PDT

I have an SSD and I am using windows 10. I recently installed Arch and I configured my SSD in a way that it has 2 EFI System partion. One for GRUB2 and another for windows boot manager.My system boots as UEFI.Eventhough I installed os prober it didn't detect windows bootloader. So I need to switch to UEFI and change boot orders everytime I need to switch. How should I configure grub2 file so it also detects windows bootloader and gives me a option to choose from either, when grub2 is loaded.

How to launch a command at the startup of only the very first shell of an X session?

Posted: 30 May 2021 06:49 AM PDT

I am running Gnome on Pop-OS! 20.04.

I am using terminator and zsh shell. I want to launch fortune only the first time I use zsh shell in a X session.

I have tried those kind of things in .zshrc:

### count the lines of  ps -ax | grep zsh  ps -ax | grep /zsh    ###  [ -z "$(pidof zsh)" ] && fortune  

PS: I had hard times finding appropriate tags for this quesion

How to get the average of multiple data in a row from multiple folders

Posted: 30 May 2021 09:40 AM PDT

My data inside a folder data looks like data1.txt, data2.txt, … data120.txt.

Inside each .txt file I have four columns (1000 data lines in each column) example:

data1.txt

1 2 3 4   4 0 1 3   3 1 1 2   2 2 2 1   ........  

data2.txt

0 1 3 4   4 2 1 3   3 1 3 2   2 3 2 1   ........  

data3.txt

1 0 3 4   4 0 0 3   0 1 1 2   2 0 2 1   ........  

data120.txt

1 2 3 1   4 1 1 3   3 1 1 1   2 1 2 1   ........  

I want to get the average .txt which looks like below but divided by 4 because I used four data samples in this example.

1+0+1+1  \  2+1+0+2  \  3+3+3+3  \  4+4+4+1  \    4+4+4+4  \  0+2+0+1  \  1+1+0+1  \  3+3+3+3  \    3+3+0+3  \  1+1+1+1  \  1+3+1+1  \  2+2+2+1  \    2+2+2+2  \  2+3+0+1  \  2+2+2+2  \  1+1+1+1  \  

I show my data this way just to make it clear - )

How I can grep data with some spaces and ignoring block?

Posted: 30 May 2021 10:01 AM PDT

I am trying to retrieve Local Address-port with port number in different block used by each IP-address in Foreign Address and PID/Program name) from the following and store it in a file:

I used:

    netstat -natp | grep '^[a-z0-9P]*'  

after that I want to ignore Recv-Q and Send-Q block and take Local Address with its port number and Foreign Address and then again ignore State and take PID/Program name.

Which regexp will be help full for me? Also, it will be more helpful if I am able keep both port numbers in different block after each address.

This is What I have:

$ netstat -natp | grep '^[a-z0-9P]*'  (Not all processes could be identified, non-owned process info   will not be shown, you would have to be root to see it all.)  Active Internet connections (servers and established)  Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name      tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      -                     tcp        0      0 127.0.0.1:5939          0.0.0.0:*               LISTEN      -                     tcp        0      0 127.0.0.53:53           0.0.0.0:*               LISTEN      -                     tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      -                     tcp        0      0 127.0.0.1:631           0.0.0.0:*               LISTEN      -                     tcp        0      0 0.0.0.0:5432            0.0.0.0:*               LISTEN      -                     tcp        0      0 0.0.0.0:7071            0.0.0.0:*               LISTEN      -                     tcp        0      0 192.168.42.157:37960    106.10.218.42:443       ESTABLISHED 21019/firefox         tcp        0      0 192.168.42.157:35636    117.18.237.29:80        ESTABLISHED 21019/firefox         tcp        1     32 192.168.42.157:40444    5.39.93.71:443          CLOSING     -                     tcp        0      0 192.168.42.157:35626    52.27.200.224:443       TIME_WAIT   -                     tcp        0      0 192.168.42.157:43004    122.252.255.200:80      ESTABLISHED 21019/firefox         tcp        0      0 192.168.42.157:35734    117.18.237.29:80        TIME_WAIT   -                     tcp        0      0 192.168.42.157:35776    52.27.200.224:443       TIME_WAIT   -                     tcp        0      0 192.168.42.157:41690    54.182.1.219:443        ESTABLISHED 21019/firefox         tcp        0      0 192.168.42.157:56472    54.182.0.97:443         ESTABLISHED 21019/firefox         tcp        1     32 192.168.42.157:48390    198.252.206.25:443      CLOSING     -                     tcp        0      0 192.168.42.157:37322    34.107.221.82:80        ESTABLISHED 21019/firefox         tcp        0      0 192.168.42.157:57724    204.79.197.204:443      ESTABLISHED 21019/firefox         tcp        0      0 192.168.42.157:43142    23.57.14.17:443         ESTABLISHED 21019/firefox         tcp        0      0 192.168.42.157:46286    13.227.138.58:443       ESTABLISHED 21019/firefox         tcp        0      0 192.168.42.157:55576    112.133.250.163:443     ESTABLISHED 21019/firefox         tcp        0      0 192.168.42.157:52328    151.101.120.193:443     ESTABLISHED 21019/firefox         tcp        0      0 192.168.42.157:35736    52.39.214.89:443        ESTABLISHED 21019/firefox         tcp        0      0 192.168.42.157:57252    99.83.135.170:443       ESTABLISHED 21019/firefox         tcp        0      0 192.168.42.157:48394    198.252.206.25:443      ESTABLISHED 21019/firefox         tcp        0      0 192.168.42.157:45020    54.182.0.113:443        ESTABLISHED 21019/firefox         tcp        0      0 192.168.42.157:50396    27.123.42.205:443       ESTABLISHED 21019/firefox         tcp        1     32 192.168.42.157:48092    198.252.206.25:443      CLOSING     -                     tcp        0      0 192.168.42.157:55798    142.250.192.99:80       TIME_WAIT   -                     tcp        0      0 192.168.42.157:34190    157.240.16.52:443       ESTABLISHED 21019/firefox         tcp        0      0 192.168.42.157:37320    34.107.221.82:80        ESTABLISHED 21019/firefox         tcp        0      0 192.168.42.157:44806    54.87.110.85:443        ESTABLISHED 21019/firefox         tcp        0      0 192.168.42.157:51202    103.229.10.173:443      ESTABLISHED 21019/firefox         

This is what I want:

Prot Local Address PortofLocalA  Foreign Address  PortofForeignA  PID/Program name      

and all its o/p below it

Transmission rewrites colons in directory names to underscores

Posted: 30 May 2021 10:21 AM PDT

Transmission appears to replace colons : in (the parent) directory name with the underscore _ character for added torrents. How to disable this behavior?

for instance:

Wikipedia: en-2020-01Wikipedia_ en-2020-01


version 3.00 (bb6b5a062e)

Store terminal output in HTML file

Posted: 30 May 2021 09:16 AM PDT

I have a CSV file that I'm counting the occurrences of column 4:

Input:

2,Serveis,14,"Restaurants, bars i hotels (Inclos hostals, pensions i fondes)",Restaurants,1  2,Serveis,11,"Finances i assegurances",Finances i assegurances,1  2,Serveis,14,"Restaurants, bars i hotels (Inclos hostals, pensions i fondes)",serveis d'allotjament,1  2,Serveis,14,"Restaurants, bars i hotels (Inclos hostals, pensions i fondes)",Bars   / CIBERCAFE,1  1,Comerç al detall,4,Equipament personal,Calçat i pell,1  2,Serveis,14,"Restaurants, bars i hotels (Inclos hostals, pensions i fondes)",Bars   / CIBERCAFE,1  2,Serveis,14,"Restaurants, bars i hotels (Inclos hostals, pensions i fondes)",Bars   / CIBERCAFE,1  1,Comerç al detall,3,Parament de la llar,Mobles i articles fusta i metall,1  2,Serveis,14,"Restaurants, bars i hotels (Inclos hostals, pensions i fondes)",Restaurants,1  

After using this code:

cat file.csv | cut -d, -f 4 | sort | uniq -c | sort -nr  

I get the following output:

6 "Restaurants  1 Parament de la llar  1 Finances i assegurances  1 Equipament personal  

And I want to write a script that runs the code and puts this output in an HTML file, in sort of a simple table or like text plain and simple. How would I do that?

speedtest-cli: ValueError: invalid literal for int() with base 10: ''

Posted: 30 May 2021 10:10 AM PDT

On 3 machines I get:

$ speedtest-cli   Retrieving speedtest.net configuration...  Traceback (most recent call last):    File "/usr/bin/speedtest-cli", line 11, in <module>      load_entry_point('speedtest-cli==2.1.2', 'console_scripts', 'speedtest-cli')()    File "/usr/lib/python3/dist-packages/speedtest.py", line 1986, in main      shell()    File "/usr/lib/python3/dist-packages/speedtest.py", line 1872, in shell      speedtest = Speedtest(    File "/usr/lib/python3/dist-packages/speedtest.py", line 1091, in __init__      self.get_config()    File "/usr/lib/python3/dist-packages/speedtest.py", line 1173, in get_config      ignore_servers = list(  ValueError: invalid literal for int() with base 10: ''  

I have tested one of these machines on two different internet connections with the same result.

Why is it not working?

Make specific key sticky

Posted: 30 May 2021 09:17 AM PDT

I have found enabling sticky keys under xorg (awesome desktop manager) on Super User, which showed me that I can enable sticky keys via

xkbset sticky -twokey  

This works as expected, but in my case I'd prefer to only make a specific key (Alt Gr) be sticky instead of all modifier keys. Can this be configured? And if so: where?

Why does POSIX require certain shell built-ins to have an external implementation?

Posted: 30 May 2021 10:00 AM PDT

From this question about whether printf is a built-in for yash, comes this answer that quotes the POSIX standard.

The answer points out that the POSIX search sequence is to find an external implementation of the desired command, and then, if the shell has implemented it as a built-in, run the built-in. (For built-ins that aren't special built-ins.)

Why does POSIX have this requirement for an external implementation to exist before allowing an internal implementation to be run?

It seems... arbitrary, so I am curious.

Changing the Visuals of GRUB in Manjaro

Posted: 30 May 2021 09:04 AM PDT

I am looking to make grub look nice. I know GRUB-Customizer exists but I have been told its incompatible with Arch. It is not in the pacman repository. So how would I make it a bit nicer.Mostly by increasing font-size and reordering and renaming the boot options.

Going above and beyond would be putting OS icons in.

Multiple images on root window with ImageMagick

Posted: 30 May 2021 10:12 AM PDT

I don't run a desktop environment, but instead use a bare window manager (StumpWM). Thus, my usual method for setting a desktop background is to run display -window root <image file> from a WM startup script, which does what I want.

When using multiple monitors, the X root window stretches across both and includes dead space. Thus, running that command line causes the image to either be stretched or tiled across the entire window, which isn't useful. I can use -geometry to display an image on either monitor, but if I run that again (to view on both monitors) it overwrites the other. There doesn't seem to be any ability with display to display two images in one invocation.

How can I use display to display different images on both monitors? Alternatively, how else can I get the "desktop background image" effect without a desktop environment?

Building python with openssl support

Posted: 30 May 2021 10:01 AM PDT

I have the following directory structure:

nchambers@nchambers-305E4A-305E5A-305E7A:~/mini-bin$ la  2to3               cpp-5.3.0            gcov-5.3.0           grmid-5.3.0         objcopy-2.25.1    python3.5-config      .sources  2to3-3.5           elfedit-2.25.1       gcov-tool-5.3.0      grmiregistry-5.3.0  objdump-2.25.1    python3.5m            strings-2.25.1  addr2line-2.25.1   g++-5.3.0            gfortran-5.3.0       gserialver-5.3.0    pydoc             python3.5m-config     strip-2.25.1  aot-compile-5.3.0  gappletviewer-5.3.0  gij-5.3.0            gtnameserv-5.3.0    pydoc3            python3-config        x86_64-unknown-linux-gnu-c++-5.3.0  ar-2.25.1          gc-analyze-5.3.0     gjar-5.3.0           idle                pydoc3.5          python-config         x86_64-unknown-linux-gnu-g++-5.3.0  .archives          gcc-5.3.0            gjarsigner-5.3.0     idle3               python            pyvenv                x86_64-unknown-linux-gnu-gcc-5.3.0  as-2.25.1          gcc-ar-5.3.0         gjavah-5.3.0         idle3.5             python2           pyvenv-3.5            x86_64-unknown-linux-gnu-gcc-ar-5.3.0  bash               gcc-nm-5.3.0         gkeytool-5.3.0       jcf-dump-5.3.0      python2.7         ranlib-2.25.1         x86_64-unknown-linux-gnu-gcc-nm-5.3.0  .binaries          gcc-ranlib-5.3.0     gnative2ascii-5.3.0  jv-convert-5.3.0    python2.7-config  readelf-2.25.1        x86_64-unknown-linux-gnu-gcc-ranlib-5.3.0  .build             gcj-5.3.0            gorbd-5.3.0          ld-2.25.1           python2-config    rebuild-gcj-db-5.3.0  x86_64-unknown-linux-gnu-gcj-5.3.0  c++-5.3.0          gcj-dbtool-5.3.0     gprof-2.25.1         ld.bfd-2.25.1       python3           size-2.25.1           x86_64-unknown-linux-gnu-gfortran-5.3.0  c++filt-2.25.1     gcjh-5.3.0           grmic-5.3.0          nm-2.25.1           python3.5         smtpd.py  nchambers@nchambers-305E4A-305E5A-305E7A:~/mini-bin$ la .sources/  bash-4.4-beta  binutils-2.25.1  boost_1_60_0  gcc-5.3.0  Python-2.7.11  Python-3.5.1  nchambers@nchambers-305E4A-305E5A-305E7A:~/mini-bin$ la .build/  bash-4.4-beta  binutils-2.25.1  boost_1_60_0  gcc-5.3.0  Python-2.7.11  Python-3.5.1  nchambers@nchambers-305E4A-305E5A-305E7A:~/mini-bin$ la .binaries/  bash-4.4-beta  binutils-2.25.1  boost_1_60_0  gcc-5.3.0  Python-2.7.11  Python-3.5.1  nchambers@nchambers-305E4A-305E5A-305E7A:~/mini-bin$   

Generally, tar balls go in .archives, unpacked sources go in .sources, building happens in .build, and the final binaries are store in .binaries. I then symlink the binaries from .binaries to ~/mini-binwhich is on my $PATH.I know its a weird workflow, but it works for me. Anyways, I am looking to rebuild python-2.7.11 (and 3.4.0, but am starting with 2.7.11) from source. It builds fine on my system, but I need it to build with ncurses, readline, ssl, etc support. Do I have to compile these libraries from source? If so, where would I put the libraries? How do I tell python to build with them. Thank you and have a good day!

cURL does not recognize certificate

Posted: 30 May 2021 07:04 AM PDT

At our company they enforce a web proxy which breaks SSL connections and replaces the certificate by its own fake certificate. (To be precise it uses a proxy cert which is signed by the company cert.)

In order to download from a https URL I therefore have to make my system trust that fake certificate (or disable certificate checking).

I therefore added both the proxy cert and the company cert to both /etc/ssl/certs/ca-bundle.crt and /etc/ssl/certs/ca-certificates.crt. (Both link to the same file.)

Now downloading with wget works fine, however downloading with curl does not work, because curl is not able to verify the certificate:

* Rebuilt URL to: https://company.net/  * Hostname was NOT found in DNS cache  *   Trying 172.18.111.111...  * Connected to 172.18.111.111 (172.18.111.111) port 3128 (#0)  * Establish HTTP proxy tunnel to company.net:443  > CONNECT company.net:443 HTTP/1.1  > Host: company.net:443  > User-Agent: curl/7.39.0  > Proxy-Connection: Keep-Alive  >  < HTTP/1.1 200 Connection established  <  * Proxy replied OK to CONNECT request  * successfully set certificate verify locations:  *   CAfile: /etc/ssl/certs/ca-bundle.crt    CApath: none  * SSLv3, TLS handshake, Client hello (1):  * SSLv3, TLS handshake, Server hello (2):  * SSLv3, TLS handshake, CERT (11):  * SSLv3, TLS alert, Server hello (2):  * SSL certificate problem: self signed certificate in certificate chain  * Closing connection 0  curl: (60) SSL certificate problem: self signed certificate in certificate chain  

What might be wrong? How can I debug further?

Port forwarding for VPN connection

Posted: 30 May 2021 08:04 AM PDT

I'm using a public wifi hotspot. Usually when I connect to these, I immediately connect to my college VPN. On this hotspot, only browsing works (no program other than a browser seems to be able to access the internet, this includes ping commands from the terminal). I'm guessing this is because of the firewall in use by the hotspot provider.

I'm given to understand that it should be possible to tunnel all of my traffic through an SSH tunnel, pointing to the correct port. I can set up a server on my home network to tunnel to, but it does not have a static IP. Is there a way around this, or possibly an easier way than SSH tunneling?

--Update--

I've since set up a raspberry pi with a static ip on my home network. Can I either tunnel all of my traffic over ssh, via the RPi, or even just tunnel the VPN connection (that I would normally use, were the ports it uses not blocked over ssh (I'm not too worried about latency).

md5sum command binary and text mode

Posted: 30 May 2021 10:41 AM PDT

The GNU md5sum command has two modes: binary mode and text mode. I guess the difference is only on how newline characters are handled? Am I right?

On GNU/Linux, the two modes always produce same result, so the only use of -b and -t options is to indicate the flag (* or ) used before the file name?

In what circumstances can the modes produce different results? On Windows/MacOS systems? (Versions for these platforms available?)

No comments:

Post a Comment