Saturday, June 26, 2021

Recent Questions - Unix & Linux Stack Exchange

Recent Questions - Unix & Linux Stack Exchange


ubuntu drive not present bios after moving partition

Posted: 26 Jun 2021 02:54 PM PDT

I have an SSD with two partitions one for ubuntu and the other one for windows, and, I don't know if this is important, but I also have a hard drive with two partitions to store windows and ubuntu data. Because I ran out of space, I got another SSD and I moved the ubuntu partition to this drive using Clonezilla, but after doing this, I can't boot windows or ubuntu, on the BIOS says "drive not present" for both OS.

I tried to fix this using boot-repair but it didn't work. I generated a Boot info summary using this tool and you can find it here.

I don't know what to do anymore, I hope someone can help me.

Skype repository key expired?

Posted: 26 Jun 2021 02:52 PM PDT

When I apt-get update, the repository for Microsoft's Skype says:

W: An error occurred during the signature verification.   The repository is not updated and the previous index files will be used.  GPG error: https://repo.skype.com/deb stable InRelease: The following signatures were invalid:  EXPKEYSIG 1F3045A5DF7587C3 Skype Linux Client Repository <se-um@microsoft.com>  W: Failed to fetch https://repo.skype.com/deb/dists/stable/InRelease     The following signatures were invalid:   EXPKEYSIG 1F3045A5DF7587C3 Skype Linux Client Repository <se-um@microsoft.com>  W: Some index files failed to download. They have been ignored, or old ones used instead.  

Why is this happening? Is there something wrong with my system? And what should I do?

ath10k_pci firmware crash

Posted: 26 Jun 2021 02:21 PM PDT

I'm on Linux Mint. My wifi randomly stops working and the pc gets laggy for about 5 seconds. This issue is recent, as I've been using Linux for almost a year now but this started happening a month ago or so

I've tried getting the latest firmware from the repo at github.com/kvalo/ath10k-firmware, as suggested here, but that didn't work

I've tried the following kernel parameters: pci=nomsi, pcie_aspm=off, pci=nommconf, and ath10k_core.skip_otp=y, but none of them solved the issue.

I've also tried the suggestions in here and here, but had no luck

The issue happens in kernels 5.8.0-55 and 5.4.0-74.

I got these logs from journal -p 3

Jun 26 15:16:33 jorgeiszlinux kernel: ath10k_pci 0000:01:00.0: failed to read device register, device is gone  Jun 26 15:16:39 jorgeiszlinux kernel: ath10k_pci 0000:01:00.0: failed to read device register, device is gone  Jun 26 15:16:42 jorgeiszlinux kernel: ath10k_pci 0000:01:00.0: failed to read device register, device is gone  Jun 26 15:16:42 jorgeiszlinux kernel: ath10k_pci 0000:01:00.0: failed to reset chip: -5  Jun 26 15:16:42 jorgeiszlinux kernel: ath10k_pci 0000:01:00.0: Could not init hif: -5  Jun 26 15:16:42 jorgeiszlinux kernel: ath10k_pci 0000:01:00.0: firmware crashed! (guid 931c5c85-8ba3-4b61-b080-9eb1a53ce38e)  Jun 26 15:16:48 jorgeiszlinux kernel: [drm:amdgpu_dm_commit_planes.constprop.0 [amdgpu]] *ERROR* Waiting for fences timed out!  Jun 26 15:16:48 jorgeiszlinux kernel: ath10k_pci 0000:01:00.0: failed to read firmware dump area: -16  Jun 26 15:16:48 jorgeiszlinux kernel: ath10k_pci 0000:01:00.0: Copy Engine register dump:  Jun 26 15:16:49 jorgeiszlinux kernel: ath10k_pci 0000:01:00.0: [00]: 0x00034400 4294967295 4294967295 4294967295 4294967295  Jun 26 15:16:49 jorgeiszlinux kernel: ath10k_pci 0000:01:00.0: [01]: 0x00034800 4294967295 4294967295 4294967295 4294967295  Jun 26 15:16:49 jorgeiszlinux kernel: ath10k_pci 0000:01:00.0: [02]: 0x00034c00 4294967295 4294967295 4294967295 4294967295  Jun 26 15:16:49 jorgeiszlinux kernel: ath10k_pci 0000:01:00.0: [03]: 0x00035000 4294967295 4294967295 4294967295 4294967295  Jun 26 15:16:49 jorgeiszlinux kernel: ath10k_pci 0000:01:00.0: [04]: 0x00035400 4294967295 4294967295 4294967295 4294967295  Jun 26 15:16:49 jorgeiszlinux kernel: [drm:amdgpu_dm_commit_planes.constprop.0 [amdgpu]] *ERROR* Waiting for fences timed out!  Jun 26 15:16:49 jorgeiszlinux kernel: ath10k_pci 0000:01:00.0: [05]: 0x00035800 4294967295 4294967295 4294967295 4294967295  Jun 26 15:16:49 jorgeiszlinux kernel: ath10k_pci 0000:01:00.0: [06]: 0x00035c00 4294967295 4294967295 4294967295 4294967295  Jun 26 15:16:49 jorgeiszlinux kernel: ath10k_pci 0000:01:00.0: [07]: 0x00036000 4294967295 4294967295 4294967295 4294967295  Jun 26 15:16:49 jorgeiszlinux kernel: wlp1s0: failed to remove key (0, 8c:5f:ad:56:34:eb) from hardware (-5)  Jun 26 15:16:49 jorgeiszlinux kernel: wlp1s0: failed to remove key (1, ff:ff:ff:ff:ff:ff) from hardware (-5)  

lshw -C network

    description: Network controller         product: QCA6174 802.11ac Wireless Network Adapter         vendor: Qualcomm Atheros         physical id: 0         bus info: pci@0000:01:00.0         version: 32         width: 64 bits         clock: 33MHz         capabilities: pm msi pciexpress cap_list         configuration: latency=0         resources: memory:e0200000-e03fffff  

My laptop is Acer Swift SF315 (AMD Ryzen 7 2700U, Radeon Vega)

Note that the issue occurs in other distros as well, so it's not Mint-related

Problems with USB to WiFi adapter TP-LINK TL-WN722N v2 and Hotspot creation

Posted: 26 Jun 2021 02:34 PM PDT

I use Linux Mint 20.1 and am trying to properly install USB WiFi device TP-LINK TL-WN722N v2

I first installed driver R8188eu following instructions from

https://github.com/lwfinger/rtl8188eu

Also this driver was recommended on following link

http://en.techinfodepot.shoutwiki.com/wiki/TP-LINK_TL-WN722N_v2

Then I followed first Answer from the topic

https://askubuntu.com/questions/1341804/tp-link-tl-wn722n-v2-driver-problem-ubuntu-20-04/1348355#1348355

and tried to instead install driver 8812au

Now when I type

sudo dkms status  

I get

rtl8812au, 5.9.3.2, 5.4.0-77-generic, x86_64: installed  

as it should be the case

but when I type

inxi -Fxxxz  

I get regarding the mentioned device

Device-3: TP-Link TL-WN722N v2 type: USB driver: r8188eu bus ID: 1-8:8 chip ID: 2357:010c serial: IF: wlxd037454c08ee state: down mac: IF-ID-1: tun0 state: unknown speed: 10 Mbps duplex: full mac: N/A  

So it's not the same USB driver as I get with dkms status

I have no idea how to change this, as I have uninstalled the first driver, than installed the new one but to no avail.

Operationally wise I can connect to existing wifi network, but I cannot use my wifi adapter to create a hotspot.

Any idea what should be done?

Two supposedly same M4A files are giving different hash results. Why?

Posted: 26 Jun 2021 02:57 PM PDT

Two files containing same song, both in M4A format, output two different results when I calculate their hashes:

md5sum  f149e2d2a232a410fcf61627578c101a  new.m4a  ad26ed675342f0f45dcb5f9de9d586df  old.m4a  

They contain same number of bytes:

ls -l  -rw-rw-r-- 1 cdc cdc 2978666 Jun 26 19:49 new.m4a  -rwxrwxr-x 1 cdc cdc 2978666 Jun 26 19:49 old.m4a  

The exiftool output differs only in dates of creation (pastebin for new.m4a and pastebin for old.m4a).

I used Audacity's tools to compare two files (by inverting and mixing them, which makes them nullify each other's similarities) and the result was silence, since there was nothing left, meaning there is no difference between two files.

Command cmp gives me:

cmp -l       54 375  23       55  51 305       56  41 112       58 375  23       59  51 305       60  45 116      170 375  23      171  51 305      172  41 112      174 375  23      175  51 305      176  41 112      270 375  23      271  51 305      272  41 112      274 375  23      275  51 305      276  41 112    cmp -b  new.m4a old.m4a differ: byte 54, line 1 is 375 M-}  23 ^S  

The only real difference is that old.m4a file was downloaded in December, 2020, and new.m4a was downloaded few hours ago.

If needed, you can download new.m4a here and old.m4a here. Originally, both were downloaded from artist's Bandcamp page.

P.S. My human ear says that these two are identical.

Can you make xfce panel applet using python?

Posted: 26 Jun 2021 02:00 PM PDT

Is it possible to make a xfce panel applet using python? Most if not all panel applets seem to use C so I doubt that I would able to.

How can I get WiFi drivers for RTL8822CE on my Arch? (repost from AskUbuntu - since that was the wrong place.)

Posted: 26 Jun 2021 01:46 PM PDT

Original post.


So I have Network controller: Realtek Semiconductor Co., Ltd. RTL8822CE 802.11ac PCIe Wireless Network Adapter integrated with my motherboard. I'm trying to find drivers, and I come across this package.
However, this package doesn't work, it gives me this error:

About to run dkms install steps...  Error! Could not locate dkms.conf file.  File: /usr/src/rtl8821ce-v5.5.2_34066.20200325/dkms.conf does not exist.  Error! Could not locate dkms.conf file.  File: /usr/src/rtl8821ce-v5.5.2_34066.20200325/dkms.conf does not exist.  Error! Could not locate dkms.conf file.  File: /usr/src/rtl8821ce-v5.5.2_34066.20200325/dkms.conf does not exist.  Finished running dkms install steps.  

And I came across this post.
Their solution was to use dpkg-reconfigure. I tried looking for an Arch equivalent to it, and its containing package debconf, until I came to the realization there may not be any Arch equivalent of either.

I also did try the steps detailed here which pretty much uses the same package. Here's some info:

Kernel Version: 5.10.46-1-lts  Hardware Model: HP HP ENVY TE01-0xxx  Processor: Intel® Core™ i7-8700 CPU @ 3.20GHz × 12  OS Type: 64-bit  

Thanks so much (this mobile-data is so painful).
Cheers!

What's the UNIX way to safely read from STDIN and create safe pipelines

Posted: 26 Jun 2021 03:11 PM PDT

I'm trying to understand how to create a program and (safely) use it in a pipeline as if it were another another terminal utility.

Specifically I'm trying to create a Python script that tells me if an image is of given resolution or ratio. My current solution is using the os module and, from inside Python, use all of the standard utilities available on Linux (such as find, grep, mv, rm, etc). The problem is each time that I want to implement something new I have to code that in Python, but always using those utilities from inside the os module, which shouldn't be required as the scope of the program should simply be filtering out files.

To solve this I would like to be able to pipe the content of another programs to my Python script, let's call it imgfilter that takes a single argument, either a resolution (e.g.: 1920x1080) or a ratio (e.g.: 16:9), and then filter out those files and output the ones that match the given criteria.

A small example is finding and moving all 1920x1080 images from ~/wallpapers to ~/1920x1080:

$ find ~/wallpapers | imgfilter 1920x1080 | xargs -I '{}' mv {} ~/1920x1080  

First question is, can I ensure that this is run sequentially? This is can I ensure that imgfilter will execute only after find has finished? Otherwise - and as seen in this post on stackoverflow - I might be running my Python script on a incomplete list of files, considering find could still be running, especially if the disk is slow or the target folder is big.

Second question is, on how to read either from STDIN or as an argument, depending on user input, in Python? This is a duplicate and has several answers online - such as the one on this post on stackoverflow - but they all seem quite old. Is the accepted answer on that post still up to date?

Command, args and process name

Posted: 26 Jun 2021 01:44 PM PDT

I learned recently of pidof, and I was comparing it to pgrep. While doing that, I noticed that pidof returned several PIDs for Firefox, while pgrep returned just one.

I checked pgrep's man page and experimented with its toggles, and got the expected output with -f:

   -f, --full            The pattern is normally only matched against the process name.              When -f is set, the full command line is used.  

Now, I knew of ps making distinctions between the full path to a command, 'simple' command name (like basename on the fullpath) and the full args, but I had never heard of process names.

In the case below, all the other processes are children of 4661, so I'm guessing Firefox forked them to take advantage of multiprocessing.

The questions, then:

  • What are process names?
  • How and why do processes set them?

Looking at these posts, it seems that it may or may not be done by changing argv[0] or calling prctl(PR_SET_NAME), and that it is used simply to help on debugging, identifying which subprocess is doing what (or, sometimes, to fool the user thinking a process is something else).

Is that so? Or are process names something in addition to argv?

$ pidof firefox  5495 5463 5391 5384 5380 5351 5330 5311 5239 5184 4661  $ pgrep firefox  4661  $ pgrep -f firefox  4661  5184  5239  5311  5330  5351  5380  5384  5391  5463  5495  $ pgrep -fl firefox  4661 firefox  5184 Web Content  5239 Web Content  5311 Web Content  5330 Web Content  5351 Web Content  5380 Web Content  5384 Web Content  5391 Web Content  5463 Privileged Cont  5495 WebExtensions  $   

Fedora 34 - Libreoffice - Could not load a pixbuf from icon theme

Posted: 26 Jun 2021 01:42 PM PDT

After upgrading to Fedora 34 I installed libreoffice. When I try to run it I get the following:

(soffice:8828): Gtk-WARNING **: 14:00:00.573: Could not load a pixbuf from icon theme.  This may indicate that pixbuf loaders or the mime database could not be found.  terminate called after throwing an instance of 'com::sun::star::uno::DeploymentException'  Application Error  

Then the program promptly crashes. It doesn't appear to be because of a broken icon.

$ rpm -qa | grep libreoffice | sort  libreoffice-7.1.4.2-1.fc34.x86_64  libreoffice-base-7.1.4.2-1.fc34.x86_64  libreoffice-calc-7.1.4.2-1.fc34.x86_64  libreoffice-core-7.1.4.2-1.fc34.x86_64  libreoffice-data-7.1.4.2-1.fc34.noarch  libreoffice-draw-7.1.4.2-1.fc34.x86_64  libreoffice-emailmerge-7.1.4.2-1.fc34.x86_64  libreoffice-graphicfilter-7.1.4.2-1.fc34.x86_64  libreoffice-gtk3-7.1.4.2-1.fc34.x86_64  libreoffice-help-en-7.1.4.2-1.fc34.x86_64  libreoffice-impress-7.1.4.2-1.fc34.x86_64  libreoffice-langpack-en-7.1.4.2-1.fc34.x86_64  libreoffice-math-7.1.4.2-1.fc34.x86_64  libreoffice-ogltrans-7.1.4.2-1.fc34.x86_64  libreoffice-opensymbol-fonts-7.1.4.2-1.fc34.noarch  libreoffice-pdfimport-7.1.4.2-1.fc34.x86_64  libreoffice-pyuno-7.1.4.2-1.fc34.x86_64  libreoffice-ure-7.1.4.2-1.fc34.x86_64  libreoffice-ure-common-7.1.4.2-1.fc34.noarch  libreoffice-writer-7.1.4.2-1.fc34.x86_64  libreoffice-x11-7.1.4.2-1.fc34.x86_64  

As you can see I have only 1 version of each package installed. I have no flatpak packages on my system (yet).

I would appreciate any assistance. Let me know if you need any more information than what I have already provided.


UPDATE: Uninstalling through dnf and installing through snap fixed the issue entirely. This could be an issue with the the rpm being broken.

Xfce panel applet that closes and opens window

Posted: 26 Jun 2021 12:25 PM PDT

I've made a few gtk apps using vala and python, and I'm wondering if it is possible to make a xfce panel applet that when clicked would open up an app and when clicked again close it. Any help would be great

How to determine when storage device containing BTRFS filesystem is failing

Posted: 26 Jun 2021 11:53 AM PDT

I noticed some corruption errors on a HDD device partition containing part of a BTRFS filesystem:

matt@ryzen3950 ~> btrfs device stats /dev/dm-2  [/dev/mapper/cryptsda1].write_io_errs    0  [/dev/mapper/cryptsda1].read_io_errs     0  [/dev/mapper/cryptsda1].flush_io_errs    0  [/dev/mapper/cryptsda1].corruption_errs  12  [/dev/mapper/cryptsda1].generation_errs  0  

Additionally, using btrfs-scrub failed to repair the errors. Does this necessarily mean that the hard drive is failing? Should I replace this hard drive and discard it? Or, is it possible that some other issue created the corruptions and the hard drive is fine? How can I determine if the hard drive is indeed healthy?

Use BTRFS replace with luks encryption

Posted: 26 Jun 2021 11:11 AM PDT

I have a failing hard disk drive, which contains a partition forming part of my root filesystem:

matt@ryzen3950 ~> btrfs device stats /dev/dm-2  [/dev/mapper/cryptsda1].write_io_errs    0  [/dev/mapper/cryptsda1].read_io_errs     0  [/dev/mapper/cryptsda1].flush_io_errs    0  [/dev/mapper/cryptsda1].corruption_errs  12  [/dev/mapper/cryptsda1].generation_errs  0  

I have another storage device that I'd like to replace this one with. To do this, I first partitioned and encrypted my new drive:

matt@ryzen3950 ~> sudo parted /dev/sdd  (parted) mklabel gpt                                                        (parted) mkpart primary btrfs 1MiB 100%                                     (parted) quit                                                               
matt@ryzen3950 ~> sudo cryptsetup luksFormat /dev/sdd1  matt@ryzen3950 ~> sudo cryptsetup open --type luks /dev/sdd1 cryptsdd1  

(I did not perform any mkfs on this)

Then, I attempted to perform btrfs replace (note, this filesystem is currently live):

sudo btrfs replace start /dev/mapper/cryptsda1 /dev/mapper/cryptsdd1 /  

However, I get the error:

/dev/mapper/cryptsdd1 appears to contain a partition table (atari).  ERROR: use the -f option to force overwrite of /dev/mapper/cryptsdd1  

This makes it sound like I should not have partitioned the drive first. Should I have done the following instead?

sudo btrfs replace start /dev/sda /devsdd /  

If not, what's the correct way to do this?

Finally, what is <path> in btrfs replace? I believe it's the filesystem mount point (e.g., / on a live root filesystem). Is this correct?

I read the full btrfs-replace manpage as well as the btrfs wiki, but was not able to find the answers to my questions.

turn off display via CLI in RHEL 8

Posted: 26 Jun 2021 11:26 AM PDT

Have a laptop running RHEL 8.4 without the gui. I would like to power off the display. I use to run vbetool dpms off on ubuntu but that doesn't work,

user@server# ls /sys/class/drm  card0  card0-DP-1  card0-HDMI-A-1  card0-LVDS-1  card0-VGA-1  renderD128  version  

[Update] I'm not using Wayland or X11, session type is shown as tty.

user@server$ loginctl show-session "$XDG_SESSION_ID" -p Type  Type=tty  

[Update] These commands complete successfully but have no affect

user@server$ setterm --blank force   user@server$ setterm --powerdown 1  

Is jq internal sort slower than GNU sort?

Posted: 26 Jun 2021 01:28 PM PDT

While filtering through this json file I did a benchmark and found out utilizing jq's internal sort and unique method is actually 25% slower than sort --unique!

Command Mean [ms] Min [ms] Max [ms] Relative
jq "[.[].category] \| sort \| unique" channels.json 172.0 ± 2.6 167.8 176.8 1.25 ± 0.06
jq "[.[].category \| select((. != null) and (. != \"XXX\"))] \| sort \| unique" channels.json 151.9 ± 4.1 146.5 163.9 1.11 ± 0.06
jq ".[].category" channels.json \| sort -u 137.2 ± 6.6 131.8 156.6 1.00
Summary    'jq ".[].category" channels.json | sort -u' ran      1.11 ± 0.06 times faster than 'jq "[.[].category | select((. != null) and (. != \"XXX\"))] | sort | unique" channels.json'      1.25 ± 0.06 times faster than 'jq "[.[].category] | sort | unique" channels.json'  

test command:

hyperfine --warmup 3 \      'jq "[.[].category] | sort | unique" channels.json'  \      'jq "[.[].category | select((. != null) and (. != \"XXX\"))] | sort | unique" channels.json' \      'jq ".[].category" channels.json | sort -u'  

If we only test sort (without uniqueness), again jq is 9% slower than sort:

Command Mean [ms] Min [ms] Max [ms] Relative
jq "[.[].category] \| sort" channels.json 133.9 ± 1.6 131.1 138.2 1.09 ± 0.02
jq ".[].category" channels.json \| sort 123.0 ± 1.3 120.5 125.7 1.00
Summary    'jq ".[].category" channels.json | sort' ran      1.09 ± 0.02 times faster than 'jq "[.[].category] | sort" channels.json'    

versions:

jq-1.5-1-a5b5cbe  sort (GNU coreutils) 8.28  

I expected using jq's internal functions would result in a faster processing than piping into an external app which itself should be spawned. Am I using jq poorly?

update Just repeated this experiment on host with FLASH storage, Arm CPU and these versions:

jq-1.6  sort (GNU coreutils) 8.32  

result:

Benchmark #1: jq "[.[].category] | sort" channels.json    Time (mean ± σ):     587.8 ms ±   3.9 ms    [User: 539.5 ms, System: 44.2 ms]    Range (min … max):   582.8 ms … 594.2 ms    10 runs     Benchmark #2: jq ".[].category" channels.json | sort    Time (mean ± σ):     606.0 ms ±   8.6 ms    [User: 569.5 ms, System: 49.0 ms]    Range (min … max):   589.6 ms … 616.2 ms    10 runs     Summary    'jq "[.[].category] | sort" channels.json' ran      1.03 ± 0.02 times faster than 'jq ".[].category" channels.json | sort'  

Now jq sort runs 3% faster than GNU sort :D

Re-mount Raid Array (mdadm) and Access Existing Files

Posted: 26 Jun 2021 02:39 PM PDT

Hopefully this question has a simple answer. I created and mounted a RAID5 array with 4 disks. I copied about 4TB of files from a external HDD and in an attempt to allow proper permissions for my media server, accidentally changed permissions for /. It was easiest to just replace the OS and after reinstalling Ubuntu I need to simply re-mount the array to access the previously copied data.

Here's what I did:

$ cat /proc/mdstat  Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]   unused devices: <none>    $ lsblk -o NAME,SIZE,FSTYPE,TYPE,MOUNTPOINT  NAME     SIZE FSTYPE            TYPE MOUNTPOINT  loop0   99.4M squashfs          loop /snap/core/11187  loop1   55.5M squashfs          loop /snap/core18/1988  loop2    219M squashfs          loop /snap/gnome-3-34-1804/66  loop3   96.1M squashfs          loop /snap/plexmediaserver/211  loop4     51M squashfs          loop /snap/snap-store/518  loop5   64.8M squashfs          loop /snap/gtk-common-themes/1514  loop6   31.1M squashfs          loop /snap/snapd/11036  sda     10.9T linux_raid_member disk   sdb     10.9T linux_raid_member disk   sdc     10.9T linux_raid_member disk   sdd     10.9T linux_raid_member disk   sde    465.8G                   disk   ├─sde1   512M vfat              part   ├─sde2   513M vfat              part /boot/efi  ├─sde3     1K                   part   └─sde5 464.8G ext4              part /  

this indicated to me that the raid mdd on each disk was still intact, however the designated /dev/md/5 file did not exist (just sd[a-d]). Since I am new I bumbled around for a bit confirming my suspicions.

$ cat /proc/mdstat  Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]   unused devices: <none>    $ sudo mkfs.ext4 -F /dev/md/5  mke2fs 1.45.5 (07-Jan-2020)  The file /dev/md/5 does not exist and no size was specified.    $ sudo mdadm --assemble --scan  mdadm: Devices UUID-f53866f3:b6bd8441:1a92a871:b29eb9e5 and UUID-f53866f3:b6bd8441:1a92a871:b29eb9e5 have the same name: /dev/md/5  mdadm: Duplicate MD device names in conf file were found.    $ cat /etc/mdadm/mdadm.conf  # mdadm.conf  #  # !NB! Run update-initramfs -u after updating this file.  # !NB! This will ensure that initramfs has an uptodate copy.  #  # Please refer to mdadm.conf(5) for information about this file.  #    # by default (built-in), scan all partitions (/proc/partitions) and all  # containers for MD superblocks. alternatively, specify devices to scan, using  # wildcards if desired.  #DEVICE partitions containers    # automatically tag new arrays as belonging to the local system  HOMEHOST <system>    # instruct the monitoring daemon where to send mail alerts  MAILADDR root    # definitions of existing MD arrays  ARRAY /dev/md/5  metadata=1.2 UUID=f53866f3:b6bd8441:1a92a871:b29eb9e5 name=myname:5    # This configuration was auto-generated on Fri, 25 Jun 2021 16:11:25 -0600 by mkconf  ARRAY /dev/md/5 metadata=1.2 name=myname:5 UUID=f53866f3:b6bd8441:1a92a871:b29eb9e5      

I don't know why I thought I need to make a filesystem, but hopefully the error means that nothing was changed. I'm also not sure why I ended up with duplicate entries in mdadm.conf, but I assume it was from one of my mdadm --detail calls or something. But removing those lines was easy enough

$ sudo cp /etc/mdadm/mdadm.conf /etc/mdadm/mdadm.conf.backup  $ sudo -i  root~# grep -v "5 UUID" /etc/mdadm/mdadm.conf.backup > /etc/mdadm/mdadm.conf  root~# cp /etc/mdadm/mdadm.conf /etc/mdadm/mdadm.conf.backup  root~# grep -v "auto-generate" /etc/mdadm/mdadm.conf.backup > /etc/mdadm/mdadm.conf  

mdadm.conf now only contains one definition for md5. I know this is not necessary for assembling the array but at this point I wanted to re-assemble and update-initramfs.

$ sudo mdadm -A /dev/md/5 /dev/sd[a-d]  mdadm: /dev/md/5 has been started with 4 drives.    $ sudo update-initramfs -u -v  $ sudo mkdir /mnt/md5  $ sudo mount /dev/md/5 /mnt/md5  

Everything seems to be working but I don't understand how to access the existing files on the array. I don't believe anything I did would have overwritten the drives but the /mnt/md5 directory is empty w/ 30+TB storage. fdisk confirms

$ sudo fdisk -l  Disk /dev/sdd: 10.94 TiB, 12000138625024 bytes, 23437770752 sectors  Disk model: ST12000NM0008-2H  Units: sectors of 1 * 512 = 512 bytes  Sector size (logical/physical): 512 bytes / 4096 bytes  I/O size (minimum/optimal): 4096 bytes / 4096 bytes      Disk /dev/sda: 10.94 TiB, 12000138625024 bytes, 23437770752 sectors  Disk model: ST12000NM0008-2H  Units: sectors of 1 * 512 = 512 bytes  Sector size (logical/physical): 512 bytes / 4096 bytes  I/O size (minimum/optimal): 4096 bytes / 4096 bytes      Disk /dev/sde: 465.78 GiB, 500107862016 bytes, 976773168 sectors  Disk model: Samsung SSD 860   Units: sectors of 1 * 512 = 512 bytes  Sector size (logical/physical): 512 bytes / 512 bytes  I/O size (minimum/optimal): 512 bytes / 512 bytes  Disklabel type: dos  Disk identifier: 0x4b8331f4    Device     Boot   Start       End   Sectors   Size Id Type  /dev/sde1          2048   1050623   1048576   512M  b W95 FAT32  /dev/sde2  *    1050624   2101247   1050624   513M ef EFI (FAT-12/16/32)  /dev/sde3       2103294 976771071 974667778 464.8G  5 Extended  /dev/sde5       2103296 976771071 974667776 464.8G 83 Linux      Disk /dev/sdc: 10.94 TiB, 12000138625024 bytes, 23437770752 sectors  Disk model: ST12000NM0008-2H  Units: sectors of 1 * 512 = 512 bytes  Sector size (logical/physical): 512 bytes / 4096 bytes  I/O size (minimum/optimal): 4096 bytes / 4096 bytes      Disk /dev/sdb: 10.94 TiB, 12000138625024 bytes, 23437770752 sectors  Disk model: ST12000NM0008-2H  Units: sectors of 1 * 512 = 512 bytes  Sector size (logical/physical): 512 bytes / 4096 bytes  I/O size (minimum/optimal): 4096 bytes / 4096 bytes      Disk /dev/md5: 32.76 TiB, 36000010076160 bytes, 70312519680 sectors  Units: sectors of 1 * 512 = 512 bytes  Sector size (logical/physical): 512 bytes / 4096 bytes  I/O size (minimum/optimal): 524288 bytes / 1572864 bytes  

Any help is appreciated with restoring the files that (presumably) still exist. Like I said hopefully this one isn't complicated!

Writing commands with a specific colour

Posted: 26 Jun 2021 01:22 PM PDT

I use the terminal and want to use a specific colour when writing commands.

For instance, suppose I write "history" on the command-line. I would like that the commands entered from the keyboard are colourised with a predefined colour.

In summary, all the commands entered by the user will show with the same colour.

enter image description here

Thus

history    grep -hir -C 8 "hello world" .  

will show in blue.

Currently I have the following for PS1

PS1='\n\[\033[01;36m\]\u@\h:'  PS1=$PS1'\n+ \[\033[38;5;214m\]\w\[\033[00m\]\n'  

Then I did the following

OPS0="$PS0" OPS1="$PS1"  t7="$(tput setaf 7)" t5="$(tput setaf 5)"  [[ -n "$PS1" ]] && PS0="\[$t7\]" PS1="\[$t7\]$PS1\[$t5\]"  

I get two strange characters at the beginning, just before 01cuneus

pete@home:~$ ls  01cuneus  Admir          Documents   logs      Public  02chaos   conjgrad.f90~  Downloads   Music     temp  03marshl  Desktop        fontconfig  Pictures  Videos  

Combining different terms in third column into one row according to sample code

Posted: 26 Jun 2021 02:00 PM PDT

I have some genomic data with three columns. Column 1 is samples, Column 2 is cancer type, and Column 3 is mechanism. Each sample has one or more gene mechanisms assigned to it (e.g. 6 ecDNA) in the third row. Currently for each sample there are multiple rows per mechanism (So for e.g. sample x has 4 rows with the "6 ecDNA" mechanism 5 rows with the "5 deletion" mechanism where each row corresponds to one genetic aberration found for that sample).

I was wondering if there was a way to display all types of mechanisms in the data for each sample in one row

(e.g. Sample Cancer Abbrev Mechanism)

     x        Bladder     6 ecDNA, 5 deletion  

Is there a way to condense the types of mechanisms for a sample into one row? Edit: This is a representative table of the OG dataset with example samples "x" and "y".

Sample, Cancer Abbrev, Mechanism

 x,       Bladder,         6 ecDNA     x,       Bladder,         6 ecDNA     x,       Bladder,         5 deletion     x,       Bladder,         5 deletion     y,       Osteosarcoma,    4 transposition     y,       Osteosarcoma,    4 transposition    y,       Osteosarcoma,    5 deletion    y,       Osteosarcoma,    6 ecDNA  

....

How can AIX jfs2 snapshot size be less than size of the data on the fs?

Posted: 26 Jun 2021 01:02 PM PDT

On every OS I ever create the snapshots of size bigger than the occupied fs space. For example: if I had a fs called /try of 250GB but the space used is 100GB I will create a snapshot of 100GB or a little more. On AIX I create a snapshot with a size under the fs occupied space..and works. On a logical volume of 2GB I create a file of 590MB

dd if=/dev/zero of=hello bs=1m count=590  590+0 records in  590+0 records out  

then I create the snapshot of 256mb instead of 590MB or more

snapshot -o snapfrom=/try -o size=256m   Snapshot for file system /try created on /dev/fslv00  

df report the correct size of fs "try"

df -hT /try  /dev/try     jfs2    2,0G  591M    1,5G  29% /try  

I can mount the snapshot

mount -o snapshot /dev/fslv00 /mnt/  

df report size of 256mb for the snapshot but with 1,5mb used!

df -hT /mnt  /dev/fslv00    jfs2  256M  1,5M    255M   1% /mnt  

For the final try I have make a md5sum check of a file..and works for both snapshot and real fs.

b7fb2207c641085a9cf02064edf1742e  /mnt/hello  b7fb2207c641085a9cf02064edf1742e  /try/hello  

The questions are simple: how can the snapshot size being under the size of the real fs? Why the space on the snapshot (1,5mb) is under the space used(591mb) of the real fs?

Disambiguating the word "command" in Linux

Posted: 26 Jun 2021 12:40 PM PDT

The word command refers to two different concepts in Linux:

  1. An executable program, such as grep (or a shell built-in, such as cd). Example usage: "Here are the top 10 Linux commands you should learn."
  2. A full text string sent to the shell for execution, such as grep com /etc/hosts. Example usage: "Type a Linux command and press Enter."

Does anyone have any best practices for avoiding this ambiguity when writing prose about Linux commands? Here are some attempts I've rejected already:

  • Using the word program or executable for meaning #1. It's inaccurate for shell built-ins.
  • Using the phrase command line for meaning #2. That's confusing because "command line" is also a synonym for "shell."
  • Using the phrase command string for meaning #2. It's imprecise because both #1 and #2 are strings.

Any advice appreciated.

How to append a string to every cell (with awk)?

Posted: 26 Jun 2021 03:10 PM PDT

This is an output of cal. What is the simplest way to append an additional <SPACE> to every cell?

     June 2021          Su Mo Tu We Th Fr Sa           1  2  3  4  5     6  7  8  9 10 11 12    13 14 15 16 17 18 19    20 21 22 23 24 25 26    27 28 29 30  

An expected result:

     June 2021  Su  Mo  Tu  We  Th  Fr  Sa           1   2   3   4   5   6   7   8   9  10  11  12  13  14  15  16  17  18  19  20  21  22  23  24  25  26  27  28  29  30  

Format data into a table

Posted: 26 Jun 2021 12:13 PM PDT

How can I get the details and transpose it to horizontal form?

Every record ends after Couse. Couse will never be blank or null.

Note: These four headers will be there for the below data: Name, City, Age, Couse

If you see the second record, there isn't any "Name": "" -> missing so it should be null in place of that and the remaining will be appended after that with a pipe separated like this: null | Ors | 11 | MB

I have data like below in the demo.txt file

"Name":"asxadadad  ,aaf dsf"  "City":"Mum"  "Age":"23"  "Couse":"BBS"  "City":"Ors"  "Age":"11"  "Couse":"MB"  "Name":"adad sf"  "City":"Kol"  "Age":"21"  "Couse":"BB"  "Name":"pqr"  "Age":"21"  "Couse":"NN"  

Expected Output:

asxadadad  ,aaf dsf | Mum  | 23 | BBS  null                | Ors  | 11 | MB  adad sf             | Kol  | 21 | BB  pqr                 | null | 21 | NN  

I tried the below code: but not working my logic

counter=0  var_0='Couse'     while read -r line       echo "$line"       counter=$(( counter + 1 ))       var_1=`echo "$line" | grep -oh "Couse"`       if [ $var_0 == $var_1 ]     then          head -$counter demo.txt > temp.txt          sed -i '1,$counter' demo.txt          counter = 0     else          echo "No thing to do"     fi     done < demo.txt  

Ubuntu 20.04 upgrades breaks keyboard/mouse

Posted: 26 Jun 2021 03:03 PM PDT

I was using 19. Ubuntu recommended an update. I clicked all the buttons and let it do its thing.

After the update restarted the computer, I tried using the mouse. Nothing. I tried the keyboard. Nothing.

A couple manual restarts later, I get the mouse to work for half a second before it goes dead. I have no idea how to fix this type of problem. I suppose I could try a fresh install in the BIOS, but is there a possibly less extreme method? I spent a lot of time setting my environment up and I don't want to lose it.

ntp won't work on read-only Raspberry Pi

Posted: 26 Jun 2021 12:48 PM PDT

I'm struggling to understand WHY ntp (the service) won't set the time correctly on my raspberry pi.

I have configured the filesystem as read only, to save my SD card, but it used to work, and I cannot seem to figure out why ntp won't work now.

In the logs I get many many lines of that message:

ntpd[415]: kernel reports TIME_ERROR: 0x41: Clock Unsynchronized  ntpd[415]: kernel reports TIME_ERROR: 0x41: Clock Unsynchronized  ntpd[415]: error resolving pool 0.debian.pool.ntp.org: Temporary failure in name resolution (-3)  ntpd[415]: error resolving pool 1.debian.pool.ntp.org: Temporary failure in name resolution (-3)  ntpd[415]: error resolving pool 2.debian.pool.ntp.org: Temporary failure in name resolution (-3)  ntpd[415]: error resolving pool 3.debian.pool.ntp.org: Temporary failure in name resolution (-3)  ntpd[415]: error resolving pool 3.debian.pool.ntp.org: Temporary failure in name resolution (-3)  ntpd[415]: error resolving pool 2.debian.pool.ntp.org: Temporary failure in name resolution (-3)  ntpd[415]: error resolving pool 1.debian.pool.ntp.org: Temporary failure in name resolution (-3)  ntpd[415]: error resolving pool 0.debian.pool.ntp.org: Temporary failure in name resolution (-3)  

My /etc/resolv.conf looks like this:

# Generated by resolvconf  nameserver 8.8.8.8  nameserver 192.168.1.22  

I have access to internet on that RPi, I can ping the pool addresses, I can ping google, I can apt update (after remounting in rw)...

I also can issue an ntpdate command manually and IT WORKS!

$ sudo ntpdate -u 0.fr.pool.ntp.org 1.fr.pool.ntp.org  24 Nov 23:04:34 ntpdate[578]: step time server 129.250.35.250 offset 2418.621037 sec  

So yeah, I'm pulling hairs here. I cannot understand why the ntp service won't work. I scourged the internet, nobody seems to have this particular issue (all have a malfunctioning dns, but mine is working)

My read-only setup is the following: https://hallard.me/raspberry-pi-read-only/

Do you guys have any idea?

Set default permissions for new files and folders

Posted: 26 Jun 2021 02:05 PM PDT

I have an upload directory which has information fed into it by users, each time a user registers they submit a picture which creates a folder for them in the upload directory and stores their photo(s) there. The issue is, by default this upload directory and the photos in them are 0755 and 0644 respectively and i cant quite figure out a way which will reliably change the defaults of new files and folders (umask seemsearching seemed fruitful but I thought i'd ask advice on my specific situation.)

I'd like it when a new folder is created in uploads that folder is only accessible by owner and the admin and the same for any files created in those folders. This is with the ultimate intention of making that their private "profile" area which only they and the admin can access. Any help would be greatly appreciated.

File permission with six bytes in git. What does it mean? [duplicate]

Posted: 26 Jun 2021 01:31 PM PDT

I performed a git commit command and it gave me the following reply:

7 files changed, 93 insertions(+), 15 deletions(-)  mode change 100644 => 100755 assets/internal/fonts/icomoon.svg  mode change 100644 => 100755 assets/internal/fonts/icomoon.ttf  mode change 100644 => 100755 assets/internal/fonts/icomoon.woff  

I know files can have user / group / other rwx permissions and those can be expressed as three bytes, like "644" or "755". But why is git showing six bytes here?

I've read the following articles but didn't find an answer:

Job queueing on a single machine

Posted: 26 Jun 2021 12:02 PM PDT

I have a shiny new server for running simulations on, with a pair of Tesla GPUs and 32 cores, running CentOS 7.2. I'd like for multiple users to be able to submit jobs to the server that get queued up and run when the previous finishes, preferably with some sort of prioritisation system and time limit, like PBS/TORQUE but for a single machine rather than a cluster. I know I can install and configure TORQUE for a single machine, but it seems like overkill - theoretically, the scheduler should only have to run when jobs finish or run overtime. I can probably homebrew a set of scripts, but I was wondering if a solution already exists?

Merging folders with mv?

Posted: 26 Jun 2021 12:48 PM PDT

If I use mv to move a folder called "folder" to a directory that already contains "folder" will they merge or will it be replaced?

Need to install VirtualBox Guest Additions 4.3.8 in CentOS 6.3

Posted: 26 Jun 2021 01:01 PM PDT

My host system is OSX 10.9.2 and I'm running VirtualBox 4.3.8.

In a Hadoop class I took recently, we were provided with a VM disk image preloaded with CentOS 6.3 and a lot of Hadoop-related software. This is what I have to work with.

As it stands, I don't have bidirectional cut/paste or any of the other goodies that VBOX Guest Additions provide.

I recently posted in this forum a question about upgrading/updating (sorry, I don't understand the distinction, although I understand that there is one) from CentOS 6.3 to 6.5. The only reason that I think I succeeded is that CentOS would no longer boot to the desktop. (It seems possible that it was no longer compatible with the Hadoop software that was on the disk.) So apparently that wasn't the solution to my problem.

I know that it is possible get VBGA working for this disk image: someone else in my class had OSX and VirtualBox, and was able to get past the problem I'm about to describe.

Here's what I've done so far:

  • DL'ed and installed VBOX 4.3.8 for OSX from https://www.virtualbox.org/wiki/Downloads
  • DL'ed http://download.virtualbox.org/virtualbox/4.3.8/Oracle_VM_VirtualBox_Extension_Pack-4.3.8-92456.vbox-extpack
  • Doubleclicked the file, the VB Manager said "The extension pack is already installed with the same version, would you like reinstall it?" I reinstalled, just for grins.
  • Booted, ran yum info kernel, saw that V2.6.32r279.el6 was "installed", and that V2.6.32r431.5.1.el6 was "available". I don't know enough about Linux to know if that means CentOS 6.3 and 6.5, or if it's something else entirely.
  • Also noted that Guest Additions functionality was not there.
  • Used VirtualBox VM > Devices > Insert Guest Additions CD Image… to mount the CD (it appeared on the desktop)
  • Ran VBoxLinuxAdditions.run
  • Building the main Guest Additions module … FAILED
  • /var/log/vboxadd-install.log: "Unable to find sources of current Linux kernel. Specify KERN_DIR= and run make again."

Off to Google. This post most closely resembled my problem. However, it differs enough (Windows, i686, VBOX 4.3.6) to make me afraid I'll just wander off into another morass.

Could someone provide baby-step guidance from here?

Make a computer act as a virtual USB device for other equipments

Posted: 26 Jun 2021 01:00 PM PDT

Some applications simulate a virtual USB or CD Rom drive as if a USB drive is attached to the computer.

Is there any configuration or application that provides a virtual USB drive, not for the the operating system itself, but for other equipments which accept USB drive, through a USB port.

So I'll have a virtual hard disk (e.g. a *.vdi file) in the computer, which is connected, through a USB socket, as a USB drive to some other equipment (e.g. a cell phone or a laptop).

No comments:

Post a Comment