Sunday, February 6, 2022

Recent Questions - Server Fault

Recent Questions - Server Fault

How to set up multiple apache + php-fpm + mariadb service groups on a single server with per-group resource limits?

Posted: 06 Feb 2022 09:21 AM PST

We have a centos8 web server hosting hundreds of websites, using 3 services: apache, mariadb and php-fpm. Because apache and mariadb do not support per-account resource limitation, all websites have the same access to many system resources (storage, database) and a single website can overload the machine (often that happens through database queries) and bring down all websites hosted there.

We would like to use Linux cgroups and possibly other container features to introduce limits to resources one website can use, mostly the number of php processes and I/O operations and database queries per second.

One (not necessarily the most performant) way to achieve this is to run one dedicated group of services (apache+mariadb+php-fpm) for each website or website group, with specific configuration of resource limits. Thus, to have hundreds of simultaneously running service groups.

What is the more straightforward way to set this up from existing setup of working 3 services? Performance is secondary and we do not want to go into creating and running hundreds of container images. Instead, we would like to have everything defined in an easily runtime-modifiable form in the root filesystem and control all services through systemd. Thus, no docker containers.

SSL subdomain doesn't redirect

Posted: 06 Feb 2022 08:03 AM PST

I am trying to visit app.example.com.

  1. When I type app.example.com, it goes to example.com.
  2. If I type https://app.example.com, it goes to app.example.com.

Why is the first happening?

Here's the nginx.conf

# APP =====================================================    server {      listen 80;      server_name app.example.com;         return 301 https://app.example.com$request_uri;  }    server {      listen 443 ssl;      server_name app.example.com;        ssl_certificate /etc/letsencrypt/live/app.example.com/fullchain.pem;      ssl_certificate_key /etc/letsencrypt/live/app.example.com/privkey.pem;        location / {          proxy_pass 'http://127.0.0.1:3000';      }  }    # WEBSITE =====================================================    server {          listen 80;      server_name example.com www.example.com;        return 301 https://example.com$request_uri;  }    server {      listen 443 ssl;      server_name www.example.com;        ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem;      ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;        return 301 https://example.com$request_uri;  }    server {      listen 443 ssl;      server_name example.com;        ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem;      ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;        location / {          proxy_pass 'http://127.0.0.1:3002';      }  }

HP software, How to get rid of this annoying message: "No system tray detected on this system"

Posted: 06 Feb 2022 07:49 AM PST

I want to know how to get rid of this annoying message: "No system tray detected on this system" after I start my PC every day. This happened since I installed HP printer and scanner drivers (HPLIP) on my PC on Ubuntu 18.04.6 LTS

Naturally, I want to keep my HP printer working. Btw, the scanner is not recognized by the system yet

enter image description here

Why is virt-install failing with “Read-only file system” when specifying initrd on a locally mounted ISO?

Posted: 06 Feb 2022 08:28 AM PST

I'm using virt-install to create an Ubuntu machine using the official Ubuntu server live ISO. This command has been working in the past, but stopped working at some point (maybe after upgrading the host OS from Ubuntu 18.04 to 20.04, but I'm not sure).

$ mkdir image-builder-68692987-installer  $ mount ubuntu-21.10-live-server-amd64.iso image-builder-68692987-installer  $ virt-install --name image-builder-68692987 \      --noreboot --os-variant ubuntu20.04 \      --disk gitlab-runner-ubuntu-base.qcow2 \      --disk image-builder-68692987-seed.iso \      --cdrom ubuntu-21.10-live-server-amd64.iso \      --boot 'kernel=/var/lib/libvirt/images/gitlab-runner/image-builder-68692987-installer/casper/vmlinuz,initrd=/var/lib/libvirt/images/gitlab-runner/image-builder-68692987-installer/casper/initrd,kernel_args=autoinstall console=ttyS0' \      --vcpus=4 \      --ram=4096 \      --network default \      --graphics none

Output:

WARNING  CDROM media does not print to the text console by default, so you likely will not see text install output. You might want to use --location. See the man page for examples of using --location with CDROM media    Starting install...  ERROR    internal error: child reported (status=125): unable to open     /var/lib/libvirt/images/gitlab-runner/image-builder-68692987-installer/casper/initrd:     Read-only file system  Domain installation does not appear to have been successful.

So for some reason, it is complaining that initrd is on a read-only filesystem. This is obviously true, since it's pointing to the Ubuntu ISO which is mounted via a loopback device—but why does it think it needs write access?

How to set max input vars on PHP_FPM?

Posted: 06 Feb 2022 06:39 AM PST

I have a LAMP stack Digitalocean droplet. I just changed to PHP_FPM and now I can't set max input vars. How to do so?

Thanks!

DNS record for UPN alias?

Posted: 06 Feb 2022 04:54 AM PST

I have a active directory domain ad.example.com and I configured an UPN alias example.com

When a user tries to logon as user@example.com how does it know the domain is actually ad.example.com?

Don't I need a SRV record or something in the parent zone to hint to the AD?

mariabackup fails with Error: cannot mkdir 2: /data/backups/mariadb/

Posted: 06 Feb 2022 06:08 AM PST

Trying to use mariabackup to take backup of all our databases. First created backup directory as

mkdir -p /mnt/data/backup/mariadb  chown -R mysql:mysql /mnt/data/backup/mariadb

And proceed to do backup

# mariabackup --backup --target-dir=/mnt/data/backups/mariadb/ --user=root --password=xxxxxxxx  [00] 2022-02-06 11:28:46 Connecting to MySQL server host: localhost, user: root, password: set, port: not set, socket: /mnt/data/mysql/mysql.sock  [00] 2022-02-06 11:28:46 Using server version 10.3.28-MariaDB  mariabackup based on MariaDB server 10.3.28-MariaDB Linux (x86_64)  [00] 2022-02-06 11:28:46 uses posix_fadvise().  [00] 2022-02-06 11:28:46 cd to /mnt/data/mysql/  [00] 2022-02-06 11:28:46 open files limit requested 0, set to 1024  [00] 2022-02-06 11:28:46 mariabackup: using the following InnoDB configuration:  [00] 2022-02-06 11:28:46 innodb_data_home_dir =   [00] 2022-02-06 11:28:46 innodb_data_file_path = ibdata1:12M:autoextend  [00] 2022-02-06 11:28:46 innodb_log_group_home_dir = ./  [00] 2022-02-06 11:28:46 InnoDB: Using Linux native AIO  2022-02-06 11:28:46 0 [Note] InnoDB: Number of pools: 1  [00] 2022-02-06 11:28:46 Error: cannot mkdir 2: /mnt/data/backups/mariadb/

tried goggling and various options it did not worked so finally tried without --target-directory option and did backup again and it worked successfully and got this

# ll  total 8  drwxr-xr-x  3 mysql mysql   21 Feb  6 11:41 backup  drwxr-xr-x 27 mysql mysql 4096 Feb  4 19:07 mysql  drwx------ 27 root  root  4096 Feb  6 11:22 xtrabackup_backupfiles

What did I do wrong or how do I run this with ----target-directory

HTB "rate" parameter limits available bandwidth

Posted: 06 Feb 2022 07:47 AM PST

Good day,

I have the following situation: 4 TCP streams of data from one machine to another. Each streams has its own destination TCP Port. 4 streams have different priorities : high, medium, low, bulk. High, medium, low generate 1.67Mbit/s and bulk generates 10Mbit/s. (iperf3 used to generate the traffic). Packets of each stream are marked with appropriate DiffServ mark (DSCP) and this mark is used for the classification of traffic in the HTB qdisc.

Goal: HTB qdisc should be configured in such way that at any time the high prio stream gets its requiered 1.67Mbit/s, medium prio is also guaranteed 1.67Mbit/s but with slightly lower prio and the rest of the traffic should be guaranteed 50kbit/s. Each stream must be able to use the whole link if it is idle and the stream generates more bandwidth than initially specified.

Generation of traffic:

High priority:  iperf3 -c 192.168.88.254 -p 5150 -t 62 -b 1.67M -l 128 -S 224   Medium priority:  iperf3 -c 192.168.88.254 -p 5160 -t 62 -b 1.67M -l 4K -S 160   Low priority:  iperf3 -c 192.168.88.254 -p 5170 -t 62 -b 1.67M -l 4K -S 96   Bulk:  iperf3 -c 192.168.88.254 -p 5180 -t 62 -b 10M -l 4K -S 0

Configuration of HTB qdisc

NI="eth2"  AC="sudo /sbin/tc class add dev "    # Delete previous qdiscs  sudo /sbin/tc qdisc del dev $NI root    # Add HTB as root with default class 40 for uncategorized traffic  sudo /sbin/tc qdisc add dev $NI root handle 1: htb default 40  sudo /sbin/tc class add dev $NI parent 1: classid 1:1 htb rate 3.5mbit ceil 1000mbit    # high priority stream DSCP 224 - 1110 0000 - 0xE0  $AC $NI parent 1:1 classid 1:10 htb rate 1.7mbit ceil 1000mbit prio 1  # medium priority stream   $AC $NI parent 1:1 classid 1:20 htb rate 1.7mbit ceil 1000mbit prio 2  # low priority stream  $AC $NI parent 1:1 classid 1:30 htb rate 50kbit ceil 1000mbit prio 3  # bulk stream  $AC $NI parent 1:1 classid 1:40 htb rate 50kbit ceil 1000mbit prio 4     # Add filters to classify packets based on dscp mark    # high priority DSCP 224 - 1110 0000 - 0xE0  sudo /sbin/tc filter add dev $NI protocol ip parent 1: prio 1 u32 match ip tos 0xE0 0xff flowid 1:10  # medium priority DSCP 160 - 1010 0000 - 0xA0  sudo /sbin/tc filter add dev $NI protocol ip parent 1: prio 2 u32 match ip tos 0xA0 0xff flowid 1:20  # low priority DSCP 96 - 1100 0000 - 0x60  sudo /sbin/tc filter add dev $NI protocol ip parent 1: prio 3 u32 match ip tos 0x60 0xff flowid 1:30  # bulk DSCP 0 - 0000 0000 - 0x00  sudo /sbin/tc filter add dev $NI protocol ip parent 1: prio 4 u32 match ip tos 0x00 0xff flowid 1:40

The traffic is classified correctly. I can see relevant counters in tc class statistics going up. I rechecked this multiple times.

Problem: This configuration correctly allocates bandwidth to high and medium prio streams. Low prio and bulk also receive their 50kbit. However, I am not able to push through the link more than the value specified in the root class1:1 as rate i.e. 3.5mbit.

In every article and manual about HTB that I read it was stated, that "rate" parameter is the minimal guaranteed rate to the class and "ceil" is the max amount it can get. In my case it seems that "rate" caps the link at specified value. This is definitely not the desired and expected behaviour.

If I set the "rate" parameter of the root class to same value as "ceil" i.e. 1000mbit, no prioritisation takes place and the available bandwidth is divided equally between all flows. This is not the desired behavior as in the case of fluctuations of available bandwidth, prio traffic will get less than 1.67Mbit/s

Have I misunderstood the meaning of "rate" parameter in the root class? Is this problem somehow related to onther HTB parameters like "quantum"? I also observed that every class has negative amount of tokens during the transmission of data. Is this bad? If so, what parameters should i tune and how?

Thank You in advance!

Did something stupid that interrupting Ethernet access for "some ports"

Posted: 06 Feb 2022 06:19 AM PST

I have just installed Ubuntu Server 20.04 on my server, but I executed an automated script without carefully watching, and it resulted in a really weird scenario.

I also have multiple webservers enabled on that server, but on alternative ports. Before I executed it, I can access them easily by typing server's Ethernet address and it's port. But after it, I just don't know why it wouldn't work. Here is some info I can provide:

  1. Ports, such as 22(SSHD), 25(Postfix), 80 and 443(Apache) all worked very well. My Nginx server listening on all other ports, for example, if I get it to listen on port 30000, I will be able to see the port occupied by Nginx via:

    lsof -i:40000

But for some reason, while listening all the other ports is not accessible, if I stop apache and make nginx listen on 80 and 443, it would work. Though, no luck if I make Nginx to listen on 81 and 444.

  1. It is not a router issue. Though, telnet Nginx address:port have no response.

  2. Accessing from Ethernet is not working, but if I do that on localhost, like if I log into my server via ssh and "telnet localhost 40000", I can get Nginx's response.

  3. All the problems was probably due to a careless iRedmail install. I am still digging it's installation script.

Please let me know if you have any idea, really don't want to reinstall a whole server just for this...Thanks in advance!

UniFi Security Gateway (Strongswan) site to site VPN with branch side has no real IP at all (e.g. 4G connections) and dual WAN fallback

Posted: 06 Feb 2022 05:59 AM PST

As title described, here is the scenario I have:

  1. HQ - USG Pro (Unified Security Gateway Pro) with static public IP
  2. Branch Office - USG with dual WAN, either one or both of the WAN port has no real IP due to using mobile 4G Network (the public IP is a 10.x.x.x Class A private network)

Ideally I would like to configure a site-to-site VPN setup but leveraging dialup client for the branch office so that it will work without a real IP. It would be great if I can get it work directly using Unifi controller (I have one on the cloud, with all the equipment provisioned using L3 IP based approach), however if it is impossible I can also use underlying configuration hack directly using the gateway json override trick:

https://help.ui.com/hc/en-us/articles/215458888-UniFi-USG-Advanced-Configuration-Using-config-gateway-json

IPTables log connection pid on OUTPUT chain

Posted: 06 Feb 2022 06:04 AM PST

I'm trying to find pid of a socket using iptables OUTPUT chain log, or even better adding it into the log.

My current iptable rule:

sudo iptables -A OUTPUT -j LOG --log-prefix='[PID]' --log-level 7 --log-uid

I'm a bit frustrated since I know that iptables' owner module can filter items by pid (using -m owner --owner-pid flag) which means that the info is there, but I can't log it.

I know that it impossible to do with INPUT chain since iptables is a kernel process, but for OUTPUT chain it should be possible.

any idea? or even of how to cross some log data in order to get PID of OUTPUT chain connections?

Removing the default DNS servers from ubuntu server 20.04

Posted: 06 Feb 2022 07:18 AM PST

I have an ubuntu 20.04 server and I want to change its DNS settings. The server is using netplan and cloud-init.

I disabled cloud-init network configuraion by creating /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg file with the following content:

network: {config: disabled}

There is an automatically generated /etc/netplan/50-cloud-init.yaml with the following content (I've added the nameservers section by myself):

# This file is generated from information provided by the datasource.  Changes  # to it will not persist across an instance reboot.  To disable cloud-init's  # network configuration capabilities, write a file  # /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:  # network: {config: disabled}  network:      version: 2      ethernets:          ens3:              dhcp4: true              match:                  macaddress: fa:16:3e:d9:e5:96              mtu: 1458              set-name: ens3              addresses: [192.168.0.48/24]              gateway4: 192.168.0.1              nameservers:                  addresses: [178.22.122.100, 185.51.200.2]

And Just to make sure nothing overrides these settings, I created a copy of the 50-cloud-init.yaml file, named 60-change-dns.yaml in the /etc/netplan/ directory with the following content:

network:      version: 2      ethernets:          ens3:              dhcp4: true              match:                  macaddress: fa:16:3e:d9:e5:96              mtu: 1458              set-name: ens3              addresses: [192.168.0.48/24]              gateway4: 192.168.0.1              nameservers:                  addresses: [178.22.122.100, 185.51.200.2]

Then I ran sudo netplan apply command and everything seemed to be working fine. To check if the name servers have changed, I executed sudo resolvectl status and here is the output (the interesting part):

Link 2 (ens3)        Current Scopes: DNS             DefaultRoute setting: yes                    LLMNR setting: yes             MulticastDNS setting: no                DNSOverTLS setting: no                    DNSSEC setting: no                  DNSSEC supported: no                Current DNS Server: 178.22.122.100           DNS Servers: 178.22.122.100                        185.51.200.2                          8.8.8.8                               8.8.4.4                   DNS Domain: openstacklocal

What the hell are 8.8.8.8 and 8.8.4.4 in the DNS server list? They were there by default and my purpose was not to add two more DNS servers on top of them. I wanted to completely replace them with my own custom DNS servers.

How do I remove 8.8.8.8 and 8.8.4.4 from the DNS server list?

VLAN support with libvirt for linux bridge to virtual machines

Posted: 06 Feb 2022 05:50 AM PST

I'm using systemd-networkd to configure network interfaces managed by libvirt for KVM (Kernel-based Virtual Machine) with Debian Bullseye on all nodes. I want to have transparent VLAN support on the virtual machines using the Linux Bridge. For the Linux Bridge this isn't supported by libvirt.

For example I have a virtual machine with three interfaces attached to the bridge with:

host ~$ virsh attach-interface guest-vm bridge br0 --config  host ~$ virsh attach-interface guest-vm bridge br0 --config  host ~$ virsh attach-interface guest-vm bridge br0 --config

Now when running the guest I will see on the bridge:

host ~$ sudo bridge link  3: enp1s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master br0 state forwarding priority 32 cost 4  30: vnet13: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master br0 state forwarding priority 32 cost 100  31: vnet14: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master br0 state forwarding priority 32 cost 100  32: vnet15: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master br0 state forwarding priority 32 cost 100

All interfaces are successfully attached to the bridge. enp1s0 is the uplink interface on the host. Now when looking at the VLAN ids on the bridge I see:

~$ sudo bridge vlan  port              vlan-id  enp1s0            10                    26                    30                    50

Only the host interface is shown with its VLAN ids.

Is there a way to also attach VLAN ids to the other interfaces vnet* of the guest so that it can use them?

Is it possible to configure Windows "resource exhaustion detector" to detect and log high cpu events, rather than just low memory events?

Posted: 06 Feb 2022 08:06 AM PST

Is it possible to configure Windows "resource exhaustion detector" feature to detect and log high cpu events, rather than just low memory events?

As some may know, if Windows detects a low memory condition, the "resource exhaustion detector" mechanism will detect that and report it in the Windows Event Logs (System), as eventid 2004, and with a source of "resource-exhaustion-detector".

Sadly, it seems only to fire on a low memory condition. I would love it to fire also on an extended high CPU condition. But I have not found any resources to suggest it's possible. Does anyone know?

Motivation:

If you may wonder why I seek this, something nice (and simple) about that event log line is it lists what processes are using the most memory at the moment of that alert, as in:

"Windows successfully diagnosed a low virtual memory condition. The following programs consumed the most virtual memory: bob.exe (9412) consumed 1253019648 bytes, jane.exe (4828) consumed 430903296 bytes, and bill.exe (2228) consumed 395128832 bytes." [Emphasis is mine, and the process "names have been change to protect the innocent", as the old Dragnet TV show used to say.]

I would love is to get a similar list of offending processes when the system is experiencing high and prolonged excessive CPU use.

Given the generic name of "resource exhaustion detector" (and not just "low memory detector"), I was hopeful I may find something on this, but so far my google fu is failing me.

I've also not found anything discussing how that memory limit is set (at what size, and for what duration). That would be interesting to know as well, if anyone may find more on configuring this detector.

(As for the secondary intent here, of knowing when CPU is high and knowing what processes are the cause, I will say that Perfmon and its data collector sets do NOT seem the solution. Those require that you identify the specific process(es) to watch. I am seeking with this something that fires even if I may not think to have some tool watching some particular process.)

Why am I missing /var/run/sshd after every boot?

Posted: 06 Feb 2022 04:31 AM PST

I'm running a Ubuntu 16.04 container under Proxmox 5.2-11. After applying the latest round of patches1 I'm unable to login at the console or over ssh.

I mounted the container root FS on the hypervisor and added pts/0 to /etc/security/access.conf (we run pam_access) and that allowed root login to the console. We have root : lxc/tty0 lxc/tty1 lxc/tty2 in access.conf which I thought was sufficient so why I needed pts/0 now is puzzling.

I noticed ssh was not running so tried starting it by hand (/usr/sbin/sshd -DDD -f /etc/ssh/sshd_config) and received this error:

Missing privilege separation directory: /var/run/sshd

I created the directory by hand, started ssh and was able to finally login, but after a reboot, the problem persists. The directory is not being created. Only useful bits in journalctl and the only interesting part is something about "operation not permitted" but no further info.

I'm not too familiar with 16.04 so wondering how I can find out more about the problem. I have no /var/log/syslog or /var/log/messages only kern.log so kind of lost.

1

systemd-sysv 229-4ubuntu21.9  libpam-systemd 229-4ubuntu21.9  libsystemd0 229-4ubuntu21.9  systemd 229-4ubuntu21.9  udev 229-4ubuntu21.9  libudev1 229-4ubuntu21.9  iproute2 4.3.0-1ubuntu3.16.04.4  libsasl2-modules-db 2.1.26.dfsg1-14ubuntu0.1  libsasl2-2 2.1.26.dfsg1-14ubuntu0.1  ldap-utils 2.4.42dfsg-2ubuntu3.4  libldap-2.4-2 2.4.42dfsg-2ubuntu3.4  libsasl2-modules 2.1.26.dfsg1-14ubuntu0.1  libgs9-common 9.25dfsg1-0ubuntu0.16.04.3  ghostscript 9.25dfsg1-0ubuntu0.16.04.3  libgs9 9.25dfsg1-0ubuntu0.16.04.3

[2]

Nov 27 10:13:48 host16 systemd[1]: Starting OpenBSD Secure Shell server...  Nov 27 10:13:48 host16 sshd[474]: Missing privilege separation directory: /var/run/sshd  Nov 27 10:13:48 host16 systemd[1]: ssh.service: Control process exited, code=exited status=255  Nov 27 10:13:48 host16 systemd[1]: Failed to start OpenBSD Secure Shell server.  Nov 27 10:13:48 host16 systemd[1]: ssh.service: Unit entered failed state.  Nov 27 10:13:48 host16 systemd[1]: ssh.service: Failed with result 'exit-code'.  Nov 27 10:13:48 host16 mysqld_safe[495]: Starting mysqld daemon with databases from /var/lib/mysql/mysql  Nov 27 10:13:48 host16 mysqld[500]: 181127 10:13:48 [Note] /usr/sbin/mysqld (mysqld 10.0.36-MariaDB-0ubuntu0.16.04.1) starting as process 499 ...  Nov 27 10:13:48 host16 systemd[1]: ssh.service: Service hold-off time over, scheduling restart.  Nov 27 10:13:48 host16 systemd[1]: Stopped OpenBSD Secure Shell server.  Nov 27 10:13:48 host16 systemd[1]: Failed to reset devices.list on /system.slice/ssh.service: Operation not permitted  Nov 27 10:13:48 host16 systemd[1]: Starting OpenBSD Secure Shell server...  Nov 27 10:13:48 host16 sshd[502]: Missing privilege separation directory: /var/run/sshd  Nov 27 10:13:48 host16 systemd[1]: ssh.service: Control process exited, code=exited status=255  Nov 27 10:13:48 host16 systemd[1]: Failed to start OpenBSD Secure Shell server.  Nov 27 10:13:48 host16 systemd[1]: ssh.service: Unit entered failed state.  Nov 27 10:13:48 host16 systemd[1]: ssh.service: Failed with result 'exit-code'.  Nov 27 10:13:48 host16 systemd[1]: ssh.service: Service hold-off time over, scheduling restart.  Nov 27 10:13:48 host16 systemd[1]: Stopped OpenBSD Secure Shell server.  Nov 27 10:13:48 host16 systemd[1]: Failed to reset devices.list on /system.slice/ssh.service: Operation not permitted  Nov 27 10:13:48 host16 systemd[1]: Starting OpenBSD Secure Shell server...  Nov 27 10:13:48 host16 sshd[503]: Missing privilege separation directory: /var/run/sshd  Nov 27 10:13:48 host16 systemd[1]: ssh.service: Control process exited, code=exited status=255  Nov 27 10:13:48 host16 systemd[1]: Failed to start OpenBSD Secure Shell server.  Nov 27 10:13:48 host16 systemd[1]: ssh.service: Unit entered failed state.  Nov 27 10:13:48 host16 systemd[1]: ssh.service: Failed with result 'exit-code'.  Nov 27 10:13:48 host16 systemd[1]: ssh.service: Service hold-off time over, scheduling restart.  Nov 27 10:13:48 host16 systemd[1]: Stopped OpenBSD Secure Shell server.  Nov 27 10:13:48 host16 systemd[1]: Failed to reset devices.list on /system.slice/ssh.service: Operation not permitted  Nov 27 10:13:48 host16 systemd[1]: Starting OpenBSD Secure Shell server...  Nov 27 10:13:48 host16 sshd[504]: Missing privilege separation directory: /var/run/sshd  Nov 27 10:13:48 host16 systemd[1]: ssh.service: Control process exited, code=exited status=255  Nov 27 10:13:48 host16 systemd[1]: Failed to start OpenBSD Secure Shell server.  Nov 27 10:13:48 host16 systemd[1]: ssh.service: Unit entered failed state.  Nov 27 10:13:48 host16 systemd[1]: ssh.service: Failed with result 'exit-code'.  Nov 27 10:13:49 host16 systemd[1]: ssh.service: Service hold-off time over, scheduling restart.  Nov 27 10:13:49 host16 systemd[1]: Stopped OpenBSD Secure Shell server.  Nov 27 10:13:49 host16 systemd[1]: ssh.service: Start request repeated too quickly.  Nov 27 10:13:49 host16 systemd[1]: Failed to start OpenBSD Secure Shell server.  Nov 27 10:13:49 host16 systemd[1]: ssh.service: Unit entered failed state.  Nov 27 10:13:49 host16 systemd[1]: ssh.service: Failed with result 'start-limit-hit'.  Nov 27 10:13:49 host16 systemd[1]: Started /etc/rc.local Compatibility.  Nov 27 10:13:49 host16 systemd[1]: Failed to reset devices.list on /system.slice/plymouth-quit.service: Operation not permitted  Nov 27 10:13:49 host16 systemd[1]: Starting Terminate Plymouth Boot Screen...  Nov 27 10:13:49 host16 systemd[1]: Failed to reset devices.list on /system.slice/plymouth-quit-wait.service: Operation not permitted  Nov 27 10:13:49 host16 systemd[1]: Starting Hold until boot process finishes up...  Nov 27 10:13:49 host16 systemd[1]: Failed to reset devices.list on /system.slice/rc-local.service: Operation not permitted  Nov 27 10:13:49 host16 systemd[1]: Started Hold until boot process finishes up.  Nov 27 10:13:49 host16 systemd[1]: Started Container Getty on /dev/pts/1.  Nov 27 10:13:49 host16 systemd[1]: Started Container Getty on /dev/pts/0.  Nov 27 10:13:49 host16 systemd[1]: Failed to reset devices.list on /system.slice/console-getty.service: Operation not permitted  Nov 27 10:13:49 host16 systemd[1]: Started Console Getty.  Nov 27 10:13:49 host16 systemd[1]: Reached target Login Prompts.  Nov 27 10:13:49 host16 systemd[1]: Started Terminate Plymouth Boot Screen.  Nov 27 10:13:52 host16 nslcd[338]: accepting connections  Nov 27 10:13:52 host16 nslcd[275]:    ...done.  Nov 27 10:13:52 host16 systemd[1]: Started LSB: LDAP connection daemon.  Nov 27 10:13:52 host16 systemd[1]: Failed to reset devices.list on /system.slice/cron.service: Operation not permitted  Nov 27 10:13:52 host16 systemd[1]: Started Regular background program processing daemon.  Nov 27 10:13:52 host16 systemd[1]: Failed to reset devices.list on /system.slice/atd.service: Operation not permitted

Added systemd-tmpfiles --create output

Really bizarre.... I checked /tmp and those files don't exist enter image description here

After VPN into work network - get error on many sites “NET::ERR_CERT_AUTHORITY_INVALID”

Posted: 06 Feb 2022 07:03 AM PST

After VPNing into a work network, I can no longer access many sites that use https such as StackOverflow, Slack, Workflowy, many more. I get this message:

Your connection is not private Attackers might be trying to steal your information from stackoverflow.com (for example, passwords, messages, or credit cards). Learn more NET::ERR_CERT_AUTHORITY_INVALID

I'm using Chrome but the same thing happens in Firefox.

Some sites allow me to bypass this message with the Advanced button, but most do not.

However, Gmail works! As do all google sites (google.com, youtube.com, etc). And outlook.Microsoft.com. And Amazon.

What could be causing this? Any workarounds? My laptop is not part of the domain. HTTP sites work fine.

Cannot connect to exchange server over internet on computer but can connect via mobile

Posted: 06 Feb 2022 08:06 AM PST

I'm trying to understand how this is possible. This is my situation

Inside the company (LAN environment)

  • Outlook 2010,2013,2016 can connect successfully to exchange server. No problem here

Outside the company (Internet environment)

  • Outlook 2010,2013,2016 CANNOT connect to exchange server.
  • Mobile device using Outlook App, Gmail App, phone default mail app, etc CAN connect to exchange server. (when setting up email choose Exchange and type in the server address)
  • If I VPN to office, then Outlook 2010,2013,2016, can connect to exchange server

The Exchange server is 2010

Extra info on Outlook 2016. If you try to add new email account when outside the company. The autodiscover failed to retrieve the configuration. If you're inside the company, the autodiscover works fine.

I've tried to use autodiscover.xml file + adding entry to registry. The auto discover works (able to get configuration), but failed at last step (Logging on to the mail server) with this error message "The connection to Microsoft Exchange is unavailable. Outlook must be online or connected to complete this action."

My question are why mobile devices works without any problem? and how can I make Outlook works without modifying anything on server side.

Thank you, any help are greatly appreciated.

Preventing an HP DL server from automatically booting - what are the scenarios?

Posted: 06 Feb 2022 04:04 AM PST

We are currently working across our environment and disabling all ways that an HP server can automatically reboot. This is in response to a massive outage which caused our servers to begin flapping, causing a service outage for several million customers. The request from "on high" is to have the servers shut down, but not reboot until a human can manually guide them back online when the "coast is clear" (we have several geographically redundant sites).

So far, I have identified the following possible causes:

  1. HP ASR automatically reboots a host. This can be disabled by switching off the ASR timer.
  2. Disable automatic-power-on in iLO. I believe this is only triggered when power is removed, and then re-applied to the host.

However, I assume there is yet another configuration that is applied when one of the server sensors passes a critical threshold, for example if the ambient temperature sensor exceeds 40 degrees C. That should absolutely shut down a host, but I'm unsure where the configuration lies to disable the automatic reboot after the ambient temperature drops. Or is this also controlled by HP ASR?

I just want to ensure that there aren't any scenarios that I have forgotten that could bite us in the butt in production.

Any help would be appreciated.

How can I get Azure to use an SSH key when doing a node.js deploy via git?

Posted: 06 Feb 2022 10:00 AM PST

I'm deploying a node.js app to Azure Web Services using git from my local machine via: git push azure master

One of our npm packages is a private repo hosted on BitBucket. Without Azure having an SSH key to clone that repo, the deployment fails.

How can I add an SSH key to my Azure server that will get used during these deployment executions?

How to set BasicAuth everywhere EXCEPT certain directories

Posted: 06 Feb 2022 06:04 AM PST

When it comes to setting BasicAuth protection for specific directory, I use simple setup (in apache2.conf file):

<Directory /var/www/somedir/>  Deny from all  AuthUserFile /var/pswd/somedir/.htpasswd  AuthName authorization  AuthType Basic  Satisfy Any  require valid-user  </Directory>

But I would like to set BasicAuth everywhere on the server by default, and unlock it only for specific directories (websites, domains).

So, how could one set BasicAuth everywhere on the server except certain directories?

How to enable LDAP over SSL/TLS in AD without installing AD Certificate Services

Posted: 06 Feb 2022 06:05 AM PST

I am installing a Sonicwall firewall into my organization. I've connected the Sonicwall with the Active Directory domain, however now on the status page of the appliance there is a huge warning:

WARNING: LDAP is being used without TLS - this is highly insecure.

I understand that connection between the FW and the DC is made with clear text and although this is not much of a problem because the Sonicwall and the Domain Controllers are in the local network and in the same subnet, we still want to encrypt the traffic to comply with our regulations.

As I made my search on other forums people are mentioning that I need to apply a certificate to the Domain Controller as per this MS article which is also mentioning the installation of AD Certificate services.

Is there any other way to do encrypt the LDAP traffic without installation of the additional role (AD CS) on the Domain Controller? Installing additional role to the Domain Controller, just for one simple task seems like an overkill to me - like nailing a needle with a sledgehammer.

Also If I am really to install and deploy a Certification Authority to our organization what would be the impact on it? I don't have experience working with it, so are there any implications and/or problems for which I am to be aware of?

Audit Logs for Environment Path Changes

Posted: 06 Feb 2022 09:04 AM PST

Something (or someone) keeps changing our environment path variables on several of our servers. I have the Audit Policy (under local sec pol) set to Success, Failure for all but Process Tracking (which is just failure). However, when changes are made to our environment path, I'm not able to find where the audit log of the change is being logged.

Can someone point me in the direction where I would find the path environment variable change log (or how to enable auditing for environment path changes, if not already)?

I'm on Windows Server 2012 Standard.

Cannot get web root to be /var/www/html, despite setting it in apache2.conf and 000-default.conf

Posted: 06 Feb 2022 05:02 AM PST

new to Linux and trying to set up a basic web server. I'm currently a bit confused, as the document root when you visit the server in a browser appears to be /var/www/.

In both apache2.conf and 000-default.conf the DocumentRoot is set to /var/www/html, and I have restarted the apache2 service numerous times with no luck. I'm unsure as to what could be causing this - I have installed mod_security, but I don't think that should have any effect.

For reference, current apache2.conf and 000-default.conf (I know some values are insanely high, will sort it once I can get everything running).

Server IP: http://167.114.71.100/

As expected from apache2.conf, this gives a 403 forbidden. 167.114.71.100/html does work, however. Any ideas how I can make the default root 167.114.71.100/html?

Thanks!

apache2.conf:

# This is the main Apache server configuration file.  It contains the  # configuration directives that give the server its instructions.  # See  for detailed information about  # the directives and /usr/share/doc/apache2/README.Debian about Debian specific  # hints.  #  #  # Summary of how the Apache 2 configuration works in Debian:  # The Apache 2 web server configuration in Debian is quite different to  # upstream's suggested way to configure the web server. This is because Debian's  # default Apache2 installation attempts to make adding and removing modules,  # virtual hosts, and extra configuration directives as flexible as possible, in  # order to make automating the changes and administering the server as easy as  # possible.    # It is split into several files forming the configuration hierarchy outlined  # below, all located in the /etc/apache2/ directory:  #  #   /etc/apache2/  #   |-- apache2.conf  #   |   `--  ports.conf  #   |-- mods-enabled  #   |   |-- *.load  #   |   `-- *.conf  #   |-- conf-enabled  #   |   `-- *.conf  #   `-- sites-enabled  #       `-- *.conf  #  #  # * apache2.conf is the main configuration file (this file). It puts the pieces  #   together by including all remaining configuration files when starting up the  #   web server.  #  # * ports.conf is always included from the main configuration file. It is  #   supposed to determine listening ports for incoming connections which can be  #   customized anytime.  #  # * Configuration files in the mods-enabled/, conf-enabled/ and sites-enabled/  #   directories contain particular configuration snippets which manage modules,  #   global configuration fragments, or virtual host configurations,  #   respectively.  #  #   They are activated by symlinking available configuration files from their  #   respective *-available/ counterparts. These should be managed by using our  #   helpers a2enmod/a2dismod, a2ensite/a2dissite and a2enconf/a2disconf. See  #   their respective man pages for detailed information.  #  # * The binary is called apache2. Due to the use of environment variables, in  #   the default configuration, apache2 needs to be started/stopped with  #   /etc/init.d/apache2 or apache2ctl. Calling /usr/bin/apache2 directly will not  #   work with the default configuration.    Include /etc/phpmyadmin/apache.conf    # Global configuration  #    #  # ServerRoot: The top of the directory tree under which the server's  # configuration, error, and log files are kept.  #  # NOTE!  If you intend to place this on an NFS (or otherwise network)  # mounted filesystem then please read the Mutex documentation (available  # at <URL:http://httpd.apache.org/docs/2.4/mod/core.html#mutex>);  # you will save yourself a lot of trouble.  #  # Do NOT add a slash at the end of the directory path.  #  #ServerRoot "/etc/apache2"    #  # The accept serialization lock file MUST BE STORED ON A LOCAL DISK.  #  Mutex file:${APACHE_LOCK_DIR} default    #  # PidFile: The file in which the server should record its process  # identification number when it starts.  # This needs to be set in /etc/apache2/envvars  #  PidFile ${APACHE_PID_FILE}    #  # Timeout: The number of seconds before receives and sends time out.  #  Timeout 300    #  # KeepAlive: Whether or not to allow persistent connections (more than  # one request per connection). Set to "Off" to deactivate.  #  KeepAlive On    #  # MaxKeepAliveRequests: The maximum number of requests to allow  # during a persistent connection. Set to 0 to allow an unlimited amount.  # We recommend you leave this number high, for maximum performance.  #  MaxKeepAliveRequests 100    #  # KeepAliveTimeout: Number of seconds to wait for the next request from the  # same client on the same connection.  #  KeepAliveTimeout 5      # These need to be set in /etc/apache2/envvars  User ${APACHE_RUN_USER}  Group ${APACHE_RUN_GROUP}    #  # HostnameLookups: Log the names of clients or just their IP addresses  # e.g., www.apache.org (on) or 204.62.129.132 (off).  # The default is off because it'd be overall better for the net if people  # had to knowingly turn this feature on, since enabling it means that  # each client request will result in AT LEAST one lookup request to the  # nameserver.  #  HostnameLookups Off    # ErrorLog: The location of the error log file.  # If you do not specify an ErrorLog directive within a <VirtualHost>  # container, error messages relating to that virtual host will be  # logged here.  If you *do* define an error logfile for a <VirtualHost>  # container, that host's errors will be logged there and not here.  #  ErrorLog ${APACHE_LOG_DIR}/error.log    #  # LogLevel: Control the severity of messages logged to the error_log.  # Available values: trace8, ..., trace1, debug, info, notice, warn,  # error, crit, alert, emerg.  # It is also possible to configure the log level for particular modules, e.g.  # "LogLevel info ssl:warn"  #  LogLevel warn    # Include module configuration:  IncludeOptional mods-enabled/*.load  IncludeOptional mods-enabled/*.conf    # Include list of ports to listen on  Include ports.conf      # Sets the default security model of the Apache2 HTTPD server. It does  # not allow access to the root filesystem outside of /usr/share and /var/www.  # The former is used by web applications packaged in Debian,  # the latter may be used for local directories served by the web server. If  # your system is serving content from a sub-directory in /srv you must allow  # access here, or in any related virtual host.  <Directory />      Options FollowSymLinks      AllowOverride None      Require all denied  </Directory>    <Directory /usr/share>      AllowOverride None      Require all granted  </Directory>    <Directory /var/www/html>      Options Indexes FollowSymLinks      AllowOverride None      Require all granted  </Directory>    #<Directory /srv/>  #   Options Indexes FollowSymLinks  #   AllowOverride None  #   Require all granted  #</Directory>          # AccessFileName: The name of the file to look for in each directory  # for additional configuration directives.  See also the AllowOverride  # directive.  #  AccessFileName .htaccess    #  # The following lines prevent .htaccess and .htpasswd files from being  # viewed by Web clients.  #  <FilesMatch "^\.ht">      Require all denied  </FilesMatch>      #  # The following directives define some format nicknames for use with  # a CustomLog directive.  #  # These deviate from the Common Log Format definitions in that they use %O  # (the actual bytes sent including headers) instead of %b (the size of the  # requested file), because the latter makes it impossible to detect partial  # requests.  #  # Note that the use of %{X-Forwarded-For}i instead of %h is not recommended.  # Use mod_remoteip instead.  #  LogFormat "%v:%p %h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" vhost_combined  LogFormat "%h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" combined  LogFormat "%h %l %u %t \"%r\" %>s %O" common  LogFormat "%{Referer}i -> %U" referer  LogFormat "%{User-agent}i" agent    # Include of directories ignores editors' and dpkg's backup files,  # see README.Debian for details.    # Include generic snippets of statements  IncludeOptional conf-enabled/*.conf    # Include the virtual host configurations:  IncludeOptional sites-enabled/*.conf

000-default.conf:

<VirtualHost *:80>      # The ServerName directive sets the request scheme, hostname and port that      # the server uses to identify itself. This is used when creating      # redirection URLs. In the context of virtual hosts, the ServerName      # specifies what hostname must appear in the request's Host: header to      # match this virtual host. For the default virtual host (this file) this      # value is not decisive as it is used as a last resort host regardless.      # However, you must set it for any further virtual host explicitly.      #ServerName www.example.com        ServerAdmin webmaster@localhost      DocumentRoot /var/www/html        # Available loglevels: trace8, ..., trace1, debug, info, notice, warn,      # error, crit, alert, emerg.      # It is also possible to configure the loglevel for particular      # modules, e.g.      #LogLevel info ssl:warn        ErrorLog ${APACHE_LOG_DIR}/error.log      CustomLog ${APACHE_LOG_DIR}/access.log combined        # For most configuration files from conf-available/, which are      # enabled or disabled at a global level, it is possible to      # include a line for only one particular virtual host. For example the      # following line enables the CGI configuration for this host only      # after it has been globally disabled with "a2disconf".      #Include conf-available/serve-cgi-bin.conf  </VirtualHost>

virtual hosts with Apache 2.4 windows

Posted: 06 Feb 2022 09:04 AM PST

Trying to upgrade the PHP version from 5.3 to 5.5

Environment: Windows 2012 Server, Apache 2.4, PHP 5.5,

Upgrading from PHP 5.3 and Apache 2.2 for *MySQL and *Drupal

I was able to get the Apache2.4/PHP 5.5 config to work using the base document root. I am able to navigate to 'http://sitetest' and call index.html or a created phpinfo.php with correct php version.

Issue is getting virtual hosts to work that are sub folders of the base document root. sitetest/sitenet sitetest/siteorg get

Internal Server Error

The server encountered an internal error or misconfiguration and was unable to complete your request.

From httpd.conf

DocumentRoot "C:/sitefolder/web"  <Directory "C:/sitefolder/web">      Options FollowSymLinks      AllowOverride All      Require all granted  </Directory>    AccessFileName .htaccess  ...  # Virtual hosts  Include conf/extra/httpd-vhosts.conf  ...  LoadModule php5_module "C:/php/php55/php5apache2_4.dll"  PHPIniDir "C:/php/php55"  AddType application/x-httpd-php .php

from the httpd-vhosts.conf

<VirtualHost *:80>      DocumentRoot  C:/sitefolder/web      ServerName sitetest      ServerAdmin admin@site.local  <Directory "C:/sitefolder/web">      Options FollowSymLinks      AllowOverride All      Require all granted  </Directory>  </VirtualHost>    NameVirtualHost *:80  <VirtualHost *:80>      DocumentRoot "C:/sitefolder/web/sitenet"      ServerName sitenet      ServerAdmin admin@site.local      ErrorLog "C:/sitefolder/web/sitenet/logs/error.log"   <Directory "C:/sitefolder/web/sitenet">      Options FollowSymLinks      AllowOverride All      Require all granted  </Directory>          </VirtualHost>    <VirtualHost *:80>      DocumentRoot  "C:/sitefolder/web/siteorg"      ServerName siteorg      ServerAdmin admin@site.local      ErrorLog "C:/sitefolder/web/siteorg/logs/error.log"   <Directory "C:/sitefolder/web/siteorg">      Options FollowSymLinks      AllowOverride All      Require all granted  </Directory>      </VirtualHost>

It doesnt matter if I try to read another easy HTML or php page in either of the sites. I tried not defining the directories for the virtual hosts and I still get the internal error.

I have read several different documents for virtual hosting and was able to get the Apache2.2 with 5.3 to work, but have run into a standstill here.

Thanks in advance.

Change Block Tracking settings not retained for a VM on VMWare 5.5

Posted: 06 Feb 2022 07:03 AM PST

I'm trying to enable Change Block Tracking as per this VMWare KB article but the setting ctkEnabled is not being retained in the advanced options or being added to the vmx file.
The setting scsi0:0.ctkEnabled is being retained and I see a file vmname-ctk.vmdk in the datastore. I've verified that the vm doesn't have any snapshots but don't see any other suggestions in the KB (or google, of course.)

Edit:

The last comment on this community post. It's a slightly different circumstance, but suggests that there is an known bug editing these settings in the web-gui.

IIS7 SSL certificate caching

Posted: 06 Feb 2022 06:05 AM PST

I changed certificate in IIS 7, changed bindings on websites, everything OK. But when I'm trying to set my old certificate (and I'm changed bindings too), I still get new certificate instead of old. I deleted cache, certificates from browser, opened from another machines, but there is no effect. When I open website from machine, where is it runned, I can see, that there is my old certificate, so IIS7 changed certificate... But where the certificate can be cached? P.S. To prevent misunderstanding - I want to return to my OLD certificate.

Hyper-V 2012 gracefully shutdown when UPS battery running out

Posted: 06 Feb 2022 04:52 AM PST

Background:

We're running a couple of Virtual servers in Hyper-V at a small office using Windows 2008 R2. We only have 1 physical host (and we aren't planning to change that any time soon). We're looking to move to Hyper-V Server 2012.

Currently we have a small UPS that keeps the server running for about 20 minutes and then sends a shutdown message to Windows 2008. The UPS Connects via USB. Windows 2008 recognises the UPS as a battery and there is also software from Eaton that we use to monitor the battery. When Windows shuts down it also shuts down the VM's properly.

We don't want the overhead of running a full version of Server 2012 just so we can have the UPS shut the server down, but we don't want Active Directory and Exchange to suddenly power off either.

Can we make Hyper-V Server 2012 shutdown the VM's gracefully when the UPS runs out of power?

How to findout which key was being used to login for an SSH session

Posted: 06 Feb 2022 06:01 AM PST

I would like to know which key was used when logging into an SSH session. I wan to correlate the key to a local database and send email to an address which corresponds to the key.

The first step is to find which key is the one being used.

User directive in nginx generates error despite running as UID root

Posted: 06 Feb 2022 05:02 AM PST

I'm running nginx on a MacOS X machine, installed with brew, and when I launch nginx, even with sudo, I get the following warning in my log file over and over again:

4/21/11 2:03:42 AM  org.nginx[3788] nginx: [warn] the "user" directive  makes sense only if the master process runs with super-user privileges,  ignored in /usr/local/etc/nginx/conf/nginx.conf:2

From nginx.conf:

user  jschuur staff;

I'm already launching nginx with sudo, since I want the thing to listen on port 80. Shouldn't that be enough to give it the proper super user privileges?

The nginx binary as it's installed:

jschuur@Glenna:sbin → master ls -la  total 4544  drwxr-xr-x   3 jschuur  staff      102 Apr 12 20:53 .  drwxrwxr-x  15 jschuur  staff      510 Apr 12 15:25 ..  -rwxr-xr-x   1 jschuur  staff  2325648 Apr 12 20:39 nginx

FWIW, I recompiled the binary to set passenger up and moved it around from it's original location into /usr/local/sbin.

Update: As it turns out MacOS X was restarting nginx after I'd stopped it, because the launchd plist in ~/Library/LaunchAgents had set it to 'KeepAlive'. However, because I installed this plist into my local user's LaunchAgents folder as opposed to /Library/LaunchAgents (or better yet /Library/LaunchDaemons, which run before you even log on), it wasn't executed as root. Because of an error about not having permissions to use port 80, it actually exited right away, but still wrote to the same log file as the nginx process I started with sudo. I had thought the errors stemming from the automatic restart were actually coming from my manual restart via sudo.

So, bottom line, problem solved. The real problem here was the homebrew instructions specifically asking you to install the plist file into an area that wouldn't allow a local site to use port 80.

single sign-on integrating SVN

Posted: 06 Feb 2022 10:00 AM PST

I need to authenticate my windows users on to a Linux Server which will act as a primary authentication source. Users need to be authenticated and use their access to run SVN or Mercurial ( with something like Tortoise SVN client), or some versioning system. The versioning system need to be authenticated against the Linux Server's authentication source, and users need to use their Windows login username and password to server. I'd have attempted to do this normally on Samba. But is there a better choice? Also how do you create a roaming profile? That is anyone should be able to access their SVN from any PC as long as they use their right Windows username and password

Posted by at

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)