Monday, May 24, 2021

Recent Questions - Server Fault

Recent Questions - Server Fault


GCP internal http(s) load balancer timeout after 300s

Posted: 24 May 2021 09:57 PM PDT

Recently, I've a problem with the internal http(s) load balancer on GCP, about the timeout of backend service (an instance group). After 300 seconds, the API calling to LB will be failed with 408 HTTP response.

I have an internal HTTPS load balancer (LB) on GCP. First, I call a quick API through LB, that worked normally. Then I set timeout for backend service to 10 seconds then call the slow API (say, 500 seconds to complete the request). The error response happens after 10 seconds as expected. But when I set the timeout to 1000 seconds and call the slow API, i receive the error timeout response after only exactly 300 seconds. I also increate the Connection draining timeout to 1000 seconds, but it still doesn't work.

Is the any parameter i need to set to allow API with reply time more than 300 seconds? Thank you.

patroni.exceptions.PatroniFatalException: 'Failed to bootstrap cluster'

Posted: 24 May 2021 09:47 PM PDT

I have a problem with patroni right here

[postgres@localhost ~]$ patroni /etc/patroni.yml  2021-05-25 11:39:56,767 INFO: Selected new etcd server http://10.102.196.138:2379  2021-05-25 11:39:56,772 INFO: No PostgreSQL configuration items changed, nothing to reload.  2021-05-25 11:39:56,831 INFO: Lock owner: None; I am etcd1  2021-05-25 11:39:56,836 INFO: trying to bootstrap a new cluster  The files belonging to this database system will be owned by user "postgres".  This user must also own the server process.    The database cluster will be initialized with locale "en_US.UTF-8".  The default text search configuration will be set to "english".    Data page checksums are enabled.    creating directory /data/patroni ... initdb: error: could not create directory "/data/patroni": Permission denied  pg_ctl: database system initialization failed  2021-05-25 11:39:56,877 INFO: removing initialize key after failed attempt to bootstrap the cluster  Traceback (most recent call last):    File "/usr/local/bin/patroni", line 8, in <module>      sys.exit(main())    File "/usr/local/lib/python3.6/site-packages/patroni/__init__.py", line 170, in main      return patroni_main()    File "/usr/local/lib/python3.6/site-packages/patroni/__init__.py", line 138, in patroni_main      abstract_main(Patroni, schema)    File "/usr/local/lib/python3.6/site-packages/patroni/daemon.py", line 100, in abstract_main      controller.run()    File "/usr/local/lib/python3.6/site-packages/patroni/__init__.py", line 108, in run      super(Patroni, self).run()    File "/usr/local/lib/python3.6/site-packages/patroni/daemon.py", line 59, in run      self._run_cycle()    File "/usr/local/lib/python3.6/site-packages/patroni/__init__.py", line 111, in _run_cycle      logger.info(self.ha.run_cycle())    File "/usr/local/lib/python3.6/site-packages/patroni/ha.py", line 1457, in run_cycle      info = self._run_cycle()    File "/usr/local/lib/python3.6/site-packages/patroni/ha.py", line 1351, in _run_cycle      return self.post_bootstrap()    File "/usr/local/lib/python3.6/site-packages/patroni/ha.py", line 1247, in post_bootstrap      self.cancel_initialization()    File "/usr/local/lib/python3.6/site-packages/patroni/ha.py", line 1240, in cancel_initialization      raise PatroniFatalException('Failed to bootstrap cluster')  patroni.exceptions.PatroniFatalException: 'Failed to bootstrap cluster'  

Here is my patroni.yml

scope: postgres  namespace: /db/  name: etcd1    restapi:      listen: 10.102.196.138:8008      connect_address: 10.102.196.138:8008    etcd:      hosts: 10.102.196.138:2379,10.102.196.139:2379,10.102.196.147:2379    bootstrap:      dcs:          ttl: 30          loop_wait: 10          retry_timeout: 10          maximum_lag_on_failover: 1048576          postgresql:              use_pg_rewind: true        initdb:      - encoding: UTF8      - data-checksums        pg_hba:      - host replication all 127.0.0.1/32 md5      - host replication all 10.102.196.138/0 md5      - host replication all 10.102.196.139/0 md5      - host all all 0.0.0.0/0 md5        users:          admin:              password: admin              options:                  - createrole                  - createdb   postgresql:      listen: 10.102.196.138:5432      connect_address: 10.102.196.138:5432      data_dir: /data/patroni      pgpass: /tmp/pgpass      authentication:          replication:              username: replicator              password: password          superuser:              username: postgres              password: password      parameters:          unix_socket_directories: '.'    tags:      nofailover: false      noloadbalance: false      clonefrom: false      nosync: false    [postgres@localhost ~]$ ls /data/  patroni_2021-05-25-11-03-59  

Can someone help?

Mounting an ocfs2 partition (with multipath removed)

Posted: 24 May 2021 08:37 PM PDT

(I previously posted this on AskUbuntu, but I've since deleted that since no one replied and I now think it's server-related and probably more suitable for here.)

I look after a set of servers running Ubuntu 20.04 which are attached to a disk array that has been formatted as an ocfs2 filesystem. I have been encountering a strange problem. The disk array has two network connections out to a router, so I have been using multipath on the servers. But, for two programs (at least) which output to both standard error and to a file, gibberish (i.e., non-ASCII characters) appear in standard error. The output file itself is fine (as far as I know). But this problem concerns me a bit...

When the same programs with the same inputs write to a externally mounted ext4 filesystem, this does not occur. As far as I can tell, the main difference between the two is the use of multipath. So...to test this "theory", I'm now removing multipath on one server.

(Indeed, if this doesn't solve the problem, then I will need to try something else.)

I disabled multipath by:

  1. Unmounted the array.
  2. Removed the multipath-tools package.
  3. Removed the entry in /etc/fstab.
  4. Rebooted.

When I restarted, I saw this in lsblk:

sdb      8:16   0  36.4T  0 disk   ├─sdb1   8:17   0  34.5T  0 part /data  └─sdb2   8:18   0   1.9T  0 part /home  sdc      8:32   0    53T  0 disk   sde      8:64   0    53T  0 disk   

I guess I have never seen this before. sdb is an ext4 partition and it is fine. But how come the disk array appears as just disk. Underneath it, I don't see partitions like sdc1.

In the /dev/ directory, I see this:

$ ls -al sdb*  brw-rw---- 1 root disk 8, 16 May 24 12:10 sdb  brw-rw---- 1 root disk 8, 17 May 24 12:10 sdb1  brw-rw---- 1 root disk 8, 18 May 24 12:10 sdb2  $ ls -al sdc*  brw-rw---- 1 root disk 8, 32 May 24 12:10 sdc  

Does something have to be done on either the disk array or on the server? I was thinking of just trying mount.ocfs2 but I think I better seek opinions first. Without multipath, I'd probably just use one of the network connections now. i.e., sdc.

I tried the partprobe command and ran it with no changes. I then ran partprobe -s and it gave this output:

/dev/sdb: gpt partitions 1 2  /dev/sdc: msdos partitions  

Does this mean I made a mistake in the very beginning and this disk array should have been made as a gpt partition? Is that why I don't see part? But multipath works with "msdos partitions"?

What are my options? The partition currently has user data, so I'm hesitant to just trying to mount without asking for opinions. It's only partially used, so I can probably find space to move files around to make room, if it is necessary.

yaml.parser.ParserError: while parsing a block mapping

Posted: 24 May 2021 08:40 PM PDT

I followed this document while configuring patroni and I have a problem with patroni.service file like this https://snapshooter.com/learn/postgresql/postgresql-cluster-patroni

[Unit]  Description=Runners to orchestrate a high-availability PostgreSQL  After=syslog.target network.target    [Service]  Type=simple    User=postgres  Group=postgres    ExecStart=/usr/local/bin/patroni /etc/patroni.yml    KillMode=process    TimeoutSec=30    Restart=no    [Install]  WantedBy=multi-user.targ  

Here is my patroni.yml

scope: postgres  namespace: /db/  name: etcd1    restapi:      listen: 10.102.196.138:8008      connect_address: 10.102.196.138:8008    etcd:      hosts: 10.102.196.138:2379,10.102.196.139:2379,10.102.196.147:2379    bootstrap:      dcs:          ttl: 30          loop_wait: 10          retry_timeout: 10          maximum_lag_on_failover: 1048576          postgresql:              use_pg_rewind: true        initdb:      - encoding: UTF8      - data-checksums        pg_hba:      - host replication all 127.0.0.1/32 md5      - host replication all 10.102.196.138/0 md5      - host replication all 10.102.196.139/0 md5      - host all all 0.0.0.0/0 md5        users:          admin:              password: admin              options:                  - createrole                  - createdb   postgresql:      listen: 10.102.196.138:5432      connect_address: 10.102.196.138:5432      data_dir: /data/patroni      pgpass: /tmp/pgpass      authentication:          replication:              username: replicator              password: password          superuser:              username: postgres              password: password      parameters:          unix_socket_directories: '.'    tags:      nofailover: false      noloadbalance: false      clonefrom: false      nosync: false  

And I got this error:

[root@localhost ~]# sudo systemctl status patroni -l  ● patroni.service - Runners to orchestrate a high-availability PostgreSQL     Loaded: loaded (/etc/systemd/system/patroni.service; disabled; vendor preset: disabled)     Active: failed (Result: exit-code) since Mon 2021-05-24 23:24:26 +07; 7s ago    Process: 3307 ExecStart=/usr/local/bin/patroni /etc/patroni.yml (code=exited, status=1/FAILURE)   Main PID: 3307 (code=exited, status=1/FAILURE)    May 24 23:24:26 localhost.localdomain patroni[3307]: self.current_event = self.state()  May 24 23:24:26 localhost.localdomain patroni[3307]: File "/usr/local/lib64/python3.6/site-packages/yaml/parser.py", line 439, in parse_block_mapping_key  May 24 23:24:26 localhost.localdomain patroni[3307]: "expected <block end>, but found %r" % token.id, token.start_mark)  May 24 23:24:26 localhost.localdomain patroni[3307]: yaml.parser.ParserError: while parsing a block mapping  May 24 23:24:26 localhost.localdomain patroni[3307]: in "/etc/patroni.yml", line 1, column 1  May 24 23:24:26 localhost.localdomain patroni[3307]: expected <block end>, but found '<block mapping start>'  May 24 23:24:26 localhost.localdomain patroni[3307]: in "/etc/patroni.yml", line 38, column 2  May 24 23:24:26 localhost.localdomain systemd[1]: patroni.service: main process exited, code=exited, status=1/FAILURE  May 24 23:24:26 localhost.localdomain systemd[1]: Unit patroni.service entered failed state.  May 24 23:24:26 localhost.localdomain systemd[1]: patroni.service failed.  

Downgrade from 2019 standard to 2019 essentials

Posted: 24 May 2021 06:05 PM PDT

We have a physical server in a datacenter. It is running Server 2016 Standard. We created a virtual server with Hyper-V as 2019 Essentials. The datacenter staff authenticated the VM using one of the 2 free virtual machines allowed with a standard license. Somehow, during the authentication, the version was upgraded to 2019 standard. Is there any way other than setting up another VM as essentials and have the datacenter transfer the license? We would rather use essentials for the 25 cals that come with it.

I have found various conflicting articles and I was hoping someone on this forum would have encountered this or a similar situation.

What triggers a redeploy in a multi container azure app service?

Posted: 24 May 2021 05:43 PM PDT

I have an azure app service configured to be multi-container like so:

version: '3.7'    services:    ui:      image: myacr.azurecr.io/angular-ui      restart: always      ports:        - 4200:80    survey-service:      image: myacr.azurecr.io/function-service      restart: always    

And I have continuous deployment enabled.

The two containers are pushed to the acr via github actions. It SEEMS like pushing to the ACR triggers the containers to get re pulled and recreated. I just can't find where the documentation says that.

So I need to understand:

  • What does the webhook do?
  • Is enabling continuous deployment what did it?
  • Does this only work for ACRs or for public docker repos too?
  • What permissions if any do I need to set in the ACR?
  • Where are the docs for this?

Apache log search keywords are not listed in referrer

Posted: 24 May 2021 05:06 PM PDT

Search keywords/phrases are not appearing in the referrer link in the Apache log.
Is there an additional Logformat or other settings needed, what am I missing? Running Apache 2.4 on Ubuntu

Here's the log format in the apache2.conf (default) :

LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined  LogFormat "%h %l %u %t \"%r\" %>s %b" common  LogFormat "%{Referer}i -> %U" referer  LogFormat "%{User-agent}i" agent  CustomLog /var/log/apache2/access.log combined  

To test the referrer capture, I entered a keyword to search on bing.com and google.com, the web site was found and listed, and I clicked the links. I expect to see the search keyword in the referrer URL. In the access.log I only see the referrer but search keywords or parameters are missing in the URL:

....  "GET / HTTP/1.1" 200 11868 "https://www.bing.com/" "Mozilla/5.0 (Linux; Android 9; Mi 9 SE) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.210 Mobile Safari/537.36"  "GET /solutions/ HTTP/1.1" 200 7569 "http://www.google.com/" "Mozilla/5.0 (Linux; Android 9; Mi 9 SE) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.210 Mobile Safari/537.36"  ....  

How to install PHP-intl on Centos PHP 7.2

Posted: 24 May 2021 04:26 PM PDT

My PHP 7.2 application is giving the error Message: Class 'NumberFormatter' not found

We are running:

  • PHP 7.2.34
  • CentOS Linux release 7.9.2009 (Core)

I tried to install PHP-intl and it seemed to work as now if I try again I get this:

# yum install php72-php-intl  # Package php72-php-intl-7.2.34-4.el7.remi.x86_64 already installed and latest version  # Nothing to do  

I have reloaded Apache but the extension is not loaded:

  • php -m does not list the module.
  • /usr/lib64/php/modules does not contain intl.so

How do I get from having php72-php-intl-7.2.34-4.el7.remi.x86_64 to having the extension actually installed and enabled in PHP?

Taking input on remote vm

Posted: 24 May 2021 03:30 PM PDT

below is my use case , I have to take a input from a user for a specific command which need to run on a remote machine.But my problem is I am not able to access that machine directly so what I am doing

Ssh to othere server and from their I am doing ssh to that server. But at that server we are not able to take any inout from user.

Below is a sample script:-

ssh user@machine1 /bin/bash <<\EOF1    ssh user@machine2 /bin/bash <<\EOF2    pwd  echo " Enter input  :"    read -r input    curl -XPOST -k -H "Content-type: application/json" -d '{ "LOGIN_URL": "https://xyz/login","LOGIN_USER": "user11@gmail.com", "LOGIN_PASSWORD": "xyz", "CID": "'"$input"'" }' 'https://abc/gettoken' > sample    scp sample.txt root@user1:/root  

It does not ask for user input and dirctly jump to scp command. Also is this a correct way to pass a variable in a curl command

Any help will be appreciated!

SSH Tunnel - Performant alternatives for exposing a local port to remote machines?

Posted: 24 May 2021 03:50 PM PDT

I have Server A running at home in a Carrier-Grade NAT environment. Due to this I cannot open a port directly in my router. I also have Server B on a cloud hosting provider, which has a static public IP address that can have ports opened to. Both machines run Ubuntu linux.

So far the best solution for me was SSH Tunneling, but it comes with a performance drawback known as TCP-over-TCP.

I have tried several solutions and each one of them had some problems:

  1. OpenVPN/Wireguard: I do not want to route Server A's entire traffic to a remote server, only expose a few ports (while keeping them accessible from the LAN, too)
  2. ngrok: Centralised service, has several limitations (like max connections, etc)
  3. LocalTunnel: HTTP(S) only. I want to forward TCP traffic.
  4. SSH Tunnel: discussed above
  5. Most other solutions: Centralised (non-selfhostable) or HTTP(S) only

I also need the original IP address of the client accessing the port through Server B to be visible to Server A. What software should I try?

Create load balancer on GCP (GAE). SSL cert domains stuck in FAILED state

Posted: 24 May 2021 09:58 PM PDT

I'm new to this topic. We're using GCP (App Engine, standard) to host one nodejs application. However, for different reasons we decided to create two services - stage and default (think as the same app running in parallel).

Default one is connected with custom domain (GAE provided SSL cert) and working properly. The stage service can be accessed with google generated URL (stage-dot-example.appspot.com) and obviously protected with ssl certificates.

Then, we had to go through security review from our partners and we used stage for this.

The result is we have to disable of TLSv1.0 and TLSv1.1. With GAE - we need to create Load Balancer and switch SSL policies to the TLS specific.

The problem: to create External HTTPS load balancer - you have to create SSL Certificate resource (i.e. you have to own domain). With custom domain I guess it should not be hard but how do I do this for stage? Do I use my stage domain (...appspot.com) in SSL Certificate resource? If so - what do I do with DNS records and external IP (you need to switch IP to external IP in A and AAAA records)?

Or if I'm doing something wrong - could you point me to the right direction?

UPDATE + UPDATE 2

I decided to go to the path proposed by Wojtek_B. So I verified stage.example.com and it worked fine without Load Balancer.

At that point, my DNS Records include 4 A and 4 AAAA records from @ with google provided IPs, and 3 CNAME records (www, stage, www.stage) pointing to "ghs.googlehosted.com."

Next, I created SSL certificate resource with 4 domains: example.com, www.example.com, stage.example.com, www.stage.example.com.

Then I added an External HTTPS Load Balancer (with external IP, for example, 1.2.3.4 and SSL cert mentioned above).

I added new A records for @, www, stage, and www.stage to point to 1.2.3.4. I've dropped CNAME records because they are excessive.

After waiting for 2-3 hours (TTL is 1/2 hour) all subdomains were activated except for example.com (stuck in FAILED_NOT_VISIBLE).

ANSWER

I've been fighting managed SSL certificate getting stuck in provisioning state for a while. I followed this tutorial where you're supposed to create external IP (v4) only. But I also had 4 AAAA records (got those during domain verification) with (obviously) ipv6. So I tried to reserve external IP (v6) and it took less than minute to push all 4 (sub)domains to the active state.

In just a few minutes both services through LB were up and running with required TLS configs.

Why isn't ssh-agent a background process by default?

Posted: 24 May 2021 04:07 PM PDT

I have always wondered why such a basic feature (loading ssh keys for persistent usage) requires a clunky command to execute in the background of a cmdline. Why isn't ssh-agent a service (for example) by default? I assume there might be a security reason, but I'm curious to get other thoughts.

Error in job agent when running a package in SSISDB catalog (SQL Server)

Posted: 24 May 2021 10:05 PM PDT

I created a simple package in visual studio. Then I deployed this package in SSISDB catalog which is on a different server. I am using Job agent to execute this package with a proxy account. But I am getting this error in Job Agent:

"Excuted as user: *\test**. Microsoft (R) SQL Server execute Package Utility Version 13.0.5153.0 for 64-bit Copyright 2016 Microsoft. All rights reserved. Failed to execute IS server package because of error 0x80131904. Server: ****, Package path: ****.dtsx , Environment reference Id: NULL. Description: Login failed for user: '*\test'**. Source: .Net SqlClient Data Provider. .... The package execution failed. The step failed."

Kindly help me with identifying this issue.

Problems with DNS and IPv6 on Server 2012 and 2016

Posted: 24 May 2021 08:03 PM PDT

I manage hundreds of servers for many customers. Most are SMB segment, having 1 to 3 servers per customer max. In past few weeks I get more and more frequent DNS errors on random domain controllers, from 2008R2 to 2016. Simply put, DC does not resolve DNS anymore. This happened on some dozen of servers lately, and I haven't figured out the cause yet.

Weird is, that for example, on same premises, 2 VMs, 2 domain controllers for 2 different companies, each with 15 users. Same ISP, same router, same switch. 1 DC works OK, no problems, while 2nd DC cannot resolve DNS anymore:

On server 1 problem local DNS... but nslookup to 8.8.8.8 works!?:

C:\Users\Administrator>nslookup  Default Server:  UnKnown  Address:  ::1    > www.google.com  Server:  UnKnown  Address:  ::1    DNS request timed out.      timeout was 2 seconds.  DNS request timed out.      timeout was 2 seconds.  *** Request to UnKnown timed-out  > server 8.8.8.8  Default Server:  google-public-dns-a.google.com  Address:  8.8.8.8    > www.google.com  Server:  google-public-dns-a.google.com  Address:  8.8.8.8    Non-authoritative answer:  Name:    www.google.com  Addresses:  2a00:1450:4001:81c::2004            172.217.16.68    >  

On server 2 no problems:

C:\Users\Administrator>nslookup www.google.com  Server:  localhost  Address:  ::1    Non-authoritative answer:  Name:    www.google.com  Addresses:  2a00:1450:4006:802::2004            216.58.206.132  

Both are AD DC in single-domain setup, DNS configured with public forwarders, DC DNS points to itself only. IPv4 and IPv6 enabled on servers, but IPv6 is disabled on router. Did not touch any of those servers for past few months.

Did MS change anything? I do not remember DNS ever before switched to IPv6....why did it switch now? And why it works on one server and not on the other, still they are both the same (actually, same deployment, just configured for 2 different domains).

NAT gateway set up doesn't work

Posted: 24 May 2021 06:01 PM PDT

I have implemented an environment for a Wordpress website. I have a loadbalncer (ALB) that is connected to an EC2 server plus an AutoScaling group(for failover).

As we need to update Wordpress and install some plugins on EC2 server, I need to set up NAT gateway to allow internet connection from wordpress server(ec2) to internet.

I have set up the NAT gateway like the following link:
https://aws.amazon.com/premiumsupport/knowledge-center/nat-gateway-vpc-private-subnet/

However I could not be able to reach the Website (DNS of ALB) afterward.

My Routetable set up looks like below:

                      Route                Subnets  Main Route Table        10.0.0.0/16 local       -                       0.0.0.0/0   natgw    Public Route Table        10.0.0.0/16 local       10.0.10.0/24, 10.0.11.0/24  (ELB subnets)                       0.0.0.0/0   igw    Private route B      10.0.0.0/16 local       10.0.20.0/24,10.0.30.0/24 (EC2 subnets)                       0.0.0.0/0   natgw      Private route C      10.0.0.0/16 local       10.0.21.0/24,10.0.31.0/24 (RDS subnets)                       0.0.0.0/0   natgw  

The Public route is assigned to two public subnets which are connected to the LoadBalancer.

The Private route B,C are assigned to private subnets: Web server()EC2 plus database-RDS.

Any help would be appreciated.

enter image description here

ldapsearch: How do I query pwdLastSet and have it make sense?

Posted: 24 May 2021 10:05 PM PDT

Hi all I have same problem as here: enter link description here, but I need it for ldapserach not in Powershell.

my command:

ldapsearch -LLL -H ldap://<HOST>:<POST> -D 'CN=<CN>' -w <PASSWD> '(samaccountname=<NAME>)' pwdlastset  

output:

dn: <SOME_DN>  pwdLastSet: 131267839972407395  

nomad shows job as dead

Posted: 24 May 2021 09:01 PM PDT

I am new to nomad. We are using nomad in production along with docker, consul, vault, ansible. When I run a nomad job from my local machine, Job registers successfully but show status as dead.

Roadrunner-2:nomad Shailesh$ nomad run -detach -address=http://nomad.example-private.com:4646 production/router.hcl   Job registration successful  Evaluation ID: c9e77cb5-ef69-1c63-e926-f131b331d800  

When I do nomad status

Roadrunner-2:nomad Shailesh$ nomad status  ID        Type     Priority  Status  metrics   system   50        running  registry  service  50        running  router    system   50        dead  

And particularly when I check status of the job it gives error.

Roadrunner-2:nomad Shailesh$ nomad status router  ID          = router  Name        = router  Type        = system  Priority    = 50  Datacenters = us-east-1  Status      = dead  Periodic    = false  Error querying job summary: Unexpected response code: 404 (job not found)  

I would highly appreciate any direction for troubleshooting above. Thanks

HP SmartArray Rebuild Logical Drive without data loss

Posted: 24 May 2021 03:03 PM PDT

First, let me explain our situation:

We had an HP server with a RAID 1 over 2 drives (for the OS) and a RAID 5 over 4 drives (for VM data). The raid recently failed. I say the word failed very loosely because to this day I still don't know what exactly caused it. We thought at first a failed drive was the problem. But, after running the HP Offline ACU the status of the drive originally thought to be failed is indeed showing no errors. However, we have another drive (which originally didn't show any errors) now showing a warning that it might fail soon.

Now, here is my question:

Can we see how the raid was configured before and configure it exactly as is and not lose any data? Or are we going to lose data no matter what? The backups we have are not completely up-to-date so I want to try to restore data back to how it was at all costs. I am just not sure if when I recreate the logical volume (exactly how it was) if my data will still be purged?

Is there a command line utility in ACU Offline that will allow me to recreate a logical volume that will NOT purge data if the GUI will?

Thanks for your advice.

How can I move packages between Spacewalk channels?

Posted: 24 May 2021 07:02 PM PDT

How I can move a set of packages from one channel to another? I haven't found any option in the interface that'd let me do this.

My current workflow is to download the rpms, delete them from the channel (which appears to delete them from all of Spacewalk!), and then re-upload them with rhnpush. For obvious reasons this is clunky as hell. I must be missing something.

Which also makes me ask: How does one remove a package from a channel and not delete it from the entire System?

Adding multiple users from a group to an Active Directory group using Powershell

Posted: 24 May 2021 05:04 PM PDT

I have a powershell script that is supposed to go through a specific ou and store the groups into a variable $groups. Here is the code I use in the script:

$Groups = Get-ADGroup -Properties * -Filter * -SearchBase "OU=GFS-USERS,OU=AFS-OU-Groups,OU=AFS,OU=FA,OU=DEPARTMENTS,DC=ou,DC=ad3,DC=blabla,DC=com" -Server "ou.ad3.blabla.com"  Foreach($G In $Groups)  {      Write-Host $G.Name      Write-Host "-------------"      $G.Members  }  

This step seems to work fine.

In my next part of my script I have it go through each group and attempt to add the users from each group into a group where they should all be combined. The code is as follows:

foreach ($group in $groups)  {      Add-ADGroupMember -Identity "CN=test,OU=AFS-OU-ACLs-EDMS,OU=AFS-OU-Groups,OU=AFS,OU=FA,OU=DEPARTMENTS,DC=ou,DC=ad3,DC=blabla,DC=com" -Members (Get-ADGroupMember $group) -Server "ou.ad3.blabla.com"  }         

When I run the script, it works fine for all users from:

OU=AFS,OU=FA,OU=DEPARTMENTS,DC=ou,DC=ad3,DC=blabla,DC=com  

but for all other users I get the following error:

Add-ADGroupMember : The server is unwilling to process the request  At line:1 char:22  

Does anyone know if this is a permissions issue or if there is something I am doing wrong?

BIRD iBGP - Route not reachable

Posted: 24 May 2021 05:04 PM PDT

i have two machines participating in the DN42 network, a darknet driven by the Chaos Computer Club and others to play around with advanced routing techniques like BGP and stuff.

The machines are connected via an OpenVPN connection and can ping each other. Now, my challenge is: Since i own both machines, i'd like to establish an iBGP connection between them, so that they work in the same AS. I already have successful BGP peerings with other AS, but in this specific case, the propagated routes are marked as unreachable.

If that helps: One of the machine is a debian server, the other one is an OpenWRT router. I am using BIRD to get the BGP connections.

I am quite new to this advanced routing stuff, so i would be glad to receive some help. This is the example BIRD config from one of the machines, the config on the other machine looks similar.

# Configure logging  log syslog { debug, trace, info, remote, warning, error, auth, fatal, bug };    # Override router ID  router id 172.23.211.129;    define myas = 4242421111;    protocol kernel {      scan time 20;           # Scan kernel routing table every 20 seconds      import all;      export where source != RTS_STATIC;  }    protocol device {      scan time 10;           # Scan interfaces every 10 seconds  }    protocol static {      route 172.23.211.128/25 via 172.23.211.129;      #route 172.23.211.0/25 via 172.23.211.1;  }    # Protocol template  template bgp PEERS {      import all;      export all;      #export where source = RTS_STATIC;   }     protocol bgp PEER2 from PEERS  {      description "iBGP to home router";      #direct;      local 172.23.211.129 as myas;      neighbor 172.23.211.1 as myas;   }  

As you can see, i have one /24 assigned to my AS, and i want to split it into two /25 subnets, the first assigned to my home machine, the second assigned to a server in a datacenter. So, now the routes between both machines are imported and exported, but birdc shows them as unreachable, and the kernel routing table shows them without any interface identifier. When i try to reach through the connection to ping some peers of the other machine, the network is unreachable...so, please help me.

nginx proxied responses terminating/truncating

Posted: 24 May 2021 03:03 PM PDT

I have this nginx config on a host (host1):

server {    ...    location /foo {      proxy_pass http://127.0.0.1:8091/;    }  }  

The backend nginx config looks like this:

server {    listen localhost:8091;    root /store;    autoindex on;  }  

The backend is actually on a different host (host2), and due to our firewall config, I need to reverse-tunnel the connections (this may be irrelevant to the question but including for completeness):

ssh -R8091:localhost:8091 host1  

This setup is for serving large-ish files (GBs). The problem is I'm finding that requests are abruptly terminating/truncating short of their full size, always a bit >1GB (e.g. 1081376535, 1082474263, ...). No errors in the logs however, and nothing jumps out from verbose debug logging. There's always a Content-Length in the response too.

After some digging I found that there's a proxy_max_temp_file_size that's 1GB by default. Indeed, from inspecting the FDs of the nginx worker in /proc, the temp file is actually being filled up to exactly 1073741824 bytes, and at a rate much faster than the downstream client is able to receive. Lowering it to 1MB mostly makes the problem go away [1] (as would, I imagine, disabling it with 0 altogether). But why would this be a problem? If there was a timeout, why no error message, why is 1GB the default, and why would the downstream client manage to receive a few additional (varying number of) bytes beyond the 1073741824th byte?

Anyway, just wondering if anyone might have an inkling as to what's up. If it makes a difference, both hosts are Ubuntu 12.04.

[1] I say mostly because the problem is replaced by another one, which is that downloads all stop at exactly 2147484825 bytes, which happens to be 0x80000499, but I haven't done enough debugging to determine if this is between (as I suspect) the frontend server and the client.

Roundcube domain change for a single account

Posted: 24 May 2021 07:02 PM PDT

I have inherited a server that is running some ugly roundcube php interface for mail accounts. I would like to change a domain for specific account to custom domain that I have set up on my DNS. I have added users to system and created aliases in my postfix configuration. In file /usr/local/www/roundcube/config/main.inc.php I have a line like $rcmail_config['default_host'] = 'mail.domain.com'; and this is setting all new accounts to have @mail.domain.com domain in their mails. I also have tried to change this by logging in to this account through roundcube interface and in Settings -> Identies I have E-Mail line that I can't change. I see there is also some mysql database for this roundcube installation. If someone could point me where exactly I should change domain name for specific accounts (not for all accounts, like in the line I quoted above).

Network Services Write/Modify Permissions on Inetpub Folder Potiential security issues

Posted: 24 May 2021 08:03 PM PDT

I have an asp.net web application that runs as a Content Management system for a website.

It was originally on an IIS6 Server. However the web application was recently moved to an II7 Server. After the move, my CMS System no longer worked as I received an error message:

Access to the path 'C:\inetpub\SITEFOLDER\FILENAME' is denied.

Every time I save a page in my CMS system I need to write to an aspx file on the server as well as update the web.sitemap file on the server.

I compared the file permissions between the two servers and noticed that the old IIS6 server had Write/Modify permissions on the inetpub folder for the "Network Services" user that carried down to all the site files on the server.

The new server does not have "Network Services" with Write/Modify permissions set on the inetpub folder. Which is obviously the reason why my CMS system isn't working on the new server.

The solution is, of course, to set the Write/Modify permissions on the Inetpub folder on the new server so that ASP.NET has the proper permissions to write and modify any file within the site.

My question is, is this an exceptionable solution on a production web server. Are there any security holes I am opening up by allowing Network Services to have Write/Modify permissions on the inetpub folder? I know giving the IUSER write/modify permissions on the entire inetpub folder can lead to security issues and you have to be careful what files are given write permissions for the IUSER, I just didn't know if the same rule applied to the "Network Services" user.

Preventing access to apache sites via server ip in browser

Posted: 24 May 2021 09:01 PM PDT

I'm trying to set up an apache webserver to serve multiple websites, and I'd like to be able to block access via the ip address of the server itself.

i.e. if someone was to put in http://84.320.65.4/site-name for example, the site in question would be served. However I only want them to be able to access the site via the domain specified in the sites-enabled configuration.

Where and how do I go about configuring this, or is this a bad idea?

truncated headers from varnish configuration file

Posted: 24 May 2021 04:03 PM PDT

I need help with Varnish, I have a varnish configuration file as default.vcl.

I can see from the output of varnishstat that hit ratio is quite high.

I've also checked varnishtop -i txurl to see what are the requests going to backend.

Now, the problem is in http headers the X-Cache header is missing and the other varnish headers.

From the default.vcl there is an option to delete those headers But I need help on how do I keep those headers in http response from varnish itself.

My default.vcl file

backend default {   .host = "127.0.0.1";   .port = "81"; }    # admin backend with longer timeout values. Set this to the same IP & port as your default server.  backend admin {    .host = "127.0.0.1";    .port = "81";    .first_byte_timeout = 18000s;    .between_bytes_timeout = 18000s;  }  # add your Magento server IP to allow purges from the backend  acl purge {    "localhost";    "127.0.0.1";  }      sub vcl_recv {      if (req.restarts == 0) {          if (req.http.x-forwarded-for) {              set req.http.X-Forwarded-For =              req.http.X-Forwarded-For + ", " + client.ip;          } else {              set req.http.X-Forwarded-For = client.ip;          }      }        if (req.request != "GET" &&        req.request != "HEAD" &&        req.request != "PUT" &&        req.request != "POST" &&        req.request != "TRACE" &&        req.request != "OPTIONS" &&        req.request != "DELETE" &&        req.request != "PURGE") {          /* Non-RFC2616 or CONNECT which is weird. */          return (pipe);      }        # purge request      if (req.request == "PURGE") {          if (!client.ip ~ purge) {              error 405 "Not allowed.";          }          ban("obj.http.X-Purge-Host ~ " + req.http.X-Purge-Host + " && obj.http.X-Purge-URL ~ " + req.http.X-Purge-Regex + " && obj.http.Content-Type ~ " + req.http.X-Purge-Content-Type);          error 200 "Purged.";      }        # switch to admin backend configuration      if (req.http.cookie ~ "adminhtml=") {          set req.backend = admin;      }        # we only deal with GET and HEAD by default      if (req.request != "GET" && req.request != "HEAD") {          return (pass);      }        # normalize url in case of leading HTTP scheme and domain      set req.url = regsub(req.url, "^http[s]?://[^/]+", "");        # static files are always cacheable. remove SSL flag and cookie      if (req.url ~ "^/(media|js|skin)/.*\.(png|jpg|jpeg|gif|css|js|swf|ico)$") {          unset req.http.Https;          unset req.http.Cookie;      }        # not cacheable by default      if (req.http.Authorization || req.http.Https) {          return (pass);      }        # do not cache any page from      # - index files      # - ...  #    if (req.url ~ "^/(index)") {  #        return (pass);  #    }        # Don't cache checkout/customer pages, product compare      if (req.url ~ "^/(index.php/)?(checkout|customer|catalog/cart/product_compare|wishlist)") {          return(pass);      }          # as soon as we have a NO_CACHE cookie pass request      if (req.http.cookie ~ "NO_CACHE=") {          return (pass);      }        # normalize Aceept-Encoding header      # http://varnish.projects.linpro.no/wiki/FAQ/Compression      if (req.http.Accept-Encoding) {          if (req.url ~ "\.(jpg|png|gif|gz|tgz|bz2|tbz|mp3|ogg|swf|flv)$") {              # No point in compressing these              remove req.http.Accept-Encoding;          } elsif (req.http.Accept-Encoding ~ "gzip") {              set req.http.Accept-Encoding = "gzip";          } elsif (req.http.Accept-Encoding ~ "deflate" && req.http.user-agent !~ "MSIE") {              set req.http.Accept-Encoding = "deflate";          } else {              # unkown algorithm              remove req.http.Accept-Encoding;          }      }        # remove Google gclid parameters      set req.url = regsuball(req.url,"\?gclid=[^&]+$",""); # strips when QS = "?gclid=AAA"      set req.url = regsuball(req.url,"\?gclid=[^&]+&","?"); # strips when QS = "?gclid=AAA&foo=bar"      set req.url = regsuball(req.url,"&gclid=[^&]+",""); # strips when QS = "?foo=bar&gclid=AAA" or QS = "?foo=bar&gclid=AAA&bar=baz"        return (lookup);  }    # sub vcl_pipe {  #     # Note that only the first request to the backend will have  #     # X-Forwarded-For set.  If you use X-Forwarded-For and want to  #     # have it set for all requests, make sure to have:  #     # set bereq.http.connection = "close";  #     # here.  It is not set by default as it might break some broken web  #     # applications, like IIS with NTLM authentication.  #     return (pipe);  # }  #  # sub vcl_pass {  #     return (pass);  # }  #  sub vcl_hash {      hash_data(req.url);      if (req.http.host) {          hash_data(req.http.host);      } else {          hash_data(server.ip);      }      if (!(req.url ~ "^/(media|js|skin)/.*\.(png|jpg|jpeg|gif|css|js|swf|ico)$")) {          call design_exception;      }      return (hash);  }    sub vcl_hit {       return (deliver);  }      sub vcl_miss {       return (fetch);  }    sub vcl_fetch {      if (beresp.status == 500) {         set beresp.saintmode = 10s;         return (restart);      }      set beresp.grace = 5m;        # add ban-lurker tags to object      set beresp.http.X-Purge-URL = req.url;      set beresp.http.X-Purge-Host = req.http.host;        if (beresp.status == 200 || beresp.status == 301 || beresp.status == 404) {          if (beresp.http.Content-Type ~ "text/html" || beresp.http.Content-Type ~ "text/xml") {              if ((beresp.http.Set-Cookie ~ "NO_CACHE=") || (beresp.ttl < 1s)) {                  set beresp.ttl = 0s;                  return (hit_for_pass);              }                # marker for vcl_deliver to reset Age:              set beresp.http.magicmarker = "1";                # Don't cache cookies              unset beresp.http.set-cookie;          } else {              # set default TTL value for static content              set beresp.ttl = 4h;          }          return (deliver);      }        return (hit_for_pass);  }    sub vcl_deliver {      # debug info      if (resp.http.X-Cache-Debug) {          if (obj.hits > 0) {              set resp.http.X-Cache = "HIT";              set resp.http.X-Cache-Hits = obj.hits;          } else {             set resp.http.X-Cache = "MISS";          }          set resp.http.X-Cache-Expires = resp.http.Expires;      } else {          # remove Varnish/proxy header          remove resp.http.X-Varnish;          remove resp.http.Via;          remove resp.http.Age;          remove resp.http.X-Purge-URL;          remove resp.http.X-Purge-Host;      }        if (resp.http.magicmarker) {          # Remove the magic marker          unset resp.http.magicmarker;            set resp.http.Cache-Control = "no-store, no-cache, must-revalidate, post-check=0, pre-check=0";          set resp.http.Pragma = "no-cache";          set resp.http.Expires = "Mon, 31 Mar 2008 10:00:00 GMT";          set resp.http.Age = "0";      }  }    # sub vcl_error {  #     set obj.http.Content-Type = "text/html; charset=utf-8";  #     set obj.http.Retry-After = "5";  #     synthetic {"  # <?xml version="1.0" encoding="utf-8"?>  # <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"  #  "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">  # <html>  #   <head>  #     <title>"} + obj.status + " " + obj.response + {"</title>  #   </head>  #   <body>  #     <h1>Error "} + obj.status + " " + obj.response + {"</h1>  #     <p>"} + obj.response + {"</p>  #     <h3>Guru Meditation:</h3>  #     <p>XID: "} + req.xid + {"</p>  #     <hr>  #     <p>Varnish cache server</p>  #   </body>  # </html>  # "};  #     return (deliver);  # }  #  # sub vcl_init {  #   return (ok);  # }  #  # sub vcl_fini {  #   return (ok);  # }    sub design_exception {  }  

Assign multiple IP addresses in NAT mode to the same VM

Posted: 24 May 2021 06:01 PM PDT

I'm aware that VirtualBox assigns the same IP to different VMs (by default 10.0.2.15), however I have a case where I have a VM with a single vNIC, and a Linux namespace inside the VM attempts to get an IP from DHCP, and it gets the same IP from VirtualBox. So both eth0 and the other interface inside my network namespace have the same IP address, which is obviously wrong.

Is there a way to have VirtualBox correctly return different DHCP responses or is it a limitation of VirtualBox that it doesn't correctly handle DHCP requests coming from different source MAC addresses from the same VM?

BAD stripes in controller and windows utility to remap bad blocks

Posted: 24 May 2021 04:03 PM PDT

Ok, here is my story. I have 3 disks raid 5, one of the disks made a few surface errors and I didn't know. I tried to repair the corrupted database table in mysql and the machine froze on write. I rebooted with hard reset. It seems the other 2 disks were writing data.

They had exchanged the bad HD so the physical HD's are ok without problems. The logical array in Adaptec 3405 controller shows bad stripes. CHKDSK /f removed some tmp files that were bad. But I have few files that I can not copy on this drive (I/O error) and I guess they are located on the "bad stripes" section of the disk.

The server is production one. I can not change it at the moment, the disk was changed and the files that do not work have no importance to the server.

Now my question is: The NTFS is behaving like if the disks would have surface errors because Adaptec controller marked them as non readable. I know I can rebuild the array but I can not because it is productions server (I will move data to another server, it will take time). The problem is not critical as the bad sector disk is replaced. The other non readable files that are in bad stripe I don't need.

But I asked adaptec if the new files will not have I/O error because the adapter marks them as bad and they told me: NO. It is a dangerous situation now because at any moment database could be writeen to bad stripe (bad block from the OS perspective).

I would only need a tool that makes surface scan and put all the bad blocks (there are 32 the imaging backup program told me) into one "bad" file and the new files will not be able to write there. The disks won't degrade more as the hard drive errors are not there any more, the faulty disk was swapped.

I know when the disks are failing it is not good to remap bad blocks, but my situation is fixed now, I just need to map bad blocks to some atrificial bad.txt file for example that would reside on the disk so nothing is ever tried to be written there.

I hope I was clear enough. I can not find such software, I've found a surface scanner but it does not make a bad file out of it :-( Ok, I will not delete my bad 3 files but there may be other sectors from the bad stripe in controller and I'd like a new file to be made pointing to this sectors so no more damage can occur in writes. I do not need to read those files at all...

No comments:

Post a Comment