| proxy_cache_bypass keeps serving old cached response if URL changes to non-cacheable Posted: 25 Nov 2021 02:28 PM PST Under normal circumstances, when using proxy_cache_bypass nginx will fetch a fresh copy from upstream, and overwrite the cached response with the new one. But if the URL changes from a cacheable to a non-cacheable response (for example to a 4xx response with Cache-Control: no-cache), then using proxy_cache_bypass will indeed serve a fresh copy from upstream, but it will leave the old copy in the cache. Which means that everytime the URL is requested without triggering proxy_cache_bypass, it will keep serving the old cache. I guess this is an intended behaviour, because proxy_cache_bypass only overwrites cached responses by saving a new one, and a no-cache response means there is nothing to save? Is this what's happening? How can I solve this? I don't want to enable caching for 4xx responses... I'm running nginx/1.14.2 proxy_cache_path /var/cache/nginx levels=1:2 keys_zone=my-cache:70m max_size=28g inactive=1d; proxy_temp_path /var/cache/nginx/tmp; proxy_cache my-cache; proxy_cache_key $remote_user$scheme$host$request_uri; proxy_cache_use_stale error timeout invalid_header http_500 http_502 http_503 http_504 http_429; proxy_cache_bypass $http_cache_control; proxy_read_timeout 90; add_header X-Cache-Status $upstream_cache_status; etag off;
|
| terraform apply error alreadyExists on untouched resources Posted: 25 Nov 2021 01:49 PM PST I am starting a new terraform project, following the official guide: https://learn.hashicorp.com/tutorials/terraform/gke?in=terraform/kubernetes&utm_source=WEBSITE&utm_medium=WEB_IO&utm_offer=ARTICLE_PAGE&utm_content=DOCS&_ga=2.91746777.2118895439.1637849824-960084622.1637849824 I have managed to get it to run. (I am running it as part of a google cloud build task triggered on commit) However, if i change something in a resource (eg i replaced "gke_num_nodes" default from 2 to 1), when i run terraform apply again this is what I get: Plan: 4 to add, 0 to change, 0 to destroy. Changes to Outputs: + kubernetes_cluster_host = (known after apply) + kubernetes_cluster_name = "workspace-auto-gke" + project_id = "workspace-auto" + region = "europe-west4" google_compute_network.vpc: Creating... ╷ │ Error: Error creating Network: googleapi: Error 409: The resource 'projects/workspace-auto/global/networks/workspace-auto-vpc' already exists, alreadyExists │ │ with google_compute_network.vpc, │ on vpc.tf line 15, in resource "google_compute_network" "vpc": │ 15: resource "google_compute_network" "vpc" { │ ╵
Is there a way to get it to not try to recreate existing untouched resources? My cloudbuild.json is as follows: { "steps": [ { "name": "hashicorp/terraform", "entrypoint": "/bin/sh", "args": [ "./cloudbuild/prepare-terraform.sh" ] } ], "logsBucket": "gs://my-bucket/logdir", "serviceAccount": "projects/my-proj/serviceAccounts/my-service-account@my-proj.iam.gserviceaccount.com" }
with prepare-terraform.sh being simply terraform init terraform plan terraform apply -auto-approve
|
| I have a sql server with a pictures table with 25 million entries and it has become too slow to query. currently have a xeon E5-2667v3 with 32g of ram [duplicate] Posted: 25 Nov 2021 01:25 PM PST I have a sql server with a pictures table with 25 million entries and it has become too slow to query i currently have a Xeon E5-2667v3 with 32g of ram, I am looking to get a hp ProLiant Gen 9 and I am not sure how to go with specs, is SQL single or multi-threaded? would I benefit from going many weaker cores or less stronger cores? does ram have a large effect? should I get more ram or faster ram? What about caching? Any help is much appreciated Thank you! |
| NFS mounts folders, but no files Posted: 25 Nov 2021 11:02 AM PST I have a problem getting files to show up using an NFSv4 mount between Server: Ubuntu 20.04, 192.168.1.1 Client: Ubuntu 18.04, 192.168.1.2
On the server, I have a folder and file /home/server/files/myfile.pdf
(note 'server' is being used as the user account name). Here is the full ownership and permissions chain of the original file: drwxr-xr-x root:root / drwxr-xr-x root:root /home/ drwxr-xr-x server:server /home/server/ drwxrwxr-x server:nfsgroup /home/server/files/ -rwxrwxrwx server:nfsgroup /home/server/files/myfile.pdf
Here you can see an example of a utility 'user':'group' I created: nfsuser uid=127 nfsgroup gid=134
Both 'nfsuser' and 'server' are in the 'nfsgroup' group. On the server, I have the /files/ directory bind-mounted to /srv/nfs4/files/ in /etc/fstab, with an entry /home/server/files /srv/nfs4/files none bind 0 0
This is successful, and I can clearly see the /srv/nfs4/files/myfile.pdf file as -rwxrwxrwx server:nfsgroup /srv/nfs4/files/myfile.pdf
Here is the full ownership and permissions chain of the mounted and served file: drwxr-xr-x root:root / drwxr-xr-x root:root /srv/ drwxrwxr-x server:nfsgroup /srv/nfs4/ drwxrwxr-x server:nfsgroup /srv/nfs4/files/ -rwxrwxrwx server:nfsgroup /srv/nfs4/files/myfile.pdf
Now, I want to export the entire /srv/nfs4/ directory, which contains the bind-mounted files/, to the client. I have this entry in the server's /etc/exports: /srv/nfs4 192.168.1.2(rw,sync,fsid=0,root_squash,all_squash,anonuid=127,anongid=134,no_subtree_check)
Note in particular that the all_squash,anonuid=127,anongid=134 group of settings causes the client user 'client' to be recognized as 'nfsuser' in the 'nfsgroup' with regards to permissions. After I export this using $ sudo exportfs -ra, I mount it to the root-level directory /nsffiles/ on the client using the command $ sudo mount -t nfs 192.168.1.1:/srv/nfs4 /nsffiles. The result is that I can see the exported files/ directory within /nsffiles/ on the client. However, I cannot see its contents: $ ls -Alhd /nsffiles/ drwxrwxr-x 8 client 134 4.0K Nov 14 20:37 /nsffiles/ $ ls -Alhd /nsffiles/files/ drwxr-xr-x 2 root root 4.0K Nov 14 20:36 /nsffiles/files/ $ ls -Alh /nsffiles/files/ total 0
The fact that the NFS client can see the files/ directory at all indicates there's nothing fundamentally wrong with my NFS services or the local network. There's no firewall issues, and I shouldn't need to install any additional packages. Also, I previously had the NFS mount working perfectly under a slightly different folder structure (files/ was in a directory outside of home, for example), so I know the basics are fine in that regard. I assume that the most likely cause of why it won't work now is a subtle ownership/permissions change between the previous setup and the current one. In particular, the mounted directory is server side: `/srv/nfs4/` drwxrwxr-x server:nfsgroup client side: `/nsffiles/` drwxrwxr-x client:134
The contained directory is server side: `/srv/nfs4/files/` drwxrwxr-x server:nfsgroup client side: `/nsffiles/files/` drwxr-xr-x root:root
That is, for the mounted directory, the permissions convert as server:nfsgroup -> client:134, which is expected. The permissions for the contained directory, however, convert as server:nfsgroup -> root:root. Why root:root and not client:134? This ServerFault question sounds similar, but it concerns Windows, and I can't tell that the only answer applies. This Unix.SE question concerns NFS file permissions for a Windows/Linux setup; all of the information I can glean from it is things I already know and think I've taken care of. Please respond only if you know what you're talking about. If you can't resist posting a wild guess, please identify it as a wild guess. Or, you know, provide enough information that your reader can tell the difference. |
| Why there are SELinux errors in permissive mode? Posted: 25 Nov 2021 10:36 AM PST I have set CentOS 8 Stream with SELinux set to permissive but I still have bunch of red lines in the log e.g.: SELinux is preventing /usr/lib/systemd/systemd from name_connect access on the tcp_socket port 80
Are these real or its just printing - what would be happening if it was in restrictive mode? |
| MariaDB high %sys load on AMD compared to Intel Posted: 25 Nov 2021 03:39 PM PST we have run into a issue with MariaDB and AMD Epyc 7402. We also have much less potent Intel servers that do not suffer from this problem, using exact same configuration (we are using ansible). Under more than about 15% total cpu load in %usr, there is an unusually high load in %sys.. If anything you need is missing, i can provide whatever is neccessary in form of logs and or configurations.   
For starters: - OS: Centos 7 (5.15.2-1.el7.elrepo.x86)
- MariaDB: 10.4.22-MariaDB
- This server is running as mysql master for total of 8 slaves but only 4 slaves are directly attached to it on second network interface
HW: - AMD EPYC 7402 24-Core Processor 2x
- Memory 528161544 kB divided evenly among sockets
- HDD: INTEL SSDPE2KX010T8 2x (mdraid, lvm and ext4)
What we tried: (single and combinations (where it makes sense)) - jemalloc 3.6.0-1.el7
- jemalloc 5.2.1-1.el7 (compiled ourselves with default and new c++ compilers)
- kernel-lt 5.4.160-1
- kernel-lt 5.4.161-1
- kernel-ml 5.15.2-1.el7.elrepo.x86
- kernel-ml 5.12.1-1.el7.elrepo.x86
- kernel 3.10.0-1160.45.1.el7
- numactl --interleave=all
Configs: [client] port = 3306 [mysql] no_auto_rehash max_allowed_packet = 1024M prompt = '\u@\h [\d]> ' default_character_set = utf8 [mysqld_safe] open_files_limit = 1024 user = mysql log-error = /var/log/mysql/error.log [mysqld] # NETWORKING bind-address = 0.0.0.0 port = 3306 socket = /var/lib/mysql/mysql.sock skip-external-locking max_connections = 3000 max_user_connections = 3000 datadir = /var/lib/mysql # BUFFERS key_buffer_size = 256M max_allowed_packet = 1024M table_open_cache = 256 sort_buffer_size = 1M read_buffer_size = 1M read_rnd_buffer_size = 4M myisam_sort_buffer_size = 64M # QUERY CACHE query_cache_type = 1 query_alloc_block_size = 8192 query_cache_limit = 1024 query_cache_min_res_unit = 512 query_cache_size = 8388608 # STORAGE default_storage_engine = InnoDB innodb_data_home_dir = /var/lib/mysql innodb_data_file_path = ibdata1:10M:autoextend innodb_log_group_home_dir = /var/lib/mysql # MEMORY LIMITS innodb_buffer_pool_size = 425G innodb_log_file_size = 60G innodb_buffer_pool_instances = 64 innodb_log_buffer_size = 8M innodb_lock_wait_timeout = 50 # COLLATION character_set_server = utf8 collation_server = utf8_unicode_ci # MyISAM variables key_buffer_size = 1024M # INNODB TWEAKING innodb_file_format = Barracuda innodb_strict_mode = ON innodb_file_per_table = 1 innodb_thread_concurrency = 0 innodb-flush-method = O_DIRECT # General Tweaks tmp_table_size = 2048M max_heap_table_size = 2048M # Replication server-id = 24 binlog-format = row log_bin = mysql-bin relay-log = mysql-relay-bin log-slave-updates = 1 expire_logs_days = 3 log_bin_trust_function_creators = 1 # LOGGING - ERROR log_error = /var/log/mysql/error.log log_warnings = 2 innodb_print_all_deadlocks = 1 # LOGGING - SLOW LOG slow_query_log_file = /var/log/mysql/slow.log slow_query_log = 1 log_queries_not_using_indexes = 0 long_query_time = 10 min_examined_row_limit = 100 # LOGGING - GENERAL LOG general_log_file = /var/log/mysql/general.log general_log = 0 # REPLICATION slave_compressed_protocol = 1 # THREADING thread_handling = pool-of-threads thread-pool-max-threads = 6000 innodb_flush_log_at_trx_commit = 1 replicate_ignore_db = norp_* [myisamchk] key_buffer_size = 128M sort_buffer_size = 128M read_buffer = 2M write_buffer = 2M [mysqlhotcopy] interactive-timeout [mysqldump] quick max_allowed_packet = 1024M
/etc/sysctl.conf kernel.panic=30 vm.swappiness=10 net.ipv4.ip_local_port_range=10240 65000 net.core.rmem_default=524288 net.core.wmem_default=262144 net.core.rmem_max=134217728 net.core.wmem_max=134217728 net.ipv4.tcp_rmem=4096 87380 134217728 net.ipv4.tcp_wmem=4096 65536 134217728 net.ipv4.tcp_moderate_rcvbuf=1 net.ipv4.tcp_fin_timeout=10 net.core.netdev_max_backlog=300000 net.ipv4.tcp_max_syn_backlog=8192 net.ipv4.tcp_synack_retries=2 net.ipv4.tcp_max_orphans=800000 net.ipv4.tcp_keepalive_intvl=20 net.ipv4.tcp_keepalive_probes=5 net.core.somaxconn=2048 net.core.dev_weight=256 net.core.netdev_tstamp_prequeue=0 net.ipv4.ipfrag_time=5 net.ipv4.tcp_ecn=1 net.ipv4.tcp_max_tw_buckets=20000 net.ipv4.tcp_retries2=7 net.ipv4.tcp_limit_output_bytes=262144 fs.file-max=500000 net.ipv4.ip_forward=1 net.ipv4.ip_nonlocal_bind=1 net.ipv4.conf.eth0.arp_ignore=1 net.ipv4.conf.eth0.arp_announce=2 vm.overcommit_memory=1 net.ipv4.conf.default.rp_filter=0 vm.overcommit_memory = 1
PT-Mysql-Summary # Status Counters (Wait 10 Seconds) ########################## Variable Per day Per second 13 secs Aborted_clients 6 Acl_database_grants 1500 Acl_proxy_users 15 Acl_users 1500 Aria_pagecache_blocks_not_flushed 70000 -16 Aria_pagecache_blocks_unused 2500 -29 Aria_pagecache_blocks_used 100000 1 Aria_pagecache_read_requests 17500000000 225000 200000 Aria_pagecache_reads 100000000 1250 3500 Aria_pagecache_write_requests 1500000000 17500 15000 Aria_pagecache_writes 600000000 7000 7000 Aria_transaction_log_syncs 3000 Binlog_commits 5000000 60 60 Binlog_group_commits 5000000 60 60 Binlog_snapshot_position 5000000000 60000 1750000 Binlog_bytes_written 35000000000 400000 1750000 Binlog_cache_disk_use 45000 1 Binlog_cache_use 5000000 60 60 Binlog_stmt_cache_use 20 Bytes_received 225000000000 2500000 2250000 Bytes_sent 1000000000000 12500000 10000000 Com_admin_commands 9000 Com_begin 70000 1 Com_call_procedure 1250 Com_commit 70000 1 Com_delete 1250000 15 Com_insert 10000000 125 450 Com_insert_select 2500 Com_rollback 600 Com_select 350000000 4000 3500 Com_set_option 60000000 600 700 Com_show_slave_status 9000 Com_show_status 12500 Com_show_tables 600 Com_show_variables 45 Com_update 20000000 225 900 Com_update_multi 70 Connections 30000000 350 350 Created_tmp_disk_tables 7000000 80 70 Created_tmp_files 80000 1 Created_tmp_tables 9000000 100 100 Delete_scan 250 Empty_queries 90000000 1000 1000 Executed_triggers 50000000 600 1000 Feature_check_constraint 6 Feature_json 6 7 Feature_subquery 1500000 20 20 Feature_trigger 6000000 70 125 Flush_commands 6 Handler_commit 400000000 5000 6000 Handler_delete 9000000 100 Handler_discover 250 Handler_icp_attempts 12500000000 150000 60000 Handler_icp_match 12500000000 150000 60000 Handler_prepare 60000000 700 2500 Handler_read_first 200000 2 1 Handler_read_key 30000000000 400000 400000 Handler_read_last 1250000 15 15 Handler_read_next 300000000000 3500000 2250000 Handler_read_prev 17500000000 200000 150000 Handler_read_rnd 12500000000 150000 125000 Handler_read_rnd_deleted 250 Handler_read_rnd_next 30000000000 300000 150000 Handler_rollback 150000 1 2 Handler_tmp_update 125000000 1500 Handler_tmp_write 17500000000 200000 200000 Handler_update 22500000 250 1250 Handler_write 12500000 150 600 Innodb_buffer_pool_bytes_data 500000000000 6000000 4500000 Innodb_buffer_pool_bytes_dirty 125000000000 1250000 1000000 Innodb_buffer_pool_pages_flushed 70000 Innodb_buffer_pool_read_ahead 125000 1 Innodb_buffer_pool_read_requests 300000000000 3500000 3000000 Innodb_buffer_pool_reads 30000000 350 250 Innodb_buffer_pool_write_requests 350000000 4000 7000 Innodb_data_fsyncs 5000000 60 70 Innodb_data_read 500000000000 6000000 4500000 Innodb_data_reads 35000000 400 250 Innodb_data_writes 5000000 60 70 Innodb_data_written 25000000000 300000 600000 Innodb_dblwr_pages_written 70000 Innodb_dblwr_writes 50000 Innodb_log_write_requests 35000000 400 900 Innodb_log_writes 5000000 60 70 Innodb_os_log_fsyncs 5000000 60 70 Innodb_os_log_written 22500000000 250000 600000 Innodb_pages_created 100000 1 3 Innodb_pages_read 35000000 400 250 Innodb_pages_written 70000 Innodb_row_lock_time 175000 1 Innodb_row_lock_waits 2250 Innodb_rows_deleted 5000000 60 Innodb_rows_inserted 4500000 50 100 Innodb_rows_read 400000000000 4500000 3000000 Innodb_rows_updated 22500000 250 1250 Innodb_system_rows_read 6 Innodb_num_open_files 2000 Innodb_available_undo_logs 800 Innodb_secondary_index_triggered_cluster_reads 60000000000 700000 600000 Key_read_requests 1250 Key_reads 400 Memory_used 10000000000 125000 1250000 Memory_used_initial 8000000000 90000 Open_table_definitions 2500 Opened_files 35000000 400 450 Opened_table_definitions 17500 1 Opened_tables 15000000 175 300 Opened_views 250 Qcache_hits 3500000 40 100 Qcache_inserts 250 Qcache_not_cached 600000000 7000 6000 Queries 500000000 6000 7000 Questions 450000000 5000 6000 Rows_read 125000000000 1500000 1250000 Rows_sent 2250000000 25000 15000 Rows_tmp_read 17500000000 200000 175000 Select_full_join 175000 2 2 Select_full_range_join 125000 1 2 Select_range 25000000 300 250 Select_range_check 125 Select_scan 2500000 30 20 Slave_connections 25 Slaves_connected 25 Slow_queries 30000 Sort_merge_passes 40000 Sort_priority_queue_sorts 20000000 225 200 Sort_range 30000000 350 300 Sort_rows 9000000000 100000 90000 Sort_scan 5000000 60 35 Subquery_cache_hit 20000 Subquery_cache_miss 2500000 30 20 Syncs 9000 Table_locks_immediate 50000 2 Table_open_cache_active_instances 6 Table_open_cache_hits 800000000 9000 17500 Table_open_cache_misses 15000000 175 300 Table_open_cache_overflows 12500000 125 300 Threadpool_idle_threads 2500 -3 Threadpool_threads 3000 Threads_created 70000 Update_scan 20 Uptime 90000 1 1 wsrep 1250000000000 15000000 1000000000 # Table cache ################################################ Size | 256 Usage | 100% # InnoDB ##################################################### Version | 10.4.22 Buffer Pool Size | 432.0G Buffer Pool Fill | 20% Buffer Pool Dirty | 3% File Per Table | ON Page Size | 16k Log File Size | 2 * 60.0G = 120.0G Log Buffer Size | 8M Flush Method | O_DIRECT Flush Log At Commit | 1 XA Support | Checksums | ON Doublewrite | ON R/W I/O Threads | 4 4 I/O Capacity | 200 Thread Concurrency | 0 Concurrency Tickets | 5000 Commit Concurrency | 0 Txn Isolation Level | REPEATABLE-READ Adaptive Flushing | ON Adaptive Checkpoint | Checkpoint Age | 2G InnoDB Queue | 0 queries inside InnoDB, 0 queries in queue Oldest Transaction | 0 Seconds History List Len | 49981 Read Views | 41 Undo Log Entries | 0 transactions, 0 total undo, 0 max undo Pending I/O Reads | 0 buf pool reads, 0 normal AIO, 0 ibuf AIO, 0 preads Pending I/O Writes | 0 buf pool (0 LRU, 0 flush list, 0 page); 0 AIO, 0 sync, 0 log IO (0 log, 0 chkp); 0 pwrites Pending I/O Flushes | 0 buf pool, 0 log Transaction States | 41xACTIVE, 148xnot started # MyISAM #####################################################
vmstat -1 procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu----- r b swpd free buff cache si so bi bo in cs us sy id wa st 66 0 0 382036288 258372 23430352 0 0 3872 99095 201601 554506 16 49 35 0 0 60 0 0 382076832 258372 23444996 0 0 3472 4880 193726 557287 17 50 33 0 0 60 0 0 382070048 258372 23430188 0 0 3568 4496 186601 533398 16 51 33 0 0 61 0 0 382029760 258372 23424988 0 0 3472 4220 190556 537991 14 54 32 0 0 75 0 0 382012704 258372 23445704 0 0 3648 4360 179687 527781 11 62 27 0 0 71 0 0 382003648 258372 23469492 0 0 3536 75513 196280 531335 16 64 20 0 0 78 0 0 382000704 258372 23495520 0 0 3552 2384 189606 543385 13 67 20 0 0 85 0 0 381998528 258372 23511724 0 0 3392 3113 232940 531062 14 74 12 0 0 80 0 0 382065728 258372 23474704 0 0 3744 3192 199559 541724 13 70 17 0 0 78 0 0 382070784 258372 23445872 0 0 3692 4314 196798 552624 15 69 17 0 0 75 0 0 382059680 258372 23464400 0 0 3536 2924 197860 511828 13 66 21 0 0 84 0 0 382105184 258372 23489224 0 0 3616 3724 206273 535082 13 73 14 0 0 84 0 0 382134016 258372 23475792 0 0 3472 2293 215286 520198 12 77 11 0 0 84 0 0 382123136 258372 23472120 0 0 3568 3097 205254 525508 13 72 15 0 0 82 0 0 382153728 258372 23494132 0 0 6912 3952 233188 538006 14 72 14 0 0 81 0 0 382128064 258372 23502312 0 0 3616 3284 184822 521364 13 70 17 0 0 75 0 0 382123776 258372 23541176 0 0 3776 2872 207498 536109 12 72 16 0 0 83 0 0 382160096 258372 23540552 0 0 3536 3306 202817 551826 15 68 18 0 0 65 0 0 382205664 258372 23504828 0 0 3648 2692 190065 551570 14 61 25 0 0 62 0 0 382176576 258372 23508872 0 0 4816 4964 186138 560683 12 58 30 0 0 55 0 0 382215264 258372 23448712 0 0 3616 4165 166252 558690 11 46 43 0 0 52 0 0 382219872 258372 23430824 0 0 3584 167357 178906 592052 15 42 41 2 0 47 0 0 382217472 258372 23379732 0 0 6752 2976 178652 547447 15 40 45 0 0 46 0 0 382210624 258372 23358184 0 0 3744 2800 186172 543690 15 38 46 0 0 47 0 0 382180192 258372 23351812 0 0 8896 3976 166141 530729 15 34 51 0 0 51 0 0 382159808 258372 23362736 0 0 3568 2772 171856 559503 18 34 47 0 0 47 0 0 382181024 258372 23364880 0 0 4112 3232 165595 516269 12 39 49 0 0 46 0 0 382229728 258372 23371168 0 0 3616 3447 151896 515363 14 36 49 0 0 52 0 0 382223136 258372 23332020 0 0 4672 2674 180962 556582 14 39 47 0 0 60 0 0 382214336 258372 23347904 0 0 3568 2960 169390 522171 12 46 42 0 0 62 0 0 382228512 258372 23384148 0 0 3696 3686 191964 532110 15 49 36 0 0 65 0 0 382184832 258372 23405920 0 0 3584 4528 183894 541259 15 54 31 0 0 72 0 0 382199488 258372 23394256 0 0 3584 2345 194048 538938 15 60 25 0 0 80 0 0 382209600 258372 23405216 0 0 12128 4376 192327 513727 13 66 21 0 0 70 0 0 382176608 258372 23437752 0 0 3776 2748 216748 533426 15 67 18 0 0 69 0 0 382180928 258372 23458208 0 0 6816 4772 192280 549678 14 62 24 0 0 61 0 0 382196640 258372 23427652 0 0 4016 57509 193263 542982 12 64 24 0 0 70 0 0 382233216 258372 23434848 0 0 3648 89949 191647 541364 12 57 30 0 0 74 0 0 382189856 258372 23442084 0 0 3520 2632 186482 573915 14 60 26 0 0 62 0 0 382155744 258372 23445880 0 0 3888 5436 178857 559573 14 55 30 0 0 73 0 0 382158304 258372 23420096 0 0 3712 3800 180457 551695 15 55 30 0 0 77 0 0 382161760 258372 23402016 0 0 3488 3768 208621 542433 16 62 22 0 0 68 1 0 382149664 258372 23412868 0 0 6704 3777 184064 534328 14 60 26 0 0 69 1 0 382120896 258372 23409916 0 0 3728 4153 178721 518152 13 59 28 0 0 83 0 0 382115968 258372 23472716 0 0 3376 4576 190548 522075 15 69 17 0 0 23 1 0 381906176 258372 23575532 0 0 3792 4595 220719 560955 20 71 10 0 0 86 1 0 382011392 258372 23485620 0 0 3680 5765 217555 560589 20 71 9 0 0 72 1 0 382006240 258372 23484148 0 0 4544 5158 195189 540771 18 67 14 0 0 75 1 0 381983744 258372 23507208 0 0 3520 4404 192868 535709 15 63 22 0 0 72 1 0 381939328 258372 23526672 0 0 3632 4000 179945 545377 15 62 23 0 0 70 1 0 381931392 258372 23511188 0 0 7952 4904 181942 528306 15 60 26 0 0 78 1 0 381927520 258372 23541316 0 0 4032 5773 202670 530372 13 63 23 0 0 79 1 0 381914752 258372 23548560 0 0 3200 3520 202867 536863 14 68 18 0 0
Numa Interleave is on cat /proc/`pidof mysqld`/numa* | grep interleave | wc -l 1430
|
| How to route multiple web apps on several AWS EC2s under 1 domain without using a bunch of Elastic IPs? Posted: 25 Nov 2021 09:11 AM PST Last night I realized that there is a 5 address limit with regard to provisioning elastic IP's. Ive done some initial research, and have come to the conclusion that a bunch of elastic Ip's is probably not the correct answer. I feel confident that I can add startup scripts to update Route 53 dns records at startup to circumvent the need for static ip addresses, but from reading, it seams like this could lead to issues with down time from DNS caching. For things like the phone system, website, and mail server; this would be less than ideal. In your experience, what is the "Right" way to handle a situation like this? For reference, I'm developing the infrastructure for my new small business, and thus far I have the following EC2's: - PBX phone software by the company 3CX. This ec2 is not using an elastic IP. The company 3CX provides me a 3cx domain to use with my desktop/mobile software.
- Samba Server to provide network file sharing. (Elastic IP)
- Online accounting software (Elastic IP as a subdomain by Route 53)
- Wordpress sever main website (Elastic IP as a subdomain by Route 53)
- Softether VPN to provide access to the Samba Server (Elastic IP)
- MailCow Mail Server (Elastic IP as a subdomain by Route 53)
Many thanks in advance! :) |
| How to migrate to Google managed certificates without downtime? Posted: 25 Nov 2021 09:10 AM PST I'm moving example.com from an external (non-Google) hosting provider into GCP. When setting up the load balancer, I noticed that I have to point example.com to the load balancer in order for the Google managed certificate to validate. I'm supposed to just change the A record of example.com to the (static) IP of the new load balancer - then it will validate. The problem is that I already have a lot of traffic to example.com, requests that happen after example.com starts pointing to the load balancer, but before the certificate is validated will generate SSL errors, and very unhappy users. Has anyone solved this? I know there are ways to avoid downtime when rotating certificates, but there must be some way to migrate large sites without downtime? |
| Mysqldump generated file that is smaller size then database Posted: 25 Nov 2021 10:56 AM PST My database is 250 MB and dump size is 82 MB. I am using latest MariaDB. running this command: mysqldump -u user -p database > t.sql
|
| What are required environment variables for GPG? Posted: 25 Nov 2021 10:53 AM PST I'm trying to setup the docker credentials, and it requires to initialize the gpg first. Here are my steps: root@remote_machine:~# gpg --gen-key gpg (GnuPG) 2.2.4; Copyright (C) 2017 Free Software Foundation, Inc. This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Note: Use "gpg --full-generate-key" for a full featured key generation dialog. GnuPG needs to construct a user ID to identify your key. Real name: NeededUser Email address: mymail@mail.com You selected this USER-ID: "NeededUser <mymail@mail.com>" Change (N)ame, (E)mail, or (O)kay/(Q)uit? O We need to generate a lot of random bytes. It is a good idea to perform some other action (type on the keyboard, move the mouse, utilize the disks) during the prime generation; this gives the random number generator a better chance to gain enough entropy. gpg: agent_genkey failed: Required environment variable not set Key generation failed: Required environment variable not set root@remote_machine:~#
I have tried to set up the GPG_TTY variable, and it is set to /dev/pts/0. What variables are necessary for GPG? |
| Attempting to create an RAID array in Ubuntu server 20.10 Posted: 25 Nov 2021 12:22 PM PST I'm attempting to create a RAID 0 RAID array on Ubuntu server 21.10, but I keep getting the following error: mdadm: An option must be given to set the mode before a second device Here is the syntax I used: ~$ sudo mdadm –create –verbose /dev/md0 –level=0 –raid-devices=2 /dev/sdb1 /dev/sdc1 mdadm: An option must be given to set the mode before a second device (–verbose) is listed
I've seen some websites that list the syntax as: sudo mdadm -–create -–verbose /dev/md0 -–level=0 -–raid-devices=2 /dev/sdb1 /dev/sdc1
To which I get: mdadm: invalid option -- '?' Usage: mdadm --help for help
I've tried removing the partitions and attempting the command without partitions, still no joy. I've tried the -c vs the --create option, still no fun. Do I need to install any additional packages in order to create an array? |
| DELL R320, Xeon E5-2450 v1, Oracle Linux 8 marks clocksource 'tsc' as unstable, random crashes under load Posted: 25 Nov 2021 11:35 AM PST I recently acquired used Dell R320 with Xeon E5-2450 v1, all firmware's are updated to most recent versions using Lifecycle controller. On boot dmesg reports: microcode: microcode updated early to revision 0x71a, date = 2020-03-24 [ 12.384040] clocksource: timekeeping watchdog on CPU9: Marking clocksource 'tsc' as unstable because the skew is too large: [ 12.395572] clocksource: 'hpet' wd_now: 3b1bb82 wd_last: 2e247ff mask: ffffffff [ 12.413476] clocksource: 'tsc' cs_now: 1c62267fd4b cs_last: 1c30b8dcf7f mask: ffffffffffffffff [ 12.425567] tsc: Marking TSC unstable due to clocksource watchdog [ 12.431666] TSC found unstable after boot, most likely due to broken BIOS. Use 'tsc=unstable'.
Then if i run phoronix-test-suite stress-run stress-ng system after aprox. one minute become unresponsive. During test i see watchdog events from network adapter: [ 705.412997] NETDEV WATCHDOG: eno1 (tg3): transmit queue 0 timed out [ 705.412997] WARNING: CPU: 9 PID: 6812 at net/sched/sch_generic.c:473 dev_watchdog+0x27d/0x281 [ 705.412997] Modules linked in: xt_CHECKSUM ipt_REJECT nf_nat_tftp nft_objref nf_conntrack_tftp nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct nf_tables_set tun rfkill scsi_transport_iscsi ip_set xt_conntrack xt_multiport xt_nat xt_addrtype xt_mark xt_MASQUERADE nft_counter xt_comment nft_compat nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 veth sunrpc iTCO_wdt intel_rapl_msr iTCO_vendor_support dcdbas intel_rapl_common sb_edac x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel vfat fat kvm irqbypass crct10dif_pclmul crc32_pclmul mgag200 ghash_clmulni_intel drm_vram_helper aesni_intel ttm crypto_simd cryptd glue_helper drm_kms_helper pcspkr drm syscopyarea sysfillrect sysimgblt fb_sys_fops lpc_ich i2c_algo_bit zfs(POE) joydev zunicode(POE) zzstd(OE) zlua(OE) mei_me zavl(POE) mei icp(POE) zcommon(POE) znvpair(POE) ipmi_ssif spl(OE) ioatdma dca ipmi_si ipmi_devintf ipmi_msghandler acpi_power_meter [ 705.412997] sch_fq_codel ip_tables xfs libcrc32c sd_mod sg ahci libahci libata mpt3sas tg3 raid_class scsi_transport_sas wmi fuse [ 705.412997] CPU: 9 PID: 6812 Comm: stress-ng Kdump: loaded Tainted: P OE 5.4.17-2136.300.7.el8uek.x86_64 #2 [ 705.412997] Hardware name: Dell Inc. PowerEdge R320/0KM5PX, BIOS 2.4.2 01/29/2015 [ 705.412997] RIP: 0010:dev_watchdog+0x27d/0x281 [ 705.412997] Code: 48 85 c0 75 e6 eb a0 4c 89 e7 c6 05 9b 59 17 01 01 e8 c7 a9 fa ff 89 d9 4c 89 e6 48 c7 c7 68 3b 53 ac 48 89 c2 e8 be f1 82 ff <0f> 0b eb 82 0f 1f 44 00 00 66 2e 0f 1f 84 00 00 00 00 00 66 66 66 [ 705.412997] RSP: 0000:ffffac6d003d0e50 EFLAGS: 00010282 [ 705.412997] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000006 [ 705.412997] RDX: 0000000000000007 RSI: 0000000000000092 RDI: ffff9e853f457d00 [ 705.412997] RBP: ffffac6d003d0e80 R08: 0000000000000514 R09: 00000000ffffffff [ 705.412997] R10: 0000000000000000 R11: ffff9e851d84f3d0 R12: ffff9e850d8e4000 [ 705.412997] R13: 0000000000000005 R14: ffff9e850d8e4480 R15: ffff9e8537d377c0 [ 705.412997] FS: 00007fa4baba5740(0000) GS:ffff9e853f440000(0000) knlGS:0000000000000000 [ 705.412997] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 705.412997] CR2: 00007f54983fad0c CR3: 0000000b99992006 CR4: 00000000000606e0 [ 705.412997] Call Trace: [ 705.412997] <IRQ> [ 705.412997] ? pfifo_fast_enqueue+0x160/0x151 [ 705.412997] call_timer_fn+0x32/0x12c [ 705.412997] run_timer_softirq+0x1a5/0x42e [ 705.412997] __do_softirq+0xe1/0x2e7 [ 705.412997] ? hrtimer_interrupt+0x12a/0x222 [ 705.412997] irq_exit+0xf3/0xf8 [ 705.412997] smp_apic_timer_interrupt+0x79/0x130 [ 705.412997] apic_timer_interrupt+0xf/0x14 [ 705.412997] </IRQ>
If i add mitigations = off to kernel command-line parameters on boot, phoronix lasts from 4 to 7 minutes and the system again become unresponsive. The same stuff happens with KVM guests, tried to install Debian 11 5 times, install freezes during either initial package install or kernel unpack. screen of freeze messages: https://ibb.co/k2Jk4QG Does anyone had similar issues ? Thanks ! P.S.: current kernel 5.4.17-2136.300.7.el8uek.x86_64, also tried with 4.18.0-305.19.1.el8_4.x86_64 without any difference |
| Kerberos Ticket Hand-off No Good on Chrome on macOS X Posted: 25 Nov 2021 11:00 AM PST I am implementing Okta as a single-sign on provider in an enterprise environment of about 90 users. One of Okta's features is Desktop Single Sign On - the ability for users to be authenticated with Okta simply by virtue of having logged into their machine and thereby authenticating with the domain. The user simply opens a browser, goes to the company's Okta tenant URL, and they are logged in. Without this feature, the user would be prompted for their credentials when loading up the Okta tenant URL. DSSO is accomplished by the browser picking up a Kerberos ticket from the OS that itself is generated when the user authenticates with the Active Directory domain. The browser then hands this ticket back to the server, and the server communicates with the Okta cloud to authenticate the user. The authentication flow in our environment goes like this: - User logs into their machine. A kerberos ticket is generated upon login and authentication with the domain.
- User opens their browser, and either tries to access an Okta-protected/integrated app, or goes directly to their Okta portal.
- User is redirected by Okta to our load balancer, which terminates the request at the IWA web app on the web server
- IWA web app challenges the browser for authentication
- Browser grabs the Kerberos ticket from the OS and hands it to the load balancer, which passes it to the IWA web app
- IWA app validates the ticket and fetches user profile from AD
- IWA app generates and digitally signs an SSO token and sends it to the browser
- Browser returns the tokent to Okta via HTML form POST
- Okta completes the sign-in request and returns the user to the app with an SSO token
The process is failing on step 5, and I know this is the case because: - Chrome prompts the user for NTLM credentials when the Okta tenant URL is requested
- This prompt happens before the IWA web app and the browser is configured properly for DSSO (per the documentation I linked at the beginning)
- The prompt does not happen on Chrome, Firefox, and Internet Explorer on Windows (DSSO works on Windows with Chrome, Firefox, and IE)
- This prompt does not happen in Safari on macOS X, but does happen with Chrome and Firefox in OS X
What I cannot figure out is why Chrome and Firefox are not picking up the Kerberos ticket from the OS in macOS X, but the same browsers in Windows are picking up the ticket without a hitch. Steps I've tried: Setting Chrome's whitelist settings with the following terminal commands (recommended by Okta's documentation): $ defaults write com.google.Chrome AuthServerWhitelist "*.example.com" $ defaults write com.google.Chrome AuthNegotiateDelegateWhitelist "*.example.com" - Setting Chrome's whitelist settings with SimpleMDM configuration push (this method actually succeeded in pushing the settings to Chrome - proven by going to chrome://policy and seeing the settings)
- Uninstalling anti-virus
- Adding every possible FQDN to the list of servers whitelisted in step 2 - first just the servers we have whitelisted in Windows (because Windows actually works), and then a list of Okta servers recommended by Okta support
I still cannot get this feature to work and I am now trying to figure out if there's a way to troubleshoot the process Chrome uses to pickup the Kerberos ticket from the OS. Some kind of debugger for Chrome's Kerberos ticket pickup mechanism would be great, but I imagine no such thing is available. |
| undefined variables in /etc/apache2/apache2.conf Posted: 25 Nov 2021 03:26 PM PST Whenever I run sudo apache2 -S on my server it throws the following error: [Sun Mar 04 16:17:22.303217 2018] [core:warn] [pid 31405] AH00111: Config variable ${APACHE_LOCK_DIR} is not defined [Sun Mar 04 16:17:22.303414 2018] [core:warn] [pid 31405] AH00111: Config variable ${APACHE_PID_FILE} is not defined [Sun Mar 04 16:17:22.303503 2018] [core:warn] [pid 31405] AH00111: Config variable ${APACHE_RUN_USER} is not defined [Sun Mar 04 16:17:22.303577 2018] [core:warn] [pid 31405] AH00111: Config variable ${APACHE_RUN_GROUP} is not defined [Sun Mar 04 16:17:22.303655 2018] [core:warn] [pid 31405] AH00111: Config variable ${APACHE_LOG_DIR} is not defined [Sun Mar 04 16:17:22.306324 2018] [core:warn] [pid 31405:tid 140656836052864] AH00111: Config variable ${APACHE_LOG_DIR} is not defined AH00526: Syntax error on line 74 of /etc/apache2/apache2.conf: Invalid Mutex directory in argument file:${APACHE_LOCK_DIR}
I've tried several things but it keeps giving me these errors... I made some adjustments to apache2.conf but that ended up not working so I reverted my changes, tried downloading the default ennvardidn't work My /etc/apache2/apache2.conf file looks like this: # This is the main Apache server configuration file. It contains the # configuration directives that give the server its instructions. # See http://httpd.apache.org/docs/2.4/ for detailed information about # the directives and /usr/share/doc/apache2/README.Debian about Debian specific # hints. # # # Summary of how the Apache 2 configuration works in Debian: # The Apache 2 web server configuration in Debian is quite different to # upstream's suggested way to configure the web server. This is because Debian's # default Apache2 installation attempts to make adding and removing modules, # virtual hosts, and extra configuration directives as flexible as possible, in # order to make automating the changes and administering the server as easy as # possible. # It is split into several files forming the configuration hierarchy outlined # below, all located in the /etc/apache2/ directory: # # /etc/apache2/ # |-- apache2.conf # | `-- ports.conf # |-- mods-enabled # | |-- *.load # | `-- *.conf # |-- conf-enabled # | `-- *.conf # `-- sites-enabled # `-- *.conf # # # * apache2.conf is the main configuration file (this file). It puts the pieces # together by including all remaining configuration files when starting up the # web server. # # * ports.conf is always included from the main configuration file. It is # supposed to determine listening ports for incoming connections which can be # customized anytime. # # * Configuration files in the mods-enabled/, conf-enabled/ and sites-enabled/ # directories contain particular configuration snippets which manage modules, # global configuration fragments, or virtual host configurations, # respectively. # # They are activated by symlinking available configuration files from their # respective *-available/ counterparts. These should be managed by using our # helpers a2enmod/a2dismod, a2ensite/a2dissite and a2enconf/a2disconf. See # their respective man pages for detailed information. # # * The binary is called apache2. Due to the use of environment variables, in # the default configuration, apache2 needs to be started/stopped with # /etc/init.d/apache2 or apache2ctl. Calling /usr/bin/apache2 directly will not # work with the default configuration. # Global configuration # # # ServerRoot: The top of the directory tree under which the server's # configuration, error, and log files are kept. # # NOTE! If you intend to place this on an NFS (or otherwise network) # mounted filesystem then please read the Mutex documentation (available # at <URL:http://httpd.apache.org/docs/2.4/mod/core.html#mutex>); # you will save yourself a lot of trouble. # # Do NOT add a slash at the end of the directory path. # ServerRoot "/etc/apache2" # # The accept serialization lock file MUST BE STORED ON A LOCAL DISK. # Mutex file:${APACHE_LOCK_DIR} default # # PidFile: The file in which the server should record its process # identification number when it starts. # This needs to be set in /etc/apache2/envvars # PidFile ${APACHE_PID_FILE} # # Timeout: The number of seconds before receives and sends time out. # Timeout 300 # # KeepAlive: Whether or not to allow persistent connections (more than # one request per connection). Set to "Off" to deactivate. # KeepAlive On # # MaxKeepAliveRequests: The maximum number of requests to allow # during a persistent connection. Set to 0 to allow an unlimited amount. # We recommend you leave this number high, for maximum performance. # MaxKeepAliveRequests 100 # # KeepAliveTimeout: Number of seconds to wait for the next request from the # same client on the same connection. # KeepAliveTimeout 5 # These need to be set in /etc/apache2/envvars User ${APACHE_RUN_USER} Group ${APACHE_RUN_GROUP} # # HostnameLookups: Log the names of clients or just their IP addresses # e.g., www.apache.org (on) or 204.62.129.132 (off). # The default is off because it'd be overall better for the net if people # had to knowingly turn this feature on, since enabling it means that # each client request will result in AT LEAST one lookup request to the # nameserver. # HostnameLookups Off # ErrorLog: The location of the error log file. # If you do not specify an ErrorLog directive within a <VirtualHost> # container, error messages relating to that virtual host will be # logged here. If you *do* define an error logfile for a <VirtualHost> # container, that host's errors will be logged there and not here. # ErrorLog ${APACHE_LOG_DIR}/error.log # # LogLevel: Control the severity of messages logged to the error_log. # Available values: trace8, ..., trace1, debug, info, notice, warn, # error, crit, alert, emerg. # It is also possible to configure the log level for particular modules, e.g. # "LogLevel info ssl:warn" # LogLevel warn # Include module configuration: IncludeOptional mods-enabled/*.load IncludeOptional mods-enabled/*.conf # Include list of ports to listen on Include ports.conf # Sets the default security model of the Apache2 HTTPD server. It does # not allow access to the root filesystem outside of /usr/share and /var/www. # The former is used by web applications packaged in Debian, # the latter may be used for local directories served by the web server. If # your system is serving content from a sub-directory in /srv you must allow # access here, or in any related virtual host. <Directory /> Options FollowSymLinks AllowOverride None Require all denied </Directory> <Directory /usr/share> AllowOverride None Require all granted </Directory> <Directory /var/www/> Options Indexes FollowSymLinks AllowOverride None Require all granted </Directory> #<Directory /srv/> # Options Indexes FollowSymLinks # AllowOverride None # Require all granted IncludeOptional sites-enabled/*.conf #</Directory> # AccessFileName: The name of the file to look for in each directory # for additional configuration directives. See also the AllowOverride # directive. # AccessFileName .htaccess # # The following lines prevent .htaccess and .htpasswd files from being # viewed by Web clients. # <FilesMatch "^\.ht"> Require all denied </FilesMatch> # # The following directives define some format nicknames for use with # a CustomLog directive. # # These deviate from the Common Log Format definitions in that they use %O # (the actual bytes sent including headers) instead of %b (the size of the # requested file), because the latter makes it impossible to detect partial # requests. # # Note that the use of %{X-Forwarded-For}i instead of %h is not recommended. # Use mod_remoteip instead. # LogFormat "%v:%p %h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" vhost_combined LogFormat "%h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" combined LogFormat "%h %l %u %t \"%r\" %>s %O" common LogFormat "%{Referer}i -> %U" referer LogFormat "%{User-agent}i" agent # Include of directories ignores editors' and dpkg's backup files, # see README.Debian for details. # Include generic snippets of statements IncludeOptional conf-enabled/*.conf # Include the virtual host configurations: IncludeOptional sites-enabled/*.conf # vim: syntax=apache ts=4 sw=4 sts=4 sr noet
What could be the problem, what am I missing? This post is a continuation of this post: https://stackoverflow.com/questions/49085166/virtualhost-apache-on-ubuntu-server-not-working EDIT I added this to my envvars file: export APACHE_RUN_USER=www-data export APACHE_RUN_GROUP=www-data
This fixed once error but now I still get these errors: [Mon Mar 05 17:52:45.978727 2018] [core:warn] [pid 9150] AH00111: Config variable ${APACHE_PID_FILE} is not defined [Mon Mar 05 17:52:45.978953 2018] [core:warn] [pid 9150] AH00111: Config variable ${APACHE_RUN_USER} is not defined [Mon Mar 05 17:52:45.979024 2018] [core:warn] [pid 9150] AH00111: Config variable ${APACHE_RUN_GROUP} is not defined [Mon Mar 05 17:52:45.979109 2018] [core:warn] [pid 9150] AH00111: Config variable ${APACHE_LOG_DIR} is not defined [Mon Mar 05 17:52:45.982417 2018] [core:warn] [pid 9150:tid 140490114955136] AH00111: Config variable ${APACHE_LOG_DIR} is not defined [Mon Mar 05 17:52:45.982709 2018] [core:warn] [pid 9150:tid 140490114955136] AH00111: Config variable ${APACHE_LOG_DIR} is not defined [Mon Mar 05 17:52:45.982771 2018] [core:warn] [pid 9150:tid 140490114955136] AH00111: Config variable ${APACHE_LOG_DIR} is not defined AH00543: apache2: bad user name ${APACHE_RUN_USER}
|
| Nginx with keepalived Posted: 25 Nov 2021 10:05 AM PST I got 2 nginx servers which are handling 1) reverse proxy and 2) load balancing on two webservers. Since I cannot provide any of the nginx machines IP to the customer so I was thinking to have keepalived on top of 2 Nginx machines with VIP it also checks the fail-over of Nginx. What I have achieved 1) Successfully installed nginx on 2 machines, so when I hit any nginx server IP I am able to see the application webpage. Even I tested the fail over it works fine. 2) I have installed keepalived on both the machines and tried to generate VIP with the following config vrrp_script chk_nginx { script "killall -0 nginx" interval 2 } vrrp_instance VI_1 { interface eth0:1 state MASTER virtual_router_id 51 priority 100 unicast_src_ip server1_ip unicast_peer { server2_ip } virtual_ipaddress { VIP } track_script { chk_nginx } }
--> So here when I hit VIP in the web browser I got following error This site can't be reached VIP refused to connect. Try: Checking the connection Checking the proxy and the firewall Not sure what I am missing here . Can someone help me to resolve this issue .So as per my understanding when I hit VIP I should see application web page ? Regards VG |
| Apache httpd mod_proxy POST issue on home page like abc.com Posted: 25 Nov 2021 03:02 PM PST I have apache server 2.4.18 on centos as a web server in the front end. Now I have multiple websites which are deployed on tomcat 8.0.33 as webapps in the default webapps folder of tomcat. Configuration on apache end: <VirtualHost 1.2.3.4:80> ServerName abc.com ServerAlias www.abc.com ProxyPass / http://localhost:8079/abc/ ProxyPassReverse / http://localhost:8079/abc/ </VirtualHost> <VirtualHost 1.2.3.4:80> ServerName def.com ServerAlias www.def.com ProxyPass / http://localhost:8079/def/ ProxyPassReverse / http://localhost:8079/def/ </VirtualHost>
Two .war files by the name, abc.war & def.war are deployed to tomcat webapps folder. abc.com works absolutely fine, as it is just an html site. but def.com is a dynamic site & it has a login form on home page. The home page after successful login should load user profile page which is: login page url -> def.com user profile page -> def.com/profile.htm Note that the home page loads successfully (means GET is executing successfully) on browser but when user tries to login(which is a POST request), it somehow tries to load: def.com/def/profile.htm Also note that if I change configuration to : <VirtualHost 1.2.3.4:80> ServerName def.com ServerAlias www.def.com ProxyPass /def/ http://localhost:8079/def/ ProxyPassReverse /def/ http://localhost:8079/def/ </VirtualHost>
then I am successfully able to open def.com/def/ & I can also successfully able to login & open def.com/profile.htm but I want my home page to be on def.com |
| Apache .htaccess <IF> statement with multiple matches Posted: 25 Nov 2021 02:33 PM PST I've tried ||, OR without success, so what is the more concise way to write these two matches on the user agent in a single statement? <If "%{HTTP_USER_AGENT} == 'Mozilla/5.0 (Windows NT 5.1; rv:7.0.1) Gecko/20100101 Firefox/7.0.1'"> Require all denied </If> <If "%{HTTP_USER_AGENT} == 'Mozilla/5.0 (Windows NT 6.1; WOW64)'"> Require all denied </If>
|
| Windows server ignoring DNS resolution on private interface Posted: 25 Nov 2021 12:03 PM PST I have a Windows 2012 server with two interfaces - public internet facing, and private. The private interface uses our internal DNS servers. This interface has no gateway. Static IP - not DHCP. I want to disable the public interface, as we use a proxy for outgoing traffic. Having disabled it, I'm finding DNS resolution failing: I cannot ping foo.external.com - host cannot be found However, I can nslookup, notice that it connects to our internal DNS, and successfully resolve foo.external.com ipconfig /displaydns DOES show the correct entry. I'm utterly confused why ping does not simply use this entry. If I manually add an entry to local hosts file, for "1.1.1.1 foo", as an experiment, and try ping foo, it fails - cannot resolve host. Same for foo.external.com, and "foo.external.com.". Pinging "foo.external.com", or "foo.external.com." also fails. If, however, I then re-enable the public interface, it all works. Including the hosts file. I can ping "foo" (the resolution aspect). Disable it, and it stops working again. I've tried all the reboots, ipconfig /flushdns, nbtstat -R stuff. a) Why is ping (and our application) not able to properly resolve things when the public interface is disabled, despite Windows being able to speak to its DNS server, and cache the result (as shown in ipconfig /displaydns). b) Why is the hosts file being ignored for the private interface? Many thanks. |
| Nginx reverse proxy in docker container - wrong IP logged Posted: 25 Nov 2021 12:35 PM PST I am running Nginx in a Docker container as a reverse proxy using the configuration below. It basically redirects the request to another docker container running on the same machine Problem The problem is that the output in the access log states the wrong client IP address, specifically I get the IP of the Docker host 172.17.0.1: 172.17.0.1 - - [24/May/2016:19:50:18 +0000] "GET /admin/ HTTP/1.1" 200 19243 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64; rv:46.0) Gecko/20100101 Firefox/46.0" "-"
I tried to follow the advice from this post regarding the set_real_ip_from directive but without any success. Configuration server { listen 8000 ssl; server_name example.com; access_log /var/log/nginx/host.access.log main; ssl_certificate /path/to/cert.pem; ssl_certificate_key /path/to/key.key; set_real_ip_from 172.17.0.1; real_ip_header X-Forwarded-For; real_ip_recursive on; proxy_set_header Host $host:$server_port; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; location / { proxy_pass http://172.17.0.1:8100/; } }
What am I missing here?
Edit: I just realised that other people have a similar problem here and here but unfortunately no solution. |
| Squid Proxy: 400 Bad Request when "%25" (Percent Sign) in URL Posted: 25 Nov 2021 01:06 PM PST I have a squid proxy that works well except for this issue: If a URL has a %25 inside of it (the percent sign), we get a 400 Bad Request and Bad Request is displayed to the web browser. Example URL: http://www.amazon.com/25%25-Percent-Off-Stickers-Adhesive/dp/B00J0IBJ0S/ Log: 12/Jan/2016:18:40:28 -0600 429 MY.IP.IS.HERE TCP_MISS/400 310 GET http://www.amazon.com/25%25-Percent-Off-Stickers-Adhesive/dp/B00J0IBJ0S/ - ROUNDROBIN_PARENT/three text/html I'm not sure if this is a bug or a configuration error. I have a round robin setup as shown above. Here is the output of squid3 -v: Squid Cache: Version 3.1.19 configure options: '--build=x86_64-linux-gnu' '--prefix=/usr' '--includedir=${prefix}/include' '--mandir=${prefix}/share/man' '--infodir=${prefix}/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--libexecdir=${prefix}/lib/squid3' '--srcdir=.' '--disable-maintainer-mode' '--disable-dependency-tracking' '--disable-silent-rules' '--datadir=/usr/share/squid3' '--sysconfdir=/etc/squid3' '--mandir=/usr/share/man' '--with-cppunit-basedir=/usr' '--enable-inline' '--enable-async-io=8' '--enable-storeio=ufs,aufs,diskd' '--enable-removal-policies=lru,heap' '--enable-delay-pools' '--enable-cache-digests' '--enable-underscores' '--enable-icap-client' '--enable-follow-x-forwarded-for' '--enable-auth=basic,digest,ntlm,negotiate' '--enable-basic-auth-helpers=LDAP,MSNT,NCSA,PAM,SASL,SMB,YP,DB,POP3,getpwnam,squid_radius_auth,multi-domain-NTLM' '--enable-ntlm-auth-helpers=smb_lm,' '--enable-digest-auth-helpers=ldap,password' '--enable-negotiate-auth-helpers=squid_kerb_auth' '--enable-external-acl-helpers=ip_user,ldap_group,session,unix_group,wbinfo_group' '--enable-arp-acl' '--enable-esi' '--enable-zph-qos' '--enable-wccpv2' '--disable-translation' '--with-logdir=/var/log/squid3' '--with-pidfile=/var/run/squid3.pid' '--with-filedescriptors=65536' '--with-large-files' '--with-default-user=proxy' '--enable-linux-netfilter' 'build_alias=x86_64-linux-gnu' 'CFLAGS=-g -O2 -fPIE -fstack-protector --param=ssp-buffer-size=4 -Wformat -Wformat-security -Werror=format-security' 'LDFLAGS=-Wl,-Bsymbolic-functions -fPIE -pie -Wl,-z,relro -Wl,-z,now' 'CPPFLAGS=-D_FORTIFY_SOURCE=2' 'CXXFLAGS=-g -O2 -fPIE -fstack-protector --param=ssp-buffer-size=4 -Wformat -Wformat-security -Werror=format-security' --with-squid=/build/squid3-FzlLQ3/squid3-3.1.19 uname -a:
Linux MyHostName 3.13.0-44-generic #73~precise1-Ubuntu SMP Wed Dec 17 00:39:15 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux (It's an Ubuntu 12.04.5 LTS server) The hack with this URL would simply be to have something strip out the %25 but that wouldn't work on all destination websites. The URL works fine when not going through the proxy. Thanks for any ideas, I'm willing to provide more config info. |
| Bitlocker data drive opens as read only - Posted: 25 Nov 2021 12:03 PM PST I have a virtualized Windows Server 2012R2 running on ESXi 6.0 There are 2 virtual disks in this system, an unencrypted C drive for the OS and a Bitlocker encrypted D drive for data sharing. Bitlocker unlock method is via a password. When the drive is set to auto unlock, the drive is available as expected. If I disable the auto unlock and need to type the password at startup, the drive is available in the clear but it is available as Read Only. Windows sees this drive as a fixed disk but Bitlocker to Go is being implemented, I assume because of the virtual hard disks. Is this potentially my issue? I have looked at some other posts that have resolved similar issues with diskpart and the group policy setting "Deny write access to fixed (and removable) drives not protected by BitLocker" Which seems counterintuitive but others appeared to have luck with. What happens (or does not happen?) at startup when password is required that may be causing this issue? |
| Rsyslog doesn't create log files on CentOS7 Posted: 25 Nov 2021 02:03 PM PST I have the following configuration file in "/etc/rsyslog.d/10-my.conf" # This file is managed by Puppet, changes may be overwritten if $programname == 'hello' then -/var/log/test/test.log & ~
On CentOS6.5 (rsyslog 5.8.10 ) this creates an empty file in /var/log/test/test.log Same configuration file on CentOS7(rsyslog 7.4.7) doesn't create an empty file.
Anyone can tell why is that? Did this behavior change in 7.4? Or is it something on my CentOS7 instance?
rsyslogd -f/etc/rsyslog.d/10-my.conf -N3
On CentOS6 and CentOS7 returns whole bunch of warnings but nothing serious.
CentOS7 SELinux is set to Permissive mode
CentOS 7 (/etc/rsyslog.conf) # file is managed by puppet ################# #### MODULES #### ################# $ModLoad imuxsock # provides support for local system logging $ModLoad imjournal # provides access to the systemd journal ########################### #### GLOBAL DIRECTIVES #### ########################### $MaxMessageSize 2k # # Set the default permissions for all log files. # $FileOwner root $FileGroup root $FileCreateMode 0600 $DirOwner root $DirGroup root $DirCreateMode 0750 $PrivDropToUser root $PrivDropToGroup root $WorkDirectory /var/lib/rsyslog $Umask 0000 # Turn off message reception via local log socket; # local messages are retrieved through imjournal now. $OmitLocalLogging on $IncludeConfig /etc/rsyslog.d/*.conf # # Emergencies are sent to everybody logged in. # *.emerg :omusrmsg:*
|
| Can't Track Down What's Causing Server To Go Out of Memory Posted: 25 Nov 2021 01:32 PM PST For a few months now I've been experiencing random crashes on my 2GB RAM VPS Linode server. Someone convinced me to switch to self-managed hosting and I for some reason thought I could handle it despite having no experience in the field. I'm going to provide as much information as possible to help: Site in question is a Wordpress/Woocommerce site, it does have quite a lot of plugins so I'm aware that's not ideal but they are mostly required. Concurrent users is usually around 10, doesn't really go over 20. I've noticed that I do tend to get a lot of random bots coming to site that sometimes tends to cause spike traffic, but even that should be manageable I feel given I have a CDN and w3cache set up. The site runs smoothly with no problem 99% of the time. I've even run a load test using loadimpact.com and the server handled it fine. Here are some logs from latest crash, maybe someone can get some clues: Error Log [Fri Jun 19 03:06:07 2015] [error] server reached MaxClients setting, consider raising the MaxClients setting
MYSQL LOG 150619 03:06:09 mysqld_safe Number of processes running now: 0 150619 03:06:12 mysqld_safe mysqld restarted 150619 3:06:15 [Warning] Using unique option prefix key_buffer instead of key_buffer_size is deprecated and will be removed in a future release. Please use the full name instead. 150619 3:06:15 [Note] Plugin 'FEDERATED' is disabled. 150619 3:06:15 InnoDB: The InnoDB memory heap is disabled 150619 3:06:15 InnoDB: Mutexes and rw_locks use GCC atomic builtins 150619 3:06:15 InnoDB: Compressed tables use zlib 1.2.3 150619 3:06:15 InnoDB: Using Linux native AIO 150619 3:06:16 InnoDB: Initializing buffer pool, size = 320.0M 150619 3:06:16 InnoDB: Completed initialization of buffer pool 150619 3:06:17 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 150619 3:06:17 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery.
SYSLOG MESSAGES Jun 19 03:06:07 panel kernel: php invoked oom-killer: gfp_mask=0x201da, order=0, oom_score_adj=0 Jun 19 03:06:07 panel kernel: php cpuset=/ mems_allowed=0 Jun 19 03:06:07 panel kernel: CPU: 1 PID: 24844 Comm: php Not tainted 4.0.5-x86_64-linode58 #1 Jun 19 03:06:07 panel kernel: 0000000000000000 ffff880055fac740 ffffffff8193b626 ffff880055fac100 Jun 19 03:06:07 panel kernel: ffffffff819360a4 0100000000000400 ffff88007fc17000 0000000000000020 Jun 19 03:06:07 panel kernel: ffff88007fc17000 0000000000000000 0000000000000000 0000000000000000 Jun 19 03:06:07 panel kernel: Call Trace: Jun 19 03:06:07 panel kernel: [<ffffffff8193b626>] ? dump_stack+0x40/0x50 Jun 19 03:06:07 panel kernel: [<ffffffff819360a4>] ? dump_header.isra.10+0x78/0x1e3 Jun 19 03:06:07 panel kernel: [<ffffffff81941bb6>] ? _raw_spin_unlock_irqrestore+0x2e/0x3f Jun 19 03:06:07 panel kernel: [<ffffffff81172f4a>] ? oom_kill_process+0xbe/0x380 Jun 19 03:06:07 panel kernel: [<ffffffff810dc57d>] ? has_ns_capability_noaudit+0x13/0x1b Jun 19 03:06:07 panel kernel: [<ffffffff8117369a>] ? __out_of_memory+0x43d/0x47d Jun 19 03:06:07 panel kernel: [<ffffffff8117381b>] ? out_of_memory+0x52/0x67 Jun 19 03:06:07 panel kernel: [<ffffffff811777e5>] ? __alloc_pages_nodemask+0x708/0x846 Jun 19 03:06:07 panel kernel: [<ffffffff811a4a7d>] ? alloc_pages_current+0xb2/0xcf Jun 19 03:06:07 panel kernel: [<ffffffff8117205d>] ? filemap_fault+0x26f/0x394 Jun 19 03:06:07 panel kernel: [<ffffffff8118f78d>] ? __do_fault+0x3f/0x79 Jun 19 03:06:07 panel kernel: [<ffffffff81193009>] ? handle_mm_fault+0x370/0xd80 Jun 19 03:06:07 panel kernel: [<ffffffff811426a1>] ? __audit_syscall_exit+0x208/0x224 Jun 19 03:06:07 panel kernel: [<ffffffff8104219d>] ? __do_page_fault+0x321/0x37b Jun 19 03:06:07 panel kernel: [<ffffffff81943f08>] ? page_fault+0x28/0x30 Jun 19 03:06:07 panel kernel: Mem-Info: Jun 19 03:06:07 panel kernel: Node 0 DMA per-cpu: Jun 19 03:06:07 panel kernel: CPU 0: hi: 0, btch: 1 usd: 0 Jun 19 03:06:07 panel kernel: CPU 1: hi: 0, btch: 1 usd: 0 Jun 19 03:06:07 panel kernel: Node 0 DMA32 per-cpu: Jun 19 03:06:07 panel kernel: CPU 0: hi: 186, btch: 31 usd: 3 Jun 19 03:06:07 panel kernel: CPU 1: hi: 186, btch: 31 usd: 0 Jun 19 03:06:07 panel kernel: active_anon:348396 inactive_anon:116687 isolated_anon:32 Jun 19 03:06:07 panel kernel: active_file:351 inactive_file:528 isolated_file:0 Jun 19 03:06:07 panel kernel: unevictable:0 dirty:0 writeback:6 unstable:0 Jun 19 03:06:07 panel kernel: free:3411 slab_reclaimable:8339 slab_unreclaimable:5537 Jun 19 03:06:07 panel kernel: mapped:169339 shmem:168953 pagetables:8887 bounce:0 Jun 19 03:06:07 panel kernel: free_cma:0 Jun 19 03:06:07 panel kernel: Node 0 DMA free:7928kB min:44kB low:52kB high:64kB active_anon:2880kB inactive_anon:3140kB active_file:0kB inactive_file:28kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15996kB managed:15912kB mlocked:0kB dirty:0kB writeback:0kB mapped:2676kB shmem:2664kB slab_reclaimable:44kB slab_unreclaimable:164kB kernel_stack:0kB pagetables:132kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:20 all_unreclaimable? no Jun 19 03:06:07 panel kernel: lowmem_reserve[]: 0 1972 1972 1972 Jun 19 03:06:07 panel kernel: Node 0 DMA32 free:5736kB min:5656kB low:7068kB high:8484kB active_anon:1390704kB inactive_anon:463608kB active_file:1488kB inactive_file:1892kB unevictable:0kB isolated(anon):128kB isolated(file):0kB present:2080768kB managed:2023584kB mlocked:0kB dirty:0kB writeback:24kB mapped:675052kB shmem:673148kB slab_reclaimable:33312kB slab_unreclaimable:21984kB kernel_stack:4304kB pagetables:35416kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Jun 19 03:06:07 panel kernel: lowmem_reserve[]: 0 0 0 0 Jun 19 03:06:07 panel kernel: Node 0 DMA: 4*4kB (UM) 5*8kB (U) 8*16kB (UM) 4*32kB (UM) 3*64kB (UE) 10*128kB (EM) 6*256kB (UEM) 1*512kB (U) 0*1024kB 0*2048kB 1*4096kB (R) = 7928kB Jun 19 03:06:07 panel kernel: Node 0 DMA32: 97*4kB (UEMR) 10*8kB (MR) 0*16kB 0*32kB 1*64kB (R) 1*128kB (R) 0*256kB 0*512kB 1*1024kB (R) 0*2048kB 1*4096kB (R) = 5780kB Jun 19 03:06:07 panel kernel: 174507 total pagecache pages Jun 19 03:06:07 panel kernel: 4598 pages in swap cache Jun 19 03:06:07 panel kernel: Swap cache stats: add 1673544, delete 1668946, find 6783627/7234079 Jun 19 03:06:07 panel kernel: Free swap = 0kB Jun 19 03:06:07 panel kernel: Total swap = 262140kB Jun 19 03:06:07 panel kernel: 524191 pages RAM Jun 19 03:06:07 panel kernel: 0 pages HighMem/MovableOnly Jun 19 03:06:07 panel kernel: 14317 pages reserved Jun 19 03:06:07 panel kernel: [ pid ] uid tgid total_vm rss nr_ptes nr_pmds swapents oom_score_adj name Jun 19 03:06:07 panel kernel: [ 1541] 0 1541 2710 0 9 2 127 -1000 udevd Jun 19 03:06:07 panel kernel: [ 2129] 0 2129 2295 34 8 2 90 0 dhclient Jun 19 03:06:07 panel kernel: [ 2175] 0 2175 6402 0 12 2 75 -1000 auditd Jun 19 03:06:07 panel kernel: [ 2238] 0 2238 60749 0 21 3 214 0 rsyslogd Jun 19 03:06:07 panel kernel: [ 2264] 25 2264 57836 0 44 2 4143 0 named Jun 19 03:06:07 panel kernel: [ 2295] 81 2295 5373 0 14 2 63 0 dbus-daemon Jun 19 03:06:07 panel kernel: [ 2350] 495 2350 80168 0 26 5 86 0 memcached Jun 19 03:06:07 panel kernel: [ 2368] 0 2368 16569 11 34 2 166 -1000 sshd Jun 19 03:06:07 panel kernel: [ 2662] 0 2662 4943 23 19 5 54 0 dovecot Jun 19 03:06:07 panel kernel: [ 2665] 498 2665 10600 0 41 6 144 0 pop3-login Jun 19 03:06:07 panel kernel: [ 2666] 498 2666 10639 0 41 6 153 0 imap-login Jun 19 03:06:07 panel kernel: [ 2667] 97 2667 3369 0 18 5 49 0 anvil Jun 19 03:06:07 panel kernel: [ 2668] 0 2668 3402 0 18 5 78 0 log Jun 19 03:06:07 panel kernel: [ 2671] 498 2671 10636 18 41 6 120 0 pop3-login Jun 19 03:06:07 panel kernel: [ 2672] 0 2672 4145 0 19 5 282 0 config Jun 19 03:06:07 panel kernel: [ 2673] 498 2673 10635 0 42 6 145 0 imap-login Jun 19 03:06:07 panel kernel: [ 2683] 47 2683 18053 15 49 7 136 0 exim Jun 19 03:06:07 panel kernel: [ 2807] 0 2807 43014 217 108 6 12952 0 /usr/local/cpan Jun 19 03:06:07 panel kernel: [ 2895] 0 2895 43249 4306 97 6 9070 0 spamd child Jun 19 03:06:07 panel kernel: [ 2952] 501 2952 4042 0 14 4 33 0 newrelic-daemon Jun 19 03:06:07 panel kernel: [ 2953] 0 2953 28675 0 16 2 52 0 abrtd Jun 19 03:06:07 panel kernel: [ 2954] 501 2954 60989 123 29 4 544 0 newrelic-daemon Jun 19 03:06:07 panel kernel: [ 2970] 0 2970 28181 0 23 4 65 0 abrt-dump-oops Jun 19 03:06:07 panel kernel: [ 2981] 0 2981 34674 9 23 2 160 0 pure-ftpd Jun 19 03:06:07 panel kernel: [ 2983] 0 2983 34047 7 27 2 114 0 pure-authd Jun 19 03:06:07 panel kernel: [ 2995] 0 2995 29217 8 16 2 148 0 crond Jun 19 03:06:07 panel kernel: [ 3009] 0 3009 5291 11 14 2 34 0 atd Jun 19 03:06:07 panel kernel: [ 3521] 0 3521 35818 389 90 6 6493 0 cpsrvd (SSL) - Jun 19 03:06:07 panel kernel: [ 3760] 0 3760 10172 158 34 4 1275 0 queueprocd - wa Jun 19 03:06:07 panel kernel: [ 3832] 0 3832 28548 0 66 5 4769 0 dnsadmin - serv Jun 19 03:06:07 panel kernel: [ 3858] 0 3858 21764 452 54 5 3245 0 tailwatchd Jun 19 03:06:07 panel kernel: [ 3934] 0 3934 29015 2 65 5 5416 0 cpdavd - accept Jun 19 03:06:07 panel kernel: [ 3964] 0 3964 7951 11 27 4 1315 0 cpanellogd - sl Jun 19 03:06:07 panel kernel: [ 4002] 0 4002 1035 1 9 4 19 0 agetty Jun 19 03:06:07 panel kernel: [ 4004] 0 4004 1031 0 9 4 17 0 mingetty Jun 19 03:06:07 panel kernel: [ 4006] 0 4006 1031 0 9 4 18 0 mingetty Jun 19 03:06:07 panel kernel: [ 4008] 0 4008 1031 0 9 4 18 0 mingetty Jun 19 03:06:07 panel kernel: [ 4011] 0 4011 2709 0 9 2 128 -1000 udevd Jun 19 03:06:07 panel kernel: [ 4012] 0 4012 2709 0 9 2 128 -1000 udevd Jun 19 03:06:07 panel kernel: [ 4013] 0 4013 1031 0 9 4 18 0 mingetty Jun 19 03:06:07 panel kernel: [ 4015] 0 4015 1031 0 9 4 18 0 mingetty Jun 19 03:06:07 panel kernel: [ 4017] 0 4017 1031 0 8 4 17 0 mingetty Jun 19 03:06:07 panel kernel: [16053] 0 16053 18080 444 59 5 138 0 httpd Jun 19 03:06:07 panel kernel: [19702] 0 19702 2883 1 15 4 71 0 mysqld_safe Jun 19 03:06:07 panel kernel: [19964] 497 19964 393396 49834 210 7 17743 0 mysqld Jun 19 03:06:07 panel kernel: [22735] 0 22735 3368 26 17 5 0 0 ssl-params Jun 19 03:06:07 panel kernel: [24373] 0 24373 20166 2064 63 5 9 0 leechprotect Jun 19 03:06:07 panel kernel: [24374] 99 24374 18247 625 48 5 129 0 httpd Jun 19 03:06:07 panel kernel: [24375] 99 24375 18259 641 48 5 127 0 httpd Jun 19 03:06:07 panel kernel: [24378] 99 24378 18283 650 48 5 129 0 httpd Jun 19 03:06:07 panel kernel: [24379] 99 24379 18207 582 48 5 131 0 httpd Jun 19 03:06:07 panel kernel: [24380] 99 24380 18294 651 48 5 129 0 httpd Jun 19 03:06:07 panel kernel: [24383] 99 24383 18251 632 48 5 129 0 httpd Jun 19 03:06:07 panel kernel: [24388] 99 24388 18275 651 48 5 129 0 httpd Jun 19 03:06:07 panel kernel: [24395] 99 24395 18232 610 48 5 129 0 httpd Jun 19 03:06:07 panel kernel: [24438] 0 24438 4252 51 12 2 0 0 anacron Jun 19 03:06:07 panel kernel: [24457] 99 24457 18249 629 48 5 129 0 httpd Jun 19 03:06:07 panel kernel: [24460] 99 24460 18243 622 48 5 129 0 httpd Jun 19 03:06:07 panel kernel: [24559] 99 24559 18234 611 48 5 129 0 httpd Jun 19 03:06:07 panel kernel: [24688] 99 24688 18283 646 48 5 131 0 httpd Jun 19 03:06:07 panel kernel: [24757] 97 24757 7930 113 30 5 0 0 auth Jun 19 03:06:07 panel kernel: [24838] 507 24838 84744 21633 134 6 0 0 php Jun 19 03:06:07 panel kernel: [24840] 507 24840 83776 20878 133 6 0 0 php Jun 19 03:06:07 panel kernel: [24841] 507 24841 82483 18820 128 7 0 0 php Jun 19 03:06:07 panel kernel: [24844] 507 24844 82937 20101 130 7 0 0 php Jun 19 03:06:07 panel kernel: [24845] 507 24845 82227 18788 129 7 0 0 php Jun 19 03:06:07 panel kernel: [24846] 507 24846 82291 19040 129 6 0 0 php Jun 19 03:06:07 panel kernel: [24847] 507 24847 82887 19792 131 7 0 0 php Jun 19 03:06:07 panel kernel: [24848] 507 24848 82934 20094 130 7 0 0 php Jun 19 03:06:07 panel kernel: [24850] 507 24850 82894 19887 130 6 0 0 php Jun 19 03:06:07 panel kernel: [24852] 507 24852 82548 19709 129 6 0 0 php Jun 19 03:06:07 panel kernel: [24859] 507 24859 82803 19724 131 6 0 0 php Jun 19 03:06:07 panel kernel: [24862] 99 24862 18149 480 48 5 132 0 httpd Jun 19 03:06:07 panel kernel: [24866] 507 24866 81465 17804 126 6 0 0 php Jun 19 03:06:07 panel kernel: [24875] 99 24875 18149 480 48 5 132 0 httpd Jun 19 03:06:07 panel kernel: [24876] 99 24876 18149 480 48 5 132 0 httpd Jun 19 03:06:07 panel kernel: [24877] 507 24877 79415 14127 121 6 0 0 php Jun 19 03:06:07 panel kernel: [24878] 507 24878 79221 13720 118 7 0 0 php Jun 19 03:06:07 panel kernel: [24885] 99 24885 18149 480 48 5 132 0 httpd Jun 19 03:06:07 panel kernel: [24886] 99 24886 18182 523 48 5 132 0 httpd Jun 19 03:06:07 panel kernel: [24887] 99 24887 18182 523 48 5 132 0 httpd Jun 19 03:06:07 panel kernel: [24888] 99 24888 18182 523 48 5 132 0 httpd Jun 19 03:06:07 panel kernel: [24892] 507 24892 77328 9824 111 6 0 0 php Jun 19 03:06:07 panel kernel: [24898] 99 24898 18149 480 48 5 132 0 httpd Jun 19 03:06:07 panel kernel: [24899] 99 24899 18149 480 48 5 132 0 httpd Jun 19 03:06:07 panel kernel: [24900] 99 24900 18182 513 48 5 132 0 httpd Jun 19 03:06:07 panel kernel: [24901] 99 24901 18149 480 48 5 132 0 httpd Jun 19 03:06:07 panel kernel: [24902] 99 24902 18149 480 48 5 132 0 httpd Jun 19 03:06:07 panel kernel: [24903] 99 24903 18149 480 48 5 132 0 httpd Jun 19 03:06:07 panel kernel: [24904] 99 24904 18149 480 48 5 132 0 httpd Jun 19 03:06:07 panel kernel: [24905] 99 24905 18149 480 48 5 132 0 httpd Jun 19 03:06:07 panel kernel: [24906] 507 24906 75260 6007 103 6 0 0 php Jun 19 03:06:07 panel kernel: [24907] 507 24907 61544 14697 114 6 0 0 php Jun 19 03:06:07 panel kernel: [24908] 507 24908 75724 6894 105 6 0 0 php Jun 19 03:06:07 panel kernel: [24909] 507 24909 74348 5097 101 6 0 0 php Jun 19 03:06:07 panel kernel: [24910] 507 24910 75746 6708 104 7 0 0 php Jun 19 03:06:07 panel kernel: [24911] 507 24911 75746 6630 104 6 0 0 php Jun 19 03:06:07 panel kernel: [24912] 507 24912 75746 6705 104 6 0 0 php Jun 19 03:06:07 panel kernel: [24913] 507 24913 76743 8656 108 6 0 0 php Jun 19 03:06:07 panel kernel: [24922] 507 24922 73138 3847 99 7 0 0 php Jun 19 03:06:07 panel kernel: [24923] 507 24923 72900 3513 96 6 0 0 php Jun 19 03:06:07 panel kernel: [24925] 507 24925 72704 3279 98 6 0 0 php Jun 19 03:06:07 panel kernel: [24926] 507 24926 72856 3395 98 6 0 0 php Jun 19 03:06:07 panel kernel: [24927] 99 24927 18149 480 48 5 132 0 httpd Jun 19 03:06:07 panel kernel: [24928] 99 24928 18149 480 48 5 132 0 httpd Jun 19 03:06:07 panel kernel: [24929] 99 24929 18149 480 48 5 132 0 httpd Jun 19 03:06:07 panel kernel: [24930] 99 24930 18149 480 48 5 132 0 httpd Jun 19 03:06:07 panel kernel: [24931] 99 24931 18149 480 48 5 132 0 httpd Jun 19 03:06:07 panel kernel: [24932] 99 24932 18149 480 48 5 132 0 httpd Jun 19 03:06:07 panel kernel: [24933] 99 24933 18149 480 48 5 132 0 httpd Jun 19 03:06:07 panel kernel: [24934] 99 24934 18149 480 48 5 132 0 httpd Jun 19 03:06:07 panel kernel: [24935] 99 24935 18149 480 48 5 132 0 httpd Jun 19 03:06:07 panel kernel: [24936] 507 24936 72900 3454 98 7 0 0 php Jun 19 03:06:07 panel kernel: [24937] 507 24937 72900 3512 100 6 0 0 php Jun 19 03:06:07 panel kernel: [24938] 507 24938 72898 3510 99 6 0 0 php Jun 19 03:06:07 panel kernel: [24940] 99 24940 18149 480 48 5 132 0 httpd Jun 19 03:06:07 panel kernel: [24941] 99 24941 18149 480 48 5 132 0 httpd Jun 19 03:06:07 panel kernel: [24942] 99 24942 18149 480 48 5 132 0 httpd Jun 19 03:06:07 panel kernel: [24944] 507 24944 72709 3228 98 6 0 0 php Jun 19 03:06:07 panel kernel: [24945] 507 24945 72707 3210 96 6 0 0 php Jun 19 03:06:07 panel kernel: [24946] 507 24946 72505 2756 97 6 0 0 php Jun 19 03:06:07 panel kernel: [24947] 507 24947 72770 3394 99 6 0 0 php Jun 19 03:06:07 panel kernel: [24948] 507 24948 72898 3511 98 7 0 0 php Jun 19 03:06:07 panel kernel: [24950] 507 24950 72920 3491 99 6 0 0 php Jun 19 03:06:07 panel kernel: [24952] 99 24952 18149 480 48 5 132 0 httpd Jun 19 03:06:07 panel kernel: [24953] 99 24953 18149 480 48 5 132 0 httpd Jun 19 03:06:07 panel kernel: [24954] 99 24954 18149 480 48 5 132 0 httpd Jun 19 03:06:07 panel kernel: [24956] 507 24956 72273 2332 95 6 0 0 php Jun 19 03:06:07 panel kernel: [24959] 507 24959 71635 1659 96 6 0 0 php Jun 19 03:06:07 panel kernel: [24960] 507 24960 72358 2601 97 7 0 0 php Jun 19 03:06:07 panel kernel: [24961] 507 24961 72431 2797 96 6 0 0 php Jun 19 03:06:07 panel kernel: [24962] 507 24962 72565 2478 97 6 0 0 php Jun 19 03:06:07 panel kernel: [24963] 99 24963 18149 480 48 5 132 0 httpd Jun 19 03:06:07 panel kernel: [24964] 507 24964 72505 2727 98 6 0 0 php Jun 19 03:06:07 panel kernel: [24965] 507 24965 71871 1883 95 6 0 0 php Jun 19 03:06:07 panel kernel: [24987] 99 24987 18149 467 48 5 132 0 httpd Jun 19 03:06:07 panel kernel: [24988] 99 24988 18149 472 48 5 132 0 httpd Jun 19 03:06:07 panel kernel: [24989] 99 24989 18149 467 48 5 132 0 httpd Jun 19 03:06:07 panel kernel: [24990] 99 24990 18149 479 48 5 132 0 httpd Jun 19 03:06:07 panel kernel: [24991] 99 24991 18149 477 48 5 132 0 httpd Jun 19 03:06:07 panel kernel: [24992] 507 24992 24451 62 72 6 0 0 php Jun 19 03:06:07 panel kernel: [24993] 99 24993 18149 478 48 5 132 0 httpd Jun 19 03:06:07 panel kernel: [24994] 99 24994 2998 13 14 4 0 0 suphp Jun 19 03:06:07 panel kernel: [24995] 507 24995 5810 9 12 4 0 0 php Jun 19 03:06:07 panel kernel: [24996] 507 24996 3880 9 10 4 0 0 php Jun 19 03:06:07 panel kernel: [24998] 507 24998 6373 16 15 5 0 0 php Jun 19 03:06:07 panel kernel: [24999] 507 24999 2767 1 3 3 0 0 php Jun 19 03:06:07 panel kernel: [25000] 0 25000 18080 432 47 5 135 0 httpd Jun 19 03:06:07 panel kernel: Out of memory: Kill process 19964 (mysqld) score 117 or sacrifice child Jun 19 03:06:07 panel kernel: Killed process 19964 (mysqld) total-vm:1573584kB, anon-rss:199336kB, file-rss:0kB Jun 19 03:06:07 panel kernel: php invoked oom-killer: gfp_mask=0x280da, order=0, oom_score_adj=0 Jun 19 03:06:07 panel kernel: php cpuset=/ mems_allowed=0
ANd here's my apache settings: Starts Servers 2 Minimum Spare Servers 6 Maximum Spare Servers 12 Server Limit 256 Max Clients 50 Max Requests Per Child 3000 Keep-Alive Off
Mysql my.cnf settings: [mysqld] innodb_file_per_table=1 open_files_limit=1024000 max_connections=75 key_buffer=32M max_allowed_packet=268435456 thread_stack=128K table_cache=32 innodb_buffer_pool_size=320M default-storage-engine=MyISAM query_cache_size=8M tmp_table_size=16M max_heap_table_size=16M thread_cache_size=4 table_open_cache=32
Please help, I've tried my best to track this thing down but I can't really pinpoint what is causing it. BTW some additional random tidbits: -Some of the pages on my site have LOTS of pictures (like over 100), not sure if these count as extra clients/requests and affecting performance somehow. -My Wordpress memory limit is set around 400MB. -Mysql database for this site is around 175MB. 91MB of it coming from WP_Options. -Mysqltuner.pl keeps telling me to increase tmp_table_size and max_heap_table_size even though I've set them as high as 200M! -Here's a screenshot of my htop right now. Seems mysql is constantly using 40% of memory and Swap memory is almost full for some reason. Is this normal? : 
UPDATE: I've lowered Max Clients to 35 but still getting some weird mysql crash. Newrelic error message is Unknown storage engine 'InnoDB' (my mysqltuner.pl is saying InnoDB is disabled but that it's default storage engine). My site is telling me to reinstall Wordpress like a fresh install, and I have to restart mysql or the server for that to go away. |
| Auto accept rsa key fingerprint from command line Posted: 25 Nov 2021 09:12 AM PST I've tried yes | ssh root@10.x.x.x to try to accept the RSA key fingerprint, but am still prompted if I'm sure I want to connect. Is there a way to make this automatic? |
| Restarting shell script with &disown using Monit Posted: 25 Nov 2021 10:05 AM PST I have a shell script that runs a C++ backend mail system (PluginHandler). I need to monitor this process in Monit and restart it if it fails. The script: export LD_LIBRARY_PATH=/usr/local/lib/:/CONFIDENTAL/CONFIDENTAL/Common/ cd PluginHandler/ ./PluginHandler
This script does not have a PID file and we run this script by executing ./rundaemon.sh &disown
./pluginhandler starts the process and starts logging into /etc/output/output.log I stop the process by identifying the process ID with [ps -f | grep PluginHandler] and then killing the process. I can check the process in Monit just fine, but I think Monit is starting the process if it is not running but it can't do &disown so the process ends as soon as it starts. This is the code in the monitrc file for checking this process: check process Backend matching "PluginHandler" if not exist then alert start "PATH/TO/SCRIPT/rundaemon.sh &disown" alert example@gmail.com only on {timeout} with mail-format {subject: "[BLAH"}
I tried to stop the script from terminating by modifying the script like the following but this does not work either. export LD_LIBRARY_PATH=/usr/local/lib/:/home/CONFIDENTAL/production/CONFIDENTAL/Common/ cd PluginHandler/ (nohup ./PluginHandler &) return
Any help to write a proper Monit rules to resolve this issue would be greatly appreciated :) |
| PHP Errors are not stored on CentOS Server Posted: 25 Nov 2021 11:09 AM PST I just adjusted the php.ini on my CentOS 64 Bits VPS in /etc/php.ini to log PHP errors: cat /etc/php.ini | grep php-errors.log error_log = /var/log/php-errors.log
I also have log_errors = on I created the log file in /var/log/ and it is CHMOD 644. I also turned on Error reporting E_ALL cat /etc/php.ini | grep error_reporting ; error_reporting error_reporting = E_ALL ; Eval the expression with current error_reporting(). Set to true if you want ; error_reporting(0) around the eval().
Then I restarted the httpd daemon. When I add a file via the WordPress uploader I see it is not uploaded because of a permission issue "cannot-open-file.png" has failed to upload due to an error Unable to create directory wp-content/uploads/2014/05. Is its parent directory writable by the server?
, but it is not stored as an error in php-errors.php: pwd /var/log ls -l | grep php -rw-r--r-- 1 root root 0 May 6 06:21 php-errors.log
All my other logs in /var/log/httpd are also root:root so I would assume the logging would work. And when I did adjust the file's permissions to apache:apache as suggested I still had no errors in the log file. Even adding error logging on to the .htaccess did not help. I also checked the PHP.ini using phpinfo() . The only ini loaded is the one I adjusted in /etc/php.ini and the user and group it is using is apache - User/Group apache(48)/48 . What am I missing? PS Could be issues with the directory for the log files as suggested here Can't configure PHP error log I am checking out more info on this. |
| It is okay to set MASQUERADE at 2 network interfaces in a Linux server? Posted: 25 Nov 2021 11:53 AM PST There is a Linux server with 3 network interfaces, eth0, eth1, eth2. IP forwarding has been turn on in this server. - eth0 is connected to 10.0.1.0/24. Its IP is
10.0.1.1. - eth1 is connected to 172.16.1.0/24. Its IP is
172.16.1.1. Server A can ping router C at 172.16.1.2. - eth2 is connected to 192.168.1.0/24. Its IP is
192.168.1.1. Server A can ping server B at 192.168.1.2. - Router C is able to route to 172.16.2.0/24 and 172.16.3.0/24.
[10.0.1.0/24] | 172.16.2.0/24------| | [C]------172.16.1.0/24------[A]------192.168.1.0/24------[B] 172.16.3.0/24------| We have set MASQUERADE at eth0. When server B (192.168.1.2) connect to 10.0.1.0/24, IP MASQUERADE will happen at eth0. Can we set MASQUERADE at eth1? Is it okay to set MASQUERADE at more than 1 network interfaces in Linux? |
| Quickest/best way to copy a portion of a large mongo database to another server? Posted: 25 Nov 2021 03:02 PM PST I have a dataset of 100m tweets stored in Mongo, unoptimized and unindexed. I need to copy all tweets from the last month onto another server, what is the best way to do this? My idea was to use a Ruby script to extract and copy the relevant tweets to a new database on the server, then run the mongo copyDatabase command to copy it over. Its taking horrendously long though, any other way to do it? require 'mongo_mapper' MongoMapper.database = 'twitter' require './models' tweets = TwitterTweet.where(:created_at => {"$gt" => 1.month.ago}).all; # about 15 million MongoMapper.database = 'monthly' # copy the tweets over to the new db tweets.each do |tweet| tweet.save! end;
|
| How to decrease the size of a KVM virtual machine disk image? Posted: 25 Nov 2021 01:10 PM PST How do you decrease or shrink the size of a KVM virtual machine disk? I allocated a virtual disk of 500GB (stored at /var/lib/libvirt/images/vm1.img), and I'm finding that overkill, so now I'd like to free up some of that space for use with other virtual machines. There seems to be a lot answers on how to increase image storage, but not decrease it. I found the virt-resize tool, but it only seems to work with raw disk partitions, not disk images. Edit: I'm using an LVM with an Ext4 formatted partition. Edit: GParted screenshot showing my LVM parition layout. I need to do a lot more then just resize the filesystem. I know of no safe way to resize an LVM. And please don't tell me to use pvresize. Despite its name, it does not support resizing LVMs. 
I did try sudo pvresize /dev/vda5, but it just says physical volume "/dev/vda5" changed but doesn't actually reduce the size. I tried start parted to manually set the partition size (very dangerous), but doing print all just gives me the error "/dev/sr0 unrecognised disk label". Edit: By following these instructions, I was able to successfully shrink both my logical and physical volumes (although I had to remember to activate and deactivate lvm before and after certain commands, which the instructions omit. Now GParted is showing 250G of unallocated free space. How do I remove this from the KVM disk image and give it back to the underlying hypervisor? |
| join_buffer_size >= 4 M is not advised? Posted: 25 Nov 2021 11:49 AM PST I get this message from MysqlTunner.pl: join_buffer_size >= 4 M This is not advised On the other hand, I read in Debian's my.cnf guide about jont_buffer_size that: This buffer is used for the optimization of full JOINs (JOINs without indexes). Such JOINs are very bad for performance in most cases anyway, but setting this variable to a large value reduces the performance impact. See the "Select_full_join" status variable for a count of full JOINs. Allocated per thread if full join is found So I'm wondering which one should I believe? Currently I've set join_buffer_size = 64M as part of efforts to cope with scalability problem of a high-traffic site whose queries are not particularly optimized. I appreciate your hints on this. |
No comments:
Post a Comment