MemCached::increment() is not atomic? - php

My understanding is that Memcached::increment is atomic.
I have this code:
include('../clibootstrap.php');
$key = 'ad_1';
$mc = $app['memcache'];
$mc->setOption(Memcached::OPT_BINARY_PROTOCOL,true);
usleep(10);
$mc->increment($key, 1, 0);
die('OK');
$mc is an instance of \MemCached
Now I try to benchmark it using Apache Bench:
# ab -n2000 -c100 http://somehost.com/foo.php
Benchmarking somehost.com (be patient)
Completed 200 requests
Completed 400 requests
Completed 600 requests
Completed 800 requests
Completed 1000 requests
Completed 1200 requests
Completed 1400 requests
Completed 1600 requests
Completed 1800 requests
Completed 2000 requests
Finished 2000 requests
Server Software: Apache/2.2.22
Server Hostname: somehost.com
Server Port: 80
Document Path: /foo.php
Document Length: 2 bytes
Concurrency Level: 100
Time taken for tests: 4.821 seconds
Complete requests: 2000
Failed requests: 0
Write errors: 0
Total transferred: 352000 bytes
HTML transferred: 4000 bytes
Requests per second: 414.82 [#/sec] (mean)
Time per request: 241.067 [ms] (mean)
Time per request: 2.411 [ms] (mean, across all concurrent requests)
Transfer rate: 71.30 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 1.0 0 5
Processing: 33 237 34.5 237 323
Waiting: 33 237 34.5 237 323
Total: 38 237 34.0 237 323
Percentage of the requests served within a certain time (ms)
50% 237
66% 255
75% 263
80% 266
90% 274
95% 279
98% 287
99% 294
100% 323 (longest request)
Now I expect the value of the 'ad_1' key to be exactly 2000, so let's check with telnet:
# telnet localhost 11211
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
get ad_1
VALUE ad_1 0 4
1997
END
version
VERSION 1.4.13
stats
STAT pid 5527
STAT uptime 248
STAT time 1414164851
STAT version 1.4.13
STAT libevent 2.0.16-stable
STAT pointer_size 64
STAT rusage_user 0.092005
STAT rusage_system 0.268016
STAT curr_connections 5
STAT total_connections 2006
STAT connection_structures 23
STAT reserved_fds 20
STAT cmd_get 1
STAT cmd_set 0
STAT cmd_flush 0
STAT cmd_touch 0
STAT get_hits 1
STAT get_misses 0
STAT delete_misses 0
STAT delete_hits 0
STAT incr_misses 0
STAT incr_hits 1997
STAT decr_misses 0
STAT decr_hits 0
STAT cas_misses 0
STAT cas_hits 0
STAT cas_badval 0
STAT touch_hits 0
STAT touch_misses 0
STAT auth_cmds 0
STAT auth_errors 0
STAT bytes_read 144026
STAT bytes_written 112049
STAT limit_maxbytes 67108864
STAT accepting_conns 1
STAT listen_disabled_num 0
STAT threads 4
STAT conn_yields 0
STAT hash_power_level 16
STAT hash_bytes 524288
STAT hash_is_expanding 0
STAT expired_unfetched 0
STAT evicted_unfetched 0
STAT bytes 73
STAT curr_items 1
STAT total_items 1998
STAT evictions 0
STAT reclaimed 0
END
I am using PHP-memcached version 2.2
Any ideas as to why the ad_1 value is not 2000 ?
And how can I make sure that MemCached::increment() becomes atomic?
Also, in the line VALUE ad_1 0 4, what does the 4 mean ?

I used a semaphore, changing my code to
include('../clibootstrap.php');
$key = 'ad_1';
$mc = $app['memcache'];
$mc->setOption(Memcached::OPT_BINARY_PROTOCOL,true);
$sem = sem_get(1234, 1);
if (sem_acquire($sem)) {
$mc->increment($key, 1, 1);
sem_release($sem);
}
usleep(10);
die('OK');
and now it works.

Related

Problem with PHP session and memcached servers

I made a lot of searches on Google, but I couldn't find anything that help me...
I have a PHP Server (Version 7.3.27-1~deb10u1), and I store PHP session with memcached PECL module (v 3.1.4), connected to two servers Memcached 1.5.6 (mc3, mc4).
Here is the php info of memcached:
memcached
memcached support enabled
Version 3.1.4
libmemcached version 1.0.18
SASL support yes
Session support yes
igbinary support yes
json support yes
msgpack support yes
Directive Local Value Master Value
memcached.compression_factor 1.3 1.3
memcached.compression_threshold 2000 2000
memcached.compression_type fastlz fastlz
memcached.default_binary_protocol Off Off
memcached.default_connect_timeout 0 0
memcached.default_consistent_hash Off Off
memcached.serializer php php
memcached.sess_binary_protocol On On
memcached.sess_connect_timeout 0 0
memcached.sess_consistent_hash On On
memcached.sess_consistent_hash_type ketama ketama
memcached.sess_lock_expire 0 0
memcached.sess_lock_max_wait not set not set
memcached.sess_lock_retries 10 10
memcached.sess_lock_wait not set not set
memcached.sess_lock_wait_max 2500 2500
memcached.sess_lock_wait_min 150 150
memcached.sess_locking On On
memcached.sess_number_of_replicas 1 1
memcached.sess_persistent Off Off
memcached.sess_prefix memc.sess.key. memc.sess.key.
memcached.sess_randomize_replica_read Off Off
memcached.sess_remove_failed_servers Off Off
memcached.sess_sasl_password no value no value
memcached.sess_sasl_username no value no value
memcached.sess_server_failure_limit 0 0
memcached.store_retry_count 2 2
php info about session:
Directive Local Value Master Value
session.auto_start Off Off
session.cache_expire 180 180
session.cache_limiter nocache nocache
session.cookie_domain no value no value
session.cookie_httponly 1 1
session.cookie_lifetime 0 0
session.cookie_path / /
session.cookie_samesite None None
session.cookie_secure 1 1
session.gc_divisor 1000 1000
session.gc_maxlifetime 1440 1440
session.gc_probability 0 0
session.lazy_write Off Off
session.name PHPSESSID PHPSESSID
session.referer_check no value no value
session.save_handler memcached memcached
session.save_path mc3:11211,mc4:11211 mc3:11211,mc4:11211
session.serialize_handler php php
session.sid_bits_per_character 5 5
session.sid_length 26 26
session.upload_progress.cleanup On On
session.upload_progress.enabled On On
session.upload_progress.freq 1% 1%
session.upload_progress.min_freq 1 1
session.upload_progress.name PHP_SESSION_UPLOAD_PROGRESS PHP_SESSION_UPLOAD_PROGRESS
session.upload_progress.prefix upload_progress_ upload_progress_
session.use_cookies 1 1
session.use_only_cookies 1 1
session.use_strict_mode 1 1
session.use_trans_sid 0
Sometimes, I have some 3 different PHP Warnings in the error.log:
PHP Warning: session_start(): Failed to read session data: memcached (path: mc3:11211,mc4:11211) in ...
PHP Warning: session_start(): Unable to clear session lock record in ...
PHP Warning: Unknown: error saving session to memcached: SUCCESS in Unknown on line 0
mc3 stats from memcached:
stats
STAT pid 4605
STAT uptime 26116746
STAT time 1623423688
STAT version 1.5.6
STAT libevent 2.1.8-stable
STAT pointer_size 64
STAT rusage_user 79124.237411
STAT rusage_system 229363.516381
STAT max_connections 1024
STAT curr_connections 6
STAT total_connections 717575493
STAT rejected_connections 0
STAT connection_structures 370
STAT reserved_fds 20
STAT cmd_get 62408739
STAT cmd_set 263762563
STAT cmd_flush 0
STAT cmd_touch 0
STAT get_hits 62374874
STAT get_misses 33865
STAT get_expired 73927
STAT get_flushed 0
STAT delete_misses 69381618
STAT delete_hits 62368677
STAT incr_misses 5084
STAT incr_hits 585249627
STAT decr_misses 0
STAT decr_hits 0
STAT cas_misses 0
STAT cas_hits 0
STAT cas_badval 0
STAT touch_hits 0
STAT touch_misses 0
STAT auth_cmds 0
STAT auth_errors 0
STAT bytes_read 2396633334387
STAT bytes_written 1068336598289
STAT limit_maxbytes 2147483648
STAT accepting_conns 1
STAT listen_disabled_num 0
STAT time_in_listen_disabled_us 0
STAT threads 4
STAT conn_yields 0
STAT hash_power_level 16
STAT hash_bytes 524288
STAT hash_is_expanding 0
STAT slab_reassign_rescues 1693964
STAT slab_reassign_chunk_rescues 97
STAT slab_reassign_evictions_nomem 0
STAT slab_reassign_inline_reclaim 85547
STAT slab_reassign_busy_items 10795
STAT slab_reassign_busy_deletes 0
STAT slab_reassign_running 0
STAT slabs_moved 34072
STAT lru_crawler_running 0
STAT lru_crawler_starts 16292401
STAT lru_maintainer_juggles 1224560001
STAT malloc_fails 0
STAT log_worker_dropped 0
STAT log_worker_written 0
STAT log_watcher_skipped 0
STAT log_watcher_sent 0
STAT bytes 29817169
STAT curr_items 2804
STAT total_items 211966175
STAT slab_global_page_pool 192
STAT expired_unfetched 33771541
STAT evicted_unfetched 0
STAT evicted_active 0
STAT evictions 0
STAT reclaimed 13256661
STAT crawler_reclaimed 20516908
STAT crawler_items_checked 4188684951
STAT lrutail_reflocked 3658
STAT moves_to_cold 41432828
STAT moves_to_warm 2713
STAT moves_within_lru 171
STAT direct_reclaims 0
STAT lru_bumps_dropped 0
END
This doesn't append everytime, so this isn't easy to reproduce.
Do you have some hints I could check?
Thanks by advance and have a good day!

How to debug php script that exits with 137 error?

I have long running worker that iterates over 5M records using batch processing. I use standard Laravel's function chunkById for this.
As long as i can see, i have not reached 200M of memory usage, which i can see in output of docker stats:
CONTAINER ID NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS
fd05760e5d96 case-place-partners_case-place-partners_app_1 21.71% 140.5MiB / 7.666GiB 1.79% 919MB / 103MB 113MB / 21.4MB 19
Additionally, i have memory_get_usage() and memory_get_usage(true) everywhere and i dont see numbers higher than 52428800.
Output of journalctl -k | grep -i -e memory -e oom:
Aug 19 09:28:41 mirokko-i3 kernel: Memory: 8023772K/8259584K available (12291K kernel code, 1319K rwdata, 3900K rodata, 1612K init, 3616K bss, 235812K reserved, 0K cma-reserved)
Aug 19 09:28:41 mirokko-i3 kernel: Freeing SMP alternatives memory: 32K
Aug 19 09:28:41 mirokko-i3 kernel: x86/mm: Memory block size: 128MB
Aug 19 09:28:41 mirokko-i3 kernel: Freeing initrd memory: 9024K
Aug 19 09:28:41 mirokko-i3 kernel: check: Scanning for low memory corruption every 60 seconds
Aug 19 09:28:41 mirokko-i3 kernel: Freeing unused decrypted memory: 2040K
Aug 19 09:28:41 mirokko-i3 kernel: Freeing unused kernel image memory: 1612K
Aug 19 09:28:41 mirokko-i3 kernel: Freeing unused kernel image memory: 2012K
Aug 19 09:28:41 mirokko-i3 kernel: Freeing unused kernel image memory: 196K
Aug 19 09:28:57 mirokko-i3 kernel: [TTM] Zone kernel: Available graphics memory: 4019344 KiB
Aug 19 09:28:57 mirokko-i3 kernel: [TTM] Zone dma32: Available graphics memory: 2097152 KiB
Output of docker inspect container_id located here
Seems like Laravel has job timeout. Just run worker with --timeout="0" to disable this feature or set your own value.

nginx forward proxy - failed (24: Too many open files),

I'm using php curl with nginx as a proxy. here is my code:
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,$url);
curl_setopt($ch, CURLOPT_PROXY, $proxy);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$curl_scraped_page = curl_exec($ch);
curl_close($ch);
echo $curl_scraped_page;
after sometime that this running the nginx load is extreamly slow and sometime it returns error 500.
the log says
failed (24: Too many open files),
some more details:
root#proxy-s2:~# ulimit -Hn
4096
root#proxy-s2:~# ulimit -Sn
1024
There is nothing else running on the server, and no other script is using this proxy.
Is it nginx bug? how to resolve it?
or
What else could it be? how can it be resolved?
I didn't change the default nginx configuration
Nginx restart solved the problem (temporarily I guess)
here is my nginx.conf
worker_processes 1;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
sendfile on;
keepalive_timeout 65;
gzip on;
server {
listen 8080;
location / {
resolver 8.8.8.8;
proxy_pass http://$http_host$uri$is_args$args;
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
}
}
top
top - 09:23:55 up 21:51, 1 user, load average: 0.09, 0.13, 0.08
KiB Mem: 496164 total, 444328 used, 51836 free, 12300 buffers
KiB Swap: 0 total, 0 used, 0 free. 336228 cached Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
8 root 20 0 0 0 0 S 0.0 0.0 4:57.56 rcuos/0
4904 nobody 20 0 97796 14128 1012 R 0.0 2.8 4:19.82 nginx
7 root 20 0 0 0 0 S 0.0 0.0 2:11.35 rcu_sched
3 root 20 0 0 0 0 S 0.0 0.0 0:18.50 ksoftirqd/0
832 root 20 0 139208 6808 172 S 0.0 1.4 0:13.11 nova-agent
45 root 20 0 0 0 0 S 0.0 0.0 0:06.21 xenbus
74 root 20 0 0 0 0 S 0.0 0.0 0:03.03 kworker/u30:1
155 root 20 0 0 0 0 S 0.0 0.0 0:02.73 jbd2/xvda1-8
46 root 20 0 0 0 0 R 0.0 0.0 0:02.39 kworker/0:1
57 root 20 0 0 0 0 S 0.0 0.0 0:01.91 kswapd0
1 root 20 0 33448 2404 1136 S 0.0 0.5 0:01.47 init
391 root 20 0 18048 1336 996 S 0.0 0.3 0:00.97 xe-daemon
1034 syslog 20 0 255840 2632 784 S 0.0 0.5 0:00.90 rsyslogd
1107 root 20 0 61364 3048 2364 S 0.0 0.6 0:00.73 sshd
40 root rt 0 0 0 0 S 0.0 0.0 0:00.29 watchdog/0
316 root 20 0 19472 456 252 S 0.0 0.1 0:00.12 upstart-udev-br
6 root 20 0 0 0 0 S 0.0 0.0 0:00.11 kworker/u30:0
1098 root 20 0 23652 1036 784 S 0.0 0.2 0:00.08 cron
7935 root 20 0 105632 4272 3284 S 0.0 0.9 0:00.07 sshd
330 root 20 0 51328 1348 696 S 0.0 0.3 0:00.06 systemd-udevd
7953 root 20 0 22548 3428 1680 S 0.0 0.7 0:00.05 bash
678 root 20 0 15256 524 268 S 0.0 0.1 0:00.04 upstart-socket-
8647 root 20 0 25064 1532 1076 R 0.0 0.3 0:00.03 top
mpstat
root#proxy-s2:~# mpstat
Linux 3.13.0-55-generic (proxy-s2) 07/09/2015 _x86_64_ (1 CPU)
09:22:17 AM CPU %usr %nice %sys %iowait %irq %soft %steal %guest %gnice %idle
09:22:17 AM all 0.94 0.00 1.63 0.16 0.00 2.16 0.92 0.00 0.00 94.20
iostat
root#proxy-s2:~# iostat
Linux 3.13.0-55-generic (proxy-s2) 07/09/2015 _x86_64_ (1 CPU)
avg-cpu: %user %nice %system %iowait %steal %idle
0.94 0.00 3.80 0.16 0.92 94.19
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
xvdc 0.01 0.02 0.00 1710 0
xvda 3.16 4.19 88.56 322833 6815612
Please try below ,do the following changes in your limits.conf.
vi /etc/security/limits.conf
For open files
soft nofile 64000
hard nofile 64000
For max user processes
soft nproc 47758
hard nproc 47758
For max memory size
soft rss unlimited
hard rss unlimited
For virtual memory
soft as unlimited
hard as unlimited
Just put this on atop of Nginx configuration file:
worker_rlimit_nofile 40000;
events {
worker_connections 4096;
}
I think I found the problem:
here is the nginx error.log
2015/07/09 14:17:27 [error] 15390#0: *7549 connect() failed (111: Connection refused) while connecting to upstream, client: 23.239.194.233, server: , request: "GET http://www.lgqfz.com/ HTTP/1.1", upstream: "http://127.0.0.3:80/", host: "www.lgqfz.com", referrer: "http://www.baidu.com"
2015/07/09 14:17:29 [error] 15390#0: *8121 connect() failed (111: Connection refused) while connecting to upstream, client: 204.44.65.119, server: , request: "GET http://www.lgqfz.com/ HTTP/1.1", upstream: "http://127.0.0.3:80/", host: "www.lgqfz.com", referrer: "http://www.baidu.com"
2015/07/09 14:17:32 [error] 15390#0: *8650 connect() failed (101: Network is unreachable) while connecting to upstream, client: 78.47.53.98, server: , request: "GET http://188.8.253.161/ HTTP/1.1", upstream: "http://188.8.253.161:80/", host: "188.8.253.161", referrer: "http://188.8.253.161/"
It was a DDOS attack on my PROXY that I stopped by allowing only my IP to access the PROXY.
I found it to be common lately - when u crawl a site, and the site identify you as a crawler, it will sometime DDOS attack your proxy until they go black.
One example of such site is amazon.com

How to get longer compile times in PHP

I was wondering how can I write a PHP script that needs a longer compile time?
I want to do this to test in the OPCache extension works.
Later edit:
When a PHP script is loading, the code is compiled into bytecode and this bytecode will be interpreted by CPU. The compilation process usually takes some milliseconds but I need to make this time extremely large to test the OPCache extension from PHP 5.5. This extension should cache the script bytecode so that it won't need to compile the script again.
As #PaulCrovella said in the comments, what I needed was ApacheBench.
By using the command ab http://localhost/index.php on a script with about 600.000 lines of code, the results were:
On the first benchmark test:
Server Software: Apache/2.4.9
Server Hostname: localhost
Server Port: 80
Document Path: /index.php
Document Length: 4927 bytes
Concurrency Level: 1
Time taken for tests: 0.944 seconds
Complete requests: 1
Failed requests: 0
Total transferred: 5116 bytes
HTML transferred: 4927 bytes
Requests per second: 1.06 [#/sec] (mean)
Time per request: 944.054 [ms] (mean)
Time per request: 944.054 [ms] (mean, across all concurrent requests)
Transfer rate: 5.29 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.0 0 0
Processing: 944 944 0.0 944 944
Waiting: 939 939 0.0 939 939
Total: 944 944 0.0 944 944
On the second benchmark test:
Server Software: Apache/2.4.9
Server Hostname: localhost
Server Port: 80
Document Path: /index.php
Document Length: 4927 bytes
Concurrency Level: 1
Time taken for tests: 0.047 seconds
Complete requests: 1
Failed requests: 0
Total transferred: 5116 bytes
HTML transferred: 4927 bytes
Requests per second: 21.28 [#/sec] (mean)
Time per request: 47.003 [ms] (mean)
Time per request: 47.003 [ms] (mean, across all concurrent requests)
Transfer rate: 106.29 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.0 0 0
Processing: 47 47 0.0 47 47
Waiting: 43 43 0.0 43 43
Total: 47 47 0.0 47 47

Not getting improved performance from Memcached

I have a prestashop based ecommerce website (GoDaddy Linux Centos VPS). I'm trying to cut down on what pingdom calls "wait time", the time it takes for my server to generate the html through php and mysql.
I believe I have memcached installed correctly, but I am not getting improved performance, when I have it enabled.
What's going on here?
STAT pid 3403
STAT uptime 3210
STAT time 1343508734
STAT version 1.4.4
STAT pointer_size 64
STAT rusage_user 0.045993
STAT rusage_system 0.162975
STAT curr_connections 10
STAT total_connections 31
STAT connection_structures 12
STAT cmd_get 4507
STAT cmd_set 1260
STAT cmd_flush 2
STAT get_hits 4491
STAT get_misses 16
STAT delete_misses 2
STAT delete_hits 2
STAT incr_misses 0
STAT incr_hits 0
STAT decr_misses 0
STAT decr_hits 0
STAT cas_misses 0
STAT cas_hits 0
STAT cas_badval 0
STAT auth_cmds 0
STAT auth_errors 0
STAT bytes_read 6144475
STAT bytes_written 21262077
STAT limit_maxbytes 536870912
STAT accepting_conns 1
STAT listen_disabled_num 0
STAT threads 4
STAT conn_yields 0
STAT bytes 147189
STAT curr_items 340
STAT total_items 1260
STAT evictions 0
Here's a recent pingdom test: http://tools.pingdom.com/fpt/#!/zdViZURr3/http://www.kikbo.com/buy
You have to configure prestashop to use memcache in the backoffice. Therefore simply go to the preferences->performance tab and scroll down. Set prestashop to use the cache and select memcache as caching option.
You then have to specify the memcache servers that should be used by prestashop. You should then see an immediate performance increase!
Unfortunately prestashop has some problems with memcache (information not getting refreshed,...). I wrote a blog article about that with some code to fix this: http://www.supnig.com/blog/caching-mysql-results-with-memcached

Categories