Zend_Search_Lucene crashes during indexing - php

I wanted to create search engine for my webpage, but during indexing on server it crashes with errors :
Warning: opendir(/admin/lucene/) [function.opendir]: failed to open dir: Too many open files in /admin/includes/Zend/Search/Lucene/Storage/Directory/Filesystem.php on line 159
Warning: readdir(): supplied argument is not a valid Directory resource in /admin/includes/Zend/Search/Lucene/Storage/Directory/Filesystem.php on line 160
Warning: closedir(): supplied argument is not a valid Directory resource in /admin/includes/Zend/Search/Lucene/Storage/Directory/Filesystem.php on line 167
Fatal error: Ignoring exception from Zend_Search_Lucene_Proxy::__destruct() while an exception is already active (Uncaught Zend_Search_Lucene_Exception in /admin/includes/Zend/Search/Lucene/Storage/File/Filesystem.php on line 66) in /admin/test.php on line 549
I am using newest version of ZF. Is there code solution for such error - I run script on localhost and it works great.
Thanks for any help.

It seems the problem is in the large number of segments in the index.
Could you check how much files does index folder contain?
There are two ways to solve this problem:
a) Optimize index more often.
b) Use another MaxBufferedDocs/MergeFactor parameters. See Zend_Search_Lucene documentation for details.
If it doesn't help, please register JIRA issue for the problem.

PHP has hit the limit on the number of files it can have open at once it seems might be an option to change in php.ini, could be an OS (quota) limit or you might be able to tell the indexer to slow down and not have so many files open simultaneously.

This is most definitely a Linux/kernel imposed limitation. Use the following command as root on your machine:
cat /proc/sys/fs/file-nr
Return values are defined as:
Total allocated file descriptors
Total free allocated file descriptors
Maximum open file descriptors
I'm also going to take a guess and say you are on a shared hosting machine. If this is the case, I imagine that this sort of issue may come up frequently.
Finally, the following article provides a good amount of information on Linux and open file descriptors even if it is a little dated.
http://www.netadmintools.com/art295.html

Related

Why is the path truncated / cut off in the PHP error log?

I have a WordPress web site running on Windows Server 2012 R2 via IIS and PHP 7.1.1. Everything works well, except once or twice a day, the site returns a white screen (HTTP 500) and the error log shows the following errors:
[17-Jan-2019 15:29:15 UTC] PHP Warning: require_once(D:\Webs\www.WebS): failed to open stream: No such file or directory in D:\Webs\www.WebSite.com\wp-content\plugins\captcha\bws_menu\bws_include.php on line 91
[17-Jan-2019 15:29:15 UTC] PHP Fatal error: require_once(): Failed opening required 'D:\Webs\www.WebSite.com/wp-content/plugins/captcha/bws_menu/bws_functions.php' (include_path='.;C:\php\pear') in D:\Webs\www.WebSite.com\wp-content\plugins\captcha\bws_menu\bws_include.php on line 91
FYI - I have changed the path off of D:\Webs to show a fictitious website name.
Notice that the path is cut off. Sometimes the path is cut off in different places. Sometimes it's to completely different files for completely different plugis. Sometimes it's even complaining about WordPress core. The only thing consistent is that once this happens, the exact same path is the failure until I recycle the App Pool. When it happens again, it's a different file, but complains about the same file until I recycle the App Pool.
What gives?
The only workaround I have is to recycle the app pool or set IIS to automatically recycle on a schedule. Neither of these are good long term solutions.
I should also note that this server has about 10 other WordPress sites running. They each have their own App Pool and run from their own folder. They never, ever have this problem.
Also, resources on the server are excellent. At the highest traffic times, the CPU's are less than 50% and there is less than 75% memory utilization. There are also many gigabytes of free disk space on the system volume as well as the data volume the site runs from.
Please don't reply with "don't run PHP or WordPress on Windows/IIS." That's not helpful - I cannot change the environment and I need to figure out the solution.
I have 100% control of the server environment and can debug/troubleshoot as needed. So if there's any way to get more information, please let me know!

MediaWiki 1.27 warning requiring SpecialUserLogin.php

I've updated my mediawiki from 1.26.2 to 1.27, the installation process finished ok, but when I try to access I received this error:
Warning:
require(/var/app/current/includes/specials/SpecialUserLogin.php):
failed to open stream: No such file or directory in
/var/app/current/includes/AutoLoader.php on line 81 Fatal error:
require(): Failed opening required
'/var/app/current/includes/specials/SpecialUserLogin.php'
(include_path='/var/app/current/vendor/pear/pear_exception:/var/app/current/vendor/pear/console_getopt:/var/app/current/vendor/pear/pear-core-minimal/src:/var/app/current/vendor/pear/mail_mime:/var/app/current/vendor/pear/mail_mime-decode:/var/app/current/vendor/pear/net_socket:/var/app/current/vendor/pear/net_smtp:/var/app/current/vendor/pear/mail:.:/usr/share/pear:/usr/share/php')
in /var/app/current/includes/AutoLoader.php on line 81
I have no idea why is this happening. If I check the files in my server they're there. I'm also having template issues if I choose vector I only get a messed up template, without styling.
I'm using PHP 5.6.
I hope someone can help me.
I just stumbled across this same exact error after upgrading to MW 1.27.
In my case, the SpecialUserlogin.php existed and all of the permissions were right BUT the word login was written in lowercase so the system thought this file didn't exist. So I just renamed the SpecialUserlogin.php to SpecialUserLogin.php and b00m, it worked!
As for your templating issues, check the common.css file. Copy paste everything out of there, so it's empty, if you don't use it. And check that you're calling your style files correctly in your template.

Named pipe in php on Windows/Mac/Linux?

I'm currently using proc_open() in php to communicate with command line tools. It automatically creates up to 3 file descriptors for STDIN, STDOUT and STDERR because more than 3 pipes breaks Windows compatibility. It works great on Windows XP and 10.7 on my Mac.
However, I would like to be able to create my own named pipes for better control. If I'm unable to do so, then I have to maintain references between the process resource and its stream resources, which is overly complicating my code. I can't use files because they fill up disk space for long tasks. I'm also trying to avoid using sockets because they aren't enabled by default in php.
Here are the links I have found so far:
http://bytes.com/topic/php/answers/557245-named-pipes-windows
https://bugs.php.net/bug.php?id=29005
Interprocess Communication using Named Pipes in C# + PHP
http://en.wikipedia.org/wiki/Named_pipe#In_Windows
http://www.phpkode.com/source/p/xp-framework/xp-framework-xp-framework-554d8b2/core/src/main/php/rdbms/mysqlx/NamedPipe.class.php
I've tried each of these in php 5.3.13 but none of them work:
var_dump(fopen("\\\\.\\pipe\\mypipe", "w+"));
var_dump(fopen("\\\\127.0.0.1\\pipe\\mypipe", "w+"));
var_dump(fopen("\\\\".php_uname('n')."\\pipe\\mypipe", "w+"));
I always get an error that looks like this:
PHP Warning: fopen(\\.\pipe\mypipe): failed to open stream: No such file or directory in C:\Documents and Settings\Administrator\Desktop\named-pipe.php on line 15
PHP Stack trace:
PHP 1. {main}() C:\Documents and Settings\Administrator\Desktop\named-pipe.php:0
PHP 2. fopen() C:\Documents and Settings\Administrator\Desktop\named-pipe.php:15
Warning: fopen(\\.\pipe\mypipe): failed to open stream: No such file or directory in C:\Documents and Settings\Administrator\Desktop\named-pipe.php on line 15
Call Stack:
0.0018 431936 1. {main}() C:\Documents and Settings\Administrator\Desktop\named-pipe.php:0
0.0018 432072 2. fopen() C:\Documents and Settings\Administrator\Desktop\named-pipe.php:15
bool(false)
So my question is, has anyone successfully opened a named pipe on Windows XP? If not, what versions of Windows support named pipes? I don't know if the pipe "file" descriptor needs to already exist, for example if it has to be created by mysql or other services.
Bonus points if you can supply cross-platform code, perhaps using posix_mkfifo() or system("mkfifo pipe") like in this answer:
http://www.php.net/manual/en/function.popen.php#22801
Ideally the solution should work with a vanilla install of php, so without sockets, etc, but anything helps thanks.
Have you realy read http://bytes.com/topic/php/answers/557245-named-pipes-windows ?
The solution is not to use dots in pipe's name.
All your samples are with dots in pipe's name.
Is this server code you are writing? It looks like you're using the client side calls perhaps you need to create the pipe first?
https://msdn.microsoft.com/en-us/library/windows/desktop/aa365590(v=vs.85).aspx
Not sure you can write the server in php, needs CreateNamedPipe().

PHP Fatal error on line number that doesn't exist

Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 523800 bytes) in /Library/WebServer/Documents/XMLDataStore.class.php on line 981
The curious thing about this error is not the memory leak, which would be easy enough to troubleshoot. Rather, it is the fact that XMLDataStore.class.php is only 850 lines long, which I have verified in multiple text editors.
This is with the PHP 5.3 bundled with Snow Leopard. I'm not using an opcode cache. Here is my php.ini:
allow_url_fopen = Off
error_reporting = -1
display_errors = 1
display_startup_errors = 1
date.timezone = 'America/Los_Angeles'
output_buffering = Off
realpath_cache_size = 0k
XMLDataStore.class.php has recently been refactored and it used to be longer than 981 lines. It's almost as if PHP has cached a 2-week-old version and is reading that. I'm positive that the current version at /Library/WebServer/Documents/XMLDataStore.class.php is only 850 lines long, though.
Could this be a line break issue? i.e. the PHP interpreter breaking lines differently than your IDE / editor? I don't know about how PHP handles Linux/Mac/Windows linebreaks, but it might be a possibility.
Can you create a fatal error somewhere in the script, and look which line number it shows you?
Could there be some over-long lines in your code (> 65535 characters) that mix up the line counting?
Also what happens if you rename the file, and include it under a new name? THis should take care of any screwed up cache issues.
PHP Seems to have issues with macintosh style line ending, It doesnt count the cr at the end of comments. I had this issue after opening a file on a friends apple.
I used kate on linux and Crimson Editor on Windows to change the end of line back to Unix style and the line numbers worked fine.
If you are getting an error on a non-existant line then it may very well be that PHP is caching it.
Maybe try renaming it then executing it.
Not to sure about Snow Leopard, but I am using it right now and have to say there are a great deal of strange bugs with the OS alone, could be that.
I experienced this error before when I was accessing a DB and storing info in an array, turns out I forgot how large the DB was and when it passed MB on the stack I got that error and it stopped.
If the situation is similar use a more processor heavy approach, such as connect, get a line/ block/ whatever, to something, then go back.
Don't go through everything, store it all, then do something.
Check this out:
Drupal.org memory allocation error
Hope this helps, not sure how clear all of this is.
Maybe show parts of your script to get an idea of what might be causing this.
Here is some more info. I tried the same code on a different Mac running the same version of Snow Leopard - one that is rarely used for development and would not have a version of this file cached - same result.
Next I tried copying the same code to a Solaris box running PHP 5.2.8. I still get the memory error - which is fine and expected, of course, since it does exist - but the PHP error message says this instead:
Fatal error: Maximum function nesting level of '100' reached, aborting! in /export/www/htdocs/XMLDataStore.class.php on line 741
This makes perfect sense, as that is the first line of a method that is being called recursively by another method. Exact same code with same line breaks (LF) checked out at the same revision # from the same SVN repository on all 3 machines.
Bug in PHP 5.3.0 for Snow Leopard? :)
While I have no idea what may cause this wrong line counting (I'm using PHP 5.3 on OSX 10.6, too. No Problems here.), you can narrow down the actual error by dividing your script into smaller parts.
that way maybe you'll find the real location of your error.
And a 850 line PHP file could seriously use some refactoring anyway.
I was experiencing this issue as well recently, in my case (PHP 5.3 on Ubuntu/Nginix under php-fpm5) I had a script that was causing a 500 error with no output (due to it being on production). When examining the error log it mentioned line 589 on index.php which only had lines going up to 453. The solution in my case was support for the short tag was disabled on production but enabled for staging...
ie. <? was disabled and <?php was required for opening PHP tags.
The error in the log was obviously not helpful in my case so it took a while for me to troubleshoot it. I am leaving this suggestion here hoping it will save someone some time.

HTML tables to PDF in PHP - neither DOMPDF nor html2ps/pdf are working

For the longest time now I've been trying to convert HTML pages containing large tables to PHP. These are styled with CSS and can be several pages long.
I first tried DOMPDF. It works great, until a document is more than one page. None of the fixes I've found work. Either it errors out, or any element that would be even partially on the second page gets lumped over the content of page 1. I've tried both the latest release and the SVN copy I checked out today.
I also tried html2ps/pdf, and it gives me "permission denied" errors writing to the cache directory, even though the directory is fully writable. The system requirements test script passes. The server is running PHP 5 under IIS (not my choice heh).
Can either of these be convinced to work, or is there a library that can do it?
Errors from the demo/test scripts:
Warning: file_put_contents(C:\Inetpub\wwwroot\JM\pdf\html2ps\cache/7d888258e9745b3716532ea81342daaf.css.compiled) [function.file-put-contents]: failed to open stream: Permission denied in C:\Inetpub\wwwroot\JM\pdf\html2ps\css.cache.class.php on line 33
Warning: fopen(C:\Inetpub\wwwroot\JM\pdf\html2ps\cache/unicode.lb.classes.dat) [function.fopen]: failed to open stream: Permission denied in C:\Inetpub\wwwroot\JM\pdf\html2ps\inline.content.builder.php on line 991
Warning: flock() expects parameter 1 to be resource, boolean given in C:\Inetpub\wwwroot\JM\pdf\html2ps\inline.content.builder.php on line 992
Warning: fwrite(): supplied argument is not a valid stream resource in C:\Inetpub\wwwroot\JM\pdf\html2ps\inline.content.builder.php on line 1011 (this error repeats several thousand times)
Warning: fopen(C:\Inetpub\wwwroot\JM\pdf\html2ps\cache/times.z) [function.fopen]: failed to open stream: Permission denied in C:\Inetpub\wwwroot\JM\pdf\html2ps\pdf.fpdf.makefont.php on line 318
Can't write to file C:\Inetpub\wwwroot\JM\pdf\html2ps\cache/times.z
A similar problem with html2pdf under IIS has been discussed here.
The solution was quite simple, yet weird. If this doesn't help you, a full error-message and / or a little bit of code would help me to help you.
I personally recommend command line applications instead of any PHP libraries.
Reasons :
PHP libraries need more time and memory (cache) for conversion process.
They need well-formatted HTML pages only, otherwise throw errors or warnings.
No support for external style sheets.
Command Line Tool:
If you run your script on a Linux server then I suggest command line tools.
Reasons :
They are extremely fast compared to PHP libraries.
Support CSS.
Accept non-well-formatted HTML.
Which command line tool to use?
wkhtmltopdf
htmltopdf
html2pdf
For more information refer to Converting HTML to PDF (not PDF to HTML) using PHP.

Categories