Force a statement to visually write to a file slowly - php

I have a want to take a File.open('somefile', 'w+') and make it read a file, take one line of text at a time, and visually write it slowly in another file. The reason I ask this question is because I can find nothing that does this already in code, nor can I find anything that actually controls the speed of how fast a program writes on the computer. I know that this can be simulated in a program such as Adobe Aftereffects so long as you provide a cursor after a character and the visual effect doesn't take place too quickly, but I've got 4,000 lines of code that I want to iterate over and can't afford to do this manually. This effect can also be achieved with a Microsoft Macro, but this requires it to be manually entered into the macro with no option of copy and paste.
-solutions preferred in Python, Ruby, and PHP-

If I understood properly, what you are trying to achieve, here you go:
input = File.read('readfrom.txt', 'r')
File.open('writeto.txt', 'w+') do |f|
input.chars.each do |c|
f.print(c) # print 1 char
f.flush # flush the stream
sleep 1 # sleep
end
end

This is one quick and dirty way of doing it in Python.
from time import sleep
mystring= 'My short text with a newline here\nand then ensuing text'
dt = 0.2 #0.2 seconds
for ch in mystring:
with open('fn_out','w+') as f:
f.write(ch)
f.flush()
sleep(dt)
f.flush() will result in updating the file with the changes.
One could make this more elaborate by having a longer pause after each newline, or a variable timestep dt.
To watch the change one has to repeatedly reload the file, as pointed out by #Tom Lord so you could run something like this beforehand to watch it in the terminal:
watch -n 0.1 cat fn_out

After some serious testing, I have finally developed a piece of code that will do the very thing I want. Tom Lord gave me some new words to use in my search terms "simulate typing" and this led me to win32ole with its SendKeys function. Here is a code that will iterate over all the characters in a file and print them out exactly as they were saved while simulating typing. I will see about making this into a gem for future use.
require 'win32ole'
wsh = WIN32OLE.new("WScript.Shell")
wsh.Run("Notepad.exe")
while not wsh.AppActivate("Notepad")
sleep 1
end
def fileToArray(file)
x = []
File.foreach("#{file}") do |line|
x << line.split('')
end
return x.flatten!
end
tests = fileToArray("readfrom.txt")
x = 0
while x <= tests.length
send = tests[x]
wsh.SendKeys("#{send}")
x += 1
sleep 0.1
end

Related

getting result from Vowpal Wabbit daemon mode

I am running VW in daemon mode. As a standalone executable, it runs perfectly fine. In daemon mode, I see something about predictions and options initially but not the end result. Not sure what exactly is going on.
This is how I call VW6
/bin64/vw --daemon --num_children 2 -t -i ~/modelbow.vw6 --min_prediction 0 --max_prediction 1 -p stdout 2>&1
I check vw6 is running fine. I send data using simple php script (removed debug lines for brevity):
$fp = fsockopen("localhost",26542, $errno, $errstr, 3);
$fp_dat = fopen("/tmp/ml.dat", "r");
$mldata = explode("\n", file_get_contents("/tmp/ml.dat"));
$mlstr = implode($mldata);
fwrite($fp, $mlstr);
$result = trim(fgets($fp, 1024));
print $result;
Print $result above prints nothing. The only thing I see in stdout is
num sources = 1
Num weight bits = 28
learning rate = 10
initial_t = 1
power_t = 0.5
decay_learning_rate = 1
predictions = stdout
only testing
average since example example current current current
loss last counter weight label predict features
While in a standalone executable mode, if I run with the same model same dat file just without the -daemon option, it happily gives a result at the end
...
...
predictions = stdout
only testing
average since example example current current current
loss last counter weight label predict features
1.000000 ba66dfc7a135e2728d08010b40586b90
Any idea what could be going wrong here with the daemon mode? I tried using -p /tmp/ option as well...ran the daemon mode with sudo but nothing helped. Is there a debug dump option or verbose option or something else to know what exactly is going on?
thanks
The reason it is not working is not in vw but in the PHP client code.
explode on "\n", strips the newlines out.
implode without a glue-string parameter results in glue-string defaulting to the empty string.
Result: newlines are stripped out.
All examples are merged into one big (and incomplete, since there's no newline at the end) example.
vw needs newlines to separate examples, without them it will be waiting forever for the 1st example to complete.
So I think you need to change the implode line of code to:
$mlstr = implode("\n", $mldata);
for it to work.
You will also need an additional ending newline to get the last line through.

PHP exec very slow processing simple 3-pipe grep

I've read here and cannot really understand how to speed up my simple exec() which basically looks like this:
zcat access_log.201312011745.gz | grep 'id=6' | grep 'id2=10' | head -n10
I've added ini_set('memory_limit', 256); to the top of the PHP document, but the script still takes about 1 minute to run (contrasted with about near instant completion in Penguinet). What can I do to improve it?
I would try some of the following:
Change your exec to just run somethig simple, like
echo Hello
and see if it still takes so long - if it does, the problem is in the process creation and exec()ing area.
If that runs quickly, try changing the exec to something like:
zcat access_log.201312011745.gz > /dev/null
to see if it is the "zcat" slowing you down
Think about replacing the greps with a "sed" that quits (using "q") as soon as it finds what you are looking for rather than continuing all the way to end of file - since it seems (by your "head") you are only interested in the first few, not all occurrences of your strings. For example, you seem to be looking for lines that contain "id=6" and also "id2=10", so if you used "sed" like below, it may be faster because "sed" will print it and stop immediately the moment it finds a line with "id=6" followed by "id2=10"
zcat access_log.201312011745.gz | sed -n '/id=2.*id2=10/p;q'
The "-n" says "don't print, in general" and then it looks for "id=2" followed by any characters then "id2=10". If it finds that, it prints the line and the "q" makes it quit immediately without looking through to end of file. Note that I am assuming "id=2" comes before "id2=10" on the line. If that is not true, the "sed" will need additional work.

completely deleting a file from server

I want to delete a file by using PHP. I have used the unlink() function, but I was wondering about the security of unlink. Is the file completely deleted from the server? I want to make sure that there is no way to get the file back and the file is completely removed from the server.
open the file in binary mode for writing, write 1's over the entire file, close the file, and then unlink it. overwrites any data within the file so it cannot be recovered.
Personally i would say use 1's instead of 0's as 1's are actual data and will always write, where as 0's may not write, depending on several factors.
Edit: After some thought, and reading of comments, i would go with a hybrid approach, depending on "how deleted" you want the file to be, if you simply wish to make it so the data cannot be recovered, overwrite the entire files length with 1's as this is fast, and destroys the data, the problem with this, is it leaves a set length of uniform data on the disk which infers a file USED to be there and gives away the files length, giving vital pieces of forensic information. Simply writing random data will not avoid this also, as if all the drive sectors around this file are untouched, this will also leave a forensic trace.
The best solution factoring in forensic deletion, obfuscation and plausible deniability (again, this is overkill, but im adding it for the sake of adding it), overwrite the entire length of the file with 1's and then, for HALF the length of the file in bytes, write from mt_rand in random length sizes, from random starting points, leaving the impression that many files of varying lengths used to be in this area, thus creating a false trail. (again, this is completely overkill and is generally only needed by serial killers and the CIA, but im adding it for the sake of doing so).
the US government used to recommend a seven step wipe, for disks.
1) all '1's
2) all '0's
3) the pattern '01'
4) the pattern '10'
5) a random pattern
6) all '1'
7) a random pattern,
re the code sample, using a language like PHP is wrong for this type of wipe as your relaying on the OS really wipeing the file and not doing something cleaver like only wipeing it the last time or just unlinking it, however...
(untested)
$filename = "/usr/local/something.txt";
$size = filesize($filename);
$pat1 = chr(0);
$pat2 = chr(255);
$pat3 = chr(170);
$pat4 = chr(85);
$mask = str_repeat($pat1, $size);
file_put_contents($filename, $mask);
$mask = str_repeat($pat2, $size);
file_put_contents($filename, $mask);
$mask = str_repeat($pat3, $size);
file_put_contents($filename, $mask);
$mask = str_repeat($pat4, $size);
file_put_contents($filename, $mask);
This might not answer HOW to perfectly delete a file "with PHP", but it answers your question: "Is the file completely deleted from the server ?"
In some cases, No! (on UNIX/POSIX OS).
According to the highest voted comment on the offical PHP unlink() manual page, the unlink function does not really delete the file, it's deleting the system link to the file's content ! As files can have several files names (!) [symlinks?] the file will only be deleted when ALL file names are unlinked. So, if your file has 2 names, then unlink() will not really delete the file unless you unlink() both file names. Dear linux guys, please correct me here if necessary.
This might be why the function is called unLINK() and not delete() !!!
Here a full quote of the excellent comment:
Deleted a large file but seeing no increase in free space or decrease of disk usage? Using UNIX or other POSIX OS? The unlink() is not about removing file, it's about removing a file name. The manpage says: `unlink - delete a name and possibly the file it refers to''. Most of the time a file has just one name -- removing it will also remove (free, deallocate) thebody' of file (with one caveat, see below). That's the simple, usual case.
However, it's perfectly fine for a file to have several names (see the link() function), in the same or different directories. All the names will refer to the file body and keep it alive', so to say. Only when all the names are removed, the body of file actually is freed. The caveat: A file's body may *also* bekept alive' (still using diskspace) by a process holding the file open. The body will not be deallocated (will not free disk space) as long as the process holds it open. In fact, there's a fancy way of resurrecting a file removed by a mistake but still held open by a process...
Have a look on unlink()'s sister function link() here.
The (imo) best way to delete a file via PHP:
The way to go to really delete a file with PHP (in linux) is to use the exec() function, which executes real bash commands (doing things with linux bash feel correct btw). In this case, the file test.jpg would be deleted by doing:
exec("rm test.jpg);
More info on how to use rm (remove) correctly can be found for example here. Please note: PHP needs the right to delete the file!
UPDATE: Unfortunatly, the linux rm command ALSO does not really delete the file if it has two names/links. Look here for more info.
I'll have a deeper research on that and give feedback...
It is possible that because of some fragmentation on the disk some parts of file will stay, even if the file is totally overwritten.
The other way is to run (by shell_exec()) external program, that is system specific. Here is an example (for Windows), however I have not tested it.
You should do multiple passes of overwriting to deminish traces. For instance using the US DoD 5220-22.M : "Overwrite all addressable locations with a character, its complement, then a random character and verify" (from killdisk site)
Here's what the EFF recommends to permanently remove a file http://ssd.eff.org/tech/deletion.
In my embedded Ubuntu device, I use: echo exec('rm /usr/share/subdirectory/subdirectory/filename'); This works for me.
if you use rm -f (--force) then linux will
ignore nonexistent files and arguments, never prompt
rm -d will
remove empty directories
If you enter rm --help at the prompt you get the help screen. The last lines read:
Note that if you use rm to remove a file, it might be possible to recover some of its contents, given sufficient expertise and/or time. For greater assurance that the contents are truly unrecoverable, consider using shred.
Since my system is a "closed" system then I'm not concerned about violating security issues. My logic being that one must have the system password to SSH into the OS and the only user interface is via web pages.
#Sliq's comments are still true to date. You need to decide for your case.

PHP CLI - get user input while still doing things in background

I'm working on a game, written in PHP and that runs in a console. Think back to old MUDs and other text-based games, even some ASCII art!
Anyway, what I'm trying to do is have things happening while also accepting user input.
For instance, let's say it's a two player game and Player 1 is waiting for Player 2 to make a move. This is easily done by just listening for a message.
But what if Player 1 wants to change some options? What if they want to view details on aspects of the game state? What about conceding the game? There are many things a Player may want to do while waiting for their opponent to make a move.
Unfortunately the best I have right now is the fact that Ctrl+C completely kills the program. The other player is then left hanging, until the connection is dropped. Oh, and the game is completely lost.
I get user input with fgets(STDIN). But this blocks execution until input has been received (which is usually a good thing).
Is it even possible for a console program like this to handle input and output simultaneously? Or should I just look at some other interface?
In short PHP is not built for this, but you might get some help from one of these extensions. I'm not sure how thorough they are, but you really probably want to use a text UI library. (And really you probably do not want to use PHP for this.)
All that said, you need to get non blocking input from STDIN character by character. Unfortunately most terminals are buffered from PHP's point of view, so you won't get anything until enter is pressed.
If you run stty -icanon (or your OS's equivalent) on your terminal to disable buffering, then the following short program basically works:
<?php
stream_set_blocking(STDIN, false);
$line = '';
$time = microtime(true);
$prompt = '> ';
echo $prompt;
while (true)
{
if (microtime(true) - $time > 5)
{
echo "\nTick...\n$prompt$line";
$time = microtime(true);
}
$c = fgetc(STDIN);
if ($c !== false)
{
if ($c != "\n")
$line .= $c;
else
{
if ($line == 'exit' || $line == 'quit')
break;
else if ($line == 'help')
echo "Type exit\n";
else
echo "Unrecognized command.\n";
echo $prompt;
$line = '';
}
}
}
(It relies on local echo being enabled to print the characters as they are typed.)
As you see, we are just looping around forever. If a character exists, add it to the $line. If enter is pressed, process $line. Meanwhile, we are ticking every five seconds just to show that we could be doing something else while we wait for input. (This will consume maximum CPU; you'd have to issue a sleep() to get around that.)
This isn't meant to be a practical example, per se, but perhaps will get you thinking in the proper direction.
It is possible to build a game like you describe using ncurses (non-blocking mode) and libevent. That way, you get close to no CPU consumption. Handling individual keys is sometimes awkward (implement Backspace yourself, it's not fun at all - and did you know various OSes send different keycodes on Backspace press?), and gets really tricky if you want to support UTF-8 properly. Still, completely viable.
In particular, it is beneficial to make extensive use of libevent, by reading both the network and keyboard (stdin) input with it. This function enables you to listen for individual keys:
http://www.php.net/manual/en/function.ncurses-cbreak.php
which you can later read using libevent API. The key to keep in mind is that you will sometimes end up reading more than 1 key at a time, and it has to be handled (so loop over everything that you have read). Otherwise, the user will be annoyed to see that not all key presses are "reaching" the application and some are lost.
Sorry Matthew, I'm going to have to un-accept your answer, because I have found it myself:
Use the following code to receive user input while still doing something else:
while(/* some condition that the code running is waiting on */) {
// perform one step or iteration of that code
exec("choice /N /C ___ /D _ /T _",$out,$ret);
// /C is a list of letters that do something
// /D is the default action that will be used as a no-op
// /T is the amount of time to wait, probably best set to one second
switch($ret) {
// handle cases - the "default" case should be "continue 2"
}
}
This can then be used to interrupt the loop and enter an options menu, or trigger some other event, or could even be used to type out a command if used right.

How to keep file to 1000 lines with Linux or PHP?

I have a file that I'm using to log IP addresses for a client. They want to keep the last 500 lines of the file. It is on a Linux system with PHP4 (oh no!).
I was going to add to the file one line at a time with new IP addresses. We don't have access to cron so I would probably need to make this function do the line-limit cleanup as well.
I was thinking either using like exec('tail [some params]') or maybe reading the file in with PHP, exploding it on newlines into an array, getting the last 1000 elements, and writing it back. Seems kind of memory intensive though.
What's a better way to do this?
Update:
Per #meagar's comment below, if I wanted to use the zip functionality, how would I do that within my PHP script? (no access to cron)
if(rand(0,10) == 10){
shell_exec("find . logfile.txt [where size > 1mb] -exec zip {} \;")
}
Will zip enumerate the files automatically if there is an existing file or do I need to do that manually?
The fastest way is probably, as you suggested, to use tail:
passthru("tail -n 500 $filename");
(passthru does the same as exec only it outputs the entire program output to stdout. You can capture the output using an output buffer)
[edit]
I agree with a previous comment that a log rotate would be infinitely better... but you did state that you don't have access to cron so I'm assuming you can't do logrotate either.
logrotate
This would be the "proper" answer, and it's not difficult to set this up either.
You may get the number of lines using count(explode("\n", file_get_contents("log.txt"))) and if it is equal to 1000, get the substring starting from the first \n to the end, add the new IP address and write the whole file again.
It's almost the same as writing the new IP by opening the file in a+ mode.

Categories