I wrote a PHP script to pull tweets from the Twitter firehose and store them into a database. Ideally I want to just let it run so that it collects tweets over time, thus, it's wrapped in a while(1) loop.
This seems to be problematic because it's timing out. If I just run it in a browser, it won't run for more than 30 seconds before timing out and giving me a 324 Error.
Question: Is there a way that I can have it run for a certain amount of time (20 seconds), auto kill itself, then restart? All in a cron job (PS...I don't know how to write a cron job)?
Background: Site hosted on Godaddy. Would ideally like to run this on my hosting server there.
The Script:
<?php
$start = time();
$expAddress = "HOSTNAME";
$expUser = "USERNAME";
$expPwd = "PASSWORD";
$database = "DBNAME";
$opts = array(
'http' => array(
'method' => "POST",
'content' => 'keywords,go,here',
)
);
// Open connection to stream
$db = mysql_connect($expAddress, $expUser, $expPwd);
mysql_select_db($database, $db);
$context = stream_context_create($opts);
while (1) {
$instream = fopen('https://USERNAME:PASSWORD#stream.twitter.com/1/statuses/filter.json','r' ,false, $context);
while(! feof($instream)) {
if(time() - $start > 5) { // break after 5 seconds
break;
}
if(! ($line = stream_get_line($instream, 100000, "\n"))) {
continue;
}
else {
$tweet = json_decode($line);
// Clean before storing
// LOTS OF VARIABLES FOR BELOW...REMOVED FOR READABILITY
// Send to database
$ok = mysql_query("INSERT INTO tweets
(created_at, from_user, from_user_id, latitude, longitude, tweet_id, language_code,
place_name, profile_img_url, source, text, retweet_count, followers_count,
friends_count, listed_count, favorites_count)
VALUES
(NOW(), '$from_user', '$from_user_id', '$latitude', '$longitude', '$tweet_id', '$language_code',
'$place_name', '$profile_img_url', '$source', '$text', '$retweet_count', '$followers_count',
'$friends_count', '$listed_count', '$favorites_count')");
if (!$ok) { echo "Mysql Error: ".mysql_error(); }
flush();
}
}
}
?>
You can have cron jobs run once a minute.
To do this follow these steps:
Make a script that runs your PHP code, for example:
#!/bin/bash
wget myurl.com/blah > /dev/null
Save it as my-cron.sh in some folder (like /var)
Add it to cron. Run crontab -e See Cron Format and Crontab usage.
This for example, will run it once a minute.
# Minute Hour Day of Month Month Day of Week Command
* * * * * /var/my-cron.sh
If I get well your need, the best thing for you is to use cron job making a script run indefinitely will not be a good idea.
As specifier in one of you comments you are using a hosting server Godaddy so probably you will not be able to have shell access, BUT depending on your cPanel version you may be able to create and define cron job.
see this link and this google search
Perhaps, if you don't have this option and you are wiling to let a browser opened I would suggest the following
create an html page as a client which would make an ajax request every hours to your PHP script, like this you emulate a cron job function
the ajax request code might look like (using jQuery)
function makeRequest(){
$.ajax({
url: "http://yourhost/url-to-your-script.php",
complete: function(data){
setTimeout(function(){
makeRequest();
}, 60 * 60 * 1000); // Minutes * Seconds * MS
}
});
}
makeRequest();
I hope this helps
EDIT
this link might help too
IMPORTANT DO NOT FORGET TO REMOVE THE INFINITE LOOP
I just had same issue.
Only cron job can do if you want run script off browser. You can set up cron job with free providers or you can set up cron job in windows's Scheduled tasks.
If your site has a good traffic then you can follow the option below that your users does the work for you.
In php you can find time in hour and seconds
$time= date(' H:i:s');
create a table to track if the code was run.
eg; table column name check with option 0 and 1;
select check from table.
enter code here
if ($minute > 59)
{
if($check==0)
{
run your code
then update the table each time when it was run
eg; update table set check='1'
}
}
then another if condition to reset your code
if(minute>0 && minute <1)
{
select check from your table.
if(check==1)
{
update table set check='0'
}
}
Related
I'm programming my own "smart home" as a learning project.
My code is running fine. I'm looking for help to improve the efficiency and of the code and/or the setup of crontab + php code.
I'm monitoring the energy consumption of my washing machine with a WIFI energy meter. Target is to notify me once the washing machine is completed so I don't forget to clear it.
on my Pi I have a crontab like so:
*/20 7-22 * * * /usr/bin/php '/home/holger/html/plugs/washer.php'
which runs following php code (I simplified for better readability):
[...]/I call the function, of course, but this function does the main task
function loop($maschine, $watt_init, $trashhold){
$max = 75;//max loops to avoid endless runs
$i = 1;//start counter
$tackt = 3;//tact time to check energy consumption
//$trashhold = 4;//ab x Watt kein standby
if ($watt_init < 1 ) {//Machine is switched off if energy consumption < 1 Watt
die;//quit
}
elseif ($watt_init < 2 ) {//Machine is switched off or in standby if energy consumption < 1 Watt
die;//quit
}
else {//Any thing else: Machine is running
while ($i < $max) {//loop as long as max loops are not reached
$watt_current = json_combine(IPplug5);//getting current energy consumption from WIFI energy meter via JSON
sleep(60*$tackt);//sleep and continue every 60s x tact time
$i++;//increase counter +1
//compare actual consumption with defined trashhold
if ($watt_current[0] >= $trashhold) {//continue while energy consumption bigger then trashhold
continue;//repeat loop
}
elseif ($watt_current[0] < $trashhold) {//stop if energy consumption lower then trashhold
break;//stop loop
}
}
echo "Program done. please clear. Runtime: " . $i*$tackt. "Min."
//[...] message me to my telegram bot
}
}
The code is running fine and I'm getting the output I need.
My question is: Is there a better way to do that?
Currently I'm afraid to overload my Pi with too many open php sessions, therefore I'm starting the code only every 20min and also let the while loop sleep for 3 Min. But for improved accuracy I like to run the cronjob more often and also let the while loop sleep only for 30s.
My requirements are to stick to my PI and php code and not to use any available software like Home Assisant.io as it contradicts with my learning approach.
Any ideas or insights welcome.
Ideally it's a not the best approach to handling and measuring power consumption. It would be best if you created an API that accepts events like on/off or threshold hold limit extends from your IP devices. Further you can create logs and store them in databases.
Although, for your current problem here is one alternate solution.
Set your cron that runs every second.
function get_powerConsumption($machine, $watt_init, $threshold)
{
if ($watt_init < 2) {
exit();
}
$time = date("Y-m-d H:i");
$filename = $machine . '_power_consumption.log'; // expecting some machine identification name here. otherwise ignore prefix
$watt_current = json_combine(IPplug5);
if ($watt_current[0] >= $threshold) {
$data = array(
$time,
$watt_current[0]
);
file_put_contents($filename, json_encode($data) . "\n", FILE_APPEND);
} elseif ($watt_current[0] < $threshold) {
$data = array(
$time,
'stopped'
);
file_put_contents($filename, json_encode($data) . "\n", FILE_APPEND);
}
}
Create another cron to look up for stopped events logged in the file. if found process the calculation based on logged data like time and consumption. You can set this cron to run based on your need like every second or minute or after some interval.
Also, handle code to delete old logs, once stopped events found.
I have a mysql table and I want it to be "emptied" every night at midnight. I have searched for an answer on the web and came across nothing that seemed to help me. I had this idea of using javascript to get the current time and then run an if statement and see if it is equal to midnight and if it was to execute a php script that deleted the information.
Javascript:
var myVar=setInterval(function(){myTimer()},1000);
function myTimer()
{
var d=new Date();
var t=d.toLocaleTimeString();
if(t == 12:00:00 AM){
$.ajax({
URL: 'delete.php';
});
};
};
delete.php:
<?php
require 'connect.php';
mysql_query("DELETE * FROM messages;");
?>
I have tested this by setting the time in the if statement to a time a few minutes ahead of my actual time and it does not work.
Implementing your own event scheduler, especially as a web page using JavaScript is a bad idea.
Use for that either
a cron job to run DELETE statement through the mysql command line interface
/path/to/mysql -u<user> -p"<password>" <db_name> -e "delete from messages"
or a MySQL event, e.g.
CREATE EVENT delete_messages_at_midnight
ON SCHEDULE EVERY 1 DAY STARTS CURDATE() + INTERVAL 1 DAY
DO DELETE FROM messages;
If you go with MySQL event approach:
use SHOW PROCESSLIST to check if the event scheduler is enabled. If it's ON you should see a process "Daemon" by user "event_scheduler".
use SET GLOBAL event_scheduler = ON;to enable the scheduler if it's currently not enabled.
More on configuring event scheduler read here
I usually update my site "by hand", entering in one page called "enterheretoupdate.php". This page refreshes every minute to do all the job I need, so while this page is "open", my site keeps refreshing every minute.
What does "enterheretoupdate.php" do? It makes things related to mysql: create tables, selects from tables, add rows to tables, etc. Apart from that, it also make calculations on php and updates .json files.
I would like to create a cron job, so that it is not necessary for me to visit "enterheretoupdate.php" on my computer for updating my site every minute.
I am quite new on this, but I have learned how to create a cron job (I use 1and1). The example cron job I have created, consisting on sending an email every minute, works fine.
But then, I tried to save "enterheretoupdate.php" as a cron job and it does not work. Is there a "limitation" on the things a cron job can do? How should I "translate" my php file to make it work as a cron job?
Any help is really welcome.
This is how my .php file looks like:
<?php
$page = $_SERVER['PHP_SELF'];
$sec = "10";
//Change 1 to reload, 0 to not to reload;
$reload=1;
$gamecode=7;
$cmp="EL";
$year=2017;
if ($reload==1) echo"<head><meta http-equiv='refresh'content=".$sec.";URL='".$page."?gamecode=".$gamecode."&cmp=".$cmp."&year=".$year."'></head>";
include("../newcon.php");
include("../formulas.php");
include_once("funLightCreateTables.php");
include_once("funLightFirstFive.php");
include_once("funLightChanges.php");
include_once("funLightLiveJsons.php");
if ($cmp=="EC") {$l="U";}
if ($cmp=="EL") {$l="E";}
//Check
$q="SELECT * FROM LightLiveSchedule WHERE year=".$year." and cmp=".$cmp." and gamecode=".$gamecode."";
$res=mysqli_query($link,$q);
while ($r=mysqli_fetch_assoc($res)){
$started=$r['started'];
}
if ($started==0){
LightCreateTables($cmp,$year,$gamecode);
$q="UPDATE LightLiveSchedule SET started=1 WHERE year=".$year." and cmp=".$cmp." and gamecode=".$gamecode."";
mysqli_query($link,$q);
}
//Read
$pbp=file_get_contents("http://thesite.com/data.json?gamecode=".$gamecode."&seasoncode=".$l.$year."");
$pbp = json_decode($pbp,true);
//Insert
mysqli_query($link,"Truncate P_Live_Temp_".$cmp."_".$year."_".$gamecode."");
$lres=0;
$vres=0;
$n=0;
for ($i=0;$i<=4;$i++){
$nplays[$i]=count($pbp[$qtitle[$i]]);
$ii=0;
for ($j=0;$j<=$nplays[$i];$j++){
//change results
if ($pbp[$qtitle[$i]][$ii]['PUNTOS_A']!=null) {
$lres=$pbp[$qtitle[$i]][$ii]['PUNTOS_A'];
}
if ($pbp[$qtitle[$i]][$ii]['PUNTOS_B']!=null) {
$vres=$pbp[$qtitle[$i]][$ii]['PUNTOS_B'];
}
//clean
if (strpos($pbp[$qtitle[$i]][$ii]['CSDESCWEB'],"(")==0) {$play=$pbp[$qtitle[$i]][$ii]['CSDESCWEB'];}
if (strpos($pbp[$qtitle[$i]][$ii]['CSDESCWEB'],"(")>0) {$play=substr($pbp[$qtitle[$i]][$ii]['CSDESCWEB'],0,strpos($pbp[$qtitle[$i]][$ii]['CSDESCWEB'],"(")-1);}
//count
$points=0;
if ($play=="Three Pointer") {$points=3;}
if ($play=="Two Pointer" or $play=="Lay Up" or $play=="Dunk") {$points=2;}
if ($play=="Free Throw In") {$points=1;}
//ntconsole=00:00 at End Game
if ($play=="End Game") {$pbp[$qtitle[$i]][$ii]['NTCONSOLA']="00:00";}
//insert
$q="INSERT INTO P_Live_temp_".$cmp."_".$year."_".$gamecode."
(orden,shteam,shloc,shvis,quarter,minute,ntconsole,pcode,play,locres,visres,points)
VALUES
(".$n.",'".$pbp[$qtitle[$i]][$ii]['NTEQUIPO']."','".$pbp['ca']."','".$pbp['cb']."',".($i+1).",
".$pbp[$qtitle[$i]][$ii]['MINUTO'].",'".$pbp[$qtitle[$i]][$ii]['NTCONSOLA']."',
'".str_replace(" ","",substr($pbp[$qtitle[$i]][$ii]['NTJUGD'],1,10))."','".$play."',".$lres.",".$vres.",".$points.")";
mysqli_query($link,$q);
$ii++;
$n++;
}
}
Do you think it is suitable for a cron job? How should I proceed? Thanks a lot!
I had similar issues but the following worked for me.
See the link to change default mysql permission
How to allow remote connection to mysql
Now change your db_server value in the sql connection file from
localhost to 127.0.0.1
In your case the you need to edit the file ../newcon.php it seems.
This is more of a logic question than language question, though the approach might vary depending on the language. In this instance I'm using Actionscript and PHP.
I have a flash graphic that is getting data stored in a mysql database served from a PHP script. This part is working fine. It cycles through database entries every time it is fired.
The graphic is not on a website, but is being used at 5 locations, set to load and run at regular intervals (all 5 locations fire at the same time, or at least within <500ms of each-other). This is real-time info, so time is of the essence, currently the script loads and parses at all 5 locations between 30ms-300ms (depending on the distance from the server)
I was originally having a pagination problem, where each of the 5 locations would pull a different database entry since i was moving to the next entry every time the script runs. I solved this by setting the script to only move to the next entry after a certain amount of time passed, solving the problem.
However, I also need the script to send an email every time it displays a new entry, I only want it to send one email. I've attempted to solve this by adding a "has been emailed" boolean to the database. But, since all the scripts run at the same time, this rarely works (it does sometimes). Most of the time I get 5 emails sent. The timeliness of sending this email doesn't have to be as fast as the graphic gets info from the script, 5-10 second delay is fine.
I've been trying to come up with a solution for this. Currently I'm thinking of spawning a python script through PHP, that has a random delay (between 2 and 5 seconds) hopefully alleviating the problem. However, I'm not quite sure how to run exec() command from php without the script waiting for the command to finish. Or, is there a better way to accomplish this?
UPDATE: here is my current logic (relevant code only):
//get the top "unread" information from the database
$query="SELECT * FROM database WHERE Read = '0' ORDER BY Entry ASC LIMIT 1";
//DATA
$emailed = $row["emailed"];
$Entry = $row["databaseEntryID"];
if($emailed == 0)
{
**CODE TO SEND EMAIL**
$EmailSent="UPDATE database SET emailed = '1' WHERE databaseEntryID = '$Entry'";
$mysqli->query($EmailSent);
}
Thanks!
You need to use some kind of locking. E.g. database locking
function send_email_sync($message)
{
sql_query("UPDATE email_table SET email_sent=1 WHERE email_sent=0");
$result = FALSE;
if(number_of_affacted_rows() == 1) {
send_email_now($message);
$result = TRUE;
}
return $result;
}
The functions sql_query and number_of_affected_rows need to be adapted to your particular database.
Old answer:
Use file-based locking: (only works if the script only runs on a single server)
function send_email_sync($message)
{
$fd = fopen(__FILE__, "r");
if(!$fd) {
die("something bad happened in ".__FILE__.":".__LINE__);
}
$result = FALSE;
if(flock($fd, LOCK_EX | LOCK_NB)) {
if(!email_has_already_been_sent()) {
actually_send_email($message);
mark_email_as_sent();
$result = TRUE; //email has been sent
}
flock($fd, LOCK_UN);
}
fclose($fd);
return $result;
}
You will need to lock the row in your database by using a transaction.
psuedo code:
Start transaction
select row .. for update
update row
commit
if (mysqli_affected_rows ( $connection )) >1
send_email();
got this code:
<?php
function test ($url){
$starttime = microtime(true);
$valid = #fsockopen($url, 80, $errno, $errstr, 30);
$stoptime = microtime(true);
echo (round(($stoptime-$starttime)*1000)).' ms.';
if (!$valid) {
echo "Status - Failure";
} else {
echo "Status - Success";
}
}
test('google.com');
?>
I want to have an option to execute this function every 5mins / 1hour / 1 day etc.. I was suggested using cron, but i never heard of cron before and after doing some research i understood that its a sepperate file, that would exeute the function every x amout of time. What if i would have multiple users, for example userA would want to run the script every 5mins, and userB would want to run the script every hour. In this sittuation i would need to create multiple cron files for each user?
Edit:
I was thinking about doing something like this:
for ($i = 0; $i < 100; $i++) {
test('google.com');
sleep(10 * 60);
}
Only in the sleep line i would have a custom $n field that each user would define by themselves. My problem with this was - it returns results only when the full cycle has finished, i would want it to return result after every "round?" (idk what its called, but basically would give 1 value, then 10mins later 2nd value and so on) .
The only way I found is to use an ajax method which will be called from the user page within a setTimeout.
See : settimeout function
May be you need use cron (UNIX) or Windows Tasks (Microsoft) depending the OS on your server.
Cron
Windows Task
Greetings,