When I'm use ibase_trans() the script start infinite loop and don't respond.
I'am requesting nusoap webservice.
If I just print the nusoap data the script execution time is 3 seconds, after I uncomment the line that start transaction it starts infinite loop.
$dbh1 = ibase_connect(DB_HOST1, DB_USER1, DB_PASS1) or die('die message');
$tr1 = ibase_trans($dbh1);
After the nusoap data loop I have those lines :
$query3 = ibase_query($tr1,"INSERT INTO EXECUTAR ( FLAG ) VALUES ( 2 )") or die('teste');
Here start the problem.
My php version : 5.6
Xampp version: 3.2.1
Firebird: 2.5
Full code :
if ($client->fault) {
echo "<h2>Fault</h2><pre>";
print_r($result);
echo "</pre>";
} else {
$error = $client->getError();
if ($error) {
echo "<h2>Error.</h2><pre>" . $error . "</pre>";
} else {
$dbh1 = ibase_connect(DB_HOST1, DB_USER1, DB_PASS1) or die('die message');
$tr1 = ibase_trans($dbh1);
$TotalProdutos = 0;
foreach($result['ConsultaEstoqueProdutosResult']['diffgram']['NewDataSet']['Estoques'] as $produto){
$TotalProdutos++;
$query1 = ibase_query($tr1,"INSERT INTO ATUALIZA_PRE ( CODIGO,
ESTOQUE,
ATIVO,
DISPONIVEL,
ESTOQUEDISPONIVEL,
PRECOSEMST,
PRECOREVENDA )
VALUES (
'{$produto['CODIGO']}',
'{$produto['ESTOQUE']}',
{$produto['ATIVO']},
{$produto['DISPONIVEL']},
{$produto['ESTOQUEDISPONIVEL']},
{$produto['PRECOSEMST']},
'".$produto['PRECOREVENDA']."')")or die('teste1');
}
$dbh1 = ibase_connect(DB_HOST1, DB_USER1, DB_PASS1) or die('die message');
$tr1 = ibase_trans($dbh1);
$query3 = ibase_query($tr1,"INSERT INTO EXECUTAR ( FLAG ) VALUES ( 2 )") or die('teste');
ibase_commit($tr1);
ibase_close($dbh1);
}
}
Can someone help me please?
Related
I am using 7 JSON File, and I am trying to insert 95 coloums + 50-100 rows at a time.
This is how my code looks like: I removed some 6 JSON files and other rows&colums. This is a short version of my code
<?php
$requestsDone = 2;
$maxRequests = 100;
$currentnumber = 1;
while ($requestsDone <= $maxRequests) {
$response = file_get_contents("https://api.themoviedb.org/3/movie/".$requestsDone."?api_key=522cexxxxxxxxxxxxx9ax34a");
if ($response != FALSE) {
$response = json_decode($response, true);
}
if ( ($response["title"]) != "" )
{
$requestsDone++;
include 'admin/connectdb.php';
try {
// prepare sql and bind parameters
$stmt = $conn->prepare("INSERT INTO TABLE (title,image,money,tag1,tag2)
VALUES (:title, :image, :money,:tag1,:tag2)");
$stmt->bindParam(':title', $title);
$stmt->bindParam(':image', $image);
$stmt->bindParam(':money', $money);
$stmt->bindParam(':tag1', $tag1);
$stmt->bindParam(':tag1', $tag2);
// insert a row
$movie_title = "".$response["title"]."";
if ( $response["image"] != "" )
$image = "".$response["image"]."";
if ( $response["money"] != "" )
$money = "".$response["money"]."";
if ( ($response["tag"][0]["name"]) != "" )
$tag1 = "".$response["tag"][0]["name"]."";
if ( ($response["tag"][1]["name"]) != "" )
$tag2 = "".$response["tag"][1]["name"]."";
$stmt->execute();
echo "New records created successfully ".$currentnumber++." <br/>";
}
catch(PDOException $e)
{
echo "Error: " . $e->getMessage();
}
$conn = null;
}
}
?>
I am getting 2 type of errors: Sometime 1st error, sometime 2nd error.
1st Error: Cloudflare time-out: Error 504 Screenshot
2nd Error: Error: SQLSTATE[HY000]: General error: 2006 MySQL server has gone away
Please let me know, if you need more information.
Edit:
This is how one of the JSON file looks like
{"adult":false,"backdrop_path":"/87hTDiay2N2qWyX4Ds7ybXi9h8I.jpg","belongs_to_collection":null,"budget":63000000,"genres":[{"id":18,"name":"Drama"}],"homepage":"http://www.foxmovies.com/movies/fight-club","id":550,"imdb_id":"tt0137523","original_language":"en","original_title":"Fight Club","overview":"A ticking-time-bomb insomniac and a slippery soap salesman channel primal male aggression into a shocking new form of therapy. Their concept catches on, with underground \"fight clubs\" forming in every town, until an eccentric gets in the way and ignites an out-of-control spiral toward oblivion.","popularity":9.765478999999999,"poster_path":"/adw6Lq9FiC9zjYEpOqfq03ituwp.jpg","production_companies":[{"name":"Regency Enterprises","id":508},{"name":"Fox 2000 Pictures","id":711},{"name":"Taurus Film","id":20555},{"name":"Linson Films","id":54050},{"name":"Atman Entertainment","id":54051},{"name":"Knickerbocker Films","id":54052}],"production_countries":[{"iso_3166_1":"DE","name":"Germany"},{"iso_3166_1":"US","name":"United States of America"}],"release_date":"1999-10-15","revenue":100853753,"runtime":139,"spoken_languages":[{"iso_639_1":"en","name":"English"}],"status":"Released","tagline":"How much can you know about yourself if you've never been in a fight?","title":"Fight Club","video":false,"vote_average":8.199999999999999,"vote_count":8047}
I have a code that runs a query on an external (slow) API and loops through a lot of variables and inserts data, sends email, ect'.
This is the main function as an example:
// MAIN: Cron Job Function
public function kas_alert() {
// 0. Deletes all the saved data from the `data` table 1 month+ ago.
// $this->kas_model->clean_old_rows();
// 1. Get 'prod' table
$data['table'] = $this->kas_model->prod_table();
// 2. Go through each row -
foreach ( $data['table'] as $row ) {
// 2.2. Gets all vars from the first query.
$last_row_query = $this->kas_model->get_last_row_of_tag($row->tag_id);
$last_row = $last_row_query[0];
$l_aaa_id = $last_row->prod_aaa_id;
$l_and_id = $last_row->prod_bbb_id;
$l_r_aaa = $last_row->dat_data1_aaa;
$l_r_and = $last_row->dat_data1_bbb;
$l_t_aaa = $last_row->dat_data2_aaa;
$l_t_and = $last_row->dat_data2_bbb;
$tagword = $last_row->tag_word;
$tag_id = $last_row->tag_id;
$country = $last_row->kay_country;
$email = $last_row->u_email;
$prod_name = $last_row->prod_name;
// For the Weekly report:
$prod_id = $last_row->prod_id;
$today = date('Y-m-d');
// 2.3. Run the tagword query again for today on each one of the tags and insert to DB.
if ( ($l_aaa_id != 0) || ( !empty($l_aaa_id) ) ) {
$aaa_data_today = $this->get_data1_aaa_by_id_and_kw($l_aaa_id, $tagword, $country);
} else{
$aaa_data_today['data1'] = 0;
$aaa_data_today['data2'] = 0;
$aaa_data_today['data3'] = 0;
}
if ( ($l_and_id != 0) || ( !empty($l_and_id) ) ) {
$bbb_data_today = $this->get_data1_bbb_by_id_and_kw($l_and_id, $tagword, $country);
} else {
$bbb_data_today['data1'] = 0;
$bbb_data_today['data2'] = 0;
$bbb_data_today['data3'] = 0;
}
// 2.4. Insert the new variables to the "data" table.
if ($this->kas_model->insert_new_tag_to_db( $tag_id, $aaa_data_today['data1'], $bbb_data_today['data1'], $aaa_data_today['data2'], $bbb_data_today['data2'], $aaa_data_today['data3'], $bbb_data_today['data3']) ){
}
// Kas Alert Outputs ($SEND is echoed in it's original function)
echo "<h1>prod Name: $prod_id</h1>";
echo "<h2>tag id: $tag_id</h2>";
var_dump($aaa_data_today);
echo "aaa old: ";
echo $l_r_aaa;
echo "<br> aaa new: ";
echo $aaa_data_today['data1'];
var_dump($bbb_data_today);
echo "<br> bbb old: ";
echo $l_r_and;
echo "<br> bbb new: ";
echo $bbb_data_today['data1'];
// 2.5. Check if there is a need to send something
$send = $this->check_if_send($l_aaa_id, $l_and_id, $l_r_aaa, $aaa_data_today['data1'], $l_r_and, $bbb_data_today['data1']);
// 2.6. If there is a trigger, send the email!
if ($send) {
$this->send_mail($l_aaa_id, $l_and_id, $aaa_data_today['data1'], $bbb_data_today['data1'], $l_r_aaa, $l_r_and, $tagword, $email, $prod_name);
}
}
}
This CodeIgniter controller is runs every day and I get this running (Cronjob) for too long using almost nothing from the CPU and RAM (RAM is at 400M/4.25G and CPU at ONLY 0.7%-1.3%).
I wonder if there's an option to split all foreach loops to smaller threads (and I'm not sure if forking will do here) and run all the foreach loops in parallel but in a way that it wont get my server crashing.
I'm no DevOps and really interested learning - What should I do in this case?
I usually work with web hosting companies but I decided to start learning working with servers to expand my knowledge.
I'll better give a real example to explain my question the best:
I have a web application that gathers data from a slow API that returns JSON data of products.
I have a function running every 1AM running a lot of queries on "id"s in my database.
Crontab:
0 1 * * * cd /var/www/html/tools; php index.php aso Cli_kas kas_alert
So this creates a process for the app (please correct me here if I'm wrong) and each process creates threads, and just to be more accurate, they are multi-threads since they do more than one thing: like pulling data from the DB to get the right variables and string them to the API queries, getting the data from the API, organizing it, searching the relevant data, and then inserting new data to the database.
The main PHP functions:
// MAIN: Cron Job Function
public function kas_alert() {
// 0. Deletes all the saved data from the `data` table 1 month+ ago.
// $this->kas_model->clean_old_rows();
// 1. Get 'prod' table
$data['table'] = $this->kas_model->prod_table();
// 2. Go through each row -
foreach ( $data['table'] as $row ) {
// 2.2. Gets all vars from the first query.
$last_row_query = $this->kas_model->get_last_row_of_tag($row->tag_id);
$last_row = $last_row_query[0];
$l_aaa_id = $last_row->prod_aaa_id;
$l_and_id = $last_row->prod_bbb_id;
$l_r_aaa = $last_row->dat_data1_aaa;
$l_r_and = $last_row->dat_data1_bbb;
$l_t_aaa = $last_row->dat_data2_aaa;
$l_t_and = $last_row->dat_data2_bbb;
$tagword = $last_row->tag_word;
$tag_id = $last_row->tag_id;
$country = $last_row->kay_country;
$email = $last_row->u_email;
$prod_name = $last_row->prod_name;
// For the Weekly report:
$prod_id = $last_row->prod_id;
$today = date('Y-m-d');
// 2.3. Run the tagword query again for today on each one of the tags and insert to DB.
if ( ($l_aaa_id != 0) || ( !empty($l_aaa_id) ) ) {
$aaa_data_today = $this->get_data1_aaa_by_id_and_kw($l_aaa_id, $tagword, $country);
} else{
$aaa_data_today['data1'] = 0;
$aaa_data_today['data2'] = 0;
$aaa_data_today['data3'] = 0;
}
if ( ($l_and_id != 0) || ( !empty($l_and_id) ) ) {
$bbb_data_today = $this->get_data1_bbb_by_id_and_kw($l_and_id, $tagword, $country);
} else {
$bbb_data_today['data1'] = 0;
$bbb_data_today['data2'] = 0;
$bbb_data_today['data3'] = 0;
}
// 2.4. Insert the new variables to the "data" table.
if ($this->kas_model->insert_new_tag_to_db( $tag_id, $aaa_data_today['data1'], $bbb_data_today['data1'], $aaa_data_today['data2'], $bbb_data_today['data2'], $aaa_data_today['data3'], $bbb_data_today['data3']) ){
}
// Kas Alert Outputs ($SEND is echoed in it's original function)
echo "<h1>prod Name: $prod_id</h1>";
echo "<h2>tag id: $tag_id</h2>";
var_dump($aaa_data_today);
echo "aaa old: ";
echo $l_r_aaa;
echo "<br> aaa new: ";
echo $aaa_data_today['data1'];
var_dump($bbb_data_today);
echo "<br> bbb old: ";
echo $l_r_and;
echo "<br> bbb new: ";
echo $bbb_data_today['data1'];
// 2.5. Check if there is a need to send something
$send = $this->check_if_send($l_aaa_id, $l_and_id, $l_r_aaa, $aaa_data_today['data1'], $l_r_and, $bbb_data_today['data1']);
// 2.6. If there is a trigger, send the email!
if ($send) {
$this->send_mail($l_aaa_id, $l_and_id, $aaa_data_today['data1'], $bbb_data_today['data1'], $l_r_aaa, $l_r_and, $tagword, $email, $prod_name);
}
}
}
For #Raptor, this is the function that get's the API data:
// aaa tag Query
// Gets aaa prod dataing by ID.
public function get_data_aaa_by_id_and_tg($id, $tag, $query_country){
$tag_for_url = rawurlencode($tag);
$found = FALSE;
$i = 0;
$data = array();
// Create a stream for Json. That's how the code knows what to expect to get.
$context_opts = array(
'http' => array(
'method' => "GET",
'header' => "Accepts: application/json\r\n"
));
$context = stream_context_create($context_opts);
while ($found == FALSE) {
// aaa Query
$json_query_aaa = "https://api.example.com:443/aaa/ajax/research_tag?app_id=$id&term=$tag_for_url&page_index=$i&country=$query_country&auth_token=666";
// Get the Json
$json_query_aaa = file_get_contents($json_query_aaa, false, $context);
// Turn Json to a PHP array
$json_query_aaa = json_decode($json_query_aaa, true);
// Get the data2
$data2 = $json_query_aaa['tag']['data2'];
if (is_null($data2)){ $data2 = 0; }
// Get data3
$data3 = $json_query_aaa['tag']['phone_prod']['data3'];
if (is_null($data3)){ $data3 = 0; }
// Finally, the main prod array.
$json_query_aaa = $json_query_aaa['tag']['phone_prod']['app_list'];
if ( count($json_query_aaa) > 2 ) {
for ( $j=0; $j<count($json_query_aaa); $j++ ) {
if ( $json_query_aaa[$j]['id'] == $id ) {
$found = TRUE;
$data = $json_query_aaa[$j]['data'] + 1;
break;
}
if ($found == TRUE){
break;
}
}
$i++;
} else {
$data = 0;
break;
}
}
$data['data1'] = $data;
$data['data2'] = $data2;
$data['data3'] = $data3;
return $data;
}
All threads are stacked one after an other, and when one thread is done, only then - the second thread can proceed, ect'.
And in technical view on this, all threads wait in the RAM until the one before them is done working "inside" the CPU. (correct me if I'm wrong again :] )
This doesn't even "tickle" the servers RAM or CPU when looking at it in the process manager (I use "htop"). RAM is at 400M/4.25G and CPU at ONLY 0.7%-1.3%.
Making me feel this isn't the best I can get from my current server, and getting slow results from my web app.
How do I get things done in a way that all threads work in parallel, but not to a point that my app crashes due to lacks of CPU or RAM?
if ( isset($_POST['update']) ){
$db=mysqli_connect("localhost","****","****","****");
$lasttime = isset($_POST['timestamp']) ? $_POST['timestamp'] : 0;
while (1){
sleep(1);
clearstatcache();
$mresult = mysqli_query($db,"SELECT * FROM tblchat WHERE msg_datetime > $lasttime");
if (!empty($mresult)){ break; }
}
$msgs = array();
while ($row = mysqli_fetch_object( $mresult )) { $msgs[] = $row; }
mysqli_free_result($mresult);
$response = array();
$response['msgs'] = $msgs;
echo json_encode($response);
flush();
mysqli_close($db);
exit();
}
The code is the server for a long polling connection with client. If update is requested, the while loops check for any new messages received after the timestamp sent with the update request. If found, it puts the result in an array and echo it back to the client.
The resulting output is something like this [msgs:[{msg_from:"",msg_to:"",msg:"",msg_datetime:""},{msg_from:"",msg_to:"",msg:"",msg_datetime:""}]]
The code works fine for the first time and send all the recent messages well encapsulated but then it again sends an empty array of messages. Please guide me.
solved the issue with mysqli_num_rows
if (mysqli_num_rows($mresult)){ $msgs = array(); while ($row = mysqli_fetch_object( $mresult )) { $msgs[] = $row; } mysqli_free_result($mresult); break; }
if (mysqli_num_rows($wresult)){ $writers = array(); while ($row = mysqli_fetch_object( $wresult )) { $writers[] = $row; } mysqli_free_result($wresult); break; }
thanks everyone for their help!
I am learning how to run iMacros from my php scripts so that PHP script calls an iMacros browser session and passes any variables that I have (url and macro name for example). The iMacros session then runs the iMacro, after the macro is done running it passes the resulting html page back to the PHP script and closes itself. In an ideal world, anyway.
Here is the iMacros calling script:
<?php
require 'src/iimfx.class.php';
$iim = new imacros();
$vars = array();
$iim->play($vars,'grab_data.iim');
?>
But when i run this script from cmd.exe [command line] on WAMP, I get this:
New imacros session started!
Using Proxy: MY_PROXY_IP:MY_PROXY_PORT
-runner -fx -fxProfile default
--------------------------------------------------------
Setting Value IP => MY_PROXY_IP
Setting Value port => MY_PROXY_PORT
Playing Macro proxy.iim
--------MACRO ERROR!-------------------
ERROR: Browser was not started. iimInit() failed?
--------------------------------------------------------
Playing Macro grab_google.iim
--------MACRO ERROR!-------------------
ERROR: Browser was not started. iimInit() failed?
P.S. MY_PROXY_IP and MY_PROXY_PORT are replaced with actual numbers both in error messages above and iimfx.class.php.
And here is code for the iimfx.class.php :
<?php
class imacros {
function __construct($proxyip = 'MY_PROXY_IP', $proxyport = 'MY_PROXY_PORT', $silent = false, $noexit = false) {
echo "--------------------------------------\nNew imacros session started!\nUsing Proxy: $proxyip:$proxyport\n";
$this->proxyip = $proxyip;
$this->proxyport = $proxyport;
if (empty ( $this->proxyip ))
echo "NO PROXY!!\n";
$this->noexit = $noexit;
$this->fso = new COM ( 'Scripting.FileSystemObject' );
$this->fso = NULL;
$this->iim = new COM ( "imacros" );
$toexec = "-runner -fx -fxProfile default";
if ($silent === true)
$toexec .= " -silent";
if ($noexit === true)
$toexec .= " -noexit";
echo $toexec . "\n";
$this->iim->iimInit ( $toexec );
if (! empty ( $this->proxyip )) {
$dvars ['IP'] = $this->proxyip;
$dvars ['port'] = $this->proxyport;
$this->play ( $dvars, 'proxy.iim' );
}
}
function __destruct() {
if ($this->noexit === false)
$this->iim->iimExit ();
}
function play($immvars = '', $macro) {
echo "--------------------------------------------------------\n";
if (is_array ( $immvars )) {
foreach ( $immvars as $key => $value ) {
echo "Setting Value $key => $value\n";
$this->iim->iimSet ( "-var_" . $key, $value );
}
}
echo "Playing Macro $macro\n";
$s = $this->iim->iimPlay ( $macro );
if($s>0){
echo "Macro successfully played!\n";
}else{
echo "--------MACRO ERROR!-------------------\n ERROR: " . $this->getLastError() . "\n";
}
return $s;
}
// This function retrieves extracts in your iMacros script if you have any.
function getLastExtract($num) {
return $this->iim->iimGetLastExtract ( $num );
}
// Returns the last error :)
function getLastError(){
return $this->iim->iimGetLastError();
}
// Enables/disables images
function setImages($images = 1) { // 1 = on 2 = off
$dvars ['images'] = $images;
$this->play ( $dvars, 'images.iim' );
}
// Enables or disables adblockplus
function enableABP($status = true){
$dvars['status'] = $status;
$this->play ( $dvars, 'abp.iim' );
}
}
?>
Is there something I am missing here?
I have iimRunner.exe running during all of this [started manually before running the script] and I have iMacros Browser V8+.
Also, my grab_data.iim and all other required .iim are in the same place as the php script that is trying to call them and execute them.
Any kind of help and/or steer towards the right direction would be greatly appreciated!!
Thanks in advance.
U must by start the immrunner, before start the script =)
http://wiki.imacros.net/iimRunner