I try to set up some basic fixtures in Mongo DB. To do so I read a set of JSON files and try to insert them into local db (I am connecting with admin rights). Here's the weird part. For some reasons I wrote to versions of the code that should work basically the same so I have one:
$client = new \MongoClient($connectionURI);
$db = $client->selectDB($database);
$collections = $db->listCollections();
foreach ($collections as $collection) {
//echo "Removing all documents from '$collection'" . PHP_EOL;
$collection->remove();
$tmp = explode('.', (string)$collection);
$collectionName = $tmp[1];
$tmp = explode('_', $tmp[0]);
$dbName = $tmp[1];
$country = $tmp[2];
if(file_exists(__DIR__."/fixtures/{$country}/{$dbName}/{$collectionName}.json")) {
echo "Inserting fixture data into '{$collection}'".PHP_EOL;
$data = json_decode(file_get_contents(__DIR__."/fixtures/{$country}/{$dbName}/{$collectionName}.json"));
$doc = $collection->insert($data, ["w" => "majority"]);
}
}
And second one based on iteration of the files to read instead of listing existing collections:
$client = new \MongoClient($connectionURI);
foreach(glob(__DIR__.'/fixtures/*', GLOB_ONLYDIR) as $dir) {
$country = basename($dir);
foreach(glob(__DIR__.'/fixtures/'.$country.'/*', GLOB_ONLYDIR) as $dbDir) {
$collections = array_diff(
scandir(__DIR__."/fixtures/{$country}/".basename($dbDir)), ['.', '..']
);
$dbName = 'test_'.basename($dbDir).'_'.$country;
foreach($collections as $collectionFile) {
$collectionName = pathinfo($collectionFile)['filename'];
$data = [];//json_decode(file_get_contents(__DIR__."/fixtures/{$country}/".basenam e($dbDir)."/{$collectionName}.json"));
// $client->$dbName->$collectionName->insert($data);
$db = $client->selectDB($dbName);
$collection = $db->selectCollection($collectionName);
$collection->insert($data, ["w" => "majority"]);
echo $country.'->'.$dbName.'->'.$collectionName.PHP_EOL;
}
}
}
The trick is that the first implementation works nicely and second throws MongoCursorException with authentication problem. The problem I am having is that both versions try to connect to exact same database and collection in fact. So I am getting following output:
Inserting fixture data into 'test_customers_poland.accounts'
PHP Fatal error: Uncaught exception 'MongoCursorException' with message 'Couldn't get connection: Failed to connect to: 127.0.0.1:27017: SASL Authentication failed on database 'test_customers_poland': Authentication failed.' in /srv/dev-api/src/tests/init_databases.php:96
Stack trace:
#0 /srv/dev-api/src/tests/init_databases.php(96): MongoCollection->insert(Array, Array)
#1 {main}
thrown in /s
rv/dev-api/src/tests/init_databases.php on line 96
Of course I also checked running those snippets separately as well with the same result so the question is: what am I doing wrong in second approach?
Related
I had php AdvancedHTMLDOM working just fine for a long time. However, about a week ago I noticed that the data I am scraping are not being updated for some reason.
I ran the script manually and got the following error:
root#telemetry:/home/telemetry/scripts/pressure# php -f get_pressure_nodes.php
PHP Fatal error: Uncaught Error: Class 'DOMDocument' not found in /home/telemetry/scripts/pressure/advanced_html_dom-master/advanced_html_dom.php:171
Stack trace:
#0 /home/telemetry/scripts/pressure/advanced_html_dom-master/advanced_html_dom.php(167): AdvancedHtmlDom->load('<html>\n<head>\n<...', false)
#1 /home/telemetry/scripts/pressure/advanced_html_dom-master/advanced_html_dom.php(747): AdvancedHtmlDom->__construct('<html>\n<head>\n<...')
#2 /home/telemetry/scripts/pressure/advanced_html_dom-master/advanced_html_dom.php(748): str_get_html('<html>\n<head>\n<...')
#3 /home/telemetry/scripts/pressure/get_pressure_nodes.php(17): file_get_html('get_pressure_no...')
#4 {main}
thrown in /home/telemetry/scripts/pressure/advanced_html_dom-master/advanced_html_dom.php on line 171
root#telemetry:/home/telemetry/scripts/pressure#
Here is my script (I kept the basics for simplicity, also I got this code from somewhere I cannot recall, so if it is yours please inform me so I can give credit where it is due):
<?php
require('advanced_html_dom-master/advanced_html_dom.php');
$html = file_get_html('get_pressure_nodes.html');
$table = $html->find('table', 1);
$rowData = array();
foreach($table->find('tr') as $row)
{
// initialize array to store the cell data from each row
$temp = array();
foreach($row->find('td') as $cell)
{
// push the cell's text to the array
$temp[] = $cell->plaintext;
}
$rowData[] = $temp;
}
foreach ( $rowData as $cell_contents )
{
print ( $cell_contents ) ;
}
?>
If AdvancedHTMLDOM has stopped working suddenly, this SO Post could be of help in fixing it.
I'm trying to stick with the "Fat Model" approach in programming my CakePHP app and am currently stumped in my attempts to get a function working in one of my models. Here's a slightly simplified version of my code.
See the part where it says "This is where the trouble starts" and a bunch of lines are commented out. I can't figure out how to save the headers info to the model instance, because I'm using "findByID" which returns an array instead of an object.
I found another thread (CakePHP 2.0 Object not Array) discussing the array vs. object issue. It seems that's part of the design of Cake 2.X, though I don't really understand why. Anyway I don't want a workaround as is suggested in that other thread, but rather I want to understand how to do this correctly:
// app/Model/Datareportdoc.php
App::uses('AppModel', 'Model');
class Datareportdoc extends AppModel {
public function parseData($id) {
$results = ""; // init
$filepath = ""; // init
$reportdoc = $this->findById($id);
$filepath = $reportdoc['Datareportdoc']['filepath'];
$headers = array(); // init
// Parse and save the report data
if (($handle = fopen($filepath, "r")) !== FALSE) {
$row = 0;
while (($datarow = fgetcsv($handle, 1000, ",")) !== FALSE) {
$num = count($datarow);
// If it's a header row, create the headers array
if ($row == 0) {
$results .= "HEADER ROW\n";
$dataheaders = implode(",", $datarow);
// *** THIS IS WHERE THE TROUBLE STARTS ***
//$this->Datareportdoc->id = $id; // NO: "Indirect modification of overloaded property"
//$this->Datareportdoc->set('dataheaders', $dataheaders); // No, because we have an array, not an object :-(
//$this->Datareportdoc->dataheaders = $dataheaders;
//$this->Datareportdoc->save(); // NO: Error: Call to a member function save() on a non-object
// Also tried this, which also doesn't work
//$reportdoc['Datareportdoc']['dataheaders'] = $dataheaders;
//$reportdoc->save();
$results .= "Datareportdoc dataheaders: ".$dataheaders."\n";
} else {
$data = array();
$i = 0;
foreach ($headers as $header) {
$data[$header] = $datarow[$i];
$i++;
}
$results .= "Got data for insert:\n".print_r($data, true)."\n\n\n";
// #TODO: Insert the data...
}
$row++;
}
fclose($handle);
} else {
$results .= "fopen failed :-(<br />\n";
//echo "fopen failed :-(<br />\n";
//exit();
}
return $results;
}
}
So give this a shot:
Instead of this:
//$this->Datareportdoc->id = $id;
//$this->Datareportdoc->set('dataheaders', $dataheaders); // No, because we have an array, not an object :-(
//$this->Datareportdoc->dataheaders = $dataheaders;
//$this->Datareportdoc->save();
Use this:
$this->save($dataheaders);
It works every time in the model when done this way.
(Background: CakePHP 2.x cookbook talks a 'little' bit about active record .... but it's not really if you're used to other platforms, not in the sense I understand the term anyway. (3.0 is ORM and active record, but not so for 2.x)
I've got a simple php script for getting a csv file into array and inserting each row into MongoDB (CD Collection). But somehow insert returns an error after first successful one:
"Fatal error: in C:\xampp\htdocs\mongo\lesson1\index.php on line 17"
Here's the code. What could possibly cause such an error? DB receives only one (first).
$filename = 'd:/cd_col.csv'; // Each line: Title;No. of CD with movie
$csvfile = fopen($filename,'rb');
while(!feof($csvfile)) {
$csvarray[] = fgetcsv($csvfile);
}
$m = new MongoClient();
$db = $m->mymovies;
$collection = $db->movies;
$id=0;
foreach($csvarray as $key=>$value)
{
$movie = explode(';', $value[0]);
$fmovie = array('_id'=>++$id, 'title'=>$movie[0], 'cdno'=>$movie[1]);
if($collection->save($fmovie)===true) { // this is line 17
echo 'Successful insert: '.$key;
}
}
SOLVED:
Cannot use save in php, that's it.
You should use $collection->insert(); instead.
First of all i want to apologize if this is the most basic question ever! I'm not that good with php but i'm learning.
I can't find a solution or even understand why it's going wrong all the time. I do want to know why this is happening
I'm trying to get the two latest tweets from a twitter account. I don't want to use massive (existing, i know) classes or codes which i don't understand. So i tried the following myself:
$timeline = "http://twitter.com/statuses/user_timeline.xml?screen_name=Mau_ries";
$data = file_get_contents($timeline);
$tweets = new SimpleXMLElement($data);
$i = 0;
foreach($tweets as $tweet){
echo($tweet->text." - ".$tweet->created_at);
if (++$i == 2) break;
}
When i first ran this code i got the text from my tweets, but when i refreshed the page i sometimes get the following error:
Warning: file_get_contents(http://twitter.com/statuses/user_timeline.xml?screen_name=Mau_ries) [function.file-get-contents]: failed to open stream: HTTP request failed! HTTP/1.0 400 Bad Request in /path/to/file on line 88
Fatal error: Uncaught exception 'Exception' with message 'String could not be parsed as XML' in /public/sites/www.singledays.nl/tmp/index.php:89 Stack trace: #0 /public/sites/www.singledays.nl/tmp/index.php(89): SimpleXMLElement->__construct('') #1 {main} thrown in /path/to/file on line 89
Lines 88 & 89 are these:
$data = file_get_contents($timeline);
$tweets = new SimpleXMLElement($data);
Really weird. Sometimes it work, sometimes not.
Does anybody know this issue and/or a solution? And why does the error seem to occur randomly (Allthough it;s now erroring for a while allready)?
Thanks!
$timeline = "http://twitter.com/statuses/user_timeline.xml?screen_name=Mau_ries";
$data = #file_get_contents($timeline);
if($data){
$fh = fopen("cache/".sha1($timeline),"w");
fwrite($fh, $data);
fclose($fh);
}else{
$fh = #fopen("cache/".sha1($timeline),"r");
$data = "";
while(!feof($fh)){ $data = fread($fh, 1024); }
fclose($fh);
}
if(!$data) die("could not open url or find a cache of url locally");
$tweets = new SimpleXMLElement($data);
$i = 0;
foreach($tweets as $tweet){
echo($tweet->text." - ".$tweet->created_at);
if (++$i == 2) break;
}
There as every one has said debugging you should really cache the results in files and if it fails to download then use the cache the above code will do it for you.
I am trying to load a file to my Cloud Files Container from a remote file.
Below is the example code a Cloud Files Support person gave me:
<?php
require('cloud/cloudfiles.php');
$res = fopen("http://images.piccsy.com/cache/images/66653-d71e1b-320-469.jpg", "rb");
$temp = tmpfile();
$size = 0.0;
while (!feof($res))
{
$bytes = fread($res, 1024);
fwrite($temp, $bytes);
$size += (float) strlen($bytes);
}
fclose($res);
fseek($temp, 0);
//
$auth = new CF_Authentication('user','token ');
//Calling the Authenticate method returns a valid storage token and allows you to connect to the CloudFiles Platform.
$auth->authenticate();
$conn = new CF_Connection($auth);
$container = $conn->create_container("example");
$object = $container->create_object("example.jpg");
$object->content_type = "image/jpeg";
$object->write($temp, $size);
fclose($temp);
?>
The problem I am getting is the below error:
Fatal error: Call to a member function create_container() on a non-object in /home2/sharingi/public_html/daily_dose/remote_test.php on line 24
Not sure exactly what I am not noticing here.
It seems $conn did not successfully instantiate an object. That is the problem, but the solution is not clear cut.
Is CF_Connection defined? Does error_reporting(E_ALL) give you more helpful info? What does var_dump($conn instanceof CF_Connection) output?
This should point you in the right direction.