I am trying to use an API that returns the following structure
<TwilioResponse>
<Call>
<Sid>CAe1644a7eed5088b159577c5802d8be38</Sid>
<DateCreated>Tue, 10 Aug 2010 08:02:17 +0000</DateCreated>
<DateUpdated>Tue, 10 Aug 2010 08:02:47 +0000</DateUpdated>
<ParentCallSid/>
<AccountSid>AC5ef872f6da5a21de157d80997a64bd33</AccountSid>
<To>+14153855708</To>
<From>+14158141819</From>
<PhoneNumberSid></PhoneNumberSid>
<Status>completed</Status>
<StartTime>Tue, 10 Aug 2010 08:02:31 +0000</StartTime>
<EndTime>Tue, 10 Aug 2010 08:02:47 +0000</EndTime>
<Duration>16</Duration>
<Price>-0.03000</Price>
<Direction>outbound-api</Direction>
</Call>
</TwilioResponse>
I can get to the XML data by using CURL which works fine like this:
$handle = curl_init();
curl_setopt($handle, CURLOPT_URL, $base_url."/Accounts/{$accountSid}/Calls");
curl_setopt($handle, CURLOPT_RETURNTRANSFER, true);
$response = curl_exec($handle);
curl_close($handle);
however once I get the data back as XML I try and put it into a simpleXML element and return it back to the page that called this function as follows:
$xml = new SimpleXmlElement($response);
if($xml)
{
return $xml;
}
else
{
return false;
}
When I return the XML to the page that called the function, I can see a whole lot of simpleXMLElement Objects if I do a print_r() but when I try and do a foreach($xml->TwilioResponse->call as $call) I get nothing and it does not seem like I can actually drill into the data at all.
Can someone help me and point out where I am going wrong with this? It has been driving me absolutely crazy for the past couple of hours.
THANKS!
It looks like there are two problems with $xml->TwilioResponse->call.
$xml contains the TwilioResponse element (the "document element"), $xml->TwilioResponse is incorrect.
(XML and) SimpleXML element names are case-sensitive, call should be Call.
Give $xml->Call a spin.
Related
I am using the Laravel Goutte package to perform some webscraping - the following code works and returns a lot of data, I am trying to filter out only the bit of data I require.
If I load up the browser (whilst injecting jQuery into the page) I am able to get the data I need using jQuery using the following in the console jQuery('ea-proclub-overview')[0]; - I am basically trying to do the equivalent of this command within the Laravel/Goutte instance below.
Using jQuery('ea-proclub-overview')[0].customCrestBaseUrl; in the console I get the exact URL I need - https://fifa21.content.easports.com/fifa/fltOnlineAssets/05772199-716f-417d-9fe0-988fa9899c4d/2021/fifaweb/crests/256x256/l'
Below is my PHP code - I am getting back in the $node variable but I am unsure how to only return the customCrestBaseUrl so it gives me the URL.
$client = new Client();
$client->setServerParameter('HTTP_USER_AGENT', 'Mozilla/5.0 (X11; Linux i686; rv:78.0) Gecko/20100101 Firefox/78.0');
$client->setServerParameter('REFERER', 'https://www.ea.com/');
$url = 'https://www.ea.com/en-gb/games/fifa/pro-clubs/ps5-xbsxs/overview?clubId=2552&platform=ps5';
$crawler = $client->request('GET', $url);
$crawler->filter('ea-proclub-overview')->each(function ($node) {
dd($node);
});
Expected Result
<ea-proclub-overview endpoints="{"settingsEndpoint":"https://proclubs.ea.com/api/fifa/settings","seasonalStatsEndpoint":"https://proclubs.ea.com/api/fifa/clubs/seasonalStats","clubsInfoEndpoint":"https://proclubs.ea.com/api/fifa/clubs/info","matchesEndpoint":"https://proclubs.ea.com/api/fifa/clubs/matches","memberStatEndpoint":"https://proclubs.ea.com/api/fifa/members/stats","memberCareerStatEndpoint":"https://proclubs.ea.com/api/fifa/members/career/stats"}" colors="{"currentDivision":{"startColor":"#FA4358","endColor":"#FA4358"},"nextDivision":{"relegationColor":"#FA4358","pointsColor":"#FA4358"},"pieChart":{"winsColor":"#19A863","lossesColor":"#C4010D","tiesColor":"#282D3B"},"stats":{"wins":{"startColor":"#19A863","endColor":"#94D85D"},"losses":{"startColor":"#C4010D","endColor":"#F80245"},"ties":{"startColor":"#282D3B","endColor":"#282D3B"}}}" match-type="["gameType9","gameType13"]" headers-labels="{"points":"Points","stats":{"wins":{"label":"Wins","description":"Wins"},"losses":{"label":"Losses","description":"Losses"},"ties":{"label":"Draws","description":"Draws"}}}" division-labels="{"title":"Division Ranking","currentDivisionTitle":"Current Division","nextDivisionTitle":"Points To Next Division","seasons":"Season","record":"Record","points":"Points","gamesPlayed":"Games Played","gamesRemaining":"Games Remaining","divisionImgBaseUrl":"https://media.contentapi.ea.com/content/dam/eacom/fifa/pro-clubs/divisioncrest","stats":{"wins":"W","losses":"L","ties":"D"}}" progressbar-labels="{"div":"Div","promotion":"Promotion","relegation":"Relegation","title":"Title"}" members-labels="{"title":"Members","linkText":"View All Members","linkUrl":"members","totalTitle":"Total Members","totalCountsLabel":"Total","memberDetails":{"proOverall":"Overall Rating","ratingAve":"Average Match Rating","gamesPlayed":"Games Played"},"memberPosition":{"defender":"Defender","forward":"Forward","goalkeeper":"Goalkeeper","midfielder":"Midfielder"},"positions":{"defender":"Defenders","forward":"Forwards","goalkeeper":"Goalkeepers","midfielder":"Midfielders"},"defaultMemberAvatar":"https://media.contentapi.ea.com/content/dam/ea/fifa/fifa-21/pro-clubs/common/pro-clubs/avatar.png"}" match-labels="{"title":"Last Match","linkText":"View All Match History","linkUrl":"match-history","altTitle":"No match data was found"}" trophies-labels="{"title":"Trophies","cupsLabel":{"leaguesWon":"Leagues Won","titlesWon":"Titles Won","totalCupsWon":"Total Cups Won"},"cupsImg":{"leaguesWonImgUrl":"https://media.contentapi.ea.com/content/dam/ea/fifa/fifa-21/pro-clubs/common/pro-clubs/league-titles-21.png","titlesWonImgUrl":"https://media.contentapi.ea.com/content/dam/ea/fifa/fifa-21/pro-clubs/common/pro-clubs/all-tiles-21.png","totalCupsWonImgUrl":"https://media.contentapi.ea.com/content/dam/ea/fifa/fifa-21/pro-clubs/common/pro-clubs/cups-won-21.png"}}" history-labels="{"title":"Club History","subTitle":"Overall Record","pts":"Points","division":"Division","historyDetails":{"seasons":"Seasons Played","totalGames":"Total Games","titlesWon":"Titles Won","bestPoints":"Highest Points Total","promotions":"Promotions","relegations":"Relegations"},"stats":{"wins":"Wins","losses":"Losses","ties":"Draws"},"statsShort":{"wins":"W","losses":"L","ties":"D"},"progressBar":{"title":"Best Season Finish","tipLabel":"DIV","startColor":"#9B7801","endColor":"#F9F1A5","divisionBaseUrl":"https://media.contentapi.ea.com/content/dam/eacom/fifa/pro-clubs/divisioncrest"}}" translations="{"4543827":"East Coast US","5723475":"West Coast US","5719381":"Western Europe","4539733":"Eastern Europe","5129557":"Northern Europe","5457237":"Southern Europe","4344147":"British Isles","5456205":"South America","4407629":"Central America","4281153":"Asia","4281683":"Australia / New Zealand"}" crest-base-url="https://fifa21.content.easports.com/fifa/fltOnlineAssets/05772199-716f-417d-9fe0-988fa9899c4d/2021/fifaweb/crests/256x256/l" custom-crest-base-url="https://fifa21.content.easports.com/fifa/fltOnlineAssets/05772199-716f-417d-9fe0-988fa9899c4d/2021/fifaweb/crests/256x256/l" default-crest-url="https://media.contentapi.ea.com/content/dam/ea/fifa/fifa-21/pro-clubs/common/pro-clubs/crest-default.png" loading-image="https://media.contentapi.ea.com/content/dam/eacom/fifa/pro-clubs/loading-animation.png" default-club-name="Disbanded"></ea-proclub-overview>
Actual Result
-- Too much post but it all of the HTML & XML
Below is pastebin of the entire response from the $crawler when dumped out using dd().
https://pastebin.com/qxUTpu9p
According to the documentation:
$customCrestBaseUrl = $crawler
->filter('ea-proclub-overview')
->first()
->extract(['custom-crest-base-url'])
;
i been trying to fetch some data from wikia website by using simple_html_dom lib for php. basically what i do is to use the wikia api to convert into html render and extract data from there. After extracting, i will pump those data into mysql database to save. My problem is that, usually i will pull 300 records and i will stuck on 93 records with file_get_html being null which will cause my find() function to fail. I am not sure why is it stopping at 93 records but i have tried various solution such as
ini_set( 'default_socket_timeout', 120 );
set_time_limit( 120 );
basically i will have to access wikia page for 300 times to get those 300 records. But mostly i will manage to get 93 records before file_get_html gets to null. Any idea how can i tackle this issue?
i have test curl as well and have the same issue.
function test($url){
$ch=curl_init();
$timeout=5;
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, $timeout);
$result=curl_exec($ch);
curl_close($ch);
return $result;
}
$baseurl = 'http://xxxx.wikia.com/index.php?';
foreach($resultset_wiki as $name){
// Create DOM from URL or file
$options = array("action"=>"render","title"=>$name['name']);
$baseurl .= http_build_query($options,'','&');
$html = file_get_html($baseurl);
if($html === FALSE) {
echo "issue here";
}
// this code for cURL but commented for testing with file_get_html instead
$a = test($baseurl);
$html = new simple_html_dom();
$html->load($a);
// find div stuff here and mysql data pumping here.
}
$resultsetwiki is an array with the list of title to fetch from wikia, basically resultsetwiki data set is load from db as well before performing the search.
practically i will it this type of error
Call to a member function find() on a non-object in
answered my own issue, seems to be the URL that i am using and i have changed to curl with post to post the action and title parameter instead
I tried implementing the following PHP code to POST JSON via PHP: cURL (SOME FORCE.COM WEBSITE is a tag that signifies the URL that I want to POST):
$url = "<SOME FORCE.COM WEBSITE>";
$data =
'application' =>
array
(
'isTest' => FALSE,
key => value,
key => value,
key => value,
...
)
$ch = curl_init($url);
$data_string = json_encode($data);
curl_setopt($ch, CURLOPT_POST, true);
//Send blindly the json-encoded string.
//The server, IMO, expects the body of the HTTP request to be in JSON
curl_setopt($ch, CURLOPT_POSTFIELDS, $data_string);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER,
array
(
'Content-Type:application/json',
'Content-Length: ' . strlen($data_string)
)
);
$result = curl_exec($ch);
curl_close($ch);
print_r($result);
echo '<pre>';
echo $_POST;
$jsonStr = file_get_contents('php://input'); //read the HTTP body.
var_dump($jsonStr);
var_dump(json_decode($jsonStr));
echo '</pre>';
The output of the above is the following:
"Your TEST POST is correct, please set the isTest (Boolean) attribute on the application to FALSE to actually apply."
Arraystring(0) ""
NULL
OK, the above confirms that I formatted the JSON data correctly by using json_encode, and the SOME FORCE.COM WEBSITE acknowledges that the value of 'isTest' is FALSE. However, I am not getting anything from "var_dump($jsonStr)" or "var_dump(json_decode($jsonStr))". I decided to just ignore that fact and set 'isTest' to FALSE, assuming that I am not getting any JSON data because I set 'isTest' to TRUE, but chaos ensues when I set 'isTest' to FALSE:
[{"message":"System.EmailException: SendEmail failed. First exception on row 0; first error: REQUIRED_FIELD_MISSING, Missing body, need at least one of html or plainText: []\n\nClass.careers_RestWebService.sendReceiptEmail: line 165, column 1\nClass.careers_RestWebService.postApplication: line 205, column 1","errorCode":"APEX_ERROR"}]
Arraystring(0) ""
NULL
I still do not get any JSON data, and ultimately, the email was unable to be sent. I believe that the issue is resulting from an empty email body because there is nothing coming from "var_dump($jsonStr)" or "var_dump(json_decode($jsonStr))". Can you help me retrieve the JSON POST? I would really appreciate any hints, suggestions, etc. Thanks.
I solved this question on my own. I was not sure if I was doing this correctly or not, but it turns out that my code was perfect. I kept refreshing my website, from where I am POSTing to SOME FORCE.COM WEBSITE. I believe that the people managing SOME FORCE.COM WEBSITE were having issues on their end. Nothing was wrong with what I did. For some reason, I got a code 202 and some gibberish text to go along with it. I would be glad to show the output, but I do not want to POST again for the sake of the people managing SOME FORCE.COM WEBSITE that I am POSTing to. Thank you guys for your help.
I'm using the default PHP Solr client:
http://php.net/manual/en/book.solr.php
Up until now, the results are amazing so I decided to continue using Solr, and develop a "suggester" service:
I did according to:
http://wiki.apache.org/solr/Suggester/
It's also working great - but I have no idea how to query it using the PHP client.
This is how I query for documents:
http://localhost:8080/solr/subject_offers/select/?q=string_to_search
This is how I query for suggestions:
http://localhost:8080/solr/subject_offers/suggest/?q=string_to_suggest
As you can see, the suggestion service have a different RequestHandler ( called 'suggest').
How can I change it on the client?
Update:
I'm using cURL for retrieving the response XML from Solr.
But I still, want to return a Solr response object, so I'm using "SolrUtils::digestXmlResponse($xml, SolrResponse::PARSE_SOLR_OBJ)".
Unfortunately I'm getting an error of "Error un-serializing response" for a valid XML:
<response><lst name="responseHeader"><int name="status">0</int><int name="QTime">2</int></lst><lst name="spellcheck"><lst name="suggestions"><lst name="the"><int name="numFound">1</int><int name="startOffset">9</int><int name="endOffset">12</int><arr name="suggestion"><str>the</str></arr></lst><lst name="the"><int name="numFound">1</int><int name="startOffset">33</int><int name="endOffset">36</int><arr name="suggestion"><str>the</str></arr></lst><lst name="collation"><str name="collationQuery">name:for the d^5 description:for the d^0.4</str><int name="hits">1</int><lst name="misspellingsAndCorrections"><str name="the">the</str><str name="the">the</str></lst></lst></lst></lst><result name="response" numFound="1" start="0"><doc><str name="1_on_1_price">9,USD</str><float name="avarage_rating">0.0</float><arr name="categories"><str>2,2,3</str></arr><int name="category_id">3</int><arr name="description"><str>was named for the directo</str><str>The Manchesowned by Peel Ports</str></arr><bool name="is_public">true</bool><str name="language">he</str><int name="lesson_type">0</int><str name="name">was named for the directo</str><int name="subject_id">12</int></doc></result></response>
Please advise,
thanks!
I had the same problem and I found answer in the source code. The solution is:
$client = new SolrClient($options);
$client->setServlet(SolrClient::SEARCH_SERVLET_TYPE, "request_handler_name");
You can use CURL extension for PHP to do this. A Small example would be:
$url = 'http://localhost:8080/solr/subject_offers/select/?';
$to_post = array('q' => urlencode('string_to_search'),
'other'=> urlencode('somthing_else'));
foreach($to_post as $key=>$value) {
$string .= $key.'='.$value.'&';
}
rtrim($string, '&');
$ch = curl_init();
curl_setopt($ch,CURLOPT_URL,$url);
curl_setopt($ch,CURLOPT_POST,count($to_post));
curl_setopt($ch,CURLOPT_POSTFIELDS,$string);
$result = curl_exec($ch);
curl_close($ch);
Hope that Helps...
I am trying to get a list of results from Freebase. I have an array of MIDs. Can someone explain how I would structure the query and pass it to the API in PHP?
I'm new to MQL - I can't even seem to get the example to work:
$simplequery = array('id'=>'/topic/en/philip_k_dick', '/film/writer/film'=>array());
$jsonquerystr = json_encode($simplequery);
// The Freebase API requires a query envelope (which allows you to run multiple queries simultaneously) so we need to wrap our original, simplequery structure in two more arrays before we can pass it to the API:
$queryarray = array('q1'=>array('query'=>$simplequery));
$jsonquerystr = json_encode($queryarray);
// To send the JSON formatted MQL query to the Freebase API use cURL:
#run the query
$apiendpoint = "http://api.freebase.com/api/service/mqlread?queries";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "$apiendpoint=$jsonquerystr");
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$jsonresultstr = curl_exec($ch);
curl_close($ch);
// Decoding the JSON structure back into arrays is performed using json_decode as in:
$resultarray = json_decode($jsonresultstr, true); #true:give us the json struct as an array
// Iterating over the pieces of the resultarray containing films gives us the films Philip K. Dick wrote:
$filmarray = $resultarray["q1"]["result"]["/film/writer/film"];
foreach($filmarray as $film){
print "$film<br>";
}
You're doing everything right. If you weren't, you'd be getting back error messages in your JSON result.
I think what's happened is that the data on Philip K. Dick has been updated to identify him not as the "writer" of films, but as a "film_story_contributor". (He didn't, after all, actually write any of the screenplays.)
Change your simplequery from:
$simplequery = array('id'=>'/topic/en/philip_k_dick', '/film/writer/film'=>array());
To:
$simplequery = array('id'=>'/topic/en/philip_k_dick', '/film/film_story_contributor/film_story_credits'=>array());
You actually can use the Freebase website to drill down into topics to dig up this information, but it's not that easy to find. On the basic Philip K. Dick page (http://www.freebase.com/view/en/philip_k_dick), click the "Edit and Show details" button at the bottom.
The "edit" page (http://www.freebase.com/edit/topic/en/philip_k_dick) shows the Types associated with this topic. The list includes "Film story contributor" but not "writer". Within the Film story contributor block on this page, there's a "detail view" link (http://www.freebase.com/view/en/philip_k_dick/-/film/film_story_contributor/film_story_credits). This is, essentially, what you're trying to replicate with your PHP code.
A similar drill-down on an actual film writer (e.g., Steve Martin), gets you to a property called /film/writer/film (http://www.freebase.com/view/en/steve_martin/-/film/writer/film).
Multiple Queries
You don't say exactly what you're trying to do with an array of MIDs, but firing multiple queries is as simple as adding a q2, q3, etc., all inside the $queryarray. The answers will come back inside the same structure - you can pull them out just like you pull out the q1 data. If you print out your jsonquerystr and jsonresultstr you'll see what's going on.
Modified a bit to include answer into question, as this helped me I've upvoted each, just thought I would provide a more "compleat" answer, as it were:
$simplequery = array('id'=>'/topic/en/philip_k_dick', '/film/film_story_contributor/film_story_credits'=>array());
$jsonquerystr = json_encode($simplequery);
// The Freebase API requires a query envelope (which allows you to run multiple queries simultaneously) so we need to wrap our original, simplequery structure in two more arrays before we can pass it to the API:
$queryarray = array('q1'=>array('query'=>$simplequery));
$jsonquerystr = json_encode($queryarray);
// To send the JSON formatted MQL query to the Freebase API use cURL:
#run the query
$apiendpoint = "http://api.freebase.com/api/service/mqlread?queries";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "$apiendpoint=$jsonquerystr");
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$jsonresultstr = curl_exec($ch);
curl_close($ch);
// Decoding the JSON structure back into arrays is performed using json_decode as in:
$resultarray = json_decode($jsonresultstr, true); #true:give us the json struct as an associative array
// Iterating over the pieces of the resultarray containing films gives us the films Philip K. Dick wrote:
if($resultarray['code'] == '/api/status/ok'){
$films = $resultarray['q1']['result']['/film/film_story_contributor/film_story_credits'];
foreach ($films as $film){
print "$film</br>";
}
}