Optimising Performance: Many CURL requests - php

I am in the process of developing a web-based RESTful api for a client. Everything works great apart from one request, in which I need to request the Foursquare API for each row.
The URL for this request is: http://api.example.com/v1/users/times.
Currently the response of a request to that url is:
{
"response": {
"user": {
... some user info ...
"times": [
{
"id": "8",
"venue_fq_id": "4b81eb25f964a52000c430e3",
"user_id": "1",
"wait_length": "4468",
"created_at": "2012-06-09 21:45:43"
},
{
"id": "9",
"venue_fq_id": "4aad285af964a520c05e20e3",
"user_id": "1",
"wait_length": "8512",
"created_at": "2012-06-09 21:45:43"
},
{
"id": "10",
"venue_fq_id": "42377700f964a52024201fe3",
"user_id": "1",
"wait_length": "29155",
"created_at": "2012-06-09 21:45:44"
},
{
"id": "11",
"venue_fq_id": "45c88764f964a5206e421fe3",
"user_id": "1",
"wait_length": "33841",
"created_at": "2012-06-09 21:45:44"
},
{
"id": "12",
"venue_fq_id": "430d0a00f964a5203e271fe3",
"user_id": "1",
"wait_length": "81739",
"created_at": "2012-06-09 21:45:44"
}
]
}
},
"stat": "ok"
}
However, the venue_fq_id being returned in the response.user.times array is relative to a venue on the Foursquare API. I tried running a curl request to the Foursquare API for each row, but the performance is incredibly slow. Please can you give some examples of ways in which I could speed up the performance while retrieving the same information I would had I accessed requested the F/Q API each time?
Here's my code:
$query = $this->db->query("SELECT * FROM `wait_times` WHERE `user_id` = ?", array($email_address));
$wait_times = $query->result();
foreach ($wait_times as $wait_time) {
$wait_time->venue = $this->venue_info($wait_time->venue_fq_id);
}
function venue_info($fq_id) {
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "https://api.foursquare.com/v2/venues/4b522afaf964a5200b6d27e3");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$response = json_decode(curl_exec($ch));
curl_close($ch);
return $response['response']['venue'];
}

You're wasting a horrendous amount of time instantiating/tearing down CURL objects. This prevents you from taking advantage of HTTP keep-alives, forcing curl to start a new tcp connection for every request you make.
Curl handles CAN be reused. e.g.
$ch = curl_init();
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
function venue_info($fq_id) {
global $ch;
curl_setopt($ch, CURLOPT_URL, "https://api.foursquare.com/v2/venues/4b522afaf964a5200b6d27e3");
$resp = curl_exec($ch) or die(curl_error($ch));
$json = json_decode($resp);
return($json);
}

The Foursquare API provides the multi endpoint to allow you to combine up to five queries into a single request.

Related

Post Json API giving error 500, what I'm doing wrong?

I'm trying to create a new client in my API using php cURL. Clients, products and everything that is created is by POST method. This is my code:
$json='{
"data": {
"type": "customers",
"attributes": {
"tax_registration_number": "5555555550",
"business_name": "Test customer",
"contact_name": "Mr. Test",
"website": "https://www.testurl.pt",
"phone_number": "2299999999",
"mobile_number": "9299999999",
"email": "test#testcustomer.pt",
"observations": "This is only a test",
"internal_observations": "This is good customer",
"not_final_customer": null,
"cashed_vat": null,
"tax_country_region": "PT-MA"
},
"relationships": {
"main_address": {
"data": {
"type": "addresses",
"id": 1
}
},
"addresses": {
"data": [
{
"type": "addresses",
"id": 1
},
{
"type": "addresses",
"id": 2
}
]
}
}
}
}';
print($json);
Here I iniciate the cURL, I have the token and the authorization already:
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,($url));
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS,($json));
curl_setopt($ch, CURLOPT_HTTPHEADER, array(
'Content-Type: application/vnd.api+json',
'Accept: application/json',
'Authorization: Bearer ' . $token,
));
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$response = curl_exec($ch);
var_dump ($response);
$response=json_decode($response,true);
curl_close ($ch);
This is the response that I have:
string(329) "{"errors":[{"status":"500 Internal Server Error","code":"JA006","detail":"Erro de sistema JA006: erro interno na base de dados. Por favor contacte o suporte técnico.","meta":{"internal-error":"in JsonapiJson::Value::operator[](ArrayIndex)const: requires arrayValue"}}],"jsonapi":{"version": "1.0","meta":{"libversion":"2.4.1"}}}"
Can someone please help me?
Thanks!
The API error states this: in JsonapiJson::Value::operator[](ArrayIndex)const: requires arrayValue.
Are you sure that JSON is the correct format? It seems like you are probably providing an object in the JSON where the server expects an array of objects. For example, check if data, or relationships, or addresses should be arrays.
My top guess would be instead of:
"addresses": {
"data": [
{
"type": "addresses",
"id": 1
},
{
"type": "addresses",
"id": 2
}
]
}
Maybe it's supposed to be
"addresses":[
{
"type": "addresses",
"id": 1
},
{
"type": "addresses",
"id": 2
}
]
I certainly can't tell you for sure, since I don't know the API you are trying to use, but I highly suspect it's a case like this where it's expecting an array, but you're providing an object.

Trying to save contacts with GetResponse api but I am unable to

My task is to authenticate on this api
https://api.getresponse.com/v3/accounts
Then need to save on the following api
https://api.getresponse.com/v3/POST/contacts
data must be in the following format
{
"name": "Hemant Maurya",
"email": "xyz#yahoo.com",
"dayOfCycle": "0",
"campaign": {
"campaignId": "6mzZL"
},
"tags": [
{
"tagId": "Xw"
},
{
"tagId": "Nn"
}
],
"scoring": 25,
"customFieldValues": [
{
"customFieldId": "n",
"value": [
"white"
]
}
],
"ipAddress": "14.143.38.111"
}
Following code can authenticate
$url = "https://api.getresponse.com/v3/accounts";
$headers = array();
$headers[] = "X-Auth-Token: api-key 17fbe43cc8a23daaf36b35093c77djet";//api key is fake at the moment
$state_ch = curl_init();
curl_setopt($state_ch, CURLOPT_URL, $url);
curl_setopt($state_ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($state_ch, CURLOPT_HTTPHEADER, $headers);
$state_result = curl_exec ($state_ch);
$state_result = json_decode($state_result);
$debug = 1;
print_r($state_result);
The response is as following
{
"accountId": "fjnfd",
"firstName": "first name",
"lastName": "last name",
"email": "xyz.com#gmail.com",
"phone": "+3798798",
"companyName": "",
"state": "state address",
"city": "city address",
"street": "street address",
"zipCode": "226010",
"countryCode": {
"countryCodeId": "100",
"countryCode": "IN"
},
"industryTag": {
"industryTagId": null
},
"numberOfEmployees": null,
"timeFormat": "12h",
"timeZone": {
"name": "Asia/Kolkata",
"offset": "+09:30"
},
"href": "https://api.getresponse.com/v3/accounts"
}
I am not getting how to save the data tried GetResponseAPI3.class.php from Github but it's not happening.
get response documentation
Links that can Help
Authentication
Saving contacts
You may vote it down but I have been trying for last 3 days and it's not happening.
The thing is that you don't need to send a request to https://api.getresponse.com/v3/accounts to authenticate your calls. The data that you receive means your authentication headers are probably correct, so can directly proceed to sending POST requests to https://api.getresponse.com/v3/contacts. Your payload looks fine so you should make it successfully.

Error getting a json response from http request

I am trying to get the response from an http request from the Hunter API.
The url is something like this :
https://api.hunter.io/v2/email-finder?domain=mydomain&api_key=myapikey&first_name=myfirstname&last_name=myname
And the response is something like this :
{
"data": {
"email": "emailfound#domain.tld",
"score": 68,
"domain": "domain.tld",
"sources": [
{
"domain": "domain.tld",
"uri": "http://somedomain.tld/",
"extracted_on": "2017-04-04"
}
]
},
"meta": {
"params": {
"first_name": "myfirstname",
"last_name": "mylastname",
"full_name": null,
"domain": "domain.tld",
"company": null
}
}
}
Here is what I am doing in my controller :
$curl = curl_init();
curl_setopt($curl, CURLOPT_URL, $url);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
$response = curl_exec($curl);
curl_close($curl);
// return response
$json = new JsonResponse();
$datas = $json->setData($response);
return json_encode($datas);
And here is what json_encode($datas) returns :
{"headers":{}}
I precise that I am working with Symfony3 and testing on my OVH serveur (Performance offer).
Any idea where does come from this response ? and why I am not getting the real response ? Thanks !
As per my comment,
Instead of using CURL (Which is a much harder system to deal with IMO), you should use file_get_contents() like so:
$json = file_get_contents($url);
$obj = json_decode($json,true);
Try this:
$json = file_get_contents('https://api.hunter.io/v2/email-finder?domain=mydomain&api_key=myapikey&first_name=myfirstname&last_name=myname');
$data = json_encode($json,true)
var_dump($data);

wowza rest api to php script

im new to php and wowza and was wondering if anyone can point me to a tutorial on how to use this wowza curl api with php? ive tried searching but cant find the answer anywhere so i came here. what im trying to achive is run this curl via php from a remote computer
this is one of the curl id like to convert to php script however i cant seem to find where or how i should start
curl -X POST --header 'Accept:application/json; charset=utf-8' --header 'Content-type:application/json; charset=utf-8' http://localhost:8087/v2/servers/_defaultServer_/vhosts/_defaultVHost_/applications/testlive -d'
{
"restURI": "http://localhost:8087/v2/servers/_defaultServer_/vhosts/_defaultVHost_/applications/testlive",
"name": "testlive",
"appType": "Live",
"description": "Testing our Rest Service",
"streamConfig": {
"restURI": "http://localhost:8087/v2/servers/_defaultServer_/vhosts/_defaultVHost_/applications/testlive/streamconfiguration",
"streamType": "live"
},
"securityConfig": {
"restURI": "http://localhost:8087/v2/servers/_defaultServer_/vhosts/_defaultVHost_/applications/testlive/security",
"secureTokenVersion": 0,
"clientStreamWriteAccess": "*",
"publishRequirePassword": true,
"publishPasswordFile": "",
"publishRTMPSecureURL": "",
"publishIPBlackList": "",
"publishIPWhiteList": "",
"publishBlockDuplicateStreamNames": false,
"publishValidEncoders": "",
"publishAuthenticationMethod": "digest",
"playMaximumConnections": 0,
"playRequireSecureConnection": false,
"secureTokenSharedSecret": "",
"secureTokenUseTEAForRTMP": false,
"secureTokenIncludeClientIPInHash": false,
"secureTokenHashAlgorithm": "",
"secureTokenQueryParametersPrefix": "",
"secureTokenOriginSharedSecret": "",
"playIPBlackList": "",
"playIPWhiteList": "",
"playAuthenticationMethod": "none"
},
"modules": {
"restURI": "http://localhost:8087/v2/servers/_defaultServer_/vhosts/_defaultVHost_/applications/testlive/modules",
"moduleList": [
{
"order": 0,
"name": "base",
"description": "Base",
"class": "com.wowza.wms.module.ModuleCore"
},
{
"order": 1,
"name": "logging",
"description": "Client Logging",
"class": "com.wowza.wms.module.ModuleClientLogging"
},
{
"order": 2,
"name": "flvplayback",
"description": "FLVPlayback",
"class": "com.wowza.wms.module.ModuleFLVPlayback"
},
{
"order": 3,
"name": "ModuleCoreSecurity",
"description": "Core Security Module for Applications",
"class": "com.wowza.wms.security.ModuleCoreSecurity"
}
]
}
}'
As mentioned above, take a look at php cURL extension. Then check out the following example script:
// If you have digest auth turned on, switch this to true.
$useDigest = false;
$username = "admin";
$password = "pass";
$json = '{
"restURI": "http://localhost:8087/v2/servers/_defaultServer_/vhosts/_defaultVHost_/applications/stackoverflow",
"name": "stackoverflow",
"appType": "Live",
"description": "Testing our Rest Service",
"streamConfig": {
"restURI": "http://localhost:8087/v2/servers/_defaultServer_/vhosts/_defaultVHost_/applications/stackoverflow/streamconfiguration",
"streamType": "live"
},
"securityConfig": {
"restURI": "http://localhost:8087/v2/servers/_defaultServer_/vhosts/_defaultVHost_/applications/stackoverflow/security",
"secureTokenVersion": 0,
"clientStreamWriteAccess": "*",
"publishRequirePassword": true,
"publishPasswordFile": "",
"publishRTMPSecureURL": "",
"publishIPBlackList": "",
"publishIPWhiteList": "",
"publishBlockDuplicateStreamNames": false,
"publishValidEncoders": "",
"publishAuthenticationMethod": "digest",
"playMaximumConnections": 0,
"playRequireSecureConnection": false,
"secureTokenSharedSecret": "",
"secureTokenUseTEAForRTMP": false,
"secureTokenIncludeClientIPInHash": false,
"secureTokenHashAlgorithm": "",
"secureTokenQueryParametersPrefix": "",
"secureTokenOriginSharedSecret": "",
"playIPBlackList": "",
"playIPWhiteList": "",
"playAuthenticationMethod": "none"
},
"modules": {
"restURI": "http://localhost:8087/v2/servers/_defaultServer_/vhosts/_defaultVHost_/applications/stackoverflow/modules",
"moduleList": [
{
"order": 0,
"name": "base",
"description": "Base",
"class": "com.wowza.wms.module.ModuleCore"
},
{
"order": 1,
"name": "logging",
"description": "Client Logging",
"class": "com.wowza.wms.module.ModuleClientLogging"
},
{
"order": 2,
"name": "flvplayback",
"description": "FLVPlayback",
"class": "com.wowza.wms.module.ModuleFLVPlayback"
},
{
"order": 3,
"name": "ModuleCoreSecurity",
"description": "Core Security Module for Applications",
"class": "com.wowza.wms.security.ModuleCoreSecurity"
}
]
}
}';
$ch = curl_init("http://localhost:8087/v2/servers/_defaultServer_/vhosts/_defaultVHost_/applications/stackoverflow");
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "POST");
curl_setopt($ch, CURLOPT_POSTFIELDS, $json);
if($useDigest){
curl_setopt($ch, CURLOPT_USERPWD, $username . ":" . $password);
curl_setopt($ch, CURLOPT_HTTPAUTH, CURLAUTH_DIGEST);
}
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, array(
'Accept:application/json; charset=utf-8',
'Content-type:application/json; charset=utf-8',
'Content-Length: '.strlen($json)
));
$contents = curl_exec($ch);
curl_close($ch);
$response = json_decode($contents);
var_dump($response);
This will yield similar output to the following:
object(stdClass)#1 (3) {
["success"]=>
bool(true)
["message"]=>
string(49) "Application (stackoverflow) created successfully."
["data"]=>
NULL
}
You can find several other cURL examples that you can simply manipulate the JSON being sent along with the verb type (in this case POST) to further leverage the REST API.
An example of a GET request to retrieve your existing applications, you can remove the CURLOPT_POSTFIELDS cURL option and modify the following lines:
$ch = curl_init("http://localhost:8087/v2/servers/_defaultServer_/vhosts/_defaultVHost_/applications");
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "GET");

How to convert provided single line of cURL code with inline parameters to a format using curl_setopt()

I'm trying to use an API from avalara.net to get sales tax data and their API documentation lists what appears to be essentially a single line of code with a long string in JSON format for connecting to their API via cURL; I'm not familiar with this single string style and am trying to convert it into a format that follows the php manual's style, for example setting options via curl_setopt() instead of as inline commands like -u and -d but I'm not getting anywhere.
I already looked at php - implement curl command line to php, tried but getting error and added their solution - since I didn't get any error message I don't know if my problem was the same but it seemed like it wouldn't hurt to add CURLOPT_FOLLOWLOCATION. Didn't seem to make any difference.
I also looked at convert curl line command in php code but it didn't help as I already had curl_exec() in my code and had been reading the php manual pages.
Their code can be seen at https://github.com/avadev/AvaTax-Calc-REST-cURL/blob/master/tax-get-POST.txt and in relevant part is:
curl -u <AccountNumber>:<LicenseKey> -H "Content-Type: text/json" -d '{
--long string of data omitted; see link above for full data--
}' "https://development.avalara.net/1.0/tax/get"
I did some research and discovered that -u is for a username/password, -H is a special header, and -d is for the data to be sent... so this is what I put together:
// Identify the target URL
$url = 'https://development.avalara.net/1.0/tax/get';
// Start the process
$curl = curl_init($url);
// Tell cURL to fail if an error occurs
curl_setopt($curl, CURLOPT_FAILONERROR, 1);
// Allow for redirects; we don't know how avalara might route requests
curl_setopt($curl, CURLOPT_FOLLOWLOCATION, 1);
// Assign the returned data to a variable
curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1);
// Set the timeout
curl_setopt($curl, CURLOPT_TIMEOUT, 5);
// Make sure to use POST method
curl_setopt($curl, CURLOPT_POST, 1);
// Set cURL to use a login:password for access
curl_setopt($curl, CURLOPT_USERPWD, "[1100033004]:[1FC8AED1543C699B]");
// Add some custom header info
curl_setopt($curl, CURLOPT_HTTPHEADER, array('Content-type: text/json'));
// Use Heredoc syntax to encapsulate data, to avoid having to escape a million quotes
$data = <<<EOD
{
"DocDate": "2013-12-23",
"CustomerCode": "12345678123456781234567812345678",
"CompanyCode": "EGU",
"DocType": "SalesInvoice",
"Commit": false,
"Client": "Cool ERP,3,5",
"DocCode": "29",
"DetailLevel": "Tax",
"CustomerUsageType": "G",
"ExemptionNo": "12334",
"Discount": 0,
"PurchaseOrderNo":"PO 23423",
"ReferenceCode":"",
"PosLaneCode":"",
"BusinessIdentificationNo":"",
"TaxOverride":
{
"Reason":"Item Returned",
"TaxDate":"2013-05-05",
"TaxOverrideType":"TaxDate"
},
"Addresses":
[
{
"AddressCode": "Origin",
"Line1": "269",
"Line2": "7723 Tylers Place Blvd",
"City": "West Chester",
"Region": "OH",
"PostalCode": "45069-4684",
"Country": "US"
},
{
"AddressCode": "Dest",
"Line1": "1060 W. Addison St",
"City": "Chicago",
"Region": "IL",
"PostalCode": "60613-4566",
"Country": "US"
}
],
"Lines":
[
{
"LineNo": "00001",
"DestinationCode": "Dest",
"OriginCode": "Origin",
"ItemCode": "SP-001",
"Description": "Eyeglasses",
"TaxCode": "PC030147",
"Qty": 1,
"Amount": 100
}
]
}
EOD;
// That EOD ends the encapsulation via Heredoc syntax
// Note that the actual code does not indent the closing of Heredoc; only indented for stack overflow code view
// Set the POST data
curl_setopt($curl, CURLOPT_POSTFIELDS, $data);
// Execute the transaction
$r = curl_exec($curl);
// Close the connection
curl_close($curl);
// Print the results for debugging
print_r($r);
However, when I try to view the web page, I see nothing - no error message, no results, nothing.
I've looked at some of the other posts here on stackoverflow and in the php manual - I'm not sure at this point how to even debug it as I'm not getting any error messages back. Any thoughts would be appreciated.
This is a demo example to demonstrate the using of API in PHP this may help you to post some data to the API
$url = '<your url here>';
$curl = curl_init($url);
$data_string = '[
{
"FirstName": "Value"
},
{
"EmailAddress": "Value"
},
{
"Phone": "Value"
}
]';
curl_setopt($curl, CURLOPT_CUSTOMREQUEST, "POST");
curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($curl, CURLOPT_HEADER, 0);
curl_setopt($curl, CURLOPT_POSTFIELDS, $data_string);
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($curl, CURLOPT_HTTPHEADER, array(
'Content-Type:application/json',
'Content-Length:'.strlen($data_string)
));
$json_response = curl_exec($curl);
$curl_errorno = curl_errno($curl);
$status = curl_getinfo($curl, CURLINFO_HTTP_CODE);
if ($status) {
echo 'Status is 200 OK! ';
}else{
echo "Sorry Something went wrong. Please retry!";
curl_close($curl);
}
figure out what you're getting in status code that will help you.
EDIT
this will be the value of your
$data_string=
'{
"DocDate": "2013-12-23",
"CustomerCode": "12345678123456781234567812345678",
"CompanyCode": "EGU",
"DocType": "SalesInvoice",
"Commit": false,
"Client": "Cool ERP,3,5",
"DocCode": "29",
"DetailLevel": "Tax",
"CustomerUsageType": "G",
"ExemptionNo": "12334",
"Discount": 0,
"PurchaseOrderNo": "PO 23423",
"ReferenceCode": "",
"PosLaneCode": "",
"BusinessIdentificationNo": "",
"TaxOverride": {
"Reason": "Item Returned",
"TaxDate": "2013-05-05",
"TaxOverrideType": "TaxDate"
},
"Addresses": [
{
"AddressCode": "Origin",
"Line1": "269",
"Line2": "7723 Tylers Place Blvd",
"City": "West Chester",
"Region": "OH",
"PostalCode": "45069-4684",
"Country": "US"
},
{
"AddressCode": "Dest",
"Line1": "1060 W. Addison St",
"City": "Chicago",
"Region": "IL",
"PostalCode": "60613-4566",
"Country": "US"
}
],
"Lines": [
{
"LineNo": "00001",
"DestinationCode": "Dest",
"OriginCode": "Origin",
"ItemCode": "SP-001",
"Description": "Eyeglasses",
"TaxCode": "PC030147",
"Qty": 1,
"Amount": 100
}
]
}'
ANSWER
Remoe the braces form this line like this
curl_setopt($curl, CURLOPT_USERPWD, "1100033004:1FC8AED1543C699B");

Categories