Manipulating curl-obtained data before outputting - php

I'm using the below code to pull the html text from a site to publish on my own.
How can I manipulate the curl handle before echo?
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_VERBOSE, 1);
curl_setopt($ch, CURLOPT_POST, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER,1);
$returned = curl_exec($ch);
curl_close ($ch);
echo $returned;


You can manipulate your returned data:
$returned = curl_exec($ch);
curl_close ($ch);
$returned is the response so you can manipulate it
Did you mean something like this

Related

How to scrape data from Ajax or Json

I want to scrape data from this url
I am able to fetch simple data from html tags usign curl but not able to fetch data from Json or Ajax, I am not sure is it Ajax or Json data.
In below screen shot I want to fetch Appliance Models Data.
Which is coming form I think json or ajax. ==>>
Here below is my script to get data from page -
$loginURL = "https://www.apwagner.com/appliance-part/wpl/wp661600";
//$file='source.html'; //create a html file to save source code
$ch = curl_init();
$timeout = 5;
curl_setopt($ch, CURLOPT_URL, $loginURL);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, $timeout);
$data = curl_exec($ch);
curl_close($ch);
Please provide some guidance to fetch this info ..
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,"https://www.apwagner.com/Product/GetPartModel");
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS,
"partNumber=wp661600&make=wpl");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$server_output = curl_exec ($ch);
curl_close ($ch);

The part of data page gets through ajax request.
see this screenshot
You need to do it with curl
after your first curl response is received
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,"https://www.apwagner.com/Product/GetPartModel");
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS,
"partNumber=wp661600&make=wpl");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$server_output = curl_exec ($ch);
curl_close ($ch);
Or try to scrap data using python script
import string
import time
from selenium import webdriver
driver = webdriver.Chrome('<path to your chrome driver>')
driver.get('https://www.apwagner.com/appliance-part/wpl/wp661600');

How to pass PHP_session_id in curl request

I am learning PHP and trying to automate a website login and then post some data to another page once logged in. I managed to login to the website and when I try to post data, I got a "Document moved".
Then I analysed the headers in firebug and realised that there was a PHP_session_id, so when I tried to manually pass this PHP_session_id it worked.
So my question is, how can I automatically get the sessionid when I login and then subsequently pass this on to my second request?
This is what my code looks like:
<?php
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,"http://www.example.com/login-main.php");
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS,
"loginType=company&username=johndoe%40gmail.com&password=1234&company=test");
ob_start();
curl_exec ($ch);
ob_end_clean();
curl_close ($ch);
unset($ch);
$ch = curl_init();
curl_setopt($ch, CURLOPT_RETURNTRANSFER,1);
curl_setopt($ch, CURLOPT_HTTPHEADER, array(
'Host:www.example.com',
'Origin:http://www.example.com',
'Cookie:PHPSESSID=na6ivnsdfab206ktb453o2au07',
'Referer:http://www.example.com/bookings/'
));
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_URL,"http://www.example.com/bookings.php");
curl_setopt($ch, CURLOPT_POSTFIELDS,
"start_date=&end_date=&type=instructor&b_type=&id=41");
$buf2 = curl_exec ($ch);
curl_close ($ch);
echo "<PRE>".htmlentities($buf2);
?>

Add to your code in all curl requests
curl_setopt($ch, CURLOPT_COOKIEJAR, $full_path_to_cookie_file);
curl_setopt($ch, CURLOPT_COOKIEFILE, $full_path_to_cookie_file);
For example
$full_path_to_cookie_file= __DIR__.'/cookie.txt';

how to send post request by curl to web services in php

I need to fetch data from myallocator.com. I have his api address api.myallocator.com,
but when I send my data to them, it failed. I wrote some code for that which is shown below. Basically that pms property management system site.By that you can manage your property with his api.
$url = "http://api.myallocator.com/";
$xmlRequestString='<?xml version="1.0" encoding="UTF-8"?>
<GetProperties>
<Auth>
<UserId>"xxxxxxx"</UserId>
<UserPassword>"xxxxx"</UserPassword>
<VendorId>"xxxxxxx"</VendorId>
<VendorPassword>"xxxxxxxx"</VendorPassword>
</Auth>
</GetProperties>';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt ($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt ($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_POSTFIELDS, $xmlRequestString);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_VERBOSE, 0);
$data = curl_exec($ch);
echo $data;
curl_close($ch);

Here Is the example of php curl with post try this
<?php
$ch = curl_init();
// set your site url
curl_setopt($ch, CURLOPT_URL,"http://testmysite.com/");
curl_setopt($ch, CURLOPT_POST, 1);
// postvar1, postvar2 .. are the parameters to send with post
curl_setopt($ch, CURLOPT_POSTFIELDS,
"postvar1=value1&postvar2=value2&postvar3=value3");
// receive server response ...
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$server_output = curl_exec ($ch);
curl_close ($ch);
if ($server_output == "OK") { ... } else { ... }
?>

AS3 do post through PHP without changing doted parameters

I am trying to communicate with Google Forms using AS3 and PHP. The problem is that when I pass parameters such 'entry.283123610', PHP changes these into 'entry_283123610'. How can I prevent PHP from changing the post variables? This is the code I am currently using:
$ch = curl_init();
curl_setopt ($ch, CURLOPT_URL, $url);
curl_setopt($ch,CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($resource, CURLOPT_POST, 1);
curl_setopt($resource, CURLOPT_POSTFIELDS, $params);
curl_setopt ($ch, CURLOPT_HEADER, 0);
ob_start();
curl_exec ($ch);
curl_close ($ch);
$string = ob_get_contents();
ob_end_clean();

curl_exec printing results when I don't want to

I am using the following code:
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_TIMEOUT, 12);
$result = curl_exec($ch);
curl_close ($ch);
However it's printing the results straight away. Is it possible to put the JSON result into a variable so I can print it out when I want to?

Set CURLOPT_RETURNTRANSFER option:
// ...
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
$result = curl_exec($ch);
Per the docs:
CURLOPT_RETURNTRANSFER - TRUE to return the transfer as a string of the return value of curl_exec() instead of outputting it out directly.

Have you tried?
curl_setopt($ch, CURLOPT_VERBOSE, 0);
This worked for me!

after php 5.1 curl will display always result you can view in documentation. for avoid it simply use
echo "< span style='display:none'>";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_TIMEOUT, 12);
$result = curl_exec($ch);
curl_close ($ch);
echo"< /span>";

Categories