I was trying to download the results of a batch and was coding a program for this.
On an aspx file, I was able to write the following PHP code since the URL included the parameters:
function get_data($url) {
$ch = curl_init();
$timeout = 5;
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, $timeout);
$data = curl_exec($ch);
curl_close($ch);
return $data;
}
for ($i=1; $i<15000; $i++) {
$url = "http://example.com/result.aspx?ClassId=342&TermId=95&StudentId=".$i;
$returned_content = get_data($url);
if (stripos($returned_content,'Roll') !== false) {
echo "Student ID:" . $i;
echo $returned_content;
}
}
However, when a result is queried on a .ASP file, the URL simply says 'results.asp' without any additional parameters. Is there a way to use CURL requests to run a for loop and download this data in a similar manner?
Thanks for any help!
Related
// From URL to get redirected URL
$url = 'https://www.shareasale.com/m-pr.cfm?merchantID=83483&userID=1860618&productID=916465625';
$ch = curl_init(); // create cURL handle (ch)
if (!$ch) {
die("Couldn't initialize a cURL handle");
}
// set some cURL options
$ret = curl_setopt($ch, CURLOPT_URL, $url);
$ret = curl_setopt($ch, CURLOPT_HEADER, 1);
$ret = curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
$ret = curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$ret = curl_setopt($ch, CURLOPT_TIMEOUT, 30);
// execute
$ret = curl_exec($ch);
if (!empty($ret)) {
$info = curl_getinfo($ch);
curl_close($ch); // close cURL handler
if (empty($info['http_code'])) {
die("No HTTP code was returned");
} else {
echo 'REDIRECTED FINAL URL'.$info['url']); // this does not give final url.
}
}
is their any way we can get final url from a url after all re-directions ?
Let me know if any changes needs to be done in this code ?
https://www.shareasale.com/m-pr.cfm?merchantID=83483&userID=1860618&productID=916465625
This is the url which has lots of redirections, i am testing code with this one but it does not return final url, it return some the url then the url which you see in url bar.
The code you have is working correctly, but it is only part of what you want. When you get to the final URL redirect, your return includes...
<HTML><head></head><body>
<script LANGUAGE="JavaScript1.2">
window.location.replace('https:\/\/loomyhome.com\/collections\/all-products\/products\/blue-my-mind-rug?sscid=71k5_300lf&')
</script>
</body></html>
So you then need to extract the URL from there. You can use a regex (not my best skill) which would be something like...
preg_match('#(https:.*?)\'\)#', $ret, $match);
echo stripslashes($match[1]);
(using stripslashes to unescape the string). Gives...
https://loomyhome.com/collections/all-products/products/blue-my-mind-rug?sscid=71k5_3097f&
I am trying to get the latest commit from github using the api, but I encounter some errors and not sure what the problem is with the curl requests. The CURLINFO_HTTP_CODE gives me 000.
What does it mean if I got 000 and why is it not getting the contents of the url?
function get_json($url){
$base = "https://api.github.com";
$curl = curl_init();
curl_setopt($curl, CURLOPT_URL, $base . $url);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false);
//curl_setopt($curl, CONNECTTIMEOUT, 1);
$content = curl_exec($curl);
echo $http_status = curl_getinfo($curl, CURLINFO_HTTP_CODE);
curl_close($curl);
return $content;
}
echo get_json("users/$user/repos");
function get_latest_repo($user) {
// Get the json from github for the repos
$json = json_decode(get_json("users/$user/repos"),true);
print_r($json);
// Sort the array returend by pushed_at time
function compare_pushed_at($b, $a){
return strnatcmp($a['pushed_at'], $b['pushed_at']);
}
usort($json, 'compare_pushed_at');
//Now just get the latest repo
$json = $json[0];
return $json;
}
function get_commits($repo, $user){
// Get the name of the repo that we'll use in the request url
$repoName = $repo["name"];
return json_decode(get_json("repos/$user/$repoName/commits"),true);
}
I use your code and it will work if you add an user agent on curl
curl_setopt($ch, CURLOPT_USERAGENT,'YOUR_INVENTED_APP_NAME');
I have a url something like this
$url ="www.domain.com/image.php?id=123&idlocation=987&number=01";
Previously i was getting the extension using following code
$img_details= pathinfo($url);
But this won't work any more since the url has other variables also . So in this case how to get the Image name and extension .
I know i should first download the file using
$contenido = file_get_contents($url);
But don't know how to get the files name/extention from this
Thanks in advance
$ch = curl_init();
$url = 'http://www.domain.com/image.php?id=123&idlocation=987&number=01';
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_NOBODY, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$results = split("\n", trim(curl_exec($ch)));
foreach($results as $line) {
if (strtok($line, ':') == 'Content-Type') {
$parts = explode(":", $line);
echo trim($parts[1]);
}
}
Return: image/png
Already answered in: Get mime type of external file using cURL and php
The above answers focus both on the mime-type and they will work in most cases they both need additional resource usage - either a second network call to get the mime type or more disk reads/writes to save the file to disk and use the exif-imagetype function. And both will not return the file name which was a part of the question. Here is a download function using curl that will return a downloaded file as an array with name,mime type and content. Additionally it will try to get the name from the URL if possible.
Sample usage
$file=downloadfile($url);
echo "Name: ".$file["name"]."<br>";
echo "Type: ".$file["mimetype"]."<br>";
Code
function downloadfile($url){
global $headers;
$headers=array();
$file=array("content"=>"","mimetype"=>"","name"=>"");
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_HEADERFUNCTION, 'readHeaders');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$file["content"]=curl_exec($ch);
if(sizeof($headers)){
foreach($headers as $header){
if(!strncasecmp($header,'content-type:',13)){
$file["mimetype"]=trim(substr($header,13));
}
elseif(!strncasecmp($header,'content-disposition:',20)){
$file["name"]=trim(substr(strstr($header,'filename='),9));
}
}
}
if(!$file["name"]){
$query=strpos("?",$url);
$file["name"]=basename(substr($url,0,($query?$query:strlen($url))));
}
unset($headers);
return $file;
}
function readHeaders($ch, $header) {
global $headers;
array_push($headers, $header);
return strlen($header);
}
I made a PHP function that works with an API to show me the dollar balance of an account. This function is called: get_balance
function get_balance($account){
$url = 'http://mycompaniesurl.com/' . $account; // url of website
global $response;
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_TIMEOUT, 3);
$response = curl_exec($ch);
curl_close($ch);
}
The function get_balance returns the output in the variable $response
I'm certain that the function works, so I have no questions on that part. However, I'm trying to process accountnumbers written down in a CSV file. I call the CSV file with the following code:
$file = new SplFileObject("test.csv");
$file->setFlags(SplFileObject::READ_CSV);
$data = call_user_func_array('array_merge', iterator_to_array($file));
$data = array_combine(range(1, count($data)), $data);
extract($data, EXTR_PREFIX_ALL, 'variable');
I'm testing my code with a csv file called test.csv, containing 4 addresses (first one has a balance of 0, other 3 have a balance of >0).
With the following code I get the balance with the accountnumber printed to my screen:
get_balance($data[2]);
if ($response > 0){
echo $response,"---------------",$data[2];
}
Because $data[1] has a balance of 0, nothing is printed. $data[2],$data[3] and $data[4] have a balance of more than 0, so they do return the balance together with the accountnumber.
Now what my question is; is there a way to 'automatically' do this? Something like
get_balance($data[]);
seems to not work. The CSV that this php file has to process is about ~1000 accountnumber and may have more in the future, so typing a get_balance($data[1]) up to get_balance($data[999]) will be a time consuming business.
Is there a (simple) way to apply the function to ALL the $data[] ?
Yes, there is, called array_map:
$balanced_data = array_map("get_balance", $data);
This will leave $balanced_data empty, because at the moment get_balance has no return, but uses a global variable. Change it to something like this
function get_balance($account){
$url = 'http://mycompaniesurl.com/' . $account; // url of website
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_TIMEOUT, 3);
$response = curl_exec($ch);
curl_close($ch);
return $response;
}
If you don't want to or can't change get_balance, you will need a workaround like this
$max = count($data);
for ($i=0; $i<$max; $i++) {
$response = 0; // reset to make sure we get a new value
get_balance($data[$i]);
$balanced_data[$i] = $response;
}
I mine data from rss links and get a bunch of urls like:
http://feedproxy.google.com/~r/electricpig/~3/qoF8XbocUbE/
.... and if I access the links in my web browser, I am redirected to something like:
http://www.electricpig.co.uk/stuff.
Is there a way in php to write a function that, when given a url "a" that redirects the user to an url "b", returns you the url "b" ?
Here you go:
function getRedirect($oldUrl) {
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $oldUrl);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$res = curl_exec($ch);
$newUrl = curl_getinfo($ch, CURLINFO_EFFECTIVE_URL);
curl_close($ch);
return $newUrl;
}
The function requires cURL, and makes use of CURLINO_EFFECTIVE_URL. You can look it up on phpdoc here
EDIT:
if you are certain the oldUrl is not redirecting to newUrl via javascript, then you can also avoid fetching the body of the newUrl using
curl_setopt($ch, CURLOPT_NOBODY, TRUE); // remove body
Put the above line before $res = curl_exec($ch); in the function getRedirect to achiever faster execution.
public function getRedirect($url) {
$headers = get_headers($url, 1);
if (array_key_exists("Location", $headers)) {
$url = getRedirect($headers["Location"]);
}
return $url;
}