i'm trying to download gz file locally from githubarchive with httpclient in php.
When i execute a wget in terminal, the gz is extracted and each folders are downloaded on my computer.
When i do the same in php code, i encounter a 404 each time.
Bellow, my code :
//Symfony\Component\HttpClient\HttpClient;
$httpClient = HttpClient::create();
$response = $httpClient->request('GET', "https://data.gharchive.org/2015-01-01-{0..23}.json.gz");
if (200 !== $response->getStatusCode()) {
throw new \Exception('status code = ' . $response->getStatusCode());
}
when i call wget https://data.gharchive.org/2015-01-01-{0..23}.json.gz in console, every files in gz are downloaded on my computer.
Maybe can i use curl but i have already used it with no success.
{0..23} is a feature of bash called brace expansion. You'll need to recreate this functionality in PHP with something like
for ($i = 0; $i < 24; $i++) {
$response = $httpClient->request('GET', "https://data.gharchive.org/2015-01-01-{$i}.json.gz");
...
}
Related
and thanks in advance. I try to build a webscraper with PHP and I use Visual Studio Code.
When I run the following code, the following problem shows up:
Use of unknown class: 'Goutte\Client'
Does anyone know how to solve that issue?
I have googled all over the place, looked at SO and asked the forbidden one, but still after three days I have not achieved any progress. (I am also a noob, so maybe it is not as difficult to solve as I think).
Looking forward to your feedback and tips.
<?php
require 'vendor/autoload.php';
use Goutte\Client;
// Initialize the Goutte client
$client = new Client();
// Create a new array to store the scraped data
$data = array();
// Loop through the pages
for ($i = 0; $i < 3; $i++) {
// Make a request to the website
$crawler = $client->request('GET', 'https://ec.europa.eu/info/law/better-regulation/have-your-say/initiatives_de?page=' . $i);
// Find all the initiatives on the page
$crawler->filter('.initiative')->each(function ($node) use (&$data) {
// Extract the information for each initiative
$title = $node->filter('h3')->text();
$link = $node->filter('a')->attr('href');
$description = $node->filter('p')->text();
$deadline = $node->filter('time')->attr('datetime');
// Append the data for the initiative to the data array
$data[] = array($title, $link, $description, $deadline);
});
// Sleep for a random amount of time between 5 and 10 seconds
$sleep = rand(5,10);
sleep($sleep);
}
// Open the output file
$fp = fopen('initiatives.csv', 'w');
// Write the header row
fputcsv($fp, array('Title', 'Link', 'Description', 'Deadline'));
I'm generating a large PDF with 2000 pages in symfony (4.2) framework. What I'm doing is just save the HTML content to the .HTML file by getting content from the twig.
Then I'm using the headless chrome to generate the PDF from the URL using the below command.
/usr/bin/google-chrome --headless --disable-gpu --run-all-compositor-stages-before-draw --print-to-pdf [URL of HTML file] --virtual-time-budget=10000
Now, the requirement is while the above command is running I have to display the loader with the progress bar in the front.
What I did is as below to get the stream response and display them on the browser.
Controller
public function streamAction()
{
$process = new Process(["pwd"]);
$process->run();
$output = new StreamedOutputService(fopen('php://stdout', 'w'));
$response = new StreamedResponse(function() use ($output, $process) {
// $process->isRunning() always returns false.
while ($process->isRunning()) {
$output->writeln($process->getOutput());
}
});
$response->headers->set('X-Accel-Buffering', 'no');
return $response;
}
Streamed Response Class
protected function doWrite($message, $newline)
{
if (
false === #fwrite($this->getStream(), $message) ||
(
$newline &&
(false === #fwrite($this->getStream(), PHP_EOL))
)
) {
throw new RuntimeException('Unable to write output.');
}
echo $message;
ob_flush();
flush();
}
What is the buggy on the above code? I'm not able to get the output of the command hence can not write it to the browser.
Below code is working fine and sending response at every 2 seconds on the browser
public function streamAction()
{
$output = new StreamedOutputService(fopen('php://stdout', 'w'));
$response = new StreamedResponse(function() use ($output) {
for($i = 0; $i <= 5; $i++) {
$output->writeln($i);
sleep(2);
}
});
$response->headers->set('X-Accel-Buffering', 'no');
return $response;
}
I have a download server for files.
And I want my users to get the files from the download server.
My download server is Linux.
I want when the user clicks on the download button.
Get the file directly from the download server.
I do not want to use the stream to download ...
I want to connect to the download server via the link and then download it using PHP
My site is with mvc
Thank you, step by step to help me
thank you
Stream stream = null;
//This controls how many bytes to read at a time and send to the client
int bytesToRead = 10000;
// Buffer to read bytes in chunk size specified above
byte[] buffer = new Byte[bytesToRead];
// The number of bytes read
try
{
//Create a WebRequest to get the file
HttpWebRequest fileReq = (HttpWebRequest)HttpWebRequest.Create(Global.UrlVideoPrice + IdCourse + "//" + IdTopic+".rar");
//Create a response for this request
HttpWebResponse fileResp = (HttpWebResponse)fileReq.GetResponse();
if (fileReq.ContentLength > 0)
fileResp.ContentLength = fileReq.ContentLength;
//Get the Stream returned from the response
stream = fileResp.GetResponseStream();
// prepare the response to the client. resp is the client Response
var resp = System.Web.HttpContext.Current.Response;
//Indicate the type of data being sent
resp.ContentType = "application/octet-stream";
//Name the file
resp.AddHeader("Content-Disposition", "attachment; filename=\"" + Topic.fldName + ".rar\"");
resp.AddHeader("Content-Length", fileResp.ContentLength.ToString());
int length;
do
{
// Verify that the client is connected.
if (resp.IsClientConnected)
{
// Read data into the buffer.
length = stream.Read(buffer, 0, bytesToRead);
// and write it out to the response's output stream
resp.OutputStream.Write(buffer, 0, length);
// Flush the data
resp.Flush();
//Clear the buffer
buffer = new Byte[bytesToRead];
}
else
{
// cancel the download if client has disconnected
length = -1;
}
} while (length > 0); //Repeat until no data is read
}
finally
{
if (stream != null)
{
//Close the input stream
stream.Close();
}
}
This is my download code.
Now I want the user to get the files directly from the download server.
Do not use site traffic to download the file.
And use server download traffic
im trying to download a remote xml file, but is not working, is not storing the file on my storage.
my code:
$url = 'http://xml.url.xml';
set_time_limit(0);
// Download file and save it on folder
$guzzleClient = new Client();
$response = $guzzleClient->get($url);
$body = $response->getBody();
$body->seek(0);
$size = $body->getSize();
$file = $body->read($size);
Storage::download($file);
The Storage::download() method is used to generate response, that will force the download in the browser.
Use Storage::put('filename.xml', $content) instead.
You can read more in the docs:
https://laravel.com/docs/5.6/filesystem
I'm working with android and try to create an app that able to upload several image to the server. I had tried to upload the image to my localhost using xampp, it works well. But when I try to upload to my enterprise server I can't find my file, in the other word. The file can't be written. I don't know what make it failed? This is my code
Upload tp XAMPP
Connection string private static final String url_photo = "http://192.168.7.110/blabla/base.php";
Path static final String path = "C:\\xampp\\htdocs\\psn_asset_oracle\\Images\\";
Upload to actual enterprise server
Connection String private static final String url_photo = "http://192.168.4.27/oracle/logam/am/data_images/android_image/base.php";
Path static final String path = "http://192.168.4.27/oracle/logam/am/data_images/android_image/";
My code to upload to server
params_p.add(new BasicNameValuePair("image_name_1",
image_name_1));
params_p.add(new BasicNameValuePair("image_name_2",
image_name_2));
params_p.add(new BasicNameValuePair("image_name_3",
image_name_3));
params_p.add(new BasicNameValuePair("image_name_4",
image_name_4));
json_photo = jsonParser.makeHttpRequest(url_photo, "POST", params_p);
ArrayList<NameValuePair> params_p = new ArrayList<NameValuePair>();
PHP code
if(isset($_POST["image_name_1"]) && isset($_POST["image_name_2"]) && isset($_POST["image_name_3"]) && isset($_POST["image_name_4"])
&& isset($_POST["image_1"]) && isset($_POST["image_2"]) && isset($_POST["image_3"]) && isset($_POST["image_4"]))
{
$image_name_1 = $_POST["image_name_1"];
$image_name_2 = $_POST["image_name_2"];
$image_name_3 = $_POST["image_name_3"];
$image_name_4 = $_POST["image_name_4"];
$image_1 = $_POST["image_1"];
$image_2 = $_POST["image_2"];
$image_3 = $_POST["image_3"];
$image_4 = $_POST["image_4"];
/*---------base64 decoding utf-8 string-----------*/
$binary_1=base64_decode($image_1);
$binary_2=base64_decode($image_2);
$binary_3=base64_decode($image_3);
$binary_4=base64_decode($image_4);
/*-----------set binary, utf-8 bytes----------*/
header('Content-Type: bitmap; charset=utf-8');
/*---------------open specified directory and put image on it------------------*/
$file_1 = fopen($image_name_1, 'wb');
$file_2 = fopen($image_name_2, 'wb');
$file_3 = fopen($image_name_3, 'wb');
$file_4 = fopen($image_name_4, 'wb');
/*---------------------assign image to file system-----------------------------*/
fwrite($file_1, $binary_1);
fclose($file_1);
fwrite($file_2, $binary_2);
fclose($file_2);
fwrite($file_3, $binary_3);
fclose($file_3);
fwrite($file_4, $binary_4);
fclose($file_4);
$response["message"] = "Success";
echo json_encode($response);
}
I've contact my DBA and asked to give me permission to write the file, and it still doesn't work. The error is json doesn't give "Success" as message that indicate the file failed to be written. I will appreciate any help. Thank you.
Does the file server allows write access permission? Sample of chmod, http://catcode.com/teachmod/
Is your enterprise server online? The IP address looks private to me, 192.168.4.27.
Dont use json use http post.
See below code
HttpClient client = new DefaultHttpClient();
String postURL = "your url";
HttpPost post = new HttpPost(postURL);
try {
MultipartEntity reqEntity = new MultipartEntity(HttpMultipartMode.BROWSER_COMPATIBLE);
ByteArrayBody bab = new ByteArrayBody(img, "image.jpg");
reqEntity.addPart("image", bab);
post.setEntity(reqEntity);
HttpResponse response = client.execute(post);
Here img is your image in ByteArray format
The problem solved after my DBA give me another file path.