Is it possible to pull track info from an audio stream using PHP? I've done some digging and the closest function I can find is stream_get_transports but my host doesn't support http transports via fsockopen() so I'll have to do some more tinkering to see what else that function returns.
Currently, I'm trying to pull artist and track metadata from an AOL stream.
This is a SHOUTcast stream, and yes it is possible. It has absolutely nothing to do with ID3 tags. I wrote a script awhile ago to do this, but can't find it anymore. Just last week I helped another guy who had a fairly complete script to do the same thing, but I can't just post the source to it, as it isn't mine. I will however get you in touch with him, if you e-mail me at brad#musatcha.com.
Anyway, here's how to do it yourself:
The first thing you need to do is connect to the server directly. Don't use HTTP. Well, you could probably use cURL, but it will likely be much more hassle than its worth. You connect to it with fsockopen() (doc). Make sure to use the correct port. Also note that many web hosts will block a lot of ports, but you can usually use port 80. Fortunately, all of the AOL-hosted SHOUTcast streams use port 80.
Now, make your request just like your client would.
GET /whatever HTTP/1.0
But, before sending <CrLf><CrLf>, include this next header!
Icy-MetaData:1
That tells the server that you want metadata. Now, send your pair of <CrLf>.
Ok, the server will respond with a bunch of headers and then start sending you data. In those headers will be an icy-metaint:8192 or similar. That 8192 is the meta interval. This is important, and really the only value you need. It is usually 8192, but not always, so make sure to actually read this value!
Basically it means, you will get 8192 bytes of MP3 data and then a chunk of meta, followed by 8192 bytes of MP3 data, followed by a chunk of meta.
Read 8192 bytes of data (make sure you are not including the header in this count), discard them, and then read the next byte. This byte is the first byte of meta data, and indicates how long the meta data is. Take the value of this byte (the actual byte with ord() (doc)), and multiply it by 16. The result is the number of bytes to read for metadata. Read those number of bytes into a string variable for you to work with.
Next, trim the value of this variable. Why? Because the string is padded with 0x0 at the end (to make it fit evenly into a multiple of 16 bytes), and trim() (doc) takes care of that for us.
You will be left with something like this:
StreamTitle='Awesome Trance Mix - DI.fm';StreamUrl=''
I'll let you pick your method of choice for parsing this. Personally I'd probably just split with a limit of 2 on ;, but beware of titles that contain ;. I'm not sure what the escape character method is. A little experimentation should help you.
Don't forget to disconnect from the server when you're done with it!
There are lots of SHOUTcast MetaData references out there. This is a good one: http://www.smackfu.com/stuff/programming/shoutcast.html
Check this out: https://gist.github.com/fracasula/5781710
It's a little gist with a PHP function that lets you extract MP3 metadata (StreamTitle) from a streaming URL.
Usually the streaming server puts an icy-metaint header in the response which tells us how often the metadata is sent in the stream. The function checks for that response header and, if present, it replaces the interval parameter with it.
Otherwise the function calls the streaming URL respecting your interval and, if any metadata isn't present, then it tries again through recursion starting from the offset parameter.
<?php
/**
* Please be aware. This gist requires at least PHP 5.4 to run correctly.
* Otherwise consider downgrading the $opts array code to the classic "array" syntax.
*/
function getMp3StreamTitle($streamingUrl, $interval, $offset = 0, $headers = true)
{
$needle = 'StreamTitle=';
$ua = 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.110 Safari/537.36';
$opts = [
'http' => [
'method' => 'GET',
'header' => 'Icy-MetaData: 1',
'user_agent' => $ua
]
];
if (($headers = get_headers($streamingUrl))) {
foreach ($headers as $h) {
if (strpos(strtolower($h), 'icy-metaint') !== false && ($interval = explode(':', $h)[1])) {
break;
}
}
}
$context = stream_context_create($opts);
if ($stream = fopen($streamingUrl, 'r', false, $context)) {
$buffer = stream_get_contents($stream, $interval, $offset);
fclose($stream);
if (strpos($buffer, $needle) !== false) {
$title = explode($needle, $buffer)[1];
return substr($title, 1, strpos($title, ';') - 2);
} else {
return getMp3StreamTitle($streamingUrl, $interval, $offset + $interval, false);
}
} else {
throw new Exception("Unable to open stream [{$streamingUrl}]");
}
}
var_dump(getMp3StreamTitle('http://str30.creacast.com/r101_thema6', 19200));
I hope this helps!
Thanks a lot for the code fra_casula. Here is a slightly simplified version running on PHP <= 5.3 (the original is targeted at 5.4). It also reuses the same connection resource.
I removed the exception because of my own needs, returning false if nothing is found instead.
private function getMp3StreamTitle($steam_url)
{
$result = false;
$icy_metaint = -1;
$needle = 'StreamTitle=';
$ua = 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.110 Safari/537.36';
$opts = array(
'http' => array(
'method' => 'GET',
'header' => 'Icy-MetaData: 1',
'user_agent' => $ua
)
);
$default = stream_context_set_default($opts);
$stream = fopen($steam_url, 'r');
if($stream && ($meta_data = stream_get_meta_data($stream)) && isset($meta_data['wrapper_data'])){
foreach ($meta_data['wrapper_data'] as $header){
if (strpos(strtolower($header), 'icy-metaint') !== false){
$tmp = explode(":", $header);
$icy_metaint = trim($tmp[1]);
break;
}
}
}
if($icy_metaint != -1)
{
$buffer = stream_get_contents($stream, 300, $icy_metaint);
if(strpos($buffer, $needle) !== false)
{
$title = explode($needle, $buffer);
$title = trim($title[1]);
$result = substr($title, 1, strpos($title, ';') - 2);
}
}
if($stream)
fclose($stream);
return $result;
}
This is the C# code for getting the metadata using HttpClient:
public async Task<string> GetMetaDataFromIceCastStream(string url)
{
m_httpClient.DefaultRequestHeaders.Add("Icy-MetaData", "1");
var response = await m_httpClient.GetAsync(url, HttpCompletionOption.ResponseHeadersRead);
m_httpClient.DefaultRequestHeaders.Remove("Icy-MetaData");
if (response.IsSuccessStatusCode)
{
IEnumerable<string> headerValues;
if (response.Headers.TryGetValues("icy-metaint", out headerValues))
{
string metaIntString = headerValues.First();
if (!string.IsNullOrEmpty(metaIntString))
{
int metadataInterval = int.Parse(metaIntString);
byte[] buffer = new byte[metadataInterval];
using (var stream = await response.Content.ReadAsStreamAsync())
{
int numBytesRead = 0;
int numBytesToRead = metadataInterval;
do
{
int n = stream.Read(buffer, numBytesRead, 10);
numBytesRead += n;
numBytesToRead -= n;
} while (numBytesToRead > 0);
int lengthOfMetaData = stream.ReadByte();
int metaBytesToRead = lengthOfMetaData * 16;
byte[] metadataBytes = new byte[metaBytesToRead];
var bytesRead = await stream.ReadAsync(metadataBytes, 0, metaBytesToRead);
var metaDataString = System.Text.Encoding.UTF8.GetString(metadataBytes);
return metaDataString;
}
}
}
}
return null;
}
UPDATE:
This is an update with a more appropriate solution to the question. The original post is also provided below for information.
The script in this post, after some error correction, works and extracts the stream title using PHP:
PHP script to extract artist & title from Shoutcast/Icecast stream.
I had to make a couple of changes, because the echo statements at the end were throwing an error. I added two print_r() statements after the function, and $argv[1] in the call so you can pass the URL to it from the command line.
<?php
define('CRLF', "\r\n");
class streaminfo{
public $valid = false;
public $useragent = 'Winamp 2.81';
protected $headers = array();
protected $metadata = array();
public function __construct($location){
$errno = $errstr = '';
$t = parse_url($location);
$sock = fsockopen($t['host'], $t['port'], $errno, $errstr, 5);
$path = isset($t['path'])?$t['path']:'/';
if ($sock){
$request = 'GET '.$path.' HTTP/1.0' . CRLF .
'Host: ' . $t['host'] . CRLF .
'Connection: Close' . CRLF .
'User-Agent: ' . $this->useragent . CRLF .
'Accept: */*' . CRLF .
'icy-metadata: 1'.CRLF.
'icy-prebuffer: 65536'.CRLF.
(isset($t['user'])?'Authorization: Basic '.base64_encode($t['user'].':'.$t['pass']).CRLF:'').
'X-TipOfTheDay: Winamp "Classic" rulez all of them.' . CRLF . CRLF;
if (fwrite($sock, $request)){
$theaders = $line = '';
while (!feof($sock)){
$line = fgets($sock, 4096);
if('' == trim($line)){
break;
}
$theaders .= $line;
}
$theaders = explode(CRLF, $theaders);
foreach ($theaders as $header){
$t = explode(':', $header);
if (isset($t[0]) && trim($t[0]) != ''){
$name = preg_replace('/[^a-z][^a-z0-9]*/i','', strtolower(trim($t[0])));
array_shift($t);
$value = trim(implode(':', $t));
if ($value != ''){
if (is_numeric($value)){
$this->headers[$name] = (int)$value;
}else{
$this->headers[$name] = $value;
}
}
}
}
if (!isset($this->headers['icymetaint'])){
$data = ''; $metainterval = 512;
while(!feof($sock)){
$data .= fgetc($sock);
if (strlen($data) >= $metainterval) break;
}
$this->print_data($data);
$matches = array();
preg_match_all('/([\x00-\xff]{2})\x0\x0([a-z]+)=/i', $data, $matches, PREG_OFFSET_CAPTURE);
preg_match_all('/([a-z]+)=([a-z0-9\(\)\[\]., ]+)/i', $data, $matches, PREG_SPLIT_NO_EMPTY);
echo '<pre>';var_dump($matches);echo '</pre>';
$title = $artist = '';
foreach ($matches[0] as $nr => $values){
$offset = $values[1];
$length = ord($values[0]{0}) +
(ord($values[0]{1}) * 256)+
(ord($values[0]{2}) * 256*256)+
(ord($values[0]{3}) * 256*256*256);
$info = substr($data, $offset + 4, $length);
$seperator = strpos($info, '=');
$this->metadata[substr($info, 0, $seperator)] = substr($info, $seperator + 1);
if (substr($info, 0, $seperator) == 'title') $title = substr($info, $seperator + 1);
if (substr($info, 0, $seperator) == 'artist') $artist = substr($info, $seperator + 1);
}
$this->metadata['streamtitle'] = $artist . ' - ' . $title;
}else{
$metainterval = $this->headers['icymetaint'];
$intervals = 0;
$metadata = '';
while(1){
$data = '';
while(!feof($sock)){
$data .= fgetc($sock);
if (strlen($data) >= $metainterval) break;
}
//$this->print_data($data);
$len = join(unpack('c', fgetc($sock))) * 16;
if ($len > 0){
$metadata = str_replace("\0", '', fread($sock, $len));
break;
}else{
$intervals++;
if ($intervals > 100) break;
}
}
$metarr = explode(';', $metadata);
foreach ($metarr as $meta){
$t = explode('=', $meta);
if (isset($t[0]) && trim($t[0]) != ''){
$name = preg_replace('/[^a-z][^a-z0-9]*/i','', strtolower(trim($t[0])));
array_shift($t);
$value = trim(implode('=', $t));
if (substr($value, 0, 1) == '"' || substr($value, 0, 1) == "'"){
$value = substr($value, 1);
}
if (substr($value, -1) == '"' || substr($value, -1) == "'"){
$value = substr($value, 0, -1);
}
if ($value != ''){
$this->metadata[$name] = $value;
}
}
}
}
fclose($sock);
$this->valid = true;
}else echo 'unable to write.';
}else echo 'no socket '.$errno.' - '.$errstr.'.';
print_r($theaders);
print_r($metadata);
}
public function print_data($data){
$data = str_split($data);
$c = 0;
$string = '';
echo "<pre>\n000000 ";
foreach ($data as $char){
$string .= addcslashes($char, "\n\r\0\t");
$hex = dechex(join(unpack('C', $char)));
if ($c % 4 == 0) echo ' ';
if ($c % (4*4) == 0 && $c != 0){
foreach (str_split($string) as $s){
//echo " $string\n";
if (ord($s) < 32 || ord($s) > 126){
echo '\\'.ord($s);
}else{
echo $s;
}
}
echo "\n";
$string = '';
echo str_pad($c, 6, '0', STR_PAD_LEFT).' ';
}
if (strlen($hex) < 1) $hex = '00';
if (strlen($hex) < 2) $hex = '0'.$hex;
echo $hex.' ';
$c++;
}
echo " $string\n</pre>";
}
public function __get($name){
if (isset($this->metadata[$name])){
return $this->metadata[$name];
}
if (isset($this->headers[$name])){
return $this->headers[$name];
}
return null;
}
}
$t = new streaminfo($argv[1]); // get metadata
/*
echo "Meta Interval: ".$t->icymetaint;
echo "\n";
echo 'Current Track: '.$t->streamtitle;
*/
?>
With the updated code, it prints the arrays of header and streamtitle info. If you only want the now_playing track, then comment out the two print_r() statements, and uncomment the echo statements at the end.
#Example: run this command:
php getstreamtitle.php http://162.244.80.118:3066
#and the result is...
Array
(
[0] => HTTP/1.0 200 OK
[1] => icy-notice1:<BR>This stream requires Winamp<BR>
[2] => icy-notice2:SHOUTcast DNAS/posix(linux x64) v2.6.0.750<BR>
[3] => Accept-Ranges:none
[4] => Access-Control-Allow-Origin:*
[5] => Cache-Control:no-cache,no-store,must-revalidate,max-age=0
[6] => Connection:close
[7] => icy-name:
[8] => icy-genre:Old Time Radio
[9] => icy-br:24
[10] => icy-sr:22050
[11] => icy-url:http://horror-theatre.com
[12] => icy-pub:1
[13] => content-type:audio/mpeg
[14] => icy-metaint:8192
[15] => X-Clacks-Overhead:GNU Terry Pratchett
[16] =>
)
StreamTitle='501026TooHotToLive';
Here is the original post using python and vlc
The PHP solution kept searching but never returned a response for me.
This is not PHP as requested, but may help others looking for a way to extract the 'now_playing' info from live streams.
If you only want the 'now_playing' info, you can edit the script to return that.
The python script extracts the metadata (including the 'now_playing' track) using VLC. You need VLC and the python libraries: sys, telnetlib, os, time and socket.
#!/usr/bin/python
# coding: utf-8
import sys, telnetlib, os, time, socket
HOST = "localhost"
password = "admin"
port = "4212"
def check_port():
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
res = sock.connect_ex((HOST, int(port)))
sock.close()
return res == 0
def checkstat():
if not check_port():
os.popen('vlc --no-audio --intf telnet --telnet-password admin --quiet 2>/dev/null &')
while not check_port():
time.sleep(.1)
def docmd(cmd):
tn = telnetlib.Telnet(HOST, port)
tn.read_until(b"Password: ")
tn.write(password.encode('utf-8') + b"\n")
tn.read_until(b"> ")
tn.write(cmd.encode('utf-8') + b"\n")
ans=tn.read_until(">".encode("utf-8"))[0:-3]
return(ans)
tn.close()
def nowplaying(playing):
npstart=playing.find('now_playing')
mystr=playing[npstart:]
npend=mystr.find('\n')
return mystr[:npend]
def metadata(playing):
fstr='+----'
mstart=playing.find(fstr)
mend=playing.find(fstr,mstart+len(fstr))
return playing[mstart:mend+len(fstr)]
checkstat()
docmd('add '+sys.argv[1])
playing=""
count=0
while not 'now_playing:' in playing:
time.sleep(.5)
playing=docmd('info')
count+=1
if count>9:
break
if playing == "":
print("--Timeout--")
else:
print(metadata(playing))
docmd('shutdown')
Example, extract metadata from Crypt Theater Station:
./radiometatdata.py http://107.181.227.250:8026
Response:
+----[ Meta data ]
|
| title: *CRYPT THEATER*
| filename: 107.181.227.250:8026
| genre: Old Time Radio
| now_playing: CBS Radio Mystery Theatre - A Ghostly Game of Death
|
+----
Related
Before everything , I know that my question is already posted thousand times but none of them concern me since i cant identify my problem.
So ,I am building a PHP rest api to get a put request from the client (angular), this last can update a file inside the form as well as some text inputs , all of this is sent as a form-data in this code :
var fileFormData = new FormData();
$http({
method: 'PUT',
url: host + '/produit/' + $scope.product.id,
data: fileFormData,
transformRequest: angular.identity,
headers: {
'Content-Type': undefined
}
}).then(function successCallback(response) {}, function errorCallback(response) {});
On the server-side I have to get the response type and then parse the data here's the full code with help from here
$method = $_SERVER['REQUEST_METHOD'];
if ($method=="PUT"){
global $_PUT;
/* PUT data comes in on the stdin stream */
$putdata = fopen("php://input", "r");
/* Open a file for writing */
// $fp = fopen("myputfile.ext", "w");
$raw_data = '';
/* Read the data 1 KB at a time
and write to the file */
while ($chunk = fread($putdata, 1024))
$raw_data .= $chunk;
/* Close the streams */
fclose($putdata);
// Fetch content and determine boundary
$boundary = substr($raw_data, 0, strpos($raw_data, "\r\n"));
if(empty($boundary)){
parse_str($raw_data,$data);
$GLOBALS[ '_PUT' ] = $data;
return;
}
// Fetch each part
$parts = array_slice(explode($boundary, $raw_data), 1);
$data = array();
foreach ($parts as $part) {
// If this is the last part, break
if ($part == "--\r\n") break;
// Separate content from headers
$part = ltrim($part, "\r\n");
list($raw_headers, $body) = explode("\r\n\r\n", $part, 2);
// Parse the headers list
$raw_headers = explode("\r\n", $raw_headers);
$headers = array();
foreach ($raw_headers as $header) {
list($name, $value) = explode(':', $header);
$headers[strtolower($name)] = ltrim($value, ' ');
}
// Parse the Content-Disposition to get the field name, etc.
if (isset($headers['content-disposition'])) {
$filename = null;
$tmp_name = null;
preg_match(
'/^(.+); *name="([^"]+)"(; *filename="([^"]+)")?/',
$headers['content-disposition'],
$matches
);
list(, $type, $name) = $matches;
//Parse File
if( isset($matches[4]) )
{
//if labeled the same as previous, skip
if( isset( $_FILES[ $matches[ 2 ] ] ) )
{
continue;
}
//get filename
$filename = $matches[4];
//get tmp name
$filename_parts = pathinfo( $filename );
$tmp_name = tempnam( ini_get('upload_tmp_dir'), $filename_parts['filename']);
//populate $_FILES with information, size may be off in multibyte situation
$_FILES[ $matches[ 2 ] ] = array(
'error'=>0,
'name'=>$filename,
'tmp_name'=>$tmp_name,
'size'=>strlen( $body ),
'type'=>$value
);
//place in temporary directory
file_put_contents($tmp_name, $body);
}
//Parse Field
else
{
$data[$name] = substr($body, 0, strlen($body) - 2);
}
}
}
$GLOBALS[ '_PUT' ] = $data;
$input= array_merge($_PUT, $_FILES);
}
and then I have to execute the query and move the file form the temp folder
$sql = "update `$table` set $set where id=$key"; //works
if(isset($name)){ //to check if file exist
chmod($tmpname,0666); //since that's the answer for most of question that are similar to mine
echo substr(sprintf('%o', fileperms($tmpname)), -4); //returns 0666
echo $tmpname." ".$name; //works too
if(move_uploaded_file($tmpname, "/home/****/****/uploads/produits/".$name))
echo "moved";
else
echo "not moved"; // and I always get this
Can you explain to me please whats wrong ?
PS:
I checked the /tmp folder the files I uploaded are still there.
The log is clean and I used this in the top of my code
ini_set('display_errors', true);
error_reporting(E_ALL);
Update :
I use this to get file params from the $input var
$values = array_map(function ($value) use ($link) {
global $method;
if ($value === null)
return null;
if (gettype($value) === "array"){
if ($method=="PUT" || $method=="POST"){
global $tmpname,$name;
$tmpname=$value['tmp_name'];
$name=uniqid().$value['name'];
$value=$name;
}
}
return mysqli_real_escape_string($link, (string) $value);
}, array_values($input));
I have been using the Yahoo Financial API to download historical stock data from Yahoo. As has been reported on this site, as of mid May, the old API was discontinued. There have been many posts addressed the the form of the new call, e.g.:
https://query1.finance.yahoo.com/v7/finance/download/AAPL?period1=315561600&period2=1496087439&interval=1d&events=history&crumb=XXXXXXXXXXX
As well as methods for obtaining the crumb:
Yahoo Finance URL not working
But I must be misunderstanding what the procedure is as I always get an error saying that it "Failed to open stream: HTTP request failed. HTTP/1.0 201 Unauthorized".
Below is my code. Any and all assistance is welcome. I have to admit that I am an old Fortran programmer and my coding reflects this.
Good Roads
Bill
$ticker = "AAPL";
$yahooURL="https://finance.yahoo.com/quote/" .$ticker ."/history";
$body=file_get_contents($yahooURL);
$headers=$http_response_header;
$icount = count($headers);
for($i = 0; $i < $icount; $i ++)
{
$istart = -1;
$istop = -1;
$istart = strpos($headers[$i], "Set-Cookie: B=");
$istop = strpos($headers[$i], "&b=");
if($istart > -1 && $istop > -1)
{
$Cookie = substr ( $headers[$i] ,$istart+14,$istop - ($istart + 14));
}
}
$istart = strpos($body,"CrumbStore") + 22;
$istop = strpos($body,'"', $istart);
$Crumb = substr ( $body ,$istart,$istop - $istart);
$iMonth = 1;
$iDay = 1;
$iYear = 1980;
$timestampStart = mktime(0,0,0,$iMonth,$iDay,$iYear);
$timestampEnd = time();
$url = "https://query1.finance.yahoo.com/v7/finance/download/".$ticker."?period1=".$timestampStart."&period2=".$timestampEnd."&interval=1d&events=history&crumb=".$Cookie."";
while (!copy($url, $newfile) && $iLoop < 10)
{
if($iLoop == 9) echo "Failed to download data." .$lf;
$iLoop = $iLoop + 1;
sleep(1);
}
#Craig Cocca this isn't exactly a duplicate because the reference you gave gives a solution in python which for those of us who use php but haven't learnt python doesn't help much. I'd love to see as solution with php. I've examinied the yahoo page and am able to extract the crumb but can't work out how to put it into a stream and GET call.
My latest (failed) effort is:
$headers = [
"Accept" => "*/*",
"Connection" => "Keep-Alive",
"User-Agent" => sprintf("curl/%s", curl_version()["version"])
];
// open connection to Yahoo
$context = stream_context_create([
"http" => [
"header" => (implode(array_map(function($value, $key) { return sprintf("%s: %s\r\n", $key, $value); }, $headers, array_keys($headers))))."Cookie: $Cookie",
"method" => "GET"
]
]);
$handle = #fopen("https://query1.finance.yahoo.com/v7/finance/download/{$symbol}?period1={$date_now}&period2={$date_now}&interval=1d&events=history&crumb={$Crumb}", "r", false, $context);
if ($handle === false)
{
// trigger (big, orange) error
trigger_error("Could not connect to Yahoo!", E_USER_ERROR);
exit;
}
// download first line of CSV file
$data = fgetcsv($handle);
The two dates are unix coded dates i.e.: $date_now = strtotime($date);
I've now managed to download share price history. At the moment I'm only taking the current price figures but my download method receives historical data for the past year. (i.e. until Yahoo decides to put some other block on the data).
My solution uses the "simple_html_dom.php" parser which I've added to my /includes folder.
Here is the code (modified from the original version from the Harvard CS50 course which I recommend for beginners like me):
function lookup($symbol)
{
// reject symbols that start with ^
if (preg_match("/^\^/", $symbol))
{
return false;
}
// reject symbols that contain commas
if (preg_match("/,/", $symbol))
{
return false;
}
// body of price history search
$sym = $symbol;
$yahooURL='https://finance.yahoo.com/quote/'.$sym.'/history?p='.$sym;
// get stock name
$data = file_get_contents($yahooURL);
$title = preg_match('/<title[^>]*>(.*?)<\/title>/ims', $data, $matches) ? $matches[1] : null;
$title = preg_replace('/[[a-zA-Z0-9\. \| ]* \| /','',$title);
$title = preg_replace('/ Stock \- Yahoo Finance/','',$title);
$name = $title;
// get price data - use simple_html_dom.php (added to /include)
$body=file_get_html($yahooURL);
$tables = $body->find('table');
$dom = new DOMDocument();
$elements[] = null;
$dom->loadHtml($tables[1]);
$x = new DOMXpath($dom);
$i = 0;
foreach($x->query('//td') as $td){
$elements[$i] = $td -> textContent." ";
$i++;
}
$open = floatval($elements[1]);
$high = floatval($elements[2]);
$low = floatval($elements[3]);
$close = floatval($elements[5]);
$vol = str_replace( ',', '', $elements[6]);
$vol = floatval($vol);
$date = date('Y-m-d');
$datestamp = strtotime($date);
$date = date('Y-m-d',$datestamp);
// return stock as an associative array
return [
"symbol" => $symbol,
"name" => $name,
"price" => $close,
"open" => $open,
"high" => $high,
"low" => $low,
"vol" => $vol,
"date" => $date
];
}
Very similar questions have been posted and answered. However every solution I found so far is using php Memcached extension rather than Memcache. I am trying to delete/expire all memcache items that begin with a predefined key. For example, so_something, so_something_else .... Any help is much appreciated.
After some research I figured how to do it using standard Memcache commands. But, unfortunately this will man trips to the memcache server so it is to be used with care; in my case it is used only after deployment:
/**
* Helper function to expire memcache entries with a specific key prefix.
*
* #param string $key_prefix
* The memcache key prefix.
* #param array $servers
* Array of used memcache servers.
* #return array
*
*/
function memcache_delete_by_key($key_prefix = '', $servers = array()) {
$mc = new Memcache();
$delete_keys = array();
foreach ($servers as $server => $default) {
list($host, $port) = explode(':', $server);
$mc->addServer($host, $port);
// Extract all the keys from Memcache.
$stats = memcache_command($server, $port, "stats items");
$stat_lines = explode("\r\n", $stats);
$slabs = array();
$keys = array();
foreach ($stat_lines as $line) {
if (preg_match("/STAT items:([\d]+):number ([\d]+)/", $line, $matches)) {
if (isset($matches[1]) && !in_array($matches[1], $slabs)) {
$slabs[] = $matches[1];
$string = memcache_command($server, $port, "stats cachedump " . $matches[1] . " " . $matches[2]);
preg_match_all("/ITEM (.*?) /", $string, $matches);
$keys[] = $matches[1];
}
}
}
$delete_keys = call_user_func_array('array_merge', $keys);
// Locate all keys that begin with our prefix.
foreach ($delete_keys as $index => $key) {
// If strpos returns 0 (not false) then we have a match.
if (strpos($key, $key_prefix) === 0) {
$mc->delete($key);
}
}
}
return $delete_keys;
}
/*
* Helper function to send memcache commands to a given Memcache Server
*/
function memcache_command($server, $port, $command) {
$s = #fsockopen($server, $port);
if (!$s) {
return die("Cant connect to:" . $server . ":" . $port);
}
fwrite($s, $command . "\r\n");
$buf = "";
while ((!feof($s))) {
$buf .= fgets($s, 256);
// Stat says 'END'.
if (strpos($buf, "END\r\n") !== FALSE) {
break;
}
// Delete says DELETED or NOT_FOUND.
if (strpos($buf, "DELETED\r\n") !== FALSE || strpos($buf, "NOT_FOUND\r\n") !== FALSE) {
break;
}
// Flush_all says ok.
if (strpos($buf, "OK\r\n") !== FALSE) {
break;
}
}
fclose($s);
return ($buf);
}
I'm trying to build a small CMS using CodeIgniter, and I need to be able to dynamically update some variables within the application/config.php
So far I did:
private function update_file ($file, $var, $var_name) {
$start_tag = "<?php if ( ! defined('BASEPATH')) exit('No direct script access allowed');\n";
if (file_exists($file)) {
require_once ($file);
$updated_array = array_merge($$var_name, $var);
$data = $start_tag."\$".$var_name." = ".var_export($updated_array, true).";";
file_put_contents($file, $data);
} else {
return false;
}
}
Everything works just fine! The result in the config.php file will be:
<?php ...;
$config = array (
'base_url' => '',
...
...
);
But what if I would like to maintain the original config.php file format with comments, spaces and
separated declared $config['key'] = 'value' ... ?
Is that possible ?
EDIT:
Thank you for your answers, very precious.
I found a slightly different solution for my needs, performing a preg_replace on the return of file_get_contents() and then write back on the file the new resulting string. File maintains the exact original clean format.
private function update_file ($file, $var, $var_name) {
if (file_exists($file)) {
require_once ($file);
$contents = file_get_contents($file);
$updated_array = array_merge($$var_name, $var);
$search = array();
$replace = array();
foreach($$var_name as $key => $val) {
$pattern = '/\$'.$var_name.'\[\\\''.$key.'\\\'\]\s+=\s+[^\;]+/';
$replace_string = "\$".$var_name."['".$key."'] = ".var_export($updated_array[$key], true);
array_push($search, $pattern);
array_push($replace, $replace_string);
}
$new_contents = preg_replace($search, $replace, $contents);
write_file($file, $new_contents);
}
Maybe it requires some slight performance improvements. But this is my baseline idea.
create the keys with empty values
$config['base_url'] = '';
then set them inside any of your controllers.
This works best if you store the values in the db, and initialize them in MY_Controller.
$this->config->set_item('base_url', 'value');
It is possible. I can't find the code , but once i have written something like that. Whole idea was based on tokenizing template file and substitute values in an array, preserving key order, line numbers and comments from the template.
[+] Found it. It's purpose was to fill values from template that looked like this (it was much bigger of course):
<?php
$_CFG = array(
// DB section
'db_host' => 'localhost',
'db_user' => 'root',
'db_pass' => '',
'db_name' => 'test',
// Site specific
'lang' => array('pl','en'),
'admin' => 'admin#example.com',
);
And the code that was doing all the magic:
$tokens = token_get_all(file_get_contents('tpl/config.php'));
$level = -1;
$buffer = '';
$last_key = 0;
$iteration = 0;
foreach($tokens as $t){
if($t === ')'){
$iteration = 0;
$last_key = 0;
$level--;
}
if(is_array($t)){
if($t[0] == T_ARRAY && strtolower($t[1]) === 'array')
$level++;
if($t[0] == T_CONSTANT_ENCAPSED_STRING){
if($last_key){
if($level){
if(isset($new_config[$last_key][$iteration])){
$buffer .= var_export($new_config[$last_key][$iteration], TRUE);
}
else
$buffer .= 'null';
$iteration++;
}
else{
if(isset($new_config[$last_key]))
$buffer .= var_export($new_config[$last_key], TRUE);
else
$buffer .= 'null';
$last_key = 0;
}
}
else{
$buffer .= $t[1];
$last_key = trim($t[1],"'");
}
}
else
$buffer .= $t[1];
}
else
$buffer .= $t;
}
file_put_contents('config.php',$buffer);
Is there a way to know the avaliable ram in a server (linux distro) with php (widthout using linux commands)?
edit: sorry, the objective is to be aware of the ram available in the server / virtual machine, for the particular server (even if that memory is shared).
If you know this code will only be running under Linux, you can use the special /proc/meminfo file to get information about the system's virtual memory subsystem. The file has a form like this:
MemTotal: 255908 kB
MemFree: 69936 kB
Buffers: 15812 kB
Cached: 115124 kB
SwapCached: 0 kB
Active: 92700 kB
Inactive: 63792 kB
...
That first line, MemTotal: ..., contains the amount of physical RAM in the machine, minus the space reserved by the kernel for its own use. It's the best way I know of to get a simple report of the usable memory on a Linux system. You should be able to extract it via something like the following code:
<?php
$fh = fopen('/proc/meminfo','r');
$mem = 0;
while ($line = fgets($fh)) {
$pieces = array();
if (preg_match('/^MemTotal:\s+(\d+)\skB$/', $line, $pieces)) {
$mem = $pieces[1];
break;
}
}
fclose($fh);
echo "$mem kB RAM found"; ?>
(Please note: this code may require some tweaking for your environment.)
Using /proc/meminfo and getting everything into an array is simple:
<?php
function getSystemMemInfo()
{
$data = explode("\n", file_get_contents("/proc/meminfo"));
$meminfo = array();
foreach ($data as $line) {
list($key, $val) = explode(":", $line);
$meminfo[$key] = trim($val);
}
return $meminfo;
}
?>
var_dump( getSystemMemInfo() );
array(43) {
["MemTotal"]=>
string(10) "2060700 kB"
["MemFree"]=>
string(9) "277344 kB"
["Buffers"]=>
string(8) "92200 kB"
["Cached"]=>
string(9) "650544 kB"
["SwapCached"]=>
string(8) "73592 kB"
["Active"]=>
string(9) "995988 kB"
...
Linux commands can be run using the exec function in PHP. This is efficient and will do the job(if objective is to get the memory).
Try the following code:
<?php
exec("free -mtl", $output);
print_r($output);
?>
Small and tidy function to get all of its values associated to their keys.
$contents = file_get_contents('/proc/meminfo');
preg_match_all('/(\w+):\s+(\d+)\s/', $contents, $matches);
$info = array_combine($matches[1], $matches[2]);
// $info['MemTotal'] = "2047442"
I don't think you can access the host server memory info without a special written PHP extension. The PHP core library does not allow (perhaps for security reasons) to access the extended memory info.
However, if your script has access to the /proc/meminfo then you can query that special file and grab the info you need. On Windows (although you've not asked for it) we can use the com_dotnet PHP extension to query the Windows framework via COM.
Below you can find my getSystemMemoryInfo that returns that info for you no matter if you run the script on a Linux/Windows server. The wmiWBemLocatorQuery is just a helper function.
function wmiWBemLocatorQuery( $query ) {
if ( class_exists( '\\COM' ) ) {
try {
$WbemLocator = new \COM( "WbemScripting.SWbemLocator" );
$WbemServices = $WbemLocator->ConnectServer( '127.0.0.1', 'root\CIMV2' );
$WbemServices->Security_->ImpersonationLevel = 3;
// use wbemtest tool to query all classes for namespace root\cimv2
return $WbemServices->ExecQuery( $query );
} catch ( \com_exception $e ) {
echo $e->getMessage();
}
} elseif ( ! extension_loaded( 'com_dotnet' ) )
trigger_error( 'It seems that the COM is not enabled in your php.ini', E_USER_WARNING );
else {
$err = error_get_last();
trigger_error( $err['message'], E_USER_WARNING );
}
return false;
}
// _dir_in_allowed_path this is your function to detect if a file is withing the allowed path (see the open_basedir PHP directive)
function getSystemMemoryInfo( $output_key = '' ) {
$keys = array( 'MemTotal', 'MemFree', 'MemAvailable', 'SwapTotal', 'SwapFree' );
$result = array();
try {
// LINUX
if ( ! isWin() ) {
$proc_dir = '/proc/';
$data = _dir_in_allowed_path( $proc_dir ) ? #file( $proc_dir . 'meminfo' ) : false;
if ( is_array( $data ) )
foreach ( $data as $d ) {
if ( 0 == strlen( trim( $d ) ) )
continue;
$d = preg_split( '/:/', $d );
$key = trim( $d[0] );
if ( ! in_array( $key, $keys ) )
continue;
$value = 1000 * floatval( trim( str_replace( ' kB', '', $d[1] ) ) );
$result[$key] = $value;
}
} else // WINDOWS
{
$wmi_found = false;
if ( $wmi_query = wmiWBemLocatorQuery(
"SELECT FreePhysicalMemory,FreeVirtualMemory,TotalSwapSpaceSize,TotalVirtualMemorySize,TotalVisibleMemorySize FROM Win32_OperatingSystem" ) ) {
foreach ( $wmi_query as $r ) {
$result['MemFree'] = $r->FreePhysicalMemory * 1024;
$result['MemAvailable'] = $r->FreeVirtualMemory * 1024;
$result['SwapFree'] = $r->TotalSwapSpaceSize * 1024;
$result['SwapTotal'] = $r->TotalVirtualMemorySize * 1024;
$result['MemTotal'] = $r->TotalVisibleMemorySize * 1024;
$wmi_found = true;
}
}
// TODO a backup implementation using the $_SERVER array
}
} catch ( Exception $e ) {
echo $e->getMessage();
}
return empty( $output_key ) || ! isset( $result[$output_key] ) ? $result : $result[$output_key];
}
Example on a 8GB RAM system
print_r(getSystemMemoryInfo());
Output
Array
(
[MemTotal] => 8102684000
[MemFree] => 2894508000
[MemAvailable] => 4569396000
[SwapTotal] => 4194300000
[SwapFree] => 4194300000
)
If you want to understand what each field represent then read more.
It is worth noting that in Windows this information (and much more) can be acquired by executing and parsing the output of the shell command: systeminfo
exec("grep MemTotal /proc/meminfo", $aryMem);
$aryMem[0] has your total ram minus kernel usage.
I don't remember having ever seen such a function -- its kind of out the scope of what PHP is made for, actually.
Even if there was such a functionnality, it would probably be implemented in a way that would be specific to the underlying operating system, and wouldn't probably work on both Linux and windows (see sys_getloadavg for an example of that kind of thing)
// helpers
/**
* #return array|null
*/
protected function getSystemMemInfo()
{
$meminfo = #file_get_contents("/proc/meminfo");
if ($meminfo) {
$data = explode("\n", $meminfo);
$meminfo = [];
foreach ($data as $line) {
if( strpos( $line, ':' ) !== false ) {
list($key, $val) = explode(":", $line);
$val = trim($val);
$val = preg_replace('/ kB$/', '', $val);
if (is_numeric($val)) {
$val = intval($val);
}
$meminfo[$key] = $val;
}
}
return $meminfo;
}
return null;
}
// example call to check health
public function check() {
$memInfo = $this->getSystemMemInfo();
if ($memInfo) {
$totalMemory = $memInfo['MemTotal'];
$freeMemory = $memInfo['MemFree'];
$swapTotalMemory = $memInfo['SwapTotal'];
$swapFreeMemory = $memInfo['SwapFree'];
if (($totalMemory / 100.0) * 30.0 > $freeMemory) {
if (($swapTotalMemory / 100.0) * 50.0 > $swapFreeMemory) {
return new Failure('Less than 30% free memory and less than 50% free swap space');
}
return new Warning('Less than 30% free memory');
}
}
return new Success('ok');
}