I am programming a PHP intranet and have problems with extracting of data from older system. It was created with Sharepoint and MSSQL. It stores the data terribly unclean.
I found the desired data searching by keyword only by accident with help of HeidiSQL (db management tool). Its fulltext filter is searching also in varbinary columns and one time it converted the varbinary content to meaningful xml data (I saw header + 50 next letters), but only for a second, then it disappeared back to the original value.
I moved data from SQLserver to MySQL via CSV export, so I can work with PHP.
The varbinary contains this value:
0xEFBBBF2F2A205F6C6369643D223130353122205F76657273696F6E3D2231342E302E34373632220D0A5F4C6F63616C42696E64696E67202A2F0D7461626C652E6D732D646973632D6261720D7B0D206261636B67726F756E642D636F6C6F723A233539353935393B0D206261636B67726F756E642D696D6167653A75726C28717569636B6C61756E63686865616465722D44434239423136342E6769662E706E673F63746167293B0D6261636B67726F756E642D7265706561743A7265706561742D783B0D20626F726465723A31707820736F6C696420233236323632363B0D746578742D616C69676E3A6C6566743B0D7D0D7461626C652E6D732D646973632048520D7B0D6865696768743A3170783B0D20636F6C6F723A233539353935393B0D7D0D7461626C652E6D732D646973632074647B0D666F6E742D73697A653A3870743B0D666F6E742D66616D696C793A7461686F6D612C73616E732D73657269663B0D7D0D7461626C652E6D732D646973632074640D7B0D766572746963616C2D616C69676E3A746F703B0D7D0D2E6D732D6469736320617B0D20636F6C6F723A234138323230433B0D746578742D6465636F726174696F6E3A6E6F6E653B0D7D0D2E6D732D6469736320613A686F7665727B0D20636F6C6F723A233030303B0D746578742D6465636F726174696F6E3A756E6465726C696E653B0D7D0D2E6D732D6469736320613A766973697465647B0D20636F6C6F723A234445343631433B0D746578742D6465636F726174696F6E3A6E6F6E653B0D7D0D2E6D732D6469736320613A766973697465643A686F7665727B0D20636F6C6F723A233030303B0D746578742D6465636F726174696F6E3A756E6465726C696E653B0D7D0D7461626C652E6D732D646973632074640D7B0D70616464696E673A303B0D20636F6C6F723A233346334633463B0D7D0D2E6D732D646973632D6E6F7061642C7461626C652074722074642E6D732D646973632D6E6F7061642C7461626C652E6D732D646973632D6E6F7061642074640D7B0D70616464696E673A3070783B0D7D0D7461626C652074722074642E6D732D646973632D70616461626F76657B0D70616464696E673A3570782035707820357078203570783B0D7D0D7461626C652E6D732D646973632D6261722074640D7B0D766572746963616C2D616C69676E3A6D6964646C653B0D6865696768743A323270783B0D70616464696E672D6C6566743A3570783B0D7D0D7461626C652E6D732D646973632D62617220494D470D7B0D766572746963616C2D616C69676E3A6D6964646C653B0D7D0D7461626C652E6D732D646973632D62617220420D7B0D766572746963616C2D616C69676E3A3130253B0D70616464696E672D72696768743A3570783B0D7D0D74642E6D732D646973632D626F7264657265642C74642E6D732D646973632D626F7264657265642D6E6F6C6566740D7B0D20626F726465723A30707820736F6C696420233539353935393B0D626F726465722D746F702D77696474683A3070783B0D7D0D7461626C652074722074642E6D732D646973632D626F7264657265642D6E6F6C6566740D7B0D626F726465722D6C6566742D77696474683A3170783B0D70616464696E673A30707820313070782035707820313070783B0D666F6E742D66616D696C793A76657264616E613B0D666F6E742D73697A653A31656D3B0D746578742D616C69676E3A6C6566743B0D7D0D7461626C652074722074642E6D732D646973632D626F7264657265647B0D70616464696E673A3070782030707820357078203570783B0D7D0D6469762E6D732D646973632D726F6F742D626F6479206469763A66697273742D6368696C6420703A66697273742D6368696C647B0D6D617267696E2D746F703A3070783B0D7D0D
How can I convert it back to meaningful XML? I tried these PHP functions: hexdec(), base64_encode(), base64_decode() - but none is working.
There is 10.000 rows there... Please help me :) Thank you in advance.
This works well also in PHP <5.4, thank you for your time:
if ( !function_exists( 'hex2bin' ) ) {
function hex2bin( $str ) {
$sbin = "";
$len = strlen( $str );
for ( $i = 0; $i < $len; $i += 2 ) {
$sbin .= pack( "H*", substr( $str, $i, 2 ) );
}
return $sbin;
}
}
Related
I have two apps one written in php and one in python and both of them use the same mysql database.
For the public id of the entries in some of the tables I use binary(16) fields(I can't change this, it must remain this way).
The question is how does python does the conversion of this binary field?
Let's take one of the entries as an example.
When I get it in php(from the db) the value of the public id is °•WiCÄ‘õ0Iò|–g, the same value is shown in SequelPro. But php myAdmin does a hex function over binary fields and shows 0bb09557691443c491f53049f27c9667. Now I managed in php to convert the binary to the value showed in php myAdmin and it works for all the entries but I've just noticed that python does another conversion. When I get the entry used in this example via python the public id is owwweye1rjnvt3i1d0ib18x3.
What I need to achieve is to convert in php what I get from MySql: °•WiCÄ‘õ0Iò|–g to what python sees: owwweye1rjnvt3i1d0ib18x3. The php app makes calls on the python one(not developed by me) and thus the id needs to be in the same format for a successfull call.
Any suggestions are welcomed. Thanks.
EDIT: If i send °•WiCÄ‘õ0Iò|–g from php to python and print it rigth away I get: °•WiCÄ‘õ0Iò|–g
Finally I've sorted this out.
Seems that python converts to base36 not hex as I've wrongly supposed.
I've tried to simply base_convert 0bb09557691443c491f53049f27c9667 from 16 to 36 but I've got owwweye1rk04k4cskkw4s08s. Not really what I needed but still a great step further as it started to look like owwweye1rjnvt3i1d0ib18x3.
This difference I supposed to appear because of the large values to be converted(loss of precision), so I've further researched and found the bellow function, written by Clifford dot ct at gmail dot com on the php.net website:
<?php
function str_baseconvert($str, $frombase=10, $tobase=36) {
$str = trim($str);
if (intval($frombase) != 10) {
$len = strlen($str);
$q = 0;
for ($i=0; $i<$len; $i++) {
$r = base_convert($str[$i], $frombase, 10);
$q = bcadd(bcmul($q, $frombase), $r);
}
}
else $q = $str;
if (intval($tobase) != 10) {
$s = '';
while (bccomp($q, '0', 0) > 0) {
$r = intval(bcmod($q, $tobase));
$s = base_convert($r, 10, $tobase) . $s;
$q = bcdiv($q, $tobase, 0);
}
}
else $s = $q;
return $s;
}
?>
I don't think others will come across this issue very often, but still if it happens hope they'll find this instead of burning their brains out like I did :))))
This question already has answers here:
PHP function to generate v4 UUID
(18 answers)
Closed 9 years ago.
I'm working on a video share project and I would like to generate "characters" id's for each video similar to how youtube does it. for example tgax-1sCgIs
Is it safe to use the following function to generate UUIDs, If for example I've 100000000 videos and I need to add new uuid, how can I be sure it's not duplicated?
function generateRandomString($length = 11) {
$characters = '0123456789abcdefghijklm-_nopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ';
$randomString = '';
for ($i = 0; $i < $length; $i++) {
$randomString .= $characters[rand(0, strlen($characters) - 1)];
}
return $randomString;
}
The following strictly addresses UUIDs. The URLs used by Youtube are not UUIDs and cannot be compared as such. They are much smaller (over 293 times smaller!) and do not have the same guarantees of such an immensely huge domain as a UUID. In this case (for "short hash tags"), duplicate checking must be used - but it need not differ than any other kind of duplicate checking.
If you create a UUID from a proper generator (e.g. a random UUIDv4 generator), then you can be assured that the probability of duplicates is "so low that it just doesn't matter".
As such, while I normally suggest not checking for duplicate UUIDs, there are cases when doing so is pertinent:
During re-mergering (i.e. a cyclic merge operation) where duplicates from prior data are expected and will occur;
The UUID comes from an untrusted generator (i.e. the UUID values be subverted/injected by an attacker or from other manual human intervention);
If used as an SQL Column/Index there is no reason why a Unique Constraint should not be applied as it is required anyway to maintain proper multiplicities.
On the other hand, while I find UUIDs very good for inter-boundary identification (such as transporting information between systems or providing "long" unique resource handles), I find UUIDs very poor to be used as a standard database "record identifier". Where I need a surrogate PK, I merely use a traditional auto-incremement column which is much easier on physical layout. (SQL Server provides a special UUID generator that is much better for indexing - but less secure - than a truly random v4 UUID.)
Unfortunately, PHPs standard uniqid (a "custom" format?) function does not provide the best guarantees. In any case, see PHP function to generate v4 UUID that shows a UUIDv4(-ish?) implementations that is much better than the posted code as they conform to a common generation technique and use a much higher grade random source. (However, please see the comments relating to how mt_rand is seeded - or not seeded - in the answers.)
If you're using a database you have a couple options:
Just use the auto increment column of the table you're storing the videos in. The number will always be unique.
Each time you generate an id, check the database to see if it exists. If it exists, re-run the function to generate a new uuid and check the database again. Do it until you query the database and no rows are returned with that id.
There are a few other posts you should look at that have a better approach at generating a true uuid:
Nice UUID class
PHP function to generate v4 UUID
How to create a UUID in php
I'm pretty sure YouTube is just encoding integer IDs in a base-X system. There's just so many, and they are created so fast, that they seem random.
The code would look something like:
<?php
$base_str = '0123456789abcdefghijklmnopqrstuvwxyz-_';
$base = strlen($base_str);
// generate a number if no input
if( ! isset($argv[1]) ) {
$number = rand(1000,1000000);
} else {
$number = intval($argv[1]);
}
printf("Input: %d\n", $number);
printf("Base: %d\n", $base);
// will hold the base-X encoded representation of the number
$repr = '';
for( $i=$number; $i>0; ) {
$remainder = $i % $base;
$digit_repr = substr($base_str, $remainder, 1);
$repr = $digit_repr . $repr;
printf("Rem: %2d Repr: %s Cur: %16d Progress: %s\n", $remainder, $digit_repr, $i, $repr);
$i = ($i - $remainder) / $base;
}
Example output:
Input: 2000000
Base: 38
Rem: 22 Repr: m Cur: 2000000 Progress: m
Rem: 1 Repr: 1 Cur: 52631 Progress: 1m
Rem: 17 Repr: h Cur: 1385 Progress: h1m
Rem: 36 Repr: - Cur: 36 Progress: -h1m
If you want to introduce a little more "randomness" into how the IDs look you can always scramble $base_str. Just keep in mind that you can only scramble it once before you start encoding IDs.
Decoding
I guess that's important, right?
<?php
$base_str = '0123456789abcdefghijklmnopqrstuvwxyz-_';
$base = strlen($base_str);
if( ! isset($argv[1]) ) {
$input = '-h1m';
} else {
$input = $argv[1];
}
printf("Input: %s\n", $input);
printf("Base: %d\n", $base);
$repr = str_split($input);
$number = 0;
for( $i=0; $i<count($repr); $i++) {
$number = $number * $base;
$value = strpos($base_str, $repr[$i]);
$number += $value;
printf("Char: %s Value: %2d Cur: %12d\n", $repr[$i], $value, $number);
}
Example output:
Input: -h1m
Base: 38
Char: - Value: 36 Cur: 36
Char: h Value: 17 Cur: 1385
Char: 1 Value: 1 Cur: 52631
Char: m Value: 22 Cur: 2000000
I have been asked/told to convert a foxpro function to PHP, however I know nothing about foxpro.
PARAMETERS cCkey
LOCAL cKey
cKey = SUBSTR(SYS(2015),2)+PADL(LTRIM(STR(INT(IIF(INT(RAND()*1000000000) = 851390329,RAND(-1),RAND())*1000000),6)),6,"0")
RETURN cKey
Above is the function they are wanting to use in a system that is being built in php to integrate with the foxpro databases.
Some of the functions are familiar from PHP, but others like the "SYS", and "IIF" are not, and being that I know someone on here will be able to take one look at it, and know exactly what it is doing.
Mind helping me out? Thanks in advance.
Sys(2015) is a handy VFP function which returns a value unique to that session of VFP. You can read it here
Iif is inline if-else-endif statement .. like Excel does
Updated
SYS(2015) in PHP ? I don't know .. but if we talking about random string in PHP, you can use this
function rand_string( $length ) {
$chars = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789";
$size = strlen( $chars );
for( $i = 0; $i < $length; $i++ ) {
$str .= $chars[ rand( 0, $size - 1 ) ];
}
return $str;
}
I got that from this link and got the basic idea from this link
About the other part MAYBE like this :
$randomresult = 0
$srandom = ""
If (INT(RAND()*1000000000) = 851390329)
{
$randomresult = int(rand(-1)) * 1000000
}
else
{
$randomresult = int(rand()) * 1000000
}
$srandom=str_pad(ltrim(strval($randomresult),"0")),6,"0",STR_PAD_LEFT)
So, MAYBE we can make the foxpro code like this in PHP :
$cKey = rand_string(10) . str_pad(ltrim(strval($randomresult),"0")),6,"0",STR_PAD_LEFT)
At least you can the idea .....
SYS(2015) Returns a unique 10-character procedure name that begins with an underscore followed by a combination of letters and numbers.
http://msdn.microsoft.com/en-us/library/684by7c1(v=vs.80).aspx
IIF Returns one of two values depending on the value of a logical expression.
http://msdn.microsoft.com/en-us/library/7ttt15k6(v=vs.80).aspx
With this information I think you can at least take a stab at creating a PHP function and then showing some PHP and asking for help if needed.
I have a set of ZLIB compressed / base64 encoded strings (done in a C program) that are stored in a database. I have written a small PHP page that should retrieve these values and plot them (the string originally was a list of floats).
Chunk of C program that compresses/encodes:
error=compress2(comp_buffer, &comp_length,(const Bytef*)data.mz ,(uLongf)length,Z_DEFAULT_COMPRESSION); /* compression */
if (error != Z_OK) {fprintf(stderr,"zlib error..exiting"); exit(EXIT_FAILURE);}
mz_binary=g_base64_encode (comp_buffer,comp_length); /* encoding */
(Example) of original input format:
292.1149 8379.5928
366.1519 101313.3906
367.3778 20361.8105
369.1290 17033.3223
375.4355 1159.1841
467.3191 8445.3926
Each column was compressed/encoded as a single string. To reconstruct the original data i am using the following code:
//$row[4] is retrieved from the DB and contains the compressed/encoded string
$mz = base64_decode($row[4]);
$unc_mz = gzuncompress($mz);
echo $unc_mz;
Yet this gives me the following output:
f6jEÍ„]EšiSE#IEfŽ
Could anyone give me a tip/hint about what I might be missing?
------ Added Information -----
I feel that the problem comes from the fact that currently php views $unc_mz as a single string while in reality i would have to re-construct an array containing X lines (this output was from a 9 line file) but... no idea how to do that assignment.
The C program that did that went roughly like this:
uncompress( pUncompr , &uncomprLen , (const Bytef*)pDecoded , decodedSize );
pToBeCorrected = (char *)pUncompr;
for (n = 0; n < (2 * peaksCount); n++) {
pPeaks[n] = (RAMPREAL) ((float *) pToBeCorrected)[n];
}
where peaksCount would be the amount of 'lines' in the input file.
EDIT (15-2-2012): The problem with my code was that I was not reconstructing the array, the fixed code is as follows (might be handy if someone needs a similar snippet):
while ($row = mysql_fetch_array($result, MYSQL_NUM)) {
$m< = base64_decode($row[4]);
$mz_int = gzuncompress($int);
$max = strlen($unc_mz);
$counter = 0;
for ($i = 0; $i < $max; $i = $i + 4) {
$temp= substr($unc_mz,$i,4);
$temp = unpack("f",$temp);
$mz_array[$counter] = $temp[1];
$counter++;
}
The uncompressed string has to be chopped into chunks corresponding to the length of a float, unpack() then reconstructs the float data from teh binary 'chunk'. That's the simplest description that I can give for the above snippet.
compress2() produces the zlib format (RFC 1950). I would have to guess that something called gzuncompress() is expecting the gzip format (RFC 1952). So gzuncompress() would immediately fail upon not finding a gzip header.
You would need to use deflateInit2() in zlib to request that deflate() produce gzip-formatted output, or find or provide a different function in PHP that expects the zlib format.
I'm using the following code to create an xls file from php.
http://www.appservnetwork.com/modules.php?name=News&file=article&sid=8
However, for some reason if the row "comments" is more than 255 characters, it doesn't output anything..
The code which writes the String to the XLS file is:
function xlsWriteString( $Row, $Col, $Value ) {
$L = strlen( $Value );
echo pack( "ssssss", 0x204, 8 + $L, $Row, $Col, 0x0, $L );
echo $Value;
return;
};
Could someone help me get this to display the field regardless of how many characters are in the string?
Thanks
EDIT: I found this: http://support.microsoft.com/kb/213841 but not sure how to implement the work around into php..
ANOTHER EDIT: Even if anyone knows how to merge cells? That would also work?? :)
Sounds like it may be related to this issue.