Background:
I have this "with rollup" query defined in MySQL:
SELECT
case TRIM(company)
when 'apple' THEN 'AAPL'
when 'microsoft' THEN 'MSFT'
else '__xx__'
END as company
,case TRIM(division)
when 'hardware' THEN Trim(division)
when 'software' THEN Trim(division)
else '__xx__'
END as division
,concat( '$' , format(sum(trydollar),0)) as dollars
FROM pivtest
GROUP BY
company, division with rollup
And it generates this output:
AAPL;hardware;$279,296
AAPL;software;$293,620
AAPL;__xx__;$572,916
MSFT;hardware;$306,045
MSFT;software;$308,097
MSFT;__xx__;$614,142
__xx__;__xx__;$1,187,058
If you have used "with rollup" queries in MySQL before, you can most likely infer the structure of my source table.
Question:
Given this raw output of MySQL, what is the easiest way to get a "tree" structure like the following?
AAPL
hardware;$279,296
software;$293,620
Total; $572,916
MSFT
hardware;$306,045
software;$308,097
Total;$614,142
Total
$1,187,058
Easiest is to do it in whatever client program you're using to receive and show the user MySQL's output -- definitely not easiest to implement presentation-layer functionality in the data layer!-) So tell us what language &c is in your client program and we may be able to help...
Edit: giving a simple Python client-side solution at the original asker's request.
With Python's DB API, results from a DB query can be most simply seen as a list of tuples. So here's a function to format those results as required:
def formout(results):
marker = dict(__xx__=' Total')
current_stock = None
for stock, kind, cash in results:
if stock != current_stock:
print marker.get(stock, stock).strip()
current_stock = stock
if kind in marker and stock in marker:
kind = ' '*8
print ' %s;%s' % (marker.get(kind, kind), cash)
marker is a dictionary to map the special marker '__xx__' into the desired string in the output (I'm left-padding it appropriately for the "intermediate" totals, so when I print the final "grand total", I .strip() those blanks off). I also use it to check for the special case in which both of the first two columns are the marker (because in that case the second column needs to be turned into spaces instead). Feel free to ask in comments for any further clarification of Python idioms and use that may be necessary!
Here's the output I see when I call this function with the supplied data (turned into a list of 7 tuples of 3 strings each):
AAPL
hardware;$279,296
software;$293,620
Total;$572,916
MSFT
hardware;$306,045
software;$308,097
Total;$614,142
Total
;$1,187,058
The space-alignment is not identical to that I see in the question (which is a little inconsistent in terms of how many spaces are supposed to be where) but I hope it's close enough to what you want to make it easy for you to adjust this to your exact needs (as you're having to translate Python into PHP anyway, the space-adjustment should hopefully be the least of it).
Related
I've searched and tried lots of things but I can't find a solution.
I am storing some SNMP OIDs in a database, and displaying them in a table with datatables.
I want the OIDs to be displayed in the correct order so for example:
1.3.6.1.2.1.1
1.3.6.1.2.1.10
1.3.6.1.2.1.2
In correct order would be:
1.3.6.1.2.1.1
1.3.6.1.2.1.2
1.3.6.1.2.1.10
A SQL query with order by on the column storing the OID string would order them:
1.3.6.1.2.1.1
1.3.6.1.2.1.10
1.3.6.1.2.1.2
I'm using serverside processing with either PHP or preferably python flask. Currently I am building the table myself in flask and have written a function that orders them by converting the OIDs to tuples and sorting. This works but I would like to use datatables to get the pagination and responsiveness.
One thing to note is there isn't a limit on the length of the OID.
Any ideas would be much appreciated.
This is kind of a hack, but it might work. If each element in the OID has a max value < 100, then create a second column in the database, where each element is converted to a 2-digit 0-filled value:
real_oid sorting_oid
1.3.6.1.2.1.1 01.03.06.01.02.01.01
1.3.6.1.2.1.10 01.03.06.01.02.01.10
1.3.6.1.2.1.2 01.03.06.01.02.01.02
You could even eliminate the periods to save space, once you tested that is is all working.
First split the string on the period and typecast to int. Then use sorted and operator.itemgetter to sort by multiple attributes. Then re-join using a period. Something like the following:
original_oids = [...]
split_and_typecast_oids = [map(int, oid.split(".")) for oid in original_oids]
sorted_oids = sorted(split_and_typecast_oids, operator.itemgetter(1,2,3,4,5,6,7))
rejoined_oids = [".".join(map(str, oid)) for oid in sorted_oids]
I have a text file that looks like this:
<http://dbpedia.org/resource/Autism> <http://www.w3.org/2000/01/rdf-schema#comment> "Autism is a disorder of neural development characterized by impaired social interaction and communication, and by restricted and repetitive behavior. The diagnostic criteria require that symptoms become apparent before a child is three years old. Autism affects information processing in the brain by altering how nerve cells and their synapses connect and organize; how this occurs is not well understood."#en .
<http://dbpedia.org/resource/Anarchism> <http://www.w3.org/2000/01/rdf-schema#comment> "Anarchism is generally defined as the political philosophy which holds the state to be undesirable, unnecessary, and harmful, or alternatively as opposing authority and hierarchical organization in the conduct of human relations. Proponents of anarchism, known as \"anarchists\", advocate stateless societies based on non-hierarchical voluntary associations. There are many types and traditions of anarchism, not all of which are mutually exclusive."#en .
<http://dbpedia.org/resource/Achilles> <http://www.w3.org/2000/01/rdf-schema#comment> "In Greek mythology, Achilles was a Greek hero of the Trojan War, the central character and the greatest warrior of Homer's Iliad. Plato named Achilles the most handsome of the heroes assembled against Troy. Later legends (beginning with a poem by Statius in the 1st century AD) state that Achilles was invulnerable in all of his body except for his heel. As he died because of a small wound on his heel, the term Achilles' heel has come to mean a person's principal weakness."#en .
I'm using code (not relevant here) to extract the name of the article in the first url in each line. Then I extract the first sentence of the description between quotes. The problem is when I try to insert that first sentence string into my table, the insert fails (echoing works fine). Just inserting the title without the description works fine. Does anyone have any idea why the description makes the insert fail?
Here's the code I'm using to get the first sentence:
$data = fgets($handle); //get line
$data = str_replace("> ", "-!-", $data);
dataArr = explode("-!-", $data);
//Get last part of uri from 1st element in array
$title = getLastPartOfUrl($dataArr[0]);
$desc=preg_replace('/(.*?[?!.](?=\s|$)).*/', '\\1', escape(substr($dataArr[2],1)));
$db->query("insert into mytable SET title = '".$title."', desc ='".$desc."'");
function escape($str)
{
$search=array("\\","\0","\n","\r","\x1a","'",'"');
$replace=array("\\\\","\\0","\\n","\\r","\Z","\'",'\"');
return str_replace($search,$replace,$str);
}
EDIT: I tried both urlencode and addslashes to no avail, in both cases including the $desc string makes the insert fail.
You don't escape the title.
I wouldn't trust your escape function either. I'm not sure what $db is, but you should use properly parameterized queries with PDO/mysqli
EDIT: DESC is a reserved word in MySQL. You need to surround it (when used as a column name) with backticks in your query.
i am using mysqlclient,
in one of my query, as shown below
sprintf (query, "select user from pcloud_session where id = '%s'", sid);
here some time this sid is with % sign in it like the example
2Cq%yo4i-ZrizGGQGQ71eJQ0
but when there is this % this query always fail, i think i have to escape this %, but how ?
i tried with \ and %% , but both of this not working, please help me here
UPDATE:
When using session.hash_bits_per_character = 6, in php session ,the default charset contains a character (comma) that will always be urlencoded(here it is %2C). This results in cookie values having this %2C in it, but session db having a comma instead of it. any idea about fixing this problem ?.. sorry for the confusion
Thanks
There's no need to escape a literal '%' in MySQL query text.
When you say the query "always fail", is it the call to the mysql_query function that is returning an error? Does it return a SQL Exception code, or is it just not returning the resultset (row) you expect?
For debugging, I suggest you echo out the contents of the query string, after the call to sprintf. We'd expect the contents of the string to be:
select user from pcloud_session where id = '2Cq%yo4i-ZrizGGQGQ71eJQ0'
And I don't see anything wrong with that SQL construct (assuming the id column exists in pcloud_session and is of character datatype. Even if id was defined as an integer type, that statement wouldn't normally throw an exception, the string literal would just be interpreted as integer value of 2.)
There should be no problem including a '%' literal into the target format of an sprintf. And there should be no problem including a '%' literal within MySQL query text.
(I'm assuming, of course, that sid is populated by a call to mysql_real_escape_string function.)
Again, I suggest you echo out the contents of query, following the call to sprintf. I also suggest you ensure that no other code is mucking with the contents of that string, and that is the actual string being passed as an argument to mysql_query function. (If you are using the mysql_real_query function, then make sure you are passing the correct length.)
UPDATE
Oxi said: "It does not return a SQL Exception code, it just does not return the result[set] I expect. I did print the query, it prints with % in it."
#Oxi
Here's a whole bunch of questions that might help you track down the problem.
Have you run a test of that query text from the mysql command line client, and does that return the row(s) you expect?
Is that id column defined as VARCHAR (or CHAR) with a length of (at least) 24 characters? Is the collation on the column set as case insensitive, or is it case sensitive?
show create table pcloud_session ;
(I don't see any characters in there that would cause a problem with characterset translation, although that could be a source of a problem, if your application is not matching the database charactarset encoding.)
Have you tested queries using a LIKE predicate against that id column?
SELECT id, user FROM pcloud_session WHERE id LIKE '2Cq\%yo4i-%' ESCAPE '\\'
ORDER BY id LIMIT 10 ;
SELECT id, user FROM pcloud_session WHERE id LIKE '2Cq%'
ORDER BY id LIMIT 10 ;
Are you getting no rows returned when you expect one row? Are you getting too many rows returned, or are you getting a different row than the one you expect?
That is an oddball value for an id column. At first, it looks almost as if the value is represented in a base-64 encoding, but it's not any standard encoding, since it includes the '%' and the '-' characters.
If you're going to do this in C without an interface library, you must use mysql_real_escape_string to do proper SQL escaping.
There shouldn't be anything intrinsically wrong with using '%inside of a string, though, as the only context in which it has meaning is either directly inprintftype functions or as an argument toLIKE` inside of MySQL.
This proves to be really annoying, but it's absolutely necessary. It's going to make your code a lot more complicated which is why using low-level MySQL in C is usually a bad idea. The C++ wrapper will give you a lot more support.
You really shouldn't escape the string yourself. The safest option is to let the MySQL API handle it for you.
For a string of maximum length n, start by allocating a string of length 2*n+1:
int sidLength = strlen(sid);
// worst-case, we need to escape every character, plus a byte for the ASCIIZ
int maxSafeSidLength = sidLength * 2 + 1;
char *safeSid = malloc(maxSafeSidLength);
// copy "sid" to "safeSid", escaping as appropriate
mysql_real_escape_string(mysql, safeSid, sid, sidLength);
// build the query
// ...
free(safeSid);
There's a longer example at the mysql_real_escape_string page on dev.mysql.com, in which they build the entire query string, but the above approach should work for supplying safeSid to sprintf.
I'm new to web design, especially backend design so I have a few questions about implementing a search function in PHP. I already set up a MySQL connection but I don't know how to access specific rows in the MySQL table. Also is the similar text function implemented correctly considering I want to return results that are nearly the same as the search term? Right now, I can only return results that are the exact same or it gives "no result." For example, if I search "tex" it would return results containing "text"? I realize that there are a lot of mistakes in my coding and logic, so please help if possible. Event is the name of the row I am trying to access.
$input = $_POST["searchevent"];
while ($events = mysql_fetch_row($Event)) {
$eventname = $events[1];
$eventid = $events[0];
$diff = similar_text($input, $event, $hold)
if ($hold == '100') {
echo $eventname;
break;
else
echo "no result";
}
Thank you.
I've noticed some of the comments mentioned more efficient ways of performing the search than with the "similar text" function, if I were to use the LIKE function, how would it be implemented?
A couple of different ways of doing this:
The faster one (performance wise) is:
select * FROM Table where keyword LIKE '%value%'
The trick in this one is the placement of the % which is a wildcard, saying either search everything that ends or begins with this value.
A more flexible but (slightly) slower one could be the REGEXP function:
Select * FROM Table WHERE keyword REGEXP 'value'
This is using the power of regular expressions, so you could get as elaborate as you wanted with it. However, leaving as above gives you a "poor man's Google" of sorts, allowing the search to be bits and pieces of overall fields.
The sticky part comes in if you're trying to search names. For example, either would find the name "smith" if you searched SMI. However, neither would find "Jon Smith" if there was a first and last name field separated. So, you'd have to do some concatenation for the search to find either Jon OR Smith OR Jon Smith OR Smith, Jon. It can really snowball from there.
Of course, if you're doing some sort of advanced search, you'll have to condition your query accordingly. So, for instance, if you wanted to search first, last, address, then your query would have to test for each:
SELECT * FROM table WHERE first LIKE '%value%' OR last LIKE '%value%' OR address LIKE '%value'
Look at below example :
$word2compare = "stupid";
$words = array(
'stupid',
'stu and pid',
'hello',
'foobar',
'stpid',
'upid',
'stuuupid',
'sstuuupiiid',
);
while(list($id, $str) = each($words)){
similar_text($str, $word2compare, $percent);
if($percent > 90) // Change percentage value to 80,70,60 and see changes
print "Comparing '$word2compare' with '$str': ";
}
You can check with $percent parameter for how strong match you want to apply.
I have a data set that is generated by a Zip Code range search:
$zips:
key -> value
11967 -> 0.5
11951 -> 1.3
The key is the Zip Code (Which I need to query the Database for), and the value is the miles from the user entered zip code. I need to take the key (Zip Code) and search the database, preferably using a MySQL query similar to my current one:
$getlistings = mysql_query("SELECT * FROM stores WHERE zip IN ($zips)");
The other alternative is to change the array somehow in my code. I tried looking in code for where the array is generated originally but I couldn't find it. Any help would be greatly appreciated!! Thanks :)
You could convert the array keys to a SQL-compatible string. For example:
'11967', '11951'
and then use the string in the query.
Since the SQL query doesn't know what a php array is and there's no good way (that I know of) to extract just the keys and surround them in quotes, so this may be your best bet.
EDIT: As Ionut G. Stan wrote (and gave an example for), using the implode and array_map functions will get you there. However, I believe the solution provided will only work if your column definition is numeric. Character columns would require that elements be surrounded by apostrophes in the IN clause.
array_keys should be what you're looking for.
$zip = array_keys($zips); # gives you simple array(11967, 11951);
implode(', ', $zip); # results in: '11967, 11951'
Cannot comment the other answers, so one additional remark from my side. Depending on the country you are in and what you do with the data... In Germany there are ZIP-Codes starting with "0" so you should make sure that you either do not store them as numerical value if you want to compare them to other data (e.g. ZIP <-> geocoord-mappings) or make sure that you convert them to int everywhere and use filtering on the output.
Old ZIP codes had four numbers, new ones have five. So displaying a new ZIP with four numbers because the leading 0 is missing will lead to confusion.
Regarding use of a temporary table i would say it depends on the size of the table and how many zip codes are used in the query.
This should do it:
// array_map sanitizes the data
$zip_codes = implode(', ', array_map('intval', array_keys($zips)));
$getlistings = mysql_query("SELECT * FROM stores WHERE zip IN ($zip_codes)");
For best performance, you should create a temporary table, fill it with your ZIP codes and query like this:
SELECT *
FROM stores
JOIN temptable
ON zip = tempvalue
Of course this will be more efficient only if you ZIP column is indexed.
I just want to throw in that the previous code snippets gave me some syntax errors and the database just spitted out one entry instead of all relevant data. The following snippet worked for me:
implode("','", $zipNumbers);