I'm having this issue with the JSONAPI for minecraft. http://mcjsonapi.com/
I am trying to use the method "files.write" or "setFileContents" to replace the contents of a file. The website states this about the method.
Pretty simple. Just rewrites the file right? Yeah but this is proving to be more difficult then I thought. At first attempt, I was trying to set 3450 characters to the file "groups.yml" on the minecraft server. Here's the code I ran in PHP:
var_dump(
$api->call("files.write", array("plugins/GroupManager/worlds/world/groups.yml", (string)$yaml))
);
The var_dump is supposed to either return a success statement or an error describing what went wrong. But instead all I get is "null". This isn't right, and I know $yaml is being casted to a string, which isn't the issue. So I decide to start testing around. After much testing, I find that the character length of what I can set is exactly 1622. Adding another space or anything causes null, otherwise, it works. This is the modified file that is below 1622 characters I tested with.
So great, you found the issue, right? No, I didn't. I thought 1622 was an odd number to stop working, so I did some further testing. I tried to set 3000 characters I generated from just smashing my keyboard, and it worked! So what's going on here?
This and this works, but this doesn't. Why is this? This app called Adminium runs this exact API, and includes a file management system inside the app which I am assuming uses the same methods I am using, but it doesn't have a problem.
I have a forum post here that I also asked on, and still haven't gotten to an answer yet.
I am a novice in working with the php language but have been learning over some time through testing and lots of time searching on the internet for examples and tutorials. Once in a while I get stumped with a task I just can't seem to figure out on my own. So I am asking that someone point me at least in the right direction in my current question.
I need to have the ability to add basically any character I want in a form field that will display the results I type in. Currently, the template uses the code below for the field and displays back only numbers. I need to upgrade or change it to allow any character I want.
Here it is
.number_format($price, 0, '.', ',')
Do I need to provide any additional info or does this get anyone's gears turning?
Much appreciated.
EDIT:
For those having issues with mysql data that won't get rid of white spaces between rows. It's because of deep tabbed indents due to an issue when copy pasting from editors/word documents.
http://bugs.mysql.com/bug.php?id=49611
Here is the bug that doesn't allow me to fix this issue, it's because of deep indents, it's a MYSQL bug that won't be fixed.
PS: Message to community, Thanks to the guys that downvoted this question without giving time to properly formulate the issue. It's a debug process, you can let people know if you need something instead of acting rude and attacking the questioner, after all as the poster I'm likely to read and see what you need. If I knew what was needed to diagnose the issue, I would've found it and would not be posting here. Since I posted here, it means I don't know where to look as to identify if the issue is a MYSQL one or a PHP one. Be polite with your community guys, don't act like wolves trying to bite someone's head off all the time because they didn't give you everything you wanted right off the bat.. Take care.
Here is the raw paste from the database:
one two three
one two three
one two three
one two three
one two three
Why is it still being like this? I str_replaced 's and \n\n\n's . I don't want to get rid of 2 \n's due to the format of the text I need in between the text.
The result I'm looking for is :
one two three
one two three
one two three
one two three
Meaning max 2 \n's.
This might be what you are after:
$normalized = preg_replace('~(\r?\n\r?){3+}~', "\n\n", $original);
But hey, it's hard to say. You want to keep stack overflow a happy place, you put some effort to posting a question.
People search in my website and some of these searches are these ones:
tapoktrpasawe
qweasd qwa as
aıe qwo ıak kqw
qwe qwe qwe a
My question is there any way to detect strings that similar to ones above ?
I suppose it is impossible to detect 100% of them, but any solution will be welcomed :)
edit: I mean the "gibberish searches". For example some people search strings like "asdqweasdqw", "paykaprkg", "iwepr wepr ow" in my search engine, and I want to detect jibberish searches.
It doesn't matter if search result will be 0 or anything else. I can't use this logic.
Some new brands or products will be ignored if I will consider "regular words".
Thank you for your help
You could build a model of character to character transitions from a bunch of text in English. So for example, you find out how common it is for there to be a 'h' after a 't' (pretty common). In English, you expect that after a 'q', you'll get a 'u'. If you get a 'q' followed by something other than a 'u', this will happen with very low probability, and hence it should be pretty alarming. Normalize the counts in your tables so that you have a probability. Then for a query, walk through the matrix and compute the product of the transitions you take. Then normalize by the length of the query. When the number is low, you likely have a gibberish query (or something in a different language).
If you have a bunch of query logs, you might first make a model of general English text, and then heavily weight your own queries in that model training phase.
For background, read about Markov Chains.
Edit, I implemented this here in Python:
https://github.com/rrenaud/Gibberish-Detector
and buggedcom rewrote it in PHP:
https://github.com/buggedcom/Gibberish-Detector-PHP
my name is rob and i like to hack True
is this thing working? True
i hope so True
t2 chhsdfitoixcv False
ytjkacvzw False
yutthasxcvqer False
seems okay True
yay! True
You could do what Stackoverflow does and calculate the entropy of the string.
Of course, this is just one of many heuristics SO uses to determine low-quality answers, and should not be relied upon as 100% accurate.
Assuming you mean jibberish searches... It would be more trouble than it's worth. You are providing them with a search functionality, let them use it however they please. I'm sure there are some algorithms out there that detect strange character groupings, but it would probably be more resource/labour intensive than just simply returning no results.
I had to solve a closely related problem for a source code mining project, and although the package is written in Python and not PHP, it seemed worth mentioning here in case it can still be useful somehow. The package is Nostril (for "Nonsense String Evaluator") and it is aimed at determining whether strings extracted during source-code mining are likely to be class/function/variable/etc. identifiers or random gibberish. It works well on real text too, not just program identifiers. Nostril uses n-grams (similar to the Gibberish Detector in the answer by Rob Neuhaus) in combination with a custom TF-IDF scoring function. It comes pretrained, and is ready to use out of the box.
Example: the following code,
from nostril import nonsense
real_test = ['bunchofwords', 'getint', 'xywinlist', 'ioFlXFndrInfo',
'DMEcalPreshowerDigis', 'httpredaksikatakamiwordpresscom']
junk_test = ['faiwtlwexu', 'asfgtqwafazfyiur', 'zxcvbnmlkjhgfdsaqwerty']
for s in real_test + junk_test:
print('{}: {}'.format(s, 'nonsense' if nonsense(s) else 'real'))
will produce the following output:
bunchofwords: real
getint: real
xywinlist: real
ioFlXFndrInfo: real
DMEcalPreshowerDigis: real
httpredaksikatakamiwordpresscom: real
faiwtlwexu: nonsense
asfgtqwafazfyiur: nonsense
zxcvbnmlkjhgfdsaqwerty: nonsense
The project is on GitHub and I welcome contributions.
I'd think you could detect these strings the same way you could detect "regular words." It's just pattern matching, no?
As to why users are searching for these strings, that's the bigger question. You may be able to stem off the gibberish searches some other way. For example, if it's comment spam phrases that people (or a script) is looking for, then install a CAPTCHA.
Edit: Another end-run around interpreting the input is to throttle it slightly. Allow a search every 10 seconds or so. (I recall seeing this on forum software, as well as various places on SO.) This will take some of the fun out of searching for sdfpjheroptuhdfj over and over again, and at the same time won't interfere with the users who are searching for, and finding, their stuff.
As some people commented, there are no hits in google for tapoktrpasawe or putjbtghguhjjjanika (Well, there are now, of course) so if you have a way to do a quick google search through an API, you could throw out any search terms that got no Google results and weren't the names of one of your products. Why you would want to do this is a whole other question - are you trying to save effort for your search library? Make your hand-review of "popular search terms" more meaningful? Or are you just frustrated at the inexplicable behaviour of some of the people out on the big wide internet? If it's the latter, my advice is just let it go, even if there is a way to prevent it. Some other weirdness will come along.
Short answer - Jibberish Search
Probabilistic Language Model works.
Logic
word is made up of sequence of characters, and if 2 characters come together more frequently and if we sum up all frequency of 2 contiguous characters coming together in word, and sum cross threshold limit (being an english word), it is said to proper english word. In brief, this logic is famous by Markov chains.
Link
For Mathematics of Gibberish and better understanding, refer to video https://www.youtube.com/watch?v=l15C8UJu17s . Thanks !!
If the search is performed on products, you could cache their names or codes and check them against that list before quering database. Else, if your site is for english users, you can build a dictionary of strings that aren't used in the english language, like qwkfagsd. Which, and agreeing with other answer, will be more resource intensive than if not there.
I am using this code to create an instant search for my site...
http://woorkup.com/2010/09/13/how-to-create-your-own-instant-search/
Some of the phrases in our database our very complex and can be easily misspelt so on top of this I wanted to use spelling suggestions.
Does anyone know of any ways to offer correct spellings based on a string provided?
Any help would be greatly appreciated.
Yes there is a jQuery plugin called After the Deadline.
If someone searches for a phrase, doesn't click any of the results, and then researches with a new similar phrase (check out levenshtein()) and does click a result, write to your database the original phrase and the new phrase.
Record each time this happens. If the phrase is already matched, increment a counter for that phrase.
Then, if someone searches for a phrase that matches one of your possibly incorrect phrases (perhaps have a threshold using your counter), you can display a Did you mean to search for...? as well as the results (if any) for the incorrect phrase.
This isn't a spell check per se, but I think it would be useful to pick up on common mistakes. Unfortunately though, you probably don't have as many people to help you build an index like Google's Did you mean?
Peter Norvig has written (and explained) a fairly basic spelling corrector; which makes for a very interesting read. It's in Python, but his explanations are invaluable (He does work for Google and this is a very bare bones representation of the Google did you mean algorithm).