I'm interested in scrapping data from a website, and pass the data for use it in PHP. I have had a look around, and the best suggestion I have been able to find is to first serialise the Python data and then pass it along.
The issue I have is that I'm unsure how to serialise the Python data.
I'm using Selenium, and I have the following code.
test = browser.find_elements_by_css_selector("table#resultstable td")
I can see the data I want to use by running the variable through a loop and printing it.
for val in test:
print(val.text)
However when I try to serialise the object, I receive the following error:
json.dumps(test)
....
TypeError: Object of type 'WebElement' is not JSON serializable!
I Hope somebody can point me in the right direction, I'm happy in PHP but I have only begun to look Python recently.
JSON the data format is limited in what it can store (numbers, strings, booleans, and then arrays or maps (what python calls dictionaries) of these elements (or other arrays or maps which at some point end in one of those formats). json the python library for manipulating json data can therefore only dump to a json string something that fits those rules. In your case, I'd suggest as a simple starting point:
columns = [val.text for val in test] # convert to a list of strings, where each string is the td.text
json.dumps(columns) should then give you something useful
Related
The goal is to unserialize a PHP serialized string and get sutable object in C#
Is there any way to make this possible in C#(.Net)?
To be more specific:
We need to make an application which comunicates (Via HTTP) to specific website which returns the needed information. Fortunately/unfortunately we dont have permission to website so the data (array mostly) that is returned from website is PHP serialized.
I suppose using JSON as an intermediary step could be useful.
You should probably write it to XML or JSON. You can construct your C# object back from the XML
Edit: Looks like there is already a XML serializer for PHP
Have an old project that someone did in PHP and am attempting to convert it over to Rails. Am encountering an issue when trying to iterate through a multiple array in rails.
The array as stored in the DB looks like this:
a:5:{i:0;s:8:"Director";i:1;s:11:"Shareholder";i:2;s:14:"Vice President";i:3;s:9:"Secretary";i:4;s:9:"Treasurer";}
Am trying to display the String values such as "Director" and "Shareholder".
Does the DB field need to be changed to a different format to work?
How would this be done in rails?
Thank you in advance
That looks like the output of PHP's serialize function to me.
If you have PHP around or don't mind installing it to help with the data migration, then I'd write a short PHP script to convert those columns to JSON using PHP's unserialize and json_encode functions: read a value, unserialize it, json_encode the result, write it back into the database (all in PHP).
After that, you should be able to use ActiveRecord's serialize with JSON as the storage format:
class Model < ActiveRecord::Base
serialize :whatever_it_is_called, JSON
end
You could also use YAML (the default for ActiveRecord's serialize) if you had YAML support in PHP-land.
If you don't want to touch PHP at all, then you'll have to write Ruby parser for PHP's serialize format (which seems to be well documented in the PHP docs) and hook that up to ActiveRecord's serialize.
You will of course be working with a copy of the real database for all this work.
I've used Google, Yahoo, AND Bing, but I can't find any good answers. I've seen jLinq, but I want to be able to query JSON in PHP in hopes of having a not an SQL database, but instead all data storage within the filesystem on my server. No, I don't care how bad it sounds.
Ideas nonetheless? I would think that there would be a PHP class on this.
-----EDIT-----
Guys, thanks for your answers so far, but I don't think that json_encode and json_decode are of much use. What I want to be able to do is encode/decode JSON, and be able to search it for specific keys with specific values. Albeit I have PROVEN to myself that I can do so, it's a lot of code for something that should be so simple. Anything else you have in mind?
You can use JsonQ package. This package can query over JSON data like Query Builder.
https://github.com/nahid/jsonq
You can call json_encode to convert your data to a string, and write this string to a file. Then when you want to use it, you read the entire file and call json_decode to convert it back to data. When you're done processing it, repeat the encode/write steps.
But if you have multiple processes doing this, they'll completely overwrite what each other is doing. So it's not a very good way to manage shared data.
You could try JSONPath it allows you to query json with xpath as you would xml.
Look into json_decode. This lets you take JSON as a string and parse it into an object in PHP. You could theoretically store the strings as files, and use file_get_contents to retrieve the string from the file.
You would have to write your own searching/indexing/updating/etc algorithms, but if you don't want to use a real database solution (since you said, No, I don't care how bad it sounds.), then I guess this would work.
To search for values in the object you get from json_decode, look into in_array and array_search
I just started doing jQuery last week, and so far I already made some basic systems with ajax, like basic jQuery CRUD and simple chat system without referencing on other's work for I decided to test myself on how far I can do systems alone in jQuery(without JSON and XML yet).
But when I decided to look at other's work (hoping to get/learn good practices and codes out there) many or almost every program that deals with ajax have some JSON in it. So I decided to study and read JSON specially this one, but I guess because it's my first time dealing with it, I'm having a problem sinking it into my brain. Yeah I know it is a "lightweight way of describing hierarchical data", I also know how to make JSON like mixing a literal array and object in JS, and how to dsplay it in js.
But my question is, what's the difference and what's the advantage than not using it?
When I can still get and store data on the server using ajax and database without JSON.
By the way I haven't focus on XML yet because based from my research it's better to use JSON in AJAX.
Can you give me some actual scenario dealing with
s1. ajax php mysql (this with what disadvantages?)
and
s2. ajax php mysql json (this with what advantages?)
I mean, my focus is to send and get data, and I already can do it with s1.
Sorry if you find my question stupid. Tia. :)
Why use JSON? The answer is portability and structure.
JSON is portable because parsers and writers are available for many, many languages. This means that JSON that a PHP script generates can be very easily understood by a JavaScript script. It is the best way to transmit complex structures like arrays and objects, and have it still be compatible with multiple languages.
JSON provides structure because the data you transmit with it can have consistent formatting. This is instead of transmitting back plain-text (i.e. unformatted) data, like comma-separated or delimited data.
Data that is merely delimited (for example, "BookName1,BookName2,BookName3") is more difficult for humans to understand, debug, and work with. If you wanted to debug a response between your server and your browser and the data was delimited (like my example above), you might have a hard time understanding it. Also, if you want to add different data types, provide separate records, etc., then your custom data format becomes more complicated. Eventually, you might end up reinventing JSON.
As a side note, JSON is indeed better than XML. It is much more efficient space-wise. There are no tag names to take up space. Structure is created via nested braces, instead of verbose tags.
Resources
Here is an interesting article on the differences and pros/cons of XML and JSON: http://www.json.org/xml.html
Examples
Per your request, here is an example of encoding JSON with PHP. This is ripped from the docs:
$arr = array ('a'=>1,'b'=>2,'c'=>3,'d'=>4,'e'=>5);
echo json_encode($arr);
Output:
{"a":1,"b":2,"c":3,"d":4,"e":5}
Contrast this to something like this, without JSON:
a,1
b,2
c,3
d,4
e,5
To parse that, you'd have to iterate through each line, split the values yourself, and then create the array. This isn't that difficult, but imagine you have a nested object:
$arr = array ('a'=> array(1,2,3),'b'=> array('a' => 1, 'b' => 2),'c'=>3,'d'=> array(1,2,3,4,5) ,'e'=>5); // etc.
With JSON, it's no different to encode it. Just use json_encode. But, encoding this manually, and then decoding it manually would be significantly more work.
Programming in any sort of programming language, you have several different types of data at your disposal, including the very useful array type.
Interchanging data between Javascript and any server side language can only happen through strings. I.e. you can send and return any text, but there's no way to send a native array or number type.
JSON is an elegant way to express array and other types using only a string. This way you can pass arbitrary data back and forth between different environments and are not limited to pure text. XML solves the same kind of problem, but is often overkill for simple AJAX requests.
I want to convert an Object into a String in PHP. Specifically, I'm trying to take a mysql query response, and I'm trying to convert it into something I can write to a file and use later.
Of course, when you try to write an Object to a file, PHP appropriately yells: Catchable fatal error: Object of class DB_result could not be converted to string in .....
Alternatively, if there is some other way of writing the result of a mysql query to a file, that works too. I'm playing around with a home-brewed caching project :)
Maybe serializing? It will take an object/array and convert it to a string (which can then be un-serialized back later)
json_encode and json_decode will also accomplish many of the properties you are looking for via serialize. The advantage is that you can send JSON-encoded data to a web browser and JavaScript can view and modify properties like a native JavaScript object. In addition, JSON is lighter weight than serialized data because its syntax is a lot more compact.