I have 30 android clients that send xml files to the server every one hour.I need to read each file which is pretty long(over 1700 lines per file) but i am not so sure how reliable is for this kind of tasks.
Should i continue reading with php or is C++ or java program better suited.
With PHP you can use 2 good library: SimpleXML for simple functions or DOM for something more complexe.
You can have a idea of what PHP can do with XML files here: https://www.php.net/manual/en/refs.xml.php
If you know the names of nodes you need to read, SimpleXML is the simple way to manage yours datas. You actually can do what you want with XML files on PHP.
Related
I've got a very large XML file (1.5GB) that I need to parse and then insert specific values into a MySQL table.
Now the way I would have usually done parsing on a DOM would be to use jQuery or PHP Simple Dom Parser but in this situation, given the file size, I don't think either are suitable. I need the emphasis to be on performance. I've read a little about SimpleXML and XML Parser for PHP and it seems each have their advantages but I'm not sure if either of these are suitable for a file with a size of 1.5GB.
I've also seen Pear's XML parser mentioned but, again, I don't know if this is suitable in this situation. From what I've read it seems that I need to load into memory only the required nodes and not the whole tree itself. Even now i'm having trouble actually viewing the document due to the size. VIM seems to be the only editor that can handle it but even then scrolling through the document can cause a crash.
If anyone can recommend one of these above the other, or even an entirely different solution that would be great.
That would then bring me to my SQL inserts which I was going to do on the fly - so after i've parse a node and pulled the values I require I will insert these into the database. Again, any advice would be great.
For such huge XML file its recommended to use SAX based XML parsers. In PHP you can do it with "XML Parser". It consumes less memory than its peers. Also its very fast.
SimpleXml and DOM are not meant for big XML files
try:
XMLReader: http://php.net/manual/en/book.xmlreader.php
or even better/faster (but slightly more complicate to use)
XMLParser: http://php.net/manual/en/book.xml.php
This question already has answers here:
Closed 11 years ago.
Possible Duplicates:
PHP what is the best approach to using XML? Need to create and parse XML responses
Parse big XML in PHP
Hello Community,
I am writing an application, that requires to parse XML files, that can minimum of 1000 MBs or more in size.
I have tried with few code that is available on internet. As file size is more, it's easy to understand that file will have lots and lots of XML tags. So, loop performance gets weak as time elapse.
So, I would need a parser:
-> Performance is considerably good as time passes, when doing execution / parsing
-> Doesn't load the whole XML file in memory
I know about following XML parsers, but not sure which to use and why?
XML Parser
SimpleXML
XMLReader
I am using PHP 5.3, so please help me guys and gals, to choose the parser.
You can even suggest me some other options, or classes.
Thanks.
EDIT
I even want to know about SAX (Simple API for XML) and StAX implementation of PHP
First of all, you can't load that much XML in memory. It depends on your machine, but if your XML file is more than 10-20 MB it generally is too much. The server may be able to handle more, but it's not a good idea to fill all the memory with one script. So you can rule out SimpleXML and DOM from the start.
The other two options, XML Parser and XMLReader, will both be good, with XMLReader being a newer extension, so probably better. But as a warning you should take notice that XMLReader also allows you to load everything in memory. Don't do that. Instead use it as a node-by-node parser and read/process your data in small bits.
You problem may go beyond the scope of choosing a parser if you need most of the data from the XML. You should also make sure that you don't load it all up in memory and use it at the end of the script. Instead use it as you get it and dispose of it once you no longer need it.
Load your giant XML files into an XML database and perform your query and manipulations through their XQuery/XSLT interfaces.
http://www.xml.com/pub/a/2003/10/22/embed.html
Background:
So i'm writing a web-service style web application as a way to increase my knowledge of how PHP and XML work together. I want to eventually take that XML data and use it in a mobile phone application but that's a different issue. I can connect to the data, pull, and process all the information with PHP and I've managed to get it exporting to CSV. I want to now begin to push that data out in XML.
Question:
What is the (a) recommended way to work with XML in PHP?
References:
PHP Manual, XML Portion
I suggest using simple XML which is way easier to handle xml operations.
I'm writing a simple cms for a client and for the umpteenth time can't decide the best strategy going in.
My options, as far as I can tell, are:
1) Store my XML data as a flat file that is referenced by my actionscript. I could manipulate this file with a php "back end".
2) Have my actionscript call a PHP script that would output the XML, bypassing the interaction between actionscript and the physical xml doc. (eliminating some caching issues).
3) Create flat .txt files (similar to tables) that php reads from and outputs XML to my actionscript call.
I'm up in the air about this because I'm not big on XML being my physical data. One little missing ">" and your whole file takes a dump. At least with option 3, one file might be corrupt, but it wouldn't bring down all of your data. Then again, I haven't made much use of PHP's built in XML classes, so could be just adding extra work to my project.
Anyone have a rock solid method for storing data when you don't have access to a Database?
Thanks in advance,
-J
I would personally take the hit on the write and use a tool (e.g. CMS) to create the XML file. You can manage the validation, creation, and manipulation within your language of choice.
I have a large XML file (600mb+) and am developing a PHP application which needs to query this file.
My initial approach was to extract all the data from the file and insert it into a MySQL database - then query it that way. The only issue with this was that it was still slow, plus the XML data gets updated regularly - meaning I need to download, parse and insert data from the XML file into the database everytime the XML file is updated.
Is it actually possible to query a 600mb file? (for example, searching for records where TITLE="something here"?) Is it possible to get it to do this in a reasonable amount of time?
Ideally would like to do this in PHP, though I could also use JavaScript too.
Any help and suggestions appreciated :)
Constructing an XML DOM for a 600+ Mb document is definitely a way to fail. What you need is SAX-based API. SAX, though, does not usually allow XPath to be used, but you can emulate it with imperative code.
As for the file being updated, is it possible to retrieve only differences anyhow? That would massively speed up subsequent processing.