eXtensible Markup Language is referred to as XML. Its purpose was to transport and store data. It was made to be readable by both humans and machines. Because of this, the design objectives of XML place an emphasis on simplicity, generality, and Internet usability.
In this tutorial, an XML file that needs to be parsed is actually an RSS feed.
Rich Site Summary (RSS), also known as Really Simple Syndication, is a web feed format family that is used to publish frequently updated information such as blog entries, news headlines, audio, and video. RSS is plain text with an XML format.
- The RSS format itself is comparatively simple for both automated and human readers.
- The top news stories RSS feed from a well-known news website is the RSS that is processed in this tutorial. Check it out right here. Our objective is to transform this XML file into another format and store it for later use after processing the RSS feed.
The code above will:
- Open the specified URL to load the RSS feed, then save it as an XML file.
- To save news as a list of dictionaries, where each dictionary represents a single news item, parse the XML file.
- News items should be saved as a CSV file.
Let us try to understand the code in pieces:
Loading and saving RSS feed
Here, we started off by making an HTTP response object by making an HTTP request to the RSS feed's URL. The XML file data is now contained in the response's content, and we save it as topnewsfeed.xml in our local directory.
To parse an XML file, we created the parseXML() function. We are aware that XML is a naturally hierarchical data format, and that a tree is the most appropriate visual representation of it.
Here, we're utilising the xml.etree.ElementTree (abbreviated ET) module. For this purpose, Element Tree has two classes: Element and ElementTree, where Element represents a single node in the tree and ElementTree represents the entire XML document as a tree. Reading and writing to/from files as well as interactions with the entire document are typically done at the ElementTree level. The Element level is where interactions with a single XML element and its children take place.
Ok, so let’s go through the parseXML() function now:
Saving Data to a CSV File
Now all that is left to do is use the savetoCSV() function to save the list of news items to a CSV file so that it can be easily used or modified in the future. To learn more about adding dictionary words to CSV files, click here.
To store all news stories as a table, the data from the hierarchical XML file has been transformed into a straightforward CSV file. This also makes expanding the database simpler.
Additionally, the JSON-like data can be used directly in applications! The best alternative for data extraction from websites that don't offer a public API but do offer some RSS feeds is this.