I have a RSS feed with more than 5000 items in it. In the user interface, I'm trying to have search feature with ability to do custom search based on different categories. First when the page loads I'm just showing the first 10 feeds which loads really quick as its supposed to be, but when we enter a string to search with a category selected, the processing is pretty slow. I want to know if there is a way to do this more efficiently than going through each and every feed item every single time.
I'm not adding any code here because I'm looking for ideas for handling/searching such large rss feeds. So far I have been using PHP (simple XML) and JavaScript.
RSS (and XML in general) are great data transport formats. They are not good formats for accessing that data via random access.
Import the feeds into a database (properly, don't just dump the raw XML in there) such as Postgresql or MySQL and use a full text search provided by the database server.
You can use a session variable to store all the feeds. Also in the background have a polling script which checks for new feed. If you get one, add it to the session. Use the session variable to search the feed.
Don't use SimpleXML for this. (In fact, it really shouldn't be used at all). Rather, use the DOMDocument class to parse through your XML.