I want to get the filepath of all files in a specific directory. The problem is that the directory is very big (about 400GB). I tried to make recursive calls to read all directories and subdirectories and it works fine - but it is slow. I got the idea that I call »find ...« with exec. When using the shell I can obtain an array of all files but I only have access to that array when the process finished. It would be great to have something to get the results while searching so I can store it in my database and other programms have access to it. I thougt of calling a function for every (or every n) results. Is something like that possible and how would you do that?
Did I read correctly that you are using exec to find all of the files?
A better way to do this would be using php its self, with some of the available features like the DirectoryIterator and RecursiveDirectorIterator.
Personally, I think that using these is a better idea than using exec to run a command. That way you are not passing the handling off to another process when you can just do it within php.
I am unsure on being able to get usable data while you are creating the array though. You could possibly store each value of the array into a database, that way, each time a value is added to the array, the database is updated. You could then use the information from the database while the script is still running.
To get the results while searching, use popen — Opens process file pointer — instead of exec — Execute an external program.