I have to write a program which checks if a particular directory on my folder has any files (of a specific extension), and if it finds any files, it reads them one by one and loads data from them into a database.
This is the rough algorithm in my mind:
Using an infinite while() loop, continuously keep checking if the directory has any files of that particular extension (e.g. check if the directory has any *.xml files). I can use the PHP glob() function.
If yes, then in a foreach loop, read data from each file and load it into the database.
Once a file's data has been loaded, delete it.
My Question:
I will be constantly checking if there any .xml files in the directory. This means that many times I will get a true (meaning/saying "Yes, there are .xml files in the directory") even for the files whose data is BEING loaded.
So once a file has been found in the directory, I need a check which checks if its data is in the process of being loaded into a database. How do I check that?
The process of data-loading is that I extract useful data from the file into a .csv file and then use LOAD DATA INFILE SQL query to load the data into my MySQL database.
One solution is to use inotifywait as suggested in this answer: https://stackoverflow.com/a/6767891/2032943 to watch event and then act on them.
Also if you want to see that the file is already being used by some other command, you can use linux lsof command to check if there is an open handle for the file used by some process:
lsof | grep <filename>
Note that these commands are specific to linux and will not work on windows.