在插入之前,在每个CSV行上插入带有MySQL检查的CSV记录

I'm uploading a CSV file of records to add to a MySQL database. With each row in the CSV file I need to check another MySQL table in the same database to see that the data we're entering is valid. This check is just a simple SELECT x FROM blah WHERE primery_key = value but with 5-10,000 records in a CSV file, it takes a long time to run through them all.

I'd like to try and make this faster, but don't know what the best options is, as none of them seem too great so far...

  1. Some kind of SELECT/INSERT FROM database WHERE IN (array created from CSV file row values)
  2. Add all the values into a new 'temporary' table, then delete the ones that don't exist in the other table via MySQL then add all the remaining records to the final table.
  3. Something completely different that you fine people will enlighten me on... hopefully?

The CSV file will probably be 5000 records tops but may be as much as 10,000 in future. The table the records need to be checked against is at around 250,000 records.

Edit to add some code:

function processCSV($file)
    {
        if(empty($file)){
            $this->default_error[] = 'No data found';
        }

        $lines=file($file);

        foreach($lines as $line){
            $row_count++;
            $pieces = str_getcsv($line, ",");
            simple SELECT id1 FROM my_table WHERE id2 = . $pieces[0]
            if(row exists){
                puts value into array to batch process inserts at the end using array_chunk()
            }
        }
   }

It's not actual code as it's called from different classes, but I'm guessing all needed is there? The main issue is a MySQL check on a huge for loop, but let me know if you want more specifics.