Is it more efficient to do multiple single queries in a php loop or to keep everything in an array and build one bigger query for elastic?
Option 1:
foreach ($rowset as $row) {
//execute query:
"query": {
"bool" : {
"must" : [
{"term" : { "a" : "'.$row['a'].'" }},
{"term" : { "b" : "'.$row['b'].'" }}
]
}
}
}
Option 2:
$search = "";
foreach ($rowset as $key => $row) {
if($key > 0) {
$search .= ',';
}
$search .= '"must" : [
{"term" : { "a" : "'.$row['a'].'" }},
{"term" : { "b" : "'.$row['b'].'" }}
]'
}
"query": {
"bool" : {
"should" : [
"must" : [
'.$search.'
]
]
}
}
The syntax may be incorrect in this minimal example but I hope the idea becomes clear. I would expect Option 2 to be faster. I didn't test it yet since I have multiple nested loops right now and wanted an optinion before I rewrite my code.
External resources requests are always slow. So it's better to create single request. External DB will process it with same speed, but you will save requests time.
In the other hand you can get such big query, that your client (e.g. browser) will timeout or you can't put everything to memory for processing.
Use single query with reasonable amount of data.
Single request using filter is the best option: https://www.elastic.co/guide/en/elasticsearch/reference/current/query-filter-context.html