I'm in the process of building a custom Collection class but I'm having some issues on filtering data.
I have a collection of financial data, ($collection) that looks like this:
App\CustomCollection {
#items: array [
0 => {
+"financials": [
0 => {
+"data_tag": {
+"tag": "revenue"
}
+"value": 1200
}
]
+"fundamental": {
+"fiscal_year": 2018
+"fiscal_period": "FY"
+"start_date": "2018-01-01"
+"end_date": "2018-12-31"
}
+"company": {#336
+"id": "com_TEST1"
}
}
1 => {
+"financials": [
0 => {
+"data_tag": {
+"tag": "revenue"
}
+"value": 300
}
]
+"fundamental": {
+"fiscal_year": 2018
+"fiscal_period": "Q1"
+"end_date": "2018-03-31"
}
+"company": {#336
+"name": "Test Company Inc"
}
}
2 => {
+"financials": [
0 => {
+"data_tag": {
+"tag": "revenue"
}
+"value": 300
}
]
+"fundamental": {
+"fiscal_year": 2018
+"fiscal_period": "Q2"
+"end_date": "2018-06-30"
}
+"company": {#336
+"name": "Test Company Inc"
}
}
... etc to Q4
]
}
I can successfully return my Fiscal Years with the following:
$fiscalYears = $collection->where('fundamental.fiscal_period', 'FY');
Returns:
App\CustomCollection {
#items: array [
0 => {
+"financials": [
0 => {
+"data_tag": {
+"tag": "revenue"
}
+"value": 1200
}
]
+"fundamental": {
+"fiscal_year": 2018
+"fiscal_period": "FY"
+"start_date": "2018-01-01"
+"end_date": "2018-12-31"
}
+"company": {#336
+"id": "com_TEST1"
}
}
]
}
I can also return just the quarters by doing:
$fiscalYears = $collection->where('fundamental.fiscal_period', '!=', 'FY');
App\CustomCollection {
#items: array [
0 => {
+"financials": [
0 => {
+"data_tag": {
+"tag": "revenue"
}
+"value": 300
}
]
+"fundamental": {
+"fiscal_year": 2018
+"fiscal_period": "Q1"
+"end_date": "2018-03-31"
}
+"company": {#336
+"name": "Test Company Inc"
}
}
1 => {
+"financials": [
0 => {
+"data_tag": {
+"tag": "revenue"
}
+"value": 300
}
]
+"fundamental": {
+"fiscal_year": 2018
+"fiscal_period": "Q2"
+"end_date": "2018-06-30"
}
+"company": {#336
+"name": "Test Company Inc"
}
}
... etc to Q4
]
}
Everything is great here and works fine, but now what I am trying to accomplish is to return only quarters that match specific values in $fiscalYears.
Here is what I am using now:
public function quartersByFiscalYears($fiscalYears)
{
$quarters = [];
foreach ($fiscalYears as $fiscalYear) {
$quarters[] = $this->quarters()->where('company', $fiscalYear->company)
->where('fundamental.end_date', '>=', $fiscalYear->fundamental->start_date)
->where('fundamental.end_date', '<', $fiscalYear->fundamental->end_date)
->values('financials');
}
return $quarters;
}
The most important part of the code above is that it returns only quarters where the $quarter end_date >= the $fiscalYear start_date and where the $quarter end_date < the $fiscalYear end_date.
This works, but it is by far the slowest "filter" in my collection. I have a feeling that I am thinking about this all wrong, although I'm not sure. It seems like looping over the entire collection every time $fiscalYears iterates is a bad idea. Can this be done in a faster/more efficient way? Or is foreach pretty common in this scenario?
It looks like each "group" in your resulting data contains the quarters for a particular company in a given year. You can use eloquent methods to get this done MUCH faster than looping over the quarters. I tested this with 40k records. Looping took ~45 secs, but this takes less than a second.
$quarters = $collection->where('fundamental.fiscal_period', '!=', 'FY');
// use `groupBy()` to organize the quarters into the right chunks
$grouped = $quarters->groupBy(function($item) {
// concatenate the company name and fiscal year to create the "group" name
return $item->company->name.'_'.$item->fundamental->fiscal_year;
});
// use `$fiscalYears` with `map()` and `flip()` to create a collection
// with the same key structure as $grouped
$filter = $fiscalYears->map(function($item) {
return $item->company->name.'_'.$item->fundamental->fiscal_year;
})->flip();
// filter the collection with `intersectByKeys()`
$result = $grouped->intersectByKeys($filter);
// If you want, replace the keys with numeric indices and convert to array
$result = $result->values()->toArray();
Just be careful about which eloquent methods you use. If you use flatten()
, that method uses recursive loops under the hood, so it will still be slow.