So i finally have setup the elasticsearch database and imported data into it.
Sometimes when i try to request data from frontend, i get 500 error( not all the time, just sometimes ).
I tried to request data from POSTMAN( to see the ES error message ).
I got:
{
"error": "SearchPhaseExecutionException[Failed to execute phase [query], all shards failed; shardFailures {[9m4uVcf3TLmQ9Kr7z_fSpQ][text][0]: QueryPhaseExecutionException[[text][0]: query[filtered(function score (blended(terms: [url_words:test, domain_words:test, title:test, body:test]), functions: [{filter(*:*), function [org.elasticsearch.common.lucene.search.function.FieldValueFactorFunction@56319fc9]}{filter(*:*), function [org.elasticsearch.common.lucene.search.function.FieldValueFactorFunction@60b46f02]}]))->cache(_type:page)],from[0],size[25]: Query Failed [Failed to execute main query]]; nested: ElasticsearchException[Missing value for field [lang_en]]; }{[9m4uVcf3TLmQ9Kr7z_fSpQ][text][1]: QueryPhaseExecutionException[[text][1]: query[filtered(function score (blended(terms: [url_words:test, domain_words:test, title:test, body:test]), functions: [{filter(*:*), function [org.elasticsearch.common.lucene.search.function.FieldValueFactorFunction@3ca7d41e]}{filter(*:*), function [org.elasticsearch.common.lucene.search.function.FieldValueFactorFunction@63daf999]}]))->cache(_type:page)],from[0],size[25]: Query Failed [Failed to execute main query]]; nested: ElasticsearchException[Missing value for field [lang_en]]; }{[9m4uVcf3TLmQ9Kr7z_fSpQ][text][2]: QueryPhaseExecutionException[[text][2]: query[filtered(function score (blended(terms: [url_words:test, domain_words:test, title:test, body:test]), functions: [{filter(*:*), function [org.elasticsearch.common.lucene.search.function.FieldValueFactorFunction@27521539]}{filter(*:*), function [org.elasticsearch.common.lucene.search.function.FieldValueFactorFunction@66dbac2b]}]))->cache(_type:page)],from[0],size[25]: Query Failed [Failed to execute main query]]; nested: ElasticsearchException[Missing value for field [lang_en]]; }{[9m4uVcf3TLmQ9Kr7z_fSpQ][text][3]: QueryPhaseExecutionException[[text][3]: query[filtered(function score (blended(terms: [url_words:test, domain_words:test, title:test, body:test]), functions: [{filter(*:*), function [org.elasticsearch.common.lucene.search.function.FieldValueFactorFunction@73bb4f5e]}{filter(*:*), function [org.elasticsearch.common.lucene.search.function.FieldValueFactorFunction@112dcf1c]}]))->cache(_type:page)],from[0],size[25]: Query Failed [Failed to execute main query]]; nested: ElasticsearchException[Missing value for field [lang_en]]; }{[9m4uVcf3TLmQ9Kr7z_fSpQ][text][4]: QueryPhaseExecutionException[[text][4]: query[filtered(function score (blended(terms: [url_words:test, domain_words:test, title:test, body:test]), functions: [{filter(*:*), function [org.elasticsearch.common.lucene.search.function.FieldValueFactorFunction@b650549]}{filter(*:*), function [org.elasticsearch.common.lucene.search.function.FieldValueFactorFunction@7fbe90f4]}]))->cache(_type:page)],from[0],size[25]: Query Failed [Failed to execute main query]]; nested: ElasticsearchException[Missing value for field [lang_en]]; }]",
"status": 500
}
Here is the request body:
{
"query": {
"function_score": {
"query": {
"multi_match": {
"query": "test",
"minimum_should_match": "-25%",
"type": "cross_fields",
"tie_breaker": 0.5,
"fields": ["title^3", "body", "url_words^2", "domain_words^8"]
}
},
"functions": [{
"field_value_factor": {
"field": "rank",
"factor": 1
}
},{
"field_value_factor": {
"field": "lang_en"
}
}]
}
},
"from": 0,
"size": 25
}
I understand that Missing value for field [lang_en] is the problem. I toyed around es with google results, but without success.
ES version: 1.5.2
Any ideas ?
EDIT:
I added "missing": 0, to second field_value_factor, but i got this error instead:
{
"error": "SearchPhaseExecutionException[Failed to execute phase [query], all shards failed; shardFailures {[9m4uVcf3TLmQ9Kr7z_fSpQ][text][0]: SearchParseException[[text][0]: from[-1],size[-1]: Parse Failure [Failed to parse source [{
\"query\": {
\"function_score\": {
\"query\": {
\"multi_match\": {
\"query\": \"test\",
\"minimum_should_match\": \"-25%\",
\"type\": \"cross_fields\",
\"tie_breaker\": 0.5,
\"fields\": [\"title^3\", \"body\", \"url_words^2\", \"domain_words^8\"]
}
},
\"functions\": [{
\"field_value_factor\": {
\"field\": \"rank\",
\"factor\": 1
}
},{
\"field_value_factor\": {
\"field\": \"lang_en\",
\"missing\": 0
}
}]
}
},
\"from\": 0,
\"size\": 25
}
]]]; nested: QueryParsingException[[text] field_value_factor query does not support [missing]]; }{[9m4uVcf3TLmQ9Kr7z_fSpQ][text][1]: SearchParseException[[text][1]: from[-1],size[-1]: Parse Failure [Failed to parse source [{
\"query\": {
\"function_score\": {
\"query\": {
\"multi_match\": {
\"query\": \"test\",
\"minimum_should_match\": \"-25%\",
\"type\": \"cross_fields\",
\"tie_breaker\": 0.5,
\"fields\": [\"title^3\", \"body\", \"url_words^2\", \"domain_words^8\"]
}
},
\"functions\": [{
\"field_value_factor\": {
\"field\": \"rank\",
\"factor\": 1
}
},{
\"field_value_factor\": {
\"field\": \"lang_en\",
\"missing\": 0
}
}]
}
},
\"from\": 0,
\"size\": 25
}
]]]; nested: QueryParsingException[[text] field_value_factor query does not support [missing]]; }{[9m4uVcf3TLmQ9Kr7z_fSpQ][text][2]: SearchParseException[[text][2]: from[-1],size[-1]: Parse Failure [Failed to parse source [{
\"query\": {
\"function_score\": {
\"query\": {
\"multi_match\": {
\"query\": \"test\",
\"minimum_should_match\": \"-25%\",
\"type\": \"cross_fields\",
\"tie_breaker\": 0.5,
\"fields\": [\"title^3\", \"body\", \"url_words^2\", \"domain_words^8\"]
}
},
\"functions\": [{
\"field_value_factor\": {
\"field\": \"rank\",
\"factor\": 1
}
},{
\"field_value_factor\": {
\"field\": \"lang_en\",
\"missing\": 0
}
}]
}
},
\"from\": 0,
\"size\": 25
}
]]]; nested: QueryParsingException[[text] field_value_factor query does not support [missing]]; }{[9m4uVcf3TLmQ9Kr7z_fSpQ][text][3]: SearchParseException[[text][3]: from[-1],size[-1]: Parse Failure [Failed to parse source [{
\"query\": {
\"function_score\": {
\"query\": {
\"multi_match\": {
\"query\": \"test\",
\"minimum_should_match\": \"-25%\",
\"type\": \"cross_fields\",
\"tie_breaker\": 0.5,
\"fields\": [\"title^3\", \"body\", \"url_words^2\", \"domain_words^8\"]
}
},
\"functions\": [{
\"field_value_factor\": {
\"field\": \"rank\",
\"factor\": 1
}
},{
\"field_value_factor\": {
\"field\": \"lang_en\",
\"missing\": 0
}
}]
}
},
\"from\": 0,
\"size\": 25
}
]]]; nested: QueryParsingException[[text] field_value_factor query does not support [missing]]; }{[9m4uVcf3TLmQ9Kr7z_fSpQ][text][4]: SearchParseException[[text][4]: from[-1],size[-1]: Parse Failure [Failed to parse source [{
\"query\": {
\"function_score\": {
\"query\": {
\"multi_match\": {
\"query\": \"test\",
\"minimum_should_match\": \"-25%\",
\"type\": \"cross_fields\",
\"tie_breaker\": 0.5,
\"fields\": [\"title^3\", \"body\", \"url_words^2\", \"domain_words^8\"]
}
},
\"functions\": [{
\"field_value_factor\": {
\"field\": \"rank\",
\"factor\": 1
}
},{
\"field_value_factor\": {
\"field\": \"lang_en\",
\"missing\": 0
}
}]
}
},
\"from\": 0,
\"size\": 25
}
]]]; nested: QueryParsingException[[text] field_value_factor query does not support [missing]]; }]",
"status": 400
}
The problem was with the dumb AWS ES version 1.5.2.
My solution: Create EC2 instance and deploy Elasticsearch manualy.
In at least one document, the field lang_en is null, empty or simply non-existent.
You need to modify your field_value_factor function in order to tell it what to do in such as case, by using the missing setting with whatever default value makes sense (0, 1, etc):
{
"field_value_factor": {
"field": "lang_en",
"missing": 1 <---- add this line
}
}