I have set up PHP-FPM with nginx for a rather spread-out API. Everything works like charm, until it doesn't. Most often, I observe errors to the tune of
2015/12/21 12:34:43 [error] 14113#0: *60938148 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 1.39.8.13, server: api.hlapp.com, request: "POST /v2/hlapp/get-fun HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "api.hlapp.com"
And the problem is that I am unable to pin-point the section of code where it times out. The API call communicates with several other things on the network, failure or latency in any of which may cause this timeout. What is the best way to go about diagnosing this problem? Can we somehow have more verbose failure logging, something like in Java?
Also, I must add, I have done all sort of things like profiling code, etc. but the failure to connect with micro-components is erratic, and not caught in usual profiling. It mostly happens in production.
So, to sum it all up, all I'm looking for is a more verbose stacktrace, which tells me exactly where the code timed out.
After much head-banging, I did come up with a solution. This is inspired by Java's programming paradigm, where almost every well-implemented latency generating endpoint is either a throwable or calls a throwable, and every known exception is caught and handled.
So, the best possible solution to this could be as thus. For every request made, identify the communication channels, be it a DB query, a secondary API response, or any other thing which might result in PHP to reach its max_execution_timeout limit, surround that in a try catch, have them set to a time limit after which they "time out". This gradually helped in pinpointing the cause of timeout (which was Redis being unreachable in my case, and so I set a limit of 5 seconds to Redis connector object).
Hope this helps someone.