php-curl没有连接到远程主机，标题似乎是空的

I configured one of my server to accept files via post, convert them and send them as response. A php server uses this functionality via curl.

The important part in my php script looks like this

$ch = curl_init();

curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_PORT, $port);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, array('file_contents' => '@' . $filename));
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 5);
curl_setopt($ch, CURLINFO_HEADER_OUT, true);

file_put_contents($filename . '.png', curl_exec($ch));

print_r(curl_getinfo($ch));
echo curl_errno($ch);

curl_close($ch);

However it appears as if curl doesn't even connect to the other server and curl_getinfo only returns

Array
(
    [url] => (redacted)
    [content_type] => 
    [http_code] => 0
    [header_size] => 0
    [request_size] => 0
    [filetime] => -1
    [ssl_verify_result] => 0
    [redirect_count] => 0
    [total_time] => 0
    [namelookup_time] => 0.036982
    [connect_time] => 0
    [pretransfer_time] => 0
    [size_upload] => 0
    [size_download] => 0
    [speed_download] => 0
    [speed_upload] => 0
    [download_content_length] => -1
    [upload_content_length] => -1
    [starttransfer_time] => 0
    [redirect_time] => 0
    [redirect_url] => 
    [primary_ip] => (redacted)
    [certinfo] => Array
        (
        )
)

curl_errno returns 28 (CURLE_OPERATION_TIMEDOUT) even if I increase the timeout. As I said before it seems like it doesn't even establish a connection to the remote host. It's possible to connect to given url using a web browser.