HTTP Connections and Timeouts

It is important to set appropriate timeouts on remote calls. In most cases such timeouts are set after problems are experienced on production deployments, but it pays to do early analysis and configure accordingly.

In worst case, without timeouts, connection could be held indefinitely or for long periods of time. Threads initiating the connections will get stuck and over time the application will become unresponsive. For certain use cases, timeouts resulting in failed remote calls can be tolerated. In other use cases, such failures cannot be tolerated and some retry or queuing mechanism needs to be implemented.

Connection and Response Timeouts

If using Apache HttpClient, two timeouts can be configured:

• Connection Timeout: Determines the timeout until a connection is established. Can be set to 5 seconds for co-located deployments, 10 seconds for remotely located deployments. 20 seconds for bad/problematic networks, but personally I have not faced any.
• Socket Timeout: Timeout waiting for data from remote call after connection is established. This depends upon the end point and requires analysis. We analyze response times and set values ranging from 10 secs to 60 seconds, and in one rare case, 120 seconds. If timed out, SocketTimeoutException is thrown.

Connection Pool Timeouts

Connection pooling is a good practice to have a control over resources and resource allocation, particularly when large number of end points are involved. Maximum number of end points and connections per end point (route) can be configured.

Connection pool timeout is set to to prevent threads waiting indefinitely to get connections, which could  make the application unresponsive. Beyond connection pool timeout interval, ConnectionPoolTimeoutException is raised. 30 seconds is a good number for connection pool timeout.

Putting them together

Number of connections per end point, connection timeout, socket timeout and connection pool time out are inter-related, and depends on number of remote calls made during high load.

num_conn_per_route =

num_calls_in_conn_pool_timeout_interval_during_high_load *

max(conn_timeout + socket_timeout) /

conn_pool_timeout