最近在工作上写了一个接口，供ITSM流程调用。

调用这个接口会触发一个任务，处理一些单据数据。因为每天的单据数量不固定，所以这个任务的完成时间也不固定。

接口刚上线时，这个接口的任务完成时间在1分钟以内，直接调用不会有任何问题。

最近因为要处理的单据数量上来了，处理时间超过1分钟了，导致调用接口会有报错，但实际后台任务是可以跑完的。

例如，执行curl请求之后，报错信息如下：

[root@bk-07 ~]# curl http://172.16.1.11:83/sync_cteam_itr

            The page is temporarily unavailable                    

           

nginx error!

       

           

The page you are looking for is temporarily unavailable.  Please try again later.

           

               

Website Administrator

               
                   

Something has triggered an error on your                     website.  This is the default error page for                     nginx that is distributed with                     Fedora.  It is located                     /usr/share/nginx/html/50x.html

                   

You should customize this error page for your own                     site or edit the error_page directive in                     the nginx configuration file                     /etc/nginx/nginx.conf.

               

           

           

                运维 【nginx报错】upstream timed out (110: Connection out) while reading response header from upstream 第1张

                运维 【nginx报错】upstream timed out (110: Connection out) while reading response header from upstream 第2张            

       
   

一开始怀疑是nginx层面因上游的uwsgi超过1分钟没返回，然后自动报错返回，故先抛开nginx，直接请求uwsgi的接口，测试发现还是有报错，如下：

[root@bk-07 welink]# curl http://127.0.0.1:5007/sync_cteam_itr curl: (52) Empty reply from server 

查看nginx的日志，如下：

[root@bk-07 welink]# tail /var/log/nginx/access.log 172.16.1.11 - - [29/Nov/2023:00:16:08 +0800] "GET /sync_cteam_itr HTTP/1.1" 504 3693 "-" "curl/7.29.0" 172.16.1.11 - - [29/Nov/2023:00:18:34 +0800] "GET /sync_cteam_itr HTTP/1.1" 504 3693 "-" "curl/7.29.0" 172.16.1.11 - - [29/Nov/2023:00:21:38 +0800] "GET /sync_cteam_itr HTTP/1.1" 504 3693 "-" "curl/7.29.0" 172.16.1.11 - - [29/Nov/2023:00:24:34 +0800] "GET /sync_cteam_itr HTTP/1.1" 504 3693 "-" "curl/7.29.0" 172.16.1.5 - - [29/Nov/2023:00:31:00 +0800] "GET /sync_cteam_itr HTTP/1.1" 504 3693 "-" "curl/7.29.0" 172.16.1.11 - - [29/Nov/2023:00:35:14 +0800] "GET /sync_cteam_itr HTTP/1.1" 504 3693 "-" "curl/7.29.0" 172.16.1.11 - - [29/Nov/2023:00:42:22 +0800] "GET /sync_cteam_itr HTTP/1.1" 504 3693 "-" "curl/7.29.0" 172.16.1.5 - - [29/Nov/2023:00:45:15 +0800] "GET /reset_cteam_status HTTP/1.1" 200 75 "-" "curl/7.29.0" 172.16.1.11 - - [29/Nov/2023:00:52:22 +0800] "GET /sync_cteam_itr HTTP/1.1" 200 50 "-" "curl/7.29.0" 172.16.1.5 - - [29/Nov/2023:00:54:03 +0800] "GET /sync_cteam_itr HTTP/1.1" 200 51 "-" "curl/7.29.0" [root@bk-07 welink]# [root@bk-07 welink]# tail /var/log/nginx/error.log 2023/11/29 00:06:57 [error] 10#10: *1 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 172.16.1.11, server: localhost, request: "GET /sync_cteam_itr HTTP/1.1", upstream: "http://127.0.0.1:5007/sync_cteam_itr", host: "172.16.1.11:83" 2023/11/29 00:10:28 [error] 10#10: *1 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 172.16.1.11, server: localhost, request: "GET /sync_cteam_itr HTTP/1.1", upstream: "http://127.0.0.1:5007/sync_cteam_itr", host: "172.16.1.11:83" 2023/11/29 00:13:34 [error] 10#10: *1 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 172.16.1.11, server: localhost, request: "GET /sync_cteam_itr HTTP/1.1", upstream: "http://127.0.0.1:5007/sync_cteam_itr", host: "172.16.1.11:83" 2023/11/29 00:16:08 [error] 10#10: *1 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 172.16.1.11, server: localhost, request: "GET /sync_cteam_itr HTTP/1.1", upstream: "http://127.0.0.1:5007/sync_cteam_itr", host: "172.16.1.11:83" 2023/11/29 00:18:34 [error] 10#10: *1 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 172.16.1.11, server: localhost, request: "GET /sync_cteam_itr HTTP/1.1", upstream: "http://127.0.0.1:5007/sync_cteam_itr", host: "172.16.1.11:83" 2023/11/29 00:21:38 [error] 10#10: *1 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 172.16.1.11, server: localhost, request: "GET /sync_cteam_itr HTTP/1.1", upstream: "http://127.0.0.1:5007/sync_cteam_itr", host: "172.16.1.11:83" 2023/11/29 00:24:34 [error] 11#11: *1 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 172.16.1.11, server: localhost, request: "GET /sync_cteam_itr HTTP/1.1", upstream: "http://127.0.0.1:5007/sync_cteam_itr", host: "172.16.1.11:83" 2023/11/29 00:31:00 [error] 10#10: *1 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 172.16.1.5, server: localhost, request: "GET /sync_cteam_itr HTTP/1.1", upstream: "http://127.0.0.1:5007/sync_cteam_itr", host: "172.16.1.11:83" 2023/11/29 00:35:14 [error] 11#11: *1 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 172.16.1.11, server: localhost, request: "GET /sync_cteam_itr HTTP/1.1", upstream: "http://127.0.0.1:5007/sync_cteam_itr", host: "172.16.1.11:83" 2023/11/29 00:42:22 [error] 10#10: *1 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 172.16.1.11, server: localhost, request: "GET /sync_cteam_itr HTTP/1.1", upstream: "http://127.0.0.1:5007/sync_cteam_itr", host: "172.16.1.11:83"

 解决方法：

1、修改uwsgi的配置

在uwsgi的配置里添加3个配置项：

harakiri = 600 http-timeout = 600 http-timeout-keep-alive = 600

2、修改nginx的配置

在nginx的http配置块里添加3个配置项：

http {

...

proxy_connect_timeout 600s;

proxy_send_timeout 600s;

proxy_read_timeout 600s;

...

}

3、重新服务

如果是docker部署的，则重启docker服务。

4、验证结果

[root@bk-07 ~]# curl http://172.16.1.11:83/sync_cteam_itr 任务执行完成，耗时:  77.0742256641388 秒 

精彩文章

评论可见，请评论后查看内容，谢谢！！！
 您阅读本篇文章共花了：