tcp - GOLANG, HTTP having "use of closed network connection" error -


i getting lot of error below mentioned,

read tcp xx.xx.xx.xx:80: use of closed network connection

read tcp xx.xx.xx.xx:80: connection reset peer

//function http connection

func getresponsebytesbyurl_raw(resturl, connectiontimeoutstr, readtimeoutstr string) ([]byte, error) {     connectiontimeout, _ /*err*/ := time.parseduration(connectiontimeoutstr)     readtimeout, _ /*err*/ := time.parseduration(readtimeoutstr)     timeout := connectiontimeout + readtimeout // time.duration((strconv.atoi(connectiontimeoutstr) + strconv.atoi(readtimeoutstr)))     //timeout = 200 * time.millisecond     client := http.client{         timeout: timeout,     }     resp, err := client.get(resturl)     if nil != err {         logger.setlog("error getresponsebytesbyurl_raw |err: ", logs.levelerror, err)         return make([]byte, 0), err     }     defer resp.body.close()     body, err := ioutil.readall(resp.body)     return body, err }

update (july 14):

server : numcpu=8, ram=24gb, go=go1.4.2.linux-amd64

i getting such error during high traffic. 20000-30000 request per minutes, , have time frame of 500ms fetch response third party api.

netstat status server (using : netstat -nat | awk '{print $6}' | sort | uniq -c | sort -n) frequency

      1 established)       1 foreign       9 listen      33 fin_wait1     338 established    5530 syn_sent   32202 time_wait

sysctl -p

**sysctl -p** fs.file-max = 2097152 vm.swappiness = 10 vm.dirty_ratio = 60 vm.dirty_background_ratio = 2 net.ipv4.tcp_synack_retries = 2 net.ipv4.ip_local_port_range = 2000 65535 net.ipv4.tcp_rfc1337 = 1 net.ipv4.tcp_fin_timeout = 5 net.ipv4.tcp_keepalive_time = 300 net.ipv4.tcp_keepalive_probes = 5 net.ipv4.tcp_keepalive_intvl = 15 net.core.rmem_default = 31457280 net.core.rmem_max = 12582912 net.core.wmem_default = 31457280 net.core.wmem_max = 12582912 net.core.somaxconn = 65536 net.core.netdev_max_backlog = 65536 net.core.optmem_max = 25165824 net.ipv4.tcp_mem = 65536 131072 262144 net.ipv4.udp_mem = 65536 131072 262144 net.ipv4.tcp_rmem = 8192 87380 16777216 net.ipv4.udp_rmem_min = 16384 net.ipv4.tcp_wmem = 8192 65536 16777216 net.ipv4.udp_wmem_min = 16384 net.ipv4.tcp_max_tw_buckets = 1440000 net.ipv4.tcp_tw_recycle = 0 net.ipv4.tcp_tw_reuse = 1 net.ipv6.bindv6only = 1 net.ipv4.conf.all.rp_filter = 1 net.ipv4.conf.all.accept_source_route = 0 net.ipv4.icmp_echo_ignore_broadcasts = 1 error: "net.ipv4.icmp_ignore_bogus_error_messages" unknown key kernel.exec-shield = 1 kernel.randomize_va_space = 1 net.ipv4.conf.all.log_martians = 1 net.ipv4.conf.default.log_martians = 1 net.ipv4.icmp_ignore_bogus_error_responses = 1 net.ipv4.ip_forward = 0 net.ipv4.conf.all.send_redirects = 0 net.ipv4.conf.default.send_redirects = 0 net.ipv4.conf.all.secure_redirects = 0 net.ipv4.conf.default.secure_redirects = 0

when making connections @ high rate on internet, it's you're going encounter some connection problems. can't mitigate them completely, may want add retry logic around request. actual error type @ point doesn't matter, matching error string use of closed network connection or connection reset peer best can if want specific. make sure limit retries backoff, systems drop or reset connections way limit request rates, , may more errors faster reconnect.

depending on number of remote hosts you're communicating with, want increase transport.maxidleconnsperhost (the default 2). fewer hosts talk to, higher can set this. decrease number of new connections made, , speed requests overall.

if can, try go1.5 beta. there have been couple changes around keep-alive connections may reduce number of errors see.


Comments

Post a Comment

Popular posts from this blog

searchKeyword not working in AngularJS filter -

i writing app using angularjs on front end. want search through table word/field; 1 search box everything. according article here: http://hello-angularjs.appspot.com/searchtable can use filter: searchkeyword this. this code, , not working expected. <label>search: <input ng-model="searchkeyword"></label> <table ng-repeat="post in posts | orderby: sort | filter: searchkeyword"> <tr> <td> {{index(post)}} </td> <td> {{post.title}} </td> <td> {{post.author}} </td> <td> {{post.content}} </td> <td> {{post.date | date: "d/m/yyyy"}} </td> </tr> </table> what should do? thank you try this: controller $scope.posts = [ {title:'title one', author:'author one', content: 'content one'}, {title:'title two', author:'author two', content: 'cont...
Read more

sequelize.js - Sequelize: sort by enum cases -

i have model ( article ) field (the name type ) of type enum('source', 'translated') , return 1 article ordered type field. source articles should returned before translated articles. like: article.findone(order: [ { type: [ 'source', 'translated' ] } ]); unfortunately there appears no built-in solution sorting enums. instead have sort by case , couldn't find documentation on how sequelize. what conventional approach here? sorting case hard in sequelize, sort field (the second answer in linked post) should possible: order: sequelize.fn('field', sequelize.col('type'), 'source', 'translated') it's supported mysql far know though
Read more

user interface - how to replace an ongoing process of image capture from another process call over the same ImageLabel in python's GUI TKinter -

i trying use following code create interactive gui changes color-space(hsv, rbg, grayscale etc) on event of button click. display opencv video in tkinter using multiprocessing being new python having problems multiprocessing , attempts on making gui change it's color space on button clicks hangs entire system. on it's implementation highly appreciated. thankyou. below code: import cv2 pil import image,imagetk import tkinter tk import numpy np multiprocessing import process , queue def quit_it(root,process): root.destroy() process.terminate() def black_andwhite(root,process): process.terminate p=process(target=capture_image, args=(5,queue, )) p.start() root.after(0, func=lambda: update_all(root, image_label, queue)) def update_image(image_label, queue): frame = queue.get() = image.fromarray(frame) b = imagetk.photoimage(image=a) image_label.configure(image=b) image_label._image_cache = b root.update() def update_all(root...
Read more