python - Socket message sending crash -

i have series of messages being sent through socket running in separate thread. short time after sending message, cpu usage process drops nothing, main thread stops running, , memory allocation skyrockets crashing computer.

has experienced issue before? how have fixed it?

i have seen before large messages. messages on socket have limitation of 65,000 bytes (2^16)

so, there's limitation in core linux os doesn't allow socket pipe communicate more @ time. if send more data limit, socket meta information overwritten , parent thread "loses" child thread - , doesnt know lost child thread ! so, keeps waiting child thread send sort of information.

this makes parent thread hang , child thread keeps sending information - eats memory.

the way avoid use tempfile send information 1 side other.

about earlier mentioned tempfile method:

i'm not sure how code is, explaining how used tempfiles in usecase.

so, socket pipes have limitation, obvious choice use other method transfer messages 1 process another. when came across this, trying communicate between child , parent process.

the way doing was:

import subprocess process = subprocess.popen(["some_command"], stdout=subprocess.pipe) process.wait() print(process.stdout.readlines()) 

now, when got error, used temporary file using tempfile module in python, so:

import subprocess stdout_file = tempfile.temporaryfile() process = subprocess.popen(["some_command"], stdout=stdout_file) process.wait() stdout_file.seek(0) print(stdout_file.read().decode("utf-8", "ignore"))