reading from a pipe — stateful or stateless

I need to write a python script that processes the output of a another program. i.e. I have to do:

./other_program | ./my_scripts.py  

My question is whether this my_script.py is state-less. For example can I remember the last line of output I processed that came out of ./other_program, or the script will only process current line, completely unaware of last line?

Add Comment
1 Answer(s)

Pipes don’t know about "lines". Bytes go in on one end, and the same bytes in the same order come out at the other end. There is a little bit of (configurable) buffer in between, but when working with them, consider them being unbuffered.

And line oriented I/O happens at a higher level, e.g. when creating a C stdio FILE object on a pipe file descriptor, or using a readline library (or similar). Or – like in your case – Python stdio print, writelines and readline.

EDIT 1:

Here are two simple python programs


writer.py

#!/usr/bin/env python3 if '__main__' == __name__:     import time     i = 0;     while True:         print(f'This is writer.py, writing a line number {i} to stdout', flush=True)         i += 1         time.sleep(0.25) 

reader.py

#!/usr/bin/env python3 if '__main__' == __name__:     i = 0     while True:         l = input()         print( f'This is reader.py, a line number {i} was read from stdin with the following content\n> {l}\n' )         i += 1 

Test them with ./writer.py | ./reader.py

Answered on August 1, 2020.
Add Comment

Your Answer

By posting your answer, you agree to the privacy policy and terms of service.