Watching something be piped to a file live with tail
I have a python program which is, slowly, generating some output.
I want to capture that in a file, but I also thought I could watch it live with tail.
So in one terminal I'm doing :
python myprog.py > output.txt
and in another terminal :
tail -f output.txt
But it seems like the tail isn't showing me anything while the python program is running. If I hit ctrl-c to kill the python script, suddenly the tail of output.txt starts filling up. But not while the python is running.
What am I doing wrong?
linux command-line pipe
asked 3 hours ago
interstarinterstar
320311
add a comment |
I have a python program which is, slowly, generating some output.
I want to capture that in a file, but I also thought I could watch it live with tail.
So in one terminal I'm doing :
python myprog.py > output.txt
and in another terminal :
tail -f output.txt
But it seems like the tail isn't showing me anything while the python program is running. If I hit ctrl-c to kill the python script, suddenly the tail of output.txt starts filling up. But not while the python is running.
What am I doing wrong?
linux command-line pipe
asked 3 hours ago
interstarinterstar
320311
2
How aboutpython myprog.py | tee output.txtinstead?
– n8te
3 hours ago
@n8te tee might show the same problem if the program isn't flushing the output buffer regularly. This needs flush() and tee.
– JPhi1618
25 mins ago
@JPhi1618 - you could be right but I just tested it with 800MB of data output and it didn't skip a beat.
– n8te
12 mins ago
add a comment |
I have a python program which is, slowly, generating some output.
I want to capture that in a file, but I also thought I could watch it live with tail.
So in one terminal I'm doing :
python myprog.py > output.txt
and in another terminal :
tail -f output.txt
But it seems like the tail isn't showing me anything while the python program is running. If I hit ctrl-c to kill the python script, suddenly the tail of output.txt starts filling up. But not while the python is running.
What am I doing wrong?
linux command-line pipe
asked 3 hours ago
interstarinterstar
320311
I have a python program which is, slowly, generating some output.
I want to capture that in a file, but I also thought I could watch it live with tail.
So in one terminal I'm doing :
python myprog.py > output.txt
and in another terminal :
tail -f output.txt
But it seems like the tail isn't showing me anything while the python program is running. If I hit ctrl-c to kill the python script, suddenly the tail of output.txt starts filling up. But not while the python is running.
What am I doing wrong?
linux command-line pipe
linux command-line pipe
asked 3 hours ago
interstarinterstar
320311
asked 3 hours ago
interstarinterstar
320311
asked 3 hours ago
interstarinterstar
320311
asked 3 hours ago
interstarinterstar
320311
asked 3 hours ago
interstarinterstar
320311
320311
2
How aboutpython myprog.py | tee output.txtinstead?
– n8te
3 hours ago
@n8te tee might show the same problem if the program isn't flushing the output buffer regularly. This needs flush() and tee.
– JPhi1618
25 mins ago
@JPhi1618 - you could be right but I just tested it with 800MB of data output and it didn't skip a beat.
– n8te
12 mins ago
add a comment |
2
How aboutpython myprog.py | tee output.txtinstead?
– n8te
3 hours ago
@n8te tee might show the same problem if the program isn't flushing the output buffer regularly. This needs flush() and tee.
– JPhi1618
25 mins ago
@JPhi1618 - you could be right but I just tested it with 800MB of data output and it didn't skip a beat.
– n8te
12 mins ago
2
2
How about
python myprog.py | tee output.txt instead?– n8te
3 hours ago
How about
python myprog.py | tee output.txt instead?– n8te
3 hours ago
@n8te tee might show the same problem if the program isn't flushing the output buffer regularly. This needs flush() and tee.
– JPhi1618
25 mins ago
@n8te tee might show the same problem if the program isn't flushing the output buffer regularly. This needs flush() and tee.
– JPhi1618
25 mins ago
@JPhi1618 - you could be right but I just tested it with 800MB of data output and it didn't skip a beat.
– n8te
12 mins ago
@JPhi1618 - you could be right but I just tested it with 800MB of data output and it didn't skip a beat.
– n8te
12 mins ago
add a comment |
2 Answers
2
active
oldest
votes
You may also need to explicitly flush the buffer for it to get piped upon generation. This is because output is typically only printed when the pipe's buffer fills up (which is in kilobytes I belive), and when the stdin message ends. This is probably to save on read/writes. You could do this after every print, or if you are looping, after the last print within the loop.
import sys
...
print('Some message')
sys.stdout.flush()
answered 3 hours ago
DaveyDavey
461
New contributor
Davey is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
add a comment |
Instead of trying to tail a live file, use tee instead. It was made to do exactly what you're trying to do.
From man tee:
tee(1) - Linux man page
Name tee - read from standard input and write to standard output and files
Synopsis
tee [OPTION]... [FILE]...
Description
Copy standard input to each FILE, and also to standard output.
-a, --append
append to the given FILEs, do not overwrite
-i, --ignore-interrupts
ignore interrupt signals
--help
display this help and exit
--version
output version information and exit
If a FILE is -, copy again to standard output.
So in your case you'd run:
python myprog.py | tee output.txt
answered 2 hours ago
n8ten8te
5,17272234
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "3"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsuperuser.com%2fquestions%2f1421123%2fwatching-something-be-piped-to-a-file-live-with-tail%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
You may also need to explicitly flush the buffer for it to get piped upon generation. This is because output is typically only printed when the pipe's buffer fills up (which is in kilobytes I belive), and when the stdin message ends. This is probably to save on read/writes. You could do this after every print, or if you are looping, after the last print within the loop.
import sys
...
print('Some message')
sys.stdout.flush()
answered 3 hours ago
DaveyDavey
461
New contributor
Davey is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
add a comment |
You may also need to explicitly flush the buffer for it to get piped upon generation. This is because output is typically only printed when the pipe's buffer fills up (which is in kilobytes I belive), and when the stdin message ends. This is probably to save on read/writes. You could do this after every print, or if you are looping, after the last print within the loop.
import sys
...
print('Some message')
sys.stdout.flush()
answered 3 hours ago
DaveyDavey
461
New contributor
Davey is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
add a comment |
You may also need to explicitly flush the buffer for it to get piped upon generation. This is because output is typically only printed when the pipe's buffer fills up (which is in kilobytes I belive), and when the stdin message ends. This is probably to save on read/writes. You could do this after every print, or if you are looping, after the last print within the loop.
import sys
...
print('Some message')
sys.stdout.flush()
answered 3 hours ago
DaveyDavey
461
New contributor
Davey is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
You may also need to explicitly flush the buffer for it to get piped upon generation. This is because output is typically only printed when the pipe's buffer fills up (which is in kilobytes I belive), and when the stdin message ends. This is probably to save on read/writes. You could do this after every print, or if you are looping, after the last print within the loop.
import sys
...
print('Some message')
sys.stdout.flush()
answered 3 hours ago
DaveyDavey
461
New contributor
Davey is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
answered 3 hours ago
DaveyDavey
461
New contributor
Davey is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
answered 3 hours ago
DaveyDavey
461
answered 3 hours ago
DaveyDavey
461
461
New contributor
Davey is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
New contributor
Davey is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
Davey is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
add a comment |
add a comment |
Instead of trying to tail a live file, use tee instead. It was made to do exactly what you're trying to do.
From man tee:
tee(1) - Linux man page
Name tee - read from standard input and write to standard output and files
Synopsis
tee [OPTION]... [FILE]...
Description
Copy standard input to each FILE, and also to standard output.
-a, --append
append to the given FILEs, do not overwrite
-i, --ignore-interrupts
ignore interrupt signals
--help
display this help and exit
--version
output version information and exit
If a FILE is -, copy again to standard output.
So in your case you'd run:
python myprog.py | tee output.txt
answered 2 hours ago
n8ten8te
5,17272234
add a comment |
Instead of trying to tail a live file, use tee instead. It was made to do exactly what you're trying to do.
From man tee:
tee(1) - Linux man page
Name tee - read from standard input and write to standard output and files
Synopsis
tee [OPTION]... [FILE]...
Description
Copy standard input to each FILE, and also to standard output.
-a, --append
append to the given FILEs, do not overwrite
-i, --ignore-interrupts
ignore interrupt signals
--help
display this help and exit
--version
output version information and exit
If a FILE is -, copy again to standard output.
So in your case you'd run:
python myprog.py | tee output.txt
answered 2 hours ago
n8ten8te
5,17272234
add a comment |
Instead of trying to tail a live file, use tee instead. It was made to do exactly what you're trying to do.
From man tee:
tee(1) - Linux man page
Name tee - read from standard input and write to standard output and files
Synopsis
tee [OPTION]... [FILE]...
Description
Copy standard input to each FILE, and also to standard output.
-a, --append
append to the given FILEs, do not overwrite
-i, --ignore-interrupts
ignore interrupt signals
--help
display this help and exit
--version
output version information and exit
If a FILE is -, copy again to standard output.
So in your case you'd run:
python myprog.py | tee output.txt
answered 2 hours ago
n8ten8te
5,17272234
Instead of trying to tail a live file, use tee instead. It was made to do exactly what you're trying to do.
From man tee:
tee(1) - Linux man page
Name tee - read from standard input and write to standard output and files
Synopsis
tee [OPTION]... [FILE]...
Description
Copy standard input to each FILE, and also to standard output.
-a, --append
append to the given FILEs, do not overwrite
-i, --ignore-interrupts
ignore interrupt signals
--help
display this help and exit
--version
output version information and exit
If a FILE is -, copy again to standard output.
So in your case you'd run:
python myprog.py | tee output.txt
answered 2 hours ago
n8ten8te
5,17272234
answered 2 hours ago
n8ten8te
5,17272234
answered 2 hours ago
n8ten8te
5,17272234
answered 2 hours ago
n8ten8te
5,17272234
5,17272234
add a comment |
add a comment |
Thanks for contributing an answer to Super User!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsuperuser.com%2fquestions%2f1421123%2fwatching-something-be-piped-to-a-file-live-with-tail%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
2
How about
python myprog.py | tee output.txtinstead?– n8te
3 hours ago
@n8te tee might show the same problem if the program isn't flushing the output buffer regularly. This needs flush() and tee.
– JPhi1618
25 mins ago
@JPhi1618 - you could be right but I just tested it with 800MB of data output and it didn't skip a beat.
– n8te
12 mins ago