I was using a tool the other day, and I started it in the background. It didn't come up and, when I looked it had stopped. When this has happened in the past I foreground it and it continues working. I've only noticed this on rare occasions, and I'd previously put it down to some misconfiguration of the system. However, one of my colleagues had also noticed it, so this was the ideal opportunity to figure out what really was going on.
The first step was to identify which process was stopped using
$ jobs -l - 25195 Running process1 & + 25223 Stopped (tty output) process2
Having done that, the next step was to find out where the process had actually stopped. This information can be obtained using
ptree which prints out the process call tree:
$ ptree 25223 511 /usr/lib/ssh/sshd 25160 /usr/lib/ssh/sshd 25161 /usr/lib/ssh/sshd 25166 -bash 25223 /bin/sh process2 25232 sed -n $p 25233 /usr/bin/csh -f /usr/bin/which java java 25234 /usr/bin/stty erase ^H
So the process has stalled in
stty setting the erase character to be
^H. The callstack, printed by
pstack, was not very enlightening.
$ pstack 25234 25234: /usr/bin/stty erase ^H feef14d7 ioctl (0, 540f, 8067988) 080516f8 main (3, 8047b24, 8047b34, 80511ff) + 40c 0805125d _start (3, 8047c08, 8047c16, 8047c1c, 0, 8047c1f) + 7d
However the interesting step is from
stty. What's interesting about
which is that it is a C-shell script. The interesting bit is the following:
#! /usr/bin/csh -f #... if ( -r ~/.cshrc && -f ~/.cshrc ) source ~/.cshrc
So which sources the .cshrc file, and my .cshrc file happened to contain
stty erase ^H. So why does this cause the process to stop?
stty controls the characteristics of the terminal, but when the script is executing in the background, there is no terminal. When there's no terminal,
stty stops and waits for one to appear!
The easiest is to move the call to
stty into my
.login file. The
.login file is only parsed at login, and not every time a shell is started. Alternatively, it's possible to check for the existence of a prompt:
if ($?prompt) then if ("$prompt" =~ ?*) then /usr/bin/stty erase ^H endif endif