An IBM fix is available.
The details:
I've raised this problem in some newsgroups almost one year ago, and
I've also opened a call at IBM at that time. They came back to me with
the solution just three weeks ago...
The problem:
Whenever "ttdbserver" is started (and it is started by default when the
machine is booted), a port >=1024 is assigned to it dynamically. Let's
call this port number n. Due to some bug in inetd, this also causes
inetd to listen on the port with number n-1. For example, if ttdbserver
is on port 1026, 1025 will also start listening.
No service is associated with that listening port, i.e. inetd will allow
incoming connections, but will not start any program for handling them.
How to check for a vulnerable machine:
1.) Use "rpcinfo -p <hostname>" to scan for machines where ttdbserver is
running, and note down the port number given to it.
2.) Use "telnet <hostname> <n-1>" to try to connect to the port with the
next lower number. If a
connection is established, and you can type something without being
thrown out and without getting any response back, you have most likely
found a vulnerable machine.
The attack:
Perform the telnets in a loop, as fast as possible. Send a few
characters (important, "empty" telnets have little or no effect), and
quit or kill telnet. The effect differs whether you quit or kill telnet,
try both and chose the one your target likes best.
Example 1:
while true
do
telnet testhost 1025 <<EOF
jbgkdbdkbdfbdbdfdgbddbkbkdgbdk
^]
quit
EOF
done
Effect:
The loop makes a few hundred or thousand successful
connects. This causes some 100 pages of memory
being consumed on the target machine.
Then, no more telnets to port 1025 are accepted:
Telnet reports a timeout.
The target machine is still operating normally: Network
operation (except for port 1025) is not affected, and
I observed no error messages or other misbehaviour.
However, as far as I can tell, the memory is never freed again,
netstat shows hundreds of CLOSE_WAIT connections
staying around until reboot, and worst of all the kernel thread
"gil" was constantly consuming about 20 % CPU in sysmode,
even after the attack ended, until the machine was rebootet
(at least for the 30 minutes I've waited before rebooting).
Example 2:
while true
do
telnet testhost 1025 <<EOF >/dev/null 2>&1 &
gbbhgdhjgd
EOF
sleep 1
kill %%
done
This brought the target machine to its knees within a few
seconds: It didn't panic, but it didn't react any longer to
any network connection attempts or any terminal input.
Hence, I was unable to find out what actually caused the
trouble, I assume lack of mbufs.
Stopping the attack didn't help, after five minutes
we decided to reboot the target machine by hitting the
reset button.
The target was a rather small machine: A 32H with 32 MB.
The effects differ depending on the size and configuration
of the target and the network speed.
As far as I can tell, killing the connection instead of terminating it
cleanly leaves two connections around according to netstat -a,
both of them with a huge queue contents (32 KB in one direction,
16 KB in the other).
The attack is about as silly and as obvious to trace back
as the classical octopus.c, but differing from octopus, it
doesn't require significant ressources on the attacking machine,
and its effect continues after the attack has ended.
The fix:
The inetd patch IX70400 fixes the problem
(fileset bos.net.tcp.client 4.1.5.12).
Don't be confused by the description of that APAR -
the APAR is for a different inetd problem, it solves
mine as a side effect.
DI. Dr. Klaus Kusche
Oberoesterreichische Landesregierung / Government of Upper Austria
Rechenzentrum / Computing Centre
Smail: Kaerntnerstrasse 16, A-4020 Linz, Austria (Europe)
Phone: +43 732 7720 - 3394 Fax: +43 732 7720 3198
Email: Klaus.Kusche@ooe.gv.at