Linux libc5.4.33 dumbness w/ mk[s]temp()

Greg Alexander (galexand@SIETCH.BLOOMINGTON.IN.US)
Sat, 11 Apr 1998 16:46:56 -0500

Linux libc5.4.33's mk[s]temp() functions require 6 X's at the end of
a filename (the BSD versions I've seen are a bit more flexible). This alone
is enough to break any claims to real BSD compatability, but wait, there's
more:
Only 1 of those 6 X's are really unique. The rest are simply pid.
So you can create exactly 62 temp files using mk[s]temp() in one pid. When
using mkstemp(), a EEXIST error is returned, and you get no fd, so you're
unlikely to do anything dumb, so it's just a really annoying DoS. When
using mktemp(), an error is also returned, ENOENT (why not EEXIST?). I was
naive when coding my program that used mktemp(), so I didn't check the
return value (the string was modified to the filename of the most recently
created temp file, so it seemed as if the call had succeeded, I just didn't
get a unique filename)...what're the odds of something like 10^6 (assuming
it only uses digits) temp files showing up in ~root/mail/, right?
The best way I can think of to handle this situation is to cause the
X's to be 3 bytes (represented in hex) of an unsigned int that is
initialized using the getpid()+time() or something and just incremented
(making it static to avoid unnecessary tests...I don't know what this does
to threadsafeness). That way you get 16^6 temp files, more than enough,
given that I have fewer than 100,000 inodes in use on my whole drive.
As usual, good programming practices (checking your return values)
saves you from the security problems, but allows the DoS to continue.
Judging by past behavior from the glibc2/libc6 team, if the problem exists
there (I haven't checked...it's a big download), it'll probably get fixed
soon enough. I apologize for not submitting a libc5 patch -- I don't expect
a lot of people to upgrade libc5 versions with libc6 so imminent.

I did test this on BSDI BSD/OS 3.1. BSD/OS seemed a lot more
intelligent (i.e. kept on creating unique filenames) but a /lot/ slower than
I would have expected (only subjectively so).

Greg Alexander - also <gralexan@indiana.edu> - http://sietch.home.ml.org/
----
Don't ever do anything that you can't later justify with "It seemed like a
good idea at the time."
-- Greg's Bible