Re: Linux kernel patch to remove stack exec permission

Mike Meissner (meissner@CYGNUS.COM)
Sat, 12 Apr 1997 08:47:24 -0400 (EDT)

| There seemed to be no patch for Linux kernel to remove execute permission
| from the stack (to prevent most buffer overflow exploits), so I decided to
| make one, I include it at the end of this message. I heard some rumours that
| GCC assumes stack frame to be executable when dealing with nested functions,
| but I couldn't reproduce that. I'm running this patched kernel for a day now,
| and everything (well, except for the exploits) seems to work fine. However,
| some programs may depend on the stack being executable... I'd like to hear
| any reports of this.

Hopefully Linus will NOT install this patch without providing a configure
option to enable/disable it. It is not a rumor but a fact that GCC depends on
the stack being executable in order to support passing the address of nested
functions via trampolines. I really would prefer not to have a new quanity of
tests fail.

Alternatively, you could try to convince RMS and Kenner that trampolines are a
bad idea (I've been trying for 8 years), or root out all of the hidden
assumptions in the compiler that trampolnes are on the stack (been there, tried
it, gave up).

Here is a test case for trampolines:

#include <stdio.h>

int
g (int a, int b, int (*gi) (int, int))
{
printf ("Inside g, a = %d, b = %d, gi = 0x%.8lx\n", a, b, (long)gi);
fflush (stdout);

if ((*gi) (a, b))
return a;
else
return b;
}

void
f (void)
{
int i, j;
int f2 (int a, int b)
{
printf ("Inside f2, a = %d, b = %d\n", a, b);
fflush (stdout);
return a > b;
}

int f3 (int a, int b)
{
printf ("Inside f3, i = %d, j = %d\n", i, j);
fflush (stdout);
return i > j;
}

if (g (1, 2, f2) != 2) {
printf ("Trampoline call returned the wrong value\n");
fflush (stdout);
abort ();
}

i = 4;
j = 3;
if (g (5, 6, f3) != 5) {
printf ("Trampoline call returned the wrong value\n");
fflush (stdout);
abort ();
}
}

int
main (void)
{
printf ("Before trampoline call\n");
fflush (stdout);
f ();
printf ("Trampoline call succeeded\n");
fflush (stdout);
return 0;
}