Subject: Mirrored swap? Good idea or not?
Author: "Paquette Trevor" <TrevorPaquette@mcc.net> at INTERNET
Date: 2/19/98 7:23 AM
I've had a couple of discussions with some folks around here and they
seem to think that it is not a good idea to mirror your swap device.
I am trying to build a system that is very fault tolerant. I will have
redundant hot swap power supplies, mirrored OS slices across different
controllers, and I am wondering about swap.
I would like to mirror swap across different controllers as well. What
will Solaris 2.6 do if on of the two internal drives dies? I know that
it will fail over to the mirrored drive for OS, but what about swap? Can
I expect a performance hit? or a gain by the round-robin read feature of
mirroring? what about geometric read?
Thoughts?
-----
Basically most of the replies said not a problem. You might take a bit
of a performance hit, but for reliability you should mirror swap as well
as the O. this is the answer that I expected. Thanx to all that answered
below.
----------------
Trevor,
If you want fault tolerance, you should mirror your swap device. There
is no point in mirroring your system if you do not also mirror swap.
As for performance, the best thing is to add enough memory so swapping
is kept to a minimum.
Regards,
Dan Lorenzini Greenwich Capital Markets
dal@gcm.com 600 Steamboat Road
203-625-6088 Greenwich, CT 06830
---------------------
Paquette, Trevor wrote:
>
> I've had a couple of discussions with some folks around here
> and they seem to think that it is not a good idea to mirror
> your swap device.
What?? Writes are done in parallel, reads in round-robin.
Performance hit is slim to none, especially with 2 controllers.
You will actually gain performance on page-ins!
> I am trying to build a system that is very fault tolerant.
> I will have redundant hot swap power supplies, mirrored OS
> slices across different controllers, and I am wondering about
> swap.
If you don't mirror swap, a disk failure will cost you a reboot
and some reconfiguration (i.e. to manually switch swap to the
new disk).
> I would like to mirror swap across different controllers as
> well. What will Solaris 2.6 do if on of the two internal
> drives dies? I know that it will fail over to the mirrored
> drive for OS, but what about swap?
Think of it this way... what if someone erased your chalkboard
half-way through your presentation?
If you don't want your pager going off, mirror your swap.
Toby
-- Toby Creek - UNIX System Administrator Alcatel Telecom, 2912 Wake Forest Road, Raleigh, NC Phone +1 919 850 6465 / Fax +1 919 850 6703 mailto:creek@aur.alcatel.com ------------------hi!!
It is advisable to mirror swap... jut take a case where there is some thing in the Swap are and the disk dies ...now if the swap is not mirrored then you could loose the data etc etc and the purpose of the mirroring and fault tolerant system is defeated!!!!!
SO I HIGHLY RECOMMEND SWAP MUST BE MIRRORED !!
THANKS
venkat Mr.Venkat D [venki21@hotmail.com] ------------------ I've had a couple of discussions with some folks around here and they > seem to think that it is not a good idea to mirror your swap device.
Mirroring swap slows down *writing* to swap, will speed up reading from swap and is *essential* if you plan to handle the loss of the swap disk!
Imagine what would happen if you had 10Mb swapped to disk 'x' then that disk failed. Panic!
> I would like to mirror swap across different controllers as well. What > will Solaris 2.6 do if on of the two internal drives dies? I know that
With Solstice Disk manager it will work OK. Indeed, when I bought my E3000, the engineer recommended this setup!
rgds Stephen Stephen Harris [sweh@mpn.com] --------------------- Hi !! Regarding your situation - I would evaluate what is going to *run* on the system, and equate it to the amount of RAM you have available v/s swap space. If your system swaps a lot - then I dont think that it is a good idea to mirror the swap device, just because there will be to many writes to the controller in question. Think about it - if your swap device fails, you can always create a swapfile on the fly - you cannot do this with filesystems, but you are talking about a swapfile in this case !! If you can afford to dedicate a controller for your mirrored device then you can go right ahead....
I recently tested a fault tolerant system that I built by pulling out the root disk - failover looked nice an everything - but it took some time to sync everything. Unless it is an *EXTREMELY* critical system , I would not recommend mirroring swap devices .... but then again - this is my opinion !!
regards ,
Rahul Roy Rahul Roy [uroy@eadev1.vanguard.com] -----------------
You'll always have a performance hit on writes because you will have to do two writes but you will get a performance increase on reads since the O/S will choose the disk that can service the request the fastest.
Whenever I mirror the O/S, I always mirror the swap for fault tolerance. If you lose an O/S disk and you have not mirrored the swap, it will have nothing to which it can fail over and your system will likely panic and reboot. I have tested shutting off an O/S disk that is mirrored: the system pauses for a little bit and then continues happily along, marking the powered off disk as needing maintenance. I was using DiskSuite 4.0 at the time of testing. DiskSuite 4.1 works quite well as does Veritas Volume Manager.
-- Nicholas Masika email: nicholas_masika@mfi.com Sr UNIX System Administator phone: (415)905-2473 Taos Mountain Consultant pager: (415)253-3725 Miller Freeman Inc, San Francisco pager e-mail: nmasika.page@taos.com------------------ The main thing I have heard about mirroring swap is that it doesn't work properly during a crash dump, and that you may not be able to recover the crash file. Check sunsolve there were some articles about it.
Marc MARC.NEWMAN@chase.com ---------------
If you mirror a disk for fault tolerance, you have to mirror the whole disk, swap included.
Your comment below which ends 'but what about swap?'. A swap device is no different than any other device. It will be failed over just like the OS.
Here is an example which hopefully explains what I am saying. This is Disksuite specific, so if you were using Veritas to do the mirroring there might be some differences.
Lets say you have two disks, the two mirrors. The partition table looks the same, with root, /usr, swap and /opt. You are mirroring /, /usr, /opt, but not swap. What happens if one disk fails? The system locks up. (actually, only if the system is using the swap device, but we will assume that is the case.) The disk will be offlined by the system. However, since the swap device is just defined as a raw device, the system will keep trying to use it and you will have lots of messages in your console.
If the swap is mirrored, the disk will be offlined, and the system will not use it. You can replace it hot if your setup is good for it, or you can wait for a good time to shutdown and replace it.
In our environment here, we have about 100 servers, and this type of thing happens all the time.
People seem to think you can get away with not mirroring swap, but my experience is that if you mirror any part of a disk, you basically have to mirror the whole disk, otherwise you have done yourself no good. At least is terms of availability. If you just need a duplication of data, it might not be important.
Kelly klfergason@amoco.com
--------------- Hi Trevor,
Is mirrored swap a bad idea? Consider this:
Let's you are running a program that writes to a unmirrored partition. When the program issues a write to that partition, it goes into a wait state. It remains in that wait state until that IO completes. Now you run a program that writes to a mirrored partition. What happens when the program issues a write? It goes into a wait state and stays in that wait state until all the volumes in the mirror set return IO completion. So, writes to a mirrored partition can be and most likely are slower to a mirrored partition. The user sees this as a slower running program.
Now lets' take the case of the mirrored swap partition. When the write is issued on the pages to be written to the swap partition, the write is not complete until all the volumes in the mirror set return completion. Who is issuing the write in this case? It is the operating system doing it on behalf of the user. Who waits in this case? The operating system waits. IF the OS waits, guess who else waits? Everybody waits.
So, mirrored swap is a bad idea from a performance standpoint. IF your goal is to have a fault tolerant system, have multiple swap partitions on multiple disk drives on mutiple IO controllers. This way, you can reduce points of failure, the usual strategy for fault tolerance. Also, if you go to the multiple partition/disk/controller strategy, guess what else you get? You get a performance boost becuase you have spread single IO's across spindles and controllers, a time-tested way of improving performance.
Cheers,
Gary Smith gsmith@lvmail1.dseg.ti.com ------------- If you are trying to keep a machine up in case of a drive or controller failure, possibly at the expense of some performance, then by all means mirror swap. If you have swapped out pages when a drive fails, you will probably panic or hang the OS.
There will probably be a swap penalty to be paid for scheduling two writes for all swap pages written. With two controllers, this is likely to be minimal.
With my last employer, we routinely mirrored /, /usr, /var, /opt, and swap on all our high-availability servers. (Basically, everything on the root disk). This is usually a one-controller arrangement too, where you would expect a higher penalty (both writes have to cross the same bus). However, there was no major performance difference. Certainly none detectable by us without measuring.
> Thoughts?
-- Darren Dunham ddunham@taos.com Unix System Administrator Taos Mountain Got some Dr. Pepper? Santa Clara, CA< Please move on, ...nothing to see here, please disperse > --------- I agree that mirroring swap isn't a good idea. Since swap is just unformatted space, why not just allocate and mount swap space across a couple of disk controllers. If for some reason one does not mount the next will. .....Robin
/dev/dsk/c0t3d0s1 - - swap /tmp - /dev/dsk/c0t1d0s1 - - swap /tmp - robin.landis@exim.gov