OpenStack Survival 101: Instance Rescue Mode
Hopefully, you never have an issue that requires an instance to be recovered from a rescue state but if you do I want to save you the heart ache that I went through. The Openstack documentation doesn’t really talk about this at all and where they do the commands tend to be inaccurate. I wanted to share this in case you find yourself searching the entire web for this information.
Before I go any deeper here is the magic command to have NOVA put an instance into rescue mode booting off a specific image.
To Set Instance in Rescue Mode
nova rescue [SERVER] --image [IMAGEID]
To Get Instance Out Of Rescue Mode
nova unrescue [SERVER]
First, let’s talk about what “rescue” mode is. Rescue mode is when you set an instance through NOVA to boot from a rescue image. This is meant to allow you to recover your instance from a major problem at the OS root level. The basic concept of this is when you put an instance in rescue mode it will do a soft reboot of the instance and boot to the rescue image. This by default is specified in your openstack configuration. If you want to boot to a specific image you need to use the –image flag and the image_id of your selected image.
When this takes place the instance reboots and mounts the image_id as the first boot source and then has the existing root volume mounted. ALL other volumes are not mounted!
The reason behind this is that you are just trying to rescue the base OS and that’s it. You're trying to get it to a bootable level.
Once you are done you will take the instance out of rescue mode by issuing the nova unrescue command which will then reboot the instance again, detach the rescue image and boot with all volumes attached as normal.
If everything worked out your back in business.
Rescue Images From ISO Files
So if you’ve read this far I’m assuming your trying to boot to a ISO file for your rescue image. There is something you need to know that seems to be left out of all documentation on this. In order to boot from an ISO file as a rescue image you need to convert that to a disk bootable ISO. What does that mean? Well, normally when you boot a an ISO from a CD it’s loaded differently than booting from a disk. Openstack Rescue mode treats the image as a bootable disk. Since it does that, a standard DVD or CD ISO WILL NOT WORK. Openstack has no concept of a CD-Rom source as a target for boot.
You need to covert that over to a disk based ISO image. You’re going to need Isohybrid (http://www.syslinux.org/wiki/index.php?title=Isohybrid) which is a tool that converts conventional ISO files to bootable device ISO files. It’s normally used for generating a USB bootable disk image. If you have another tool you prefer, use it but remember it needs to be a bootable ISO disk not a CD-Rom ISO for rescue to work.
Hopefully, this has been helpful and if you have additional suggestions, would like to share your experiences, or have ideas for other topics please feel free to comment below.