Vishal Gupta's Blog

Exadata Storage/Compute Node remote re-imaging

Posted by Vishal Gupta on Oct 23, 2011

When you have a Exadata machine in the lab and you are testing lot of different things or giving hand-ons training to production DBA on lab Exadata to familiarize them with Exadata patching, one has to frequently start from scratch i.e . a particular storage/compute node image. Wouldn’t it be nice if you could write a script to re-image the servers? But alas!!! that is not possible with Exadata. One has to do all the reimaging manually. Even with the manual process, one has to insert the external USB into each storage/compute node and then remote it before reboot to complete the process. With ILOM (Integrated Lights Out Management) it is possible to mount a remote cdrom image (*.iso) file as a virtual cdrom. One can create an storage/compute node image iso using computeImageMaker_<imageversion>.x86_64.tar or cellImageMaker_<imageversion>.x86_64.tar file.

For example for compute node, please note we navigate into dl380 for compute node

# tar -pxvf computeImageMaker_11.
# cd dl380
# computeNodeImage.iso

Or for cell node, please note we navigate into dl180 directory for cell.

# tar -pxvf cellImageMaker_11.
# cd dl180
# cellImage.iso

Would it not have been nice, if one could just mount this iso file via ILOM as virtual cdrom, change the boot order in BIOS by booting into bios, which can also be force either via ILOM gui or via /usr/bin/biosconfig -set_boot_override <xmlfile> command, and choose virtual cdrom as first boot device. But problem with this approach is cell/compute node imaging process resets the ILOM, which means that even our virtual cdrom iso image is also removed during this process and this results in imaging process not completing properly.

One could try leaving a external USB permanently attached on lab exadata, and then via ILOM try to reimage, so that during ILOM reset boot image device is not detached and once can simply reconnect to ILOM to continue answering the on-screen messages. But one problem with this approach is, as part of imaging one has to remove the external USB stick otherwise automated configuration scripts dont funtion properly. I tried it on the lab exadata and could get the cell to reimage properly with external USB still attached. Even after manually changing the boot order to internal USB and then harddrive, which is what cell checks for during each reboot validation, cell just sat on a blinking prompt without going forward. I left the cell overnight as well, thinking that properly some processing is going on, but with no luck. Once i removed the external USB stick, automated configuration scripts were able to properly complete the imaging process and came to automated run of ipconf script, which set the various setting on the cell at first boot after reimaging process.

Would it have been nice, it all this could be done remotely via ILOM once ilom has been connected to the network, as ilom network configuration is not reset during re-imaging process. But that would be wishful thinking !!!

I was trying this with image, as that was my starting image on compute/cell nodes in prod/dr/test, so wanted to replicated the same history. I have not tried with later image versions yet to see if process has been improved in this regard, that trial would be for some time later.




4 Responses to “Exadata Storage/Compute Node remote re-imaging”

  1. jc nars said

    Created a USB image & tried booting the cell with the generated image and got this:

    [ERROR] get_node_type: Not a supported hardware model Virtualbox. dmidecode -s system-product-name returns invalid value

    Ever came across this?


  2. JC Nars,

    You can not use the USB image to install cell image in a VM. USB Image also tries to install all the firmware for ILOM, BIOS and other hardware which dont have in the VM. So trying to install cell in VM using the USB image will fail. Installing cell in a VM involves using a standard OEL5.5 VM server and carrying out manual steps to install the software. There are install scripts which can be run straight from the downloaded cell image tar file (skipping the firmware bits and cellos bits).


  3. However a standard backup strategy, if in place can save the organization a lot of organizations in the Pittsburgh area.
    If anything goes wrong and information is lost, data recovery can be costly.
    Only thing is that you have to be careful when
    storing the data, since if you will loose the
    data you will loose the data loss will be business considerably high.

  4. kkovachki said

    Hello All,

    Maybe this link will be useful.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: