Configure Oracle Rac Automatic Atorage Management

Build Your Own Oracle RAC 10g Release 2 Cluster on Linux and FireWire
by Jeffrey Hunter - OTNInstall & Configure Oracle RAC Automatic Storage Management (ASMLib 2.0)

Most of the installation and configuration procedures should be performed on all nodes. Creating the ASM disks, however, will only need to be performed on a single node within the cluster.

In this section, you will configure ASM to be used as the filesystem / volume manager for all Oracle physical database files (data, online redo logs, control files, archived redo logs) and a Flash Recovery Area.

The ASM feature was introduced in Oracle Database 10g Release 1 and is used to alleviate the DBA from having to manage individual files and drives. ASM is built into the Oracle kernel and provides the DBA with a way to manage thousands of disk drives 24x7 for both single and clustered instances of Oracle. All the files and directories to be used for Oracle will be contained in a disk group. ASM automatically performs load balancing in parallel across all available disk drives to prevent hot spots and maximize performance, even with rapidly changing data usage patterns.

I start this section by first discussing the ASMLib 2.0 libraries and its associated driver for Linux plus other methods for configuring ASM with Linux. Next, I will provide instructions for downloading the ASM drivers (ASMLib Release 2.0) specific to your Linux kernel. Last, you will install and configure the ASMLib 2.0 drivers while finishing off the section with a demonstration of how to create the ASM disks.

Methods for Configuring ASM with Linux (For Reference Only)

When I first started this guide, I wanted to focus on using ASM for all database files. I was curious to see how well ASM works with this test RAC configuration with regard to load balancing and fault tolerance.

There are two different methods to configure ASM on Linux:
ASM with ASMLib I/O: This method creates all Oracle database files on raw block devices managed by ASM using ASMLib calls. Raw devices are not required with this method as ASMLib works with block devices.
ASM with Standard Linux I/O: This method creates all Oracle database files on raw character devices managed by ASM using standard Linux I/O system calls. You will be required to create raw devices for all disk partitions used by ASM.

We will examine the "ASM with ASMLib I/O" method here.

Before discussing the installation and configuration details of ASMLib, however, I thought it would be interesting to talk briefly about the second method, "ASM with Standard Linux I/O." If you were to use this method (which is a perfectly valid solution, just not the method we will be implementing here), you should be aware that Linux does not use raw devices by default. Every Linux raw device you want to use must be bound to the corresponding block device using the raw driver. For example, if you wanted to use the partitions we've created, (/dev/sda2, /dev/sda3, and /dev/sda4), you would need to perform the following tasks:

Edit the file /etc/sysconfig/rawdevices as follows:

  1. raw device bindings
  2. format: <rawdev> <major> <minor>
  3. <rawdev> <blockdev>
  4. example: /dev/raw/raw1 /dev/sda1
  5. /dev/raw/raw2 8 5

/dev/raw/raw2 /dev/sda2
/dev/raw/raw3 /dev/sda3
/dev/raw/raw4 /dev/sda4
The raw device bindings will be created on each reboot.

You would then want to change ownership of all raw devices to the "oracle" user account:

  1. chown oracle:dba /dev/raw/raw2; chmod 660 /dev/raw/raw2
  2. chown oracle:dba /dev/raw/raw3; chmod 660 /dev/raw/raw3
  3. chown oracle:dba /dev/raw/raw4; chmod 660 /dev/raw/raw4

The last step is to reboot the server to bind the devices or simply restart the rawdevices service:

  1. service rawdevices restart

As I mentioned earlier, the above example was just to demonstrate that there is more than one method for using ASM with Linux. Now let's move on to the method that will be used for this article, "ASM with ASMLib I/O."

Download the ASMLib 2.0 Packages

First download the ASMLib 2.0 libraries (from OTN) and the driver (from my web site). Like OCFS, you need to download the version for the Linux kernel and number of processors on the machine. You are using kernel 2.6.9-11.0.0.10.3.EL #1 on single-processor machines:

  1. uname -a

Linux linux1 2.6.9-11.0.0.10.3.EL #1 Tue Jul 5 12:20:09 PDT 2005 i686 i686 i386 GNU/Linux
Oracle ASMLib Downloads for Red Hat Enterprise Linux 4 AS
oracleasm-2.6.9-11.0.0.10.3.EL-2.0.0-1.i686.rpm - (Driver for "up" kernels)
-OR-

oracleasm-2.6.9-11.0.0.10.3.ELsmp-2.0.0-1.i686.rpm - (Driver for "smp" kernels)
oracleasmlib-2.0.0-1.i386.rpm - (Userspace library)
oracleasm-support-2.0.0-1.i386.rpm - (Driver support files)

Install ASMLib 2.0 Packages

This installation needs to be performed on all nodes as the root user account:
$ su -

  1. rpm -Uvh oracleasm-2.6.9-11.0.0.10.3.EL-2.0.0-1.i686.rpm oracleasmlib-2.0.0-1.i386.rpm oracleasm-support-2.0.0-1.i386.rpm

Preparing… ########################################### [100%]
1:oracleasm-support ########################################### [ 33%]
2:oracleasm-2.6.9-11.0.0.########################################### [ 67%]
3:oracleasmlib ########################################### [100%]

Configure and Loading the ASMLib 2.0 Packages

Now that you downloaded and installed the ASMLib Packages for Linux, you need to configure and load the ASM kernel module. This task needs to be run on all nodes as root:
$ su -

  1. /etc/init.d/oracleasm configure

Configuring the Oracle ASM library driver.

This will configure the on-boot properties of the Oracle ASM library
driver. The following questions will determine whether the driver is
loaded on boot and what permissions it will have. The current values
will be shown in brackets ('[]'). Hitting <ENTER> without typing an
answer will keep that current value. Ctrl-C will abort.

Default user to own the driver interface []: oracle
Default group to own the driver interface []: dba
Start Oracle ASM library driver on boot (y/n) [n]: y
Fix permissions of Oracle ASM disks on boot (y/n) [y]: y
Writing Oracle ASM library driver configuration: [ OK ]
Creating /dev/oracleasm mount point: [ OK ]
Loading module "oracleasm": [ OK ]
Mounting ASMlib driver filesystem: [ OK ]
Scanning system for ASM disks: [ OK ]

Create ASM Disks for Oracle

In Section 10, you created three Linux partitions to be used for storing Oracle database files like online redo logs, database files, control files, archived redo log files, and a flash recovery area.

Here is a list of those partitions we created for use by ASM: Oracle ASM Partitions Created
Filesystem Type Partition Size Mount Point File Types
ASM /dev/sda2 50GB ORCL:VOL1 Oracle Database Files
ASM /dev/sda3 50GB ORCL:VOL2 Oracle Database Files
ASM /dev/sda4 100GB ORCL:VOL3 Flash Recovery Area
Total 200GB

The last task in this section it to create the ASM Disks. Creating the ASM disks only needs to be done on one node as the root user account. I will be running these commands on linux1. On the other nodes, you will need to perform a scandisk to recognize the new volumes. When that is complete, you should then run the oracleasm listdisks command on all nodes to verify that all ASM disks were created and available.
$ su -

  1. /etc/init.d/oracleasm createdisk VOL1 /dev/sda2

Marking disk "/dev/sda2" as an ASM disk [ OK ]

  1. /etc/init.d/oracleasm createdisk VOL2 /dev/sda3

Marking disk "/dev/sda3" as an ASM disk [ OK ]

  1. /etc/init.d/oracleasm createdisk VOL3 /dev/sda4

Marking disk "/dev/sda4" as an ASM disk [ OK ]
Note: If you are repeating this guide using the same hardware (actually, the same shared drive), you may get a failure when attempting to create the ASM disks. If you do receive a failure, try listing all ASM disks using:

  1. /etc/init.d/oracleasm listdisks

VOL1
VOL2
VOL3
As you can see, the results show that I have three volumes already defined. If you have the three volumes already defined from a previous run, go ahead and remove them using the following commands and then creating them again using the above (oracleasm createdisk) commands:

  1. /etc/init.d/oracleasm deletedisk VOL1

Removing ASM disk "VOL1" [ OK ]

  1. /etc/init.d/oracleasm deletedisk VOL2

Removing ASM disk "VOL2" [ OK ]

  1. /etc/init.d/oracleasm deletedisk VOL3

Removing ASM disk "VOL3" [ OK ]
On all other nodes in the cluster, you must perform a scandisk to recognize the new volumes:

  1. /etc/init.d/oracleasm scandisks

Scanning system for ASM disks [ OK ]

You can now test that the ASM disks were successfully created by using the following command on all nodes as the root user account:

  1. /etc/init.d/oracleasm listdisks

VOL1
VOL2
VOL3

Reference [http://www.rampant-books.com/art_hunter_rac_oracle_asm_automatic_storage_management.htm]