Placement of Voting disk and OCR Files in Oracle RAC 10g and 11gR1 [ID 293819.1]

Placement of Voting disk and OCR Files in Oracle RAC 10g and 11gR1 [ID 293819.1]
	Modified27-OCT-2010TypeBULLETINStatusPUBLISHED

In this Document
Purpose
Scope and Application
Placement of Voting disk and OCR Files in Oracle RAC 10g and 11gR1
Linux–
AIX –
HP–
Solaris–
Tru64 –
References

---

Applies to:

Oracle Server - Enterprise Edition - Version: 10.1.0.5 to 11.1.0.7 - Release: 10.1 to 11.1
Information in this document applies to any platform.
OCR, Vote disk, clusterware

Purpose

In Oracle Clusterware 10g and 11g, there are two important cluster files, the Oracle Cluster Repository
(OCR) and the Voting Disk. Due to the nature of these files, they must be placed on shared devices; either
on cluster filesystems, NFS or shared raw devices.

The aim of this document is to provide a clear and concise method to create the OCR and voting disk on
raw devices using native OS utilities; i.e, without the need of extra licensable cluster software tools. The
intent of the document is to augment the platform specific documentation for Clusterware.

This document will focus on Oracle Clusterware 10g and 11g Release 1. Oracle Clusterware 11gR2
provides the capability to store the cluster files in ASM. Please check the Oracle documentation, Oracle®
Database 2 Day + Real Application Clusters Guide 11g Release 2, for 11g Release 2 installation
instructions.

Scope and Application

The procedures will be presented for the following operating systems:

o Linux
o AIX
o HP-UX
o Solaris
o Tru64

Placement of Voting disk and OCR Files in Oracle RAC 10g and 11gR1

Please also reference:Document ID 428681.1OCR / Vote disk Maintenance Operations (ADD/REMOVE/REPLACE/MOVE), including moving from RAW Devices to Block Devices

Linux–

Prior to installing Oracle CRS, the raw devices that will house the OCR and voting disk files, must be
defined and initialized using the following procedure. Note, this procedure is applicable for RH/OEL 4 and
above as well as SUSE 9 and above. Additionally, this section does not describe how to setup Linux
multipathing, if multipathing is employed, then use the appropriate the multipath device name in place of
the devices listed below

1. Determine which disk devices will be used store the OCR and voting disk file.

2. Execute cat /proc/partitions command on all nodes to verify that the selected devices are available on nodes of the cluster.

3. Run fdisk command to create and label 280 MB partition for each the OCR and voting disk you plan on deploying.

4. Re-run cat /proc/partitions to verify that the partitions are available.

$ cat /proc/partitions
8 33 32098 sdc1 1023842 0 2047682 528080 1023708 0 2047416
681320 0 0
8 65 2048256 sde1 23151853 0 557694461 15652280 3991671 0
122778038 100

In our example, the /dev/sdc1 and /dev/sde1 will be used for OCR and for voting disk; respectively.

5. Initialize the devices using dd.
OCR disk device

dd if=/dev/zero of=/dev/sdc1 bs=125829120 count=1

Voting disk

dd if=/dev/zero of=/dev/sde1 bs=20971520 count=1

6. For each raw device that you specified in the rawdevices file, enter commands similar to the
following to set the owner, group, and permissions on the device file. The root.sh script will to
reset to the appropriate permissions.
OCR

chown root:dba /dev/sdc1
chmod 0640 /dev/sdc1

Voting disk

chown oracle:dba /dev/sde1
chmod 0660 /dev/sde1

7. When installing Oracle CRS, OUI will request the location of the OCR and voting disk. List the appropriate disk names as requested.
For more details see:
http://download.oracle.com/docs/cd/B28359_01/rac.111/b28252/toc.htm

AIX –

Prior to installing Oracle CRS, the raw devices that will house the OCR and voting disk files, must be
defined and initialized using the following procedure. Note this procedure uses raw partitions for these
files and not shared-concurrent logical volumes. The latter requires HACMP.

1. Determine which disk device will be used store the OCR file and voting disk. In our example, the /dev/rhdisk1 and /dev/hdisk2 will be used for the OCR and voting disk; respectively. Make sure that the devices chosen have PVIDs on them.

/usr/sbin/lspv

If no PVID exists, then alter the disk(s)

/usr/sbin/chdev –l hdiskn –a pv=yes

Note, the disk and its PVID must be visible and consistent between nodes. If the device name associated with the PVID for a disk that you want to use is different on any node, you must create a new device file for the disk on each of the nodes using a common unused name. In this case, create a dummy disk; e.g., hdisk99 using the mknod command on both the nodes with the minor and major number of the actual disk being used for OCR or voting disk. This will ensure the disk name is the same across all the nodes.

2. To allow concurrent IO access to this disk device and prevent the device driver from
locking the hdisks with a reservation on open, a no reservation flag must be set. Use the following chdev command to disable this reservation.
All MPIO-capable (ESS, DS8000, DS6000 devices):

chdev -l hdisk1 –a reserve_policy=no_reserve
chdev -l hdisk2 –a reserve_policy=no_reserve

for EMC (Symettrix &Clariion), HDS, IBM DS4000, and non-MPIO capable
devices, perform the following:

chdev -l hdisk1 –a reserve_lock=no
chdev -l hdisk2 –a reserve_lock=no

3. Change the permissions and ownership on the two devices.
OCR disk device

chown root:dba /dev/rhdisk1
chmod 0640 /dev/rhdisk1

Voting disk

chown oracle:dba /dev/rhdisk2
chmod 0660 /dev/rhdisk2

When installing Oracle CRS, OUI will request the location of the OCR and voting disk. List the appropriate disk device paths as listed in Step 1.

HP–

Prior to installing Oracle CRS, the raw devices that will house the OCR and voting disk files, must be
defined and initialized using the following procedure. Note, the following procedures are only necessary if
the devices in question appear as different device names (SCSI paths). If the device names are same, then
skip to step 5.

1. Determine which disk will be used store the OCR and voting disk file.

2. Execute ioscan -fnC disk command on all nodes of cluster to obtain the hardware path and Instance number for the selected devices.

On node1:

ioscan -fnC disk

This will generate output similar to the following:

disk 46 255/255/0/2.2 sdisk CLAIMED DEVICE HSV100 HP
/dev/dsk/c12t2d2 /dev/rdsk/c12t2d2

On node2:

ioscan -fnC disk

This will generate output similar to the following (note the differing devices name and Instance numbers between the two nodes).

disk 47 255/255/0/2.2 sdisk CLAIMED DEVICE HSV100 HP
nb /dev/dsk/c14t2d2 /dev/rdsk/c14t2d2

3. Create new devices using the above same devices:

mksf -C disk -I 47 -H 255/255/0/2.2 /dev/dsk/c12t2d2

and for the raw device:

mksf -C disk -I 47 -H 255/255/0/2.2 -r /dev/rdsk/c12t2d2

4. Re-running ioscan -fnC disk, provides the following output:

disk 47 255/255/0/2.2 sdisk CLAIMED DEVICE HSV100 HP
/dev/dsk/c12t2d2 /dev/rdsk/c12t2d2
/dev/dsk/c14t2d2 /dev/rdsk/c14t2d2

5. Initialize the devices using dd.
OCR disk device

dd if=/dev/zero of=/dev/rdsk/c12t2d2 bs=125829120 count=1

Voting disk

dd if=/dev/zero of=/dev/rdsk/c14t2d2 bs=20971520 count=1

6. Change the permissions and ownership on the two devices.
OCR disk device

chown root:dba /dev/rdsk/c12t2d2
chmod 0640 /dev/rdsk/c12t2d2

Voting disk

chown oracle:dba /dev/rdsk/c14t2d2
chmod 0660 /dev/rdsk/c14t2d2

7. When installing Oracle CRS, the OUI will request the location of the OCR and voting disk. List the appropriate disk device paths as listed in Step 1.

Solaris–

Prior to installing Oracle CRS, raw devices that will house the OCR and voting disk files, must be defined
and initialized using the following procedure:

1. Determine which disk devices will be used store the OCR and voting disk file. Use the format command to list the devices, and select the appropriate device. Create 280 MB partitions for each OCR/Vote you plan on deploying. Note, the disk partition to be used must skip the first 1Mb to avoid overwriting the disk VTOC.

2. Make sure the other nodes can see those devices. If the other nodes see those devices at a different path, then common symbolic links or mknod command must be used to point to the same location.

3. Initialize the devices using dd.

dd if=/dev/zero of=/dev/rdsk/c3t4d12s6 bs=125829120 / count=1
dd if=/dev/zero of=/dev/rdsk/c3t4d15s6 bs=20971520 / count=1

4. Change the permissions and ownership on the two devices.
OCR disk device

chown root:dba /dev/rdsk/c3t4d12s6
chmod 0640 /dev/rdsk/c3t4d12s6

Voting disk

chown oracle:dba /dev/rdsk/c3t4d15s6
chmod 0660 /dev/rdsk/c3t4d15s6

5. When installing Oracle CRS, the OUI will request the location of the OCR and voting disk. List the appropriate disk names as requested.

Tru64 –

Prior to installing Oracle CRS, the directories that will house the OCR and voting disk files, must be
identified. Since HP TruCluster software is required for CRS/RAC installation, it is recommended that
TruCluster Cluster Filesystem (CFS) be implemented. CFS provides single-image view of devices;
therefore, the CRS files can be located on the cluster filesystem.

1. Determine the path and name of the OCR and voting disk. For example

/oradata1/crs/ocr.dat
/oradata1/crs/voting_disk

2. Change the permissions and ownership on the two devices.
OCR disk device

chown root:dba /oradata1/crs/ocr.dat
chmod 0640 /oradata1/crs/ocr.dat

Voting disk

chown oracle:dba /oradata1/crs/voting_disk
chmod 0660 /oradata1/crs/voting_disk

3. When installing Oracle CRS, the OUI will request the location of the OCR and voting disk. List the appropriate disk device paths as listed in Step 1.

References

NOTE:428681.1- OCR / Vote disk Maintenance Operations: (ADD/REMOVE/REPLACE/MOVE)

------------------------------------------------------------------------------