On our server "sirius" we are using the "Sun Enterprise Volume Manager" (current version 2.5)

With this software we can manage our disks. We can use some facilities like "mirroring" and RAID. In the following I will describe the most important concept and also what has been done on our system.
 

Sun Enterprise Volume Manager 2.5 User's Guide

Sun Enterprise Volume Manager 2.5 System Administrator's Guide

• Definitions
• Physical Objects
• Volume Manager Objects
• Relationships between VxVM Objects
• RAID
• our hardware
• our directories concept
• manager's tasks
• add new disks to the Disk Array
• Creating a new disk group
• Removing a VM disk from a disk group
• Creating a new volume
• Resizing a File System

 

Physical objects are 'physical disks' and 'partitions'.
A physical disk is the storage device (media) installed in our machine. It does not necessarily be under control of the Volume Manager. In the following we will use the short PD for 'physical disk'. The naming convention is the usual in Solaris: c#t#d# where "#" can be any interger 0-9. The first letter ( c ) refers to the controller, the second ( t ) to the target in the corresponding SCSI bus, the third (d) defines the disk number . A forth letter (s) defines the slice (or partition) number (0-7). The s2 partition define the whole disk.

Thus a partition is a part of a physical disk and is identified to the Volume manager as c#t#d#s#. It appears as rectangular icon within the physical disk icon which is itself a cylinder labeled PD.

Back to Content

Volume Manager Objects are:
• VM disks
• Disk Groups
• Subdisks
• Plexes
• Volumes
A VM disk is a contiguous area of disk space from which the Volume Manager allocates storage. It corresponds to AT LEAST one partition and is typically composed of a public region (from which storage is allocated) and a private region (where configuration information is stored). Appears as cylindrical icon labeled D.

A VM disk is one that is both under Volume manager control and assigned to a disk group.

A Disk Group is a collection of VM disks that share a common configuration. A common configuration consists of a set of records containing detailed information about existing Volume Manager objects, their attributes and their relationships. Volumes are created within a disk group and are restricted to using disks within that group. Each disk group is represented as a "View Window" instead of an icon.

A Subdisk is a set of contiguous disk blocks and represents the basic unit in which the Volume Manager allocates disk space. Each subdisk represents a specific portion of a VM disk, which is mapped to a specific region of a physical disk. Subdisks appear within VM disks as rectangular shaped icons. Subdisks are the lowest-level building blocks in a volume and need to be created in order to build a volume.

A plex consists of one or more subdisks located on one or more disks.The Volume Manager uses subdisks to build virtual entities called plexes (also referred as mirrors). There are three way that data can be organized on the subdisks that constitute a plex:

• Concatenation
• Striping (RAID 0)
• RAID-5
Concatenation maps data in a linear manner onto one or more subdisks in a plex. Concatenation using subdisks that reside on more than one VM disk is also called spanning. This can be useful if there is no sufficient space on any one disk and also for load balancing between disks and for head optimization on a particular disk.
A volume is a virtual disk device which appears to applications, databases, and file systems like a physical disk partition, but does not have the physical limitations of a physical disk partition. A volume consists of one or more plexes, each holding a copy of the data in the volume. Thus a volume has at least one plex with at least one associated subdisk. Volumes appear as cyl;indrical icons labeled V. RAID-5 volumes are labeled RAID-5 instead of V.

Back to Content

Relationships Between VxVM Objects
Volume Manager objects are of little use until they are combined to build volumes:
• A volume is composed of one or more plexes
• One or more subdisks are combined top form plexes
• VM disks are placed under VxVM control and grouped into disk groups
• encapsulation is a process that converts existing partitions on a specific disk to volumes. /etc/vfstab entries are modified correspondly.
Back to Content
RAID implementations
• A Redundant Array of Inexpensive Disks (RAID) is a disk array in which duplicate information about the data stored in it makes it possible to regenerate the data in the event of a disk failure. The Volume Manager supports the following levels of RAID:
• RAID-0 (striping)
• RAID-1 (mirroring)
• Raid-0 plus RAID-1 (striping and mirroring)
• RAID-5
Striping (RAID-0) is a tecnique of mapping data so that the data is interleaved among two or more physical disks: a striped plex contains two or more subdisks, spread out over two or more physical disks. Data is allocated alternately and evenly to the subdisks of a striped plex. Thw subdisks are grouped into "columns" with each physical disk limited to one column. Each column contains one or more subdisks and can be derived from one or more physical disks.

Mirroring (RAID-1) is a technique of using multiple mirrors (plexes) to duplicate the information contained in a volume. Although a volume can have a single plex, at least two plexes are required to provide redundancy of data. Each of these plexes should contain disk space from different disks in order for redundancy to be effective.

Striping plus mirroring offers the benefits of spreading data across multiple disks while providing redundancy of data. The striped plex and its mirror must be allocated from separate disks.

RAID-5 provides redundancy thorough the use of parity by performing an exclusive OR (XOR) procedure on the data. The resulting parity is then written to the volume.In a traditional RAID-5 array disks are organized in columns and rows and requires all disks (partitions), columns and rows to be of equal size. Due to the virtual nature of its disks and other objets, the Volume Manager does not need to use rows. Instead it uses columns consisting of variable length subdisks.

Back to Content

Our hardware

Our server is a SUN Ultra 2 with 2 UltraSPARC 168 MHz, 256 MB RAM, node name is "sirius-ifh" (we just call it "sirius"). The server is connected through two optical fiber interface to a SUN Enterprise Network Array "A 5000" (cosy name "photon_1") in which there are several 9 GB disks.

The management of the disk space is done with the Sun Enterprise Volume Manager software.

Back to Content

Our directories concept

The mounting point of all file systems which can be exported is under "/export".

File systems with user's data (home directories) are mounted under /export/home

Home directories are distributed among many "disks". Historically this was really a disk unit, i.e. better a "partition" on a disk unit. Now with the Volume Manager we are merely working with virtual disks. The actual data can be spread among different physical disks to improve reliability (RAID-5)

The user will not notice any difference between the old and the new concept, as the mounting point is the same as before. In the virtual space we distributed the home directories according to the following rules:

1. all home directories on sirius are stored as RAID-5 file system
2. some users have their home directory on a "dedicated" machine
3. software is stored in a non-redundand way
Back to Content

Manager's tasks

• Add new disks to the Disk Array (called PHOTON)


1. just insert the new disks into the enclosure (Hot Swap capability, no need to shut down)
2. execute the following commands on a shell as root:
/bin/sh ; cd / ; _INIT_RECONFIG=1 ; export _INIT_RECONFIG
/etc/rcS.d/S50drvconfig ; /etc/rcS.d/S60devlinks
3. now instruct the PHOTON to recognize the new disks:
/usr/sbin/vxdctl enable
4. check that it sees the new disks with  vxdisk list
5. use vxdiskadm to interactively add the disks:
• Select an operation to perform: 1
• Select disk devices to add: c3t0d0
• Which disk group: users
• Use a default name for the disk: no
• Add disk as a spare disks for users: no
• Continue with operation?: y
• Encapsulate disk device?: n
• Instead of encapsulating, initialize? y
• Enter disk name for c3t0d0: sirius61
 
6. Create the mount directory:
   mkdir /export/home/sirius61 ; chmod 775 !$ ; chown root:sys !$
7. use the vxva utility to generate a new RAID5 volume:
• Click with the middle mouse botton the disks involved (they get green)
• Pull_down menu Basic_Ops ==> volume operations ==> Create ==> RAID5
• Fill in the form:
Volume name: sirius61
Volume size: 2000m
Create log subdisk: No
Number of column: 5  (if you selected 5 disks)
Stripe unit size: 32
Create file system: Yes
Mount file system: Yes
Mount point: /export/home/sirius61
Mount automatically: Yes

 
• The vxva utility runs the following commands:

vxassist -g users make sirius61 2000m layout=raid5,nolog nstripe=5 \
stripeunit=32 users11 users12 users13 users14 users15
mkfs /dev/vx/rdsk/users/sirius61 4096000
mount  /dev/vs/dsk/users/sirius61 /export/home/sirius61
8. check /etc/vfstab:
/dev/vx/dsk/users/sirius61  /dev/vx/rdsk/users/sirius61  /export/home/sirius61  ufs 3 yes -
9. edit /etc/dfs/dfstab:
share -F nfs -o rw=felder:gaas -d "RAID-5 on photon_1" /export/home/sirius61
10. edit /etc/nisdata/auto_home_ifh:
sirius61     -rw     sirius:/export/home/sirius61
11. enable incr_backup on jabba: becomes user "ifhbup" on sirius and execute:
{sirius:[ifhbup]:34}% start_bkup_for sirius sirius61

Back to Content
 
• Creating a new disk group
• use the vxdg command as follows:
vxdg init group_name your_disk_name=devicename
Example: vxdg init software_one  soft18=c2t18d0
 
• Creating a new volume
• Go to the view window of the disk group in which you want to perform this operation
• if you want to involve a specific disk click on the VM disk icon
• From the Basic-Ops menu, select Volume Operations and then "create" and specify the type of volume (simple, stripe,mirror, RAID-5, etc). The corresponding form appears.
• Enter Volume_name, size and other information. Then click "Apply"
• Or using the command line:


/usr/sbin/vxassist -g dsk_grp -U fsgen make vol_name 2000m

This command will generate a file system : /dev/vx/dsk/dsk_grp/vol_name

Back to Content
 

• Resizing a File System
• Go to the view window of the disk group in which you want to perform this operation.
• Select the volume whose file system has to be resized.
• From the Basic-Ops menu, select File System Operations, then Resize.
• Complete the File System Resize Form.
• Select Apply to resize the file system.
• Removing a VM disk from a Disk group
• Go to view window of the disk group considered
• select VM disk to be removed
• From the Advanced-Ops menu , select Disk group then Remove Disk
• Create a mirror
• you should have a disk in the disk_group you are working on
• click to the disk and click to the partition where you want to put the mirror
• Choose "Basic Operations" => Volume Operations" => Add Mirror => simple mirror

Back to Content