Solaris 10 disk mirroring using Solaris Volume Manager

 

Below are the steps to do disk mirroring using SVM.

• My primary Disk: c0t0d0s0
• My Second Disk:c0t1d0s0

 

1. Copy the partition table of the first disk to second disk(mirror disk)

bash-3.00# prtvtoc /dev/rdsk/c0t0d0s0 >/vtoc.bak
bash-3.00# fmthard -s /vtoc.bak /dev/rdsk/c0t1d0s0

2. Creating state database replicas on each disk

bash-3.00# metadb -a -f -c 3 c0t0d0s7
bash-3.00# metadb -a -f -c 3 c0t1d0s7

3. Check the state database

bash-3.00# metadb
        flags           first blk       block count
     a        u         16              8192            /dev/dsk/c0t0d0s7
     a        u         8208            8192            /dev/dsk/c0t0d0s7
     a        u         16400           8192            /dev/dsk/c0t0d0s7
     a        u         16              8192            /dev/dsk/c0t1d0s7
     a        u         8208            8192            /dev/dsk/c0t1d0s7
     a        u         16400           8192            /dev/dsk/c0t1d0s7

A state database replica contains configuration and state information about the meta devices.

4.Create the root slice mirror and its first sub-mirror 

bash-3.00# metainit -f d10 1 1 c0t0d0s0
d10: Concat/Stripe is setup
bash-3.00# metainit -f d20 1 1 c0t1d0s0
d20: Concat/Stripe is setup
bash-3.00# metainit d0 -m d10
d0: Mirror is setup

5. Next we need to run metaroot to prepare /etc/vfstab and /etc/system (do this only for the root slice)

Take a backup before run metaroot command

bash-3.00# cp vfstab vfstab.orig
bash-3.00# cp system system.orig

bash-3.00# metaroot d0

6. create swap slice mirror and its first sub-mirror

bash-3.00# metainit -f d11 1 1 c0t0d0s1
d11: Concat/Stripe is setup
bash-3.00# metainit -f d21 1 1 c0t1d0s1
d21: Concat/Stripe is setup
bash-3.00# metainit d1 -m d11
d1: Mirror is setup

7. Now my HOME partition

bash-3.00# metainit -f d12 1 1 c0t0d0s3
d12: Concat/Stripe is setup
bash-3.00# metainit -f d22 1 1 c0t1d0s3
d22: Concat/Stripe is setup
bash-3.00# metainit d2 -m d12
d2: Mirror is setup

 

8. My VAR partition

bash-3.00# metainit -f d13 1 1 c0t0d0s4
d13: Concat/Stripe is setup
bash-3.00# metainit -f d23 1 1 c0t1d0s4
d23: Concat/Stripe is setup
bash-3.00# metainit d3 -m d13
d3: Mirror is setup

 

9. My OPT partition

bash-3.00# metainit -f d14 1 1 c0t0d0s5
d14: Concat/Stripe is setup
bash-3.00# metainit -f d24 1 1 c0t1d0s5
d24: Concat/Stripe is setup
bash-3.00# metainit d4 -m d14
d4: Mirror is setup

10. Modify vfstab to mount all mirrors after boot.
The entry for the root device (/) has already been altered by the metaroot command we executed before.

#device         device          mount           FS      fsck    mount   mount
#to mount       to fsck         point           type    pass    at boot options
#
fd      -       /dev/fd fd      -       no      -
/proc   -       /proc   proc    -       no      -
/dev/md/dsk/d1  -       -       swap    -       no      -
/dev/md/dsk/d0  /dev/md/rdsk/d0 /       ufs     1       no      -
/dev/md/dsk/d3  /dev/md/rdsk/d3 /var    ufs     1       no      -
/dev/md/dsk/d2  /dev/md/rdsk/d2 /home   ufs     2       yes     -
/dev/md/dsk/d4  /dev/md/rdsk/d4 /opt    ufs     2       yes     -
/devices        -       /devices        devfs   -       no      -
sharefs -       /etc/dfs/sharetab       sharefs -       no      -
ctfs    -       /system/contract        ctfs    -       no      -
objfs   -       /system/object  objfs   -       no      -
swap    -       /tmp    tmpfs   -       yes     -

11.Reboot the system

bash-3.00#lockfs -fa
bash-3.00# sync;sync;init 6

12. Now Attach the sub-mirrors to all mirrors

bash-3.00# metattach d0 d20
d0: submirror d20 is attached
bash-3.00# metattach d1 d21
d1: submirror d21 is attached
bash-3.00# metattach d2 d22
d2: submirror d22 is attached
bash-3.00# metattach d3 d23
d3: submirror d23 is attached
bash-3.00# metattach d4 d24
d4: submirror d24 is attached

 

You can use metastat to track the mirroring progress

bash-3.00#metastat

Networker free reporting tool NSR_REPORTER

 

NSR Reporter is an excellent free Networker reporting tool developed by Thierry Faidherbe, you can download it from below link.

http://users.skynet.be/networker_reporter/index.html

You can see the full documentation of this tool on above link and also the author is very supportive, you will definitely get answer for your mails.

I am running this tool weekly basis and configured to send the tabular report in an excel sheet to the management, I will explain it below. I am using blat.exe here for sending the attachment.

1. Install NSR_REPORTER

2. Go to NRU500\data\servername\config folder and open the file smtp_daily_excel_report_by_user.txt

3. Add your servers in the file as below, I have arranged my clients based on roles.

You can add on below format

servername1;outputfilename1

servername2;outputfilename2

So it will create report for my servers on separate excel sheets based on outputfilename

For Example:-

###DB Servers -–> output file name will be as nsr_reporter_servername_status_for_DBServers.xls###
sqlsrev1;DBServers
sqlserv2;DBServers
oraclesrv1:DBServers

### File server--> output file name will be nsr_reporter_servername_status_for_FileServers.xls###
Fileserv1;FileServers
FileServ2;FileServers

### Web Servers-->output file name will be nsr_reporter_servername__status_for_WebServers.xls ###
Web1;WebServers
Web2;WebServers

4. Run the reporting tool

NSR_REPORTER_5.0.exe -s bkpserv -q "1 week ago" -D MM/DD/YYYY -graph -nodns –noicmp

5.After it completed successfully  you can see the report file under C:\NRU500\data\servername\export folder

here it’s the content of my file nsr_reporter_servername__status_for_WebServers.xls

It will have client summary tab and detailed tab.

6.Now download and install blat.exe

blat -install <smart hostname>

Create a .bat file to send the mail as below format

blat.exe -to user1@company.com, user2@company.com -subject "Weekly Backup Report" -f fromaddress@company.com -attach C:\NRU500data\servername\export\nsr_reporter_servername__status_for_WebServers.xls

7. Schedule the nsr_reporter script and .bat file for sending mail using windows scheduler task according to your requirements.

Redirecting an Exchange 2010 standalone server snapshot backup to another database or to the recovery database (RDB)

This topic describes how to redirect a snapshot backup of an Exchange 2010 standalone server to another database or the recovery database (RDB). To redirect an Exchange 2010 standalone server snapshot backup to the recovery Database

1. The database or the recovery database must already exist.

To restore to the RDB, create the RDB on an Exchange server, if necessary and leave the RDB dismounted

To create recovery database, logon to Exchange Management Shell and execute the following command

New-MailboxDatabase -Recovery -Name RDB1 -Server EXCH-MBCH-SRV

Or

New-MailboxDatabase -Recovery -Name RDB -Server EXCH-MBCH-SRV

-EdbFilePath "C:\Recovery\RDB\RDB.EDB" -LogFolderPath "C:\Recovery\RDB"

You can see the Recovery DB on Exchange GUI now

1. Open the Backup, Archive, and Restore.

2. Click File > Specify NetBackup Machines and Policy Type.

3. Select  Policy type MS-Exchange-Server

4. Click File > Select Files and Folders to Restore > from Normal Backup

5. From the NetBackup History pane, select one of the following:

· The last full backup, or

· The last full backup and all subsequent differential backups, or

· The last full backup and the last cumulative backup

6. In the All Folders pane, expand Microsoft Information Store.

7. Select the database you want to restore.

8. Click Actions > Restore.

9. Change the destination path to another database or to the RDB:

· Click the General tab.

· Select Restore everything to a different location.

In the Destination box, provide the name of the alternate database you want to restore to. Or provide the name of the RDB you previously created. Exchange 2010 does not automatically redirect a database to its RDB, if it exists.

10. Click the Microsoft Exchange tab.

11. Check Commit after last backup set is restored.

If you choose to restore backup images separately, you must check Commit after last backup set is restored only when you restore the last incremental backup set.

13 Click Start Restore.

Restore Mailbox from Recovery Database

After restoring the database backup into recovery database we can move the required mailbox into exchange database. To perform this identify the required mailbox and perform proper backup before starting the mailbox restore.

Launch the Exchange Management Shell and execute the following commands

Restore-Mailbox -Identity <mail box name> -RecoveryDatabase RDB

Example:

Redirecting an Exchange 2010 Database Availability Group (DAG) snapshot backup to another database or to the recovery database (RDB)

To redirect an Exchange 2010 snapshot backup to another database or to the recovery database

1. Manually suspend replication. This step applies for any snapshot provider you use

2. Perform all steps of Exchange 2010 standalone server explained above.

Networker backup reverse DNS error when multiple NICs on backup client

 

Have you ever came across this scenario?

If you have a backup client with multiple NICs , backup server may be able to reach only one of the NIC and if you ping the host name you will be getting reply from IP configured on other NIC

See the image below

Here Networker give you a reverse DNS lookup error while it try for a reverse lookup with the IP 172.16.10.22 as the hostname resolution is on IP 10.10.100.22.

If you are facing above issue then configure backup as below;

Step 1: On client SQLSERVER create a hosts file (or on DNS) entry as below

This way the IP 172.16.10.22 will have a dummy hostname

Step 2: On Networker server create host file (or on DNS) entry as below

Now if you ping SQLSERVER or SQLSERVER_bkp you will get reply from 172.16.10.22

Step 3: On Networker configuration

Create a client with name SQLSERVR

Step 4: On Globals (1 of 2) tab add aliases as below

You can run the backup successfully now.

If your client is on a Cluster then add the backup IP(172.16.10.22) as a cluster resource and add the dummy hostname on both cluster nodes.

Backup Selection list

Backup selection is always confusing, there is no point of taking a full backup of Windows, you cannot restore c:\Windows or most of the Application binaries so if your OS crashed then you have to reinstall OS and reinstall the Application binaries.Similarly other OS also having same scenarios.

If you affectively prepare your backup selection list then you will save the backup window and backup media.

I have made a backup selection list for my organization, this is based on OS and Application, this may not be accurate but probably it will help you to have a good start for your backup selection list.

Windows 2003/2008

• System State (This includes Quarum if it is on Windows Cluster(MSCS))
• User Data

Windows 2003/2008 with IIS Web Server

• System State Backup
• Certificate backup (Export certificate to local drive and copy to backup media)
• MetaData(use IISBack.vbs tool,copy it on local drive and then copy to backup media)
• %Windir%\System32\inetsrv
• Web Contents

Unix/Linux

• /etc
• User Data
• Web server configuration (if any)
• Web contents (if any)
• Application mount points

MSSQL Server

• All DBs
• Transaction logs

MS-Exchange Server

CAS:

• AD
• IIS -Metafile
• C:\Program Files\Microsoft\Exchange Server\V14

HUB

• Active Directory
• Transport queue databases
• C:\Program Files\Microsoft\Exchange Server\V14(Message tracking logs)

EDGE

• C:\Program Files\Microsoft\Exchange Server\V14\Scripts
• Export Edge configuration to xml the copy to backup media(.\ExportEdgeConfig.ps1 -cloneConfigData C:\EdgeConfig\ex2-edgeconfig.xml)

Mail box Server

• Mail box store
• Public store

SharePoint Server

• Active Directory
• Full Farm backup

TMG/ISA Server

• Export the configuration to local drive and copy to backup media.

Oracle

• Entire Database
• Archive logs

These are the common OS and Applications used in our environment, please note each Application will have it’s on method of backup and also please read the Application documentation for vendor backup recommendations.

Networker-duplicate the backup using nsrclone

nsrclone duplicates your saveset, the duplicate copy can be placed in a different media.

Here I will explain three different scenarios.

1. You want to copy entire volume to another media.

create a new Clone pool and assign the media.

if you want to clone entire contents of volume 0060 then

nsrclone –n <ClonePoolName> volume

Ex:- nsrclone -b “TestClonePool” 0060

 

2. You want to copy only one saveset of one server from a particular date

Use mminfo to get the save set id, example below

[root@mybkpsrv]# mminfo -avot | grep 04/19/2010 | grep AppServer

000060 AppServer 04/19/2010 10:30:17 AM 1209 KB 2444045309 cb full C:\Configs

Note the saveset id 2444045309

now run

nsrclone -s <backup server> -b <ClonePoolName> -S <save set id>

Ex: nsrclone -s mybkpsrv -b "TestClonePool" -S 2444045309

 

3. You want to clone entire backup taken on a particular date.

Copy all save set id of that date to a file

[root@mybkpsrv]# mminfo -avot | grep 04/19/2010 | |awk '{ print $8 }' >saveset.txt

This will copy all saveset id of 04/19/2010 to saveset.txt

now run

nsrclone -s <backup server> -b <ClonePoolName> –S –f <file>

Ex:- nsrclone -s mybkpsrv -b "TestPool" -S -f saveset.txt

You also can also set a different retention period for your clone as below

nsrclone -s mybkpsrv -b "TestPool" -y "6 months” -S -f saveset.txt

This will clone entire save sets with 6 months retention

Clariion LUN Expansion on Navisphere.

Claiion LUN expansion can be done online; it will create a Meta LUN combining the LUNs

LUN expansion features:

• A metaLUN is created by combining LUNs
• Dynamically increase LUN capacity, it can be done on-line while host I/O is in progress
• A Meta LUN can be further expanded
• Meta LUN will appear as a single LUN to hosts
• Meta LUN can be used with MirrorView, SnapView, or SAN Copy

Below are the steps to expand a LUN, I am expanding 450GB LUN presented to a Windows Server adding another 100GB to make it 550GB.

1. Create a new 100GB LUN, this can be created in same Raid Group or another Raid Group (Must be same Raid type)

2.Right click the LUN you wants to expand and select Expand, in my case it is LUN 153 which is 450GB size.

3. This will open a Wizard window, click on Next

4.Select whether you want Striping or Concatenation, please read the details of both on this window, it is self explanatory

5. Select the LUN you need to expand to, it will show the summary on the bottom.

6.Choose the capacity required, you may always choose Maximum Capacity.

7. Click Finish.

8. You can see the new Meta LUN under LUN Folders-MetaLUNs in Navisphere.

9. Now in Windows if you check the size of the drive still it show the same 450GB, if you open the Disk Manger you can see the total space 550 GB which is divided as 450GB (old space) plus added 100GB (Expanded space) as separate unused space.

You need to combine both space using DiskPart utility which comes with Windows.

10. Open Command Prompt

Type DiskPart then enter,then List Disk

You can see now Disk-1 is 550 GB disk and Free space showing 100GB

11. Type List volume and Select your volume to expand , my volume is volume 0, so select volume 0 then type extend and enter.

12.Check the disk space now, you can see the disk space is expanded now.

Please note you cannot extend a system partition also you cannot extend if pagefile is configured on this partition, you need to remove page file and reboot windows to extend the partition.

Netbackup, Networker and CommVault Simpana

I works with a hosting company and we are using both Symantec Netbackup and EMC Networker in our company, I have been working with these backup products for almost 5 yrs.

We recently did a POC of CommVault Simpana, I am very much impressed with simpana.

Both Netbackup and Networker has it’s on advantages and disadvantages, Below are the con’s of these products which I am experiencing.

Con’s of EMC Networker

• Seems they lost their focus on Networker,still they don't have an agent for Exchange 2010 and Domino 8.x!!!!.
• Frequent changes in the licenses, no more Exchange server module and domino module, EMC says Exchange 2010 will be supported by NMM and domino by NetWorker Module for Databases and Applications. Still no response from licensing team for my queries on how I can convert my exiting Exchange and Lotus domino license.
• Poor reporting, advanced reporting(EBA) is expensive.
• De-duplication is so expensive.
• Customer support is very bad(Networker Support).

Con’s of Symantec Netbackup

• Message level backup in Exchange 2010 is possible but only via GRT.
• Poor reporting features, advanced reporting (Ops center Analytics)is expensive also so complex.
• Archiving required enterprise vault and it is very expensive.
• Backup selection is based on Policy not for each client level.
• Licenses are expensive, in active/passive cluster, the passive node also required a license, for ex: if you have an MSSQL active/passive cluster you need to buy 2 SQL agent licenses.

I am so happy with Netbackup other than above issues I am facing, being hosting company all above are so important for us probably it’s not for others.

Our POC was mainly focused on these areas..

1. Exchange Server Message Level Backup.
2. Tight integration with Cloud computing & VMs.
3. De-Duplication support.
4. Easy deployments of backup agents.
5. Backup over WAN link.
6. User friendliness.
7. Ability to Archive Mails, SPS Docs and File servers.
8. Legal holds & other governance policies is in backup.
9. Powerful reporting.

Simpana works awesome on above areas, some of the interesting features I have noticed:-

• MYSQL support, Networker and Netbackup doesn't support this yet.
• De-duplication is works very well, very good compressing mechanism.
• Exchange  Server Message level backup possible, Netbackup support this only via GRT which  required disk storage.
• Remote deployment of agent is so easy compared to others.
• Built in SRM(Storage Resource manager), this is great feature.
• VM monitoring and utilization reports along with Simpana.
• Built in Archiving solutions for Exchange, SPS & File systems.
• Possible to restore only File Permissions.
• Powerful reporting features, a variety of reports.
• Very user friendly backup and recovery GUI for OS and Applications, Oracle,SQL, SAP, Exchange, VMs, and SPS backup and recovery is so easy.

Con’s of CommVault Simpana

I am very happy with this product after the POC but still I cannot judge simpana, we decided to replace EMC Networker with Simpana probably I will be able to tell you more about it in 2 months time.

Networker client install on Linux

Prerequisites

Make sure that communication between client and Networker server is in place. Also add required dns entries or hosts file entries for hostname resolution.

Download the Networker software for your OS from http://powerlink.emc.com

Extract the software

Install package using rpm command

rpm –ivh lgtoclnt-7.xxx.i686.rpm

Start Networker client /etc/init.d/networker start

Failed dependencies??

When you install Networker client on linux sometimes it fails with dependencies error, it will ask for below packages.

openmotif

libXp.so.6

If you are on a linux which support ‘yum’ package manager then you can resolve it using below command, you don't have to download and install above dependency packages

yum localinstall --nogpgcheck lgtoclnt-xx.xx.i686.rpm

this will install the the Networker client without dependency packages.

Networker client install on AIX

Networker client installation on AIX is very simple.

Prerequisites

Make sure that communication between client and Networker server is in place. Also add required dns entries or hosts file entries for hostname resolution.

Download the Networker for AIX from the http://powerlink.emc.com

Execute smit or smit install on AIX console

Select Software Installation and Maintenance

Select Install and Update Software

Select Install Software

Input the source file path and press Esc + 4 to list the files

Select LGTOnw.clnt

Enter to install the software

Start the service

make sure the Networker client is running ps –ef | grep nsr

Clariion LUN Migration

The rules of  LUN Migration on Clariion

• The Destination LUN should be equal in size or larger than source LUN
• A destination LUN can be on the same type of drives as the source, or a different type of drive. For example, you can migrate a LUN from Fibre Drives to ATA Drives .
• It can be same or different Raid type (Ex: you can migrate from Raid-5 to Raid 1/0)
• It can  be a regular or Meta LUN
• LUN migration is an online activity, it wont stop IO hence you don't have to bring down your services.

It is a two step process. First it is a block by block copy of a “Source LUN” to its new location “Destination LUN”. After the copy is complete, it then moves the “Source” LUNs location to its new place in the Clariion.

Below i am migrating a LUN from Raid-5 to Raid 1/0. Source is LUN-6 under Raid Group 1 and destination is LUN-29 under Raid Group 10

1. Create the destination LUN, this should be equal or bigger than source LUN, I have created an equal size of source

Now my source is under Raid Group1 and Destination under Raid Group 10

2. Right click the source LUN which is LUN 6 under Raid Group 1 and choose Migrate..

3.This will open a new window where you can select the destination, please note the source and destination LUN capacity

4. Select the LUN and click OK, click Yes on the warning window

5. Go the source LUN status, you will it is on Transitioning state

Check the properties, you can see the Migration status

6. Finally LUN is migrated and can see the source LUN id under the Raid Group 10, now it is Raid 5