David Muegge’s Blog

PowerShell, Log Parser and other stuff.

HDS AMS 2000 Storage Resource Reporting with PowerShell

November 29th, 2009 tagsUncategorized userdmuegge commentsno comments

Welcome!

I have been creating a few PowerShell scripts for use with the HDS AMS 2000 array. One thing I found I needed was a quick way to look at DP(Dynamic Provisioning) pools, raid groups, and LUNs. I also wanted to be able to see the associations and filter easily. Since I am working on a new deployment I have been creating Raid Groups and Luns often. I needed a quick way to see what I currently had while creating new resources.

I created a PowerShell script that would quickly show existing resources by raid group or DP pool. It also uses nickname info for the devices that are maintained in three csv files(LU_Nicknames.csv,RG_Nicknames.csv,DP_Nicknames.csv). These are simple comma delimited text files which contain the ID and nickname of each resource. The files are updated as storage resources are added. This allows me to easily identify the resources and to filter for specific devices.

The script executes three HSNM2 CLI commands and reads the information into object form. The LUN information is then shown grouped by raid group or DP pool.

Here is the output with the nickname search parameter set to “DB”. This will return all database resources based on the naming standard. If this is left null it will return all resources.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

PS D:\DTools\Scripts\PS\HDS-SNM2> D:\DTools\Scripts\PS\HDS-SNM2\get-LunsByDPRG.ps1
Nickname Search: DB
 
-- DP Pool  Number: 0  Nickname: DB_SQL_OLTP  Raid Level: 5( 5D+1P)  Type: SAS  Capacity: 1986.0 GB  Used: 734.0 GB ----------------------------------------------------------------
 
 
lu                                      nickname                                capacity                                stripesize                              status
--                                      --------                                --------                                ----------                              ------
3                                       DB01_OLTP_DATA                          300.0                                   256                                     Normal
20                                      DB01_OLTP_DATA                          300.0                                   256                                     Normal
22                                      DB01_OLTP_DATA                          300.0                                   256                                     Normal
29                                      DB01_DSS_DATA_TEMP                      600.0                                   256                                     Normal
 
 
 
-- RAID GROUP  Number: 4  Nickname: DB_LOG  Raid Level: 1+0( 2D+2D)  Type: SAS  Capacity: 535.7 GB  Free: 0.0 GB -------------------------------------------------------------------
 
 
lu                                      nickname                                capacity                                stripesize                              status
--                                      --------                                --------                                ----------                              ------
5                                       DB01_OLTP_LOG                           267.0                                   64                                      Normal
21                                      DB01_OLTP_LOG                           268.7                                   64                                      Normal
 
 
 
-- RAID GROUP  Number: 7  Nickname: DB_OS  Raid Level: 5( 3D+1P)  Type: SAS  Capacity: 398.4 GB  Free: 183.4 GB --------------------------------------------------------------------
 
 
lu                                      nickname                                capacity                                stripesize                              status
--                                      --------                                --------                                ----------                              ------
9                                       DB01_OS_DATA                            215.0                                   256                                     Normal
 
 
 
-- RAID GROUP  Number: 8  Nickname: DB_LOG  Raid Level: 1+0( 2D+2D)  Type: SAS  Capacity: 535.7 GB  Free: 535.7 GB -----------------------------------------------------------------

Here is the script:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160

# -- Setup login info and SNM2 environment ------------------
. ./start-session.ps1
Set-Location $env:STONAVM_HOME
 
# -- Import Nickname tables ---------------------------------------
$LUNicknames = Import-Csv $SCRIPTHOME\LU_Nicknames.csv
$RGNicknames = Import-Csv $SCRIPTHOME\RG_Nicknames.csv
$DPNicknames = Import-Csv $SCRIPTHOME\DP_Nicknames.csv
 
# -- Get storage resource info -----------------------------------
$luresults = & ./auluref.exe -unit $DEFAULTARRAY -g -nosublu -totalsize
$rgresults = & ./aurgref.exe -unit $DEFAULTARRAY -g
$dpresults = & ./audppool.exe -unit $DEFAULTARRAY -refer -g
 
# -- Setup object collections -----------------------------------
$allrgdata = @()
$allludata = @()
$alldpdata = @()
 
# -- Get LUN data -----------------------------------------------------------
foreach($luresult in $luresults){
 
	if ($luresult -match 'LU\s+Capacity\s+Size\s+Group\s+Pool\s+Level\s+Type\s+Status'){
 
		$LUDataRead = $TRUE
 
	}elseif($LUDataRead){
 
		if($luresult -match '(\d+)\s+(\d+\.\d+)\sGB\s+(\d+)KB\s+(\d+|N/A)\s+(\d+|N/A)\s+(([5,6]|1\+0)\(\s+\d+D\+\d+[D,P]\))\s+(SAS|SATA)\s+(.+)'){
			$ludata = "" | Select lu,nickname,capacity,stripesize,raidgroup,dppool,raidlevel,type,status
			$ludata.lu = $matches[1]
			$Nickname = $LUNicknames | where {$_.LU -eq $ludata.lu} | select nickname
			$ludata.nickname = $Nickname.nickname
			$ludata.capacity = $matches[2]
			$ludata.stripesize = $matches[3]
			$ludata.raidgroup = $matches[4]
			$ludata.dppool = $matches[5]
			$ludata.raidlevel = $matches[6]
			$ludata.type = $matches[8]
			$ludata.status = $matches[9]
			$allludata += $ludata
		}
	}
}
 
# --  Get Raid Group data --------------------------------------------------------
foreach($rgresult in $rgresults){
 
	if ($rgresult -match 'Group\s+Level\s+Groups\s+Type\s+Total\sCapacity\s+Free\sCapacity\s+Priority\s+Status'){
 
		$RGDataRead = $TRUE
 
	}elseif($RGDataRead){
 
		if($rgresult -match '(\d+)\s+(([5,6]|1\+0)\(\s+\d+D\+\d+[D,P]\))\s+(\d+)\s+(SAS|SATA)\s+(\d+\.\d+)\sGB\s+(\d+\.\d+)\sGB\s\(\s*\d+\.{1}\d+%{1}\)\s+(.+\s.+)\s+(.+)'){
 
			$rgdata = "" | Select rg,nickname,raidlevel,raidconfig,paritygroups,type,totalcapacity,freecapacity,priority,status
			$rgdata.rg = $matches[1]
			$Nickname = $RGNicknames | where {$_.RG -eq $rgdata.rg} | select nickname
			$rgdata.nickname = $Nickname.nickname
			$rgdata.raidlevel = $matches[2]
			$rgdata.raidconfig = $matches[3]
			$rgdata.paritygroups = $matches[4]
			$rgdata.type = $matches[5]
			$rgdata.totalcapacity = $matches[6]
			$rgdata.freecapacity = $matches[7]
			$rgdata.priority = $matches[8]
			$rgdata.status = $matches[9]
			$allrgdata += $rgdata
		}	
	}
}
 
 
# --  Get DP pool data -------------------------------------------------------------
foreach($dpresult in $dpresults){
 
	if ($dpresult -match 'Pool\s+Level\s+Total\sCapacity\s+Consumed\sCapacity\s+Type\s+Status'){
 
		$DPDataRead = $TRUE
 
	}elseif($DPDataRead){
 
		if($dpresult -match '(\d+)\s+(([5,6]|1\+0)\(\s+\d+D\+\d+[D,P]\))\s+(\d+\.\d+)\sGB\s+(\d+\.\d+)\sGB\s+(SAS|SATA)\s+(.+)'){ 
 
			$dpdata = "" | Select dp,nickname,raidlevel,totalcapacity,capacityused,type,status
			$dpdata.dp = $matches[1]
			$Nickname = $DPNicknames | where {$_.DP -eq $dpdata.dp} | select nickname
			$dpdata.nickname = $Nickname.nickname
			$dpdata.raidlevel = $matches[2]
			$dpdata.totalcapacity = $matches[4]
			$dpdata.capacityused = $matches[5]
			$dpdata.type = $matches[6]
			$dpdata.status = $matches[7]
			$alldpdata += $dpdata
		}	
	}
}
 
 
# -- Get search string -----------------------------
$searchstring = Read-Host "Nickname Search"
 
# -- Display LUN's by DP pool --------------------------------------------------------------
foreach($dppool in $alldpdata){
 
	$dpnum = $dppool.dp
	$dpcapacity = $dppool.totalcapacity
	$dpused = $dppool.capacityused
	$dpnickname = $dppool.nickname
	$dplevel = $dppool.raidlevel
	$dptype = $dppool.type
 
	if($dpnickname -like "*$searchstring*"){
		$message = "-- DP Pool  Number: $dpnum  Nickname: $dpnickname  Raid Level: $dplevel  Type: $dptype  Capacity: $dpcapacity GB  Used: $dpused GB -"
		$charadd = (180 - $message.length)
		for($x=1;$x -le $charadd;$x++){$message += "-"}
		Write-Host ""
		Write-Host $message
		Write-Host ""
	}
	if($searchstring){
		$allludata | where {(($_.nickname -like "*$searchstring*") -and ($_.dppool -eq $dpnum))} | FT -property lu,nickname,capacity,stripesize,status
	}
	else
	{
		$allludata | where {$_.dppool -eq $dpnum} | FT -property lu,nickname,capacity,stripesize,status
	}	
}
 
# -- Display LUN"s by raid group -------------------------------------------------
foreach($rgroup in $allrgdata){
 
	$rgnum = $rgroup.rg
	$rgcapacity = $rgroup.totalcapacity
	$rgfree = $rgroup.freecapacity
	$rgnickname = $rgroup.nickname
	$rglevel = $rgroup.raidlevel
	$rgtype = $rgroup.type
 
	if($rgnickname -like "*$searchstring*"){
		$message = "-- RAID GROUP  Number: $rgnum  Nickname: $rgnickname  Raid Level: $rglevel  Type: $rgtype  Capacity: $rgcapacity GB  Free: $rgfree GB -"
		$charadd = (180 - $message.length)
		for($x=1;$x -le $charadd;$x++){$message += "-"}
		Write-Host ""
		Write-Host $message
		Write-Host ""
	}
	if($searchstring){
		$allludata | where {(($_.nickname -like "*$searchstring*") -and ($_.raidgroup -eq $rgnum))} | FT -property lu,nickname,capacity,stripesize,status
	}
	else
	{
		$allludata | where {$_.raidgroup -eq $rgnum} | FT -property lu,nickname,capacity,stripesize,status
	}
}
 
# -- CD back to script directory and set window title ---------------------
Set-Location $SCRIPTHOME
$Host.UI.RawUI.WindowTitle = "DP-RG-LU Info"

The script uses the start-session.ps1 file to establish connectivity with the HDS array. Additional information regarding the use of this include file can be found at this post. Then the script executes HSNM2 CLI commands to return information on DP pools raid groups and LUNS. The script uses regular expressions to parse the output and convert it into objects. It also reads in the nickname files and adds the data to the custom objects.

The objects are then output using the built-in PowerShell formating engine with a little custom formating thrown in for the group headers.

Here is an example nickname file:

1
2
3
4
5
6
7

RG,Nickname
0,MG_OS
1,AP_OS
2,DT_OS
3,DB_SQL_OLTP
7,DB_OS
8,DB_LOG

I suppose this may not be necessary with the use of Device Manager, but I am still learning it and I could not quite get this view with it. Besides I am more of a scripting kind of guy. I also really like the output of this script as it gives me just the view of the array I need when I am allocating new storage and setting up new resources. I use this script in conjuction with two other scripts for creating LUN’s and raid groups. I plan to post those scripts soon.

Hope this helps,

Dave

HDS AMS 2000 Performance Analysis with PowerShell and PowerGadgets

October 23rd, 2009 tagsAMS 2000, PowerShell userdmuegge commentsno comments

Welcome!

It has been a while since my last post due to a very busy schedule with a SAN and virtualization project.

I have been working on an implementation of a HDS AMS 2500 midrange array for a VMWare vShere 4 environment. So far everything has been working and performing well. The management software included with the HDS AMS 2000 series array is SNM2(Storage Navigator Modular 2), A java based web application. This software also has a command line version which appears to be pretty comprehensive. It consists of a series of DOS executables, which can be run from PowerShell. There are a series of scripts I have been working on for viewing and creating storage resources on the array. I will share many of these in future posts. In this post I want to share some scripts I have written to extend the functionality of the performance monitoring utility in SNM2.

The base functionality of the array allows you to capture performance statistics to a text file. The file can be captured manually or automatically for a specified time period and interval down to one minute. One text file is produced per capture or all captures can be written to one file. Also, I believe based on the information I read in the SNM2 manual you can do some graphing with the web interface, but it requires an additional license and personally I think the PowerGadgets graphs are better.

The 4 scripts I have started with are get-performance_processor.ps1, get-performance_ports.ps1, get-performance_raidgroups.ps1, get-performance_luns.ps1, which do pretty much what they say and produce the following PowerGadgets charts.

Chart

The chart group is a tabbed interface which allows you to tab through the controllers and ports/RG/LU/Procs depending on the script being used. Each script generates different groups of charts for different performance counters. I have not implemented all of the performance counters just the ones which are most important to me now. I will be improving these scripts over time and implementing more counters. Here is an example of how the script works.

Script

After executing the script it will ask whether or not to collect data, if yes it will prompt for interval in minutes and time period. If no it will use previously collected data in the default output directory. Next it will ask to list data in text output. Then it will prompt for generation of each group of charts for ports, raid groups, luns or processors depending on the script run.

Now to the script. All of the scripts rely on the start-session.ps1 script and also require a password file be set for logging into the array. Additionally, an array has to be registered.

Example 1 shows a PowerShell script which will register an array and set the admin password.

1
2
3
4
5
6
7

$env:STONAVM_HOME="C:\Program Files\Storage Navigator Modular 2 CLI"
$env:STONAVM_ACT="on"
$env:STONAVM_RSP_PASS="on"
$env:LANG="en"
cd "C:\Program Files\Storage Navigator Modular 2 CLI"
./auunitadd -unit ARRAYNAME -ctl0 192.168.1.1 -ctl1 192.168.1.2
./auaccountenv -set -uid USERNAME

You will need to replace ARRAYNAME, USERNAME and the IP Addresses for your environment.

 Example 2 shows the start-session PowerShell script which defines environmental information.

1
2
3
4
5
6
7
8
9

# Comand Enviroment
$env:STONAVM_HOME="C:\Program Files\Storage Navigator Modular 2 CLI"
$env:STONAVM_ACT="on"
$env:STONAVM_RSP_PASS="on"
$env:LANG="en"
 
# Global
$SCRIPTHOME = "C:\Scripts\HDS-SNM2"
$DEFAULTARRAY = "ARRAYNAME"

You will need to change the paths and ARRAYNAME for your environment.

Example 3 shows the get-performance_processor.ps1 script

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98

 
# -- User Defined Constants ---------------------------
$LOG_PATH = "C:\Scripts\HDS-SNM2\pfmlog"
 
# -- Modify controller and core options as needed -----
$Controllers = @(0,1)
$cores = @("X","Y")
 
# -- Modifications below this line should not be required ---------------------------------------
 
 
 
 
# -- Setup environment ------------------------------
$testsnapin = $null
$testsnapin = get-pssnapin | where { $_.Name -eq "PowerGadgets"}
if(-not $testsnapin){add-pssnapin -Name PowerGadgets}
. ./start-session.ps1
Set-Location $env:STONAVM_HOME
 
 
# --  Object to hold processor usage data ------------------------
$allprocusagedata = @()
 
# -- Processing flags -------------------------------------
$PROCUsageDataRead = $FALSE
 
 
# -- Specify whether to run new collection if not existing files are used --
Write-Host ”Collect Data Y or N...”
$modekey = $Host.UI.RawUI.ReadKey(”NoEcho,IncludeKeyDown”)
if($modekey.Character.ToString().ToLower() -eq "y"){
	$collectiontime = Read-Host "Collection Time/Count in minutes"
	Write-Host "Collecting Data..."
	./auperform.exe -unit $DEFAULTARRAY -auto 1 -count $collectiontime -path $LOG_PATH -pfmstatis	
}
 
 
# -- Get data from files --------------------------------------------
Write-Host "Analyzing Data..."
$perffiles = Get-ChildItem $LOG_PATH
 
foreach($perffile in $perffiles){
	$filedata = Get-Content $LOG_PATH\$perffile
	foreach($line in $filedata){
 
		# Get collection times
		if($line -match '^(\d{4}\/(?:0[1-9]|1[0-2]?)\/(?:0[1-9]|[12][0-9]|3[01]?)\s(?:0[0-9]|1[0-9]|2[0-3]?)\:(?:0[0-9]|1[0-9]|2[0-9]|3[0-9]|4[0-9]|5[0-9]?)\:(?:0[0-9]|1[0-9]|2[0-9]|3[0-9]|4[0-9]|5[0-9]?))\s-\s(\d{4}\/(?:0[1-9]|1[0-2]?)\/(?:0[1-9]|[12][0-9]|3[01]?)\s(?:0[0-9]|1[0-9]|2[0-3]?)\:(?:0[0-9]|1[0-9]|2[0-9]|3[0-9]|4[0-9]|5[0-9]?)\:(?:0[0-9]|1[0-9]|2[0-9]|3[0-9]|4[0-9]|5[0-9]?))$'){
 
			$collectfrom = $matches[1]
			$collectto = $matches[2]				
		}
 
		# Get Proc Usage Data
		if ($line -match 'CTL\s+Core\s+Usage'){
 
			$PROCUsageDataRead = $TRUE
 
		}elseif($PROCUsageDataRead){
 
			$line -match '([0,1]{1})\s+([X,Y]{1})\s+(\d+)' | Out-Null 
			$procusagedata = "" | Select ctl,core,usage,datetime
			$procusagedata.ctl = $matches[1]
			$procusagedata.core = $matches[2]
			$procusagedata.usage = $matches[3]
			$procusagedata.datetime = $collectto
			$allprocusagedata += $procusagedata
 
			if(($procusagedata.ctl -eq 1) -and ($procusagedata.core -eq "Y")){$PROCUsageDataRead = $FALSE}
		}
	}
}
 
 
# -- List Data ---------------------------------
Write-Host ”List Processor Data Y or N...”
$modekey = $Host.UI.RawUI.ReadKey(”NoEcho,IncludeKeyDown”)
if($modekey.Character.ToString().ToLower() -eq "y"){
	$allprocusagedata | ft -prop *
}
 
 
# -- Create Processor Usage Chart Group ----------------------
Write-Host ”Show Processor Usage Charts Y or N...”
$modekey = $Host.UI.RawUI.ReadKey(”NoEcho,IncludeKeyDown”)
if($modekey.Character.ToString().ToLower() -eq "y"){
	Write-Host "Generating Charts..."
	foreach($Controller in $Controllers){
		foreach($core in $cores){
			$ChartTitle = "Processor Usage Controller:$Controller Core:$core"
			$allprocusagedata | where {($_.core -eq $core)-and ($_.ctl -eq $Controller)} | sort datetime | Select core,usage,datetime | `
			Out-Chart -Values usage -Series_0_Text "Processor Usage %" -Label datetime `
			-Title $ChartTitle -Gallery Lines -Group "PROCUSAGE" -Name $ChartTitle -Caption "Processor Usage %"
		}
	}
}
 
Set-Location $SCRIPTHOME

This script collects the data from the array in separate files. Reads the pertinent data from the files and transforms it into object form which is fed into the PowerGadgets out-chart cmdlet. The other three scripts are longer as they digest more information.

To use these scripts you will need PowerShell, PowerGadgets( this a pay product with a free trial ), SNM2 CLI, and the script files attached to this post. Oh and an HDS AMS 2000 array.

Here are the script dowmloads
start-Session.txt
get-performance_processor.txt
get-performance_ports.txt
get-performance_raidgroups.txt
get-performance_luns.txt

Save the files to your script directory and change the extensions to .ps1

I hope someone finds this useful.

Regards,

Dave

Exchange DB Reporting with PowerShell and Log Parser

July 1st, 2009 tagsUncategorized userdmuegge commentsno comments

I ran across a useful post today as I was roaming through Google Analytics.

Using PowerShell, LogParser and PowerGadgets to get Exchange 2003 storage information – Part 1

Wes Stahler uses Log Parser and PowerShell to report on the free space in an Exchange Database.

This is a task I have done in the past. I will add this script to my toolkit.

Regards,

Dave

previous page