EMC Isilon Platform API and PowerShell Part II

Posted on March 24, 2013
by dmuegge

In my last post I talked about the Isilon REST based platform API. I have been experimenting more with the Isilon API and PowerShell. I thought I would share the progress so far.

I decided to create a PowerShell module to leverage the functionality exposed by the Isilon platform API. The module provides some advanced functions, script cmdlets if you prefer, to provide API access. Right now I have some ‘get’ cmdlets and a lot of work to do before I have full coverage of the API. Maybe I can catch up by the time there is full coverage of the Isilon functionality in the platform APIJ

The module uses basic authentication at this time as I am still working on other authentication options. The first step in using the module is to download it and place it in your modules directory. Then load the module and create a password file for logging on to the Isilon cluster. The New-PasswordFile command creates an encrypted file containing the supplied password. This file is used to supply authentication automatically. The file can only be used by the user who was logged on when the file was created.

import-module IsilonPlatform
 
PS C:Windowssystem32> New-PasswordFile -Path c:temp -Filename is7test01-admin_pwd.txt
Password: ********
 
 
    Directory: C:temp
 
 
Mode                LastWriteTime     Length Name
----                -------------     ------ ----
-a---         3/10/2013   7:06 PM          0 pwdfile.txt

import-module IsilonPlatform

PS C:Windowssystem32> New-PasswordFile -Path c:temp -Filename is7test01-admin_pwd.txt

Password: ********

Directory: C:temp

Mode LastWriteTime Length Name

---- ------------- ------ ----

-a--- 3/10/2013 7:06 PM 0 pwdfile.txt

Once this is complete the following code will load a console with the cmdlets and setup authentication.

# Load Module
Import-Module IsilonPlatform
 
# Ignore certificate errors
Disable-CertificateValidation
 
# Set required connection information
Set-ISIAPIConnectionInfo -username "admin" -passwordfile "C:Tempis7test01-admin_pwd.txt" -baseurl "https://is7test01:8080"
 
# Show module commands
Get-Command -Module IsilonPlatform

# Load Module

Import-Module IsilonPlatform

# Ignore certificate errors

Disable-CertificateValidation

# Set required connection information

Set-ISIAPIConnectionInfo -username "admin" -passwordfile "C:Tempis7test01-admin_pwd.txt" -baseurl "https://is7test01:8080"

# Show module commands

Get-Command -Module IsilonPlatform

This code uses the password file we created earlier. Once it is executed we see the available Isilon cmdlets and we have a console to execute Isilon commands and scripts. I put this code into a script to launch an Isilon management console.

DM - Isilon Platform API Toolkit
 
CommandType     Name                                               ModuleName
-----------     ----                                               ----------
Function        Disable-CertificateValidation                      IsilonPlatform
Function        Get-ISIADSProvider                                 IsilonPlatform
Function        Get-ISIADSProviderDomainControllers                IsilonPlatform
Function        Get-ISIADSTrustedDomains                           IsilonPlatform
Function        Get-ISIFileProvider                                IsilonPlatform
Function        Get-ISIGroup                                       IsilonPlatform
Function        Get-ISIGroupMembers                                IsilonPlatform
Function        Get-ISINFSExports                                  IsilonPlatform
Function        Get-ISINFSExportsSummary                           IsilonPlatform
Function        Get-ISINFSSettings                                 IsilonPlatform
Function        Get-ISIProviderSummary                             IsilonPlatform
Function        Get-ISIQuota                                       IsilonPlatform
Function        Get-ISIQuotaLicense                                IsilonPlatform
Function        Get-ISIQuotaNotification                           IsilonPlatform
Function        Get-ISIQuotaReport                                 IsilonPlatform
Function        Get-ISIQuotaReportAbout                            IsilonPlatform
Function        Get-ISIQuotaReportSettings                         IsilonPlatform
Function        Get-ISIQuotaSummary                                IsilonPlatform
Function        Get-ISISMBOpenFiles                                IsilonPlatform
Function        Get-ISISMBSessions                                 IsilonPlatform
Function        Get-ISISMBSettings                                 IsilonPlatform
Function        Get-ISISMBShares                                   IsilonPlatform
Function        Get-ISISMBSharesSummary                            IsilonPlatform
Function        Get-ISISnapshot                                    IsilonPlatform
Function        Get-ISISnapshotLicense                             IsilonPlatform
Function        Get-ISISnapshotSummary                             IsilonPlatform
Function        Get-ISIUser                                        IsilonPlatform
Function        Get-ISIUserMappingRules                            IsilonPlatform
Function        Get-ISIUserMemberOf                                IsilonPlatform
Function        Get-ISIZone                                        IsilonPlatform
Function        Get-PasswordFromFile                               IsilonPlatform
Function        New-PasswordFile                                   IsilonPlatform
Function        Set-ISIAPIConnectionInfo                           IsilonPlatform

DM - Isilon Platform API Toolkit

CommandType Name ModuleName

----------- ---- ----------

Function Disable-CertificateValidation IsilonPlatform

Function Get-ISIADSProvider IsilonPlatform

Function Get-ISIADSProviderDomainControllers IsilonPlatform

Function Get-ISIADSTrustedDomains IsilonPlatform

Function Get-ISIFileProvider IsilonPlatform

Function Get-ISIGroup IsilonPlatform

Function Get-ISIGroupMembers IsilonPlatform

Function Get-ISINFSExports IsilonPlatform

Function Get-ISINFSExportsSummary IsilonPlatform

Function Get-ISINFSSettings IsilonPlatform

Function Get-ISIProviderSummary IsilonPlatform

Function Get-ISIQuota IsilonPlatform

Function Get-ISIQuotaLicense IsilonPlatform

Function Get-ISIQuotaNotification IsilonPlatform

Function Get-ISIQuotaReport IsilonPlatform

Function Get-ISIQuotaReportAbout IsilonPlatform

Function Get-ISIQuotaReportSettings IsilonPlatform

Function Get-ISIQuotaSummary IsilonPlatform

Function Get-ISISMBOpenFiles IsilonPlatform

Function Get-ISISMBSessions IsilonPlatform

Function Get-ISISMBSettings IsilonPlatform

Function Get-ISISMBShares IsilonPlatform

Function Get-ISISMBSharesSummary IsilonPlatform

Function Get-ISISnapshot IsilonPlatform

Function Get-ISISnapshotLicense IsilonPlatform

Function Get-ISISnapshotSummary IsilonPlatform

Function Get-ISIUser IsilonPlatform

Function Get-ISIUserMappingRules IsilonPlatform

Function Get-ISIUserMemberOf IsilonPlatform

Function Get-ISIZone IsilonPlatform

Function Get-PasswordFromFile IsilonPlatform

Function New-PasswordFile IsilonPlatform

Function Set-ISIAPIConnectionInfo IsilonPlatform

Now that we have a console with the commands loaded we can use them to get information from the Isilon system. Here is an example of a command to return all Isilon groups.

PS C:Windowssystem32> get-isigroup
 
 
dn               :
dns_domain       :
domain           : UNIX_GROUPS
generated_gid    : False
gid              : @{id=GID:0}
id               : wheel
member_of        :
name             : wheel
provider         : lsa-file-provider:System
sam_account_name : wheel
sid              : @{id=SID:S-1-22-2-0}
type             : group
 
dn               :
dns_domain       :
domain           : UNIX_GROUPS
generated_gid    : False
gid              : @{id=GID:14}
id               : ftp
member_of        :
name             : ftp
provider         : lsa-file-provider:System
sam_account_name : ftp
sid              : @{id=SID:S-1-22-2-14}
type             : group
 
dn               :
dns_domain       :
domain           : UNIX_GROUPS
generated_gid    : False
gid              : @{id=GID:31}
id               : guest
member_of        :
name             : guest
provider         : lsa-file-provider:System
sam_account_name : guest
sid              : @{id=SID:S-1-22-2-31}
type             : group
 
.
.
.

PS C:Windowssystem32> get-isigroup

dn :

dns_domain :

domain : UNIX_GROUPS

generated_gid : False

gid : @{id=GID:0}

id : wheel

member_of :

name : wheel

provider : lsa-file-provider:System

sam_account_name : wheel

sid : @{id=SID:S-1-22-2-0}

type : group

dn :

dns_domain :

domain : UNIX_GROUPS

generated_gid : False

gid : @{id=GID:14}

id : ftp

member_of :

name : ftp

provider : lsa-file-provider:System

sam_account_name : ftp

sid : @{id=SID:S-1-22-2-14}

type : group

dn :

dns_domain :

domain : UNIX_GROUPS

generated_gid : False

gid : @{id=GID:31}

id : guest

member_of :

name : guest

provider : lsa-file-provider:System

sam_account_name : guest

sid : @{id=SID:S-1-22-2-31}

type : group

Here we can see all groups are returned with all properties. We also see the results are returned as PowerShell objects. This gives us all of the PowerShell goodness when working with the Isilon platform API. The next example uses PowerShell to display just the information we want in an easier to read format.

PS C:Windowssystem32> get-isigroup | select id,provider,domain
 
id                                             provider                                       domain
--                                             --------                                       ------
wheel                                          lsa-file-provider:System                       UNIX_GROUPS
ftp                                            lsa-file-provider:System                       UNIX_GROUPS
guest                                          lsa-file-provider:System                       UNIX_GROUPS
ifs                                            lsa-file-provider:System                       UNIX_GROUPS
admin                                          lsa-file-provider:System                       UNIX_GROUPS
Administrators                                 lsa-local-provider:System                      BUILTIN
Users                                          lsa-local-provider:System                      BUILTIN
Guests                                         lsa-local-provider:System                      BUILTIN
Backup Operators                               lsa-local-provider:System                      BUILTIN
Isilon Users                                   lsa-local-provider:System                      IS7TEST01

PS C:Windowssystem32> get-isigroup | select id,provider,domain

id provider domain

-- -------- ------

wheel lsa-file-provider:System UNIX_GROUPS

ftp lsa-file-provider:System UNIX_GROUPS

guest lsa-file-provider:System UNIX_GROUPS

ifs lsa-file-provider:System UNIX_GROUPS

admin lsa-file-provider:System UNIX_GROUPS

Administrators lsa-local-provider:System BUILTIN

Users lsa-local-provider:System BUILTIN

Guests lsa-local-provider:System BUILTIN

Backup Operators lsa-local-provider:System BUILTIN

Isilon Users lsa-local-provider:System IS7TEST01

The following example shows how we can query the groups, filter by type and control the display format. This example retrieves all users with the domain type of BUILTIN and displays the name and provider in a results table.

PS C:Windowssystem32> Get-ISIGroup | where domain -eq BUILTIN | select name,provider
 
name                                                                  provider
----                                                                  --------
Administrators                                                        lsa-local-provider:System
Users                                                                 lsa-local-provider:System
Guests                                                                lsa-local-provider:System
Backup Operators                                                      lsa-local-provider:System

PS C:Windowssystem32> Get-ISIGroup | where domain -eq BUILTIN | select name,provider

name provider

---- --------

Administrators lsa-local-provider:System

Users lsa-local-provider:System

Guests lsa-local-provider:System

Backup Operators lsa-local-provider:System

The cmdlets will also allow using the pipeline. Although, I do not have pipline functionality in all cmdlets yet. The user, group and provider cmdlets do. The following example shows a listing of all Isilon ADS BUILTIN groups and the members of each group.

PS C:Windowssystem32> Get-ISIGroup | where domain -eq BUILTIN | foreach {Write-Host $_.id; $_ | Get-ISIGroupMembers | select name,type | out-string}
 
Administrators
 
name                                                                  type
----                                                                  ----
LABdomain admins                                                     group
 
 
Users
 
name                                                                  type
----                                                                  ----
Authenticated Users                                                   wellknown
LABdomain users                                                      group
 
 
Guests
 
name                                                                  type
----                                                                  ----
Guest                                                                 user
 
 
Backup Operators

PS C:Windowssystem32> Get-ISIGroup | where domain -eq BUILTIN | foreach {Write-Host $_.id; $_ | Get-ISIGroupMembers | select name,type | out-string}

Administrators

name type

---- ----

LABdomain admins group

Users

name type

---- ----

Authenticated Users wellknown

LABdomain users group

Guests

name type

---- ----

Guest user

Backup Operators

Hopefully someone finds this interesting and I will try to provide more useful examples as I add functionality to the module. Here are the download links for the module and the console launch script. You will need to modify the console launch script(Start-IsilonPlatform.ps1) for your environment.

IsilonPlatform
Start-IsilonPlatform

This module is a work in progress so use at your own risk. I hope to provide more functionality soon and I hope to see EMC add more coverage to the API. I would really like to see more around cluster configuration and performance statistics. I think this is great functionality provided by EMC, more please!

Feedback on the module is welcome.

Regards,

Dave

Using the Isilon 7.0 ReST API with PowerShell

Posted on February 16, 2013
by dmuegge

EMC recently released Isilon 7.0 “Mavericks” version of the OneFS operating system. This release has many great new features, which you can read all about here and here. One of these great new Isilon features is the ReST API, which allows programmatic access to the platform. If you are not familiar with ReST, it stands for Representational State Transfer. This is a lightweight, platform independent and stateless method of programming web services.

PowerShell allows an easy method to access the Isilon ReST API. Working with ReST is a new for me, but I thought it might be useful for some to follow along while I am learning. Also, if anyone has tips for me on this process I welcome the knowledge.

The Isilon ReST API is not enabled by default. To enable the functionality it requires changing options on the HTTP settings page in the protocols section, see below.

Isilon_HTTP_Settings

The HTTP interface can use active directory authentication, but in this post I will use basic authentication and show examples of reading data from the cluster. I hope to show more advanced examples as I learn.

PowerShell v3 has some great built-in functionality for working with ReST API’s. The Invoke-RestMethod cmdlet is exactly the functionality required to leverage the Isilon ReST API. The first challenges when working with the API will be related to authentication and certificates. The Isilon cluster will use a self-signed certificate by default. This results in a certificate error when connecting via HTTPS and can be seen when connecting to the Isilon cluster via a browser. The following code will allow a work around to the problem by ignoring the error.

# Ignore SSL errors
add-type @"
using System.Net;
using System.Security.Cryptography.X509Certificates;

    public class IDontCarePolicy : ICertificatePolicy {
    public IDontCarePolicy() {}
    public bool CheckValidationResult(
        ServicePoint sPoint, X509Certificate cert,
        WebRequest wRequest, int certProb) {
        return true;
    }
}
"@
[System.Net.ServicePointManager]::CertificatePolicy = new-object IDontCarePolicy

# Ignore SSL errors

add-type @"

using System.Net;

using System.Security.Cryptography.X509Certificates;

public class IDontCarePolicy : ICertificatePolicy {

public IDontCarePolicy() {}

public bool CheckValidationResult(

ServicePoint sPoint, X509Certificate cert,

WebRequest wRequest, int certProb) {

return true;

}

[System.Net.ServicePointManager]::CertificatePolicy = new-object IDontCarePolicy

In a production environment the correct way to handle this would be to install a certificate issued by a trusted certificate authority. The next step is to setup a proper HTTP header for basic authentication.

# Set username and password
$username = '****'
$upassword = '****'

# Encode basic authorization header
$auth = $username + ':' + $upassword
$Encoded = [System.Text.Encoding]::UTF8.GetBytes($auth)
$EncodedPassword = [System.Convert]::ToBase64String($Encoded)
$headers = @{"Authorization"="Basic $($EncodedPassword)"}

# Set username and password

$username = '****'

$upassword = '****'

# Encode basic authorization header

$auth = $username + ':' + $upassword

$Encoded = [System.Text.Encoding]::UTF8.GetBytes($auth)

$EncodedPassword = [System.Convert]::ToBase64String($Encoded)

$headers = @{"Authorization"="Basic $($EncodedPassword)"}

Once this is complete all we have to do is build the proper URL and issue the request. The code below will retrieve and display the SMB and NFS settings of the cluster.

# Setup base address and objects
$baseurl = 'https://192.168.1.239:8080'

# Create full address of resource to be retrieved
$resource = '/platform/1/protocols/smb/settings/global'
$url = $baseurl + $resource

# Execute request and display SMB Settings
$SMBGlobal = Invoke-RestMethod -Uri $url -Headers $headers -Method Get 
$SMBGlobal.settings

# Create full address of resource to be retrieved
$resource = '/platform/1/protocols/nfs/settings/global'
$url = $baseurl + $resource

# Execute request and display NFS Settings
$NFSGlobal = Invoke-RestMethod -Uri $url -Headers $headers -Method Get
$NFSGlobal.settings

# Setup base address and objects

$baseurl = 'https://192.168.1.239:8080'

# Create full address of resource to be retrieved

$resource = '/platform/1/protocols/smb/settings/global'

$url = $baseurl + $resource

# Execute request and display SMB Settings

$SMBGlobal = Invoke-RestMethod -Uri $url -Headers $headers -Method Get

$SMBGlobal.settings

# Create full address of resource to be retrieved

$resource = '/platform/1/protocols/nfs/settings/global'

$url = $baseurl + $resource

# Execute request and display NFS Settings

$NFSGlobal = Invoke-RestMethod -Uri $url -Headers $headers -Method Get

$NFSGlobal.settings

The output from the above examples is shown below. As you can see this gives a quick concise view of the protocol settings.

access_based_share_enum     : False
audit_fileshare             : none
audit_global_sacl           : {}
audit_logon                 : all
dot_snap_accessible_child   : True
dot_snap_accessible_root    : True
dot_snap_visible_child      : False
dot_snap_visible_root       : True
enable_security_signatures  : False
guest_user                  : nobody
ignore_eas                  : False
onefs_cpu_multiplier        : 1
onefs_num_workers           : 0
require_security_signatures : False
server_string               : Isilon Server
service                     : True
srv_cpu_multiplier          : 4
srv_num_workers             : 0
support_netbios             : False
support_smb2                : True

lock_protection : 2
nfsv2_enabled   : True
nfsv3_enabled   : True
nfsv4_domain    : localdomain
nfsv4_enabled   : False
rpc_maxthreads  : 16
rpc_minthreads  : 16
service         : True

access_based_share_enum : False

audit_fileshare : none

audit_global_sacl : {}

audit_logon : all

dot_snap_accessible_child : True

dot_snap_accessible_root : True

dot_snap_visible_child : False

dot_snap_visible_root : True

enable_security_signatures : False

guest_user : nobody

ignore_eas : False

onefs_cpu_multiplier : 1

onefs_num_workers : 0

require_security_signatures : False

server_string : Isilon Server

service : True

srv_cpu_multiplier : 4

srv_num_workers : 0

support_netbios : False

support_smb2 : True

lock_protection : 2

nfsv2_enabled : True

nfsv3_enabled : True

nfsv4_domain : localdomain

nfsv4_enabled : False

rpc_maxthreads : 16

rpc_minthreads : 16

service : True

While this is only a simple example of retrieving data from the cluster, the possibilities are endless. When considering where we are in the transformation to cloud and automation. This type of enabling technology will be the foundation of great things to come.

Stay tuned…

Regards,

Dave

Merge Multiple EMC NAR files with PowerShell

Posted on December 18, 2012
by dmuegge

While working on a project the other day I found the need to merge multiple NAR files. The NaviSecCLI provides a method to merge two NAR files but does not allow an option to merge multiple files. I was searching on the web for methods to do this and ran across a couple of scripts.

The first script I found was done in VBScript http://blog.edgoad.com/2011/03/merging-multiple-emc-nar-files.html

The second script I found was bash for linux http://jslabonte.wordpress.com/2012/02/01/how-to-merge-nar-files/

I thought this is something that PowerShell can do much easier so here is a script to merge multiple NAR files. This script will require the NaviSecCLI to be installed to work properly.

$TempFile = "C:Tempnar_outTemp.nar"
$OutputFile = "C:Tempnar_outNAR_Merge.nar"
$NarFiles = Get-Childitem "C:Tempnar_in"

$filecount = 0
foreach($NFile in $NarFiles){

    if($filecount -eq 0){Copy-Item $NFile.FullName $OutputFile}Else{
        
        Copy-Item $OutputFile $TempFile
        Remove-Item $OutputFile
        ./NaviSECCli.exe analyzer -archivemerge -data $TempFile $NFile.FullName -out $OutputFile

    }
    

    $filecount ++
}

$TempFile = "C:Tempnar_outTemp.nar"

$OutputFile = "C:Tempnar_outNAR_Merge.nar"

$NarFiles = Get-Childitem "C:Tempnar_in"

$filecount = 0

foreach($NFile in $NarFiles){

if($filecount -eq 0){Copy-Item $NFile.FullName $OutputFile}Else{

Copy-Item $OutputFile $TempFile

Remove-Item $OutputFile

./NaviSECCli.exe analyzer -archivemerge -data $TempFile $NFile.FullName -out $OutputFile

}

$filecount ++

}

I hope someone finds this useful.

Regards,

Dave

EMC VNXe Performance Analysis with PowerShell Part II

Posted on December 2, 2012
by dmuegge

I appreciate the positive feedback I have received from the VNXePerformance module so far. I thought I would add to it and provide a script to generate a basic report. The script can be downloaded here.

The script will produce an HTML report and associated graphics with the following information.

Capacity Information system and pools(Total and Allocated)
- Maximum, Minimum, Average, Median
- Historical graphs for system and each pool
Bandwidth usage per protocol
- Maximum, Minimum, Average, Median
- Historical graphs
IOPS usage per protocol
- Maximum, Minimum, Average, Median
- Historical graphs

Here is a sample

The previous post used PowerGadgets for the charting functionality. This tool is not free and it is also not yet supported with PowerShell 3.0. To correct this issue I provided a function in this reporting script which uses the charting functionality in the .Net 4.0 framework. While this fixes the two issues mentioned it does require more work to use, but it will work well for our purposes here. This script uses the VNXePerformance.ps1 module from my previous post and a few new functions to produce an html report and associated graphic files. A command line example to run the script is shown below.

Start-VNXeHTMLPerformanceReport.ps1 –SQLiteDBLocation “c:TmpDM” –VNXeReportOutputLocation “c:TmpRPT” –HTMLReportName

1	Start-VNXeHTMLPerformanceReport.ps1 –SQLiteDBLocation “c:TmpDM” –VNXeReportOutputLocation “c:TmpRPT” –HTMLReportName

The script uses data provided by the VNXePerformance module and the functions in the script to format and write the report data. Here is a brief description of the functions used.

Out-DataTable – this function is used to convert the PSObject data provided as output from the module functions to the system.data.datatable type. This is required for databinding to produce charts.

Out-LineChart – This function provides chart generating functionality to produce a line chart based on provided datatable and generate a .png graphic file.

Get-SeriesRollup – This function creates summary data (maximum, minimum, average, median) for series data.

The following functions create HTML report output

ConvertTo-SeriesRollupHTML
Write-ChartHTML
Write-BlankHTMLTable
Write-HeaderHTMLTable

The first part of the script defines parameters, loads charting assembly, contains the functions declarations and module import.

Param([String]$SQLiteDBLocation="C:TempDB",[String]$VNXeReportOutputLocation="C:Temp",[String]$HTMLReportName="VNXeReport.html")

# Load the charting assembly 
[void][Reflection.Assembly]::LoadWithPartialName("System.Windows.Forms.DataVisualization")

# Imports
Import-Module -Name VNXePerformance 

function Out-DataTable…
function Out-LineChart…
function Get-SeriesRollup…
function ConvertTo-SeriesRollupHTML…
function Write-ChartHTML…
function Write-BlankHTMLTable…
function Write-HeaderHTMLTable…

Param([String]$SQLiteDBLocation="C:TempDB",[String]$VNXeReportOutputLocation="C:Temp",[String]$HTMLReportName="VNXeReport.html")

# Load the charting assembly

[void][Reflection.Assembly]::LoadWithPartialName("System.Windows.Forms.DataVisualization")

# Imports

Import-Module -Name VNXePerformance

function Out-DataTable…

function Out-LineChart…

function Get-SeriesRollup…

function ConvertTo-SeriesRollupHTML…

function Write-ChartHTML…

function Write-BlankHTMLTable…

function Write-HeaderHTMLTable…

The next portion of the sets the location of the SQLite database and begins the HTML report string.

# Set DB Location
Set-VNXeSQLiteLocation -Path $SQLiteDBLocation

# Open HTML Document
$HTMLReportText = "<html><body>"

# Capacity
$VNXeSystemTotalsDetail = Get-VNXeSystemTotalsDetail
$LastSystemTotalSpace = $VNXeSystemTotalsDetail | select total_space -Last 1
$LastSystemTotalSpaceGB = $LastSystemTotalSpace.total_space / 1024 / 1024 / 1024
$HTMLTemp = "<table><tr><td>System Total Usable Space GB: </td><td>" + $LastSystemTotalSpaceGB + "</td></tr></table>"
$HTMLReportText += $HTMLTemp

# Set DB Location

Set-VNXeSQLiteLocation -Path $SQLiteDBLocation

# Open HTML Document

$HTMLReportText = "<html><body>"

# Capacity

$VNXeSystemTotalsDetail = Get-VNXeSystemTotalsDetail

$LastSystemTotalSpace = $VNXeSystemTotalsDetail | select total_space -Last 1

$LastSystemTotalSpaceGB = $LastSystemTotalSpace.total_space / 1024 / 1024 / 1024

$HTMLTemp = "<table><tr><td>System Total Usable Space GB: </td><td>" + $LastSystemTotalSpaceGB + "</td></tr></table>"

$HTMLReportText += $HTMLTemp

The next portion of the script completes the report by using the VNXePerformance module to retrieve object data then output HTML using the script functions.

# System Space
$HTMLReportText += Write-BlankHTMLTable
$HTMLReportText += Write-HeaderHTMLTable "System Allocated Space Rollup"
$SystemAllocatedRollup = Get-SeriesRollup -InputObject $VNXeSystemTotalsDetail -Property "allocated_space"
$HTMLReportText += ConvertTo-SeriesRollupHTML -InputObject $SystemAllocatedRollup

# Capacity Charts
$HTMLReportText += Write-BlankHTMLTable
$ChartFullPath = $VNXeReportOutputLocation + "VNXeDiskSpace.png"
$VNXeSystemTotalsDetail | Out-LineChart -YValues "allocated_space,total_space" -XValue "timestamp" -Width 800 -Height 600 -ChartTitle "Historical Disk Space" -ChartType "png" -ChartFullPath $ChartFullPath
$HTMLReportText += Write-ChartHTML $ChartFullPath

# Pool Space Charts
$VNXePoolsDetail = Get-VNXePoolsDetail
$HTMLReportText += Write-BlankHTMLTable
$ExtremePoolDetail = $VNXePoolsDetail | Where {$_.pool_id -eq "extreme_performance"}
$ChartFullPath = $VNXeReportOutputLocation + "VNXeExtremePoolDiskSpace.png"
$ExtremePoolDetail | Out-LineChart -YValues "allocated_space,total_space" -XValue "timestamp" -Width 800 -Height 600 -ChartTitle "Historical Disk Space - Extreme Performance Pool" -ChartType "png" -ChartFullPath $ChartFullPath
$HTMLReportText += Write-ChartHTML $ChartFullPath
.
.
.

# SPB
$HTMLReportText += Write-BlankHTMLTable
$HTMLReportText += Write-HeaderHTMLTable "Flare SPB Read IO/Sec Rollup"
$FlareSPBIORollup =  Get-SeriesRollup -InputObject $VNXeBasicSummaryFlareSPBDetail -Property LUNDiskReadsPerSec
$HTMLReportText += ConvertTo-SeriesRollupHTML -InputObject $FlareSPBIORollup
$HTMLReportText += Write-HeaderHTMLTable "Flare SPB Write IO/Sec Rollup"
$FlareSPBIORollup =  Get-SeriesRollup -InputObject $VNXeBasicSummaryFlareSPBDetail -Property LUNDiskWritesPerSec
$HTMLReportText += ConvertTo-SeriesRollupHTML -InputObject $FlareSPBIORollup
$ChartFullPath = $VNXeReportOutputLocation + "VNXeFlareSPBIOPerSec.png"
$VNXeBasicSummaryFlareSPBDetail | Out-Linechart -YValues "LUNDiskReadsPerSec,LUNDiskWritesPerSec" -Xvalue "stored_timestamp" -Width 800 -Height 600 -ChartTitle "Flare IO/Sec - SPB" -ChartType "png" -ChartFullPath $ChartFullPath
$HTMLReportText += Write-ChartHTML $ChartFullPath

# System Space

$HTMLReportText += Write-BlankHTMLTable

$HTMLReportText += Write-HeaderHTMLTable "System Allocated Space Rollup"

$SystemAllocatedRollup = Get-SeriesRollup -InputObject $VNXeSystemTotalsDetail -Property "allocated_space"

$HTMLReportText += ConvertTo-SeriesRollupHTML -InputObject $SystemAllocatedRollup

# Capacity Charts

$HTMLReportText += Write-BlankHTMLTable

$ChartFullPath = $VNXeReportOutputLocation + "VNXeDiskSpace.png"

$VNXeSystemTotalsDetail | Out-LineChart -YValues "allocated_space,total_space" -XValue "timestamp" -Width 800 -Height 600 -ChartTitle "Historical Disk Space" -ChartType "png" -ChartFullPath $ChartFullPath

$HTMLReportText += Write-ChartHTML $ChartFullPath

# Pool Space Charts

$VNXePoolsDetail = Get-VNXePoolsDetail

$HTMLReportText += Write-BlankHTMLTable

$ExtremePoolDetail = $VNXePoolsDetail | Where {$_.pool_id -eq "extreme_performance"}

$ChartFullPath = $VNXeReportOutputLocation + "VNXeExtremePoolDiskSpace.png"

$ExtremePoolDetail | Out-LineChart -YValues "allocated_space,total_space" -XValue "timestamp" -Width 800 -Height 600 -ChartTitle "Historical Disk Space - Extreme Performance Pool" -ChartType "png" -ChartFullPath $ChartFullPath

$HTMLReportText += Write-ChartHTML $ChartFullPath

# SPB

$HTMLReportText += Write-BlankHTMLTable

$HTMLReportText += Write-HeaderHTMLTable "Flare SPB Read IO/Sec Rollup"

$FlareSPBIORollup = Get-SeriesRollup -InputObject $VNXeBasicSummaryFlareSPBDetail -Property LUNDiskReadsPerSec

$HTMLReportText += ConvertTo-SeriesRollupHTML -InputObject $FlareSPBIORollup

$HTMLReportText += Write-HeaderHTMLTable "Flare SPB Write IO/Sec Rollup"

$FlareSPBIORollup = Get-SeriesRollup -InputObject $VNXeBasicSummaryFlareSPBDetail -Property LUNDiskWritesPerSec

$HTMLReportText += ConvertTo-SeriesRollupHTML -InputObject $FlareSPBIORollup

$ChartFullPath = $VNXeReportOutputLocation + "VNXeFlareSPBIOPerSec.png"

$VNXeBasicSummaryFlareSPBDetail | Out-Linechart -YValues "LUNDiskReadsPerSec,LUNDiskWritesPerSec" -Xvalue "stored_timestamp" -Width 800 -Height 600 -ChartTitle "Flare IO/Sec - SPB" -ChartType "png" -ChartFullPath $ChartFullPath

$HTMLReportText += Write-ChartHTML $ChartFullPath

The final portion of the script closes out the html file and writes it to disk.

# Close HTML Document
$HTMLReportText += "</body></html>"

# Write HTML Report to file
$HTMLReportFullPath = $VNXeReportOutputLocation + "" + $HTMLReportName
$HTMLReportText | Set-Content -Path $HTMLReportFullPath

# Close HTML Document

$HTMLReportText += "</body></html>"

# Write HTML Report to file

$HTMLReportFullPath = $VNXeReportOutputLocation + "" + $HTMLReportName

$HTMLReportText | Set-Content -Path $HTMLReportFullPath

This should provide a good starting point to use for reporting. It has much room for improvement. Everyone please comment with information discovered about the SQLite data and information added to the report.

Start-VNXeHTMLPerformanceReport.zip
Regards,

Dave

EMC VNXe Performance Analysis with PowerShell

Posted on October 25, 2012
by dmuegge

Hello again, It has been a long time, but the day job takes precedence. Recently, I have been working on a little pet project I thought I would share. The EMC VNXe is a great entry level storage array which has been popular with SMB and remote office branch office deployments in the enterprise. I was recently asked to look at performance on one of these arrays. While investigating this request I discovered the VNXe has reduced functionality around monitoring compared to big brother, VNX. Performance information is collected differently and the VNXe does not currently have the ability to utilize Unisphere Analyzer.

While searching on the web I found the following article http://henriwithani.wordpress.com/2011/12/01/hidden-vnxe-performance-statistics/ . Henri shows where to find additional performance statistics on the VNXe. He also shows how to export the data to CSV for reporting. The VNXe performance data is stored in a public domain C library based SQL database called SQLite. While investigating the use of the SQLite database and tools I discovered there is an ADO.Net provider assembly available for SQLite. Since I am somewhat partial to PowerShell and any .NET object is accessable from PowerShell. I thought I would take a shot at building a PowerShell Module for looking at VNXe performance data.

Here is what I came up with, the VNXePerformance.psm1 PowerShell module. The module currently consists of 21 CmdLets. One CmdLet is to set the location of the SQLite database tables and the remaining 20 are to return data stored in those tables. The CmdLets currently work with the following three SQLite tables:

capacity.db
stats_basic_summary.db
stats_basic_default.db

These three tables appear to hold most of the interesting information so I concentrated on them first. The primary purpose of the module is to query the database and return PowerShell objects from the data. A secondary purpose of the module is to add calculated properties to some of those returned objects. The calculated properties currently implemented provide bytes per second for each of the three IP storage protocols supported NFS, CIFS, and iSCSI and also read and write IO’s per second. This was the first info I needed and was pretty simple to figure out. I plan to add additional calculated fields in the future as I learn more about the data.

Using the module

Install the SQLite ADO.Net Provider Assembly

The VNXePerformance module requires the SQLite ADO.Net Provider to be installed, which can be downloaded from the following page http://system.data.sqlite.org/index.html/doc/trunk/www/downloads.wiki

I recommend using the setup package for the framework and OS version you are using.

The next step is to download the module, unzip it to the modules folder. The proper modules folder can be identified with the $env:PSModulePath variable.

Download the module VnxePerformance.zip

Unzip the module to the PowerShell modules folder, open PowerShell and verify module is available. The VNXePerformance module should be listed when executing the get-module -ListAvailable command, if installed in the proper location.

PS C:&gt; get-module -ListAvailable

ModuleType Name                      ExportedCommands
---------- ----                      ----------------
Manifest   VNXePerformance           {Get-VNXeBasicSummaryGMSSPADetail, Get-VNXeSystemTotalsDetail, Get-VNXeB...

PS C:> get-module -ListAvailable

ModuleType Name ExportedCommands

---------- ---- ----------------

Manifest VNXePerformance {Get-VNXeBasicSummaryGMSSPADetail, Get-VNXeSystemTotalsDetail, Get-VNXeB...

Then set the following line to the location of the System.Data.SQLite.dll and save changes as necessary. The location defined in the module is the default installation path for the assembly if installed with the prepackaged installation.

$AssemblyPath="C:Program FilesSystem.Data.SQLite2010binSystem.Data.SQLite.dll"

1	$AssemblyPath="C:Program FilesSystem.Data.SQLite2010binSystem.Data.SQLite.dll"

We are now ready to use the module so open a PowerShell prompt and import the module.

PS C:&gt;Import-Module VNXePerformance

1	PS C:>Import-Module VNXePerformance

We can now use the get-command cmdlet to see the new functionality provided by our module.

PS C:&gt;Get-Command -Module vnxeperformance | select name
Name
----
Get-VNXeApplicationsDetail
Get-VNXeApplicationTotalsDetail
Get-VNXeBasicDefaultDart2Detail
Get-VNXeBasicDefaultDart3Detail
Get-VNXeBasicDefaultFlareSPADetail
Get-VNXeBasicDefaultFlareSPBDetail
Get-VNXeBasicDefaultOSSPADetail
Get-VNXeBasicDefaultOSSPBDetail
Get-VNXeBasicSummaryDart2Detail
Get-VNXeBasicSummaryDart3Detail
Get-VNXeBasicSummaryFlareSPADetail
Get-VNXeBasicSummaryFlareSPBDetail
Get-VNXeBasicSummaryGMSSPADetail
Get-VNXeBasicSummaryGMSSPBDetail
Get-VNXeBasicSummaryOSSPADetail
Get-VNXeBasicSummaryOSSPBDetail
Get-VNXeInitTimeDetail
Get-VNXePoolsDetail
Get-VNXePoolTotalsDetail
Get-VNXeSystemTotalsDetail
Set-VNXeSQLiteLocation

PS C:>Get-Command -Module vnxeperformance | select name

Name

----

Get-VNXeApplicationsDetail

Get-VNXeApplicationTotalsDetail

Get-VNXeBasicDefaultDart2Detail

Get-VNXeBasicDefaultDart3Detail

Get-VNXeBasicDefaultFlareSPADetail

Get-VNXeBasicDefaultFlareSPBDetail

Get-VNXeBasicDefaultOSSPADetail

Get-VNXeBasicDefaultOSSPBDetail

Get-VNXeBasicSummaryDart2Detail

Get-VNXeBasicSummaryDart3Detail

Get-VNXeBasicSummaryFlareSPADetail

Get-VNXeBasicSummaryFlareSPBDetail

Get-VNXeBasicSummaryGMSSPADetail

Get-VNXeBasicSummaryGMSSPBDetail

Get-VNXeBasicSummaryOSSPADetail

Get-VNXeBasicSummaryOSSPBDetail

Get-VNXeInitTimeDetail

Get-VNXePoolsDetail

Get-VNXePoolTotalsDetail

Get-VNXeSystemTotalsDetail

Set-VNXeSQLiteLocation

We can also use the get-help CmdLet to find out the functionality provided by each function. A particulary helpful piece of info in the function help is the example field list and data.

PS C:&gt; get-help Get-VNXeSystemTotalsDetail -full

NAME
    Get-VNXeSystemTotalsDetail

SYNOPSIS
    Returns a psobject for the SQLite capacity.db system_totals table

SYNTAX
    Get-VNXeSystemTotalsDetail [<commonparameters>]

DESCRIPTION
    Returns a psobject for the SQLite capacity.db system_totals table

PARAMETERS
    <commonparameters>
        This cmdlet supports the common parameters: Verbose, Debug,
        ErrorAction, ErrorVariable, WarningAction, WarningVariable,
        OutBuffer and OutVariable. For more information, see
        about_CommonParameters (http://go.microsoft.com/fwlink/?LinkID=113216).

INPUTS
    None, Only requires set-SQLiteLocation to be called prior in session

OUTPUTS
    Array of PSObjects representing table records

    Record Example: system_totals
     --------------------------------------------------------
        timestamp            : 2011-05-03 23:59:15
        closing_time         : 2011-05-03 23:59:15
        allocated_space      : 0
        total_space          : 0
        free_space           : 0
        allocated_protection : 0
        total_protection     : 0
     --------------------------------------------------------


    -------------------------- EXAMPLE 1 --------------------------

    C:PS&gt;$detailrecords = Get-VNXeSystemTotalsDetail


RELATED LINKS

</commonparameters></commonparameters>

PS C:> get-help Get-VNXeSystemTotalsDetail -full

NAME

Get-VNXeSystemTotalsDetail

SYNOPSIS

Returns a psobject for the SQLite capacity.db system_totals table

SYNTAX

Get-VNXeSystemTotalsDetail [<commonparameters>]

DESCRIPTION

Returns a psobject for the SQLite capacity.db system_totals table

PARAMETERS

This cmdlet supports the common parameters: Verbose, Debug,

ErrorAction, ErrorVariable, WarningAction, WarningVariable,

OutBuffer and OutVariable. For more information, see

about_CommonParameters (http://go.microsoft.com/fwlink/?LinkID=113216).

INPUTS

None, Only requires set-SQLiteLocation to be called prior in session

OUTPUTS

Array of PSObjects representing table records

Record Example: system_totals

--------------------------------------------------------

timestamp : 2011-05-03 23:59:15

closing_time : 2011-05-03 23:59:15

allocated_space : 0

total_space : 0

free_space : 0

allocated_protection : 0

total_protection : 0

--------------------------------------------------------

-------------------------- EXAMPLE 1 --------------------------

C:PS>$detailrecords = Get-VNXeSystemTotalsDetail

HDS AMS 2000 Storage Resource Reporting with PowerShell

Posted on November 29, 2009
by dmuegge

Welcome!

I have been creating a few PowerShell scripts for use with the HDS AMS 2000 array. One thing I found I needed was a quick way to look at DP(Dynamic Provisioning) pools, raid groups, and LUNs. I also wanted to be able to see the associations and filter easily. Since I am working on a new deployment I have been creating Raid Groups and Luns often. I needed a quick way to see what I currently had while creating new resources.

I created a PowerShell script that would quickly show existing resources by raid group or DP pool. It also uses nickname info for the devices that are maintained in three csv files(LU_Nicknames.csv,RG_Nicknames.csv,DP_Nicknames.csv). These are simple comma delimited text files which contain the ID and nickname of each resource. The files are updated as storage resources are added. This allows me to easily identify the resources and to filter for specific devices.

The script executes three HSNM2 CLI commands and reads the information into object form. The LUN information is then shown grouped by raid group or DP pool.

Here is the output with the nickname search parameter set to “DB”. This will return all database resources based on the naming standard. If this is left null it will return all resources.

PS D:DToolsScriptsPSHDS-SNM2> D:DToolsScriptsPSHDS-SNM2get-LunsByDPRG.ps1
Nickname Search: DB

-- DP Pool  Number: 0  Nickname: DB_SQL_OLTP  Raid Level: 5( 5D+1P)  Type: SAS  Capacity: 1986.0 GB  Used: 734.0 GB ----------------------------------------------------------------


lu                                      nickname                                capacity                                stripesize                              status
--                                      --------                                --------                                ----------                              ------
3                                       DB01_OLTP_DATA                          300.0                                   256                                     Normal
20                                      DB01_OLTP_DATA                          300.0                                   256                                     Normal
22                                      DB01_OLTP_DATA                          300.0                                   256                                     Normal
29                                      DB01_DSS_DATA_TEMP                      600.0                                   256                                     Normal



-- RAID GROUP  Number: 4  Nickname: DB_LOG  Raid Level: 1+0( 2D+2D)  Type: SAS  Capacity: 535.7 GB  Free: 0.0 GB -------------------------------------------------------------------


lu                                      nickname                                capacity                                stripesize                              status
--                                      --------                                --------                                ----------                              ------
5                                       DB01_OLTP_LOG                           267.0                                   64                                      Normal
21                                      DB01_OLTP_LOG                           268.7                                   64                                      Normal



-- RAID GROUP  Number: 7  Nickname: DB_OS  Raid Level: 5( 3D+1P)  Type: SAS  Capacity: 398.4 GB  Free: 183.4 GB --------------------------------------------------------------------


lu                                      nickname                                capacity                                stripesize                              status
--                                      --------                                --------                                ----------                              ------
9                                       DB01_OS_DATA                            215.0                                   256                                     Normal



-- RAID GROUP  Number: 8  Nickname: DB_LOG  Raid Level: 1+0( 2D+2D)  Type: SAS  Capacity: 535.7 GB  Free: 535.7 GB -----------------------------------------------------------------

PS D:DToolsScriptsPSHDS-SNM2> D:DToolsScriptsPSHDS-SNM2get-LunsByDPRG.ps1

Nickname Search: DB

-- DP Pool Number: 0 Nickname: DB_SQL_OLTP Raid Level: 5( 5D+1P) Type: SAS Capacity: 1986.0 GB Used: 734.0 GB ----------------------------------------------------------------

lu nickname capacity stripesize status

-- -------- -------- ---------- ------

3 DB01_OLTP_DATA 300.0 256 Normal

20 DB01_OLTP_DATA 300.0 256 Normal

22 DB01_OLTP_DATA 300.0 256 Normal

29 DB01_DSS_DATA_TEMP 600.0 256 Normal

-- RAID GROUP Number: 4 Nickname: DB_LOG Raid Level: 1+0( 2D+2D) Type: SAS Capacity: 535.7 GB Free: 0.0 GB -------------------------------------------------------------------

lu nickname capacity stripesize status

-- -------- -------- ---------- ------

5 DB01_OLTP_LOG 267.0 64 Normal

21 DB01_OLTP_LOG 268.7 64 Normal

-- RAID GROUP Number: 7 Nickname: DB_OS Raid Level: 5( 3D+1P) Type: SAS Capacity: 398.4 GB Free: 183.4 GB --------------------------------------------------------------------

lu nickname capacity stripesize status

-- -------- -------- ---------- ------

9 DB01_OS_DATA 215.0 256 Normal

-- RAID GROUP Number: 8 Nickname: DB_LOG Raid Level: 1+0( 2D+2D) Type: SAS Capacity: 535.7 GB Free: 535.7 GB -----------------------------------------------------------------

Here is the script:

# -- Setup login info and SNM2 environment ------------------
. ./start-session.ps1
Set-Location $env:STONAVM_HOME

# -- Import Nickname tables ---------------------------------------
$LUNicknames = Import-Csv $SCRIPTHOMELU_Nicknames.csv
$RGNicknames = Import-Csv $SCRIPTHOMERG_Nicknames.csv
$DPNicknames = Import-Csv $SCRIPTHOMEDP_Nicknames.csv

# -- Get storage resource info -----------------------------------
$luresults = & ./auluref.exe -unit $DEFAULTARRAY -g -nosublu -totalsize
$rgresults = & ./aurgref.exe -unit $DEFAULTARRAY -g
$dpresults = & ./audppool.exe -unit $DEFAULTARRAY -refer -g

# -- Setup object collections -----------------------------------
$allrgdata = @()
$allludata = @()
$alldpdata = @()

# -- Get LUN data -----------------------------------------------------------
foreach($luresult in $luresults){
		
	if ($luresult -match 'LUs+Capacitys+Sizes+Groups+Pools+Levels+Types+Status'){
		
		$LUDataRead = $TRUE
	
	}elseif($LUDataRead){
	
		if($luresult -match '(d+)s+(d+.d+)sGBs+(d+)KBs+(d+|N/A)s+(d+|N/A)s+(([5,6]|1+0)(s+d+D+d+[D,P]))s+(SAS|SATA)s+(.+)'){
			$ludata = "" | Select lu,nickname,capacity,stripesize,raidgroup,dppool,raidlevel,type,status
			$ludata.lu = $matches[1]
			$Nickname = $LUNicknames | where {$_.LU -eq $ludata.lu} | select nickname
			$ludata.nickname = $Nickname.nickname
			$ludata.capacity = $matches[2]
			$ludata.stripesize = $matches[3]
			$ludata.raidgroup = $matches[4]
			$ludata.dppool = $matches[5]
			$ludata.raidlevel = $matches[6]
			$ludata.type = $matches[8]
			$ludata.status = $matches[9]
			$allludata += $ludata
		}
	}
}

# --  Get Raid Group data --------------------------------------------------------
foreach($rgresult in $rgresults){
	
	if ($rgresult -match 'Groups+Levels+Groupss+Types+TotalsCapacitys+FreesCapacitys+Prioritys+Status'){
		
		$RGDataRead = $TRUE
	
	}elseif($RGDataRead){
	
		if($rgresult -match '(d+)s+(([5,6]|1+0)(s+d+D+d+[D,P]))s+(d+)s+(SAS|SATA)s+(d+.d+)sGBs+(d+.d+)sGBs(s*d+.{1}d+%{1})s+(.+s.+)s+(.+)'){
		
			$rgdata = "" | Select rg,nickname,raidlevel,raidconfig,paritygroups,type,totalcapacity,freecapacity,priority,status
			$rgdata.rg = $matches[1]
			$Nickname = $RGNicknames | where {$_.RG -eq $rgdata.rg} | select nickname
			$rgdata.nickname = $Nickname.nickname
			$rgdata.raidlevel = $matches[2]
			$rgdata.raidconfig = $matches[3]
			$rgdata.paritygroups = $matches[4]
			$rgdata.type = $matches[5]
			$rgdata.totalcapacity = $matches[6]
			$rgdata.freecapacity = $matches[7]
			$rgdata.priority = $matches[8]
			$rgdata.status = $matches[9]
			$allrgdata += $rgdata
		}	
	}
}


# --  Get DP pool data -------------------------------------------------------------
foreach($dpresult in $dpresults){
	
	if ($dpresult -match 'Pools+Levels+TotalsCapacitys+ConsumedsCapacitys+Types+Status'){
		
		$DPDataRead = $TRUE
	
	}elseif($DPDataRead){
	
		if($dpresult -match '(d+)s+(([5,6]|1+0)(s+d+D+d+[D,P]))s+(d+.d+)sGBs+(d+.d+)sGBs+(SAS|SATA)s+(.+)'){ 
		
			$dpdata = "" | Select dp,nickname,raidlevel,totalcapacity,capacityused,type,status
			$dpdata.dp = $matches[1]
			$Nickname = $DPNicknames | where {$_.DP -eq $dpdata.dp} | select nickname
			$dpdata.nickname = $Nickname.nickname
			$dpdata.raidlevel = $matches[2]
			$dpdata.totalcapacity = $matches[4]
			$dpdata.capacityused = $matches[5]
			$dpdata.type = $matches[6]
			$dpdata.status = $matches[7]
			$alldpdata += $dpdata
		}	
	}
}


# -- Get search string -----------------------------
$searchstring = Read-Host "Nickname Search"

# -- Display LUN's by DP pool --------------------------------------------------------------
foreach($dppool in $alldpdata){
	
	$dpnum = $dppool.dp
	$dpcapacity = $dppool.totalcapacity
	$dpused = $dppool.capacityused
	$dpnickname = $dppool.nickname
	$dplevel = $dppool.raidlevel
	$dptype = $dppool.type
	
	if($dpnickname -like "*$searchstring*"){
		$message = "-- DP Pool  Number: $dpnum  Nickname: $dpnickname  Raid Level: $dplevel  Type: $dptype  Capacity: $dpcapacity GB  Used: $dpused GB -"
		$charadd = (180 - $message.length)
		for($x=1;$x -le $charadd;$x++){$message += "-"}
		Write-Host ""
		Write-Host $message
		Write-Host ""
	}
	if($searchstring){
		$allludata | where {(($_.nickname -like "*$searchstring*") -and ($_.dppool -eq $dpnum))} | FT -property lu,nickname,capacity,stripesize,status
	}
	else
	{
		$allludata | where {$_.dppool -eq $dpnum} | FT -property lu,nickname,capacity,stripesize,status
	}	
}

# -- Display LUN"s by raid group -------------------------------------------------
foreach($rgroup in $allrgdata){
	
	$rgnum = $rgroup.rg
	$rgcapacity = $rgroup.totalcapacity
	$rgfree = $rgroup.freecapacity
	$rgnickname = $rgroup.nickname
	$rglevel = $rgroup.raidlevel
	$rgtype = $rgroup.type
	
	if($rgnickname -like "*$searchstring*"){
		$message = "-- RAID GROUP  Number: $rgnum  Nickname: $rgnickname  Raid Level: $rglevel  Type: $rgtype  Capacity: $rgcapacity GB  Free: $rgfree GB -"
		$charadd = (180 - $message.length)
		for($x=1;$x -le $charadd;$x++){$message += "-"}
		Write-Host ""
		Write-Host $message
		Write-Host ""
	}
	if($searchstring){
		$allludata | where {(($_.nickname -like "*$searchstring*") -and ($_.raidgroup -eq $rgnum))} | FT -property lu,nickname,capacity,stripesize,status
	}
	else
	{
		$allludata | where {$_.raidgroup -eq $rgnum} | FT -property lu,nickname,capacity,stripesize,status
	}
}

# -- CD back to script directory and set window title ---------------------
Set-Location $SCRIPTHOME
$Host.UI.RawUI.WindowTitle = "DP-RG-LU Info"

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

# -- Setup login info and SNM2 environment ------------------

. ./start-session.ps1

Set-Location $env:STONAVM_HOME

# -- Import Nickname tables ---------------------------------------

$LUNicknames = Import-Csv $SCRIPTHOMELU_Nicknames.csv

$RGNicknames = Import-Csv $SCRIPTHOMERG_Nicknames.csv

$DPNicknames = Import-Csv $SCRIPTHOMEDP_Nicknames.csv

# -- Get storage resource info -----------------------------------

$luresults = & ./auluref.exe -unit $DEFAULTARRAY -g -nosublu -totalsize

$rgresults = & ./aurgref.exe -unit $DEFAULTARRAY -g

$dpresults = & ./audppool.exe -unit $DEFAULTARRAY -refer -g

# -- Setup object collections -----------------------------------

$allrgdata = @()

$allludata = @()

$alldpdata = @()

# -- Get LUN data -----------------------------------------------------------

foreach($luresult in $luresults){

if ($luresult -match 'LUs+Capacitys+Sizes+Groups+Pools+Levels+Types+Status'){

$LUDataRead = $TRUE

}elseif($LUDataRead){

if($luresult -match '(d+)s+(d+.d+)sGBs+(d+)KBs+(d+|N/A)s+(d+|N/A)s+(([5,6]|1+0)(s+d+D+d+[D,P]))s+(SAS|SATA)s+(.+)'){

$ludata = "" | Select lu,nickname,capacity,stripesize,raidgroup,dppool,raidlevel,type,status

$ludata.lu = $matches[1]

$Nickname = $LUNicknames | where {$_.LU -eq $ludata.lu} | select nickname

$ludata.nickname = $Nickname.nickname

$ludata.capacity = $matches[2]

$ludata.stripesize = $matches[3]

$ludata.raidgroup = $matches[4]

$ludata.dppool = $matches[5]

$ludata.raidlevel = $matches[6]

$ludata.type = $matches[8]

$ludata.status = $matches[9]

$allludata += $ludata

}

# -- Get Raid Group data --------------------------------------------------------

foreach($rgresult in $rgresults){

if ($rgresult -match 'Groups+Levels+Groupss+Types+TotalsCapacitys+FreesCapacitys+Prioritys+Status'){

$RGDataRead = $TRUE

}elseif($RGDataRead){

if($rgresult -match '(d+)s+(([5,6]|1+0)(s+d+D+d+[D,P]))s+(d+)s+(SAS|SATA)s+(d+.d+)sGBs+(d+.d+)sGBs(s*d+.{1}d+%{1})s+(.+s.+)s+(.+)'){

$rgdata = "" | Select rg,nickname,raidlevel,raidconfig,paritygroups,type,totalcapacity,freecapacity,priority,status

$rgdata.rg = $matches[1]

$Nickname = $RGNicknames | where {$_.RG -eq $rgdata.rg} | select nickname

$rgdata.nickname = $Nickname.nickname

$rgdata.raidlevel = $matches[2]

$rgdata.raidconfig = $matches[3]

$rgdata.paritygroups = $matches[4]

$rgdata.type = $matches[5]

$rgdata.totalcapacity = $matches[6]

$rgdata.freecapacity = $matches[7]

$rgdata.priority = $matches[8]

$rgdata.status = $matches[9]

$allrgdata += $rgdata

}

# -- Get DP pool data -------------------------------------------------------------

foreach($dpresult in $dpresults){

if ($dpresult -match 'Pools+Levels+TotalsCapacitys+ConsumedsCapacitys+Types+Status'){

$DPDataRead = $TRUE

}elseif($DPDataRead){

if($dpresult -match '(d+)s+(([5,6]|1+0)(s+d+D+d+[D,P]))s+(d+.d+)sGBs+(d+.d+)sGBs+(SAS|SATA)s+(.+)'){

$dpdata = "" | Select dp,nickname,raidlevel,totalcapacity,capacityused,type,status

$dpdata.dp = $matches[1]

$Nickname = $DPNicknames | where {$_.DP -eq $dpdata.dp} | select nickname

$dpdata.nickname = $Nickname.nickname

$dpdata.raidlevel = $matches[2]

$dpdata.totalcapacity = $matches[4]

$dpdata.capacityused = $matches[5]

$dpdata.type = $matches[6]

$dpdata.status = $matches[7]

$alldpdata += $dpdata

}

# -- Get search string -----------------------------

$searchstring = Read-Host "Nickname Search"

# -- Display LUN's by DP pool --------------------------------------------------------------

foreach($dppool in $alldpdata){

$dpnum = $dppool.dp

$dpcapacity = $dppool.totalcapacity

$dpused = $dppool.capacityused

$dpnickname = $dppool.nickname

$dplevel = $dppool.raidlevel

$dptype = $dppool.type

if($dpnickname -like "*$searchstring*"){

$message = "-- DP Pool Number: $dpnum Nickname: $dpnickname Raid Level: $dplevel Type: $dptype Capacity: $dpcapacity GB Used: $dpused GB -"

$charadd = (180 - $message.length)

for($x=1;$x -le $charadd;$x++){$message += "-"}

Write-Host ""

Write-Host $message

Write-Host ""

}

if($searchstring){

$allludata | where {(($_.nickname -like "*$searchstring*") -and ($_.dppool -eq $dpnum))} | FT -property lu,nickname,capacity,stripesize,status

}

else

{

$allludata | where {$_.dppool -eq $dpnum} | FT -property lu,nickname,capacity,stripesize,status

}

# -- Display LUN"s by raid group -------------------------------------------------

foreach($rgroup in $allrgdata){

$rgnum = $rgroup.rg

$rgcapacity = $rgroup.totalcapacity

$rgfree = $rgroup.freecapacity

$rgnickname = $rgroup.nickname

$rglevel = $rgroup.raidlevel

$rgtype = $rgroup.type

if($rgnickname -like "*$searchstring*"){

$message = "-- RAID GROUP Number: $rgnum Nickname: $rgnickname Raid Level: $rglevel Type: $rgtype Capacity: $rgcapacity GB Free: $rgfree GB -"

$charadd = (180 - $message.length)

for($x=1;$x -le $charadd;$x++){$message += "-"}

Write-Host ""

Write-Host $message

Write-Host ""

}

if($searchstring){

$allludata | where {(($_.nickname -like "*$searchstring*") -and ($_.raidgroup -eq $rgnum))} | FT -property lu,nickname,capacity,stripesize,status

}

else

{

$allludata | where {$_.raidgroup -eq $rgnum} | FT -property lu,nickname,capacity,stripesize,status

}

# -- CD back to script directory and set window title ---------------------

Set-Location $SCRIPTHOME

$Host.UI.RawUI.WindowTitle = "DP-RG-LU Info"

The script uses the start-session.ps1 file to establish connectivity with the HDS array. Additional information regarding the use of this include file can be found at this post. Then the script executes HSNM2 CLI commands to return information on DP pools raid groups and LUNS. The script uses regular expressions to parse the output and convert it into objects. It also reads in the nickname files and adds the data to the custom objects.

The objects are then output using the built-in PowerShell formating engine with a little custom formating thrown in for the group headers.

Here is an example nickname file:

RG,Nickname
0,MG_OS
1,AP_OS
2,DT_OS
3,DB_SQL_OLTP
7,DB_OS
8,DB_LOG

RG,Nickname

0,MG_OS

1,AP_OS

2,DT_OS

3,DB_SQL_OLTP

7,DB_OS

8,DB_LOG

I suppose this may not be necessary with the use of Device Manager, but I am still learning it and I could not quite get this view with it. Besides I am more of a scripting kind of guy. I also really like the output of this script as it gives me just the view of the array I need when I am allocating new storage and setting up new resources. I use this script in conjuction with two other scripts for creating LUN’s and raid groups. I plan to post those scripts soon.

Hope this helps,

Dave

HDS AMS 2000 Performance Analysis with PowerShell and PowerGadgets

Posted on October 23, 2009
by dmuegge

Welcome!

It has been a while since my last post due to a very busy schedule with a SAN and virtualization project.

I have been working on an implementation of a HDS AMS 2500 midrange array for a VMWare vShere 4 environment. So far everything has been working and performing well. The management software included with the HDS AMS 2000 series array is SNM2(Storage Navigator Modular 2), A java based web application. This software also has a command line version which appears to be pretty comprehensive. It consists of a series of DOS executables, which can be run from PowerShell. There are a series of scripts I have been working on for viewing and creating storage resources on the array. I will share many of these in future posts. In this post I want to share some scripts I have written to extend the functionality of the performance monitoring utility in SNM2.

The base functionality of the array allows you to capture performance statistics to a text file. The file can be captured manually or automatically for a specified time period and interval down to one minute. One text file is produced per capture or all captures can be written to one file. Also, I believe based on the information I read in the SNM2 manual you can do some graphing with the web interface, but it requires an additional license and personally I think the PowerGadgets graphs are better.

The 4 scripts I have started with are get-performance_processor.ps1, get-performance_ports.ps1, get-performance_raidgroups.ps1, get-performance_luns.ps1, which do pretty much what they say and produce the following PowerGadgets charts.

Chart

The chart group is a tabbed interface which allows you to tab through the controllers and ports/RG/LU/Procs depending on the script being used. Each script generates different groups of charts for different performance counters. I have not implemented all of the performance counters just the ones which are most important to me now. I will be improving these scripts over time and implementing more counters. Here is an example of how the script works.

Script

After executing the script it will ask whether or not to collect data, if yes it will prompt for interval in minutes and time period. If no it will use previously collected data in the default output directory. Next it will ask to list data in text output. Then it will prompt for generation of each group of charts for ports, raid groups, luns or processors depending on the script run.

Now to the script. All of the scripts rely on the start-session.ps1 script and also require a password file be set for logging into the array. Additionally, an array has to be registered.

Example 1 shows a PowerShell script which will register an array and set the admin password.

$env:STONAVM_HOME="C:Program FilesStorage Navigator Modular 2 CLI"
$env:STONAVM_ACT="on"
$env:STONAVM_RSP_PASS="on"
$env:LANG="en"
cd "C:Program FilesStorage Navigator Modular 2 CLI"
./auunitadd -unit ARRAYNAME -ctl0 192.168.1.1 -ctl1 192.168.1.2
./auaccountenv -set -uid USERNAME

$env:STONAVM_HOME="C:Program FilesStorage Navigator Modular 2 CLI"

$env:STONAVM_ACT="on"

$env:STONAVM_RSP_PASS="on"

$env:LANG="en"

cd "C:Program FilesStorage Navigator Modular 2 CLI"

./auunitadd -unit ARRAYNAME -ctl0 192.168.1.1 -ctl1 192.168.1.2

./auaccountenv -set -uid USERNAME

You will need to replace ARRAYNAME, USERNAME and the IP Addresses for your environment.

Example 2 shows the start-session PowerShell script which defines environmental information.

# Comand Enviroment
$env:STONAVM_HOME="C:Program FilesStorage Navigator Modular 2 CLI"
$env:STONAVM_ACT="on"
$env:STONAVM_RSP_PASS="on"
$env:LANG="en"

# Global
$SCRIPTHOME = "C:ScriptsHDS-SNM2"
$DEFAULTARRAY = "ARRAYNAME"

# Comand Enviroment

$env:STONAVM_HOME="C:Program FilesStorage Navigator Modular 2 CLI"

$env:STONAVM_ACT="on"

$env:STONAVM_RSP_PASS="on"

$env:LANG="en"

# Global

$SCRIPTHOME = "C:ScriptsHDS-SNM2"

$DEFAULTARRAY = "ARRAYNAME"

You will need to change the paths and ARRAYNAME for your environment.

Example 3 shows the get-performance_processor.ps1 script

# -- User Defined Constants ---------------------------
$LOG_PATH = "C:ScriptsHDS-SNM2pfmlog"

# -- Modify controller and core options as needed -----
$Controllers = @(0,1)
$cores = @("X","Y")

# -- Modifications below this line should not be required ---------------------------------------




# -- Setup environment ------------------------------
$testsnapin = $null
$testsnapin = get-pssnapin | where { $_.Name -eq "PowerGadgets"}
if(-not $testsnapin){add-pssnapin -Name PowerGadgets}
. ./start-session.ps1
Set-Location $env:STONAVM_HOME


# --  Object to hold processor usage data ------------------------
$allprocusagedata = @()

# -- Processing flags -------------------------------------
$PROCUsageDataRead = $FALSE


# -- Specify whether to run new collection if not existing files are used --
Write-Host ”Collect Data Y or N...”
$modekey = $Host.UI.RawUI.ReadKey(”NoEcho,IncludeKeyDown”)
if($modekey.Character.ToString().ToLower() -eq "y"){
	$collectiontime = Read-Host "Collection Time/Count in minutes"
	Write-Host "Collecting Data..."
	./auperform.exe -unit $DEFAULTARRAY -auto 1 -count $collectiontime -path $LOG_PATH -pfmstatis	
}


# -- Get data from files --------------------------------------------
Write-Host "Analyzing Data..."
$perffiles = Get-ChildItem $LOG_PATH

foreach($perffile in $perffiles){
	$filedata = Get-Content $LOG_PATH$perffile
	foreach($line in $filedata){
	
		# Get collection times
		if($line -match '^(d{4}/(?:0[1-9]|1[0-2]?)/(?:0[1-9]|[12][0-9]|3[01]?)s(?:0[0-9]|1[0-9]|2[0-3]?):(?:0[0-9]|1[0-9]|2[0-9]|3[0-9]|4[0-9]|5[0-9]?):(?:0[0-9]|1[0-9]|2[0-9]|3[0-9]|4[0-9]|5[0-9]?))s-s(d{4}/(?:0[1-9]|1[0-2]?)/(?:0[1-9]|[12][0-9]|3[01]?)s(?:0[0-9]|1[0-9]|2[0-3]?):(?:0[0-9]|1[0-9]|2[0-9]|3[0-9]|4[0-9]|5[0-9]?):(?:0[0-9]|1[0-9]|2[0-9]|3[0-9]|4[0-9]|5[0-9]?))

This script collects the data from the array in separate files. Reads the pertinent data from the files and transforms it into object form which is fed into the PowerGadgets out-chart cmdlet. The other three scripts are longer as they digest more information.

To use these scripts you will need PowerShell, PowerGadgets( this a pay product with a free trial ), SNM2 CLI, and the script files attached to this post. Oh and an HDS AMS 2000 array.

Here are the script dowmloads
<a href="http://muegge.com/blogfiles/start-Session.txt">start-Session.txt</a>
<a href="http://muegge.com/blogfiles/get-performance_processor.txt">get-performance_processor.txt</a>
<a href="http://muegge.com/blogfiles/get-performance_ports.txt">get-performance_ports.txt</a>
<a href="http://muegge.com/blogfiles/get-performance_raidgroups.txt">get-performance_raidgroups.txt</a>
<a href="http://muegge.com/blogfiles/get-performance_luns.txt">get-performance_luns.txt</a>

Save the files to your script directory and change the extensions to .ps1 

I hope someone finds this useful.

Regards,

Dave
){
		
			$collectfrom = $matches[1]
			$collectto = $matches[2]				
		}
		
		# Get Proc Usage Data
		if ($line -match 'CTLs+Cores+Usage'){
			
			$PROCUsageDataRead = $TRUE
		
		}elseif($PROCUsageDataRead){
		
			$line -match '([0,1]{1})s+([X,Y]{1})s+(d+)' | Out-Null 
			$procusagedata = "" | Select ctl,core,usage,datetime
			$procusagedata.ctl = $matches[1]
			$procusagedata.core = $matches[2]
			$procusagedata.usage = $matches[3]
			$procusagedata.datetime = $collectto
			$allprocusagedata += $procusagedata
			
			if(($procusagedata.ctl -eq 1) -and ($procusagedata.core -eq "Y")){$PROCUsageDataRead = $FALSE}
		}
	}
}


# -- List Data ---------------------------------
Write-Host ”List Processor Data Y or N...”
$modekey = $Host.UI.RawUI.ReadKey(”NoEcho,IncludeKeyDown”)
if($modekey.Character.ToString().ToLower() -eq "y"){
	$allprocusagedata | ft -prop *
}


# -- Create Processor Usage Chart Group ----------------------
Write-Host ”Show Processor Usage Charts Y or N...”
$modekey = $Host.UI.RawUI.ReadKey(”NoEcho,IncludeKeyDown”)
if($modekey.Character.ToString().ToLower() -eq "y"){
	Write-Host "Generating Charts..."
	foreach($Controller in $Controllers){
		foreach($core in $cores){
			$ChartTitle = "Processor Usage Controller:$Controller Core:$core"
			$allprocusagedata | where {($_.core -eq $core)-and ($_.ctl -eq $Controller)} | sort datetime | Select core,usage,datetime | `
			Out-Chart -Values usage -Series_0_Text "Processor Usage %" -Label datetime `
			-Title $ChartTitle -Gallery Lines -Group "PROCUSAGE" -Name $ChartTitle -Caption "Processor Usage %"
		}
	}
}

Set-Location $SCRIPTHOME

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

# -- User Defined Constants ---------------------------

$LOG_PATH = "C:ScriptsHDS-SNM2pfmlog"

# -- Modify controller and core options as needed -----

$Controllers = @(0,1)

$cores = @("X","Y")

# -- Modifications below this line should not be required ---------------------------------------

# -- Setup environment ------------------------------

$testsnapin = $null

$testsnapin = get-pssnapin | where { $_.Name -eq "PowerGadgets"}

if(-not $testsnapin){add-pssnapin -Name PowerGadgets}

. ./start-session.ps1

Set-Location $env:STONAVM_HOME

# -- Object to hold processor usage data ------------------------

$allprocusagedata = @()

# -- Processing flags -------------------------------------

$PROCUsageDataRead = $FALSE

# -- Specify whether to run new collection if not existing files are used --

Write-Host ”Collect Data Y or N...”

$modekey = $Host.UI.RawUI.ReadKey(”NoEcho,IncludeKeyDown”)

if($modekey.Character.ToString().ToLower() -eq "y"){

$collectiontime = Read-Host "Collection Time/Count in minutes"

Write-Host "Collecting Data..."

./auperform.exe -unit $DEFAULTARRAY -auto 1 -count $collectiontime -path $LOG_PATH -pfmstatis

}

# -- Get data from files --------------------------------------------

Write-Host "Analyzing Data..."

$perffiles = Get-ChildItem $LOG_PATH

foreach($perffile in $perffiles){

$filedata = Get-Content $LOG_PATH$perffile

foreach($line in $filedata){

# Get collection times

if($line -match '^(d{4}/(?:0[1-9]|1[0-2]?)/(?:0[1-9]|[12][0-9]|3[01]?)s(?:0[0-9]|1[0-9]|2[0-3]?):(?:0[0-9]|1[0-9]|2[0-9]|3[0-9]|4[0-9]|5[0-9]?):(?:0[0-9]|1[0-9]|2[0-9]|3[0-9]|4[0-9]|5[0-9]?))s-s(d{4}/(?:0[1-9]|1[0-2]?)/(?:0[1-9]|[12][0-9]|3[01]?)s(?:0[0-9]|1[0-9]|2[0-3]?):(?:0[0-9]|1[0-9]|2[0-9]|3[0-9]|4[0-9]|5[0-9]?):(?:0[0-9]|1[0-9]|2[0-9]|3[0-9]|4[0-9]|5[0-9]?))

This script collects the data from the array in separate files. Reads the pertinent data from the files and transforms it into object form which is fed into the PowerGadgets out-chart cmdlet. The other three scripts are longer as they digest more information.

To use these scripts you will need PowerShell, PowerGadgets( this a pay product with a free trial ), SNM2 CLI, and the script files attached to this post. Oh and an HDS AMS 2000 array.

Here are the script dowmloads

<a href="http://muegge.com/blogfiles/start-Session.txt">start-Session.txt</a>

<a href="http://muegge.com/blogfiles/get-performance_processor.txt">get-performance_processor.txt</a>

<a href="http://muegge.com/blogfiles/get-performance_ports.txt">get-performance_ports.txt</a>

<a href="http://muegge.com/blogfiles/get-performance_raidgroups.txt">get-performance_raidgroups.txt</a>

<a href="http://muegge.com/blogfiles/get-performance_luns.txt">get-performance_luns.txt</a>

Save the files to your script directory and change the extensions to .ps1

I hope someone finds this useful.

Regards,

Dave

){

$collectfrom = $matches[1]

$collectto = $matches[2]

}

# Get Proc Usage Data

if ($line -match 'CTLs+Cores+Usage'){

$PROCUsageDataRead = $TRUE

}elseif($PROCUsageDataRead){

$line -match '([0,1]{1})s+([X,Y]{1})s+(d+)' | Out-Null

$procusagedata = "" | Select ctl,core,usage,datetime

$procusagedata.ctl = $matches[1]

$procusagedata.core = $matches[2]

$procusagedata.usage = $matches[3]

$procusagedata.datetime = $collectto

$allprocusagedata += $procusagedata

if(($procusagedata.ctl -eq 1) -and ($procusagedata.core -eq "Y")){$PROCUsageDataRead = $FALSE}

}

# -- List Data ---------------------------------

Write-Host ”List Processor Data Y or N...”

$modekey = $Host.UI.RawUI.ReadKey(”NoEcho,IncludeKeyDown”)

if($modekey.Character.ToString().ToLower() -eq "y"){

$allprocusagedata | ft -prop *

}

# -- Create Processor Usage Chart Group ----------------------

Write-Host ”Show Processor Usage Charts Y or N...”

$modekey = $Host.UI.RawUI.ReadKey(”NoEcho,IncludeKeyDown”)

if($modekey.Character.ToString().ToLower() -eq "y"){

Write-Host "Generating Charts..."

foreach($Controller in $Controllers){

foreach($core in $cores){

$ChartTitle = "Processor Usage Controller:$Controller Core:$core"

$allprocusagedata | where {($_.core -eq $core)-and ($_.ctl -eq $Controller)} | sort datetime | Select core,usage,datetime | `

Out-Chart -Values usage -Series_0_Text "Processor Usage %" -Label datetime `

-Title $ChartTitle -Gallery Lines -Group "PROCUSAGE" -Name $ChartTitle -Caption "Processor Usage %"

}

Set-Location $SCRIPTHOME

To use these scripts you will need PowerShell, PowerGadgets( this a pay product with a free trial ), SNM2 CLI, and the script files attached to this post. Oh and an HDS AMS 2000 array.

Here are the script dowmloads
start-Session.txt
get-performance_processor.txt
get-performance_ports.txt
get-performance_raidgroups.txt
get-performance_luns.txt

Save the files to your script directory and change the extensions to .ps1

I hope someone finds this useful.

Regards,

Dave

Exchange DB Reporting with PowerShell and Log Parser

Posted on July 1, 2009
by dmuegge

I ran across a useful post today as I was roaming through Google Analytics.

Using PowerShell, LogParser and PowerGadgets to get Exchange 2003 storage information – Part 1

Wes Stahler uses Log Parser and PowerShell to report on the free space in an Exchange Database.

This is a task I have done in the past. I will add this script to my toolkit.

Regards,

Dave

ADAM Administration with SharePoint and PowerShell

Posted on May 10, 2009
by dmuegge

Welcome!

Recently I worked on finding a simple way to create a web based administrative interface for an ADAM directory. The requirements were to create a simple web based interface to allow business personnel to manage users and groups for an application directory. It was also desirable if this solution would easily integrate with SharePoint.

After doing a little searching on the web, I found a combination that fit the bill.

The Quest AD Management Shell CmdLets – This is a PowerShell Snap-In that allows administration of AD and ADAM. The cmdlets are from Quest Software you can find more info here. I have used them in other scripts and they have come in very handy. To make these work in this solution from SharePoint, the Quest Snapin .dll and it’s dependants need to be copied to the global assembly cache and entered as a safecontrol in the sharepoint web.config.

The iLoveSharePoint PowerWebPart 3.0 – This is a web part which allows the execution of PowerShell code from the web part. This web part is from the CodePlex project iLoveSharePoint by Christian Glessner. I was impressed with this web part. It is easy to install and configure and relatively simple to use.

The PowerWebPart allows you to execute scripts that will render asp.net web controls in the web part. This allows you to retrieve user input from the controls to use as script inputs. The possibilities are endless. For my purposes I only needed a very simple user interface.

I wanted a way to use this for different ADAM partitions so I tried to allow for different configuration scripts. The design I decided on consisted of three levels of scripts one for configuration one for data access and one UI script for each web part.

The code sample below is the configuration and connection script. This script defines the user and group containers and the directory connect and disconnect functions.

# ---------------------------------------------------------------------------
### <Library name='ADAM_Config_Lib.ps1'>
### <Author>David Muegge</Author>
### <Description>
### 	Adam connection and configuration library
### </Description>
### <Dependencies>
###	Quest AD CmdLets 1.2
### </Dependencies>
### </Library>
# ---------------------------------------------------------------------------
$SnapInName = "Quest.ActiveRoles.ADManagement"
$testsnapin = $null
$testsnapin = get-pssnapin | where { $_.Name -eq $SnapInName}
if(-not $testsnapin){add-pssnapin -Name $SnapInName}

# Container paths
$usercontainer = "cn=Users,cn=ADAMDEV"
$groupcontainer = "cn=Groups,cn=ADAMDEV"


function connect-ADAM{
	
	# This allows the use of a windows account for ADAM athentication without granting the application pool account rights to ADAM
	$passwd = Get-Content "C:testpwd.txt" | convertto-securestring
	$adamcred = new-object -typename System.Management.Automation.PSCredential -argumentlist "TestSVR1adamadmin",$passwd

	connect-QADService -Service localhost -Credential $adamcred


}

function disconnect-ADAM{

	param($adamconnection)
	
	Disconnect-QADService -Connection $adamconnection

}

# ---------------------------------------------------------------------------

### <Library name='ADAM_Config_Lib.ps1'>

### <Author>David Muegge</Author>

### <Description>

### Adam connection and configuration library

### </Description>

### <Dependencies>

### Quest AD CmdLets 1.2

### </Dependencies>

### </Library>

# ---------------------------------------------------------------------------

$SnapInName = "Quest.ActiveRoles.ADManagement"

$testsnapin = $null

$testsnapin = get-pssnapin | where { $_.Name -eq $SnapInName}

if(-not $testsnapin){add-pssnapin -Name $SnapInName}

# Container paths

$usercontainer = "cn=Users,cn=ADAMDEV"

$groupcontainer = "cn=Groups,cn=ADAMDEV"

function connect-ADAM{

# This allows the use of a windows account for ADAM athentication without granting the application pool account rights to ADAM

$passwd = Get-Content "C:testpwd.txt" | convertto-securestring

$adamcred = new-object -typename System.Management.Automation.PSCredential -argumentlist "TestSVR1adamadmin",$passwd

connect-QADService -Service localhost -Credential $adamcred

}

function disconnect-ADAM{

param($adamconnection)

Disconnect-QADService -Connection $adamconnection

}

The next script is the function library for data access to the ADAM directory.

# ---------------------------------------------------------------------------
### <Library name='ADAM_Lib.ps1'>
### <Author>David Muegge</Author>
### <Description>
### 	Adam directoy function library
### </Description>
### <Dependencies>
###	Quest AD CmdLets 1.2
### </Dependencies>
### </Library>
# ---------------------------------------------------------------------------
function get-Users{
	
	$adamconnection = connect-ADAM
	$results = get-QADObject -SearchRoot $usercontainer -Connection $adamconnection
	disconnect-ADAM -adamconnection $adamconnection
	return $results
	
	
}


function get-Groups{

	$adamconnection = connect-ADAM
	$results = get-qadgroup -SearchRoot $groupcontainer -Connection $adamconnection
	disconnect-ADAM -adamconnection $adamconnection
	return $results

	
}


function new-User{

	param([String]$logonid)
	
	# Random password generator here
	$newpassword = "password123$"
	
	$adamconnection = connect-ADAM
	New-QADUser -Name $newlogonid -UserPassword $newpassword -ParentContainer $usercontainer -Connection $adamconnection
	disconnect-ADAM -adamconnection $adamconnection
	
}

function delete-user{

	param([String]$logonid)
	
	$adamconnection = connect-ADAM
	$UserDN = "CN=$logonID," + $usercontainer
	Remove-QADObject -Identity $UserDN -Connection $adamconnection
	disconnect-ADAM -adamconnection $adamconnection

}


function new-Group{

	param([String]$groupid)
	
	$adamconnection = connect-ADAM
	New-QADGroup -Name $groupid -GroupType "Default" -ParentContainer $groupcontainer -Connection $adamconnection
	disconnect-ADAM -adamconnection $adamconnection
}


function delete-group{

	param([String]$groupid)
	
	$adamconnection = connect-ADAM
	$GroupDN = "CN=$groupid," + $groupcontainer
	Remove-QADObject -Identity $GroupDN -Connection $adamconnection
	disconnect-ADAM -adamconnection $adamconnection

}


function Add-UserToGroup{

	param([string]$username,[String]$groupname)
	
	$adamconnection = connect-ADAM
	# Get user dn
	$results = Get-QADUser -Name $username -Connection $adamconnection
	foreach($result in $results){$userdn = $result.dn.ToString()}
	
	# Get Group DN
	$results = Get-QADGroup -Name $groupname -Connection $adamconnection
	foreach($result in $results){$groupdn = $result.dn.ToString()}
	
	# Add User to Group
	Add-QADGroupMember -Identity $groupdn -Member $userdn -Connection $adamconnection
	disconnect-ADAM -adamconnection $adamconnection
	
}

# ---------------------------------------------------------------------------

### <Library name='ADAM_Lib.ps1'>

### <Author>David Muegge</Author>

### <Description>

### Adam directoy function library

### </Description>

### <Dependencies>

### Quest AD CmdLets 1.2

### </Dependencies>

### </Library>

# ---------------------------------------------------------------------------

function get-Users{

$adamconnection = connect-ADAM

$results = get-QADObject -SearchRoot $usercontainer -Connection $adamconnection

disconnect-ADAM -adamconnection $adamconnection

return $results

}

function get-Groups{

$adamconnection = connect-ADAM

$results = get-qadgroup -SearchRoot $groupcontainer -Connection $adamconnection

disconnect-ADAM -adamconnection $adamconnection

return $results

}

function new-User{

param([String]$logonid)

# Random password generator here

$newpassword = "password123$"

$adamconnection = connect-ADAM

New-QADUser -Name $newlogonid -UserPassword $newpassword -ParentContainer $usercontainer -Connection $adamconnection

disconnect-ADAM -adamconnection $adamconnection

}

function delete-user{

param([String]$logonid)

$adamconnection = connect-ADAM

$UserDN = "CN=$logonID," + $usercontainer

Remove-QADObject -Identity $UserDN -Connection $adamconnection

disconnect-ADAM -adamconnection $adamconnection

}

function new-Group{

param([String]$groupid)

$adamconnection = connect-ADAM

New-QADGroup -Name $groupid -GroupType "Default" -ParentContainer $groupcontainer -Connection $adamconnection

disconnect-ADAM -adamconnection $adamconnection

}

function delete-group{

param([String]$groupid)

$adamconnection = connect-ADAM

$GroupDN = "CN=$groupid," + $groupcontainer

Remove-QADObject -Identity $GroupDN -Connection $adamconnection

disconnect-ADAM -adamconnection $adamconnection

}

function Add-UserToGroup{

param([string]$username,[String]$groupname)

$adamconnection = connect-ADAM

# Get user dn

$results = Get-QADUser -Name $username -Connection $adamconnection

foreach($result in $results){$userdn = $result.dn.ToString()}

# Get Group DN

$results = Get-QADGroup -Name $groupname -Connection $adamconnection

foreach($result in $results){$groupdn = $result.dn.ToString()}

# Add User to Group

Add-QADGroupMember -Identity $groupdn -Member $userdn -Connection $adamconnection

disconnect-ADAM -adamconnection $adamconnection

}

The next script is an example of a UI script for the web part. When a new PowerWebPart is created a template script is added by default. This script provides a framework and some sample code. Christian also has an add-on which allows you to use PowerGui to edit your script from SharePoint. The entire solution contains one script similar to this for each web part.

########### Initialize ############
. c:ADAMAdminADAM_Lib.ps1

### declare global variables and functions
$groupnametext = New-Object System.Web.UI.WebControls.TextBox
$createbutton = New-Object System.Web.UI.WebControls.Button
$groupnamelabel = New-Object System.Web.UI.WebControls.Label
$deletebutton = New-Object System.Web.UI.WebControls.Button


############## Load ##############

### first time the OnLoad fires before CreateChildControls
#function OnLoad
#{
#	# Check if GET Request (first request). 
#	if($isPostBack -eq $false)
#	{
#		$label.Text = 'GET request.'
#	}
#}

######## 3. Create Controls ########

## create child controls
function CreateChildControls($controls)
{
	$controls.Add($groupnametext)	
 	
	$createbutton.Text = 'Add Group'
	Subscribe-Event $createbutton 'Click' 'OnCreateButtonClicked'
  	$controls.Add($createbutton)
	
	$deletebutton.Text = 'Delete Group'
	Subscribe-Event $deletebutton 'Click' 'OnDeleteButtonClicked'
  	$controls.Add($deletebutton)
	
	$groupnamelabel.Text = "Groupname"
	$controls.Add($groupnamelabel)
}

######## Events ########

## handle control events
## subscribe to an event with "Subscribe-Event($control, 'eventName','callback function name')"
function OnCreateButtonClicked($sender, $Args)
{
   
   $newgroupid = $groupnametext.text
   # Create new group
   new-Group -groupid $newgroupid
   $groupnametext.text = [string]::Empty
   
   
}

function OnDeleteButtonClicked($sender, $Args)
{
   
   $newgroupid = $groupnametext.text
   # Delete group
   Remove-Group -groupid $newgroupid
   $groupnametext.text = [string]::Empty
   
   
}

########## Render  #########

### render html
function Render($writer)
{
    $writer.Write("<table><tr><td>")
	$groupnamelabel.RenderControl($writer)
	$writer.Write("</td><td>")		
	$groupnametext.RenderControl($writer)
	$writer.Write("</td></tr><tr><td>&nbsp</td><td colspan=2>")
  	$createbutton.RenderControl($writer)
	$writer.Write("&nbsp")
	$deletebutton.RenderControl($writer)
  	$writer.Write("</td><td></td></tr></table>")
	
}

######### Error Handling #########

### render custom error message
##function OnError($exception, $writer){}

########## Unload #########

### clean up
##function OnUnload()
##{
##}

########### Initialize ############

. c:ADAMAdminADAM_Lib.ps1

### declare global variables and functions

$groupnametext = New-Object System.Web.UI.WebControls.TextBox

$createbutton = New-Object System.Web.UI.WebControls.Button

$groupnamelabel = New-Object System.Web.UI.WebControls.Label

$deletebutton = New-Object System.Web.UI.WebControls.Button

############## Load ##############

### first time the OnLoad fires before CreateChildControls

#function OnLoad

# # Check if GET Request (first request).

# if($isPostBack -eq $false)

# {

# $label.Text = 'GET request.'

# }

######## 3. Create Controls ########

## create child controls

function CreateChildControls($controls)

{

$controls.Add($groupnametext)

$createbutton.Text = 'Add Group'

Subscribe-Event $createbutton 'Click' 'OnCreateButtonClicked'

$controls.Add($createbutton)

$deletebutton.Text = 'Delete Group'

Subscribe-Event $deletebutton 'Click' 'OnDeleteButtonClicked'

$controls.Add($deletebutton)

$groupnamelabel.Text = "Groupname"

$controls.Add($groupnamelabel)

}

######## Events ########

## handle control events

## subscribe to an event with "Subscribe-Event($control, 'eventName','callback function name')"

function OnCreateButtonClicked($sender, $Args)

{

$newgroupid = $groupnametext.text

# Create new group

new-Group -groupid $newgroupid

$groupnametext.text = [string]::Empty

}

function OnDeleteButtonClicked($sender, $Args)

{

$newgroupid = $groupnametext.text

# Delete group

Remove-Group -groupid $newgroupid

$groupnametext.text = [string]::Empty

}

########## Render #########

### render html

function Render($writer)

{

$writer.Write("<table><tr><td>")

$groupnamelabel.RenderControl($writer)

$writer.Write("</td><td>")

$groupnametext.RenderControl($writer)

$writer.Write("</td></tr><tr><td>&nbsp</td><td colspan=2>")

$createbutton.RenderControl($writer)

$writer.Write("&nbsp")

$deletebutton.RenderControl($writer)

$writer.Write("</td><td></td></tr></table>")

}

######### Error Handling #########

### render custom error message

##function OnError($exception, $writer){}

########## Unload #########

### clean up

##function OnUnload()

##{

##}

The screenshot below shows the complete solution. This method was simple, effective and easy to create. I dot sourced the corresponding web part script and the connection script in each web part.

This is a pretty quick and easy way to expose some simple administrative or user functionality on a SharePoint Intranet.

I hope this helps.

Regards,

Dave

Connection History with PowerShell and NetStat

Posted on March 20, 2009
by dmuegge

Welcome!

This is a little trick some might find useful. I was working on decommissioning some servers and I needed a way to find out what was connecting to these machines. I decided to create a script to log connections. I have done this in the past in various ways which usually involved logging a bunch of data and then querying against it to find the unique connections.

This time it finally occurred to me, just filter the data as it is being collected. So I set out to write a PowerShell script that would keep a running list of client TCP connections to a given machine. This information would be stored in a text file.

The first step was to collect the information and put it into a PowerShell object.

# Create array to hold netstat data
$allnetstatdata = @()

# Get TCP connections from netstat
$netstatResults = & netstat -p TCP 
foreach($line in $netstatResults){

	# Create object data and add to object
	$netstatdata = "" | Select local,port,foreign
	if(-not (($line.Trim() -eq [String]::Empty) -or ($line.contains("Active Connections")) -or ($line.contains("Local Address")))){
		if ($line -match 's*(TCP)s{1,}(DS{1,})s{1,}(DS{1,})s{1,}(DS{1,}$)'){
			$localhost = $matches[2].Split(":")
			$netstatdata.local = $dchost[0]
			$netstatdata.port = $dchost[1]
			$foreignhost = $matches[3].Split(":")
			$netstatdata.foreign = $foreignhost[0]
			$allnetstatdata = $allnetstatdata + $netstatdata
		}
	}			
}

# Create array to hold netstat data

$allnetstatdata = @()

# Get TCP connections from netstat

$netstatResults = & netstat -p TCP

foreach($line in $netstatResults){

# Create object data and add to object

$netstatdata = "" | Select local,port,foreign

if(-not (($line.Trim() -eq [String]::Empty) -or ($line.contains("Active Connections")) -or ($line.contains("Local Address")))){

if ($line -match 's*(TCP)s{1,}(DS{1,})s{1,}(DS{1,})s{1,}(DS{1,}$)'){

$localhost = $matches[2].Split(":")

$netstatdata.local = $dchost[0]

$netstatdata.port = $dchost[1]

$foreignhost = $matches[3].Split(":")

$netstatdata.foreign = $foreignhost[0]

$allnetstatdata = $allnetstatdata + $netstatdata

}

Then the next step was to read the file with the previous information and add it to the PowerShell object.

# Define Log File
$logfile = C:TCPConnections.txt

# Get data from file and add to object
$filedata = Get-Content $logfile
if($filedata){
	foreach($line in $filedata){
		$netstatdata = "" | Select local,port,foreign
		$entry = $line.Split("`t")
		$netstatdata.port = $entry[0]
		$netstatdata.local = $entry[1]
		$netstatdata.foreign = $entry[2]
		$allnetstatdata = $allnetstatdata + $netstatdata
	}
}

# Define Log File

$logfile = C:TCPConnections.txt

# Get data from file and add to object

$filedata = Get-Content $logfile

if($filedata){

foreach($line in $filedata){

$netstatdata = "" | Select local,port,foreign

$entry = $line.Split("`t")

$netstatdata.port = $entry[0]

$netstatdata.local = $entry[1]

$netstatdata.foreign = $entry[2]

$allnetstatdata = $allnetstatdata + $netstatdata

}

We can now remove the duplicates from the combined information and save the updated file.

# Get unique rows
$newnetstatdata = $allnetstatdata | Select-Object * -unique 

# Add combined data to string
$tempcontent = New-Object System.Text.StringBuilder
foreach($netstat in $newnetstatdata){
	if($netstat.port){$tempcontent.Append($netstat.port.ToString()) | Out-Null}
	$tempcontent.Append("`t") | Out-Null 
	if($netstat.local){$tempcontent.Append($netstat.local.ToString()) | Out-Null}
	$tempcontent.Append("`t") | Out-Null 
	if($netstat.foreign){$tempcontent.Append($netstat.foreign.ToString()) | Out-Null}
	$tempcontent.Append("`n") | Out-Null 
}
		
# Write to file
Set-Content -path $logfile $tempcontent.ToString()

# Get unique rows

$newnetstatdata = $allnetstatdata | Select-Object * -unique

# Add combined data to string

$tempcontent = New-Object System.Text.StringBuilder

foreach($netstat in $newnetstatdata){

if($netstat.port){$tempcontent.Append($netstat.port.ToString()) | Out-Null}

$tempcontent.Append("`t") | Out-Null

if($netstat.local){$tempcontent.Append($netstat.local.ToString()) | Out-Null}

$tempcontent.Append("`t") | Out-Null

if($netstat.foreign){$tempcontent.Append($netstat.foreign.ToString()) | Out-Null}

$tempcontent.Append("`n") | Out-Null

}

# Write to file

Set-Content -path $logfile $tempcontent.ToString()

We can run this script in a scheduled task at whatever interval is required. Now we have a log of unique inbound TCP connections.

Best Regards,

Dave

Share this:

Share this:

Share this:

Share this:

Share this:

Share this:

Share this:

Share this:

Share this:

Share this: