XtremIO PowerShell Module Update

Posted on September 29, 2015
by dmuegge

I have been working with version 4.0 of XtremIO. This version of XtremIO has an updated REST API which provides a bunch of new functionality. The MTSXtremIO PowerShell Module provides PowerShell management of the EMC XtremIO storage array.

This new version of the REST API includes new performance features, new snapshot functionality, and many new objects. I have updated the MTSXtremIO PowerShell module for XtremIO to support the new API and functionality. The current version of the XtremIO PowerShell module contains 85 cmdlets for working with XtremIO.

The first four Cmdlets are to setup a connection to the XtremIO XMS, which is how you connect to the REST API. One XMS can manage multiple XtremIO clusters.

Disable-CertificateValidation

Get-PasswordFromFile

New-PasswordFile

Set-XIOAPIConnectionInfo

There are 41 get Cmdlets to retrieve various XtremIO objects.

Get-XIOPerformance

Get-XIOAlert

Get-XIOAlertDefinition

Get-XIOAPITypes

Get-XIOBBU

Get-XIOBrick

Get-XIOCluster

Get-XIOConsistencyGroup

Get-XIOConsistencyGroupVolume

Get-XIODAE

Get-XIODAEController

Get-XIODAEPSU

Get-XIODataProtectionGroup

Get-XIOEmailNotifier

Get-XIOEvent

Get-XIOInfinibandSwitch

Get-XIOInitiator

Get-XIOInitiatorGroup

Get-XIOIscsiPortal

Get-XIOIscsiRoute

Get-XIOItem

Get-XIOLDAPConfiguration

Get-XIOLocalDisk

Get-XIOLunMap

Get-XIOPerformance

Get-XIOScheduler

Get-XIOSlot

Get-XIOSnapshot

Get-XIOSnapshotSet

Get-XIOSNMPNotifier

Get-XIOSSD

Get-XIOStorageController

Get-XIOStorageControllerPSU

Get-XIOSYRNotifier

Get-XIOSyslogNotifier

Get-XIOTag

Get-XIOTarget

Get-XIOTargetGroup

Get-XIOUserAccount

Get-XIOVolume

Get-XIOXenvs

Get-XIOXms

There are 11 Cmdlets for creating XtremIO objects.

New-XIOConsistencyGroup

New-XIOInitiator

New-XIOInitiatorGroup

New-XIOIscsiPortal

New-XIOIscsiRoute

New-XIOLunMap

New-XIOScheduler

New-XIOSnapshot

New-XIOTag

New-XIOUserAccount

New-XIOVolume

There are 13 Cmdlets for removing XtremIO objects.

Remove-XIOConsistencyGroup

Remove-XIOConsistencyGroupVolume

Remove-XIOInitiator

Remove-XIOInitiatorGroup

Remove-XIOIscsiPortal

Remove-XIOIscsiRoute

Remove-XIOLunMap

Remove-XIOScheduler

Remove-XIOSnapshot

Remove-XIOSnapshotSet

Remove-XIOTag

Remove-XIOUserAccount

Remove-XIOVolume

There are 16 Cmdlets for changing various XtremIO objects.

Add-XIOConsistencyGroupVolume

Set-XIOAlertDefinition

Set-XIOConsistencyGroup

Set-XIOEmailNotifier

Set-XIOInitiator

Set-XIOInitiatorGroup

Set-XIOLDAPConfiguration

Set-XIOScheduler

Set-XIOSnapshot

Set-XIOSNMPNotifier

Set-XIOSYRNotifier

Set-XIOSyslogNotifier

Set-XIOTag

Set-XIOTarget

Set-XIOVolume

Update-XIOSnapshot

This provides a good basic set of functionality and coverage over 100% of the API. There are still many improvements to be made but this provides a good base set of functionality. There are some items which are at the top of the list to address such as;

Improvements in pipeline support across functions. One known issue that exists with the cmdlets is the ability to pipe the output of a get into a set. There is currently a problem with the set cmdlet using property name as input correctly.
Complete multiple cluster support across all functions
Implement ShouldProcess in all new, set, remove functionality.

Other items on roadmap or under consideration

Complete Get-XIOPerformance function. This function only provides very raw object output and has an incomplete implementation.
Implement use of certificate based authentication
Test use of server certificates on XMS
Add informational output objects on new, set, and remove functions
Implement additional error handling to include informational messages

Get the latest version of the MTSXtremIO XtremIO PowerShell Module . I will work on pipeline improvements, multiple cluster support, and support for should process next. These items will provide a solid foundation for the module functionality. I also have a fair amount of help to complete. Please let me know what other items are missing and where things can be improved.

Regards,

Dave

XtremIO Snapshots SQL Server and PowerShell Part IV

Posted on September 23, 2015
by dmuegge

I have continued to work on the MTSXtremIO module and adding functionality for XtremIO 4.0. One interesting feature in 4.0 that provides even more benefit with snapshots is the ability to refresh a snapshot. With SQL server this saves some steps with LUN mapping and mounting volumes. I have done some testing using two methods to control the snapshot. I will talk about both methods and show a scripting example using each method.

The first method uses a PowerShell Cmdlet included in the PowerShell Toolkit in EMC Storage Integrator for Windows. The XtremIO 4.0 features are found in the latest version on the software, version 3.8. To use this method the EMC Storage Integrator for Windows must be installed on the machine to be used for scripting. This could potentially be each of the participating SQL Servers.

Here is an example script refreshing a database using the PowerShell Toolkit included with version 3.8 of EMC Storage Integrator for Windows.

Import-Module ESIPSToolkit
Import-Module MTSMSSQL

# Initial Snapshot Setup and LUN Mapping
$XtremIOCluster = 'XtremIO01'
$SourceVolume = 'DMSQLVol01'
$SourceDatabase = 'SnapTest01'
$TargetDatabaseName = 'SnapTest01_QA'
$TargetDeviceName = 'DMSQLVol01.Snap_20150923-102044'
$TargetSnapDeviceID = '514f0c53d2600159'
$VMName = 'DAMAnalysis'
$DriveLetter = 'K:'
$AccessPath = 'K:\'
$datapath = 'K:\SnapTest01.mdf'
$logpath = 'K:\SnapTest01_log.ldf'


# Detach Database
Set-DBDetach -SQLServer $VMName -dbname $SourceDatabase

# Get Disk Number
$OldDisk = Get-Disk | Where UniqueId -eq $TargetSnapDeviceID

# Offline Disk
Set-Disk -Number ($OldDisk.Number) -IsOffline $true

# Connect to XtremIO
$XioSystem = Get-EmcStorageSystem -ID $XtremIOCluster

# Refresh SnapShot
Restore-EmcXtremIOSnapshots -XioStorageSystem $XIOSystem -FromVolumeName $SourceVolume -ToVolumeName $TargetDeviceName

# Rescan Windows Disks
Update-HostStorageCache

# Get Disk Number
$NewDisk = Get-DIsk | Where UniqueId -eq $TargetSnapDeviceID

# Set mount and drive path 
Set-Disk -Number ($NewDisk.Number) -IsReadOnly $false
Set-Disk -Number ($NewDisk.Number) -IsOffline $false

Start-Sleep -Seconds 1
if(( -not (get-partition -DiskNumber ($NewDisk.Number)).DriveLetter -eq $DriveLetter)){
    Add-PartitionAccessPath -DiskNumber ($NewDisk.Number) -PartitionNumber 1 -AccessPath $AccessPath
}

# Verify Files and attach database
if ((Test-Path -Path $datapath) -and (Test-path -Path $logpath)){
 
	# Attach Database
	$DBAttachResult = Set-DBAttach -dbname $TargetDatabaseName -datapath $datapath -logpath $logpath
 
}

Import-Module ESIPSToolkit

Import-Module MTSMSSQL

# Initial Snapshot Setup and LUN Mapping

$XtremIOCluster = 'XtremIO01'

$SourceVolume = 'DMSQLVol01'

$SourceDatabase = 'SnapTest01'

$TargetDatabaseName = 'SnapTest01_QA'

$TargetDeviceName = 'DMSQLVol01.Snap_20150923-102044'

$TargetSnapDeviceID = '514f0c53d2600159'

$VMName = 'DAMAnalysis'

$DriveLetter = 'K:'

$AccessPath = 'K:\'

$datapath = 'K:\SnapTest01.mdf'

$logpath = 'K:\SnapTest01_log.ldf'

# Detach Database

Set-DBDetach -SQLServer $VMName -dbname $SourceDatabase

# Get Disk Number

$OldDisk = Get-Disk | Where UniqueId -eq $TargetSnapDeviceID

# Offline Disk

Set-Disk -Number ($OldDisk.Number) -IsOffline $true

# Connect to XtremIO

$XioSystem = Get-EmcStorageSystem -ID $XtremIOCluster

# Refresh SnapShot

Restore-EmcXtremIOSnapshots -XioStorageSystem $XIOSystem -FromVolumeName $SourceVolume -ToVolumeName $TargetDeviceName

# Rescan Windows Disks

Update-HostStorageCache

# Get Disk Number

$NewDisk = Get-DIsk | Where UniqueId -eq $TargetSnapDeviceID

# Set mount and drive path

Set-Disk -Number ($NewDisk.Number) -IsReadOnly $false

Set-Disk -Number ($NewDisk.Number) -IsOffline $false

Start-Sleep -Seconds 1

if(( -not (get-partition -DiskNumber ($NewDisk.Number)).DriveLetter -eq $DriveLetter)){

Add-PartitionAccessPath -DiskNumber ($NewDisk.Number) -PartitionNumber 1 -AccessPath $AccessPath

}

# Verify Files and attach database

if ((Test-Path -Path $datapath) -and (Test-path -Path $logpath)){

# Attach Database

$DBAttachResult = Set-DBAttach -dbname $TargetDatabaseName -datapath $datapath -logpath $logpath

}

The Second method is using an open source PowerShell Module I developed to manage XtremIO. It is a PowerShell interface to the XtremIO REST API. It provides a comprehensive PowerShell management interface including the snapshot refresh functionality in 4.0.

Here is an example refreshing a database using the Update-XIOSnapshot Cmdlet found in MTSXtremIO module.

Import-Module MTSXtremIO -Force
Import-Module MTSMSSQL

Disable-CertificateValidation
Set-XIOAPIConnectionInfo -username "admin" -passwordfile "C:\Users\dmuegge\pwd\XtremIO.txt" -hostname "XtremIO01"

# Initial Snapshot Setup and LUN Mapping 
$SourceVolume = 'DMSQLVol01'
$SourceDatabase = 'SnapTest01'
$TargetDatabaseName = 'SnapTest01_QA'
$TargetDeviceName = 'DMSQLVol01.Snap_QA'
$TargetSnapDeviceID = '514f0c53d2600159'
$VMName = 'DAMAnalysis'
$DriveLetter = 'K:'
$AccessPath = 'K:\'
$datapath = 'K:\SnapTest01.mdf'
$logpath = 'K:\SnapTest01_log.ldf'


# Detach Database
Set-DBDetach -SQLServer $VMName -dbname $SourceDatabase

# Get Disk Number
$OldDisk = Get-Disk | Where UniqueId -eq $TargetSnapDeviceID

# Offline Disk
Set-Disk -Number ($OldDisk.Number) -IsOffline $true

# Refresh SnapShot
Update-XIOSnapshot -SourceVol $SourceVolume -DestVol $TargetDeviceName

# Rescan Windows Disks
Update-HostStorageCache

# Get Disk Number
$NewDisk = Get-Disk | Where UniqueId -eq $TargetSnapDeviceID

# Set mount and drive path 
Set-Disk -Number ($NewDisk.Number) -IsReadOnly $false
Set-Disk -Number ($NewDisk.Number) -IsOffline $false

Start-Sleep -Seconds 1
if(( -not (get-partition -DiskNumber ($NewDisk.Number)).DriveLetter -eq $DriveLetter)){
    Add-PartitionAccessPath -DiskNumber ($NewDisk.Number) -PartitionNumber 1 -AccessPath $AccessPath
}

# Verify Files and attach database
if ((Test-Path -Path $datapath) -and (Test-path -Path $logpath)){
 
	# Attach Database
	$DBAttachResult = Set-DBAttach -dbname $TargetDatabaseName -datapath $datapath -logpath $logpath
 
}

Import-Module MTSXtremIO -Force

Import-Module MTSMSSQL

Disable-CertificateValidation

Set-XIOAPIConnectionInfo -username "admin" -passwordfile "C:\Users\dmuegge\pwd\XtremIO.txt" -hostname "XtremIO01"

# Initial Snapshot Setup and LUN Mapping

$SourceVolume = 'DMSQLVol01'

$SourceDatabase = 'SnapTest01'

$TargetDatabaseName = 'SnapTest01_QA'

$TargetDeviceName = 'DMSQLVol01.Snap_QA'

$TargetSnapDeviceID = '514f0c53d2600159'

$VMName = 'DAMAnalysis'

$DriveLetter = 'K:'

$AccessPath = 'K:\'

$datapath = 'K:\SnapTest01.mdf'

$logpath = 'K:\SnapTest01_log.ldf'

# Detach Database

Set-DBDetach -SQLServer $VMName -dbname $SourceDatabase

# Get Disk Number

$OldDisk = Get-Disk | Where UniqueId -eq $TargetSnapDeviceID

# Offline Disk

Set-Disk -Number ($OldDisk.Number) -IsOffline $true

# Refresh SnapShot

Update-XIOSnapshot -SourceVol $SourceVolume -DestVol $TargetDeviceName

# Rescan Windows Disks

Update-HostStorageCache

# Get Disk Number

$NewDisk = Get-Disk | Where UniqueId -eq $TargetSnapDeviceID

# Set mount and drive path

Set-Disk -Number ($NewDisk.Number) -IsReadOnly $false

Set-Disk -Number ($NewDisk.Number) -IsOffline $false

Start-Sleep -Seconds 1

if(( -not (get-partition -DiskNumber ($NewDisk.Number)).DriveLetter -eq $DriveLetter)){

Add-PartitionAccessPath -DiskNumber ($NewDisk.Number) -PartitionNumber 1 -AccessPath $AccessPath

}

# Verify Files and attach database

if ((Test-Path -Path $datapath) -and (Test-path -Path $logpath)){

# Attach Database

$DBAttachResult = Set-DBAttach -dbname $TargetDatabaseName -datapath $datapath -logpath $logpath

}

The way both of these scripts work relies on some initial setup of a snapshot copy mounted to a server. This could be the same or different servers. The diagram below shows the test scenario used in the example scripts above.

In either scenario above the copy process happens very quickly. In my tests it only took a few seconds. The only difference in the scripts above is the command to connect to the resources and the command to refresh the snapshot. Either method is simple and really makes database snapshots easy. The scripting is very straightforward and easy to understand.

The main benefit of the ESI Toolkit is it is officially supported by EMC. It does require an install an just provides some core provisioning functionality. The MTSXtremIO PowerShell Module does not require installation and can be used for many other XtremIO management and reporting tasks. It is an open source project located here.

Regards,

Dave

Share this:

Share this: