Nutonian Eureqa

Eureqa API and SQL Server Data Load

RoundTower Technologies offers an analytics solution called FastAnswers powered by Eureqa. It is an amazing piece of software created by the brilliant people at Nutonian. Eureqa can use sample data to automate the creation of predictive models. If you are interested, the Data Scientists at Nutonian and RoundTower can explain the mathematics and science behind the technology. Please visit the RoundTower and Nutonian links above to learn more about the solution.

One of the main goals of Eureqa is to provide data science skills to non-PHD data analysts. This is why Eureqa is a very intriguing technology to me. I have an interest in analytics and machine learning as well as some BI background, but I do not have a deep understanding of statistics or machine learning. Eureqa is great because it helps bridge the skills gap.

The primary user interface of the Eureqa application is web based, very intuitive, and well documented. Nutonian also provides a Python API for programmatic access to Eureqa functionality. I have been experimenting with Eureqa and the Python API so I thought I would share some things I learned. Hopefully it will be helpful to someone else using Eureqa. Also, if you are interested, the current API documentation is located at http://eureqa-api.nutonian.com/en/stable/ . This contains basic information on using the API and a few helpful examples.

A question that always comes up when talking about Eureqa is “how does it connect to database sources”. So one of the first things I decided to learn about Eureqa was how to load data from SQL Server. The goal of this post is to show the basics of using the API as well as getting data from SQL.

To use the Eureqa Python API you will at least need to have Python 2.7 installed. I prefer to use a distribution that includes Python and many common libraries including the popular machine learning libraries. I also like to use a Python IDE for development, so my preferred environment is Anaconda2 and JetBrains PyCharm. There is also a good Python IDE called Spyder included with Anaconda2. If you need more assistance getting started there is a plethora tutorials on Python, Anaconda2, and various Python IDE’s on the web.

Once the development environment is setup the next step is installing and enabling the API. To start using the API your Eureqa account or local installation must be licensed and given access to use the API. If this has been done you will see the API options shown below on the settings page after logging in to Eureqa.

Eureqa_Settings_API
Eureqa_Settings_API

This page provides the ability to download the Python API and an access key, which are the two things required to use the API. The Eureqa API installation is easy, just use pip from your default Python directory using the command shown on the settings page. The next step is the API key, which is also easy, just click the get access key button on the settings page and the following dialog box is shown, which allows us to give the key a logical name.


Eureqa API KeyGen

After assigning a name click generate key. Then a dialog like the one below is shown with the API key.

Eureqa API KeyGen Result

Now we can use this key to interact with the Eureqa API. The following code example will show how to connect to SQL server retrieve data and load into a Eureqa data set. In this example we will be using data from concrete strength testing, which is publicly available from the University of California Irvine (UCI) machine learning department. The first step in our Python code will be to define our connection variables and load the required libraries, which is shown below.

Once that is done we will define two functions; one to retrieve data from SQL Server and write to a temporary .csv file and one to load data from the temporary .csv file into a Eureqa data source.

Both of these functions are fairly straightforward and self-explanatory. Notice the first function uses the pyodbc and csv Python libraries we loaded in the first step. These libraries provide database access and csv text processing functionality.

The next piece of code is the main part of the application. The first step is to connect to Eureqa, which is done using the Eureqa interface. This is the entrance point for all programmatic operations in Eureqa. After we have a connection to Eureqa we then define our SQL query and execute the two functions to retrieve and load the data.

The screenshots below show what we see in the Eureqa interface before we execute the code. A Eureqa installation with no data sets.

Eureqa Data Set

We execute the script and see a very uneventful output message that tells us the script ran successfully.

Eureqa Script Result

Now after refreshing the Eureqa data sets window there is a new data set called Concrete_Strength_Data.

Eureqa Data Set Result

Here is a subset of the imported data, which strangely enough looks like the data returned from the SQL query in our code.

Eureqa Data Subset

Now that we have a data set loaded it can be used to run searches and build models. So if you happen to be interested in a predictive model to estimate concrete strength. Here is the model Eureqa built based on the UCI concrete strength data, which it solved in minutes. Eureqa!!

Eureqa Model Result

I’ll expand on this next time.

Regards,

Dave

EMC Unity REST API and PowerShell

Earlier this month at EMC World the new Unity storage array was unveiled. There are some cool new features with this product and the architecture is a bit different from the VNX. I like to experiment in my home lab and especially with PowerShell and REST APIs. So when I heard a Unity VSA would be available including the EMC Unity REST API. I was waiting for the EMC World release so I could get my hands on it.
Unity

I downloaded the EMC Unity VSA and set it up the first day of EMC World, then started some initial testing with the Unity REST API and PowerShell. I quickly found some distinct differences in the API when compared to the Isilon and XtremIO API’s. The first difference is the authentication mechanism which took some time to figure out. I found some quirks with the PowerShell Invoke-RestMethod and Invoke-WebRequest cmdlets. These cmdlets did not provide enough control over the web requests so I had to resort to creating my own with the .NET framework.

The EMC Unity REST API uses basic authentication via a request header, but also uses login session cookies. While trying to get this working I found I needed to use the .NET class System.Net.HttpWebRequest object for greater control. The two problems I had with the PowerShell cmdlets was the inability to control auto redirect and use request cookies. The use of request cookies is required to maintain session after initial login. The data required to create one of the cookies was returned in a response after an auto redirect so this had to be turned off to capture the information successfully.

The two cookies which have to be captured during the login process are MOD_AUTH_CAS_S and mod_sec_emc which are then used in all subsequent requests to the API. There are also a couple of additional cookies which are passed back and forth during the authentication process. I created a couple of functions to complete the login process, one of which is a recursive function which handles the auto redirects and collects the required login session cookies into global variables. The complete code is more than makes sense to show in this post, but the below example shows the main elements of building the request. This code is called recursively and collects the required session information to be passed in subsequent requests.

Once the login process is complete and the required session cookies are collected data requests to the EMC Unity REST API can be issued. The below code is an example of issuing a request for system information.

The above code also includes the header EMC_CSRF_TOKEN which is actually only required when doing a POST or DELETE. Another thing to notice in the code above is the use of the fields query string. The desired object properties to be returned must be specified using this method. The output is below.

The EMC Unity REST API was a bit of a challenge to get started and took some time with fiddler to figure out. I will say though for the initial release, the API documentation was pretty good. Unity does seem pretty cool, easy to use, and it’s awesome to have the Unity VSA for experimentation. I will try to get some more comprehensive example code on GitHub soon.

Regards,

Dave

EMC VNXe Performance PowerShell Module

vnxePoSH

I thought I would revisit the VNXe performance analysis topic. In 2012 I published some posts around performance analysis of an EMC VNXe storage array. This information applies only to the 1st generation VNXe arrays not to the newer 2nd generation arrays.

In my previous articles I posted a PowerShell module to use to access the VNXe performance database. Since that time I fixed a bug around rollover of timestamps and made a couple other small improvements. The module has also now been published to GitHub.

Here is some background and you can see my previous posts for additional information.

http://muegge.com/blog/emc-vnxe-performance-analysis-with-powershell/

http://muegge.com/blog/emc-vnxe-performance-analysis-with-powershell-part-ii/

The VNXe collects performance statistics in a sqlite database, which can be accessed via scp on the array controllers. It is also available via the diagnostic data package, which can be retried via the system menu in Unisphere. There are a few database file which hold different pieces of performance data about the array. The MTSVNXePerformance PowerShell module provides cmdlets to query the database files and retrieve additional performance information over what is provided in the Unisphere GUI.

I will show some examples of using the module to get performance information. The first one is a simple table with pool capacity information.

The first step is to load the modules, set file path variables, and the sqlite location. This also uses a module to provide charting from the .Net MSChart controls.

The next step is to get some data from the VNXe sqlite tables.

This give us the information we need about pools. So now we can look at rollup Information by using a command like so.

Next we will look at IOPS from the dart summary data. Data is stored in different tables based on type of information and time period. As data is collected it I summarized and moved to historical tables storing longer time periods at less data resolution. Here we are going to get dart store stats which gives us all IO information for each of the data movers

This produces the following charts using the MSChart .Net charting controls.

 

The module can be used to produce complete VNXe performance reports like the one below.

The script that produces the report above is included in the examples folder in the GitHub project.
 

I recieved a fair amount of interest on the first posts around this topic. I hope this update and refresher is still useful to some folks.

Regards,

Dave

XtremIO PowerShell Module Update

I have been working with version 4.0 of XtremIO. This version of XtremIO has an updated REST API which provides a bunch of new functionality. The MTSXtremIO PowerShell Module provides PowerShell management of the EMC XtremIO storage array.

This new version of the REST API includes new performance features, new snapshot functionality, and many new objects. I have updated the MTSXtremIO PowerShell module for XtremIO to support the new API and functionality. The current version of the XtremIO PowerShell module contains 85 cmdlets for working with XtremIO.

The first four Cmdlets are to setup a connection to the XtremIO XMS, which is how you connect to the REST API. One XMS can manage multiple XtremIO clusters.

Disable-CertificateValidation

Get-PasswordFromFile

New-PasswordFile

Set-XIOAPIConnectionInfo

 

There are 41 get Cmdlets to retrieve various XtremIO objects.

Get-XIOPerformance

Get-XIOAlert

Get-XIOAlertDefinition

Get-XIOAPITypes

Get-XIOBBU

Get-XIOBrick

Get-XIOCluster

Get-XIOConsistencyGroup

Get-XIOConsistencyGroupVolume

Get-XIODAE

Get-XIODAEController

Get-XIODAEPSU

Get-XIODataProtectionGroup

Get-XIOEmailNotifier

Get-XIOEvent

Get-XIOInfinibandSwitch

Get-XIOInitiator

Get-XIOInitiatorGroup

Get-XIOIscsiPortal

Get-XIOIscsiRoute

Get-XIOItem

Get-XIOLDAPConfiguration

Get-XIOLocalDisk

Get-XIOLunMap

Get-XIOPerformance

Get-XIOScheduler

Get-XIOSlot

Get-XIOSnapshot

Get-XIOSnapshotSet

Get-XIOSNMPNotifier

Get-XIOSSD

Get-XIOStorageController

Get-XIOStorageControllerPSU

Get-XIOSYRNotifier

Get-XIOSyslogNotifier

Get-XIOTag

Get-XIOTarget

Get-XIOTargetGroup

Get-XIOUserAccount

Get-XIOVolume

Get-XIOXenvs

Get-XIOXms

 

There are 11 Cmdlets for creating XtremIO objects.

New-XIOConsistencyGroup

New-XIOInitiator

New-XIOInitiatorGroup

New-XIOIscsiPortal

New-XIOIscsiRoute

New-XIOLunMap

New-XIOScheduler

New-XIOSnapshot

New-XIOTag

New-XIOUserAccount

New-XIOVolume

 

There are 13 Cmdlets for removing XtremIO objects.

Remove-XIOConsistencyGroup

Remove-XIOConsistencyGroupVolume

Remove-XIOInitiator

Remove-XIOInitiatorGroup

Remove-XIOIscsiPortal

Remove-XIOIscsiRoute

Remove-XIOLunMap

Remove-XIOScheduler

Remove-XIOSnapshot

Remove-XIOSnapshotSet

Remove-XIOTag

Remove-XIOUserAccount

Remove-XIOVolume

 

There are 16 Cmdlets for changing various XtremIO objects.

Add-XIOConsistencyGroupVolume

Set-XIOAlertDefinition

Set-XIOConsistencyGroup

Set-XIOEmailNotifier

Set-XIOInitiator

Set-XIOInitiatorGroup

Set-XIOLDAPConfiguration

Set-XIOScheduler

Set-XIOSnapshot

Set-XIOSNMPNotifier

Set-XIOSYRNotifier

Set-XIOSyslogNotifier

Set-XIOTag

Set-XIOTarget

Set-XIOVolume

Update-XIOSnapshot

 

This provides a good basic set of functionality and coverage over 100% of the API. There are still many improvements to be made but this provides a good base set of functionality. There are some items which are at the top of the list to address such as;

  • Improvements in pipeline support across functions. One known issue that exists with the cmdlets is the ability to pipe the output of a get into a set. There is currently a problem with the set cmdlet using property name as input correctly.
  • Complete multiple cluster support across all functions
  • Implement ShouldProcess in all new, set, remove functionality.

 

Other items on roadmap or under consideration

  • Complete Get-XIOPerformance function. This function only provides very raw object output and has an incomplete implementation.
  • Implement use of certificate based authentication
  • Test use of server certificates on XMS
  • Add informational output objects on new, set, and remove functions
  • Implement additional error handling to include informational messages

 

Get the latest version of the MTSXtremIO XtremIO PowerShell Module . I will work on pipeline improvements, multiple cluster support, and support for should process next. These items will provide a solid foundation for the module functionality. I also have a fair amount of help to complete. Please let me know what other items are missing and where things can be improved.

 

Regards,

 

Dave

 

XtremIO Snapshots SQL Server and PowerShell Part IV

I have continued to work on the MTSXtremIO module and adding functionality for XtremIO 4.0. One interesting feature in 4.0 that provides even more benefit with snapshots is the ability to refresh a snapshot. With SQL server this saves some steps with LUN mapping and mounting volumes. I have done some testing using two methods to control the snapshot. I will talk about both methods and show a scripting example using each method.

The first method uses a PowerShell Cmdlet included in the PowerShell Toolkit in EMC Storage Integrator for Windows. The XtremIO 4.0 features are found in the latest version on the software, version 3.8. To use this method the EMC Storage Integrator for Windows must be installed on the machine to be used for scripting. This could potentially be each of the participating SQL Servers.

  1. Here is an example script refreshing a database using the PowerShell Toolkit included with version 3.8 of EMC Storage Integrator for Windows.

    The Second method is using an open source PowerShell Module I developed to manage XtremIO. It is a PowerShell interface to the XtremIO REST API. It provides a comprehensive PowerShell management interface including the snapshot refresh functionality in 4.0.

  2. Here is an example refreshing a database using the Update-XIOSnapshot Cmdlet found in MTSXtremIO module.

The way both of these scripts work relies on some initial setup of a snapshot copy mounted to a server. This could be the same or different servers. The diagram below shows the test scenario used in the example scripts above.

In either scenario above the copy process happens very quickly. In my tests it only took a few seconds. The only difference in the scripts above is the command to connect to the resources and the command to refresh the snapshot. Either method is simple and really makes database snapshots easy. The scripting is very straightforward and easy to understand.

The main benefit of the ESI Toolkit is it is officially supported by EMC. It does require an install an just provides some core provisioning functionality. The MTSXtremIO PowerShell Module does not require installation and can be used for many other XtremIO management and reporting tasks. It is an open source project located here.

Regards,

Dave

Hyper-V Cluster with Storage Spaces

I was recently discussing the topic of using Microsoft Storage Spaces in conjunction with Hyper-V and failover clustering. The Hyper-V cluster with Storage Spaces architecture has been around for a couple of years. Storage Spaces was introduced in Windows Server 2012 and was improved in Windows Server 2012 R2. There are many great resources available on the topic so I will not cover the details here. If you would like an overview of the architecture here are some great places to start.

Aidan Finn’s Blog – http://www.aidanfinn.com/2013/07/yes-you-can-run-hyper-v-on-your-storage-spaces-cluster-sofs/ – This talks about the architecture I am following. He also has a ton of additional information on Hyper-V, Storage Spaces, and other Microsoft virtualization topics.

TechNet is always a great place to start as well as the Microsoft blogs. Here are some links I found useful.

Failover Clustering – https://technet.microsoft.com/en-us/library/hh831579.aspx

Hyper-V and Failover Clustering – https://technet.microsoft.com/en-us/library/Cc732181(v=WS.10).aspx

Storage Spaces – https://technet.microsoft.com/en-us/library/hh831739.aspx

Microsoft Clustering and High Availability Blog – http://blogs.msdn.com/b/clustering/archive/2012/06/02/10314262.aspx

Now on to the primary point of this post. A couple colleagues of mine said they would like to see a demo of this architecture and in particular a successful failover of a virtual machine. I have seen some videos with parts of the configuration of Hyper-V failover clustering and also Storage Spaces. However, I was not able to find any demonstrations of the failover process. I decided to setup a cluster and create a demo video.

First I will explain the lab environment I used. It is a basic cluster-in-a-box using a virtualized Hyper-V failover cluster on ESXi 5.5. Hyper-V on Server 2012 R2 does not support nested virtualization, but Microsoft says it’s coming in Windows Server 2016. VMware ESXi 5.5 does support nested virtualization with a few small tweaks which you can read about on Derek Seaman’s Blog. Here is the basic setup I used.

Here is the VMware configuration of each Hyper-V virtual host.

There are a couple of items to call out here. The clustered disks use a different SCSI controller from the OS disks which are set to allow SCSI bus sharing for the virtual disks. Hard Disk 2 is used as a disk witness for the cluster quorum and Hard Disks 3-8 are used for the Storage Spaces pool and virtual volume. This volume becomes the clustered shared volume for virtual machine storage. The screenshots below show this configuration.

The Hyper-V failover cluster is running one Windows 7 virtual machine. In the demo video below, I will show the failover process for this VM by simulating the loss of the active cluster host.

One aspect of this configuration, which has some real benefits is a low cost and highly available virtualization solution that does not require and additional software other than Windows. While the cluster configuration in VMware is still more simplified it requires the use of VCenter and the additional cost.

Regards,

Dave

SQL Server, PowerShell, and XtremIO Snapshots Part 3 (VSS Native)

During the writing of this post EMC announced the GA release of XtremIO 4.0 on July 2nd. The new documentation and the native VSS provider are now available on EMC support. This will provide the ability to script application consistent snapshots without using AppSync. Unfortunately we do not have our lab upgraded to 4.0 yet, but that will be coming soon and I will test the script and process soon. In the meantime I will talk about how this will provide another way to do application consistent snapshots for XtremIO. I will show the architecture and a mock script of how I think it will work at this point.

In order to use the XtremIO VSS provider it must be installed on the server where we want to do an application consistent snapshot. The install is downloaded from EMC support, at the time of this post the file name for the latest version is XtremIOVSSProvider-1.0.8.msi. After installation the connection to XtremIO is configured through the control panel using the applet.

The control panel applet is only to configure the connection to the XtremIO XMS.

The VSS Provider installation can be verified by opening a command line as administrator and typing vssadmin list providers. This shows us the XtremIO VSS Provider.

The process to use an XtremIO snapshot for a database copy using the VSS provider is very similar to the process used in part 1 of this blog series. The primary difference is the VSS Provider is called to create the XtremIO snapshot. The following image shows the basic VSS architecture.

The test environment is a SQL Server virtual machine on vSphere. The SnapTest01 volume is on a 50GB RDM on XtremIO and the SnapTest01_QA volume is a snapshot of the SnapTest01 volume.

 

The example script will show the process to refresh the QA volume with a new snapshot copy. The script is almost identical to the part 1 script. The first step is to load a few PowerShell modules, define some constants, and connect to VCenter and XtremIO. This is done by using the PowerCli and a function from my MTSXtremIO module, read about that here. This function uses the XtremIO REST API to create the snapshot. I also use a couple of other modules with some of my common functions and a NTFS Security Module which I did not write. I will put links to those at the end of the post.

The example above loads module dependencies and connects to VCenter and XtremIO. The SQL Management Objects are loaded to provide SQL Server functionality.

The next step is to detach the current QA database copy, remove the virtual hard disk, and remove snapshots.

The example above uses SQL Management Objects to access SQL and detach the database. It the uses the VMware PowerCli to remove the RDM from the virtual machine. Then connects to the XtremIO via REST API and deletes snapshots

Now we are ready to create a new snapshot, add it to the lunmap, add the disk to the vm, and attach the database.

The above example creates a snapshot via the VSS provider and maps the volume to the host using the XtremIO REST API. It also rescans the disks and then adds the RDM to the virtual machine. Then the database is attached using SQL SMO.

Although I have not been able to test this code it as we need to upgrade XtremIO first. I had hoped the VSS provider would work with 3.0 but unfortunately I received the following message when I tried it.

“The registered provider does not support shadow copy of this volume”

Hopefully this example should be pretty close.

Regards,

Dave

MTSXtremIO Module
NTFSSecurity Module
MTSAuthentication Module
 

SQL Server, PowerShell, and XtremIO Snapshots Part 2 (EMC AppSync)

In my last post I talked about using PowerShell to do crash consistent snapshots on XtremIO. Then mount the snapshot and use for a secondary SQL Server database copy.

In this post I will talk about scripting application consistent XtremIO snapshots using EMC AppSync and EMC Storage Integrator for Windows (ESI). I will also talk about how the process and architecture differs from part 1 where we just did crash consistent snapshots. EMC AppSync is a server application that can create and manage snapshots for application and virtualization environments using EMC storage arrays. It uses a Host plug-in which allows the AppSync server to control host operations. ESI is a windows application and set of tools which provides EMC management capabilities from a windows machine. One of the features of ESI is a PowerShell toolkit that allows controlling EMC arrays and EMC AppSync using PowerShell CmdLets.

We’ll use the same basic scenario as the last time, but utilizing EMC AppSync. The test environment is a SQL Server virtual machine on vSphere. The SnapTest01 volume is on a 50GB RDM on XtremIO and the SnapTest01_QA volume is a snapshot of the SnapTest01 volume.

With our previous test the script would generally run on the SQL Server to perform the snapshot operations. With the AppSync application the script does not have a requirement to run locally as the agent performs local operations. Here is a diagram of the architecture.

070115_2119_SQLServerPo2

 

In this scenario the script would run on the ESI host using a windows account that has SQL admin rights on the SQL server. This is required to attach the database. The latest version of appsync as of this writing (2.2) does not have the ability to attach a copy from an XtremIO volume. Here is the statement from the admin guide on page 105.

This version of Appsync does not support automated restore of copies on XtremIO. Learn

to use AppSync and XtremIO to restore SQL Server databases to a point in time with a

semi-manual restore process.”

However, this is easily worked around using PowerShell and SQL SMO to fill in the gap. The end script is still simpler than in part 1 because AppSync does a lot of the tedious work for us and we get application consistent snapshots to boot. The other benefit of the architecture is it allows you to use a central host to control all of your database refresh operations. Here is the first part of the script which loads the ESI module, SQL SMO, and defines some constants.

 

 

The next section detaches the database, initializes the AppSync Objects, gets and dismounts the current snapshot.

 

 

The detach database operation uses SQL Server Management Objects and the AppSync objects are all created using the ESI PowerShell Toolkit CmdLets.

The next section creates a new snapshot mounts it to the host and attaches the database.

 

 

This also uses ESI for the AppSync operations and SQL SMO for the database attach.

An important point to note is we did not have to create steps for all of the individual disk operations and LUN mapping. The AppSync server and host agent took care of the details for us.

That is it for application consistent snapshots with AppSync. Next time I will talk about the method we will be able to use when XtremIO 4.0 gets here and the native VSS Provider is available.

Regards,

Dave

SQL Server, PowerShell, and XtremIO Snapshots Part 1

XtremIO is becoming a popular platform for SQL Server. It performs excellent and has a great space benefit for database copies when using XtremIO snapshots. I have had a few customers asking questions about scripting snapshots for SQL Server on XtremIO using PowerShell.

Most Windows administrators these days are using PowerShell and many SQL DBA’s are also starting to use it. I decided to setup EMC AppSync with XtremIO in our lab, do a little testing, and create some PowerShell scripts to help our customers get started. My plan was to test two different scenarios, one for crash consistent and one for application consistent snapshots with EMC AppSync, but EMC announced a new native VSS provider with XtremIO 4.0 so now there will also be another way to do application consistent snapshots when 4.0 is released. This post will cover creating a crash consistent copy of a SQL database volume using XtremIO snapshots and mounting the snapshot to create a secondary QA database. This method will use the XtremIO REST API to create a snapshot of the source volume. PowerShell will be used to execute the required steps on XtremIO, VMware, and Windows.

The test environment is a SQL Server virtual machine on vSphere. The SnapTest01 volume is on a 50GB RDM on XtremIO and the SnapTest01_QA volume is a snapshot of the SnapTest01 volume.

 

SQL_Server

The example script will show the process to refresh the QA volume with a new snapshot copy. Here is the basic logical flow of the process.

SnapSshot

The first step is to load a few PowerShell modules, define some constants, and connect to VCenter and XtremIO. This is done by using the PowerCli and a function from my MTSXtremIO module, read about that here. This function uses the XtremIO REST API to create the snapshot. I also use a couple of other modules with some of my common functions and a NTFS Security Module which I did not write. I will put links to those at the end of the post.

The example above loads module dependencies and connects to VCenter and XtremIO. The SQL Management Objects are loaded to provide SQL Server functionality.

The next step is to detach the current QA database copy, remove the virtual hard disk, and remove snapshots. We will also do a couple of rescans in the process and remove the old snapshots.

The example above uses SQL Management Objects to access SQL and detach the database. It the uses the VMware PowerCli to remove the RDM from the virtual machine. Then connects to the XtremIO via REST API and deletes snapshots

Now we are ready to create a new snapshot, add it to the lunmap, add the disk to the vm, and attach the database.

The above example creates a snapshot and maps the volume to the host using the XtremIO REST API. It also rescans the disks and then adds the RDM to the virtual machine. Then the database is attached using SQL SMO.

This completes the database refresh. I hope someone finds this helpful. Next post will introduce AppSync into the mix for application consistency and some additional benefits.

Regards,

Dave

MTSXtremIO Module
NTFSSecurity Module
MTSAuthentication Module

EMC Knowledge Sharing Competition – Article Selected for Publishing

In the fall last year I learned about the 2015 EMC Knowledge Sharing Competition. This is a writing competition hosted by EMC to promote sharing of information about technology topics. All EMC certified professionals are welcome to submit abstracts for consideration in the contest. I thought the work I had done with Data Domain Boost (DD Boost) for SQL Server would a good topic and decided to give the contest a shot.

The previous work I have done on this topic can be found at the links below.

Data Domain Boost for SQL – Deployment Planning Considerations for DBAs

SQL Server Data Domain Boost Scripting Toolkit

Toolkit Project on GitHub

In November I was notified my abstract was accepted and it was time to get started writing the article. It was a lot of work especially because I was in the middle of writing and publishing the SQL Server Data Domain Boost Scripting Toolkit. It turned out to be a great experience and last week I was notified my article, How to Boost SQL Server Backups with Data Domain, was selected for publishing. It will be included in the 2015 book of abstracts and published at some point during the year.

Find our more about the EMC Knowledge Sharing Competition here

While I was not selected as a finalist, the experience was enjoyable and interesting. I think this is a very cool event EMC does, I appreciate being part of it, and I’m sure it takes a lot of work internally. I will be looking forward to the awards session at EMC World 2015 to hear the winners announced and congratulate them.

Regards,

Dave