Use PowerShell
The Shell Is Calling
The Shell Is Calling
Jun 5th
I was tagged by SQL Server Expert Brent Ozar in his response to a great, thought provoking blog post called Give Me a Coconut and Six Months by Tim Ford (SQLAgentMan on Twitter).
The short summary of the post is if you had six months free of distraction, what would you turn your attention to. Tim’s choices included backups, security, and monitoring, which I think is a great “solid foundation” to work from.
Brent posits that if he became more effective at data mining, he would be able to provide a business with critical insight with which to improve sales, product focus, and develop key personnel.
If I had more time (and skills), I could tell executives things like:
- These are the top five customers who are about to leave us.
- These are the top five products that are about to go viral, and we need to stock more ASAP.
- These are the top five salespeople who need coaching to produce more revenue.
Walk into an executive’s office with this kind of information, and you’re a hero.
Here’s the shocker Brent… I agree.
I agree that delivering that kind of data to management is the Holy Grail of IT projects. Before coming into the IT realm, I ran a small business and I would have killed for information along those lines.
One thing I think is missing from Brent’s discussion is that there are three parts to this type of data mining:
Brent continues on to cover a common area of disagreement between us.
I kick the PowerShell horse a lot, and here it comes again. If you’re in IT, listen up: you’re either cutting costs, or making money. Guess which one has more upside. If you truly bust your hump, become an amazing scripting deity, and save 99% of your time, you just saved 99% of your salary. If you’re really good, you might save 10 people 99% of their time.
I work as the sole admin/accidental DBA/desktop support/multimedia support/”if it has a blinking LED light” support for an agency that collects lots and lots of data. As a scripting practitioner, if I save myself 25%, 40%, or even 60% of my time not having to solve the same problems over and over, I’m free to plume the mysteries of my database and convert the bits stored their into meaningful data and even information that is usable and actionable.
Not every environment has the luxury of being able to afford someone of Brent’s caliber to come in and learn their business and help them develop methods of turning their data into knowledge. In my situation and that of many small to medium businesses, scripting is the tool that enables the IT generalist to explore and branch out into these other areas.
Brent may feel ” you can go into data mining and make 100 salespeople twice as effective” and that IT people who keep things running are replaceable and he may be right, but I believe, especially in tighter economic times, specialists become a luxury that only a few can afford where efficient generalists that know the business become more in demand.
Don’t be mad Brent… it’s okay to be wrong every once in a while!
So if I had six months of no interruptions and could focus on certain specific projects, I would:
I personally think that EVERY technical person should have a grasp on the basic business environment, be aware of what is happening in their company’s industry (or at least their department’s industry for larger organizations), and begin to develop more in depth domain knowledge as to the business processes.
Due to the odd path I took to becoming an IT person, I’ve actually accumulated quite a bit of domain knowledge about the law enforcement, the various jobs, information requirements, and the ins and outs of our workflows and data. This background gives me a great starting place when going to look at my data, since scripting has given me the time to do so.
Judging from the pattern in Tim’s and Brent’s posts, I’m supposed to tag a few people..
How about:
Apr 8th
UPDATE: The script was moved to Google Code. The links in the post have been updated to reflect that. Or you can just go here… http://code.google.com/p/poshcodegen/
I’ve been working on some data conversion at work, converting records from one system to a new system. I’ve built quite a library of SQL queries with PowerShell wrappers for dealing with data in the first system, but I don’t have the same luxury with the new system.
The new system does, however, have a nice set of stored procedures that make moving data into their application much easier.
I started writing my conversion scripts in PowerShell, since I do have to do some processing on the records to accommodate the new workflow and data layout. I was looking at having to call almost 100 stored procedures through various parts of this process. That is a lot of boiler plate code or referring back to the database often to check parameter names and types. So, I’ve written a little PowerShell script that will take stored procedures (either as a parameter or from the pipeline) and create a function that wraps that stored procedure.
The benefits of this are great with V2 CTP3 or with a more advanced editor (one that will provide tab completion on parameters).
One a wider scope, I think that this type of utility is one of PowerShell’s great strengths. Using PowerShell for metaprogramming (another example here on the Telling Machine blog) can be a great time saver. I spent a couple of hours working on this script, but it would have cost me much more time to handle each case individually.
Now, when I hear “metaprogramming”, my head starts to hurt a bit as I start to think about programs about programs about programs, but this isn’t that bad. PowerShell makes this pretty easy to understand though. To create a function dynamically, all that is needed is a string that contains text that the PowerShell runtime can evaluate (PowerShell will check for syntax errors, but not logic errors – as is the case with any script or function).
Example:
$text = ‘Get-ChildItem *.ps1 | Measure-Object’
Set-Item –Path function:global:Get-PowerShellScriptCount –Value $Text
This takes advantage of the Function provider and creates a function object in the global scope with the specified name and $Text is turned into a scriptblock. I can then call that function as needed.
Since I know PowerShell, to build dynamic functions I just have to create text that can be evaluated to do the function I need.
Here’s what my script does after it runs a query against the database to get the stored procedure’s text:
What are you automating with PowerShell?
How about trying to automate some of your automation code?
New-StoredProcFunction.ps1 here.
PSMDTAG:metaprogramming sql stored procedure
UPDATED SCRIPT: Thanks to Chad Miller for the idea.. instead of parsing the text of the stored procedure, the parameter information is available in the Information_Schema.Parameters
Mar 26th
Previously, I published a script on comparing what tables two databases contained. Going a bit further, I put together a script that compares the columns and what type of data they store.
Compare-DatabaseColumns has similar parameters to the Compare-DatabaseSchema script.
This script can also take pipeline input, either strings or a property of “Name” or “TableName”.
You can find this script on PoshCode.org.
Feb 12th
I regularly am working with several versions of a database for an application that I manage (a live database, training database, test database, and previous version database). Occasionally, I need to know what the differences between the databases are, especially after our vendor updates my test environment or right after an update in my training or live environment.
Since I spend a good portion of my day in PowerShell, I wrapped some system table queries in a PowerShell script and use Compare-Object to find any differences in the tables and compare the column definitions as well. The queries targets only user tables.
Compare-DatabaseSchema.ps1 takes several parameters.
I still have to add some checks for the various constraints, but that will come later.
You can find the script on PoShCode.org.