Blog Archives
My essential Python toolbox
Since Python 3.0 was released I’m regularly checking essential python packages that I currently use. And since I’m visiting more or less the same sites on a regular basis I decided to write down my list of packages that I use with the flag [python 3=Y, x86=Y|N, x64=Y|N] if there is at least support for Python 3 on Windows:
-
Essential packages:
- Python interpreter [python 3.1=Y, x86=Y, x64=Y]
- Python Extensions for Windows [python 3.1, x86=Y, x64=Y]
- cx_Oracle [python 3.1=Y, x86=Y, x64=Y]
- PyQt4 – GPL version [python 3.1=Y, x86=Y, x64=N]
- Py2exe [python 3.1=N]
- cx_freeze [python 3.1=Y, x86=Y, x64=Y]
- matplotlib
- MySQL for Python
- ActiveState recipe – Colorize Python source [N/A]
- ReportLab Toolkit (pdf library)
And some less often used modules:
[Note: Last update on February 16, 2010]
VMWare Infrastructure Toolkit for Windows
It was only a question of time when it will be released – a VMWare specific toolkit for Windows PowerShell. I hope I’ll have a time to test this toolkit soon with VMWare Server 2.0 (after it’ll be released); nevertheless, even on paper it sounds promising with 120+ powershell cmdlets centered around managing and automation of VM.
Note: As you might know I was not a particular fan of PowerShell, but I changed my mind :-). I was skeptical about PowerShell when Microsoft announced new scripting environment (what’s wrong with WSH that I’m already familiar with?), but now I understand the importance and potential of this product, it really deserves power in it’s name. If you’re Windows system administrator you’re better off to start learning Powershell right now (if you’re already not) – it’ll make your life much easier and your employment prospect safer;-).
sed one-liners
SED and AWK are my favorites when it comes to (pre)processing data (…before I go and write python script). While my primary source of reference is book titled Sed & Awk, 2nd ed. (O’Reilly), I also have some prints of various sed one-liners hanging on the wall in my office. I never bothered to write down url on which we can find many “popular” sed one-liners, so here it is:
http://sed.sourceforge.net/sed1line.txt
Just an example of how handy this list of one-liners can be. It helped me the other day when someone on the dba-village forum asked about how to load data from comma separated file. There are five fields in the file, but the problem is that the third field may contain any number of commas. (This is a common type of a question that I receive from time to time at my workplace as well — and I almost always resolve for a solution to the sed.)
For example, let’s say that we have file (test.txt) with comma separated data such as this:
COL1,COL2,COL3-with, some, data, with, commas,COL4,COL5
We should enclose field 3 within quotes and then use OPTIONALLY ECNLOSED BY clause in SQL*Loader control file to load the data.
Basically we need to replace second comma with ,’ and next to last comma with ‘, .
The first part is easy:
cmd> cat test.txt | sed "s/,/,'/2" ....will replace second comma with ,'
The second part, replacing next to the last comma with ‘, is more complex and was finally derived from an example written in the one-liner list mentioned above:
cmd> cat test.txt | sed "s/\(.*\)\,\(.*\,\)/\1',\2/"
We're using grouping ( ) and then addressing each of the two groups with 1 and 2. Backslashes are there to escape characters that are part of the syntax, but admittedly making command line string harder for humans to parse. But consider this, lighter example where we want to split days with two consecutive semicolons) :
cmd> echo MondayTuesday | sed "s/\(Monday\)\(Tuesday\)/\1::\2/"
Monday::Tuesday
So that in the end we can get desired result:
cmd> cat test.txt | sed "s/,/,'/2" | sed "s/\(.*\)\,\(.*\,\)/\1',\2/" > test_out.txt
cmd> cat test_out.txt
COL1,COL2,'COL3-with, some, data, with, commas',COL4,COL5
Free PowerShell Commands for Active Directory
Only recently I stumble over Quest “ActiveRoles Management Shell for Active Directory”, a free cmdlets for scripting with AD as a source (and destination). For example, all you need to spool users with their descriptions from AD into csv file is this one liner:
[PS] E:\>Get-QADUser -sizeLimit 0 | select name, Description | Export-Csv ADusers.csv
In addition to “ActiveRoles Management Shell for Active Directory”, you’ll need to install PowerShell itself and Microsoft Core XML Services – if they’re not already installed on your machine (Quest installer will prompt you if that’s is the case).
In “Quest Software” program directory you’ll find link named “ActiveRoles Management Shell for Active Directory” which starts PowerShell.
One problem that I could not find a solution for, is related to spooled file encoding which will loose all non-ASCII characters in my environment (CP1250 for GUI, CP852 for command prompt), even after specifying UTF8 encoding like this:
[PS] E:\>Get-QADUser -sizeLimit 0 | select name, Description | Export-Csv ADusers.csv [PS] E:\>$OutputEncoding BodyName : utf-8 EncodingName : Unicode (UTF-8) HeaderName : utf-8 WebName : utf-8 WindowsCodePage : 1200 IsBrowserDisplay : True IsBrowserSave : True IsMailNewsDisplay : True IsMailNewsSave : True IsSingleByte : False EncoderFallback : System.Text.EncoderReplacementFallback DecoderFallback : System.Text.DecoderReplacementFallback IsReadOnly : True CodePage : 65001
I also tried with console encoding:
[PS] E:\>$OutputEncoding = [console]::OutputEncoding [PS] E:\>$OutputEncoding IsSingleByte : True BodyName : ibm852 EncodingName : Central European (DOS) HeaderName : ibm852 WebName : ibm852 WindowsCodePage : 1250 IsBrowserDisplay : True IsBrowserSave : True IsMailNewsDisplay : False IsMailNewsSave : False EncoderFallback : System.Text.InternalEncoderBestFitFallback DecoderFallback : System.Text.InternalDecoderBestFitFallback IsReadOnly : True CodePage : 852
No luck – in .csv file all “special” characters are lost.
PowerShell columns by Jeffery Hicks
As a long time subscriber on MCP Magazine I also get regular weekly newsletter from them. What I really like to read lately are columns by Jeffery Hicks, introducing the real power of PowerShell with short and simple examples, hence serving me well to learn some basics.
Some direct links to columns by Jeffery Hicks about PowerShell:
- Tee Off
- On File
- PowerShell AKA
- Smooth Operator
- The Two Faces of PowerShell
- PowerShell in Lockdown Mode
- The Big Kahuna
- Member(ship) Has Its Privileges
- Help Is On The Way
- Automated Power for IT
- “Clean” and Mean PowerShell Sessions
- How To Make Windows Spill the Beans
- Jog Your Memory with PowerShell
- Profile of a PowerSheller
- PowerShell Title Prompts
- Look Mom, No Scripting!
- Switching Days in PowerShell
- Color My World
- Put Some Power in Your Event Logs
- PowerShell Pipe Dream
dbaportal.eu 