Spring cleaning at the Exchange server

Windowclipping_71
One of our clients had a problem this morning -- mail just wouldn't arrive. The mail server was up, but unfortunately the c:-disk had become so filled that Exchange hit the parking break. I managed to squeeze some air from temporary files and the mail flow continued.

The root problem was that many users have huge mailboxes. As a collective excercise with the customer, i decided to try and hit them with some stats, so that only the people with actually huge mailboxes need be bothered with "clean up your mess"-nags from the administration.

So here's what i did. It's quick and dirty and it's built on mostly manual labour.

Step one - get the numbers 

Connect to the Exchange server. Fire up Exchange PowerShell. Enter the following chant:

Get-MailboxStatistics |ft displayname,totalitemsize,itemcount

Since our customer only has as many mailboxes that can fit one screen, i then selected all the text lines, copied them and pasted them into Excel twenty-ten (or whatever your poison).

Massage the data 

After pasting the lines, click the Data tab and the Text to columns button. The defaults will guess what you want, which is fixed-width data.

After getting your data into three columns, manually remove the "B" symbol at the end of each line. I'm sure you can do that with some nifty scripting, but with only sixty or seventy boxes, a succession of F2-Del-Enter pressings was the fastest. If you have a proper number of mailboxes, it's worth thinking of the automation bits.

Prettify

Give your columns some nifty headings (Mailbox, size, items). Add a new column. Call it Size in MB and give it the value of the left-neighbour cell divided by (1024*1024). Mark all of your data and choose Home -> Format as a table. Hide the column with size in bytes.

Now select the values in the column Size in MB. Select Conditional formatting -> Data bars -> Gradient fill (go for red ones for effect).

Send to the nice person in the admin at the customer and smile.

 

Disk space alarms for Exchange 2007/2010 and MS SQL Server 2008 using PowerShell

Disks fill up, and when they do, bad things happen. If the data disks for the mail or database servers fill up, mail stops and the production halts. Not good.

Fortunately, it's possible to get an early warning for Exchange 2007/2010 and SQL Server 2008 R2, using PowerShell. Both server products come with their customized PowerShell modules, which you'll need to use to make the magic happen.

Obligatory disclaimer: I am not a coder and i'm a PowerShell neophyte. I'm most likely making stoopid noobie errors here, but at least i get the job done with this.

The first question to have answered when monitoring disk space is which disk volumes should i be watching? The system drive and the drive(s) with the databases or mailboxes, of course. But which disks are they on?

There are two ways to answer that question. Either you go to the Exchange or SQL Server management interface and browse, or you create a script which asks for the information and works from there. As the second option creates a portable solution, that's the path for me.

Exchange

Let's begin with the Exchange server. The following will list of the names and database files for the mailbox databases on the server:

Get-MailboxDatabase | select Name,EdbFilePath

If you have a slightly bigger organization you'll probably have more than one database, otherwise you'll end up with a single line answer.

Then you iterate through the list to get the drive the database file is on and ask WMI how much space there is free and in total on the disk:

Get-MailboxDatabase | select Name,EdbFilePath | foreach { 
  $drive = $_.EdbFilePath.DriveName
  $disk = Get-WmiObject Win32_LogicalDisk -filter "DeviceID='$drive'"
  $free = 100 * $disk.FreeSpace / $disk.Size
}

We now need to check whether "free" is below an alarm limit, so add this to the top of the script

$alarm = 10

...and this before the last bracket...

if ($free -lt $alarm) {
  $msg = $_.Name + " @ $drive {0:n1}% free" -f $free
  $msg
}

This would output a warning for each volume containing an Exchange mail store with less than ten percent free space (with one decimal, that's what the {0:n1} stuff does).

If you'd rather have the message go to the Event log, then change the last $msg line to

Write-EventLog -logname Application -source Application -eventID 42000 -entrytype Warning -message $msg

Change the values for -source and -eventID to something which makes sense for your system. I'm no Event log guru, so i do not know what values are sensible or not. Since we use Kaseya VSA on the systems we monitor, my source is KaseyaCollector.

SQL Server

The story is pretty much the same on SQL server, except that you need to run it on the SQL PowerShell interface, sqlps.exe

First, you connect to the database server's management bits and from there on it's much like with the Exchange server script. There is one ugly hack that i'm particularly ashamed of, and that's when i typecast the Root property into a string to strip off the trailing slash from it so that i can use it in the WMI query. I'm sure it's possible to get the information without throwing out the semantics, but i haven't found it yet. Suggestions welcome.

$alarm = 10

$srv = New-Object Microsoft.SqlServer.Management.SMO.Server("(local)")
$srv.Databases | select Name,PrimaryFilePath | foreach {

  $vol = Get-ItemProperty -path $_.PrimaryFilePath | select Root
  $did = $vol.Root.Name.substring(0,2)
  $disk = Get-WmiOhject Win32_LogicalDisk -filter "DeviceID='$did'"
  $free = 100 * $disk.FreeSpace / $disk.Size

  if ($free -lt $alarm) {
    $msg = $_.Name + " @ $did {0:n1}% free" -f $free
    Write-Eventlog -logname Application -source Application -eventID 42002 -entrytype Warning -message $msg
  }
}

Up next: Automating it all

In a future posting, i'll write how to automate this process. If you're impatient, google how to schedule Exchange 2007/2010 PowerShell scripts. It's not hard but it's tricky.

Sunset-sensitive garden lights with awk, bash and the intertubez

Update: Added code to illuminate the garden in the morning as well.

As the darkness moves later by the day, it was time to change the garden light schedule. As a reminder, my garden lights are controlled by a relay card, which is controlled by a Nanode (an Arduino with an Ethernet) which is controlled by a small Linux plug computer. The Linux box runs cron, which sends http requests over curl to the Nanode, which in its turn switches the lights on or off.

Now the easy way to change the schedule would be to edit the crontab. But that would be uncool, since darkness falls at a different time each day. The nifty way would be to have the lights on only between sunset and sleepy-time. So that's how i wanted it.

Step One: when does the sun set.

There are a few Python scripts out there which return the times of sunrise and sunset. As my tiny Linux box didn't have the proper libraries and it's running an ancient Linux (so it can't be easily updated using apt), i couldn't use that. And after all, why duplicate the effort if the effort's been made elsewhere? And when there's nothing to learn in the process?

The answer can be found at Earthtools, using an URL not quite unlike this:

http://www.earthtools.org/sun/$LAT/$LONG/$DAY/$MON/$TZ/$ISDST

To make this work, you'll need enter the values for LATitude, LONGitude, DAY, MONth, TimeZone and whether it IS DST. 

So we need to add a Step Zero, or actually a multiude of them:

  • Where are we?
  • What date is it?
  • Is it Daylight Savings Time?

The last two questions can be answered with some bash scripting, the date command and a line of awk. Note that as i am quite the awk and bash neophyte, things are probably a bit less elegant here as they would be from a seasoned shell scripter.

MON=$(date +%m)
DAY=$(date +%d)
TZ=$(date +%:::z)

DST=$(date +%Z | grep S)
ISDST=0
if [ $DST ]; then
  ISDST=1
fi

This script fails miserably if the character S appears in your time zone, so you'll have to localize a bit before deploying, alright?

To make an educated guess as to where we are, we can either consult an on line mapping service, your friendly GPS or a web service. The web service seed like the most lazy option, and is to be found here:

http://www.geobytes.com/IpLocator.htm?GetLocation&template=php3.txt

This will return a HTML file with some funky headers containing your assumed whereabouts. It probably won't be exact enough to deploy missiles to, but good enough for a sunset value.

Now to the parsing bits, which is where the awk knowledge of the Internets way outweighs mine.

First the location. The file name of the location output above is in the variable LOCFN.

LAT=$( awk -F'"' '/name="latitude"/ {print $4}' $LOCFN)
LONG=$( awk -F'"' '/name="longitude"/ {print $4}' $LOCFN)

Then the stellar bits.

sunrise=$(awk -F'[<|>]' '/sunrise/ {print $3}' $SUNFN)
sunset=$(awk -F'[<|>]' '/sunset/ {print $3}' $SUNFN)

ssunrise=$(date -d $sunrise +%s)
ssunset=$(date -d $sunset +%s)

smorning=$(date -d 06:30 +%s)
snight=$(date -d 22:30 +%s)

The first two lines extract the sunrise and sunset from the file, whose name is stored in the variable SUNFN. The next two lines parse that value into seconds since The Epoch, which makes calculations doable. The last two linea are to allow calculation when the garden lights will illuminate in the morning or go out in the evening (this bit was fixed from yesterday's edition).

Like this. No special calculations are done for weekends, which is kind of lazy, but we take care of that later.

if [[ ( ( "$ssunset" -lt "$snow" ) && ( "$snow" -lt "$snight" ) )    || ( ( "$smorning" -lt "$snow" ) && ( "$snow" -lt "$ssunrise" ) ) ]] ;
then
        /usr/bin/curl http://relaybox/[1-4]/HIGH
else
        /usr/bin/curl http://relaybox/[1-4]/LOW
fi

My Nanode-connected relay box lives at the address relaybox. Or that's what i'd like you to think when you try to hack my LAN. The relays one through four are set to high if the window of darkness is upon my garden (wooh), otherwise they're drawn low.

Finally, and this is the bit i'm not entirely proud of, an edit of the crontab:

*/2 6-10 * * 1-5 /path/to/sunset/script
*/2 16-22 * * 1-5 /path/to/sunset/script

This will hit the script every two minutes in the morning and evening. If the sun rises after ten-oh-oh, the lights will stay on until sixteen-hundred. It's a bug, not a feature.

Soure available upon request.

Android phones practically illegal for pre-teens?

I came across the Google Account Age Requirements page (which i had no idea even existed). Google says one needs to be over 13 to create a Google account (14 in Spain and South Korea, 16 in the Netherlands). I haven't checked whether these age requirements hold for Google app for domains accounts, but if they do like i suppose they do, we're in trouble. Or more so, i am.

My not-yet-teenage daughter (but surely acting like one at times) has an Android phone and when her brother will get a phone, it'll probably be an Android too. But ack. As a Google account is practically required for an Android phone, how can somebody pre-teen use an Android phone?

I also haven't checked if there are age reqs for .mac and Live! accounts (and if they're vital for iPhone or Windows phone use) but i wouldn't be surprised if they have similar demands.

What options are there for communication and sharing stuff in a secure, child-compatible way, without using a Google account? Set up your own mail server and Diaspora pod? Last time i looked, there wasn't even a Diaspora client for Android, and how the heck am i supposed to get anyone else on Diaspora when i can't even get them on Google+?

http://support.google.com/plus/bin/answer.py?hl=en&answer=1350409

Sharing files from a Windows 7 box to an XP

Microsoft has created a rather nifty option to share resources in a server-less environment (eg a "home" or just a small office), called HomeGroup sharing. It works in Windows 7 and somehow magically builds on IPv6.

But HomeGroup sharing isn't available on XP, so you'll have to take the traditional road of right-clicking the folder you want to share, choosing Advanced sharing and setting the security bits just right (eg Share security: write for Everybody, full control for Administrators; File security: as tight as you need).

But sometimes that's not enough.

At a client with a mixed machine environment, suddenly they couldn't get to a file share residing on a Windows 7 box when coming from an XP box. I got system error 58 (or was is system error 85?). Windows croaked. I tried firewall settings, tried user rights, tried whether the fact that they'd connected to another file share on another XP box.

While i did a whole bunch of seemingly arbitrary fixing moves, after doing the following, things started happening again.

  • Start the local security policy editor, secpol.msc
  • Go to Local Policies > Security Options
  • Scroll down to Network Security: LAN Manager authentication level
  • Change the setting from "Send NTLMv2 response only" to "Send LM & NTLM - use NTLMv2 session security if negotiated"

The reasoning behind this is that Windows 7 and Vista works on a newer and more secure variant of the Lan Manager protocol than XP. You might get similar problem if connecting from a Linux box or a NAS (which very well may be a Linux box) to a Windows 7. Changing this setting starts the discussion using the older version and switches to the more secure one if the two endpoints can agree on that.

Took me several hours to get this right. I hope this will help you make a swifter fix.

Blinking lights on the LAN

The sorry state of my garden lights has taken one pathetic step to the brighter. I can now blink my garden lights over the local network, using a serverized version of the Lua script i wrote earlier to do the same over the command line. Okay, so it's one step better than ssh'ing into the box and entering relay commands over Bash :)

I first thought of learning some more Lua to make the script socket-aware. But then it hit me: Linux has all this built in, into a "super server" called inetd, which i've never actually used before but kinda know of due to its history of security issues. Still, i feel like such a late bloomer for never having used inetd before.

Here's how, if you're one of the inetd virgins. Edit /etc/services and add the name, port and protocol of my service. In my case, i called it relay 11647/tcp. Call update-inetd --add ...and a bunch of arguments separated by the \t character, as one wonky string. Yeah, it's hairy.

I needed to make just one change to the Lua script. Since the script won't read command line arguments anymore -- and i don't know if it's by design or my deficiency -- i had to add if rel == Nil then rel, act = io.read( "*number", "*number" ) end into the code.

Still, now i can say echo 4 1 | nc gardengnome 11647 which is a sure sign about a turn for the better :)

Did this make any sense at all to you?

Ping watchdog with Powershell

We have one unreliable virtual server which sometimes just loses its network connection. Until we have a permanent solution, the temporary fix is to have the computer reboot if it loses network connection.

Note: There is nothing as permanent as a temporary solution. With that warning, let's get to it.

Step 1 - Produce the following Powershell script into c:\bin\pingdog.ps1

$netup = new-object Test-Connection -quiet "goo.gl"
if( $netup -eq $False ) {
  Restart-Computer
}

Step 2 - Fix your security settings

Start Powershell. Enter Set-ExecutionPolicy RemoteSigned or whatever level you're comfortable with that'll still run your script.

Step 3 - Schedule it

If you're using Windows Server 2003 (like we, ungh) schedule a daily task to run %SystemRoot%\system32\WindowsPowerShell\v1.0\powershell.exe c:\bin\pingdog.ps1

From the task's Properties, the Schedule tab, press the Advanced button. Tick Repeat Task, every 12 minutes (or whatever you fancy), and repeat for 24 hours.

If you're using Windows Server 2008 or later, you should be able to set the task to run evey x minutes right from the interface.

Step 4 - Check your logs

Keep an eye on the failing computer's system log to see if it's restarted when you looked away.

It blinks!

A while ago, i bought an Ethernet module from Sure electronics. I wasn't sure what to use it for or if i indeed would be able to use it for anything, but it was cheap and hey, you could connect it to the Ethernet. Can't be all bad :)

Having less than a raving success with controlling my garden lights with a Nanode, i decided i needed a break. And so i googled "Arduino Sure-electronics ethernet", and lo && behold(), up comes a blog post with just what i needed, but just didn't understand to expect: that it's really easy to interface an Arduino with the Ethernet, and that the magic is built on the EtherShield library which i was already using on Nanode.

So i soldered the seven pins needed to have the two devices talk with each other. Wired them together with neat and correctly coloured jumper wires. Copied the code into my editor. Changed the MAC and IP addresses. Verified the code ("It compiles! Ship it!"). Uploaded it to the Arduino board. And it - just - worked!

You can't imagine how excited i was when i was able to open up a web page, served from the Arduino, and was able to toggle a LED on and off over my LAN. I't just so neat! (or if you do know me, you can probably imagine just how excited i was :) I should have taken a picture, or a video!

So my thanks go out to Henrik Denhart who wrote the blog post and to Andrew Lindsay, coder and maintainer of the EtherShield library, and indeed the whole Arduino maker movement who with their enthusiasm and drive for openness make the platform such a wonderful thing it is!

Toys and frustrations

We live in such wondrous times when it comes to toys! I'm not only talking about prefabricated toys like touch sensitive cardboard electric guitars, affordable remote controllable helicopters that avoid walls and are meant for inside-the-house use or not-really-that-intelligent robots. I'm talking about web servers no larger than a deck of cards -- both wired and wireless (and the wireless one is more like the size of a credit card) -- stamp size microcontrollers that can speak with servos, LCD displays, lasers, motion sensors and RFID readers. These are the kind of toys that really drive me wild.

As these kind of wonderful microcontrollers with their peripherials have become both available and affordable, i've bought some of 'em to tinker with. My latest purchase is the Nanode, which is an Arduino clone with wired Ethernet built in. It's cheap, made by hackers (=enthusiasts, makers) and was delivered by mail as a bag of discrete components and a little slip of paper with an URL to the web page which has instructions on how to build the little bugger.

So i soldered. I bought magnifying glasses (yes, plus-glasses --- i'm now officially getting old) and that soldering station i've been drooling about for so long. And it was wonderful! I assume the first builders of the first Altair and Apple computers must have felt a little like this (though i freely admit that they were truly hard core, whereas i was trodding tested ground). And you can't believe my exhilaration when i managed to flash the Nanode with a DHCP test program and got the signs of life over serial that the device was really working and that it had received an IP address from my router. That, my friends, is confirmation of existence!

It hasn't been all roses of course. My current sorrow is that the Nanode i have doesn't want to co-operate with the FTDI breakout board i got from eBay.

And there are plenty of challenges in the past. I bought a Web server module from Dangerous Prototypes (love the name!). The first one had a bug in the hardware and i felt fairly stupid for quite a while since i really had no way of knowing if it was me being incapable -- a valid guess -- or the device being faulty. Ian, the very nice dude behind DP helped me get the device fixed and returned at no extra cost. I've bought a small wireless web server prototype from Sure electronics, but the only thing you can do with it is flash a LED and check its ambient temperature. They're not very explicit about it, but it seems like you should be able to reflash the hardware yourself, if i could only get the source code from somewhere. And i got the Texas Instruments wrist-mountable computer Chronos because it was on sale, but i haven't got around to coding it yet.

But hey, i'll get something working because boy, these toys are fun!