// This is the script to give summary on the main page.
Think IPM

Thursday, July 13, 2017

vSphere Enterprise licensing Plus Throw Back Thursday [2017 Update]!

Image result for licensingSo the last time I posted about vSphere Enterprise licensing was in 2010!  At the time, Enterprise licenses were not available for net new purchasing.  You could however upgrade your existing Enterprise licenses to vSphere 6 enterprise licenses.   Sever years later, still the same! Smile   Oh and also the same, Client confusion. Smile   So today I was asked about it and did a little research to refresh my own memory and came across an updated KB article (from 2016) here:

End of availability of vSphere Enterprise, vSphere with Operations Manager (vSOM) Standard and Enterprise edition
https://kb.vmware.com/kb/2143987

 

This confirms (again?) the end of availability of Enterprise for new vSphere license purchases. Oh! and a fancy new chart as well.

Unfortunately the 50% upgrade discount just expired a few weeks ago (sorry!).

So what do you get with new Enterprise Plus licensing?

DRS, DPM, Storage DRS, Storage I/O Control, Network I/O Control, SR-IOV Support, NVIDIA GRID vGPU support, Proactive HA, Distributed Switches, Host Profiles and Auto Deploy.  Go cheap and get vSphere Advanced and you don’t get any of them. Disappointed smile

 

So that’s the short and long of it. 

Click Here to Continue Reading >>

Tuesday, July 11, 2017

My Smart Home – a LOOK at the parts that make up the sum.

imageI’ve put up a few posts about certain specific tech in the house [Check it out here] and some people have asked that I do a more complete write up of my actual Home Assistant Smart Home system.  Lots of information can be found on my Github Repo but this will be an attempt or at least a starting point to document most of the high level systems in the house.   From here, I’ll be able to write additional pieces that dive more in depth for particular systems.   If this is not your thing, look away NOW.  You’ve been warned.

The basic foundation for the system is a Raspberry Pi running Home Assistant.  Home Assistant is an open Source piece of software that in a lot of ways reminds me of VMware.   It’s an abstraction of hardware and software.  It allows me to buy and deploy pretty much whatever I want in my house in terms of hardware and provides the software layer/glue/translation for it all to talk to each other.  Smart House as a Service basically. Smile  My Raspberry Pi is a standard 16GB SD card version with the All In One install on it.  Other hardware attached to it is an Aeon Labs Z Wave Stick and a 433Mhz Transmitter.  I am also running Dasher (to provide access to my Amazon Dash buttons) and HomeBridge (to provide Apple HomeKit support) on the Pi itself.

 

Lights:

Lights are usually the first place you start with Home Automation.  Turning lights on and off automatically makes a ton of sense in the home setting.  For my lights, I first started with HUE Lights.  The hubs are standard Zigbee light controllers.  I have a mixture of both GE Lights and Hue proper lights connected to them.  I have about 40 or so lights and have 2 Hue hubs deployed in my house.  These 2 hubs control the majority of the lights for the interior of my house.  I also have a Wink Hub deployed that has a few outside lights connected to it via ZWAVE.  Much of the automations built in the house revolve around lights turning on and off through out the day.  Lights turn on at Sunset, when we turn the TV on, walk into rooms or turn off when we get into bed.   There is a pretty cool automation that keeps a consistent light level throughout the day by dimming and adjusting lights as they turn on based on the time of day.   No one like a 100% bright bulb at 2am.  I never appreciated how accent lighting and light levels can affect the overall house mood and look.  I’m a huge fan of it now.

imageimage

Outlets:

For the most part, most of my lighting is done by bulbs.  There is a never ending debate about whether you should use switches or bulbs.. I don’t think there is a right answer for everyone and for me, I chose bulbs.  They allow for more granular control of the space and it’s lighting.  That said, I do have a few controllable outlets scattered around the house.  Two main types.  Zwave outlets that control my outdoor landscaping lights and my trusty (cheap) 433Mhz outlets for accent lighting switching around the house.  For the 433Mhz operations, I have a few 6 buck Etekcity outlets that are controlled with the 433Mhz Transmitter attached to the Pi.  This worked out to be the most economical way for me to turn accent rope lighting on and off throughout the house with automations. 
imageimage

HVAC, Smoke Detectors and Irrigation:

For the Heating and AC control in the house, I chose Nest thermostats.  These devices are pretty much set it and forget it.   The thermostats keep the house cool when needed and are aware of our presence to save energy when we are not home. I have built some rules that allow Home Assistant to turn off the HVAC when someone opens a door or window for more than 5 minutes.   Once the door/window is closed, the system resumes it’s normal state.

For Smoke Detectors, I have Nest Protects deployed.  You can read about that deployment and how I came to those decisions here.

Since I live in Florida, irrigation is key to keeping the lawn green and the HOA at bay.  I have put in a Rachio smart irrigation control to control how much and how often the lawn is watered.  It’s another set it and forget it type device.

The three of these devices/systems work nicely together in the event of a fire.  The Protects will sense the fire, turn off the AC (to avoid spreading smoke through the central air) and turn on the sprinklers.  Additionally, HA will be notified and will turn ALL lights on and set our outside lights to flash to draw attention.   If we are not home, remote alerts let us know there is an emergency situation.
imageimage

Security:

For security, I did roll my own.  You can read about it here.  It’s basically NodeMCUs and the pre-existing wires.  Having 17 individual zones all feeding into Home Assistant has given me an abundance of data to build rules around such as the HVAC rule above.

We also have a SkyBell HD doorbell with motion and camera capabilities.  If people ring the bell (or just walk to the door) various lights will blink and turn on to notify us and the person that we are around and aware.   We also have 2 garage doors that are fully automated.  Using Garadget components, we get notifications when they open or close.  They also act as sensors feeding information into HA for rule processing.  A great example is at sunset, we are notified via our Text to Speech system that one or both of the doors are open.  Another neat example of how Home Assistant bridges the gap is when our Garadget Doors open at night, the outdoor Hue lights shine white for us to see.  Two separate systems working together for one experience.

image 

Whole House Speech:

Giving the home the ability to talk to us has really changed the experience with the Smart Home.  For the longest, we have been able to use speech to control the automations with Amazon Echoes (really just to override whatever automations the house was currently implementing) but with the addition of Amazon’s Polly TTS voice, I am able to broadcast all sorts of information over the whole house surround speakers.  Example use cases for this are when the Nest Thermostats turn on or off, reminders to close the windows or doors, announcement messages when we come home giving us a status of the lights, windows and doors.   Reactions to sensors also have voice outputs such as broadcasting a dog barking when there is motion in front of the house (picked up by the SkyBell).  We are also able to stream internet radio over all the speakers throughout the house.  The speakers themselves are normal ceiling mounted speakers but they are connected to ChromeCast Audios and a cheap car AMP.

Presence Detection:

Presence detection in the house is key to a LOT of the successful automations we have.  If the house knows when we are home or not, so many more things can happen without us intervening.  For presence detection, we use a simple NMAP component that just pings the network and takes note of when certain devices are online or off.  For our home, tracking the Wi-Fi status of mine and my wife’s iPhones provides a pretty accurate sensor for when we are home or not.  We also have a SleepIQ bed that has pressure sensors to know when we are in bed or not so that the nighttime routines are fully automated.  When the house knows we are both in bed, all the lights begin shutting down and entering the nighttime scenes. Likewise when we wake up. All Pretty seamlessly.

image

Final thoughts:

The house will never be finished.  The project will never end.  As new technology and newer capabilities become available I’ll work to implement them in my Smart home.  When things don’t work correctly, my family notices which means to me that these types of automations and conveniences are becoming part of the normal household routine.

Click Here to Continue Reading >>

Monday, July 3, 2017

Visualizing the Smart Home. Using Home Assistant, Fire Tablets and FloorPlan.

clip_image001

I’ve been running Home Assistant for my home automation for almost 3 years now.  The system has grow quite large and the User Interface was becoming pretty unwieldy and cumbersome.  For the longest time, I was able to shift most of the interface to voice using Amazon Alexa supported by an excellent HA component called emulated_hue.  This allowed us to interact with almost all of the HA objects using Alexa and the native Hue support.  “Alexa, Open the garage doors”, worked beautifully!  My philosophy when it comes to automating the house has always been to try to have the house anticipate what was needed without us even needing an interface.  The interfaces (voice or screen) should just be needed for occasional overrides.

The most recent security system I rolled has kind of changed a lot of that though.  Since all the windows and doors are now represented in the HA system, my lack of attention to the actual UI was now severely affecting the User Experience.   Having to scroll pages and pages of text items to find out if a window was opened or closed was not going to cut it anymore.  Voice helped (leave a window open for more than 5 minutes and Joanna comes in over the whole house surround system and let’s us know the situation and then also turns off the HVAC for us).

 

image

All the info is there but it is just not a great user experience.  There had to be a better way!

Timing is everything and Petar Kozul had just released a great Home Assistant extension called FloorPlan.  Floorplan allowed Home Assistant users to create visual maps of their sensors and HA objects.  Named Floorplan but it could do so much more.  It’s just a framework to put up SVG images and then tie HA object IDs to them and on the fly change the images and data via CSS.  It was perfect.   And it resulted in THIS:
image

This one screen gave me all the essential information I needed to see.  Date, time, weather (inside and outside temps), lights, switches and a few commonly used buttons for easy access.  And at a glance, it showed me all the windows and doors and Nest Protects in the house.  If they were RED, I would know they were left open or in the case of the Protects, OFFLINE.  It’s perfect!

 

My friend Steven helped me out with the visuals and the overall graphic design but thanks to the HA and Floorplan frameworks, the whole system will stay customizable as the needs of the house change.

Of course with the old alarm system now replaced, I had those ugly panels still on the wall (with the boring keypads).  Time to replace them with $50 Fire OS Tablets.  These 7 inch beauties can be wall mounted and are the perfect replacement for the ugly keypad artifacts.

I ended up using another great Open Sourced software called Wall Panel.  This allowed Fullscreen, MQTT support and a few other handy goodies to make this an awesome solution.

image 

Hardware parts List:

1 x FireOS tablet

1 x Wall Mount clips

1 x Recessed Outlet box

1x Magnetic tipped Micro USB cables
This allows us to just grab the tablet when we want to surf or do something without worrying about yanking the cords out.

 

Other software I used were Magic Plan to create the actual floor plan of the house, Inkscape for SVG editing and Atom for all text edits. 

 

Like all my Home Automation Projects, you can visit my repo for all the additional code and details.   Be sure to Star the repo if you want updates.

 

 My Github Repo

 

Happy Building

-Carlo

Click Here to Continue Reading >>

Tuesday, June 20, 2017

PSA: Check out your smoke detectors (once every 10 years)

I have owned my house for just about a year now and I have been automating and ‘smartening’ it up like crazy.  My home automation platform of choice is Home Assistant.  If you haven’t checked it out, it’s an Open source platform that runs on a raspberry pi and becomes the glue or in VMware terms, the abstract layer between all the various hardware.   Yes.  I have a software defined house.

So last month, I got to thinking about my smoke detectors.  I have quite a few of them in the house and honestly didn’t know much about them.  I knew 2 of them were connected to my alarm system and the rest were hardwired into power.  There was also an interconnect wire between them all so that if one goes off, they all go off.  I learned this when one of them went into a low battery state and triggered them all to go off. at night. on a week day.

In addition to the ‘dumb’ status they held, I also learned after some googling that ALL smoke and carbon monoxide detectors go bad after 10 years.  They have a chemical pad in them that ‘smells’ the smoke or CO and then triggers the alert.  This chemical has a 10 year effective lifespan.  My current detectors were 14 years old (they have a manufacturer date stamped on them) and basically didn’t do ANYTHING except eat batteries and then chirp for more.  The house could be on fire and they wouldn’t respond unless a battery happened to fall out of them.

Image result for nest protect

Of course, they needed to be replaced and I went with the Nest Protects.  They are not the cheapest guys on the block but I think they are the smartest and were the easiest to integrate into my existing smart home.  By code, I needed 9 of them for the house.  1 in each room, 1 in all bedroom hallways and 1 in the kitchen.  The ones I purchased are hardwired into power and use a wireless interconnect to spread the word of danger.  You can read up on all the cool nest features elsewhere so I wanted to just share the couple of small additions I made to my set up.

Home Assistant already had a Nest component so once I set them up on the network and added them to my Nest account, all the various protect sensors just showed up in the interface.   My Nest package can be found in my Github repo here.

Out of the box, if the Nest Protects sense an emergency condition, they will of course alert you but additionally, they will

  • turn off your HVAC (assuming controlled by a Nest Thermostat)
  • turn on your sprinkler system (if controlled by a Rachio system)

Thanks to Home Assistant, mine will also trigger my emergency script.  The script sequence will

  • switch all outside lights to red to indicate an emergency
  • flash all lights in the house 4 times to grab everyone’s attention (although the piercing siren from the detectors should do that as well)
  • turn on all interior lights to 100% brightness in the house.
  • switch my front LED strips to a white flashing strobe
  • open both garage doors if we are home. (via Garadget) [Not implemented yet]

It’s been about a month with the Protects and no false alarms so I’m pretty happy with that.  The guide light feature is a welcome bonus as well.

Click Here to Continue Reading >>

Friday, June 16, 2017

Building my Home alarm system (Hardware Phase)

Image result for burglar 

I have a house.  It has prewired windows and doors that I want to use for my own purposes.  I DON’T have an alarm company.  I refuse to pay some monthly fee for monitoring.  I just don’t feel as though I have gotten the value that I expected in the past from the security companies.  Basically I’ll just pay myself and become my own 24/7/365 monitoring solution. Winking smile

 

So after a ton of Googl-fu and trying to decide what type of system I should use to replace ADT with, I settled in on using the awesome ESP8266 chips in the NodeMCU form factor.   These tiny (and cheap) little devices could be wired up to all my existing wired doors and windows and then programmed to interface with my Home Assistant home automation platform.  They are about 10 bucks and have built in Wi-Fi and a good amount of support out there on the internet for DIY projects.  They really are awesome once you start working with them.

Here is the part list for my project : 

Let’s take a look at where I started :

imageIt was your standard alarm box.  All wires from the windows and doors fed back through to the walls to this 1980’s circuit board.   They were all simple reed switches (red and black) with magnets at the windows and doors.  When the window/door is closed, a magnet keeps the reed switch closed and the circuit is complete.  When the window/door opened, the magnet is moved away, switch is opened and the circuit is broken.  Easy for 1980’s tech to understand, easy for me to understand and easy for me to get the ESPs to understand.

First thing first, rip out all the wires from the motherboard(carefully). Smile 

Hopefully they are all labeled.  Mine were not so I basically just peeled them off in pairs and used a multimeter to test them.  Set it to tone and then walk around opening and closing windows until the tone goes away. (side note: I found 6 windows that had broken reed switches that were permanently closed through a process of elimination).  Once you identify all of your windows and doors, LABEL them!

ScreenClipAfter I had most of the physical wires labeled, I started with the NodeMCUs.  There are a ton of great resources around to explain how to program the NodeMCUs so I’ll let you find your own way through that but will give you the high level points :

I had 22 windows/doors but bundled a few together for a total of 17 zones.  I purchased 3 NodeMCUs from amazon and figured I would could do 7 zones on each one of them.

I used ESPEASY software to flash and configure the NodeMCUs.  You can see what GPIO pins I used in the diagram to the right.  Green were good, but the red Xs gave me issues.  This may have to do with my novice understanding of the whole ESP8266 architecture and microelectronics in general.

I used MQTT to talk back to my Home Assistant.  If you don’t know about MQTT, it is a machine to machine messaging protocol.  Super lightweight and perfect for these types of communications.

The way ESPEASY works in my set up is that every time a reed sensor is tripped (circuit broken), it will update MQTT.  Home Assistant is configured to watch those MQTT topics and act accordingly when it detects a change. (window opening or closing).  Based on that information, HA will do things.

ESPEASY uses a very nice web interface to do all of the configuration.  It was pretty easy to use once the flashing was completed.

 

image

Once the NodeMCUs were flashed and configured, I started wiring them up. 

image image

image image

Since I don’t like soldering (I’m not good at it), I decided to go with breadboards and wire jumper connections.  It made it super easy to connect everything in.  Plus if one of my ESPs goes bad, I can just pop it out and replace with a new one.  The wiring was pretty easy.   All the grounds (black) would be bundled together and then put to a GND on the NodeMCU, and each of the reds would go to a GPIO pin on the boards.

With everything wired up and tested, I removed the old motherboard and carefully put everything back into my alarm case.

image

image

 

The end result in Home Assistant was this:

image

It’s not the prettiest interface (I’m working on fixing that anyway) but it is WAAY more flexible than anything else I could have gotten.  I have 17 zones that I can now use for input for various things in my Smart Home.

The Home Assistant code for all of this is located here :
https://github.com/CCOSTAN/Home-AssistantConfig/blob/master/packages/alarm.yaml

One of the first automations I put in place was to turn off the HVAC systems if any of the openings were open for more than 5 minutes.  Then make a short announcement and wait to turn the HVAC back on when the window/door was closed.   I love it!

automation:
  - alias: 'Turn off HVAC in window/door is opened'
    trigger:
      - platform: state
        entity_id:
          - binary_sensor.MCU1_GPIO4
          - binary_sensor.MCU1_GPIO5
          - binary_sensor.MCU1_GPIO10
          - binary_sensor.MCU1_GPIO12
          - binary_sensor.MCU1_GPIO13
          - binary_sensor.MCU1_GPIO14
          - binary_sensor.MCU2_GPIO4
          - binary_sensor.MCU2_GPIO5
          - binary_sensor.MCU2_GPIO9
          - binary_sensor.MCU2_GPIO10
          - binary_sensor.MCU2_GPIO12
          - binary_sensor.MCU2_GPIO13
          - binary_sensor.MCU2_GPIO14
          - binary_sensor.MCU3_GPIO4
          - binary_sensor.MCU3_GPIO5
          - binary_sensor.MCU3_GPIO10
          - binary_sensor.MCU3_GPIO14
        state: 'on'
        from: 'off'
        for:
          minutes: 5
    action:
      - service: climate.set_operation_mode
        data:
          entity_id: climate.downstairs
          operation_mode: 'off'
      - service: script.speech_engine
        data_template:
          value1: "The {{ trigger.to_state.attributes.friendly_name }} has been opened for about 5 minutes.  I will shut down the Air Conditioner so you can enjoy the fresh air."
          call_outside_weather: 1
          call_inside_weather: 1

 

The home assistant code is all based on YAML and is pretty easy to learn and write for.  At some point in the future, I’ll have to do a quick write up on the main Home Assistant system running on my raspberry Pi.

So that’s about it; I ripped out my old alarm system, replaced it with 3 ESP8266 NodeMCU chips, stuffed it back into the old case, integrated it with Home Assistant and then started writing automations using my new sensors.

I built all of this a while ago and I have been running with it without issue.  In fact, it works incredibly well. 

Since then, I have also added in a light sensor to know if someone opens the panel case.

image

 

There is a ton of potential with this system and I plan to keep building it out but for now, this should get you started.  I feel like this post might have been all over the place so if there are any BIG items I may have missed or glossed over, feel free to hit me up on twitter.

Click Here to Continue Reading >>

Tuesday, June 13, 2017

Those are some BIG log files you have there buddy!

Sometimes a well run, stable production environment that runs and runs and runs is hiding some hidden dangers from you.  My client was recently doing some housekeeping on their shared storage and came across some larger than normal Virtual Machine log files.

ScreenClip

By default, these log files normally will get rotated out when a VM is powered off/on or vMotion’d.  For machines that do neither and just run happily in production, you could have instances where the log files could grow much more than you expect.  By default, VMware will keep 6 rotations of log files but each log file is enabled for unlimited growth (or until the LUN is consumed!).   For machines that have lots of login/logoff activities (think Terminal services / Citrix machines), you might want to consider reviewing the current logging options.

 

As a refresher, here is the KB article to brush up on your logging options.  Image result for lumberjack

https://kb.vmware.com/kb/8182749

You can safely delete the older vmware-x.log files without issue but sometimes will run into issues deleting the current active log file.  The VM will need to be powered off or vMotion’d to access that log file.   It’s also worth mentioning that you should take a moment or two to investigate the contents to make sure the log spew is normal and you aren’t missing something that should/could be addressed.

For machines that are generating large log files with benign messaging, you might want to consider turning off logging for those VMs.

To turn logging to off, enter logging=false in the virtual machines .vmx file

Happy Logging!

-Carlo

Click Here to Continue Reading >>

Thursday, June 8, 2017

Home Protection from Power outages (Sort of) with the BGE70.

So June is the official start of Hurricane season down here in Florida.   This is basically when it rains every day like clockwork from 4pm to 5:30.  Unfortunately with the rain usually comes heavy lightning and more recently (in my new neighborhood), power outages.  They last just a few seconds but due to my heavy investment in IOT and home automation, create absolute havoc for my house.  Basically when there was a power blip in the house, all the smart lights would immediately turn on to full brightness when the power was restored and all devices would start rebooting.  Since the Cable Modem takes the longest to reboot, most of the hubs would be back up before it and complain about not having internet access and end up in some weird state where they would need one more reboot.  My quick solution for the time being is a $35 buck UPS (APC Back-UPS Connect UPS Battery Backup).  Pretty standard stuff I guess but I found this niche one that is specifically created for home networking gear.  It doesn’t support three prong devices and is created to specifically protect your cable modem and other internet infrastructure that typically has two prongs.  Apparently this allows a smaller battery and much longer runtimes off it in contrast to it’s traditional three prong siblings.

 
Click the Picture for the link to the device.

For me and my situation, I decided to just connect the Cable Modem, a Network switch and my Home Assistant Raspberry Pi to it.   This will keep the internet connection alive in the house (assuming upstream cable company equipment isn’t affected) and my home automation software running.   Everything else (hubs, switches, lights, controllers) can reboot and come back on their own.  Since the Home Automation server is also protected, I should be able to write some logic on it to detect when these power events happen and adjust the house accordingly. 
(Middle of the night == Turn all the lights back off; Evening with us home == Turn off MOST of them)

Eventually I will get around to writing about my home automation software (powered by Home Assistant) but for now, if you want to know more about it, check out my Github repo and associated descriptions within it.

P.S. The other great thing about this UPS is it only beeps ONCE when the power goes off.  It doesn’t continually whine about being on battery power like other Battery Backup devices I have owned in the past.

 

- Carlo

Click Here to Continue Reading >>

Wednesday, June 7, 2017

Upgrading your Data Domain OS

Today I ran into a replication issue between 2 Data Domain that showed up as VERY slow transfer rates between sites.  After working with EMC support, it was advised that I should upgrade the OS versions of the boxes to 5.7.4.0-XXXXXX.

The process was time consuming but pretty straight forward.

1) Upgrade Destination DD first.

2) Make sure your boxes are no longer doing anything.  I disabled the replication pairs and paused any backup jobs or cleaning tasks.

3) Upload the new RPM to the box via the GUI.

4) Run the upgrade via the command line.

5) Repeat on Source DD server.

 

ScreenClip

You can upload the RPM via the GUI but do the upgrade via Command line.

ScreenClip

During the upgrade, the Box will reboot and the Putty connection will become unavailable for about 15 minutes.   Once your PING returns, you can log back into the DD Box via Putty and run
system upgrade status – Verify that the Box shows Completed.

Current Upgrade Status: DD OS upgrade In Progress
      Node 0: phase 4/4 (Finalize 100%)

Current Upgrade Status: DD OS upgrade Succeeded

uname  - Verify the Current OS has been upgraded.

 

- Easy Peasy.

 

Be sure to  remove the RPMs via the GUI after successfully updating the Data Domains to reclaim the space.  -Thanks for the tip Eric!

Click Here to Continue Reading >>

Monday, June 5, 2017

Video: #Syn706 – Building a XenApp real-time session monitoring dashboard

You’ve read about it in the recap and now you can watch the video!  Sam Jacobs’ full session is now available on demand on SynergyTV.

image

The presentation is now available on demand:

http://live.citrixsynergy.com/2017/player/ondemandplayer.php?presentation_id=86fbc5c30e5b490b894bca2c0772ef051d

Users need to register (no charge) to see SynergyTV content.

Click Here to Continue Reading >>

Thursday, June 1, 2017

Total Uninstall -- A paean to the most indispensable IT tool I know!

Here is a great product write-up by my colleague Jacques Bensimon.


It is said that LBJ, who famously favored a powerful walk-in shower that came at him from all angles, once commented to an aide (apparently from the shower) something along the lines of "If you ain't got one of these, your @ss ain't clean!". Historically true or not, LBJ's comment perfectly captures my own feelings about good installation tracking software: if you ain't got one of these, ...

 

It can easily be argued that the rampant overuse of application virtualization and of "siloing" (the segregation of applications to their own sets of servers) in many TS/RDS/XenApp environments (and even traditional workstation environments or VDI) is the direct result of administrators installing applications and system components "blindly", with not a clue as to what any given installation added, deleted or modified in the target machine's file system and Registry. Not only does this require a constant state of regression testing ("What existing apps might the last installation have broken?"), but any discovered issue that doesn't have an obvious explanation and solution quickly leads to the new app being summarily declared "incompatible" and either virtualized (when possible), siloed, or in some cases even abandoned. The net result is typically the proliferation of different images, each requiring ongoing maintenance (app updates, system patches, etc.), a poor use of resources (more hardware or VMs to support the different silos, some probably underutilized), lack of application interoperability, and user confusion. App virtualization, which has its place, is far from being a universal solution given its impact on performance, resource utilization, and application visibility/interoperability (when it can be used at all).

 

These issues largely go away if, with proper installation tracking and the immediate remediation which it allows when conflicts are detected (by the installing engineer rather than by users somewhere down the line), a reduced set of images (often a single one) incorporating the bulk of an organization’s or department’s applications can be created. Before your head spontaneously combusts at the thought, let me point out (as I often have to when explaining this strategy to clients) that (a) an installed application that is not currently running takes up no resources (aside from the disk space it occupies and maybe a microscopically larger Registry footprint in memory), (b) the fact that an application is installed does not necessarily mean it is accessible to all users of the image – that’s what ACLs and/or app policies are for, and (c) a single image does not preclude some of its applications from only being accessible from certain silos, should the need arise for whatever reasons (legal, political, etc.) – the same image being employed in multiple silos (with proper control of which apps are available in each one) still solves the ongoing maintenance issue.

So, what is “installation tracking software”?

At the start of my IT consulting career, I got into the habit of using PC Magazine’s InCtrl5 utility whenever I installed anything (or needed to know where an app stored some of its settings) – I maintained a Ziff-Davis subscription for years just in order to have legitimate access to this tool. While slow and flawed in many other ways, InCtrl5 was a lot better than nothing: it operated by taking a system “snapshot” (basically a directory of file system contents and a copy of the Registry) before a tracked action (installation, reconfiguration, …), then another such snapshot once the action was completed, and finally compared the two snapshots to create a fairly primitive report showing file & Registry items that were new, modified or deleted. The reports could be saved as individual files (text or HTML) but not much else could be done with them other than reading them and maybe doing a global search on their contents using other tools. The snapshots were not retained in any useful form, so they couldn’t be chained for re-use – every tracking task thus required taking fresh pre- and post-action snapshots. But it did the job.

 

And then came 64-bit! As a 32-bit app, InCtrl5, even when it could be coaxed to run on a new platform via compatibility settings, immediately lost visibility of (charitably) half the contents of the 64-bit systems I was working on, and so began a search for a replacement. History doesn’t record the numerous alternatives I looked at and tested, some few freeware – generally useless – and most to some extent or other commercial – somewhat better but still lacking – before finally coming across Gavrila Martau’s Total Uninstall (Oh happy day!):

clip_image002

Originally conceived as an intelligent uninstaller that can ferret out and remove all the pieces of any installed application (the “Installed Programs” module), which I’m sure it does competently given Mr. Martau’s evident talents and understanding of Windows applications, Total Uninstall (or TU) was the solution I sought (and much more) thanks to its “Monitored Programs” module, which is really the only TU aspect I’m discussing here (other included modules are described on the program’s web site). Like InCtrl5, TU compares pre- and post-action snapshots to generate its reports, but the similarity to InCtrl5 stops there. With apologies to the program’s author for what I’m sure will be its many omissions, here’s my list of the TU features that have made it not only my (and many of my clients’) installation tracking software of choice, but also practically my application management hub, and a local, live, navigable & searchable repository of the complete history of the Windows images I create or help maintain (automatic documentation anybody?):

· TU is fast when creating & comparing system snapshots, especially after the first snapshot following machine startup, even when the option to capture & compare executable file versions is used (something I recommend for permanent installation reports even if it does somewhat slow down the snapshot process). 

· TU offers the ability to create and use multiple scanning profiles (i.e. what to include in or exclude from system snapshots, drives/folders/file-patterns/versions/alt-streams/keys) and comparison profiles (what differences to consider or ignore when creating reports). Intelligently considered default scanning & comparison profiles are provided (they include areas likely to be modified by setups and ignore typical “system noise”), but they can be modified or supplemented as you see fit. You can essentially apply the “curiosity level” appropriate to a given tracked action.

· TU retains a configurable number of most recent snapshots (up to a configurable amount of disk space) and not only allows the use of the last one as the starting point for new reports (“chaining”) but also offers the ability to create comparison reports involving any two snapshots at any time. In particular, this means you can retroactively capture system changes by comparing the last snapshot to the current system state. This also means you can create a report of the Windows installation itself (huh?) by comparing a “capture-nothing” empty snapshot with the current system state immediately after Windows (& TU) setup.

· TU installation reports are navigable trees displayed within the main program window and are magically useful: they clearly display color-coded additions, changes (even just to attributes) & deletions, can be used to jump to items in Explorer or Regedit, allow for different views (additions/changes/deletions only, missing items since setup, leftover items following uninstall, and custom views), can be edited to remove “noise”, can be used to Undo/Redo Registry items or entire subtrees, can open files in the text or hex viewers of your choice, can be printed (they look great in PDF or XPS formats), etc. etc. Oh, you can also attach a permanent note to any installation report (e.g. setup source, special considerations, whatever).

clip_image004

· TU displays and can, when possible, apply pending moves, renames & deletions without reboot.

clip_image006

· TU reports are searchable, individually or globally, providing an immensely useful ability to understand the full history of any given file system or Registry item, from original creation through all subsequent changes/deletions/recreations, all the way to its current status. Use TU religiously and you’ll never again wonder where the heck something on your image came from.

clip_image008

· Not only can TU export any branch of Registry changes (as “do” or “undo” .reg files), but it can also create a full zipped backup (files & Registry) of any installation (best done after the report has been cleaned of irrelevant items). The backups can later be restored to the same or different machine, if sufficiently similar to the original, and often even if not. (See also the TU Ultimate description below).

· I knew I’d run out of steam before fully capturing all the wondrous capabilities and uses of TU, but I’d be remiss if I didn’t say something about its author: in all my (and my colleagues’ and clients’) years of using TU, I’ve found Mr. Martau to be as capable, knowledgeable, responsive and courteous a developer as there is. (Example: I recently asked about possibly displaying Registry key modification timestamps, and it’s now upcoming). How does he handle bug reports? I don’t know, never found any! Smile

Total Uninstall Ultimate: This relatively recent edition (“TUU”) adds a command line tool that can be used to automate the creation of system snapshots (not a bad idea as a standard part of machine startup or shutdown) and to automate restoring in unattended fashion any previously created application backup. This turns TU into a competent and fully configurable application repackager and deployment tool, especially useful to perform fast just-in-time app installations or updates on provisioned machines during startup when no other silent setup option is available or trustworthy.

Cost: TU & TUU are not free, but they are dirt cheap! While they currently average between ~$25 and $40 for small corporate purchases, depending on edition (there’s also a portable Technician’s Edition you can run from USB stick at your clients), the real deal is in larger TU/TUU site licenses. I won’t presume to guess at their current price, but several clients have in recent times obtained 1000-seat TUU licenses for a couple of bucks per machine. Contact the author for details (and download a free trial from the web site).

There, my love letter to Total Uninstall, finally done after years of false starts – I didn’t think I could possibly do it justice, and I’m quite sure I still haven’t.

JB

Follow Jacques Bensimon on Twitter @JacqBens

Click Here to Continue Reading >>