// This is the script to give summary on the main page.
Think IPM

Monday, December 30, 2013

"Short changed" by Citrix Provisioning Services!

By Jacques Bensimon.

A couple of years ago, a client showed me a freshly P2V’ed Windows image on which Microsoft Office programs were no longer functioning correctly (as just one of many issues, one could for example open Outlook, but Inbox messages would not open when double-clicked).  A quick Process Monitor trace session later, the cause of the issues became clear:   a large number of Office components, both EXEs and DLLs, are registered via their “short” or “8.3” paths (Registry entries that start with something like “C:\PROGRA~2\MICROS~2\Office12\...”), but somewhere during the P2V process the short name of the “Microsoft Office” folder had changed from its original “MICROS~2” (on the source disk) to “MICROS~1” (on the virtualized disk image), the net effect being that many registered components could no longer be located.  Similar issues were observed with many other programs that had fallen victim to a similar folder short name change.  I didn’t attach much importance to the issue at the time, figuring that it was the result of whatever (clearly flawed) process had been used for P2V [heck, the running joke at IPM is that I’m the old fuddy-duddy endlessly railing against wanton virtualization, so in my world view, these people were just asking for it, no? ;)] and I wound up cobbling together a custom script to restore all short names on the virtual disk to whatever they were on the source disk (which thankfully was still available for comparison).

I later ran into a similar issue at another client when a share hosting a number of network-based applications was migrated from one storage system to another (I believe RoboCopy was the main migration tool on that occasion) and an important application suddenly stopped working, again (as it turned out upon investigation) the result of its folder having acquired a new short name that no longer matched its registration on the various client computers.  Fixed easily enough, so still not panicking.

What took the cake and recently set me on a course to expose and eradicate this class of issues was something that hit much closer to home:  after reverse-imaging a Citrix Provisioning Services vDisk back to the physical disk of a XenApp server (using the standard Citrix BNImage tool, which I happen to prefer to the alternative XenConvert, an opinion that is now under serious review), it occurred to me to perform a quick spot check of the resulting file and folder short names versus those on the source disk (can you say DIR /X  ?) and was horrified [too strong?  You don’t know how seriously I take my images! :)] to discover a large number of differences. 

According to this Citrix KB article entitled “When Capturing an Operating System using BNImage.exe, Microsoft Office 2010 Might not Function as Expected” (duh!), using XenConvert (which doubles as Citrix’s P2V tool in the XenServer arena) instead of BNImage avoids the issue.  Be that as it may, now aware that most mass copy tools (including RoboCopy and the ubiquitous XCopy) suffer from this lack of any respect for existing 8.3 short names, I generalized and extended my old batch script into a full-blown ex post facto 8.3 name “compare & repair” utility, the attached Match83 (32-bit and 64-bit versions included – you cannot use the 32-bit version on a 64-bit OS).  Running it against my reverse image repaired literally *thousands* of mismatches in both folder and file names, most of which I’m sure wouldn’t have done any harm, but many of which would most definitely have.


To conclude, a few notes:

(1) There is apparently no functionality in SMB for remotely modifying short names, so Match83 (and the Windows tool fsutil.exe that it uses under the covers to make short name changes using the syntax fsutil file setshortname <file-specification> <new-short-name>) must target items on a local NTFS-formatted drive.

(2) Why do short name mismatches occur in such large numbers during copy operations (using short-name-unaware copy tools)?  Two main reasons:

a) The original short name of a file or folder was in most cases auto-generated by the file system during the creation of the item, so for example if the folder “Microsoft Silverlight” was created before “Microsoft Office” within the same parent folder, it would likely have acquired the short name “MICROS~1” while “Microsoft Office” acquired “MICROS~2”, but when the parent folder and all its contents are copied to a new location, the operation generally proceeds alphabetically, with the result that “Microsoft Office” is copied first and acquires “MICROS~1” while “Microsoft Silverlight” later acquires “MICROS~2” or even something else like “MICROS~5” if other “Microsoft …” folders have been added to the source since its original creation.

b) Some original short names were not auto-generated but rather were the result of an app installation (often MSI-based) by an installer program that explicitly specified particular short names for files and folders, names that bear no resemblance to what the file system would have auto-generated.  For example, on the Windows 7 computer on which I’m writing this, I have a folder called “Microsoft Research” with the short name “MI4430~1” – no way that the auto-generated short name of a copy of that folder would ever match the original.

(3) It’s of passing interest to note that Microsoft is well aware of the issues that can arise from component registrations that use short names:  starting with the Vista/2008 version of fsutil, the command  fsutil 8dot3name scan [/s] [/v] <folder-path>  became available to report all Registry entries that could potentially be invalidated by short name changes under the specified folder.

(4) Under some rare circumstances, Match83 might be unable to correct a particular item’s short name.  On the assumption that no permissions issue is involved (you should of course run this tool as a full admin), the reason will be that a *new* item has been created on the target since the original copy operation and that its system-generated short name has “hijacked” what should have been another item’s short name.  If you have reason to believe that the original item “needs” its original short name (unlikely), you can use the fsutil syntax provided above to manually swap their short names.

Happy hunting. (and New Year!)

Be sure to follow Jacques on Twitter: @JacqBens

Click Here to Continue Reading >>

Thursday, December 26, 2013

Updated TSFlag / TSFlag-x64 (v1.1)

You may remember Jacques Bensimon introducing us to TSFlag in this post.  He has a new version available here.


As long as TSFlag was already parsing executable headers, I added another item of information to the display:  the executable's so-called "subsystem" (typically either "Windows GUI" or "Windows CUI", CUI = Character User Interface, i.e. console).  TSFlag actually recognizes other possible subsystems -- here's the full list (exactly as TSFlag will display them):

· Windows GUI
· Windows CUI (Console)
· Native (Driver)
· OS/2 CUI
· Posix CUI
· Native (Win9x Driver)
· Windows CE GUI

Safe to say that if you run across one of the bottom four, you're probably doing something wrong! :)

Be sure to follow JB at @JacqBens.

Click Here to Continue Reading >>

Monday, December 23, 2013

Ever hear of 1e100.net?

Neat post from Aaron Silber.
While doing some research at a client, I noticed that IE from time to time goes out to a weird website. It was 1e100.net, so of course I googled it and found this:
What is 1e100.net?
1e100.net is a Google-owned domain name used to identify the servers in our network.
Following standard industry practice, we make sure each IP address has a corresponding hostname. In October 2009, we started using a single domain name to identify our servers across all Google products, rather than use different product domains such as YouTube.com, blogger.com, and Google.com. We did this for two reasons: first, to keep things simpler, and second, to proactively improve security by protecting against potential threats such as cross-site scripting attacks.
Most typical Internet users will never see 1e100.net, but we picked a Googley name for it just in case (1e100 is scientific notation for 1 googol).
Click Here to Continue Reading >>

Wednesday, December 18, 2013

Surviving Windows 8 annoyances with HiDrop

Post by Jacques Bensimon:

Some Windows 8.x annoyances you may be up against, with solutions.

(1) By default, Windows 8 administrative shares (C$, Admin$, etc.) cannot be accessed from a remote computer, regardless of any firewall considerations.

The solution:

Although this looks like a policy, it’s not exposed by any policy template, so the entry has to be made manually.

(2) By default, a process that’s been “Run as administrator” (more about that later) cannot “see” or access drive mappings created by a “non-admin” process such as Explorer or a normal CMD process, even if the mappings are persistent. 

The solution:

Again, this setting is not available via any policy template and has to be made manually.  It also requires a reboot (I forget if (1) does, but you’ll want both anyway).  Unlike (1), I’m pretty sure this one and the one that follows apply to Windows Server 2012 as well.  They also apply to Windows 7 and 2008 R2 *if* you don’t configure the lowest UAC level possible.

(3) This one really annoys me and required a new utility to get around:  you cannot drag-and-drop a file from an Explorer window to an “Administrator” command prompt, a consequence of the more general principle that a process that’s been “Run as administrator” will not accept drop messages (and most other types of window messages) from normal processes. 

Before I describe the workaround, let me clarify something about the wildly inaccurate misnomer that is the “Run as administrator” launch option:  as you can see from the following partial screenshot of “Process Explorer” displaying information about a command window that’s been “Run as administrator” and one launched normally, there is in fact no difference in the user name that owns the process (so there’s no “Run As” going on here involving an actual Administrator account and a secondary logon) – the difference is in that last column (which I encourage you to add to your Process Explorer windows) labeled “Integrity” (for “Integrity Level”, which you can read all about at here) and is part of a process isolation feature that prevents processes from sending messages to or hooking processes running at higher integrity levels than themselves.  When UAC is turned all the way down in Windows 7 and Windows 2008 R2, all processes run at the High integrity level, but in Windows 8 and Server 2012, even with UAC turned down to its minimum level, most processes (including the shell itself and therefore all the processes launched normally from the shell) run at the Medium integrity level unless “Run as administrator” is used explicitly (via context menu or shortcut option) or the executable contains a manifest that requests the High integrity level (which you’ll recognize by that little “UAC shield” overlay that Explorer applies to its icon).


The solution:
This is where the attached “hiDrop” utility (for “high integrity Drop”) comes in:  it runs at the High integrity level *but* uses the appropriate APIs to allow drop messages from lower integrity processes, in particular from Explorer, and then turns around and drops whatever it receives into the last CMD window to have focus, a sort of “proxy drop”.  As long as I was at it, I added the option of dropping instead into the last Notepad window to have focus (useful when you want to drop fully qualified filenames rather than file contents into Notepad).  You can run two simultaneous instances of hiDrop with different targets in order to access both capabilities simultaneously (as in the screenshot below).  Since hiDrop allows dragging multiple files simultaneously, it also has options for separating multiple filenames with either a space (the default) or a newline (only allowed when Notepad is the target – too dangerous otherwise), as well as options for enclosing file paths in double-quotes always, never or only when there is a space in the path (“smart quoting”).  The multiple-file drag-and-drop capability is useful for example when you want to capture the results of an Explorer search window.


If you can think of any other useful features, I’ll be glad to entertain them. (Leave comments below)

One last little tidbit related to process integrity levels:  since there are High and Medium integrity levels available, you can guess that Low is another possibility (and if you read the article linked above, you’ll see that there are in fact two others: Untrusted at the low end, and System at the high end).   For example, the iexplore.exe processes corresponding to the various tabs of IE windows all run at the Low integrity level, as do some Java-related processes and some Adobe Acrobat, Flash and Shockwave processes).  Well, if you’ve ever wondered what the %USERPROFILE%\AppData\LocalLow folder and the HKCU\Software\AppDataLow Registry key are all about, they’re pretty much the only places to which processes running at the Low integrity level are (by default) allowed to write.  So, if not knowing this was annoying to you, consider this another annoyance resolved! :)


Be sure to follow Jacques on Twitter @JacqBens

Click Here to Continue Reading >>