It's a blong, blong, blong road...: Ancient Blog Posts

Here are my older posts from other blogs that I’ve managed to locate using the Internet Archive Way Back Machine. Not many, I know. But anyway, here they are. I hunted these out of the past to see what I wrote about the VBScript overflow error, that at the time was partly researched using WinDbg and veered into the world of SSE2 instructions.

19th June 2004 Conditional Defines, etc.
19th June 2004 Pure evil? Me?
26th November 2006 Stories Of The Demise Of Brian's Blog Are Greatly Overstated
26th November 2006 Shortcuts
27th November 2006 TechEd Developers 2006
20th December 2006 Problems at Work

19th June 2004 - Conditional Defines, etc.

Ever needed to write code that works for certain versions of the compilers? If so then doubtless you've made use of conditional defines (aka conditional compilation symbols, conditional symbols or metasymbols). These are symbols that can be used in $IFDEF / $ENDIF conditional compilation directives or, as of Delphi 6, in $IF / $IFEND directives.

To write different code in a single file that targets Windows, Linux and Delphi you could use this type of conditional compilation block:

{$IFDEF LINUX}
  //do Linux-specific code here
{$ENDIF}
{$IFDEF MSWINDOWS}
  //do Windows-specific code here
{$ENDIF}
{$IFDEF CLR}
  //do .NET-specific code here
{$ENDIF}

Or alternatively this form, which is functionally equivalent:

{$IF Defined(LINUX)}
  //do Linux-specific code here
{$ELSEIF Defined(MSWINDOWS)}
  //do Windows-specific code here
{$ELSEIF Defined (CLR)}
  //do .NET-specific code here
{$IFEND}

Over the various versions of Turbo Pascal, Borland Pascal, Delphi, C++Builder & Kylix the Delphi (née Pascal) compiler has offered many different predefined conditional symbols but it is difficult to locate a map of all the symbols and which products defined them.

The following table attempts to fill this gap by listing all known predefined conditional symbols.

Conditional Symbols

Conditional Symbol	Meaning	Defined In
VER30	compiler version 3.0	Delphi Prism 3.0
VER80	compiler version 8	Delphi 1
VER90	compiler version 9	Delphi 2
VER93	compiler version 9.3	C++Builder 1
VER100	compiler version 10	Delphi 3
VER110	compiler version 11	C++Builder 3
VER120	compiler version 12	Delphi 4
VER125	compiler version 12.5	C++Builder 4
VER130	compiler version 13	Delphi/C++Builder 5
VER140	compiler version 14	Delphi/C++Builder 6, Kylix 1, 2 & 3
VER150	compiler version 15	Delphi 7
VER160	compiler version 16	Delphi 8 for .NET, Delphi 8 for .NET IDE Integration Pack (Delphi 7.1)
VER170	compiler version 17	Delphi 2005
VER180	compiler version 18	Delphi 2006 and Delphi 2007 for Win32
VER185	compiler version 18	Delphi 2007 for Win32
VER190	compiler version 19	Delphi 2007 for .NET
VER200	compiler version 20	Delphi 2009
VER210	compiler version 21	Delphi 2010
VER220	compiler version 22	Delphi XE
VER230	compiler version 23	Delphi XE 2
VER240	compiler version 24	Delphi XE 3
BCB	Delphi code being compiled from within C++Builder	C++Builder
GAMMA_TEST	Delphi code being compiled from a pre-release (gamma test) product	C++Builder 4 (this was a bug and should not be defined in any shipping product)
MANAGEDCODE	.NET managed code being generated	Delphi for .NET
NATIVECODE	Non-.NET native code being generated	Not sure which compiler introduced this. Presumably Delphi 2005
WINDOWS	compiling for Win16 platform	Delphi 1
MSWINDOWS	compiling for any Windows platform	Delphi 6 and later
WIN32	compiling for 32-bit Windows	32-bit Delphi versions from Delphi 2 onwards
WIN64	compiling for 64-bit Windows	64-bit Delphi versions from Delphi XE2 onwards
LINUX	compiling for any Linux platform	Kylix
LINUX32	compiling for 32-bit Linux platform	Kylix
CLR	compiling for the Common Language Runtime (Microsoft .NET platform)	Delphi for .NET, Delphi Prism
CIL	Common Intermediate Language (CIL) code being generated	Delphi for .NET
POSIX	compiling for a POSIX-compliant OS	Kylix and DCCOSX (Delphi XE2 and later)
POSIX32	compiling for a 32-bit POSIX-compliant OS	Kylix and DCCOSX (Delphi XE2 and later)
MACOS	targeting Mac OS X	DCCOSX (Delphi XE2 and later)
CPU386	compiling on an 80386 or better	all versions of Kylix and 32-bit Delphi
CPUX86	compiling in a 32-bit Delphi compiler	dcc32 (Delphi XE2 and later)
CPUX64	compiling in a 64-bit Delphi compiler	dcc64 (Delphi XE2 and later)
PIC	position independent code (PIC) being generated	Kylix when compiling shared objects, and dccosx
ELF	compiling an ELF format executable file	Kylix
PC_MAPPED_EXCEPTIONS	program counter based exception handling, as opposed to OS-based exception handling	Kylix and dccosx
DECLARE_GPL	compiling a GPL application	Kylix Open Edition
UNICODE	Unicode compiler	Delphi 2009 and later
CONDITIONALEXPRESSIONS	conditional directives support expression evaluation	Delphi/C++Builder 6 and later, Delphi for .NET and Kylix
PRISM	Delphi Prism	Delphi Prism
PRISM30	Delphi Prism 3.0	Delphi Prism 3.0
PRISM30UP	Delphi Prism 3.0 and up	Delphi Prism 3.0 and up
MONO	compiler running in Mono	Delphi Prism
DOTNET	compiler running in .NET	Delphi Prism

As of Delphi 6 times support for conditional expression evaluation within compiler directives was introduced and the compiler and RTL have (supposedly) unique version numbers defined with the RTLVersion and CompilerVersion constants. This allows you to write code like this at the top of a unit to ensure an appropriately recent version of a compiler targeting a specific platform is used:

{$IFDEF CONDITIONALEXPRESSIONS}
  {$IF not Defined(MSWINDOWS)}
    {$MESSAGE ERROR
     'Must be compiled with Delphi version 6 or later'}
  {$IFEND}
  {$IF Declared(RTLVersion) and (RTLVersion < 14.11)}
    {$MESSAGE ERROR
     'Oldest permitted compiler is Delphi 6 with Update Pack 1'}
  {$IFEND}
{$ELSE}
  'This requires a more recent version of the compiler (Delphi 6 or later)'
{$ENDIF}

Again, tables of these constants against the defining products are difficult to source, so here is my list.

Conditional Constants

Product	RTLVersion value	CompilerVersion value
Kylix 1	14.00	not defined
Delphi 6.00	14.10	14.01
Delphi 6.01	14.11	14.01
Delphi 6.02	14.20	14.01
Delphi 6.02 with RTL Updates	14.31	14.01
C++Builder 6	14.20	14.01
Kylix 2	14.20	14.10
Kylix 3	14.50	14.50
Delphi 7.0x	15.00	15.00
Delphi for .NET Preview Edition	not defined	16.00
Delphi 7.1 (Delphi 8 for .NET IDE Integration Pack)	15.00	16.00
Delphi 8.x for .NET	16.00	16.00
Delphi 2005	17.00	17.00
Delphi 2006	18.00	18.00
Delphi 2007	18.00	18.50
Delphi 2009	20.00	20.00
Delphi 2010	21.00	21.00
Delphi XE	22.00	22.00
Delphi XE2	23.00	23.00
Delphi XE3	24.00	24.00

Also just for good measure, here is (to the best of my knowledge) the list of product release dates from the Delphi/Kylix/C++Builder teams.

Product Release Dates

Product	Version	Released
Delphi	1.00	February 1995
Delphi	1.01	April 1995
Delphi	1.02	August 1995
Delphi	2.00	February 1996
Delphi	2.01	June 1996
C++Builder	1.00	February 1997
Delphi	3.00	April 1997
Delphi	3.01	August 1997
Delphi	3.02	December 1997
C++Builder	3.00	February 1998
Delphi	4.00	June 1998
Delphi	4.01	October 1998
C++Builder	4.00	January 1999
Delphi	4.02	February 1999
C++Builder	4.01	June 1999
Delphi	5.00	August 1999
Delphi	5.01	January 2000
C++Builder	5.00	February 2000
C++Builder	5.01	August 2000
Kylix	1.00	February 2001
Delphi	6.00	May 2001
Delphi	6.01	October 2001
Kylix	2.00	November 2001
C++Builder	6.00	February 2002
Delphi	6.02	March 2002
C++Builder	6.01	June 2002
Kylix	3.00	July 2002
Delphi	7.00	August 2002
Delphi for .NET Compiler	Preview	August 2002
C++Builder	6.02	August 2002
C++ Mobile Edition	1.02	December 2002
Delphi for .NET Compiler	Preview Update 1	November 2002
Delphi for .NET Compiler	Preview Update 3 (sic)	February 2003
C++Builder	6.04 (sic)	February 2003
C++ Mobile Edition	1.1	February 2003
C#Builder	1.00	Summer 2003
C++BuilderX	1.00	September 2002
Delphi for .NET	8.00	December 2003
Delphi for .NET	8.01	February 2004
Delphi for .NET	8.02	March 2004
C++BuilderX 1.5 Mobile Edition	1.50	April 2004?
Delphi	7.01	May 2004
Delphi for .NET	8.03	March 2005
Delphi 2005	9.00	October 2004
Delphi 2005	9.01	December 2004
Delphi 2005	9.02	Mar 2005
Delphi 2005	9.03	May 2005
Delphi 2006	10.00	Nov 2005
Delphi 2007 for Win32	11.00	Mar 2007
Delphi 2007 for .NET	11.00	Sep 2007
Delphi 2009	12.00	Aug 2008
Delphi 2010	14.00 (Delphi skipped version 13)	Aug 2009
Delphi XE	15.00	Aug 2010
Delphi XE2	16.00	Aug 2011
Delphi XE3	17.00	Aug 2012

19th June 2004 - Pure evil? Me?

Am I really "pure evil"? According to an online post, that's the description I've been given.... You know, I think I could grow to like that :-)

I had a question passed to me, which if applied to Delphi has a trivial answer. The question was as follows:

I want to be able to call a method on all forms in an application. To do this I must be able to get a list of all forms in an application. The general wisdom appears to be that this is not possible except in an MDI application (which doesn’t help me because it isn’t an MDI application). The generally accepted approach to this is to ensure that every form which is created is added to your own global list of forms and then you have access to all of the forms. I want a better, more generic solution.

My first port of call was a contact at Microsoft who asked the WinForms Lead PM. The response back was:

There is no built in way to do this prior to Whidbey.

Oh dear. So spelunking is required. In the timeframe I could allot to the question I found nothing useful at all so I tried the question out on Borland Support's Roy Nelson. Rather intriguingly he came back with:

I can see two possible ways of doing this, both iffy!

I pressed him for more information and he posted his two solutions on his personal blog. He also noted that for making him think so hard I am "pure evil"! You know, on reflection (no .NET pun intended... really) I reckon I can live with that. It makes me seem more interesting than I really am. And with regard to the comments in the post containing your second solution Roy, I'm sure we can increase your readership :-) Just keep pumping out well researched nuggets like this.

The two solutions use reflection to access internal data structures used by WinForms and extract the required list of forms. The first one accesses the nested type System.Windows.Forms.Application.ThreadWindows, and the second one takes advantage of a private static field of System.Windows.Forms.NativeWindow called hashBuckets, which is an array of HandleBucket structs, each of which appears to correspond to a window handle managed by WinForms.

Kudos to you Roy for such productive spelunking. I'll list the code from Roy's blog here, simply because the original formatting has gone for a Burton [ Update - I see he's taken the hint and made them a little shinier now ]. Note this is Delphi for .NET syntax, which should be trivial to translate into C#. And again, note, this is Roy's code (although I have removed some redundancy and tweaked it ever so slightly).

const
  AllBindingFlags: BindingFlags = BindingFlags.Instance or
    BindingFlags.Static or BindingFlags.Public or
    BindingFlags.NonPublic or
    BindingFlags.FlattenHierarchy;

procedure TWinForm1.Button1_Click(sender: System.Object; e: System.EventArgs);
const
  CtorParams: array[0..1] of &Type = (TypeOf(Control), TypeOf(Boolean));
type
  TIntPtrArrayType = array of IntPtr;
var
  MSCorlibModule: Module;
  ThreadWindowType: &Type;
  FldInfo: FieldInfo;
  ThreadWindowsCtor: ConstructorInfo;
  ThreadWindowsObject: &Object;
  ThreadWindowsParams: array[0..1] of &Object;
  ThreadWindowCount, i: integer;
  WindowsArr: TIntPtrArrayType;
  TempControl: Control;
begin
  //We could "MSCorlibAssembly := Assembly.Load('mscorlib');" but this is
  //easier... there are numerous ways to skin this cat.
  MSCorlibModule := Self.GetType.BaseType.Module;
  if not Assigned(MSCorlibModule) then
    Exit;
  ThreadWindowType := MSCorlibModule.GetType('System.Windows.Forms.Application+ThreadWindows');
  if not Assigned(ThreadWindowType) then
    Exit;
  ThreadWindowsCtor := ThreadWindowType.GetConstructor(AllBindingFlags, nil,
    CallingConventions.HasThis, CtorParams, nil);
  if not Assigned(ThreadWindowsCtor) then
    Exit;
  ThreadWindowsParams[0] := nil; //Look for all controls on the thread
  ThreadWindowsParams[1] := &Object(False); //False ONLY gets WinForms
  //Create the ThreadWindows Object
  ThreadWindowsObject := ThreadWindowsCtor.Invoke(ThreadWindowsParams);
  if not Assigned(ThreadWindowsObject) then
    Exit;
  //Get the number of windows on the thread
  FldInfo := ThreadWindowType.GetField('windowCount', AllBindingFlags);
  ThreadWindowCount := Integer(FldInfo.GetValue(ThreadWindowsObject));
  if ThreadWindowCount <= 0 then
    Exit;
  //Get all window handles
  FldInfo := ThreadWindowType.GetField('windows', AllBindingFlags);
  if not Assigned(FldInfo) then
    Exit;
  WindowsArr := TIntPtrArrayType(FldInfo.GetValue(ThreadWindowsObject));
  if not Assigned(WindowsArr) then
    Exit;
  for i := 0 to ThreadWindowCount - 1 do
  begin
    TempControl := Control.FromHandle(WindowsArr[i]);
    if Assigned(TempControl) then
      //Get the caption of each control
      MessageBox.Show(TempControl.Text);
  end;
end;

procedure TWinForm1.Button2_Click(sender: System.Object; e: System.EventArgs);
var
  MSCorlibModule: Module;
  HandleBucketType: &Type;
  hashBucketField: FieldInfo;
  hashBuckets: &Array;
  HandleBucket: &Object;
  HandleField: FieldInfo;
  WinHandle: IntPtr;
  FoundControl: Control;
  i: Integer;
begin
  MSCorlibModule := Self.GetType.BaseType.Module;
  if not Assigned(MSCorlibModule) then
    Exit;
  HandleBucketType := MSCorlibModule.GetType('System.Windows.Forms.NativeWindow+HandleBucket');
  if not Assigned(HandleBucketType) then
    Exit;
  HandleField := HandleBucketType.GetField('handle', AllBindingFlags);
  hashBucketField := TypeOf(NativeWindow).GetField('hashBuckets', AllBindingFlags);
  if not Assigned(hashBucketField) then
    Exit;
  hashBuckets := System.Array(hashBucketField.GetValue(nil));
  for i := 0 to hashBuckets.Length - 1 do
  begin
    HandleBucket := hashBuckets.GetValue(i);
    if Assigned(HandleBucket) then
    begin
      WinHandle := IntPtr(HandleField.GetValue(HandleBucket));
      if (WinHandle.ToInt32 <> 0) then
      begin
        FoundControl := Control.FromHandle(WinHandle);
        if FoundControl is System.Windows.Forms.Form then
          MessageBox.Show(FoundControl.Text);
      end;
    end;
  end;
end;

With reference to the WinForms PM's comment I am not sure how to interpret "built in". Surely reflection is built into .NET? So therefore there is a built in way to do it. Maybe what we should infer from the comment is that there is no trivial, effortless way of doing it.

26th November 2006 - Stories Of The Demise Of Brian's Blog Are Greatly Overstated

Hello all and welcome to my part of the 4 Chaps From Blighty blog. My name is Brian Long and this is (pretty much) my first foray into the world of blogging. I did briefly have a blog for 5 or 6 minutes a couple of years back, but that particular relationship didn’t work out. Myself and the blog wanted different things out of the relationship, and after some hard times we parted quite acrimoniously. The blog retained custody of the 2 posts I had made. So here is my first concerted effort at a real blog: Blong’s blog.

I’m sure I speak for the other chaps when I say this collective contribution to the blogosphere (is that a real word now?) represents an interesting departure from simply telling each other the things that interest or perplex us and that we all look forward to sharing views, tips and thoughts with the world at large through this medium.

Before we kick off, I should introduce myself a bit more. 11.75 years ago I worked at Borland International, having spent 4 years supporting their programming languages (Turbo Pascal, Turbo Basic, Turbo C, Turbo C++, Turbo Assembler, and Turbo Prolog) and a year doing some other stuff.

I left Borland just a few days before Delphi 1 was released and spent a large part of the next decade running training courses for that original release and the successive versions of Delphi and also C++Builder. Between training courses I wrote many articles for The Delphi Magazine and have been brought in by various clients on a consultancy basis to advise on project direction and to help troubleshoot tricky problems. I also got onto the conference circuit and delivered numerous sessions on areas of my interest on the subjects over the years. The training was great for instilling discipline in me and the trouble-shooting came quite naturally after the support background at Borland.

Earlier this century I got hit by the .NET bug, as did so many other Window programmers. Ever since, I have been working on the platform with C# (and also Delphi), picking up techniques and looking into lesser known areas of interest. I was fascinated at the innate ability of any developer to write a very flexible profiling tool with the Profiling API and spent time building helper tools that track exceptions occurring in .NET applications, trace method execution and simply log a vast amount of information on the internal operation of the application. Interoperability of managed and unmanaged code also piqued my curiosity and so I spent quite some time specialising in the subject.

The old troubleshooting spirit remains vibrant, though, as I am frequently found knee-deep in low-level debugging tools, such as CorDbg, WinDbg and SOS, the SysInternals tools and various other utilities, in order to solve problems and introspect into suspect systems. I’ve used some of the lesser known tools have to manually detect viruses and to remove malware (such as rootkits) before general purpose tools existed for the job.

Debugging, troubleshooting, lower level technologies: that gives you some sort of picture of the sort of stuff that you might expect to hear from me through this blog. I should also add that I spent some time a few years back getting to know the Microsoft Speech API (as supplied with every copy of Windows XP and above). The SAPI implementation has recently been updated in Windows Vista and so I shall be looking into what’s new at some point soon.

Okay. That’s my hello post out of the way. Currently I am getting on very nicely with my blog. I just hope this relationship works out better than my last one did. Until the next post...

26th November 2006 - Shortcuts

Whilst flitting from company to company doing the consultancy and training jobs that occupy a lot of my time, I tend to see that many developers are not aware of some of the basic convenient keystrokes and useful shortcuts that Windows makes available. So with that in mind I thought I’d write them out here to make a permanent record of them. Naturally you’ll know a greater or lesser amount of them already, but I’d like to think there will be something new in the list for everyone who reads it.

I should perhaps mention that I tend to prefer using the keyboard over the mouse, as it keeps me (to some extent) from having one arm darting back and forth over the desk all the time. Also, some machines are hooked up to very weird mice that take a lot of getting used to. So shortcuts are more important to me than I sometimes find with other techy people.

First things first, let’s look at the Windows key on the typical Windows keyboard (ÿ) that, as you doubtless know, brings up the Start menu when pressed.

This can be very useful for saving time aimlessly searching through the arbitrarily configured Start menu on any random machine to get at common options.

ÿ`+R`	Invoke the Run... dialog (much the same as bringing up the Start menu with ÿ and then pressing `R`, assuming something else with a shortcut of `R` hasn’t been added to the main Start menu)
ÿ`+E`	Run a copy of Windows Explorer (much the same as pressing ÿ`+R`, then typing `explorer` and pressing `Enter`)
ÿ`+L`	Lock the workstation
ÿ`+M`	Minimise all applications so you can (probably) see your desktop
ÿ`+Shift+M`	Undo the minimise all applications operation instigated by ÿ`+M`
ÿ`+D`	show desktop (this brings the desktop window temporarily above all other windows)
ÿ`+F1`	Windows help
ÿ`+Break`	System Properties Control Panel applet. This is the same as right-clicking on My Computer (if you can find it) and choosing `Properties`.
ÿ`+U`	Dialog reader. This uses the Microsoft SAPI (Speech API) TTS (Text To Speech) engine to read on-screen dialogs.
ÿ`+Tab`	Switch between items on Task Bar without actually seeing the selection a you do with `Alt+Tab`. Use enter to select the one you want

Other shortcuts that do not involve the Windows key include:

`Ctrl+Shift+Escape`	Invokes the Task Manager or whatever replacement you might have installed, such as Process Explorer from SysInternals. Note that in XP and later `Ctrl+Alt+Del` does the same, but previously on NT-based Windows it would bring up a system modal dialog with options of logging off, running Task Manager, shutting down and so on.
`Alt+Tab`	You doubtless know this allows you to switch between applications. What’s less known is that when you have a large number of applications and your fingers get trigger happy and take you past the one you want, you can introduce the `Shift` key to reverse through the list. `Alt+Shift+Tab` on its own simply starts at the end of the list of applications.
`Alt+F4`	If you’ve paid attention to any system menu you will know that this is the generic shortcut hey for closing any regular window in Windows.
`Ctrl+F4`	This is a variation of `Alt+F4` and is specific to closing an MDI child window.
`Alt+Space`	This drops down the System menu for the active window. So you can typically shut any window with `Alt+Space`, `C` (the `Close` menu on a system menu will have a shortcut of `C`) as an alternative to `Alt+F4`. However my most common use of this keystroke is when I have gone from 2 monitors to 1 or dropped resolution, and some application starts up at the same location as last time and it happens to be off-screen. It can be disconcerting to start an application, see it on the Task Bar, but not see the window. But given the most likely thing is that it is off-screen I press `Alt+Space`, `M` (for `Move`) and use the cursor keys to try and bring it back on screen. Holding down the left or up cursor key often does the job, though if it doesn’t another possibility is that the stored location has got corrupted. If playing with the cursor keys doesn’t do it, running up WinSpector will allow me to find the window size and position fairly quickly. Note that the mouse will only kick in and allow you to move a window with it when the window is onscreen.
`Alt+Hyphen`	This brings up the system menu for an MDI child window. `Alt+Hyphen` and `Ctrl+F4` work just as well with most faux MDI windows, such as in Excel 2003 (where `Ctrl+W` is advertised as the keystroke to close the MDI child windows).
`Shift+F8`	once upon a time I was the only person I knew of that was aware of this listbox keystroke. `Shift+F8` offers a mouseless way of making a discontiguous selection in a multi-selection listbox. The keystroke starts the listbox selection bar flashing, and you can use the cursor keys to navigate around the items, pressing `Space` to toggle selection on any of them. `Shift+F8` can also be used to turn this selection mode off when you are done. Note that this does not work with a listview, only a listbox (a list view is more intuitive, using `Ctrl` to keep discontiguous selections in place and `Space` to toggle them). The reason I implied that more people know about it now is that in August Raymond Chen wrote about it in his UI blog. The multi-select listbox also supports other keystrokes: `Ctrl+/` selects all items and `Ctrl+\` deselects all items bar the active one.
`Alt+Enter`	anywhere you can select something, right-click on it and choose a `Properties` menu item to get a Properties dialog, you can typically use this keystroke. Windows Explorer would be a prime example of such a context where this keystroke is useful.
`Alt+PrtScr` / `Alt+PrintScreen`	`PrtScr` on its own is widely known to copy an image of the whole Windows desktop onto the clipboard. `Alt+PrtScr` is often more useful in that it copies an image of the active window only to the clipboard.
Dragging plus modifiers	When dragging in many applications, the drag operation alone will move the item (for example the text in Word, or the file in Windows Explorer). If you hold the `Ctrl` key down during the drag you will ensure you perform a copy, whereas holding `Shift` confirms you want to move the item. In Windows Explorer, holding `Ctrl+Shift` will cause a shortcut to the dragged item to be created.

Drag drop operations were mentioned above; it should be mentioned that it can be really helpful to drag a file or a folder to both the Run dialog and to the command prompt (either the traditional Windows command prompt or the recent Windows PowerShell). This can be helpful in changing directories and so forth.

When navigating around Windows, using either the command prompt or Windows Explorer, appropriate use of environment variables can help you get where you want a bit quicker. In case you are too young to know the deal with environment variables, they are set it the System Properties Control Panel applet (ÿ+Break) on the Advanced page through the Environment Variables button. You can set variables to be user-specific or to be available for all users. All the variables are available to any application launched after they have been set. There are various predefined variables set up by default and then all the user-defined variables are added on top.

In a Windows command prompt (cmd.exe) you can see all the environment variables available by executing the SET command. In Windows PowerShell you can use Get ChildItem env: or dir env: to get the same result. The reason I am bringing this up is that you can use environment variables in both Windows Explorer and in the available command prompts. For example, if you often need to get to the .NET Framework SDK 2 directory, you could set the environment variable SDK2 to have a value of C:\Program Files\Microsoft Visual Studio 8\SDK\v2.0. To get to this directory in Windows Explorer, just enter:

%SDK2%

in the address bar. In Windows command prompt, enter:

cd /D %SDK2%

In Windows PowerShell use:

Set Location $env:SDK2

or alternatively:

cd $env:SDK

The useful predefined environment variables include:

windir - maps to the base Windows directory, such as C:\windows
userprofile – point to your user profile directory. The desktop is located in %userprofile%\Desktop and your personal documents directory is in %userprofile%\My Documents
computername – as it suggests, your computer’s name
username – the currently logged on user

Both the original Windows command prompt and the newer Windows PowerShell support tab completion. Some users find the command prompt tab completion is not active for them. This will almost certainly be due to the registry setting not being set. You can set HKEY_CURRENT_USER\Software\Microsoft\Command Processor\CompletionChar to a DWord value that maps to the desired control character; for the Tab key that will be the value 9.

Thanks to the introduction of .NET I have often wanted to create directories starting with a full stop (or period, if you prefer). If you’ve tried this in Windows Explorer, you won’t have got very far. Windows Explorer rejects the idea with short shrift, and you therefore might understandably have decided that it is simply not possible. In fact it is; Explorer is just fussy. The best thing to do is to drop to a command prompt, change to the relevant parent directory and create it with the mkdir command, for example:

mkdir .net

In PowerShell, you get the same result with

New-Item -path .net -ItemType directory

mkdir .net

Okay, so nothing ground-breaking in this post, but it gives me a quick reference to a number of keystrokes I find useful in and around Windows, and hopefull will also be of use to some of you out there.

27th November 2006 - TechEd Developers 2006

At the beginning of November I had the privilege of being asked to speak at the foremost European Microsoft developer-oriented conference: Microsoft TechEd Developers. As many of you know, this year’s TechEd was held in the beautiful city of Barcelona, a place to which I have not travelled to before, which added great value to the trip for me.

I had just the one session to deliver on Wednesday the 8th: what was scheduled as a whiteboard session entitled “Alternative .NET Debugging Facilities.”

The abstract for the session was as follows:

Debugging is a necessity. Sometimes an IDE-resident debugger doesn't have what it takes to track a problem down. There are a variety of other debugging tools available from Microsoft that can come to the rescue in these and other circumstances and in this session we take a look at what's available and how you can use these tools to improve your debugging prowess. This Whiteboard discussion will sidestep Visual Studio's debugger, and maybe brush over DbgCLR and MDbg, and will focus squarely on WinDbg with Son of Strike and various supporting tools and will look at application introspection, memory consumption and miscellaneous lower level trickery.

As you can see the session is on my pet subject of low-level and lesser known debugging tools, and is designed to ease you into the world of these debuggers giving you the information you need to get started and show you what sort of things are possible. In truth the session really warrants being a breakout, but it was scheduled as a breakout and so I planned accordingly. Whiteboard discussions are in smaller rooms than the regular breakout sessions, seating about 80 people. I was thrilled to find the session fill to capacity and it seemed to go well. There weren’t many questions to force an actual two way discussion, so the laws of nature had their way and I delivered what was essentially a slideless breakout session: all demos and relevant explanation.

The feedback was great, which was very gratifying. With that and the full house I was asked to repeat the session on the Friday, which I was naturally very happy to do. The repeat session wasn’t on the printed agenda, but was broadcast on the update monitors around the conference. Given that most attendees use their personal printed agenda to organise their session schedules I wasn’t expecting many people to turn up, but was happy with the crowd I got. There were more questions that in the first session and so this one was steered more by the audience than by me, which was actually more fun than I had anticipated. That sort of thing keeps you on your toes, for sure!

Outside of my own talks I chatted to acquaintances, both new and old, and attended a lot of great sessions. It was great to catch up with people I have known for years, such as Danny Thorpe (once the guy who was the main architect of the Delphi compiler after Anders Hejlsberg went to Microsoft, now one of the engineers behind Windows Live), Steve Teixeira (former Delphi R&D engineer, now a Visual C++ Program Manager) and Donald Farmer (one time Delphi programmer and now formally a Group Program Manager and informally a SQL Server Integration Services wizard in Microsoft’s SQL Server Business Intelligence department). It was similarly good to have chats with Chad Hower and Hadi Hadiri. Additionally, I unexpectedly bumped into a colleague, Richard, from the bank I am contracting for at the moment (I wasn’t aware he was going as an attendee so that was a nice surprise).

Jeff Prosise’s pre-conference tutorial was a day-long education on ASP.NET AJAX (formerly the Atlas framework) and was an excellent leg-up into that technology. It has certainly cut down my learning time significantly and Jeff has an enjoyable, easy going lecture style.

I went to see the inimitable Anders Hejlsberg do two sessions, one on LINQ and one on LINQ to SQL (formerly DLINQ). Great sessions they were with demos built up step by step to demonstrate the key features of these frameworks. I’ve since chatted with a number of people about LINQ to SQL and the more cynical have an immediate knee-jerk reaction that it will be a bad thing. They feel that having something spewing out SQL for you is guaranteed to be a retrograde step, and feel it is no benefit to take away the requirement for SQL skills from the 3GL programmer.

Anders suggests that LINQ addresses the “impedance mismatch” of the set-based SQL and programming languages like C# and VB, where SQL instructions are represented as strings and errors show up at runtime. LINQ allows you to use all the sorts of operators you are familiar with and have learnt in SQL, but in native C# or VB. LINQ itself operates against regular classes, but LINQ to SQL and LINQ to XML extend this to operating against relational data and XML. LINQ to SQL makes some very sensible looking SQL statements (you can log all the SQL to ensure it is behaving itself), and defers execution of any queries until the results are actually required. LINQ to XML makes XML creation, navigation and manipulation much simpler, and provides XPath functionality in a style consistent with all the other LINQ options. On top of all this, you can run JOINs across SQL and XML, if it becomes appropriate in your application. All in all I think LINQ looks an exciting technology that I look forward to seeing materialise in C# 3.0, expected in the .NET Framework 3.5.

I enjoyed Ingo Rammer’s session on performance and scalability and was delighted to discuss debugging tools over lunch with him one day. As an interesting coincidence, Ingo also delivered a whiteboard discussion session on WinDbg et al, but pitched slightly more advanced than my own, and his session was also repeated. Additionally, I discovered from my bank colleague Richard that he had brought Ingo over in his consultancy capacity to advise on the department’s project last year. What a small world!

Having not explored the world of Python (other than the Monty variety, naturally) I was intrigued to see what IronPython is all about (this is Microsoft’s implementation of Python for .NET with a BSD-style license). Mahesh Prakriya’s session on the matter was very enlightening. Some neat demos showed how trivial it was to talk to COM objects, such as Microsoft Agent and the Tablet PC API. It was really educational, just in term of using those two APIs. I have a Tablet PC (a nice, compact ThinkPad X41) and so want to check out this Ink Recognition API. It’s one of the things on my To Do list. My To Do list is quite long. I need to work on that.

I watched Steve Teixeira highlight what is coming in the next Visual C++ release in the “Orcas” product, which was useful. I also caught Martin Calsyn showing what could be done with Microsoft Robotics Studio. This was purely of academic interest for me, but has an indirect relevance as a friend of mine, Steve Tudor will be demonstrating it at the fourth DeveloperDeveloperDeveloper day (DDD 4).

DDD is a community-run event hosted by Microsoft for free on a Saturday, about twice per year thus far. It is very popular and tends to get “sold” out pretty quickly (there are only 300 places available). I spoke at the original DDD and also at DDD 2, once on my debugging friends and once on reverse-engineering .NET to gain a better understanding of it. The sessions seemed to be quite well received, even if only going by the comments of Benjamin Mitchell, Ian Smith (DDD), Ian Smith again (DDD2) and Charles Cook. With DDD 3 the organisers introduced a session voting system where prospective attendees can vote for the sessions they fancy seeing. My session proposals on speech and interop got narrowly pipped at the post, and so I missed that one. My current job means I am too busy to do appropriate preparation so I skipped submitting for DDD 4. Whilst I won’t be speaking at DDD 4 I am certainly going to be attending; there are a number of sessions I am looking forward to.

So, back to the subject of TechEd. There was a great selection of sessions, many of which I saw, but I’m really annoyed that missed Jeffrey Snover doing a session on PowerShell, the product he is the Architect for.

Speakers were allocated times to be available on the Microsoft stand in the Ask The Expert area, in order to help out answer questions that the attendees either thought up during the event or had brought along with them. This was very enjoyable, talking to the attendees and also mingling with Microsoft staff and MVPs from around Europe. At the end of my last ATE session, I was snapped with my customised badges (or buttons if you prefer) on as you can see here.

That was my first TechEd and what an extremely enjoyable and rewarding experience it was. I hope to be able to attend many more in the years to come.

20th December 2006 - Problems at Work

To keep myself out of mischief, I am currently mid-contract at an investment bank in the heart of the City of London. What I’m working on isn’t really relevant at this point in time, but an issue turned up in another group a month or so back that I got myself involved in, as getting to the bottom of it seemed to warrant my current favourite tool: WinDbg. The department I work in builds one main web application, which has a lot of ASP and is being extended in ASP.NET 1.1. I contributed a little to the ASP.NET side, but am currently working on project that is adjacent to.

Included in the department is an infrastructure team that deals with all the hardware and the deployment issues for the client-facing application. An unexpected error started becoming prevalent as the infrastructure guys moved the deployment surface (a web farm) to new hardware running Windows Server 2003. Intermittently (or so it appeared) an ASP page would report:

Microsoft VBScript runtime error '800a0006'
 
Overflow

and then report the file and line number. This hadn’t happened with the previous Windows 2000 server setup, and the error seemed to be occurring on a simple floating point operation that didn’t seem to be likely to overflow. A bit of research found other people with the same issue on the Internet, such as this post as one example.

Naturally this prompted an immediate use of the Microsoft support agreement, and a very helpful engineer spent many hours trying out various things to no immediate avail. As the hours wore on it was observed that the error occurred on machines that were built on Intel processors, but did not occur on machines with AMD processors installed and suspicion initially arose as to whether blame should therefore be apportioned to Intel, and indeed whether a suitable way forward might be to strip out the Intel hardware and focus solely on AMD.

I had been keeping an ear on the progress of the issue and had some thoughts on the problem. Unexpected floating point exceptions are relatively common when mixing unmanaged modules compiled with Borland compilers and Microsoft Visual C++, thanks to their different setup of the floating point control word register in the FPU (the FPCW register). Additionally, similar issues have been reported when trying to use some unmanaged DLLs. So I started getting involved and suggested a few possibilities.

I first tested out the hypothesis that it might the FP control word by knocking up a trivial COM object in Delphi that reported the current value. Unfortunately in this case the default Microsoft value of 0x27F prevailed. The only Borland-compiled module in the whole setup was a third-party component used in one of the C++ COM modules that at one point was the prime suspect. These findings seemed to clear it of guilt.

The Microsoft engineer had been analysing crash dumps of the failing machine using a system debugger (from Debugging Tools for Windows) and beat me to the next step of examining more advanced floating point capabilities. He rather keenly noticed that on the Intel machines the MMX control status register (MXCSR, which was actually introduced with SSE instructions) value was being changed, whereas on the AMD hardware it remained fixed. Additionally a colleague had narrowed down a test case that always failed on Intel hardware. It relied on executing part of the system in a certain way and then a simple floating point division would cause the overflow error to appear.

This new state of affairs gave something to immediately bite onto. The documentation for SSE2 machine instructions can found, among other places, in the AMD64 Architecture Technical Docs. The instructions themselves are documented in Volume 4 of the AMD64 Architecture Programmer’s Manual and the MXCSR register is described in section 4.3.2 of Volume 1. MXCSR is rather like an SSE version of FPCW, in that some of its bits relate to masking and unmasking floating point exceptions. Bits 7 to 12 control whether various floating point exceptions will be masked or not, whilst bits 0 to 5 are set by the processor when a floating point exception occurs. The default value of MXCSR is 0x1F80, meaning all exceptions are masked and no exceptions have occurred. However the code execution was causing it to be changed to values such as 0x1FA0 (precision exception has occurred) and 0x1FA8 (precision and overflow exceptions have occurred).

Aha! That starts to makes sense.

And so it made sense to unleash WinDbg (from Debugging Tools for Windows) and use it to hone in on the issue using a test case that had been found to reliably cause the issue when running against Intel server hardware. WinDbg was attached to the ASP.NET worker process with F6, and then some basic features were used to employ the divide and conquer technique to isolate the issue that caused the masked overflow exception. These included loading up source files (Ctrl+O), setting up pertinent breakpoints (F9) and viewing registers. You can dump all the regular floating point registers with the rF command in the command window (Alt+1) or just see a named register using r, as in r @mxcsr. An alternative is to bring up the watches window (Alt+2) and enter a watch on a register, e.g. @mxcsr or just use the registers window (Alt+4). When you attach to a process the process is paused. You can continue execution with F5 or use the g command. And you can stop debugging with Shift+F5.

It quickly became apparent that a COM object written in C++ and compiled with Visual Studio 2003 was at the root of it. Some tedious and repetitive tracing later (let’s just say it was dark outside by the time we’d got to the nub of it) we had identified the root of the issue. Of course, immediately after doing using various manual techniques, it became very clear there may well have been better options, but so goes the process of learning a tool. I wondered whether trying to set a breakpoint that triggered when the register was modified might have helped, but I’m not sure offhand if that is possible in WinDbg, and have a suspicion that it isn't. Alternatively manually unmasking the overflow exceptions by setting bit 10 to 0 (so changing the default to 0x1A80) might have aided the quest. This would cause the exception to be immediately fired upon error, de-cloaking the culprit without so much dilly-dallying. Anyway, it’s easy to see better ways when the pressure is off… That said it may not have worked if VBScript, say, resets the MXCSR value to the real default. It seems a call to the maths function exp() was being passed data outside the permitted range (which is something along the lines of -708.3964 to 709.7827). The function executed an execution branch consisting of SSE2 instructions and it was ultimately a MULPD instruction (see the documentation in Volume 4 of AMD’s Architecture Programmer’s Manual) that caused the processor to set the overflow exception bit in MXCSR. The hardware exception was masked at this point, but the bit remained. Now as far as the COM object was concerned, this was neither here nor there. However, further on down the execution path the VBScript engine apparently unmasked the overflow exception bit in MXCSR and so the overflow exception reared its ugly head and showed up in the browser.

In the meantime, whilst this investigation had been going on it had been discovered that isolating the COM objects into the COM+ system also seemed to avoid the error popping up at all.

All of this begs some fairly obvious questions:

Why were SSE2 instructions being executed, when no processor specific optimisation option was used?
What was different on the AMD hardware?
Why did COM+ help?
What was the resolution?

It seems that the VS 2003 C++ RTL decides to use SSE2 features of the Intel CPUs by default for the floating point functions, if appropriate hardware is detected. This is regardless of the state of the compiler optimisation switches. However, you can instruct it not to do so by calling _set_SSE2_enable(0) in the startup code, such as in DllMain.

It turns out that the AMD hardware in use happens not to have the SSE2 extensions, which totally explains why the issue didn’t arise there. No SSE2 hardware means the code runs normal Intel opcodes, and so MULPD doesn’t cause the overflow that sets the overflow exception bit in MXCSR and the problem doesn’t percolate up to the browser.

The thing about COM+ is that COM objects installed into it are run, by default, in a separate worker process that instils a certain amount of control over it. When in COM+ the COM object will be resident in a separate process from the ASP VBScript code and interpreter. If we add on to this the fact that each thread in each process has its own copy of the full register set, which gets switched out to a context block any time thread switching occurs, we might understand why this helps. When the actual overflow occurs in the exp() function the bit gets set in the thread in the worker process. The VBScript code is running on a different thread in a different process, and so never sees the overflow bit having been set. It therefore doesn’t have any cause to report the overflow exception. Simple! Unfortunately there were technical reasons why all the suspect COM servers could not be switched into COM+, so this could not be used as a cheap and cheerful workaround, much to the team's chagrin.

As for the resolution, well I can’t say the problem has been completely resolved (at least to my satisfaction) as yet. Yes, calling _set_SSE2_enable(0) in each of the COM modules does indeed remove the manifestation of the error. However, this doesn’t change the fact bogus values would appear to be passed into the maths functions. Now as far as the documentation is concerned, if you pass in an out of range value you get either INF (infinite) for an overflow or 0 for an underflow passed back. I didn’t follow the logic through as it went through various complex paths, but maybe those out of range result values are okay in the grand scheme of the code.

My immediate thought was towards error handling. The C++ code has no error handling (in respect of inputs to these floating point RTL functions). It doesn’t filter the values on the way in and it has no C++ matherr handler. Your typical matherr handler looks something like this:

int _matherr( struct _exception *except )
{
  if( except->type == _OVERFLOW )
  {
    //failing function is except->name
    //offending values are except->arg1 and
    //    (if appropriate) except->arg2
    //you can return a sensible value using except->retval
    except->retval = 0;
    return 1; 
  }
  return 0; 
}

It is called whenever a floating point error occurs, and in this case responds to an overflow error, causing the failing function to return 0, which may or may not be suitable. As you can see there is scope for checking the name of the function that failed, and also examining the argument or arguments that cased the error.

But then again, maybe they aren’t needed if INF is catered for in the remainder of the logic. And as it turns out, adding in a matherr handler doesn’t really help the original problem much as it doesn’t automatically reset the MXCSR register bits. Indeed if you needed to fix the problem in a matherr handler you would need to include something like this in it:

long MMXDefaultStatus = 0x1f80;
__asm
{
  LDMXCSR MMXDefaultStatus
}

However that would only be wise to run if you knew for sure that the RTL was using the SSE2 instructions. If you wanted to use them where possible, you could call _set_SSE2_enable(1) in the startup code, and store the return value in a variable, which will be non-zero if SSE2 instructions will be used. This variable can be used to decide whether to reset MXCSR in the matherr handler.

Having pondered the issue for a bit, I decided the compiler was remiss in its behaviour inasmuch as its default settings allow a floating point overflow situation to set a status bit in a status register and then leave it set. Whilst at TechEd I bumped into two Visual C++ Program Managers: Steve Teixeira, an old friend from Borland days, and Ayman Shoukry. I described to them the problem of the compiler causing code to execute that does not reset the register status bit anywhere. They confirmed there were a number of deficiencies in the floating point support in Visual C++ 2003. They also suggested that things have improved markedly in Visual C++ 2005 just out of the box. Additionally there are new floating point compiler switches, including one to ensure the C++ code actually generates an exception immediately an overflow occurs in the floating point functions (the /fp:except option).

Currently the department in the bank hasn’t upgraded to Visual studio 2005, so these options cannot be tested “in the field” but it is good to know that things have been improved.

Pages

Ancient Blog Posts