CLR Hoser - Loader

What do I want for v2: Applications?

Rich Lander — Thu, 23 Oct 2008 17:40:49 GMT

I’m addressing the v2 question first. It is hard to know what to do in v1 if you don’t know where you want to end up when you go to build and deploy v2 (and later versions) of your product.

First, I'm going back to the three different product categories that I outlined earlier. I'm going to collapse the two different types of applications into just one category. Today's post is about applications. The next one will be about components.

Applications

For v2, you will want one of two situations:

my v2 application completely replaces my v1 version entirely, or
my v2 application installs side-by-side with v1 and the two versions do not affect or interact with one another

In most cases, I suspect that folks want the former. As an example, almost no one expects or wants to continue to use Office 2003 after installing Office 2007 on their machine. Something like TurboTax is the example that breaks that rule.

If you opt to install your application side-by-side with the older version of your application, then you might need to consider the safety of side-by-side versions of the various libraries that you depend on. The specific scenario that I’m imagining is that v2 of your application might be perfectly side-by-side with the v1 version, but the newer version of the MathLib library that you depend on might not be properly designed for a side-by-side install. Let's assume that you purchased MathLib from MathCorp, and that the versioning/servicing scheme of MathLib is different (naturally) than what you employ for the libraries that you build for your own application.

There are (at least) two obvious gotchas that I can imagine:

MathLib doesn’t install side-by-side, but always overwrites older library versions with newer versions, and the MathLib developers made a breaking change this release (oops!), or
MathLib reads mathematic constants from a file at a particular location, and the format of the file changed, but the location did not (oops!), which will break the old version

And to the other option, if you opt to replace the earlier version of your application, you don't risk breaking the earlier version of your app (since it is now gone), but you still have the risk that you might affect another app on the machine that relies on MathLib in the same ways described above.

What I’ve just described is essentially the well-known “dll hell” problem. I’d hope that most developers are familiar with this problem and have enough information to avoid it. However, it is sometimes non-obvious that you are about to walk off that cliff. This post is really intended to get folks to think a little more about that cliff.

This whole scenario assumes that MathLib is installed in some global fashion, either to the GAC, or to a global directory, such as "C:\Program Files\MathCorp\MathLib". If MathLib is installed with the application, in the application directory, then this whole problem goes away. At that point, you really are back to the monolithic application that I discussed earlier where the entirety of the application is installed and serviced uniformly, in time and location.

To be clear, I'm not trying to push developers toward private installs. There are downsides to approach that too. The meta-message is that you need to understand the potential impact of deploying your v2, and align your v2 goals around avoiding those impacts.

What's with the picture in the post? I recently purchased a digital SLR and am getting into photography as a hobby of sorts. The pic is just to add something extra to the post. I like this particular picture since there is a lot to look at visually, and because the focus is pretty narrow and well off center. It is also fun to see a child’s toy as an artifact to study.

Looking into Deployment – Shipping your v2

Rich Lander — Fri, 26 Sep 2008 16:57:56 GMT

Lately, I’ve been thinking about the overall .NET deployment and servicing story. There are a number of aspects of deployment that one can consider. The particular issues that are salient for your product depend a lot on the kind of code you deploy onto end-user machines.

First, let’s look at the different kinds of products that developers might build, and need to deploy:

- Monolithic applications

o A single exe that is not dependent on any libraries, other than the .NET Framework itself, or

o Solely dependent on libraries that are serviced/upgraded with the application

- Aggregate applications

o Dependent (at least partially) on libraries (or controls or components) that are installed/serviced/upgraded separately (in ‘time’ and/or ‘location’) from the application

o These libraries are likely sourced from a 3^rd party

- Components, controls or class libraries

o You build assemblies that other developers reference in their applications (both ‘Monolithic’ and ‘Aggregate’ applications as described above) or other libraries

While we’re thinking broadly, one is left wondering what the critical questions are to think about for each of the deployment cases above. There are a lot of questions that one can imagine looking into. They range from .NET Framework and application deployment, to servicing multiple versions of your applications out in the wild, to safely deploying a new version of your API (class libraries) onto end-user machines w/o any adverse impact.

The first one that I’d like to look at is: “Am I building my product correctly for the future?” This question, though, really boils down into two important questions:

- What do I need to think about when shipping v1 of my product?

- What do I need to do to safely, correctly and seamlessly ship v2 of my product?

Those questions seem very simple, and they are on the surface. The answers are not overly complicated, but do require some up-front thought to ensure that you end up where you want to.

We have had folks come to us after a successful v1, not quite sure what to do for v2. You may be wondering what the crux of this is … you just hit ‘go’ on csc/vbc another time and you’re done, right? Maybe. It all depends on how you want your code to behave when you ship your second version.

Note that I call out v2 specifically (and not v3 and v4) since v2 is the first version after your initial one, and will be the time when you need to address any issues that come up with shipping again. Once you’ve got a plan in place, you’ll be able to rinse and repeat for each subsequent version (provided that you have a good plan).

In my next posts, I’ll answer this question for each of the product types listed at the top of the post.

Getting the list of loaded assemblies from another App domain

Rich Lander — Wed, 20 Dec 2006 20:40:10 GMT

I wrote a recent post about getting the list of loaded assemblies. That’s a pretty straight-forward operation as shown by the code listed in that post. I’d like to step up the problem one notch by looking at getting the list of assemblies from another domain. Seems like a very reasonable thing to do ... Or we’ll see …

To properly setup this example, I need to create another domain and keep a reference to that domain. That’s easy:

//Create new domain

AppDomain domain = AppDomain.CreateDomain("domain2");

Next, we need to load some code in that domain, or else there won’t be any assemblies loaded there to actually query. Note that Host is one of my types, as you’ll see later. The method below loads the DomainHost assembly in domain2, creates an instance of the DomainHost.Host type in domain2 and also returns a reference to that instance in the current domain. That’s why you see the cast to Host.

Host host = (Host)domain.CreateInstanceAndUnwrap("DomainHost", "DomainHost.Host");

OK … this is the part that where it gets interesting. It is actually a little non-obvious on how you are intended to get the list of loaded assemblies from the other domains. Let’s first look at the most intuitive approach for doing this. There is the same GetAssemblies() method on AppDomain that I used in the last post. So, given that I have a reference to the domain2 AppDomain instance, why not just call GetAssemblies() on it? This is what the code would look like:

domain.GetAssemblies();

For a variety of reasons, assemblies are not marked as serializable objects, which means that they cannot be remoted across an appdomain (or any other) boundary. You can get around this limitation by remoting and then loading an assembly as a Byte[], but that’s a different blog post. This call gets around the limitation by remoting an array of representative AssemblyName objects, as opposed to the actual assemblies. The loader then does the equivalent of the following, assuming the AssemblyName[]is called asmNames:

foreach (AssemblyName asmName in asmNames)

{

Assembly.Load(asmName);

}

This behavior is not really expected or desirable. First, you really want the AssemblyName objects, not the Assembly objects, and you definitely do not want to cause the assemblies to be loaded in this domain. In addition, if the assemblies are not located in the GAC, and the other domain has a different AppBase, then it is very likely that the assembly loads will fail, resulting in the loader throwing an exception that you might not be prepared to catch. Ouch. That’s really bad. It is also important that not all assemblies play nicely with this behavior. For example, Reflection.Emit assemblies cannot be re-loaded cross domain, for fairly obvious reasons. Mixed-mode assemblies have another set of problems. So, let’s skip that option.

Another option is to call AppDomain.GetAssemblies() in the other domain, and then to remote the Assembly[] back through to the current domain. That will have the same effect as what I just discussed above.

I’ve skipped over some important details that I need to explain. My DomainHost.Host type is a regular old that type that happens to inherit from System.MarshalByRefObject. The fact that it inherits from this type is more of a marking than anything else, as I don’t override any of MBRO’s methods, nor do I have to know what they are. This is very conceptually similar to marking a type with the [Serializable] attribute. The difference is that MBRO provides copy-by-ref semantics as opposed to copy-by-value across a boundary, which is what the [Serializable] attribute provides. This means that MBRO-inherited types can be remoted across a domain, process or even machine boundary as a reference that you can easily call across. The method calls only affect the domain in which the type actually resides (not where the reference resides), with the exception that return types (which must be either MBRO or [Serializable]) are returned to where the call was made from (the current domain). The reference is called a transparent proxy, and you can see that in the debugger when you try to peer into what turns out to be the largely opaque System.Runtime.Remoting.Proxies.__TransparentProxy type. Woah! The fact that DomainHost.Host is MBRO is going to turn out quite useful, as we’ll see.

OK, now to the real solution. I need to add a method to DomainHost.Host that I can call that returns AssemblyName[] and that does not interact with the loader in any way on this side of the domain boundary, as I want to avoid any assembly loads on these AssemblyName objects. Let’s see what the method would look like:

public AssemblyName[] GetAssemblyNames()

{

AssemblyName[] names;

Assembly[] asms;

asms = AppDomain.CurrentDomain.GetAssemblies();

names = new AssemblyName[asms.Length];

for (Int32 i = 0; i < asms.Length; i++ )

{

names[i] = asms[i].GetName();

}

return names;

}

This method uses the AppDomain.GetAssemblies() method to get the list of loaded assemblies, but doesn’t return that. It creates an equal length AssemblyName[] array to the Assembly[] that it already has. It then populates that array with the assembly name of each of the assemblies. After the assembly name array is populated, the method returns that array of serializable objects back to the caller, which happens to be on the other side of the appdomain boundary. And how does one call such a method on the other side of the appdomain boundary? Easy:

AssemblyName[] asmNames = host.GetAssemblyNames();

This ease of cross-boundary calls is the beauty of .NET Remoting. You just call methods as if the instances were located directly beside you. Cool.

Well, that’s now about it. Let’s take a look at the whole program:

Main program:

using System;

using System.Reflection;

using DomainHost;

namespace GetAssemblyNamesFromDomain

{

class Program

{

static void Main(string[] args)

{

AppDomain domain;

Host host;

AssemblyName[] asmNames;

//Create new domain

domain = AppDomain.CreateDomain("domain2");

//Load assembly in domain2 and create instance of DomainHost.Host

//A reference to the instance of DomainHost.Host is returned

host = (Host)domain.CreateInstanceAndUnwrap("DomainHost", "DomainHost.Host");

//Most obvious method for getting the list of loaded assemblies.

//This method will cause the assemblies in the remote domain to

//be loaded in this domain, which frequently won't work. Ouch!

//domain.GetAssemblies();

asmNames = host.GetAssemblyNames();

Console.WriteLine();

Console.WriteLine("Printing assemblies:");

foreach (AssemblyName asmName in asmNames)

{

Console.WriteLine(asmName.FullName);

}

DomainHost.Host (in DomainHost.dll)

using System;

using System.Reflection;

namespace DomainHost

{

public class Host : MarshalByRefObject

{

public Host()

{

Console.WriteLine("Loading host in domain {0}",AppDomain.CurrentDomain.FriendlyName);

}

public AssemblyName[] GetAssemblyNames()

{

AssemblyName[] names;

Assembly[] asms;

asms = AppDomain.CurrentDomain.GetAssemblies();

names = new AssemblyName[asms.Length];

for (Int32 i = 0; i < asms.Length; i++ )

{

names[i] = asms[i].GetName();

}

return names;

}

The output of the program looks like:

Loading host in domain domain2

Printing assemblies:

mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089

Microsoft.VisualStudio.HostingProcess.Utilities, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a

DomainHost, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null

The fact that I see “Microsoft.VisualStudio.HostingProcess.Utilities” loaded is a VS weirdness. VS is loading an assembly in domain2 to somehow “help me”, but I don’t know why. This only occurs when I’m running/debugging my app through VS. If I run the program outside of VS, I don’t see that assembly loaded. You have noticed executables ending in vshost.exe. That’s a similar issue, but for another day.

How to avoid assembly loads

Rich Lander — Thu, 07 Dec 2006 20:25:44 GMT

A guy from a well-known company mailed me asking how to avoid deploying statically referenced assemblies that he knows will not be loaded at runtime. The issue is that the assembly is being loaded, but he doesn’t believe that it is being actually used. His current workaround is to deploy the assembly to avoid FileNotFound exceptions, which nobody likes.

I have two questions for the guy:

1. Why are you so certain that the assembly will not be needed?

2. Is the assembly used in other scenarios, just not this one? There must be a reason that it is statically referenced by your app in the first place.

As you can guess, this whole exercise is a bit of a dangerous situation. You can spend a lot of time ensuring that certain code paths will never be called, and then your users do something unexpected and low-and-behold, that darned FileNotFound exception is thrown. Ouch.

Why is this a problem?

Anyway, the core of the problem is at the level of the JIT (just-in-time compiler). The JIT jits code (MSIL) at a method-level basis. The following is how things work at a high-level:

1. The app calls a method

2. The method cannot be executed because it hasn’t yet been jitted to machine code

3. The JIT compiles the method (MSIL) into x86 (or X64 or IA64) code

4. For every method call that the JIT sees, it must fully understand the signature of that method, specifically the return type and the arguments/parameters. This requirement may cause an assembly load.

5. The method can now be called. The method will not need to be jitted again (at least in this app domain).

For example, the return type might be a value type, requiring the JIT to know how large that type is. To get that information, the JIT must load the type, and must request that the assembly (in which the type is contained) be loaded if it is not already loaded. This is still the case even if the method call is within an if statement, and might not actually be called.

Code that exposes the problem

As already suggested, you are going to run into this problem anytime you have a direct cross-assembly method call anywhere in a method you know that you will call, and hence JIT. This is even true under an if statement where the condition will not be true in this scenario.

static void Method1()

{

if (condition)

{

//directly calling method from ClassLibrary2

Class2.Multiply(100, 200);

}

Code that avoids this problem

It is pretty easy to avoid this problem. Write the following code instead.

static void Method1()

{

if (condition)

{

//indirectly calling method from ClassLibrary2

Method2();

}

static void Method2()

{

//directly calling method from ClassLibrary2

Class2.Multiply(100, 200);

}

As you can see, the call to Class2.Multiply() is now an indirect call and does not require the JIT to know anything about the Class2 or require ClassLibrary2 to be loaded (the symptom to be avoided), unless of course the condition is met and Method2() is called. This trick is a little awkward sometimes, but it is a great option to at least defer and even completely avoid assembly loads.

Gotcha

Unfortunately, there is a gotcha. The JIT does optimize code by inlining methods. When this happens, you lose your indirect call, and you are now in the same bad boat you were in before. I don’t know much about the JIT inlining policy, so cannot provide a list of where this happens. This is merely a caveat that my workaround doesn’t work in all cases. If folks are interested, I can talk to the folks on the JIT and perf teams to learn more about this gotcha.

Getting the list of loaded assemblies

Rich Lander — Tue, 05 Dec 2006 23:51:00 GMT

We often talk about the loaded assemblies list over here. This is the list of assemblies currently loaded in the app domain. Big surprise. There isn't anything incredible special about this list. It is pretty easy to access and use. At the minimum, you can print out the list of assemblies. Beyond that, you can load and instantiate any of the types within those assemblies. That's where reflection comes in.

Here's some code that prints out the list of loaded assemblies:

using System;

using System.Reflection;



namespace GetLoadedAssemblies

{

class Program

{

static void Main(string[]
args)

{

Assembly[] asms;



asms = AppDomain.CurrentDomain.GetAssemblies();



Console.WriteLine("There
are {0} assemblies loaded:", asms.Length);

foreach (Assembly
asm in asms)

{

Console.WriteLine(asm.FullName);

}



Console.WriteLine();

Console.WriteLine("The
currently executing assembly is:");

Console.WriteLine(Assembly.GetExecutingAssembly().FullName);



}

}

}

FileVersion Sample

Rich Lander — Wed, 29 Nov 2006 03:39:29 GMT

I took another look at the FileVersion sample. I wish the API was actually a little different. The API actually makes sense as a general use API, but it isn't as user-friendly as I would like. I wish that there were an instance method on the Assembly class called "GetFileVersion" or something like that it took nothing and returned a Version class.

Here is more of less what it would look like, except that the GetFileVersion wouldn't be static, it wouldn't take anything and would be on the assembly class.

If you look @ the FileVersion class, there is a lot of stuff on there, and it is a super wonky API anyway. I don't understand why it has a single static method that more or less acts that the instance constructor. Why not just have a constructor that takes a string or a FileInfo or even a FileStream. Bad API design. I prefer the Version class a lot more since it is super simple. I

using System;

using System.Reflection;

using System.Diagnostics;

namespace FileVersion

{

class Program

{

static void Main(string[]
args)

{

Assembly asm;

Version ver;



asm = Assembly.Load("mscorlib,
Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089");

ver = GetFileVersion(asm);

Console.WriteLine(ver.ToString());

}



static Version
GetFileVersion(Assembly asm)

{

FileVersionInfo versionInfo;

Version ver;



versionInfo = FileVersionInfo.GetVersionInfo(asm.Location);

ver = new Version(versionInfo.FileMajorPart,
versionInfo.FileMinorPart, versionInfo.FileBuildPart, versionInfo.FilePrivatePart);



return ver;

}

}

}

If you look @ the FileVersion class, there is a lot of stuff on there, and it is a super wonky API anyway. I don't understand why it has a single static method that more or less acts like an instance constructor. Why not just have a constructor that takes a string or a FileInfo or even a FileStream. Bad API design. I prefer the above method (for the assembly case) that returns the Version class since it is super simple. I realize that the native file version is a string, so can contain more stuff, but the 4-part version number is really all I want.

Native File Version

Rich Lander — Wed, 29 Nov 2006 00:00:36 GMT

The file version number is a native code concept – meaning not originating from the .NET Framework. This version number is a resource found within the resource section of the windows PE (portable executable) format of a managed or native code dll. This resource is named “FILEVERSION”. This version number is used for information purposes only, not for any runtime purposes such as binding. In addition, this version number does not have to conform to a particular format, but is only a string, although it does typically takes the form of a simple four-part number (i.e. 1.2.3.4).

Reading the File Version

The easiest way to view this number is to view the properties of a file in Windows Explorer. The version number listed is the file version number. The product version is also listed, although I don’t know how the two numbers differ exactly. Naturally, you can access the file version from code. The following code does just that, largely using the System.Diagnostics.FileVersionInfo class, which I’ve never used before. In fact, I had to ask someone else on the loader team for that information.

using System;

using System.Reflection;

using System.Diagnostics;

namespace FileVersion

{

class Program

{

static void Main(string[] args)

{

Assembly asm = Assembly.Load("mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089");

System.Diagnostics.FileVersionInfo fvi = FileVersionInfo.GetVersionInfo(asm.Location);

Console.WriteLine(fvi.FileVersion);

}

Setting the File Version

The CLR provides an assembly-level custom attribute to set this version number for an assembly from managed code. This attribute is called System.Reflection.AssemblyFileVersion. You can see how to set it below.

using System;

using System.Reflection;

[assembly:System.Reflection.AssemblyFileVersion("2.3.4.5")]

namespace ConsoleApplication1

{

class Program

{

static void Main(string[] args)

{

Console.WriteLine("I just set the file version!");

}

You can actually set this attribute in Visual Studio 2005 via the properties menu. In that case, you cannot set it in code, as I’ve done above, since you’ll then have two instances of the attribute. You only need to set the attribute directly, as I’ve done above, if you are using the compiler directly, from the commandline.

.NET Framework Versions Numbers

Rich Lander — Tue, 28 Nov 2006 23:53:35 GMT

Versioning and version numbers are always a bit confusing. For the CLR and the .NET Framework, we’ve got lots of version numbers to think about. I’d like to debunk any confusion around them, explain what each version number means, how to view it and how to set it (if appropriate). Let’s take a look …

The version numbers that I’m going to discuss are:

· Native file version

· Managed assembly version

· Metadata version

· Metadata format version

· .NET Framework versions

I’m going to discuss these version numbers (and anything else that comes up) across the next few posts.

Assembly Identity Concept

Rich Lander — Thu, 02 Nov 2006 23:12:05 GMT

I made reference to the concept of textual identity in my last post, but didn’t go into a lot of detail. In this post, I’d like to describe the broader concept of assembly identity to provide folks with further insight as to what I was going on about. The following text is from a spec that I wrote; it should apply equally to v1.0, v1.1 and v2.0 and future .NET Framework versions.

Assembly Identity

The assembly identity is the name of an assembly. The filename, path, file hash or other characteristics are not part of the identity. The identity is used in two different ways: (1) to define the name of an assembly, and (2) to reference an assembly by name. These are sometimes referred to as “def” and “ref”. Both of these are assembly identity. In the case of a reference, the identity is used during binding to determine if and where an assembly is available.

Identity Composition

The assembly identity is composed of several distinct attributes that detail different characteristics about an assembly. Each attribute is used in binding if it is provided. The allowed attributes follow:

· Simple name

o Format: string

o Description: The name is the simple name of the assembly. It is essentially the name without all the other attributes

o Note: The name should always be the same as filename minus extension

· Version

o Format: Four 16-bit integers separated by “.”

o Description: A four-part version number (Major.Minor.Build.Revision)

o Note: Each one of the 16-bit integers overflow at 165536 and underflow at -1

· PublicKeyToken or PublicKey

o Format: An 8-byte or variable length (48- to 2048-byte) string, respectively or “neutral” or “null”

o Description: The public key token or key specifies the cryptographic signature of an assembly, guaranteeing that the assembly is from a particular publisher or set of assemblies (with that same token or key)

o Note: The token is almost always provided instead of the much longer key

· Culture

o Format: string or “neutral” or “null”

o Description: An arbitrary string that represents a culture installed on the system

· ProcessorArchitecture

o Format: “MSIL” or “X86” or “X64” or “IA64”

o Description: The processor architecture (PA) attribute specifies the requirement of a particular platform to execute a particular assembly. “MSIL” is an agnostic PA, as “MSIL” assemblies are allowed to be executed on any processor. All other PA options are “bit-specific” and must be run on a specific platform.

o Note: PA doesn’t relate directly to processor or CPU. For example, X86 assemblies can be execute on X64 machines using the WoW64 infrastructure.

· Retargetable

o Format: “yes” or “no”

o Description: The retargetable attribute specifies that an assembly can be retargeted to another assembly, meaning a reference to another assembly can be retargeted to this one.

o Note: The retargeting mechanism is more complicated than described here, and is not at all a common scenario

Note: In all cases, the attributes and enumeration values are matched during binding case insensitively.

Textual Identity

The assembly identity can be specified in a string format, most often referred to as a “textual identity”. This form of the assembly identity is used by many APIs within the .NET Framework. There are also APIs that parse the textual identity string and return the identity back as a class, removing the need for developers to parse or create a textual identity string. More on that class later.

Textual Identity Specification

The specification of this format follows:

simple_name (“,” attribute “=” value)+

The textual identity starts with the simple name, and then a set of attributes and their values. Each attribute starts with a “,” and then its name, followed by a “=” and then a quoted or non-quoted value for that attribute. Whitespace can occur pretty much anywhere within the string, except within the attribute names, which are essentially tokens.

The allowed attributes, mapping directly to the components described in the previous section, are listed below:

Simple name à has no attribute name, requiring only its value
Version à “Version”
Culture à “Culture”
Public key or token à “PublicKey” or “PublicKeyToken”
Processor architecture à “ProcessorArchitecture”
Retargetable à “Retargetable”

The following characters can be escaped as part of the identity, with the escape character “\”:

’
t
r
n
\
=
‘
U(HexChar)

Textual Identity Error Cases

The textual identity parser will error in the following cases:

The textual identity doesn’t match the correct general format (“,” attribute “=” value)+
The textual identity includes attributes that are unknown (i.e. foo=bar)
There is a value that doesn’t match a member of a given enumeration (i.e. “powerpc”)
A part of any version number under- or over-flows the integer

AssemblyName Class

The .NET Framework includes a class called System.Reflection.AssemblyName. An AssemblyName takes the textual identity in its constructor, parses the string and then provides access to each attribute of the identity via handy properties. The constructor throws if the provided string is invalid according to the specification provided above.

The term “assembly name” is sometimes used interchangeably with “assembly identity”. It is generally best to think of “assembly name” as the AssemblyName class, and “assembly identity” as the boarder concept of the multi-part name of an assembly as described earlier.

Strong versus Partial versus Weak Names

Up until this point, the assembly identity has been described as if all the parts always have to be there. That is not the case. There are essentially three categories of names, depending on how much of the identity attributes are specified.

Strong name

· Simple name

· Version

· Culture

· PublicKey or PublicKeyToken

Weak name

· Simple name

Partial name

· A weak name, or

· Additional attributes beyond a weak name, but not enough to be considered a strong name

There are two other attributes: ProcessorArchitecture and Retargetable. ProcessorArchitecture is always optional, and just really makes a strong name more stronger. Retargetable is not commonly used.

Loading by Identity

Rich Lander — Thu, 26 Oct 2006 06:43:59 GMT

The CLR loader offers a few APIs for loading assemblies, each of which have a slightly different model and behaviour for how they go about the task. I'd like to cover those and hopefully set folks on the best path for using loading assemblies.

The APIs that I have in mind are:

Assembly.Load(), and
Assembly.LoadWithPartialName()

First, you need to understand the concepts of fully-specified and partial assembly names. A fully specified name contains the following parts: the simple name, version, culture, and public key or public key token. A partial name contains at least the simple name and optionally any of the other parts of the full-specified name. This concept is most relevant to the textual identity of an assembly. For example, here is the fully-specified assembly textual identity for v2.0 System.dll: "System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089". The textual identity is a specially formatted string that is used to describe some or all of the aspects of identifying an assembly.

Assembly.Load() takes a string as one of the overloads. The textual identity, as you see formatted above, is just the sort of string that Load() is expecting. Load() also takes other types, such as AssemblyName, but that's not important for the moment. The textual identity can be fully or partially specified, and which one it is has an important impact on the way that these loader APIs operate. The one you see above for System.dll is fully specified, since all the parts are there. Note that there is an additional part (or attribute) of the textual indentity that I've missed -- ProcessorArchitecture -- but that complicates the discussion too much for not much benefit, so I'm not going to cover it here.

The following bit of code shows the use of these APIs with full- and partially-specified names. The comments inline should pretty well explain what to expect

using System;

using System.Reflection;



namespace BindingByIdentity

{

class Program

{

static void Main(string[]
args)

{

Assembly asm;



//LoadWithPartialName()
looks in both the appbase and the GAC, looking for the best match,


//where
"best match" isn't very clearly defined


//In
addition, this method has been deprecated, which means "don't use this anymore, and
it may be removed later"


try


{

Console.WriteLine("Calling
LoadWithPartialName");

asm = Assembly.LoadWithPartialName("System");

Console.WriteLine("Loaded:
{0}",asm.FullName);

}

catch(Exception
e)

{

Console.WriteLine("LoadWithPartialName
threw with the following exception:");

Console.WriteLine(e.Message);

}





//Load()
with a partial name looks only in the appbase


//There
is no gaurantee that you will get the assembly that you want


//In
the case that you have private bin paths specified, there are no order gaurantees


//This
particular use of the Assembly.Load() will always throw, since this assembly will
not be found (because it is GAC'd)


try


{

Console.WriteLine("Calling
Load");

asm = Assembly.Load("System,
PublicKeyToken=b77a5c561934e089");

Console.WriteLine("Loaded:
{0}", asm.FullName);

}

catch(Exception
e)

{

Console.WriteLine("Load
threw with the following exception:");

Console.WriteLine(e.Message);

}



//Load()
with a fully-specified name looks in the appbase and the GAC and gaurantees that you


//will
get the assembly that you asked for


try


{

Console.WriteLine("Calling
Load");

asm = Assembly.Load("System,
Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089");

Console.WriteLine("Loaded:
{0}", asm.FullName);

}

catch (Exception
e)

{

Console.WriteLine("Load
threw with the following exception:");

Console.WriteLine(e.Message);

}





}

}

}

Output:

Calling LoadWithPartialName
Loaded: System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089
Calling Load
Load threw with the following exception:
Could not load file or assembly 'System, PublicKeyToken=b77a5c561934e089' or one
of its dependencies. The system cannot find the file specified.
Calling Load
Loaded: System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089

The moral of this story is that Assembly.LoadWithPartialName() is always bad and shouldn't be used since its behaviour isn't well-defined or predictable, and that you should always fully-specify the textual identity strings that you pass into Assembly.Load(). It may be the case that partially specified strings "work" since the assemblies you are attempted to load are always going to be in the app-base, but the fully-specified names do make your code and your intentions more clear, particularly for the next guy that has to look at the code you wrote. The funny thing is that sometimes "the next guy" is you, just two months later ;)

There is also the case that your code might not be strong-named. You should still specify everything but the publickey/token in that case.

Convention over Configuration as part of the CLR Loader

Rich Lander — Wed, 21 Jun 2006 20:30:11 GMT

I've heard "convention over configuration" a number of times, largely in reference to Ruby on Rails. It's actually a really interested concept. It is simple in nature, and those kind of concepts are always the best. As an aside, I think I'm done with complicated concepts at this point ;) The question is where does the CLR loader lie w/rt CoC and is that the correct place to be.

I can tell you right away that the loader and binder are way, way over on the side of configuration. And the default behaviour -- the behaviour you get before you need to get into configuration -- is only useful for basic scenarios. Let me offer an example if this is seeming a little too vague. So, you've got assembly A (A, Version=2.0.0.0) and then you want to service it, giving us A' (A, Version=2.0.0.1). That's all good and seems to make sense. It makes sense to increment the (managed) version number to differentiate the two versions of the assembly. The only problem is that apps bound to A (A, Version=2.0.0.0) will still load 2.0.0.0 and not 2.0.0.1 (assuming A and A' were in the GAC).

Hmmm ... where does that leave us, since that's not quite the behaviour (read:convention) that we wanted. Well, we get to configure. Here are the choices available to us: re-compile all apps out there with A', configure (app.config) all apps to bind to A' or generate and install publisher policy to centrally configure all apps to bind to A'. And there is one more option still available that differs from the rest: don't change the (managed) version number in the first place to better conform to the loader convention. Ughh!

Hmmm again ... none of that sounds too good. So, what does Microsoft do, since you'd think that they are going with the better option or maybe have some secret hidden option for their use alone. Well, we (for the most part) go with not changing the (managed) version number when we service our binaries, thereby conforming to the convention. Pretty crappy, eh?

The good news is that we're moving much more toward convention-based approaches for v3. There will still be plenty of configurability via XML, but the vast majority of cases should just fall out of convention. Yeahh!!

This sort of thing really does make you need to stop and think. Are the default behaviours correct? Do they cover enough of the common scenarios? Do they address those common scenarios deeply enough? There's nothing worse than software that covers a lot of scenarios, but only 50% deeply enough in each. 100% enough in each is where I'm headed.

Early Adopter Program for CLR v3 -- Who's in?

Rich Lander — Wed, 21 Jun 2006 00:38:12 GMT

I've been running an early adopter program for a couple months now for a bunch of changes that we're making to the CLR binder/loader in the CLR v3 timeframe. The changes are related to the binding model, versioning, servicing and an add-in model. Also, just to be clear, this isn't .NET Framework v3.0 (AKA WinFX). This is the one after that. Here is the blurb that I sent folks who I thought might be interested. Please @ mail me if you are interested in participating.

We have been busy designing some exciting new changes to the CLR loader/binder. We'd like to start collecting feedback from customers about our change early, in order to ensure that the final product is rock solid. The first phase of the program is to validate that our changes are useful and to understand the compatibility of these changes. In order to collect this information, we’d like you to run two tools on your system so that we can learn more about how you – actually the managed apps you use – use the loader and GAC. The one tool is a tool that collects very basic information about the GAC, while the second tool is actually an instrumented CLR which logs data about how the loader, binder and domains are used. The next phases involve running a prototype version of the CLR that includes the new changes, but we’ll get to that later, as it comes available.

There are also a couple legal documents to sign, specifically an MSFT NDA and a license. I'm happy to discuss those if you want to go to that stage.

GACUtil v2.0 -- Where to Get it

Rich Lander — Tue, 06 Dec 2005 06:43:16 GMT

I posted almost a month ago about all the searches that I've been noticing on my blog about gacutil.exe. I believe I've determined the question that at least some of the folks are wanting to ask. Folks don't know where to get it.

gacutil.exe and other tools such as ildasm.exe are part of the .NET Framework SDK. The .Net Framework admin tool, which used to be part of the redist, is now part of the SDK.

There may be some confusion around gacutil in particular. It has never shipped with the redist, but apparently a servicing package of the redist in the Everett (v1.1) timeframe used it as part of its servicing logic (oops) and didn't properly clean up after itself (double oops). As a result, there may be developers out there that have gotten used to gacutil being in the framework directory. If that's the case, we're sorry for giving the wrong impression. gacutil is intended to be a developer-only tool, used as part of development.

Instead of using gacutil at deployment time (as our not-so-smart servicing package did), you should use MSI or code against the fusion APIs yourself (with the former being the preferred method). For more info, you might take a look through Junfeng's blog.

Happy GACing.

The Wonders of Whidbey Factoring Features – Part III: Friend Assemblies

Rich Lander — Mon, 21 Nov 2005 18:46:14 GMT

We’re now at least half way into this set of posts on factoring features in Visual Studio 2005. The more I’ve thought about getting this set of posts fully written up, the more I’ve found these features interesting. My team produced them, so that helps, but I believe that a lot of developers will end up needing these features as a part of their versioning plan.

AKA InternalsVisibleTo

Friend assemblies are a nickname for the InternalsVisibleTo attribute in the System.Runtime.CompilerServices namespace. Not much of a stretch from the name, the use of the attribute allows you to specify which assemblies (your friends) should have access to your internal types and members. Just to remind you, “internal” != “private” – “internal” == “internal”.

As you might guess, this attribute must be set at the assembly-level. You might hope, however, that you could set this attribute at a more fine-grained level. I can explain that more later.

Syntax

The attribute is set in the following way:

[assembly:InternalsVisibleTo(“MyOtherAssembly, PublicKey=4asdsadasdsd”)

Couple things … The constructor of the attribute -- remember attributes are just types -- takes a textual assembly identity. Next, you must specify at least an assembly name and optionally a public key token. You can specify a culture as long as it is neutral. You cannot specify a version. Some of this may be a surprise to you. I’ll explain that below.

Design Decision Points

In retrospect, I’m not sure that making the public key token optional was the absolute best idea, but it was probably was necessary for a lot of interesting scenarios. First, friend assemblies is not a security feature, in the sense that visibility is not a security feature. Friend assemblies is just about changing the usual bounds of visibility to something that is more convenient for developers in a lot of cases.

That all being said, friend assemblies are really best kept to strong-named assemblies. Weak- or simple-named are very easily spoof-able (by definition). As a result, someone can simply use a particular name for their assembly and then get access to the internal fields in a simply-named assembly from which you’ve granted friendship. Even if you only ship your code internally within your business, this still isn’t a great plan. Think about it for a while and you’ll probably begin to better understand why.

It probably mostly goes without saying, but strong-named assemblies can only have strong-named friends. Strong-named assemblies can only depend on strong-named assemblies, so this point is moot anyway.

I do feel strongly that disallowing the version number was the right thing to do. The reason for that is that CLR binding doesn’t always produce the version of an assembly that you expect. There are a lot of mechanisms that can be at play, such as publisher policy, binding-redirects and other binding/versioning changes that we have planned for the CLR v3. As a result, you cannot assume that an assembly that depends on you is always going to be the same. Remember, with friends, it isn’t that you are assuming that a dependency of yours is going to be a particular version, but an assembly that is dependent on you. We would have potentially made a different decision if friends was the opposite way around, being a statement about your dependencies, but it isn’t.

The ability to specify culture provided that it is neutral is mostly a red-herring. There is no harm to specifying it, which is why we allowed it, but it is best to think in general that culture isn’t allowed. The main reason here is that you should not have code in culture-specific assemblies.

The Wonders of Whidbey Factoring Features – Part II: Type Forwarders in Detail

Rich Lander — Tue, 04 Oct 2005 03:18:21 GMT

It’s high time for the next installment in this series. I was planning on getting to it sooner, but I’ve had a bunch of crazy things going on to do with getting the .NET Framework integrated into Windows Vista and planning for the next couple versions of the .NET Framework. None of that is calming down, but it’s time to get out another installment none the less.

I took a quick and somewhat vague look at type forwarders in my previous post (on purpose). Some of you were likely satisfied by that level of information and others are likely interested in more detail. As the title of this post suggests, this is the place to get more detail. To best do this, I’ve created a simple – and not particularly useful – application to demonstrate how forwarders really work. I’ll include this app in my next post.

Here is the entire program (it is only the Main method):

using System;

using StringUtilities;

using MathUtilities;

namespace TypeForwarderApp

{

class Program

{

static void Main (string[] args)

{

Int32 int1, int2, intSum, stringCharacterCount;

String initialString, reversedString;

String appMessage;

appMessage = "This app doesn't really do anything useful";

Console.WriteLine(appMessage);

int1 = 16;

int2 = 14;

initialString = "Happy 30th Birthday Microsoft!";

intSum = MathUtil.Add(int1, int2);

reversedString = StringReverse.GetReversedString(initialString);

stringCharacterCount = StringCharacterCount.GetCount(initialString);

Console.WriteLine();

Console.WriteLine("Initial String : {0}", initialString);

Console.WriteLine("Reversed String: {0}", reversedString);

Console.WriteLine("String Length : {0}", stringCharacterCount);

Console.WriteLine();

Console.WriteLine("Integers: {0}, {1}", int1, int2);

Console.WriteLine("Sum : {0}", intSum);

Console.WriteLine();

Console.WriteLine(StringReverse.GetReversedString(appMessage));

}

Like, I said, the program doesn’t do anything useful. I included the birthday message given that two Fridays ago was the annual Microsoft company meeting at SafeCo Field in Seattle . The theme was “Beyond 30” celebrating the first 30 years of Microsoft, although the content was much more about the future than the past (which I imagine most folks there appreciated). The 25^th anniversary meeting was much more of a history lesson. I think that was the one that Sinbad was at ;)

I’m going to concentrate on one type, the StringCharacterCount type. At the moment, it is part of the StringUtilties assembly. If we look at the MSIL code within TypeForwarderApp.exe, we’ll see that clearly.

This is the MSIL code within the exe that calls the static method GetString on the StringCharacterCount type:

IL_0030: call int32 [StringUtilities]StringUtilities.StringCharacterCount::GetCount(string)

As you can see, the fact that StringCharacterCount lives within the StringUtilities assembly is pretty apparent. Actually, you could go as far as saying it is hard-coded. Hence the need for forwarders …

Well, I’m a typical developer and I’ve decided in my second version of StringUtilites and MathUtilites – I happen to own them both – that StringCharacterCount is really a math utility and less of a string utility. As a result, I’m going to move this cool type to the MathUtilities assembly.

I just moved it to the MathUtilties assembly. You can see that it is indeed part of the MathUtilities assembly via the MetaInfo (CTRL-M) view in ILDasm.

TypeDef #2 (02000003)

-------------------------------------------------------

TypDefName: StringUtilities.StringCharacterCount (02000003)

Flags : [Public] [AutoLayout] [Class] [AnsiClass] [BeforeFieldInit] (00100001)

Extends : 01000001 [TypeRef] System.Object

Method #1 (06000003)

-------------------------------------------------------

MethodName: GetCount (06000003)

Notice that the StringCharacterCount type is still part of the StringUtilities namespace and not MathUtilties. This is for two reasons: the namespace is part of the type name and type forwarders do not change the type name in any way. Put another way, if I moved StringCharacterCount from the one assembly to the other *and* changed its namespace, then I would have a breaking change that even type forwarders could not mitigate.

This all makes sense, but we have yet to see the interesting part. What does the metadata within the new version of StringUtilties look like? Let’s take a look at the manifest (another ILDasm view) within the StringUtilties assembly. There are actually two additional directives that we didn’t have before.

.assembly extern MathUtilities

{

.ver 1:0:0:0

}

.class extern forwarder StringUtilities.StringCharacterCount

{

.assembly extern MathUtilities

}

StringUtilties now has a dependency on MathUtilties and there is this “.class extern forwarder” line for “StringUtilities.StringCharacterCount”. Clearly, we’ve hit our jackpot. The dependency on MathUtilities is clearly required, given that StringUtilities intends to forward to it. The second directive, the class directive, is a way to signal to the runtime that the assembly knows about the class, but that it is located at another location.

If we look further yet, we’ll see that there is an entry in ExportedType table.

ExportedType #1 (27000001)

-------------------------------------------------------

Token: 0x27000001

Name: StringUtilities.StringCharacterCount

Implementation token: 0x23000002

TypeDef token: 0x00000000

Flags : [NotPublic] [AutoLayout] [Class] [AnsiClass] [Forwarder] (00200000)

We’re now looking at metadata instead of the MSIL view (the directives) above. The ExportedType table is generally used to publish all the types in child netmodules of an assembly – build a multi-module assembly and you’ll notice this first-hand. In the case of multi-module assemblies, the implementation token would point to the netmodule, not to a separate assembly. In the case of type forwarders, the implementation token points (naturally) to a separate assembly. We can easily prove that too. Look at the AssemblyRef entry for the MathUtilities Assembly below.

AssemblyRef #2 (23000002)

-------------------------------------------------------

Token: 0x23000002

Public Key or Token:

Name: MathUtilities

Version: 1.0.0.0

Major Version: 0x00000001

Minor Version: 0x00000000

Build Number: 0x00000000

Revision Number: 0x00000000

Locale: <null>

HashValue Blob:

Flags: [none] (00000000)

Notice that its token (0x23000002) matches the implementation token of our forwarded type (0x23000002). There you go. That’s it! Now you know everything I’m going to tell you about type forwarders.

The Wonders of Whidbey Factoring Features – Part I: Type Forwarders

Rich Lander — Wed, 14 Sep 2005 19:30:12 GMT

Type forwarders are an interesting feature. If you’ve never needed them, you’ve probably never thought about or imagined them. They were designed to allow developers to move types from one assembly to another without breaking existing code that was dependent on types living within a particular assembly. There are a bunch of reasons why you might want to move types around; however existing type references will make that problematic.

You may be thinking “I move types between assemblies all the time and nothing breaks”. That may be true if you are building applications that happen to have dependent assemblies. At the point that you build frameworks that other developers build apps on *and* you distribute those libraries as pre-compiled binaries (not as source) *and* you promote apps built against earlier versions of your framework to newer versions (say with publisher policy or binding re-directs) without re-compiling apps, then you can easily run into this problem.

The .NET Framework hits this scenario dead-on. Everett apps, when run on Whidbey (i.e. Whidbey-only machines), for example, run against the Whidbey .NET Framework libraries. That work without issue, but those Everett apps still expect types to be in the same place as they where in Everett , which in the case of System.String is mscorlib.dll. If we moved System.String to system.dll, for example, we’d break 100% of Everett apps run on Whidbey. Why? The CLR loader would no longer be able to resolve the type reference – [mscorlib]System.String -- stored in the app’s metadata, but instead would throw a System.TypeLoadException exception. The app wouldn’t like that and neither would it’s users ;)

Before anyone gets the wrong idea, we didn’t move System.String or any other existing types in Whidbey. I’m merely using the .NET Framework as an example of how framework developers could similarly run into this problem, which would be the precursor for needing type forwarders.

Enter type forwarders. Type forwarders are a new MSIL directive that essentially say “type x used to be in this assembly, but it is now in this other one. Maybe you should go look over there”. Let me just show you.

Create type t1 in assembly asm1
Create an app that uses [asm1]t1 (that’s just short-hand for saying type t1 that lives in assembly asm1)
Run app.
1. Loader loads asm1
2. Loader loads t1
3. Everything works
Close app and move back to VS 2005.
Move type t1 from asm1 to asm2 (these will be two different projects)
Create a forwarder in asm1 that points to [asm2]t1
Recompile asm1 and asm2. Do not recompile the app.
Run app
1. Loader loads asm1
2. Loader notices forwarder directive in asm1, pointing to asm2
3. Loader loads asm2
4. Loader loads t1 (this time for asm2)
5. Everything works

You do need to realize though that forwarders are really just a temporary crutch for older apps. Notice that we didn’t re-compile the app above. That’s actually the point of the whole scenario. We’re assuming that we don’t have the option of re-compiling the app, because we don’t own it – in this scenario, we own the framework, not the app. When the owner of the app does get the chance, we get the following list of activities:

Re-compile app against the latest version of asm2
Run new version of app
1. Loader loads asm2
2. Loader loads t1
3. Everything works

Notice that we no longer visit the forwarder in asm1. When the app owner re-compiled the app, the compiler found t1 in asm2. As a result, we no longer needed the mis-step in asm1. Like I said earlier, forwarders are merely a crutch for apps compiled against older versions of your framework. Once the apps are re-compiled against the new version of your framework, the forwarder is no longer needed (for that app). Unfortunately, you’ll need to keep that forwarder in place for some time, as there are likely a whole host of other apps and add-ins to those apps still reliant on the forwarder to work properly.

My next post will deal with the details of forwarders and I’ll post some source that uses the feature.

The Wonders of Whidbey Factoring Features – Part 0: Introduction

Rich Lander — Tue, 13 Sep 2005 17:17:13 GMT

My last “Wonders of Whidbey” series seemed to be quite well received, at least in terms of aggregator traffic, so I thought that I’d do another one. This time, I’d like to talk about the new factoring features in Whidbey. This topic has been bouncing around my head now for quite some time, so it was easy to choose. I also have the next WoW series picked out, but you’ll have to wait a little longer for that one.

I’d like to give you some insight into these features first. I’m a little vague on the birth (early 2003?) of these two features, since I was not on the CLR team yet, but working in another part of Microsoft at that time. I joined the team in November 2003. In early 2003, a subset of the CLR team (and a bunch of other folks from across the company) was working on some major improvements to managed code versioning. They had a particular take on the problematic parts of the current versioning system and had come up with quite an interesting solution. As part of that, the team was going to do some serious factoring (moving types between assemblies) of the .NET Framework, but that approach would have the negative effect of breaking pretty much 100% of existing applications since type references included the assembly identity. As a result, the team developed some cool factoring features in the CLR to mitigate those problems.

We’re no longer going down the same path with versioning as we thought we were back in 2003, but we’ve kept the factoring features in the product. We do have some other uses for these features, but they are not quite as far reaching as the earlier versioning plan. I think that Microsoft platform teams are realizing, more and more, that many of the engineering problems that they face are not unique to Microsoft, but are also faced by other software companies. That realization is actually the basis of the Visual Studio Team System products in a lot of ways. That’s a long way of saying that we’ve kept these factoring features in the product, believing that they will be useful to enough managed code developers out there. That being said, I don’t expect a lot of folks out there to use these features. In fact, I’m hoping that adoption of these features is left to niche scenarios, as I can imagine some folks painting themselves into some bad corners if they do not use them judicially.

The features in question are colloquially called “Friend Assemblies” and “Type Forwarders”. They have different names within the product, but I recommend using these names since they are accurate and easier for folks to grasp than InternalsVisibleTo and TypeForwardedTo. Learn more about them in the following posts in this series.

The Wonders of the Whidbey GAC -- Part IV: Shipping Everett and Whidbey Versions of the Same Code

Rich Lander — Wed, 24 Aug 2005 01:38:10 GMT

I mentioned earlier that a customer question started off this thread. This is the post that most closely aligns with their question. You can likely see why I've waited until now to answer it, since the earlier posts have all provided information on how the GACs work, either separately or in tandem.

This post isn't exclusive to the GAC, but we'll keep it in the same theme since we're on a roll here and since it does apply w/o too much of a stretch.

Ship both Everett and Whidbey Versions
This is the approach that will make your customers most happy. Everett customers have a library that they can code against. At this point, there are way more Everett customers out there than Whidbey ones, so providing an Everett version likely makes business sense in the short-term. At the same time, you recognize that Whidbey customers have different requirements. Folks who have been playing (or more than playing with our Beta 2 "Go Live" license) with Whidbey for a number of months now (or longer) have gotten used to the Whidbey way of doing things. A bunch of them probably now say "of T" at the end of all their sentences, even at home. Kinda like how I always say "eh" ;) In the same way that you want to keep existing customers on Everett happy, you want to ride the Whidbey adoption curve, picking up sales (if that's the kind of business you are in) from folks coding exclusively in VS2005 with a Whidbey version of your product.

This post isn't intended to cover exactly how you produce these two separate version. I should consider a post on this topic at a later time. The two basic approaches are: (A) have two separate source trees, or (B) a single one with lots of conditional code. There are advantages and disadvantages to both and you need to decide which path you will go down.

Versioning The Everett and Whidbey Versions
Finally, we're at the core of the customer question. Now that you've decided to ship the two versions in tandem, what do you do about version numbers and that sort of thing. Hmmm.

Scenario 1 - The Whidbey version is the same code as the Everett version, just compiled with a Whidbey compiler
- For starters, this is not a super-high-value proposition for class libraries. You definitely are doing your due diligence to avoid breaking changes, but that's it. Also, you can run your Everett code through the Whidbey compilers to do compatibility testing as part of your general testing.
- This is a high-value-proposition for applications (you get an .exe from the compiler). By compiling an Everett app with a Whidbey compiler, you enable running natively on a 64-bit machine. Everett apps always run under WoW64 on 64-bit machines. This post isn't about applications though, but class libraries.
- Anyway, if you decide to go this route, I would keep the version numbers of the Everett and Whidbey components exactly the same, but give the binaries different names (i.e. FooWidgetE.dll, FooWidgetW.dll). Since they are compiled against different CLRs, the code will end up in different GACs, which will further separate them.

Scenario 2 - The Whidbey and Everett versions are mostly the same code, compiled out of the same codebase with pre-processor directives
- First, I would only do this if the differences between the Everett and Whidbey versions -- which will directly affect the number of pre-processor directives you have in your code -- is not dramatic
- I'm assuming that the Everett and Whidbey versions have the same general object models, with a few members and types being specific to Whidbey
- I would do the same thing as Scenario 1 above with differently named assemblies
- If they are on radically different ship cycles, I might start to lean towards scenario 3

Scenario 3 -- The Whidbey and Everett versions are different code
- This approach will be the best one if you intend to move your object model in a different direction in the Whidbey codebase
- I would name the assemblies differently, more differently than in scenario 1 above
- I would not make any effort to keep the version numbers in sync. They might stay in sync, however, if you always ship the two versions on the same schedule
- I would probably change the namespace names too, to make it clear that this is a new thing. This is debatable though since it makes migration harder. The thing that this does do though is make it clear when a customer moves to the Whidbey version that they have more than just a re-compile on their hands

Your code may not fall perfectly into the above buckets, but you'll at least get the idea of some possible directions to take. As always, you need to come up with a plan that makes sense for your product, not just blindly follow a plan outlines on some blog, particularly one that is called "hoser".

The Wonders of the Whidbey GAC -- Part III: Binding Order Among the v1.x, v2.0 MSIL and v2.0 Bitness GACs

Rich Lander — Tue, 23 Aug 2005 02:47:45 GMT

In my two (one and two) earlier "Wonders of the Whidbey GAC" posts, I talked about processor architecture and interop with the Everett GAC. I left out one piece of interesting data because it wouldn't make as much sense until you had read the earlier posts. This data point is the binding order among the GACs. Naturally, you cannot look at them all at once, so we defined an order of probing the GACs that we always adhere to and tends to make sense.

Here is is:

Bitness GAC
- "gac_32" when running on a 32-bit machine or under WoW64 on a 64-bit machine
- "gac_64" when running natively (not WoW64) on a 64-bit machine (this is the normal case for Whidbey apps on a 64-bit machine)
Bitness agnostic GAC
- "gac_msil" on all machines
- assemblies in this GAC run equally well everywhere (X64, X86, IA64)
Legacy v1.x GAC
- "gac" on all machines
- this is where v1.0 and everett assemblies go

You can read this same information on Junfeng's blog too, as he implemented all of this. I just slow him down ;)

Junfeng referes to MSIL assemblies as "portable". I use the term "agnostic". It is the same thing.

If you write managed code in your favourite language (i.e. c#, vb) and don't do anything special, your assemblies will fall into this category. You can do the same thing with MC++ code too, if you compile it as /clrpure; otherwise, mc++ code is by design "bit-specific". Oh, there's another term, but I'm sure you can figure that one out.

If you are unsure of which of the three GAC buckets above your assembly will fall into, and your assembly is strong-name signed, install it into the GAC with gacutil /i from the Whidbey SDK. Gacutil will install the assembly into the correct GAC and then you'll know for sure. You can also open up the assembly (not required to be strong-name signed) in ildasm. That will also tell you the same answer.

The Wonders of the Whidbey GAC -- Part II: Interop with the Everett GAC

Rich Lander — Mon, 22 Aug 2005 04:39:12 GMT

I received a question from a customer a few days ago about assembly versioning. I receive questions quite often, of which >90% surprisingly enough come via Brad. I can only imagine what his inbox looks like every morning! Anyway, the question touched on the focus for this post which is the relationship between the Everett and Whidbey GACs. Before I go any further, I’m happy to report that the relationship is sensible and easy to understand. Too often, things don’t make a lot of sense and that always bugs me.

The customer’s question was actually about assembly versioning in the case that you want to support Everett and Whidbey, which I hope to cover tomorrow, since that’s when I told them I’d give them an answer ;)

First things first, on the surface it seems like interop between the GACs is what is wanted. In actual fact, it is the behaviour of the CLR loader and gacutil.exe that is really the interesting part. The GACs themselves are passive and don’t do anything other than act as a quite compelling directory structure for holding code w/o any naming conflicts.

Migrating the Everett GAC to the Whidbey GAC

We don’t do anything like this. This would have been the wrong direction for multiple reasons. Just thought I’d take care of this one up-front.

Hard linking assemblies between the Everett and Whidbey GACs as a Perfomance Enhancement

I’m glad that we didn’t do anything like this either. Ouch. OK, now onto the real answers.

Whidbey CLR Loader GAC Probing

The Whidbey CLR loader – Fusion in this case – probes for assemblies in the Everett and Whidbey GACs when an assembly request comes through the system. I’ll skip the exact algorithm and any optimizations that we might do. The high-order bit here is that we do indeed look in both GACs, so there is no need to stop using the Everett GAC for Everett assemblies since we look there. In fact, as you’ll find out below, you cannot stop using the Everett GAC.

Gacutil.exe, the Fusion APIs and installation via MSI (installers)

There are several ways that you can get assemblies installed into the GAC, with the last one being the recommended option. The big thing I wanted to know when I asked the question is what the expected behaviour was when I added an assembly to the GAC via these different methods.

Question:

Do the Whidbey tools put Everett assemblies into the Everett GAC?

Given:

Everett and Whidbey .NET Frameworks are installed on the machine

Assemblies:

1) 1 assembly compiled against Everett csc.exe (csharp compiler)

2) 1 assembly compiled against Whidbey vbc.exe (vb compiler)

Note: the compilers (vb, csharp) don’t matter for the assemblies above. I’m just trying to mix it up a little. It is the .Net Framework versions that matter.

Tools:

A) Everett Gacutil

B) Everett Fusion APIs

C) Whidbey Gacutil

D) Whidbey Fusion APIs

E) MSI installation

Scenarios + expected outcome:

1. (1) + (A) -- Everett GAC

2. (1) + (B) -- Everett GAC

3. (1) + (C) -- Everett GAC

4. (1) + (D) -- Everett GAC

5. (1) + (E) -- Everett GAC

6. (2) + (A) -- failure

7. (2) + (B) -- failure

8. (2) + (C) -- Whidbey GAC

9. (2) + (D) -- Whidbey GAC

10. (2) + (E) -- Whidbey GAC

Scenarions 6 and 7 fail since Whidbey-compiled assemblies are not loadable via the Everett CLR. Also, since processor architecture is a major part of Whidbey, it would be difficult to store Whidbey assemblies in the Everett GAC in a reasonable manner.

Why does Whidbey put Everett assemblies into the Everett GAC?

We continue putting Everett assemblies into the Everett GAC for two reasons:

- Everett apps still require those assemblies and will only be able to use them if they are in the Everett GAC since the Everett CLR loader knows nothing about the Whidbey GAC. It is much easier and smarter for the Whidbey loader to look in the Everett GAC.

- Everett assemblies didn’t have the concept of processor architecture, which the Whidbey GAC is heavily based on, so it doesn’t make sense to store Everett assemblies in that new structure.

- Everett is going to live for a long time. Let’s not mess with a good thing ;)

That was actually 3 reasons, if you count the last one.

Is there an Everett GAC if there is no Everett .NET Framework on the Machine, but only Whidbey?

Yes. The Everett GAC is created as needed when Everett is not on machine. This means that the Everett GAC structure is auto-created when the first Everett-era assembly is added to the Whidbey-only machine. If Everett was also on this machine, the Everett GAC would have been created at the point that the Everett .NET Framework was installed.

The Wonders of the Whidbey GAC -- Part I: Processor Architecture

Rich Lander — Sat, 20 Aug 2005 05:02:47 GMT

The GAC -- Global Assembly Cache -- is a popular topic among managed code developers. It serves two main purposes: (A) as a catalogue of the globally publicly available assemblies available on the system, indexed by strong name, and (B) the location of those same files. It is not my intent to describe the GAC, however; look here for more info. My intent is to describe some interesting changes that have been made to the GAC for Whidbey and to promote some good practices. This is part I and is dedicated to processor architecture; we'll see how far we get with the series ;)

Whidbey is the first CLR release in which we support 64-bit execution. The CLR team has been working on porting the CLR to 64-bit for several years now and is now releasing it to the world with Whidbey. Woohoo! Not surprisingly, 64-bit has required a lot of changes all over the CLR, as it imports a bunch of new concepts onto us. My favourite is the general concept of processor architecture, of which there are four that we now support: X86, X64, IA64 and MSIL. We always supported the first and the last, but there was never a strong reason to differentiate them considerably as we only ran on a 32-bit system.

With respect to the v1.x GAC (v1.0 and Everett share the same GAC), you can see pretty clearly that processor architecture was not a design point.

V1.x GAC location
C:\Windows\assembly\GAC

V1.x System.dll location
C:\WINDOWS\assembly\GAC\System\1.0.5000.0__b77a5c561934e089\System.dll

Where would you have tagged the processor architecture? Nowhere that I can see. As a result, we changed the GAC structure in Whidbey to accommodate processor architecture, which is now an incredible important aspect of the GAC.

V2.0 GAC locations on an x86 machine
C:\Windows\assembly\GAC_MSIL
C:\Windows\assembly\GAC_32

V2.0 System.dll and mscorlib.dll locations on an x86 machine
C:\WINDOWS\assembly\GAC_MSIL\System\2.0.0.0__b77a5c561934e089\System.dll
C:\WINDOWS\assembly\GAC_32\mscorlib\2.0.0.0__b77a5c561934e089\mscorlib.dll

V2.0 GAC locations on an x64 or IA64 machine
C:\Windows\assembly\GAC_MSIL
C:\Windows\assembly\GAC_32
C:\Windows\assembly\GAC_64

V2.0 System.dll and mscorlib.dll locations on an x64 machine
C:\WINDOWS\assembly\GAC_MSIL\System\2.0.0.0__b77a5c561934e089\System.dll
C:\WINDOWS\assembly\GAC_32\mscorlib\2.0.0.0__b77a5c561934e089\mscorlib.dll
C:\WINDOWS\assembly\GAC_64\mscorlib\2.0.0.0__b77a5c561934e089\mscorlib.dll

You can now see that it is easy to tag assemblies with processor architecture. We’ve created three GACs to do just that! As you can see, the GAC_64 directory only shows up on 64-bit machines, which is hopefully quite intuitive.

I’ve run out of time to get any further in this post, but hopefully it gives you a good idea of how and why the GAC changed, at least with respect to processor architecture in Whidbey. I also hope I left you with the opinion that these changes were a good idea ;)

A question to ponder: If you call System.Reflection.Assembly.Load(“System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089”) and there is both a copy in the bit-specific GAC and another in the bit-agnostic (MSIL) GAC, which do you get?

Coming to you in 64 bits

Rich Lander — Thu, 19 May 2005 19:35:01 GMT

Some time ago, I moved my blog to a 64-bit X64 server. This server was running Windows Server 2003 SP1 Beta 1 and an early version of Whidbey Beta 2. I thought that this was an excellent opportunity to dogfood (MS term for early adoption) the 64-bit version of the framework, not to mention Server 2003 SP1. Another side aspect of this exercise was that I was testing out the backwards compatibility of the .NET Framework, since DasBlog is built with the v1.1 version of the Framework.

I'm happy to report that I've had zero problems with this new configuration, even with beta software. The OS has now shipped, so there is no issue there, but the version of Whidbey is still only Beta 2 and it is chugging day after day without issue. Doesn't get better than that! It also shows the incredible value of MSIL -- what C#, VB.Net, Eiffel, IronPython, Cobol and any other managed code compiler compiles down to -- given that the Everett DasBlog code "just works" on 64-bit Whidbey, even though 64-bit was not a supported (or possible) configuration of Everett.

One of the major features of Whidbey ASP.Net is XHTML compliance. DasBlog was not designed to be 100% XHTML compliant, so would not work on the first try on Whidbey. We had to use a special config switch to turn off XHTML compliance in Whidbey. After that, everything worked perfectly, in terms of performance, backwards compatibility and general behaviour of the app. I'm very happy and can say that Whidbey Beta 2 is ready for prime-time development. To that end, the .NET Framework is allowing companies to "go live" with our "go live" license. This means that you can build/migrate a public-facing application with Whidbey Beta 2.

Back to 64-bit ... I must say that the 64-bit support in Whidbey is very cool. I have two high-end X64 boxes in my office, one from Intel and the other from AMD. I use them for various purposes and am quite happy with them. In particular, I really like building and debugging apps on my 64-bit machines, having them run in native 64-bit, but then being able to deploy them to both 32- and 64-bit machines. The portability of managed code assemblies across different CPU architectures (IA64, X64, X86) is a huge win.

I've also been running early 64-bit versions of Longhorn on these two X64 machines. I haven't done any tests using >2GB of virtual address space, but I would like to see a managed app that does that, taking full advantage of the 64-bit address space. I can imagine that companies running SQL Server will immediately see the benefit of that removed barrier.

LoadFromContext MDA in VS 2005

Rich Lander — Thu, 12 May 2005 01:27:53 GMT

Managed Debugging Assistants are an existing feature (Customer Debug Probes) improved on and expanded. They are exposed in a similar manner to exceptions (don't let that scare you) but occur not because of an error condition, but because the runtime has noticed a condition that could cause you problems. We've talked to lots of customers over the years and have received feedback that there are a set of scenarios that cause developers pain more often than others. As a result, we've written a bunch of code -- the MDAs themselves -- that can be used while your code is running to protect you against these pain points ... that's good, right?

The MDA infrastructure is always "on", but only a few MDAs are activated by default. That's easily changed, by selecting the "Debug" menu and "Exceptions" within it. From there, you can turn on or off any MDA of your choosing, as you can see in the image below. There is only one MDA turned on in that part of list in the image below, but you turn on as many as you want.

Anyway, I want to introduce you to the LoadFromContext MDA. If you want to know more about the joys of LoadFrom, check out Suzanne's blog. I'll post more on binding contexts at some point, but want to stay on the subject of this MDA for the moment. Here is some code that exercises this MDA -- this is just an exe project attempting to load a built assembly from a class library project in the same solution (hence the strange pathing) ...

using System;
using System.Reflection;

namespace LoadFromMDA
{
class Program
{
static void Main(string[] args)
{
Assembly asm = Assembly.LoadFrom(@"..\..\..\SomeAssembly\bin\debug\SomeAssembly.dll");
Console.WriteLine(asm.FullName);
}
}
}

If the MDA is enabled, I get the following exception-looking popup window as soon as hit the first line of code.

The reason that this MDA is so useful is that it is really quite difficult to determine for certain which binding context an assembly was loading into, other than following a certain set of rules and closely watching the way that the binder binds to dependent assemblies. However, with this MDA, you can know for sure if you are loading assemblies into the LoadFrom context.

There are several other MDAs to play with. I recommend taking a look through them and turning on ones that might apply to your program. There is no harm and you may avoid your program throwing an exception in some edge-case scenarios for reasons that may not be super clear to you.

.NET Framework Assembly Version Number for Whidbey

Rich Lander — Mon, 09 May 2005 05:36:31 GMT

Brad sent me mail on Friday night asking about our plans (or non-plans) to change the assembly version number again for Whidbey. The mail was the result of a developer -- let's call him Marc -- who had pinged Brad earlier in the week through his blog asking that exact question ...

Here is the basic question from Marc:

I'm using Beta 2 of the .NET Framework SDK 2.0 ... I'm curious, if I roll this app out, built on a beta ... Would the app continue to function or would it need to be rebuilt with the 2.0 final SDK?

There's too many exciting features to stick with the 1.x version of .NET. :-) Every day I find something new and exciting that can be done. It makes programming fun again!

I'm glad to hear that Marc likes Whidbey. That's definitely the plan from building 42 (where the CLR team works)! Back to the original question: will Marc's app continue to work or is a re-compile required? Marc isn't specific about which beta he is using, so I'm going to have to resort to a multi-part answer, unfortunately ...

1. If Marc is on Whidbey Beta 1 (or earlier), then the answer is "yes ... a recompilation is in your future".

2. If Marc is on Whidbey Beta 2 (or later), then the answer is "no ... your app should work just fine".

W/rt (2), I do recommend re-compiling against the final product even though it isn't strictly required (from a technical perspective). Naturally, there will have been further changes/fixes in the compilers and the metadata sub-system that will very likely result in a better built assembly.

Some of you might wonder what this is all about. I'll attempt to explain w/o going on any tangents. Please email if there are some tangents that you are interested in ;)

We set the assembly version number of each of the assemblies (all the same version fortunately) in the .NET Framework at major milestones, treating it as an contract between the .NET Framework and it's consumers. We then break that contract by changing the assembly version number again when we start a new milestone (i.e. Beta 1 versus Beta 2; Everett versus Whidbey). Why do we do that? We change the assembly version number for two reasons: (A) the contract has changed since the code inside the assemblies is very substantially different, and (B) a strong identifier (i.e. a version number) needs to be changed to remain a strong identifier.

We changed the assembly version number last early in the Beta 2 milestone, for the reasons described above. We will not change it again in the Whidbey product since the rate of changes to the product has slowed down and because we only want to make changes at this point that are absolutely required to ship the product.

As a result of how we've changed the assembly number and since the assembly binder binds strictly on the whole version number, apps built with Whidbey Beta 2 will continue to reference Whidbey RTM .NET Framework assemblies since their version number will be the same as what the Whidbey Beta 2 app is expecting.

Hope that helps Marc!

Getting Roped Into Reflection

Rich Lander — Sat, 15 Jan 2005 05:10:21 GMT

I decided to whip up a quick app today to test some behaviour of Assembly.LoadFrom() that I wasn't 100% certain about. In particular, I was curious about the dependency resolution behaviour of Assembly.LoadFrom(), as compared to Assembly.Load(). I know that Load() will find static dependencies for you, provided that the dependencies are in a place that the loader looks (i.e. GAC, appbase, private bin paths). I also know that Assembly.LoadFile() doesn't do any dependency resolution for you; you have to pre-load dependencies when you use LoadFile(). So, the test is to determine which camp -- dependency resolution or not -- that LoadFrom falls into. I'm pretty sure that it will fall into the dependency resolution camp, but I've never explicitly tested this case in issolation, so I'm checking it out. I've also written a bunch of apps that depended on LoadFrom resolving dependencies for you, so this likely isn't going to end up being a big surprise.

An Exercise in Reflection

So, I have to admit up front that I'm not great at using Reflection -- another bloke works on that. Anyway, I had to brush up on my reflection skills to get this working and ask Joel if I'd done it properly once I was done, which turned out to be mostly the case.

Here's the main program:

        static void Main (string[] args)
        {
            Assembly a;
            Type t;
            MethodInfo m;
            Object[] myArgs;
            Object myInt;

            Console.WriteLine("In main program");

            //Loading assembly
            //a = Assembly.LoadFile(@"C:\Documents and Settings\rlander\My Documents\Visual Studio 2005\Projects\LoadFromApp\FooLibrary\bin\debug\FooLibrary.dll");
            a = Assembly.LoadFrom(@"../../../FooLibrary/bin/debug/FooLibrary.dll");

            //Getting type to call into
            t = a.GetType("FooLibrary.Foo");

            //Getting the specific method that I'm interested in
            m = t.GetMethod("GetInt", BindingFlags.Static | BindingFlags.Public, null, new Type[] { typeof(System.Int32)},null);

            //args to pass to the method
            myArgs = new Object[1] {5};

            try
            {
                //Calling the method and printing the return value
                myInt = m.Invoke(null, myArgs);
                Console.WriteLine(myInt.ToString());
            }
                //in case something bad happens
            catch (TargetInvocationException e)
            {
                Console.WriteLine(e.InnerException.ToString());
            }
        }

Indeed, there is more to this program than just the Main method, however, the additional aspects are not particularly interesting. The code above is the whole of a Console exe project, called LoadFromApp, in VS 2005. There are two other projects that I wrote too: (1) a Class Library project called FooLibrary that the exe loads via Assembly.LoadFrom(), and (2) another Class Library project called BarLibrary to which FooLibrary statically links. You can see these too/two in the code sample, which is linked at the bottom of the entry.

The expected behaviour, now that you have a mental model of my project, is that BarLibrary is automatically loaded by the CLR loader when I load FooLibrary from LoadFromApp via Assembly.LoadFrom(). BarLibrary is necessarily in the same directory as FooLibrary; otherwise, there would be no way that LoadFrom() could find BarLibrary, unless BarLibrary was in the GAC or something like that. This is why VS adds referenced assemblies into your bin/debug (or release) folder when you build an assembly.

The interesting thing is that this exercise turned out to be more of a lesson in reflection than in the loader, but that's how these things end up. I'll tell you what I learned about LoadFrom and then reiterate what Joel told me about reflection.

What I Learned

First, the loader part. So, it does indeed turn out to be true that Assembly.LoadFrom() does resolve dependencies, provided that there are in a findable place, which turns out to be any place that would be consulted for Assembly.Load() and the path (minus filename.dll) of the Assembly.LoadFrom(String path). That wasn't too hard, but was worth the exercise.

You'll also notice that I commented out Assembly.LoadFile(), just above Assembly.LoadFrom(). I tried this too for kicks, which resulted in an exception of type System.Reflection.TargetInvocationException. That's the “something bad happens” and what the exception handler is for above. So, this proves that LoadFile() does not resolve depencies, which I already knew, but was a good test anyway.

Now, onto the reflection aspect. The comments in the source above mostly explain what is going on, I hope. If not, I load an assembly, get a known type from that assembly, then get a known method from that type and then call through on that method with known data. Do remember/realize that this isn’t normally what .net code looks like. Reflection is one of the mechanisms of handling late bound scenarios for .net coders. It is probably the most flexible, but not the most approachable.

Back to the interesting review of reflection … The line that I'm most interested in is the call to t.GetMethod, which is the instance method MethodInfo.GetMethod. I'll re-print it here:

//Getting the specific method that I'm interested in
m = t.GetMethod("GetInt", BindingFlags.Static | BindingFlags.Public, null, new Type[] { typeof(System.Int32)},null);

The GetMethod method allows you to specify enough information for the reflection system to get a single method for you from a given type. The thing is that you'll get some type of exception if you haven't specified enough information to single out your desired method from the set of methods on the type. Put another way, you’ll get this exception if the reflection system ends up finding >1 methods that fit your description since the method only returns MethodInfo, not MethodInfo[]. GetMethods() returns MethodInfo[] and obviously doesn’t have this particular problem. So, you need to be pretty specific about the method that you want. It may not be neccesary this version, but you may add overloads next version and forget that your GetMethod call was pretty sloppy and then stuff breaks, which is bad.

It is the first two and fourth parameters that are the most interesting to me. The first one is the method name. So, this differentiates the method from all other methods. You can make the search case insensitive via the BindingFlags, but I would not recommend this approach since the CLR doesn’t work this way and it will slow down the whole operation. The next parameter is the BindingFlags. I've stated that the method I'm looking for is static and public. So, the search will not retreive any instance or internal/protected/private methods that happen to have the same name, which is quite possible. You’ll need to pick among these flags carefully to ensure that you have the correct level of specificity. The fourth parameter specifies the signature of the method. Rememer, the CLR doesn't consider the return type when determining which overload of a method to call, so the signature is just the types of the parameters that the method takes. At this point, you will have provided enough data that the reflection system can narrow its search of methods on the given time to 1, or 0 if it is not there.

The actual code sample that I wrote is on another machine. I'll post it in a day or two when I get a chance. The sample is nothing big, but you'll be able to see how LoadFrom() and LoadFile() work for yourself with my small contrived example.