# Thursday, August 09, 2007

The comments on yesterday's post on compatibility are pretty interesting.

There are effectively four options available when you find yourself a few versions into a platform and realize that you've made mistakes:

  1. Keep the mistakes, and potentially propogate those mistakes into other parts of the platform
  2. Rip out the mistakes, re-design the feature (sans mistakes) and accept the breaking change and broken apps
  3. Re-design the feature without the mistakes, add that re-designed feature to the product, deprecate the old broken feature, and accept duplication for some period of time until the older feature is ripped from the product
  4. Re-design the feature, add some elaborate compatibility mechanism to emulate the old behaviour, and sign up to maintain the elaborate compatibility mechanism for a long time

There may be some other options, but these are the ones that come to mind.

The comments from yesterday were in favour of (2).

I'm in favour of (3) as the primary model for making the product better, with a little bit of (2). I'm also willing to accept some of (1). I really dislike (4), as it doesn't benefit anyone in the long-term. (3) works well for individual APIs, but doesn't work well for whole sub-systems, such as the loader. If you change the loader in an incompatible way, you're really looking at options (2) or (4).

We've actually made a bunch of changes in Silverlight to improve the platform. As examples, we've removed non-generic collections, and have changed the way String comparisons work. We also changed the binder pretty significantly. Not to mention a new security system ... Oh, and we also included a stripped down XMLReader/Writer!

Compatibility is a tough balance relative to product evolution. We've been discussing this exact issue a lot over the last year, and the best way to evolve the project forward. The trick is coming up with a model that you really life, doesn't hugely impact the engineering process, doesn't inhibit product evolution going forward, and is explainable to customers. Taking submissions now ...

Thursday, August 09, 2007 6:45:33 AM (GMT Daylight Time, UTC+01:00)  #    Disclaimer  |  Comments [0]  | 
# Wednesday, August 08, 2007

I was just reading an Old New Thing blog entry about compatibility. I get the point that Raymond is making about compatibility. It reminds me of a lot of conversations that we have in building 42.

I talk to folks with the same proposal of pulling out the 'compat crap' to make the product better. I have two thoughts on that:

  • We'll be having this same conversation one release from now when we've implemented and shipped another set of designs that you don't like, when looked at in hind-sight
  • There are a ton of folks that have invested a ton of time and their livelihood into the apps that they've bet on our platform. What do you want me to tell them?

Compatibility is feature number 1. Without it, you don't have a product, at least for very long.

Wednesday, August 08, 2007 7:16:31 AM (GMT Daylight Time, UTC+01:00)  #    Disclaimer  |  Comments [3]  | 

I noticed that my last posts were about Vista 2005 SP1 on Vista. I had a bunch of trouble with that experience. I'm sure that the rough edges were polished out later on, but my initial experience could have been better.

I've been running Visual Studio 2008 (AKA 'Orcas') B1 and now B2 on Vista SP1 builds to write Silverlight v1.1 apps. I've got to say that the experience has been great. A bunch of work has been done to support UAC (the fundamental issue) by the VS and .NET Framework teams, and this on its own will be worth a big part of the price of admission.

I've also been building/debugging DasBlog (the blog software used to run this site) in VS 2008 B2 on Vista SP1 with IIS7. IIS7 is a pretty big change. After a few tweaks, help from the ASP.NET/IIS team and a bunch of help from the DasBlog team, I got it working. See Scott's post and post for more info.

There was only one problem in VS 2008 B2 w/IIS7 integration. That issue will be fixed in the next public release.

All in all, it's great to see a quality product, one that I'll be using a lot, coming from the VS team.

Wednesday, August 08, 2007 6:03:17 AM (GMT Daylight Time, UTC+01:00)  #    Disclaimer  |  Comments [0]  | 

I've been working on various aspects of Silverlight since sometime last year. The project has been going on much longer than that. It took a while for the managed API to come online and then even longer for good tools support, and all of this is before the product was announced. And all that time, you'd think that folks close to the product were generating a ton of samples and cool apps. Well, not really. There were some, but most folks (including me) were just cranking on getting the thing ready for Mix and then the refresh/RC that followed.

So, I'm now happily playing with the product. I'm in the middle of creating a photo gallery website for myself to replace the lame DHTML site that I wrote back in 2003 on ASP.NET v1.1.

I'm left with some random thoughts on the product and how it affects and empowers developers:

  • Compatibility between the server, the web-client (Silverlight) and the client (WPF/Winforms) is incredibly important
    • At the very least, developers benefit from knowledge transfer across the platform
    • Compatibility makes life easier for developers
    • More importantly, it opens up a ton of interesting scenarios between the different parts of the platform
  • A highly flexible and easy to target ActiveX control is liberating
    • I've written a bunch of javascript to improve user experience in the browser. I'd be happy to never do that again, given that the development experience is pretty bad.
    • You can create amazing visuals and immersize experiences that would never be possible with Javascript.
  • Silverlight also opens up a lot of questions for its use in the browser
    • Do you use Silverlight for your entire site, or only the parts that make sense?
    • What do you about navigation, particularly as it relates to forward and back buttons?
    • What do you do about folks that what a URL for a specific part of your site to send around, or return to later (bookmark)?
    • Flash has been around for a while, and has all of these same problems. Some sites use Flash for ads, others for widgets and still others that do the whole site thing. I've always disliked the whole site thing as a user.
  • Silverlight has an opportunity to hugely improve on the current Web experience
    • Most importantly, we've just let lose all those managed developers out on that ;)
    • This is the start of real internet applications, and of much more interesting web sites. Cannot wait to see what Amazon chooses to do.
    • This opens up a new x-browser (and OS) application platform to developers
    • Flash isn't always the best experience. It does crash and you sometimes get this dialog about 'waiting for a long-running script'.
  • Is this yet another Microsoft technology that developers have to learn?
    • Yes and no. If you know WPF already, then you'll be at home.
    • If you don't know the .NET Framework, then Silverlight is a great way to learn an impressive set of basics.
    • There are some differences to learn, but it's all minor
  • Is Microsoft serious about x-brower and x-OS?
    • Yes.
  • Are .NET Framework teams stoked about Silverlight?
    • Last time I checked ;)

As I work more on my photo gallery site, I'm pretty sure I'll develop more thoughts and hopefully some answers to some of the questions that I've raised.

Wednesday, August 08, 2007 5:46:30 AM (GMT Daylight Time, UTC+01:00)  #    Disclaimer  |  Comments [3]  | 
# Thursday, December 28, 2006

For those of you out there that are smarter than me ... and after yesterday's post, I'm thinking that there are a *lot* of you ... You probably noticed that I was a little confused between the effects of IE7 protected mode (Low Rights IE or LoRIE), the VS 2005 SP1 release and the debugging problems that I was experiencing on Vista.

I've known about IE7 protected mode, sandboxing and virtualization for many months now and was even cognizant of that when I was having my debugging problems. I even glanded down at the bottom bit of the IE chrome and noticed "Protected Mode: On", but for some reason I thought http://localhost would *not* be affected by the protected mode. I was dead wrong. Arghh.

So, the answer to the basic debugging problems that I was having w/IE7 on Vista are entirely solved by adding http://localhost as a trusted site in IE7. Remember to uncheck the "https://" checkbox when adding the site, or else you won't be able to add http://localhost.

There are a bunch of other blog posts on the subject if you are interested.

Thursday, December 28, 2006 10:37:57 PM (GMT Standard Time, UTC+00:00)  #    Disclaimer  |  Comments [0]  | 
# Wednesday, December 27, 2006

I decided to re-build dasblog (the blog software that I use) on Whidbey (VS 2005) over the holidays. It is something that I had always wanted to do and it seems like the dasblog folks have not yet publicly taken that on. Before anyone gets any strange ideas, I have no desire for a thinkjot-like schism, but wanted to move the codebase to Whidbey and work on getting the software to run under partial-trust. If this all works out, and the dasblog team wants to adopt my changes, cool, otherwise, I'm going to just use it myself.

Anyway, I decided to tackle this task over the holidays. Believe it or not, I almost exclusively use the free VS Express SKUs for my coding work. The "free" part isn't a big deal for me as I have a full copy of VS team suite a couple meters away from me, and I can download and install any program (including "Microsoft Bob") from the MS network that I want. Still, the Express SKUs are super convenient since I can download and install them in about 10mins and they satisfy most of my needs. Anyway, I downloaded the dasblog source and started playing with it. It became clear that I needed to use both VS express C# and VS express Web to get this done since the Web product didn't appear to support multiple projects in a single solution; in fact, it didn't appear to support the solution concept at all. And I then realized that since VS C# doesn't support JIT attach debugging (due to licensing issues), that the whole thing wasn't going to work at all. That's when I reached over for the quite large (requires two hands) VS team suite box to do some "real developlment" ;)

After a fairly lengthy install (and I didn't even install MSDN since I use the web mostly), I started back at it. I was able to get the web and class library projects into one solution in VS. Cool! I then hit F5 and it was immediately clear that something was terribly broken. The web project launched as expected, but was immediately detached from the debugger. Huh? I tried a couple more times, and I had the same experience. I was able to attach VS to WebDev.WebServer.Exe and then refresh the page, and then my breakpoints were hit. This approach though is anything but a good experience.

I had heard that there were some incompatibilities with VS on Vista, but I was under the impression that it was more niche issues, of which this is not. It is also very strange that I didn't have these same problems with the Express products, which I've been using on Vista for months. I wonder why the full product has some additional problems. I'm sure someone in building 41 knows.

Time to install VS 2005 SP1. I went to the following page. I downloaded the Vista-specific update. That didn't work, claiming that I was missing a file or two. I then downloaded and installed the non-Vista-specific SP1 package, which is just shy of 1/2 GB. Ouch! That worked. Upon launching VS, it claimed that I needed the Vista-specific update. Oh, I see, you need to install the generic VS 2005 service pack, and then the Vista-specific update. That was not at all clear to me from the VS 2005 SP1 page. Grrrr. Anyhow, now you can avoid the trouble that I had.

I then launch VS, but it claims that I need to launch the app elevated. I was actually expecting that, but thought that they would have manifested the application to force the elevation dialog. I guess not, or maybe that's still coming. Developers are going to go nuts if they have to remember to right click on the VS 2005 icon and hit "Run as administrator" every time, or just turn off UAC on their dev-boxes, which is a bad idea.

OK, launch VS again, but elevated, and voila, everything is working correctly again. Peace and harmony have now returned to my development experience.

I'm very thanksful that the VS team has pulled off this pretty significant service pack ... *before* Vista is generally available. I'm glad to be back and productive again. The directions on MSDN could use some improvement.

Wednesday, December 27, 2006 10:29:08 PM (GMT Standard Time, UTC+00:00)  #    Disclaimer  |  Comments [1]  | 
# Wednesday, December 20, 2006

I wrote a recent post about getting the list of loaded assemblies. That’s a pretty straight-forward operation as shown by the code listed in that post. I’d like to step up the problem one notch by looking at getting the list of assemblies from another domain. Seems like a very reasonable thing to do ... Or we’ll see …

To properly setup this example, I need to create another domain and keep a reference to that domain. That’s easy:

            //Create new domain

            AppDomain domain = AppDomain.CreateDomain("domain2");

 

Next, we need to load some code in that domain, or else there won’t be any assemblies loaded there to actually query. Note that Host is one of my types, as you’ll see later. The method below loads the DomainHost assembly in domain2, creates an instance of the DomainHost.Host type in domain2 and also returns a reference to that instance in the current domain. That’s why you see the cast to Host.

            Host host = (Host)domain.CreateInstanceAndUnwrap("DomainHost", "DomainHost.Host");

 

OK … this is the part that where it gets interesting. It is actually a little non-obvious on how you are intended to get the list of loaded assemblies from the other domains. Let’s first look at the most intuitive approach for doing this. There is the same GetAssemblies() method on AppDomain that I used in the last post.  So, given that I have a reference to the domain2 AppDomain instance, why not just call GetAssemblies() on it? This is what the code would look like:

            domain.GetAssemblies();

 

For a variety of reasons, assemblies are not marked as serializable objects, which means that they cannot be remoted across an appdomain (or any other) boundary. You can get around this limitation by remoting and then loading an assembly as a Byte[], but that’s a different blog post. This call gets around the limitation by remoting an array of representative AssemblyName objects, as opposed to the actual assemblies. The loader then does the equivalent of the following, assuming the AssemblyName[]is called asmNames:

            foreach (AssemblyName asmName in asmNames)

            {

                Assembly.Load(asmName);

            }

This behavior is not really expected or desirable. First, you really want the AssemblyName objects, not the Assembly objects, and you definitely do not want to cause the assemblies to be loaded in this domain. In addition, if the assemblies are not located in the GAC, and the other domain has a different AppBase, then it is very likely that the assembly loads will fail, resulting in the loader throwing an exception that you might not be prepared to catch. Ouch. That’s really bad. It is also important that not all assemblies play nicely with this behavior. For example, Reflection.Emit assemblies cannot be re-loaded cross domain, for fairly obvious reasons. Mixed-mode assemblies have another set of problems. So, let’s skip that option.

Another option is to call AppDomain.GetAssemblies() in the other domain, and then to remote the Assembly[] back through to the current domain.  That will have the same effect as what I just discussed above.

 I’ve skipped over some important details that I need to explain. My DomainHost.Host type is a regular old that type that happens to inherit from System.MarshalByRefObject. The fact that it inherits from this type is more of a marking than anything else, as I don’t override any of MBRO’s methods, nor do I have to know what they are. This is very conceptually similar to marking a type with the [Serializable] attribute. The difference is that MBRO provides copy-by-ref semantics as opposed to copy-by-value across a boundary, which is what the [Serializable] attribute provides. This means that MBRO-inherited types can be remoted across a domain, process or even machine boundary as a reference that you can easily call across. The method calls only affect the domain in which the type actually resides (not where the reference resides), with the exception that return types (which must be either MBRO or [Serializable]) are returned to where the call was made from (the current domain). The reference is called a transparent proxy, and you can see that in the debugger when you try to peer into what turns out to be the largely opaque System.Runtime.Remoting.Proxies.__TransparentProxy type. Woah! The fact that DomainHost.Host is MBRO is going to turn out quite useful, as we’ll see.

OK, now to the real solution. I need to add a method to DomainHost.Host that I can call that returns AssemblyName[] and that does not interact with the loader in any way on this side of the domain boundary, as I want to avoid any assembly loads on these AssemblyName objects. Let’s see what the method would look like:

        public AssemblyName[] GetAssemblyNames()

        {

            AssemblyName[] names;

            Assembly[] asms;

 

            asms = AppDomain.CurrentDomain.GetAssemblies();

            names = new AssemblyName[asms.Length];

 

            for (Int32 i = 0; i < asms.Length; i++ )

            {

                names[i] = asms[i].GetName();

            }

 

            return names;

        }

This method uses the AppDomain.GetAssemblies() method to get the list of loaded assemblies, but doesn’t return that. It creates an equal length AssemblyName[] array to the Assembly[] that it already has.  It then populates that array with the assembly name of each of the assemblies. After the assembly name array is populated, the method returns that array of serializable objects back to the caller, which happens to be on the other side of the appdomain boundary. And how does one call such a method on the other side of the appdomain boundary? Easy:

            AssemblyName[] asmNames = host.GetAssemblyNames();

 

This ease of cross-boundary calls is the beauty of .NET Remoting. You just call methods as if the instances were located directly beside you. Cool.

Well, that’s now about it. Let’s take a look at the whole program:

Main program:

using System;

using System.Reflection;

using DomainHost;

 

namespace GetAssemblyNamesFromDomain

{

    class Program

    {

        static void Main(string[] args)

        {

            AppDomain domain;

            Host host;

            AssemblyName[] asmNames;

 

            //Create new domain

            domain = AppDomain.CreateDomain("domain2");

//Load assembly in domain2 and create instance of DomainHost.Host

//A reference to the instance of DomainHost.Host is returned

            host = (Host)domain.CreateInstanceAndUnwrap("DomainHost", "DomainHost.Host");

 

            //Most obvious method for getting the list of loaded assemblies.

            //This method will cause the assemblies in the remote domain to

            //be loaded in this domain, which frequently won't work. Ouch!

            //domain.GetAssemblies();

 

            asmNames = host.GetAssemblyNames();

            Console.WriteLine();

            Console.WriteLine("Printing assemblies:");

            foreach (AssemblyName asmName in asmNames)

            {

                Console.WriteLine(asmName.FullName);

            }

 

        }

    }

}

 

DomainHost.Host (in DomainHost.dll)

using System;

using System.Reflection;

 

namespace DomainHost

{

    public class Host : MarshalByRefObject

    {

        public Host()

        {

            Console.WriteLine("Loading host in domain {0}",AppDomain.CurrentDomain.FriendlyName);

        }

 

        public AssemblyName[] GetAssemblyNames()

        {

            AssemblyName[] names;

            Assembly[] asms;

 

            asms = AppDomain.CurrentDomain.GetAssemblies();

            names = new AssemblyName[asms.Length];

 

            for (Int32 i = 0; i < asms.Length; i++ )

            {

                names[i] = asms[i].GetName();

            }

 

            return names;

        }

 

    }

}

 

The output of the program looks like:

Loading host in domain domain2

Printing assemblies:

mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089

Microsoft.VisualStudio.HostingProcess.Utilities, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a

DomainHost, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null

 

The fact that I see “Microsoft.VisualStudio.HostingProcess.Utilities” loaded is a VS weirdness. VS is loading an assembly in domain2 to somehow “help me”, but I don’t know why. This only occurs when I’m running/debugging my app through VS. If I run the program outside of VS, I don’t see that assembly loaded. You have noticed executables ending in vshost.exe. That’s a similar issue, but for another day.

Wednesday, December 20, 2006 8:40:10 PM (GMT Standard Time, UTC+00:00)  #    Disclaimer  |  Comments [0]  | 
# Thursday, December 07, 2006

A guy from a well-known company mailed me asking how to avoid deploying statically referenced assemblies that he knows will not be loaded at runtime. The issue is that the assembly is being loaded, but he doesn’t believe that it is being actually used. His current workaround is to deploy the assembly to avoid FileNotFound exceptions, which nobody likes.

I have two questions for the guy:

1.      Why are you so certain that the assembly will not be needed?

2.      Is the assembly used in other scenarios, just not this one? There must be a reason that it is statically referenced by your app in the first place.

 

As you can guess, this whole exercise is a bit of a dangerous situation. You can spend a lot of time ensuring that certain code paths will never be called, and then your users do something unexpected and low-and-behold, that darned FileNotFound exception is thrown. Ouch.

Why is this a problem?

Anyway, the core of the problem is at the level of the JIT (just-in-time compiler). The JIT jits code (MSIL) at a method-level basis.  The following is how things work at a high-level:

1.      The app calls a method

2.      The method cannot be executed because it hasn’t yet been jitted to machine code

3.      The JIT compiles the method (MSIL) into x86 (or X64 or IA64) code

4.      For every method call that the JIT sees, it must fully understand the signature of that method, specifically the return type and the arguments/parameters. This requirement may cause an assembly load.

5.      The method can now be called. The method will not need to be jitted again (at least in this app domain).

 

For example, the return type might be a value type, requiring the JIT to know how large that type is. To get that information, the JIT must load the type, and must request that the assembly (in which the type is contained) be loaded if it is not already loaded. This is still the case even if the method call is within an if statement, and might not actually be called.

Code that exposes the problem

As already suggested, you are going to run into this problem anytime you have a direct cross-assembly method call anywhere in a method you know that you will call, and hence JIT. This is even true under an if statement where the condition will not be true in this scenario.

        static void Method1()

        {

            if (condition)

            {

                //directly calling method from ClassLibrary2

                Class2.Multiply(100, 200);

            }

        }

 

Code that avoids this problem

It is pretty easy to avoid this problem. Write the following code instead.

        static void Method1()

        {

            if (condition)

            {

                //indirectly calling method from ClassLibrary2

                Method2();

            }

        }

        static void Method2()

        {

                //directly calling method from ClassLibrary2

                Class2.Multiply(100, 200);

        }

As you can see, the call to Class2.Multiply() is now an indirect call and does not require the JIT to know anything about the Class2 or require ClassLibrary2 to be loaded (the symptom to be avoided), unless of course the condition is met and Method2() is called. This trick is a little awkward sometimes, but it is a great option to at least defer and even completely avoid assembly loads.

Gotcha

Unfortunately, there is a gotcha. The JIT does optimize code by inlining methods. When this happens, you lose your indirect call, and you are now in the same bad boat you were in before. I don’t know much about the JIT inlining policy, so cannot provide a list of where this happens. This is merely a caveat that my workaround doesn’t work in all cases.  If folks are interested, I can talk to the folks on the JIT and perf teams to learn more about this gotcha.

Thursday, December 07, 2006 8:25:44 PM (GMT Standard Time, UTC+00:00)  #    Disclaimer  |  Comments [2]  | 
# Tuesday, December 05, 2006

We often talk about the loaded assemblies list over here. This is the list of assemblies currently loaded in the app domain. Big surprise. There isn't anything incredible special about this list. It is pretty easy to access and use. At the minimum, you can print out the list of assemblies. Beyond that, you can load and instantiate any of the types within those assemblies. That's where reflection comes in.

Here's some code that prints out the list of loaded assemblies:

using System;
using System.Reflection;

namespace GetLoadedAssemblies
{
class Program
{
static void Main(string[] args)
{
Assembly[] asms;

asms = AppDomain.CurrentDomain.GetAssemblies();

Console.WriteLine("There are {0} assemblies loaded:", asms.Length);
foreach (Assembly asm in asms)
{
Console.WriteLine(asm.FullName);
}

Console.WriteLine();
Console.WriteLine("The currently executing assembly is:");
Console.WriteLine(Assembly.GetExecutingAssembly().FullName);

}
}
}
Tuesday, December 05, 2006 11:51:00 PM (GMT Standard Time, UTC+00:00)  #    Disclaimer  |  Comments [0]  | 
# Wednesday, November 29, 2006

I took another look at the FileVersion sample. I wish the API was actually a little different. The API actually makes sense as a general use API, but it isn't as user-friendly as I would like. I wish that there were an instance method on the Assembly class called "GetFileVersion" or something like that it took nothing and returned a Version class.

Here is more of less what it would look like, except that the GetFileVersion wouldn't be static, it wouldn't take anything and would be on the assembly class.

If you look @ the FileVersion class, there is a lot of stuff on there, and it is a super wonky API anyway. I don't understand why it has a single static method that more or less acts that the instance constructor. Why not just have a constructor that takes a string or a FileInfo or even a FileStream. Bad API design. I prefer the Version class a lot more since it is super simple. I

using System;
using System.Reflection;
using System.Diagnostics;
namespace FileVersion
{
class Program
{
static void Main(string[] args)
{
Assembly asm;
Version ver;

asm = Assembly.Load("mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089");
ver = GetFileVersion(asm);
Console.WriteLine(ver.ToString());
}

static Version GetFileVersion(Assembly asm)
{
FileVersionInfo versionInfo;
Version ver;

versionInfo = FileVersionInfo.GetVersionInfo(asm.Location);
ver = new Version(versionInfo.FileMajorPart, versionInfo.FileMinorPart, versionInfo.FileBuildPart, versionInfo.FilePrivatePart);

return ver;
}
}
}

If you look @ the FileVersion class, there is a lot of stuff on there, and it is a super wonky API anyway. I don't understand why it has a single static method that more or less acts like an instance constructor. Why not just have a constructor that takes a string or a FileInfo or even a FileStream. Bad API design. I prefer the above method (for the assembly case) that returns the Version class since it is super simple. I realize that the native file version is a string, so can contain more stuff, but the 4-part version number is really all I want.

Wednesday, November 29, 2006 3:39:29 AM (GMT Standard Time, UTC+00:00)  #    Disclaimer  |  Comments [0]  |