I recently posted twice (one and two) on anonymous delegates. I guess they really caught my attention for some reason, even though I've known about them for a long time. The remaining question to close out this debate is whether anonymous delegates are the .Net Framework answer to closures. Another related topic is continuations. I'm certain that these are not that.

First, what is a closure?

Here is how wikipedia defines closures:  "Unlike garden-variety functions which retain no memory of what happened in previous calls, closures are capable of storing information across function calls."

Here is another explanation from guile, which is an open source scheme interpreter that I've never heard of before: "The concept of closure is the idea that a lambda expression "captures" the variable bindings that are in lexical scope at the point where the lambda expression occurs. The procedure created by the lambda expression can refer to and mutate the captured bindings, and the values of those bindings persist between procedure calls. "

My definition is (as of 5 mins ago): a closure is the outcome of capturing both the in-scope variables used by a block of code and the block of code itself to the end of preserving their relationship beyond the normal lifetime of those in-scope local variables. As a result, the local variables can be shared across all calls to the block of code (i.e. method). 

So, do anonymous delegates in C# v2.0 conform to the definition of lexical closures? That seems to be a matter of great debate. It is amazing when you start to look at a topic and then you realize that there has been a whole crowd to the party before you. Anonymous delegates in C# appear to be just such a topic. Brad appears to have been interested in the topic about a year ago. Brad takes the stance (at least at that time) that anonymous delegates are not closures. abhinaba also takes the same stance. Dan Muller takes the opposite opinion, believing that C# has implemented closures.

I've come to the conclusion that anonymous delegates are closures. If you look at my somewhat-in-depth explanation of anonymous delegates in my last post, you'll see that the way they work seems to conform to the three definitions above. In summary, the C# compiler generates a class on your behalf which contains both an instance method (which is your anonymous delegate) and fields that map directly to the in-scope variables that you access in the containing method. All access to these variables are wired up to these generated fields on the class. That generated class is in itself the closure. Being heap-allocated, it naturally can have a different lifetime than what the containing method has on the stack, given that stack-frames have a tendency to go way.

I have a mail out to the C# team asking them their opinion on this interesting subject. I'm curious to see how they'll respond.

Here is a better example of using an anonymous delegate that proves, in my mind, that they are lexical closures. I sure hope my definition of closures is correct ;)

Here's the code. Please do guess what it does or better yet, compile and run it. The important aspect is to guess which values of "s" do or do not match.

using System;

using System.Threading;

 

 

namespace AnonDelegateTest2

{

    class Program

    {

        delegate void FunkyDelegate();

 

        static void Main(string[] args)

        {

            FunkyDelegate fd = GetDelegate();

 

            for (Int32 i = 0; i < 10; i++)

            {

                fd();

            }

        }

 

        static FunkyDelegate GetDelegate()

        {

            String s = DateTime.Now.Ticks.ToString();

 

            FunkyDelegate fd = delegate()

            {

                Console.WriteLine("Entering fd");

                Console.WriteLine(s);

                s = DateTime.Now.Ticks.ToString();

                Thread.Sleep(100);

                Console.WriteLine(s);

                Console.WriteLine("Leaving fd");

                Console.WriteLine();

            };

 

            return fd;

        }

    }

 

}

I posted an interesting bit of C# a couple weeks ago relating to anonymous delegates. I asked in that post what the result of the following code is. Here is the code and then the result

Code:
using System;
using System.Threading;

namespace AnonMethods
{
    class Program
    {
        static void Main(string[] args)
        {
            for (Int32 i = 0; i < 20; i++)
            {
                Thread t = new Thread(delegate() { Thread.Sleep(10); Console.WriteLine(i); });
                t.Start();
            }
            Thread.Sleep(5000);
        }
    }
}

Result:
6
6
6
6
6
9
12
12
12
13
13
13
15
15
19
20
20
20
20
20

Why?
That's the strangest result from a for loop that I've ever seen. First, I'd suggest that you compile the code for yourself (remember, C# Express is free) and then take and then decompile it in either ildasm or Reflector. I find the C# in Reflector a whole lot easier to read than the IL, so I'd recommend that you start with it. I imagine that you'll get a real kick out of what you see.

First, I'd like to caveat my explanation by saying that this is a CLR guy's explanation of what the C# compiler is doing. I haven't talked to the C# team, but just took a look at the code emitted by the compiler, which both you and I are free to ponder. All that being said, here's the basic idea of what I think is going  on without going into great detail of the mechanics of anonymous delegates.

Once the C# source code is compiled, the anonymous delegate goes away and is replaced by the mechanism that I'm about to describe. Put another way, anonymous delegates are a C# source-code-only feature. For every anonymous delegate, there is a strangely, but probably predictably, named nested class added to the class that indirectly contained (within one of the methods within the class) the anonymous delegate. The class, as you might expect, exposes some surface area that is accessed from the parent class. The class is actually private, which means it cannot be accessed from anywhere but the parent class. There is a default constructor, which does nothing. There is a method that is the anonymous delegate transformed into a regular named method. Remember, anonymous delegates are a C#, not a CLR feature, so the CLR does need a named method to call. Lastly, there are a set of fields exposed on this compiler-generated class -- and this is the part that gets interesting -- that are the local variables that are accessed from both the containing method and the anonymous delegate. Variables that are defined in the containing method but not accessed from within the anonymous delegate or that are defined and used within the anonymous delegate do not get this special treatment. As you might guess, the C# compiler wires up the method that takes the delegate and the generated named method. In addition, all accesses to the shared local variables are wired up through the public fields on the generated class instead of through the actual locals. Very interesting.

OK, so that's the basic explananation of how anonymous delegates work. There are some other subtleties that I noticed, but they are not important for this explanation. I'm going to get further insight on those from the C# team and will post on those when I learn more about them.

So, then, how does all that lead to the strange and unexpected results from the for loop. Well, since the "i" variable is accessed by both the containing method and the anonymous delegate, the C# compiler rewires all accesses to "i" to the "i" field on the generated class. And since each call to the anonymous delegate in our example does its work on a separate thread, then "i" is updated by the for loop on a different schedule than it is written by Console.WriteLine in the anonymous delegate. That's why the value printed out by each call to the anonymous delegate is essentially a race condition between the for loop continually iterating and threads being created and getting time on the processor(s).

Another aspect of this is that shared locals with anonymous delegates essentially turn value types into reference types. That's a really strange behaviour and realization.

Like I said in the earlier post, the trick is determining a scenario where this behaviour is useful. I haven't found that yet, although I'm sure it will be very interesting once I find it.

I'm hanging out at Microsoft Calgary at this point. I'm still recovering from the Calgary launch yesterday, which involved standing for several hours answering lots and lots of questions. Actually, the standing wasn't that bad. My favourite part though was the Q&A at the end of the day. I'd say that Calgary was probably our best stop on the tour for Q&A, with the caveat that I did not attend the Toronto Q&A (even though I was in the building). The Calgary folks provided us with a constant stream of high-quality questions, including a string of them on Windows OneCare, which we really were not able to answer.

The only bad thing is that my notebook hard-drive failed yesterday at the launch. I knew something was bad when the BSOD popped up on my XP machine. Gotta say that that never happens on XP, contrary to the belief of the Slashdot crowd. They'll likely need to put on their tin-foil hats to read/hear that. My buddy Marc -- whom I'm staying with in Calgary -- graciously leant me his PIII Thinkpad for the day. It is slower than my regular machines (particularly compared to my X64 machines with 4 times more RAM back at the office), but it sure works better than my cooked laptop.

Well, I'm at the end of my tour-of-duty on the VS 2005 Launch: Canadian Edition. I really enjoyed myself hanging out with the MS Canada folks, in Toronto, Ottawa, Vancouver and Calgary. In particular, there was a set of us that spent a lot of time in the experts area (while not presenting), answering and routing customer questions among each other. It was tough to beat the expertise held by the Count and the Saint. They could pretty much answer anything from Com+ to Windows Forms, with a whole lot in between. I noticed a bunch of interesting diagrams that the both of them constructed on the paper boards (cannot think of a better term). Can you say WCF? And Zed was our man when it came to VSTS, particularly licensing and the apparent mystery of VS SKUs. We were constantly calling him over, as there were *a lot* of questions relating to VSTS.

I also enjoyed "experting" with the bunch of the folks that joined us in one or more of the other cities. We had a lot of fun together, on and off the field.

That's all not to mention all of the developer/IT Pro folks that we met in each city. I had a lot of interesting conversations with our customers all over Canada. You are the ones that make our platform so successful. You are also the reason why I put so much effort into the CLR, back in Redmond. It sounds like a marketing-line, but it is true. We spend a lot of time in conference rooms, hallways and each other's offices (not to mention mail) talking about how customers are going to use our product. Invariably, we broaden what seems like a simple scenario into something a fair bit more significant, but that will delight and empower a much larger (hopefully all of them) set of customers. That's what it's all about.

Thanks again to everyone I met on the launch tour. I hope that you got as much out of the experience as I did.

I'm now back to Team CLR, back in rainy Redmond.

We’re now at least half way into this set of posts on factoring features in Visual Studio 2005. The more I’ve thought about getting this set of posts fully written up, the more I’ve found these features interesting. My team produced them, so that helps, but I believe that a lot of developers will end up needing these features as a part of their versioning plan.

AKA InternalsVisibleTo

Friend assemblies are a nickname for the InternalsVisibleTo attribute in the System.Runtime.CompilerServices namespace. Not much of a stretch from the name, the use of the attribute allows you to specify which assemblies (your friends) should have access to your internal types and members. Just to remind you, “internal” != “private” – “internal” == “internal”.

As you might guess, this attribute must be set at the assembly-level. You might hope, however, that you could set this attribute at a more fine-grained level. I can explain that more later.

Syntax

The attribute is set in the following way:

[assembly:InternalsVisibleTo(“MyOtherAssembly, PublicKey=4asdsadasdsd”)

Couple things … The constructor of the attribute -- remember attributes are just types -- takes a textual assembly identity. Next, you must specify at least an assembly name and optionally a public key token. You can specify a culture as long as it is neutral. You cannot specify a version. Some of this may be a surprise to you. I’ll explain that below.

Design Decision Points

In retrospect, I’m not sure that making the public key token optional was the absolute best idea, but it was probably was necessary for a lot of interesting scenarios. First, friend assemblies is not a security feature, in the sense that visibility is not a security feature. Friend assemblies is just about changing the usual bounds of visibility to something that is more convenient for developers in a lot of cases.

That all being said, friend assemblies are really best kept to strong-named assemblies. Weak- or simple-named are very easily spoof-able (by definition). As a result, someone can simply use a particular name for their assembly and then get access to the internal fields in a simply-named assembly from which you’ve granted friendship. Even if you only ship your code internally within your business, this still isn’t a great plan. Think about it for a while and you’ll probably begin to better understand why.

It probably mostly goes without saying, but strong-named assemblies can only have strong-named friends. Strong-named assemblies can only depend on strong-named assemblies, so this point is moot anyway.

I do feel strongly that disallowing the version number was the right thing to do. The reason for that is that CLR binding doesn’t always produce the version of an assembly that you expect. There are a lot of mechanisms that can be at play, such as publisher policy, binding-redirects and other binding/versioning changes that we have planned for the CLR v3. As a result, you cannot assume that an assembly that depends on you is always going to be the same. Remember, with friends, it isn’t that you are assuming that a dependency of yours is going to be a particular version, but an assembly that is dependent on you. We would have potentially made a different decision if friends was the opposite way around, being a statement about your dependencies, but it isn’t.

The ability to specify culture provided that it is neutral is mostly a red-herring. There is no harm to specifying it, which is why we allowed it, but it is best to think in general that culture isn’t allowed. The main reason here is that you should not have code in culture-specific assemblies.

I'll be attending the Visual Studio 2005 launches in both Vancouver and Calgary next week.

I'm definitely looking forward to both. I'm up to Vancouver all the time, being only about 2 hours to the Canadian border from Redmond, plus parts of my family lives in Cloverdale. I'm very curious what the Vancouver .Net crowd is like. I have to admit that the Toronto crowd blew me away, with a turnout of >3000 developers and IT professionals. I guess I should admit that a bunch of those might just have happened to be there for SQL Server 2005 or Biztalk 2006. That being said, it did seem like the VS 2005/.NET Framework "experts" in the "Ask the Experts" area were particularly busy.

I haven't been to Calgary since ~2001. My best friend lives there, so we'll likely hit the slopes (Rockie mountains) if the weather co-operates.

Please introduce yourself at the launches if you are able to make it. I'll be in the experts area most of the time. Also, please mail me if you'd like to setup some time to meet, either at the launch or elsewhere, to discuss development issues. I had good luck meeting with banks in Toronto and would like to have similar meetings in the West. I guess I should ring up the oil patch while in Calgary and see how they're using .Net.

Us "experts" typically have dinner together directly after the launch. Please mail me if you'd like to join us. This is a good chance to spend 90mins with the likes of John and Jerome. John refers to himself as a plumber.

Most blog engines support some notion of referrers, listing where your traffic came from. This includes search engines and the search terms that were used. There are definitely some strong trends, largely related to what MSN Search and Google think my blog is about. If I wrote about new moons being discovered in pluto's orbit, then I'd start to get more traffic on that.

That all being said, I'm getting a fair number of searches coming through on gacutil v2.0. That isn't strange on its own. I'm glad that folks are using it. My question is what in particular folks are searching for. If you are one of those folks that are searching for something on gacutil v2.0 in VS 2005, please enter a comment below describing the data that you are looking for. Feel free to mail me @ rlander@microsoft.com.

As an aside, I get pretty much no spam at the address above, which is why I don't mind listing it. The combination of Exchange 12 and Outlook 2003 must be pretty amazing at eating spam. I'm on Exchange 12 dogfood, as is the rest of the CLR team.

What does the following C# code do? Guess and then run it.

using System;
using System.Threading;

namespace AnonMethods
{
    class Program
    {
        static void Main(string[] args)
        {
            for (Int32 i = 0; i < 20; i++)
            {
                Thread t = new Thread(delegate() { Thread.Sleep(10); Console.WriteLine(i); });
                t.Start();
            }
            Thread.Sleep(5000);
        }
    }
}

We were playing with this and similar code in the experts area at the Ottawa VS launch yesterday. Fun, fun.

My task ahead is to find a scenario that makes the subtleties of anonymous delegates useful. Anonymous delegates are certainly useful. It is the trick above with "i" that I'm talking about. I'll post in a couple days with more data on what is going on.

Today marked the second of ten stops on the Canadian launch of VS 2005, in Ottawa. You can check out launch pics @ flickr. Looks like the pics are limited to Toronto at this point, but I expect that to change over the next few days.

The Ottawa launch was another success. It was definitely more cozy than Toronto, being in a hotel, rather than a convention center. This different actually had its benefits. Faces started looking familiar more quickly and people didn't get lost ;)

The ask the experts area was staffed by a bunch of the same experts as Toronto. By the second go, we definitely had our act together. For example, we did a better job of getting the best person to answer questions. I don't know a lot about databases or of our System.Data classes. Fortunately, Bruce Johnson did, another .Net expert from ObjectSharp.

I managed to break away from the experts area for a while to go see one of the talks. Gotta say that Jerome and Barry did a great job. John also did a great job during the keynotes. I didn't get to see him in action during the talks.

The Ottawa launch started with us arriving in town last night. Ottawa is one of my favourite cities in Canada, and certainly my favourite in Ontario. As a result, I was really looking forward to it. The trip started a little rough with a couple flights to Ottawa being cancelled and then me forgetting my jacket at gate 122 in Toronto airport. The dumb part is that they announced a lost jacket just before the plane left the gate (with everyone seated), but I was already deep into conversation with my two seat-mates. Anyway, a quick call to Toronto Airport from Ottawa solved the problem after I was patched back through to gate 122. The jacket was there and we were good to go. My Swanndri jacket is particularly important to me as I picked it up when Annie and I were married in New Zealand five years ago.

We drove from the airport to Microsoft Canada. We were late and were denied admittance at first by building security at Microsoft Ottawa. Note that the building has several tenants, so security was not MS-specific. Anyway, we explained the situation and got past that. Odd. We then spent the next hour with a bunch of local Ottawa .Net folks. We had some great conversations with the local developers over beer and cheese. I spent about 30mins talking with one guy in particular. We had a very interesting conversation about dynamic languages and some of the subtleties of anonymous delegates and methods in C#. I certainly knew about anonyomous delegates, but didn’t know about the subtleties that he mentioned. We talked more about the way that these work today at the launch and I now have a better idea. I’ll post more on this later.

After the beer and cheese, we headed to dinner with Barry Gervin of ObjectSharp. He's the left-most guy in this pic. Definitely a very smart guy. He's a regional director for a bunch of Microsoft technologies and also seems to like Italian food.