Charles Young

BizTalk Server 2009 Announced

2008-09-05T20:25:00Z

Microsoft announced BizTalk Server 2009 today, and gave the green light to talking about the new version. It’s due for release in the first half of next year, and is shaping up nicely. Microsoft is casting BizTalk Server 2009 as a major new version in its own right, rather than just an updated 'release' of BizTalk Server 2006. This is an important move, and one I strongly welcome. There is certainly enough in BizTalk Server 2009 to warrant thinking of it as a major revision of the product, although it retains the same familiar functionality and tooling we have been using since 2006 (or even 2004).

I've been fortunate in getting my hands on the current non-public CTP in the last month or so, and putting aspects of the new version through its paces. It's not wise to go into much detail about this first CTP because some of those details will doubtless change in forthcoming betas, driven in part from the feedback Microsoft is getting. However, I will say a little about the new development and build features and expand just a little on the press releases. These are the areas I have been looking at in depth.

BizTalk Server 2009 will ship with bindings for Visual Studio 2008, and you will need to upgrade to this version of the IDE if you are still using Visual Studio 2005 (which you will be, of course, if you are a BizTalk Server 2006 user). It happens that, in previous months, I've seen more than one BizTalk shop where developers are having to run both versions of the IDE side by side in order to continue developing for BizTalk Server 2006 whilst exploiting features like WCF and WF in .NET 3.5. For many, the changes in BizTalk Server 2009 will come as a relief. This is not a cynical ploy by Microsoft to force people into upgrades for no additional benefit. The move to Visual Studio 2008 is accompanied by a very welcome move to a new project format, bringing BizTalk Server 2009 into line with mainstream .NET development. The new BizTalk project type is defined using MS-Build, just like C# or VB.NET projects. This has major implications in a number of areas. First, it means that, if you use TFS Build, you can now build BizTalk Server 2009 projects without having to write complex scripts that shell out to DevEnv. Just like C# projects, you can let TFS Build do the majority of the grunt work for you, and concentrate your attention more fully on ensuring that your automated build scripts are comprehensive and robust. This is worth the upgrade in its own right, and removes a major source of current irritation. Thanks to the rough edges in the current CTP, one of my colleagues has had the opportunity to get to grips with this side of BizTalk Server 2009 in some depth, and we can report that it is all looking very good indeed, once a few remaining gremlins have been chased out.

The other aspect of BizTalk Server 2009 which we have spent some time on is unit testing and debugging. In one sense, this is not really so much about new functionality, but more about bringing what has previously been considered 'black-belt' into the mainstrem. In Biztalk Server 2006, we currently use BizUnit to drive black-box (grey-box?), end-to-end testing, but we have also created some additional code of our own to support unit testing of BizTalk maps, schemas and pipelines. Similarly, if you know the undocumented registry key in BizTalk Server 2006, you can get BizTalk to retain the generated C# code for debugging purposes, manually attaching to BtsNtSvc.exe processes in order to debug into orchestrations, etc. This has proved a life saver in certain situations. The new version of BizTalk Server is now designed to support these approaches seamlessly. Of course, it is designed to work with the integrated unit testing features of the IDE, rather than BizUnit. There are currently some outstanding questions with regard to the debugging support which I won't go into here, because they simply reflect the unfinished state of the first CTP. However, unit testing certain works smoothly with BizTalk Server 2009 projects, and will help to raise the bar in terms of the approach that is taken to development of BizTalk solutions. Again, I will avoid going into further here because I will hit issues that are still to be fully resolved, but things are generally looking good.

I have only mentioned those areas of BizTalk Server 2009 which I have been looking at in any depth. There is a lot, lot more. One thing I will monitor closely is the inclusion of ESB Guidance 2.0. I have some problems with some aspects of ESB Guidance 1.0, like the unfortunate way in which the UDDI resolver violates the UDDI standard! However, having spent the last year doing little else but designing and implementing service bus patterns using BizTalk Server and WCF (or, in one case, WSE), I regard the inclusion of ESB Guidance 2.0 as a really intriguing, and hopefully worthwhile, aspect of the new version. It is also intriguing that Microsoft has decided to release their implementation of UDDI 3.0 as part of the BizTalk package. Let no one tell you that BizTalk has no role to play in building service buses, or that it is to be regarded as 'merely a hub-and-spoke message broker'. In my experience, the exploitation of the dynamic features of BizTalk (dynamic ports, BAM interception, the rules engine, etc.,) in a rigorous, policy-driven fashion provides good support for implementing many of the core patterns described within the world of ESB. The core design of BizTalk Server predates later ESB thinking, but is much better aligned to it that some people will admit.

Roll on 2009.

IBM to buy ILOG

2008-07-28T22:54:00Z

IBM has announced plans to buy ILOG for $360m, subject to antitrust scrutiny. ILOG specialises in rules-based processing centred on their JRules engine. Although it has been obvious for some time that ILOG was interested in being purchased, the news still came as something of a surprise. In recent times, they have been pursuing a greater presence in the .NET world, releasing a .NET version of their engine, becoming a Microsoft Global ISV partner and joining Microsoft's prestigious Business Process Alliance. Not surprising, then, that a few months ago there were rumours circulation of an impending purchase of the company by Microsoft. I have to say that, when I put this to a Microsoft contact a few months ago, it was strenuously and unequivocally denied that any such deal was being, or had ever been, considered. I have no idea if there was ever any truth in the rumour. Still, I can't help wondering if an opportunity has been missed.

Now that the company is to be bought by IBM, and their technology is to be incorporated into WebSphere, I wonder what will become of their partnership with Microsoft and their .NET offering.

MS BRE: Using Compensation handlers

2008-01-23T01:11:00Z

Being known for my interest in rules processing, I quite often get asked to help with problems with MS BRE. A couple of days ago, I was asked to help investigate an issue occurring in production for a BizTalk Server application. Occasionally, in a fairly high throughput system, BizTalk logs an error stating that a problem has been encountered while executing a rule set. That is the only information provided, with no hint of what the problem might be, and because the issue only occurs intermittently under real-world conditions in the production environment, it was not obvious how to obtain further insight without disrupting live operations.

This article investigates one way of handling this deilemma through the use of the compensation handling feature of Microsoft's Business Rules Engine. It goes on to discuss the broader use of compensation handling in rule processing.

http://geekswithblogs.net/cyoung/archive/2008/01/23/118804.aspx

MS BRE: The Rules Engine Update (REU) Service and Policy Execution

2008-01-23T01:09:00Z

A question came up tonight on BizTalkGurus on my favourite subject of rule engines. I don’t blog enough these days, so this gives me an excuse. Essentially, the question concerned an incorrect, but understandable, suspicion that MS BRE may be using remoting to execute rule sets out-of-process. This is not the case. You can find an article describing what actually happens at:

http://geekswithblogs.net/cyoung/archive/2008/01/13/118506.aspx

Using .NET 3.5 and Visual Studio 2005

2007-12-01T11:25:00Z

The recent release of Visual Studio 2008 and .NET Fx 3.5 is causing some confusion. Microsoft released these two technologies together for good reason. The wonderful new LinQ technologies introduced in .Net 3.5 rely on explicit compiler-level support, and therefore require LinQ aware compilers in Visual Studio. The new version of visual Studio provides these compilers, allowing developers to take advantage of the new monadic syntax. In addition, Visual Studio has several new features designed to make it easier to exploit NET 3.5 features such as Ajax and the foundation libraries (WCF, WF and WPF).

The problem is that by tying the release of .NET 3.5 to Visual Studio 2008, the impression is given that, unless you are ready to upgrade to the new version of the IDE, there is no point thinking about upgrading to the new version of the framework. This is simply not the case. The .NET framework does not have any built-in dependency on Visual Studio, let alone a specific version of Visual Studio. More to the point, Microsoft has long since split the versioning of the framework from the versioning of the run-time environment. .NET 3.5 continues to exploit version 2.0 of the CLR. Visual Studio 2005 is perfectly happy to compile your code against .NET 3.5 assemblies. They are just assemblies. Even more compelling is the realisation that most of the assemblies in .NET 3.5 are identical to those in .NET 3.0 (same version number). There are some new assemblies with new features. .NET 3.5 is just .NET 3.0 with extra stuff.

Why is this important? Well, not everyone is ready to upgrade to Visual Studio 2008. Apart from the expense this involves, consider the dilemma of BizTalk Server developers. Currently, there are no Visual Studio 2008 bindings for BizTalk Server (i.e., you can't create BizTalk Server project types in the new IDE). This, we are assured, will be addressed at some point, but that could be months away. For the time being, BizTalk developers are stuck with Visual Studio 2005 :-( Hence, some people are currently discounting the possibility of using .Net 3.5 because they believe, quite incorrectly, that it requires an upgrade to Visual Studio 2008.

There are issues, of course. As well as the absence of compiler support for LinQ, Visual Studio 2005 does not have access to various new project and file templates and tools that support the new version of the framework. Developers may need to do more coding in Visual Studio 2005 than would be necessary in Visual Studio 2008. This is often a small price to pay, however, in order to access the improvements in 3.5. As an example, consider the new integration between WF and WCF, provided in the new System.WorkflowServices assembly. The integration is provided via the new WorkflowServiceHost class and a couple of new activities. Visual Studio 2008 has new template support for building workflow services, and comes with a very useful new WCF test harness. However, exploiting this new functionality in Visual Studio 2005 is trivial. Create a WF workflow library, add a reference to System.WorkflowServices and add the new activities to your tool box. Finally, use the WCF Service template to add a service class to your project and you are just about in the same position as you would be in Visual Studio 2008 if you used the new Workflow Service project template. You'll need to write a couple of lines of code to use WorkflowServiceHost to host your service, of course. Off you go, and enjoy .NET 3.5.

Microsoft 'Oslo' - the vNext SOA Platform

2007-10-30T22:43:00Z

Noon (Redmond time) on Tuesday 30th October marks the moment in history we can begin to talk publically and openly about Microsoft’s 'Oslo'. Oslo is the codename for the next generation of Microsoft's SOA platform...'Microsoft SOA vNext' if you will. It is a huge vision, and one which Microsoft is throwing some serious resource at. Their strategy will unfold over several years, but the first public view has been provided today.

Get more details at http://geekswithblogs.net/cyoung/archive/2007/10/30/116456.aspx.

BizTalk Services: An overview

2007-10-09T12:18:00Z

Earlier this year, Microsoft released a CTP version of a new technology stack called 'BizTalk Services'. In July, they updated this with a new release. It is only now, at long last, that I've got around to playing with BizTalk Services, thanks to the gentle pursuasion of my colleague, Andy James (CTO at SolidSoft). I must say that I am very pleased that Andy applied some pressure. This is a really interesting development in the BizTalk world, and having now had a chance to stub my toe against the tyres (I'll leave the full-blown kicking until a future date), I feel rather excited about the possibilities this technology opens.

I have provided an overview of BizTalk Services at http://geekswithblogs.net/cyoung/archive/2007/10/09/115944.aspx. I have also written an article on the Identity Service that is part of the stack. This is at http://geekswithblogs.net/cyoung/archive/2007/10/09/115943.aspx. I have published some notes on solving problems when installing the SDK. These are at http://geekswithblogs.net/cyoung/archive/2007/10/09/115942.aspx.

WF Rules and MS BRE - Comparing Performance

2007-08-14T17:27:00Z

I’ve been asked a few times how the performance of WF (Windows Workflow Foundation) Rules compares with that of the Microsoft Business Rules Engine (MS BRE). Having done no testing, I could only guess at the answer.

I’ve now undertaken some initial performance testing to compare WF and MS BRE, and decided to publish the results. You can read my write-up of the results here.

http://geekswithblogs.net/cyoung/archive/2007/08/12/114597.aspx

BizTalk Server or WF for rules and tracking?

2007-07-19T19:12:00Z

I got an email today requesting help in deciding the appropriate selection of rule processing technology for a workflow application. I’ve got requests like this before, so I’ve decided to post a reply publically.

http://geekswithblogs.net/cyoung/archive/2007/07/19/114061.aspx

MS BRE: Controlling rule side effects

2007-04-14T03:57:00Z

For almost two years now, I've been intending to write an article about the mysterious 'side effects' flag used in Microsoft Business Rule Engine policies. Microsoft documents this feature (see http://msdn2.microsoft.com/en-us/library/aa559124.aspx), and describes very briefly how to control it. The mystery that surrounds this flag arises because it is represented by an attribute named 'sideeffects' in Microsoft's BRL (Business Rule Language) although it actually controls a caching mechanism, and because Microsoft has not provided access to the flag in their Rules Composer, thereby giving the impression that it is not a 'first-class' feature of rule definitions. In reality, this flag is an important aspect of the functional approach adopted in Microsoft's Business Rule Language (BRL), and can have a significant effect on both behaviour and performance of your policies. Understanding the flag also requires a greater understanding of the way in which BRL binds functions to custom code, providing a clean mechanism for implementing domain-specific extensions to the rule language.

The article is located at http://geekswithblogs.net/cyoung/articles/111169.aspx

Microsoft BRE: Fact Identity in the Microsoft Rules Engine, or how the author chased a non-existant bug

2007-04-04T22:24:00Z

I recently had reason to revisit the exact mechanisms Microsoft use when you assert facts to the Microsoft Rules Engine. I was discussing stuff on-line with a fellow rules enthusiast when a terrible thought occurred to me. Can the MS BRE always uniquely identify each different fact, or is there a chance that sometimes it might confuse two facts with each other?

For a couple of days, I was convinced that I had stumbled on a significant bug. Indeed, at one point, I thought there was such a serious problem that I would have to recommend to my company that we desist from any further use of Microsoft’s rules engine. Melodramatic, huh! Fortunately, after a bit more research, I discovered that I was quite wrong. The MS BRE does not suffer from a terminal flaw and can be trusted to always distinguish correctly between all your facts…except when…well, I’ll get onto that later. It’s worth recording my suspicions, mistakes and eventual enlightenment. There is something useful to learn, here, about the inner workings of the engine, and also about Microsoft’s implementation of the Hashtable class in .NET.

The first point to make is that Microsoft uses the term ‘fact’ rather loosely in MS BRE. A fact is a data item ‘asserted’ to the working memory of a rules engine. The working memory, itself, is simply some mechanism for storing and retrieving facts, and is typically implemented as a set of node-specific collections (a rule set is represented as a node graph at runtime). Microsoft makes extensive use of the non-generic Hashtable class in the MS BRE to implement node memories. In most similar engines, a fact is always a simple data tuple – i.e., a collection of attribute values. The attributes are usually named, though sometimes un-named and ordered. It is also almost always the case that each tuple contains some kind to type identifier. Many engines provide a metadata definition language which rule developers use to define tuple templates. These definitions are called ‘deftemplates’ in several engines.

Microsoft does not support the direct equivalent of deftemplates, and ‘facts’ do not always convert neatly to single tuples. The MS BRE philosophy is to apply set-based rule processing directly to data stored in .NET objects. You can therefore assert any POCO (Plain Old CLR Object) to the engine as a fact. In this case, a POCO is roughly equivalent to a tuple because it provides named properties (MS BRE does not access fields directly). Of course, it also provides normal methods as well, and rules can exploit this by calling methods in order to obtain data values, evaluate predicates or invoke custom actions.

Microsoft provides four ‘Typed Fact’ classes for wrapping specific types of .Net object and asserting them to the engine. These all implement the ITypedFact interface and are as follows:
TypedXmlDocument: wraps an XML document or node
TypedDataTable: wraps an ADO.NET DataTable
TypedDataRow: wraps an ADO.NET DataRow

DataConnection: wraps an ADO.NET DataSet and either SQL Server or OLE DB .NET data adapters, connections and transactions. A DataConnection object is used to access a specific table in an external data source via the wrapped data connection.

A TypedDataRow is the closest equivalent to a data tuple. The other Typed Fact classes are used to instantiate what Microsoft calls ‘facts’, but are, in reality, sources of multiple facts. Once asserted, the engine uses information embedded in the rules in the currently executing policy to internally select and assert the required facts from these fact sources. For example, it uses XPaths to select node sets from XML documents, and then uses additional XPaths to access individual ‘fields’ within these nodes.

The important thing to note here is that you only ever assert .NET objects to the engine. In some cases the engine automatically asserts additional .NET objects internally. MS BRE must be able to distinguish between all these objects unambiguously. It must be able to tell when two objects represent two different facts and when they reference the same fact. The only way this can be done generically across all asserted facts is to use some notion of object identity.

Unfortunately, object identity is a rather slippery subject in .NET. There is a good reason for this. In un-managed code, a compiler will normally distinguish between two different objects by using their references. Each object reference maps directly to a memory address at which the object state is located. The exact layout of the state varies from compiler to compiler and may depend of whether the object has a vTable or equivalent. The important point, however, is that once created, an object does not generally move. Hence, references stay valid over time.

Not so in the CLR (the .NET Common Language Runtime). The runtime environment provides automatic garbage collection and memory management. One consequence of this is that objects move. They cannot be guaranteed to remain at the same memory location for any significant period of time. Hence, you cannot depend on an underlying memory reference to provide object identity except in a very limited sense. If you want to test two objects to see if they are identical, you can test their underlying references at a point in time (or, at least, an approximation to a point in time). If the references are equal, the objects are identical. Microsoft supports this via the static ReferenceEquals() method of SystemObject.

In itself, this is simply not adequate to meet MS BREs needs. MS BRE stores fact objects in its working memory over a period of time, and needs to be able to distinguish between existing and new facts unambiguously. Each time you assert a fact to the engine, MS BRE needs to know if this is a new fact of which it has prior knowledge, or if you are reasserting an existing fact. Hence the engine needs some reliable form of identity that persists over time. We might consider using the values of a field in each fact as unique identifiers. A POCO, though, can contain any data, and there is no guarantee that two entirely different POCOs might not contain identical data. In any case, you would have to impose some kind of contract on your POCOs…meaning that they are not really POCOs anymore from the engine’s perspective.

Most .NET developers are aware of the GetHashCode() method. GetHashCode() is inherited from System.Object, and therefore is a member of all .NET objects. It is also callable on value types. Microsoft provides two default implementations of this method for reference and value types. For value types (structs, in C#), the hash code is provided by calling GetHashCode() of the first field in the value type. Hence, not all fields in a struct are equal. The first one has a special role. For reference types, the default implementation in .NET 2.0 is a randomly-generated 32 bit signed integer.

The hash code is often confused for an object identifier. In many cases, it is almost good enough to be used as such, but not quite. Hash codes, according to Microsoft’s guidance, should be associated with the ‘value’ of an object. This is the value which is tested for equality when you use the Equals() method. If two objects evaluate as equal, they should both return the same hash code. Of course, the ‘value’ of an object may change over time. For example, a string object’s value is the text associated with that object, and the hash code is derived algorithmically from that text. If you change the value of a string, the hash code generally changes as well. The default implementation of GetHashCode() generates a number randomly which is then associated with the object by the runtime and returned on future calls. Hence, it is almost like an object reference. Almost, but not quite. The fundamental problem is that there is a small chance that two entirely different objects could, by chance, have identical hash codes. The default implementation of GetHashCode() does not guarantee uniqueness of hash codes.

One of the reasons I suspected there might be bug in MS BRE is that the implementation of GethashCode() in System.Object was different in .NET 1.x. In the earlier version of the framework, Microsoft returned the index of the object’s sync block as a 32-bit positive integer. A sync block is a structure used to manage thread synchronisation, and the index value is incremented each time a sync block is created. If you called GetHashCode(), and there was no sync block, the object would create one internally in order to obtain a hash code. Because the indexes were incremented, and because (I believe) index numbers were recycled only when objects were garbage collected, there was no chance that two hash codes would collide. However, this was never part of the hash code contract, and Microsoft, from day one, reserved the right to change this behaviour.

No doubt you can begin to see why I entertained suspicions about MS BRE. MS BRE was originally written for .NET 1.x, and apart from a few bug fixes and minor enhancements, it has hardly changed in the .NET 2.0 version. When I used Reflector to inspect how Microsoft handles fact identity, I realised that the approach is based on calls to GetHashCode(). It therefore occurred to me that Microsoft might have incorrectly depended on the coincidental nature of the default implementation of GetHashCode in .NET 1.x, and not realised that this behaviour had changed in .Net 2.0. I suspected that, because hash codes can collide in .NET 2.0, facts might sometimes be confused. The engine might think you had reasserted an existing fact when you had actually asserted a new fact.

I needn’t have worried. The issue is robustly dealt with within MS BRE by using the Hashtable class. I had a reasonable idea of how this class works, but there was one detail of which I was not aware, although in hindsight it appears blindingly obvious that this is the only sensible way in which Microsoft could implement this class. I’m sure many developers are aware of this feature, although I haven’t yet come across any detailed discussion or documentation that describes the exact behaviour.

When you assert a fact to the engine, its type is tested, and it is handed off to a ‘class node’. This is a type-specific node object in the Rete. A Rete is a node graph that provides an efficient and executable run-time representation of a rule set. Class nodes first test to see if a fact is already stored in working memory. A class node implements its working memory as a simple hashtable, and it uses the object itself as the key value. It tests the hashtable to see if the object already exists in the collection. If the hashtable determines that the fact is not in the memory, it wraps it in an instance of a class called ‘WorkingMemoryElement’ (WME), and then adds the WME to the hashtable. Incidentally, the WorkingMemoryElement class overrides GetHashCode() in order to delegate to the GetHashCode() method of the fact it wraps. If the hashtable already contains the fact, the engine assumes that you are re-asserting an existing fact. It retracts the existing fact, and then asserts the new version of the fact.

Now, as I say, it is blindingly obvious with the benefit of hindsight that this must be implemented as a robust mechanism. After all, imagine what life would be like if, when you exploit a .NET 2.0 hashtable that uses objects as keys, it sometimes fails to distinguish between the objects it stores. This would be terrible, and we would all have heard about it. It would break huge amounts of code causing strange, arbitrary and infrequent exceptions. The trouble is that I knew just enough to know that, when you use an object as a key, the hashtable calls GetHashCode() internally. You can see why I was worried.

Hashtables compute an internal hash code by calling GetHashCode() and ensuring that the negation bit is set to 0. This is necessary because the negation bit (the first high-order bit in the 32-bit integer) is used for collision detection purposes. Hashtables detect hash code collisions and use an approach called ‘rehashing’ to deal with this.

When the MS BRE tests for an existing fact, the hashtable calls GetHashCode() on the fact and computes the internal hash code. It then looks this hashcode up in the collection to see if it is already used for a given fact. If it isn’t, the hashtable knows unambiguously that this is a new fact. However, if it does find the hash value, it cannot be certain if it already contains that fact, or if a collision has occurred. This brings us to the feature of which I was unaware. Very simply, if it finds an existing item with the hash code, the hashtable performs an additional test. It tests the equality of the fact object, using the instance Equals() method, against the object it already contains (the one with the same hash value). For performance purposes, it only performs this additional test when needed.

The default implementation of the Equals() method, inherited by all reference types from System.Object, internally does a ReferenceEquals() check to discover, unambiguously, if the two objects are actually identical. So, if your facts use the default implementation of Equals(), the hashtable will always correctly distinguish one object from another. The ‘TypedFact’ wrapper classes all use the default implementation, and your POCO facts will also use the default implementation unless you have overridden it, or are using a class that overrides this method.

In conclusion, therefore, we can see that MS BRE safely handles fact identity as long as you never assert .NET objects that have a poor or incorrect implementation of Equals(). For a long time, I have wondered if I ought to recommend overriding GetHashCode() and Equals() on fact classes. My conclusion is that I should definitely recommend that, unless you have a very good reason to do so, you should NOT override these methods. For safety, use the default implementations, or existing overrides on classes you trust (like ‘System.String’, for example). If you do have a reason to provide overrides, make sure you fully understand the characteristics of the code that you implement and ensure that you code conforms rigorously to the ‘rules’ specified by Microsoft. If you don’t, you may well live to regret it. And, unlike me, don’t worry…Microsoft developers really do have some idea (generally) of what they are doing J

MVP Global Summit Day 3

2007-03-15T14:06:00Z

Day three of the MVP summit was held at the Microsoft Campus. Things started badly with a long and slow bus ride through heavy traffic leaving me with no time for breakfast when I finally arrived at the conference centre. I boarded the shuttle for building 43 to find most of my fellow BizTalk MVPs (the ones who have been staying in downtown Seattle since the weekend) equally late, and we arrived en masse about 15 minutes into the first session. Jesus Rodriguez was on his feet presenting the forthcoming WCF adapter SDK with a particular focus on building LOB adapters with metadata harvesting and dynamic, consumer-orientated contracts. Sonu Arora presented code examples.

After a frantic and barely successful search for coffee, I sat through the next session with Gruia Pitigoi-Aron. This continued the focus on the new adapter framework, discussing the development of custom WCF channels from the perspective of BizTalk developers. Custom channels can be exploited in BizTalk land using the WCF Custom Binding adapter that will ship with R2. Gruia sees this as a new extension approach to be used alongside custom code in orchestrations and pipeline components. I need to do more thinking at some point about this approach. Although the adapter allows custom channels to interact with the BizTalk message box, this does not mean that you automatically get the recoverability and tracking you might need in a real-world solution. The appropriate use of this new facility will, I trust, become clear as we begin to apply it for real.

Next up was Marty Wasznicky discussing the BizTalk Server disaster recovery model and support for high availability. Microsoft has certainly improved this side of things, and we saw a good demo (slightly cursed by the demo gods at the end) of a BizTalk installation being recovered. Marty had to present under the eagle-eye d scrutiny of Lee Graber who, although now working with the SQL Server product team, joined us for part of the day. I was amused by Lee’s comments about BizTalk’s feature history. ‘No’, he said on more than one occasion, ‘that feature wasn’t introduced in BTS 2006; it was in BTS 2004 SP2’. Ah, the voice of authority.

Not being a system administrator, I was suitably impressed by the DR features that Marty demonstrated. This was, however, the subject of some discussion later in the day, with alternative viewpoints and strategies being discussed. The inability to support SQL Server mirroring was a matter of some concern. Again, BizTalk DR is a subject I need to look more deeply into.

After lunch, we were briefly back to adapter-land with a presentation by Tapas Nayak looking at Microsoft’s new WCF SAP adapter. This was especially interesting in terms of seeing the application of dynamic contract creation features in the new framework.

The direction of the day now changed with a fascinating presentation by Pravin Indurkar on the next release of WF. This will be part of .NET 3.5 which will ship in the ‘Orcas’ time frame. Pravin discussed and demonstrated a number of extensions that more closely integrate WF and WCF, allowing workflows to be exposed as services via WCF bindings and behaviours. WF will include new Send and Receive activities, and will support a context exchange protocol that will support a wide spectrum of service-orientated interaction patterns, including ‘conversations’ similar to patterns used in SQL Server Service Broker. This is very powerful, and I couldn’t help thinking that, conceptually at least, we were seeing something of a stepping stone on the way to integrating .NET 3 into a future version of BizTalk. WF 2.5 will support durable services via a persistence provider, and may even offer BizTalk-like features such as property promotion.

Next up was Paul Andrews, a Microsoft product manager, who discussed the drivers for choosing between different integration and workflow technologies (specifically BizTalk and WF). We’ve all sat through presentations were an attendee annoys us by reacting visibly and negatively to the presentation. On this occasion, I was that attendee, so I apologise to Paul if I made his life difficult. However, I profoundly disagree with the line he took. A major aspect of my work as a BizTalk consultant is to help clients understand how to best select and fit available technologies to their requirements, and I therefore want to see a Microsoft approach that is grounded in a solid understanding of the nature of specific technologies. To me, Paul’s approach would be analogous to comparing and contrasting the IP stack to Internet Explorer, and suggesting that while IE might be suitable technology for use by ‘enterprise’ customers, an ISV would probably want to install the stack. After all, draw up a tick-box list of features offered by IE and the IP stack, and you will see that the stack essentially offers a subset of the features you can access via your browser. So, if you don’t have ‘enterprise’ requirements, you only need the IP stack to access the Internet. Enterprise clients, of course, don’t need the stack. Huh?

For IE, read BizTalk, and for the IP stack, read WF, and you may begin to understand where I am coming from on this. Try to imagine how many minutes I would be allowed to stay in a client’s building before being marched back to the entrance if I was to take this approach in helping them to select appropriate technologies. Maybe three?

The point, surely is that WF is foundational. It is a framework on which we build. It is not a ‘product’ to be compared and contrasted with other products. BizTalk will, in future, build on this foundation. K2 ‘Black Pearl’ will build on this foundation. MOSS has already built on this foundation. MIIS is building on this foundation. Developers will build on this foundation when implementing custom workflow solutions. Edge servers will exploit this foundation and then interact with BizTalk which builds on this foundation and integrates with other services which build on this foundation. You’ve got to understand the foundational nature of WF before you can realistically advise clients in regard to technology selection. This is not enabled by plugging WF into a tick-box product comparison matrix.

OK, I bet (and hope) I get some kick-back on that. Let’s move on.

Brad Paris and Tiho Tarnavski were next up with a session on the WCF and WF BAM interceptor extensions in BizTalk R2. A few months ago, a colleague and I spent a couple of hours reflecting on WF in order to work out how its tracking infrastructure works. I won’t discuss here, but it is suitably impressive. At the time, we compared and contrasted this with the instrumentation you would need to do if using the BAM interceptor APIs directly. I was therefore delighted to discover that the WF BAM interceptor is built on the tracking infrastructure, which means that it should prove really powerful. There are a couple of caveats though. The point was well made that Microsoft does not offer anything like TPE (the BizTalk Tracking profile Editor) in order to exploit interception in WF workflows. This means that configuration must be done long-hand by editing verbose XML, and that this makes BAM instrumentation difficult. A good point, which is, of course (yet again) a tooling issue. I hope Microsoft takes note. A second point is that the interceptors do not use buffered event streams. This is potentially a killer. I do not understand why Microsoft persists in coupling BAM buffered event streams so closely to the BizTalk message box. Why not re-factor the technology to allow persisted events to be streamed out to any persistence provider.

Anush Kumar presented the next session on RFID. I’m very familiar with this subject thanks to the focus we have in Solidsoft, and many conversations with Simon Holloway who heads up our efforts in this area. I wanted to question Anush more closely on Microsoft’s story with regard to simulation and the current lack of support for ALE. However, there was so much interest from others that I didn’t get the chance after the presentation. Maybe I will email my questions. I did, at least, get to raise my concerns briefly about the lack of support for rules development when using the BRE event handler, and was later able to explain this in more depth to Jon Flanders and others.

The day finished with a session on Identity Management from Richard Turned and an extra session on MIIS which is built around WF. I’ve known Richard for many years, and I spent some time last year building a managed card demonstrator using CardSpace which we demonstrated at the Microsoft Security Summit last year, so am familiar with the technology and Kim Cameron’s laws of identity on which it is based. I am less familiar with MIIS, so was very interested to see this demonstration. MIIS certainly appears to provide a flexible approach to identity management and process governance.

In the evening, we got to share some quality time with members of the product team. I met with some of the guys working the next version of BizTalk orchestration, and also had a long conversation with Clemens Vasters who, years ago, sparked my initial interest in BizTalk. A group of us were able to articulate some of our concerns and questions about the future of BizTalk, and Clemens promised to look more closely at how roadmap information can be shared more effectively with the MVP community.

I’d love to add more and more names to my emerging list of BizTalk MVPs who I have met, but time does not allow. It’s great to meet all you guys.

MVP Global Summit Day 2

2007-03-14T08:18:00Z

Day two of the MVP summit is over. We were at the Washington State Convention & Trade Centre all day, but will be decamping to the Redmond campus tomorrow. The day started with an unscheduled bout of ‘pre-match’ chanting from the Canadian MVPs. They are an excitable crowd, and have spent much of the day ensuring that we all know which country they come from (Canada, I believe). It appears that all Canadian MVPs (with the exception, I am glad to report, of Brian Loesgen) wear identical clothing. Very strange.

VP Richard Kaplan got things underway informing us that the conference is being attended by 1,700 MVPs from 90 countries. The MVP community is certainly a very diverse group which helps give the conference a rather different feel than, say, the PDC. Bill Gates was up next, and provided us with a typical Gatesian keynote tour de force. He talked about the implications of moving to 64 bit computing and painted lots of cameo pictures of possible technology futures, emphasising some of his favourite topics along the way such as ink, recognition, etc. It’s interesting to note his acceptance of the ongoing transition of the programming ‘backbone’ to the Internet. He also discussed what was to become something of a recurrent theme during the day; namely the move to multi-core parallel processing, and the implications on software.

Wisely, Bill kept the speech fairly short, and then moved on to a Q&A session. There were too many questions to record here, but they included topics such as the problems faced by the IT in western countries due to the lack of graduate entrants into the industry, philanthropy, ‘Software as a Service’, the role of blogging, how to improve IT support for dealing with natural disasters, his all-time favourite Microsoft product, low-cost computing in the third world, the future of Live Search and a fascinating, but all too brief discussion on the evolution of technologies from tools to social insignia (yes, I’m serious).

One rather sour note was the reaction of a handful of attendees to some of the questioners whose first language was not English. I would like to see them attempt to ask a question publically in, say, Japanese, and cope with the same level of sniggering and rude comments. Not good, guys. Where I come from, we consider this attitude to be a form of racism and regard its practitioners as social pariahs. The MVP community is a global community, and should be respected as such.

Sean O’Driscoll then finished off the session by announcing the dates for the 2008 MVP summit – 14th-17th April.

After the break, I attended the Developer Division roadmap session led by S Somasegar (Soma) who is the Developer Division corporate VP. He introduced us to the highlights of the Orcas release of Visual Studio. Eric Carter demonstrated the Tools for Office and the way in which Outlook can be easily extended using Form Regions. Soma talked about friction-free deployment and the introduction of cross-platform WPF/e. Brian Goldfarb then demonstrated the use of ink overlaid on graphics and videos in WPF. WPF is certainly impressive.

Soma finished the session by discussing some of the future directions for Team Foundation Server. There will be an emphasis on testing tools and the capture and tracking of business requirements. He also briefly discussed the challenges of multi-core programming and talked about the investment Microsoft is making in this area.

After lunch, I attended a session with Anders Hejlsberg. Anders, despite suffering the wrath of the software demo gods, provided us with an intensive introduction to LINQ. I’ve played a bit with this technology, but there is surely nothing better than to sit in on an introduction from Anders himself. LINQ is fabulous. I realise the functional programming crowd probably find it little amusing that the rest of the development world is just now discovering monadic coding, but after years of using datasets, XML DOM and the like, the advent of LINQ in its various forms is truly exciting.

Anders graciously demonstrated the Literal XML features of VB.NET 9.0, leaving the C# developers feeling distinctly uncomfortable for a minute or so as we realised that perhaps there is a point to VB.NET after all! He then salvaged our faith in C# by demonstrating a Visual Studio add-in that auto-generates C# XLINQ code directly from XML. We sat through a huge number of demonstrations, but the overall themes were the declarative nature of the technology, the reduction in coding this brings, the clarity of the code in focussing on the ‘what’ rather than the ‘how’ and the clear and logical factored structure of the LINQ stack. All very good stuff indeed.

The third, and final, session of the day was with Don Box and Chris Anderson. Don did the talking and Chris provided a kind of silent running commentary by typing a stream of consciousness in Notepad. Chris also wrote the code for the demos. The session was the subject of some discussion over dinner this evening, and everyone seemed to have a different view and interpretation. My perspective is that Don was inviting us to think a certain way about XAML. Specifically, he described how XAML and the .NET 3.0 framework can be perceived as a platform for building DSLs (Domain Specific Languages). Don took us through an example where he defined a domain-specific schema as a set of C# classes, and created XAML to act as a model within that domain. He ‘interleaved interpretations’ of the domain with the schema definition (in plain English, he implemented additional functionality within the classes), and created transformational code in order to transform from the ‘newer to the older’ – i.e., from the domain model to actual run-time code. As part of this, Don showed how C# 3.0 lambda expressions and expression trees could be used effectively to represent and transform abstract syntax trees (ASTs). Don also showed us a simple graphical tool that represents any XAML in a generic fashion and provides a simple graphical DSL interface.

I am not well acquainted with the world of XAML, so the session provided some food for thought in the lead up to the forthcoming release of Orcas. I’ve played with Microsoft’s DSL toolkit which is a core component if their software factory initiative, but Don’s approach is quite separate to that, and is really a kind to general pattern for thinking about, designing and implementing code in .NET 3.0.

In the evening, we were taken to the Seattle Museum of Flight for the evening meal. I spent the evening chatting to various MVPs including Jeffrey Juday whom I met for the first time today. I also met Joao Martins from Portugal this morning, and, very briefly, Abhilash Shanmugan.

Well, time again to sleep. Tomorrow is an early start up at the campus. Goodnight.

Name-dropping in Seattle - Day one of the 2007 MVP Global Summit

2007-03-13T07:20:00Z

I'm in soggy Seattle at the MVP Global Summit, and, like so many of my fellow MVPs, it seems appropriate to keep some kind of daily journal - you know the form - let you know how fortunate I am to be on a big Microsoft freebie, and how unfortunate you are to, er..., not be...and, of course, get a bit of name-dropping in as well.

The first round of name-dropping has nothing to do with the summit. A week ago I was at the Microsoft Architect Insight conference at the Celtic Manor outside Newport, Wales. The Celtic Manor is famous for golf and is the venue for the 2010 Ryder Cup. It is also famous, these days, for software architects working on the Microsoft platform. This was the second year of the conference, and Solidsoft, having attended last year, was a co-sponsor of the event this year. The conference started with a keynote from Merlin Hay who, later in the bar, entertained a group of us with his views on hereditary peerage and 'breeding' (what else - he is the 24th Lord Erroll!), and, more seriously, provided some interesting insights into the way that political decision-making happens at a national level. Later that evening, the Solidsoft team had the pleasure of a couple of hours with Ivar Jacobson, one of the 'three amigos' (the gentlemen responsible for UML) and an early advocate of component development. Perhaps the best thing was that, instead of deep discussions on the applicability of agile methodologies in enterprise integration, we instead talked love, live and the pursuit of happiness. Hey, we do have lives! I enjoyed the final plenary with Ivar, Jack Greenfield (the software factory guy) and Colin Bird sharing their collective insights and views on a range of architecture and methodological subjects.

It was nice to briefly meet Ed Gibson again. Ed is Microsoft's Chief Security Advisor in the UK, and always a great pleasure to talk to. I was also glad to, oh so briefly, catch up with Planky (Steve Plank) from Microsoft who fronted a series of sessions on security and identity management. Well done to Matt Deacon and Daniel Nissel who made the whole event happen. It was good to see my old colleague, James Saul, and I enjoyed his session on real world architecture (I left the heckling to Andrew Rivers ;-) ). Most embarrassing moment was parading the Solidsoft corporate train set (don't ask!) in front of rival companies' stands. The train set did fulfil its role, though, as a talking point around RFID, and Simon Holloway did a great presentation on the subject at the end of the conference. Thanks to Andy James for keeping the Solidsoft team on the straight and narrow, and well done to Pam McClelland who fronted the Solidsoft stand.

The conference, itself, was very worthwhile. Software, infrastructure and enterprise architecture covers very wide territory. Real effort was made to focus the conference on a range of architectural issues, and to capture the collective wisdom of the delegates. I have personally never attended an IT conference that was so interactive, and hope that this aspect is continued and built on in future years. Listening to discussions on issues like the development of IT skills in higher education or the effective presentation of enterprise architectures to business stakeholders took me into unfamiliar territory which broadened my horizons and provided food for thought.

Well, that’s a lot of name-dropping from the Welsh event. Now back to Seattle. Today was the first day of the conference, and has mainly been a matter of registration and networking. However, things really got underway yesterday evening when a group of BizTalk MVPs got together at Kells, an Irish pub in downtown Seattle. Mike Woods, the BizTalk Server Senior Technical Product Manager, was there, and he surprised us by paying for the meal - thanks to him for that. I had already spent a pleasant couple of hours with Gregory Van de Wiele in the hotel bar (mainly drinking orange juice, honest), and Kells provided the opportunity to meet a whole lot of people I know via their blogs - Alan Smith, Paul Somers, Jon Flanders, Brian Loesgen, Romualdas Stonkus, Jon Fancey and Evangelos Hadjichristodoulou. We were amused by the live Irish music - roughly equivalent to several rounds of 'Old MacDonald had a farm...' sung with Irish accents. We eventually decided to head off to the relative quiet of one of the hotels where we discussed all the things we would like to raise with the BizTalk product team while we are here.

Today, after registration, I attended the expo. It was nice to see Lorna Williamson, our UK MVP lead. I still need to introduce myself to Akim Boukhelif who I saw from a distance. Much to my delight, I ran into Mick Badran. I've know Mick for ages, but haven't seen him since he moved back to Australia four or five years ago. He now has two kids and a thriving business based on BizTalk and MOSS training and consulting. Fantastic. I always felt Mick and I saw technology from a similar perspective, and after all this time we soon fell into sharing development experiences and violently agreeing with each other. Romualdas joined us after a while, and he and I got some late lunch. Romualdas is Lithuanian and works closely with Microsoft in that corner of Europe.

This evening, Microsoft hosted various regional dinners. The EMEA dinner was well attended, and buzzing. After a couple of hours the EMEA BizTalk MVPs decided to gatecrash the Americas dinner, which meant walking past or through the APAC and Japan regional dinners. I am therefore in a position to authoritatively state that, of the four groups, it was the EMEAns who clearly were in the lead in terms of party spirit, despite being uniformly jet-lagged. The other regions have some catching up to do. By about 11:00 pm, though, we had all had enough for one day. Tomorrow kicks off with breakfast and a keynote from Bill Gates. For now, it’s time to stop the name-dropping and get some sleep.

Rules engines and dependancy injection - a response to Nick Malik

2007-03-09T23:52:00Z

Nick Malik published a thought-provoking article on rules engines recently. You can read it at http://blogs.msdn.com/nickmalik/archive/2007/03/06/perhaps-it-is-time-to-declare-victory-in-the-battle-of-rules-engines-vs-dependency-injection.aspx. He adopts a fairly negative attitude to rules engines in general, suggesting various reasons why he considers that they are not often useful. His preferred route is to use dependency injection patterns, implementing rule logic in code components that are then loaded in some dynamic or configurable fashion into consuming applications.

I believe Nick is quite wrong to contrast rules engines and dependency injection in this way. In reality, many modern rules engines provide the ideal mechanism for implementing and managing dependency injection patterns. You can read my response at http://geekswithblogs.net/cyoung/articles/108411.aspx.