The .NET Platform
Development Tools
COM & COM+
Data Access
Web Development
XML Technologies
Windows Servers
Wireless & Mobile
Security issues
Design & Process
Career Development
Analysis & Comment
Disposable Objects
We found this book useful!
You are not logged in: login here to access all areas.
Talking BizTalk Server
Andy James is Chief Technology Officer of SolidSoft, one of the country?s leading BizTalk specialists. He spoke to Matt Nicholson at the company?s Basingstoke office about BizTalk past and present, Service-Oriented Architecture and much more.
Author: Matt Nicholson
Last updated: Jun 2004
Andy: We were looking for strategies to take us through the collapse of the dot-com boom, working out how businesses would react, and how we could continue making money. There would certainly be a problem with new projects, because the dot-com boom had sucked in all the investment money. If a collapse did happen, companies would have to pare back. So we’d be left with legacy systems. How can you extend the lifespan of legacy systems? The answer is integration - putting something in between legacy systems and a more modern front-end. We started thinking about how to make existing systems last another five years. Matt: This was before the Web Services drive? Andy: The Visual Basic SOAP Toolkit was available, but SOAP hadn’t really taken off. People had Web front-ends, and were looking to leverage back-end systems cheaply. BizTalk 2000 struck us as really cool because when an analyst said, “This is what I want the process to be,” the developer actually had to produce it. That rigor was what we were looking for. Normally the analyst says, “This is my business process,” and then the developer tries to produce something similar but shies away from the difficult bits. Matt: Was .NET on your horizon at that time? Andy: Initially we didn’t know what .NET was going to be, but then I heard Bill Gates talk about it at the PDC 2000, and it became obvious this was a set of tools that would let us rapidly produce cheap front-ends to legacy systems with BizTalk. Now all our developers have to specialise in both .NET and BizTalk. Matt: What was it about the first version of BizTalk that was new – what could it do that other products couldn’t? Andy: There’s a great website called EAI Patterns (www.eaipatterns.com). It’s a bible for integration, laying down all the things you need for a messaging or integration hub that exposes the content of back-end systems to users. BizTalk 2000 fulfilled all of these requirements, and let us build solutions very quickly without having to build components to validate this document, or access that file. Schema validation, encryption, decryption, transformation, decisions about getting data from one place to another, and making decisions based on that data - all these facilitators and toolsets are provided for you. Matt: It’s all XML-based, isn’t it? Andy: Yes pure XML. You bring in whatever data format you like from outside, but once inside it’s all XML. Matt: What struck me when .NET and BizTalk were announced, was that they were two totally separate worlds – on the one hand building .NET apps, using web services, and so on; on the other, BizTalk. Andy: They were in those days. I remember sitting with a beta of BizTalk and finding that I had to go back to C++ to overcome limitations in Visual Basic 6. Then along came .NET, but there was still a gap. We found that COM Interop could solve the problem, so we became Interop experts. Then we looked at BizTalk again and wished that all these tools were inside Visual Studio. BizTalk 2004 Matt: So what’s new in BizTalk 2004? Andy: For a start it’s what Biztalk should always have been: fully integrated into .NET. It’s a .NET application; the toolset sits inside VS.NET; and it produces .NET assemblies which you deploy in the standard way. The mapper’s come across too, but you are no longer limited to just VBScript or JScript running under the functoids: you can use compiled C#. The real plus, however, is its use of XSD and proper namespacing. Before this version, BizTalk was really two separate products - BizTalk Orchestration and Biztalk Messaging - with a little crossover. Matt: And you can do all this in Visual Studio .NET? Andy: Yes - it’s a separate tool, but you can call it from the Visual Studio .NET environment. Matt: This sounds very reminiscent of the business information features of SQL Server 2005 [codenamed ‘ Yukon’], with the OLAP cube support. Andy: That’s some way off, but even now you can see that the next version of BizTalk, when sat on SQL Server 2005, will take what we’re getting in BAM now and make it look trivial. The downside is that the more you give to senior management, the more they want! Matt: How are you distinguishing between human workflow and business process management? Andy: Business process management is about shifting data in a controlled way between automated systems. Human workflow means data flow that is the result of human decision. Human workflow sometimes gets confused with document workflow, but it’s not that either: documents are much bigger than messages, which is what BizTalk is about. In BizTalk 2004, we have the fledgling version of what human workflow might look like in BizTalk in a few years time. Matt: What else is noteworthy about BizTalk 2004? Andy: Its support for InfoPath and Office tools are big plus factors. We also have more facilitators - more tools on the toolbelt. Matt: Do you think developers will cope easily with the added complexity of BizTalk 2004? Andy: We understand BizTalk 2004 because we live and breathe it, but for companies that have just done one project in an earlier version, it can be a nightmare. People must have really good .NET skills to understand it, and with product take-up very high, it’s difficult to recruit those skills at the moment. My fear for BizTalk 2004 is that there won’t be enough people with the right skillset to meet the demand. Service-Oriented Architecture Matt: Will the point release have new features? Andy: There are a couple of things, but no major shifts in what’s happening. Also tied in will be the launch of Indigo, which is the codename for the GXA [Global XML Web Services Architecture] support in Longhorn, covering secure web service transactions - that sort of thing. An Indigo adapter is planned for BizTalk, and that’s important because all the things that are wrong with web services, such as transactions and security, are now being addressed. Matt: To your mind, does service-oriented architecture [SOA] specifically imply web services? Andy: It does in Microsoft-speak. In my mind the question is, “What am I trying to provide?” If you’re trying to build systems that are easy and simple to use, then web services leap to mind, but at the moment the technology doesn’t have enough support for things like transactions across web services – it’s not quite there yet. Service-oriented architecture is the way forward, which is why Indigo and Longhorn are critical. Then we’ll have BizTalk with Indigo adapters, exposing messaging pipelines as web services. Matt: What’s the difference between a message-oriented architecture and SOA? Andy: They’re really the same thing. Some people say that SOA has to be fronted by a web service based on SOAP, and so on. Message-oriented architectures aren’t so tightly defined. To me, it’s irrelevant – it’s the solution that matters. Matt: Finally, can you give us an example of how this all fits together in practice? Andy: Take a CRM system, an ERP system and Microsoft Commerce Server. Say I work for a company supplying surgical goods. I’ve got an InfoPath form, and a customer wants a quote for supplies. The CRM system is called by the middle tier and supplies the customer’s business history, credit rating etc – this is ‘information working’ territory, as Microsoft calls it. The ERP system provides pricing and so on, and the Commerce Server supplies the discount levels. SolidSoft was set up by CEO Garth Pickup and Liam Kelly, Sales Director and Strategy Officer, some 11 years ago. The company concentrates on, and has a strategy of working at the cutting edge of Microsoft technology. 
Matt: What made BizTalk worth looking at?
We looked around and saw players such as IBM and Tibco in the £1 million-plus project field. We guessed that companies would want something in the Microsoft box region, and BizTalk seemed to be it.
The downside of BizTalk 2000 was that we hit its limitations very quickly – we saw that there was going to be a performance overhead, for example, and there was little documentation.
You need to look into BizTalk to see its virtues though. I remember showing my wife a demo I’d built in which I dropped a file in one directory and it appeared in another. “That’s easy, ” she said, “you can just drag the file across the desktop.” What she didn’t realize was that the data inside the file had been changed by BizTalk in the process.
And it gets better the further you go. You open up the Orchestration side, and there’s the missing link. You start to see how you can build a high-level solution very quickly, and present it in a way that’s understandable to business people. That’s all the analyst needs to give to the development team for them to develop into a full-scale system. The analyst can stay focused on the business solution.
Orchestration was the biggest buzz in BizTalk 2000 and 2002, but it was also my biggest frustration because of performance and service-specificity. It was also a nightmare to debug, but I could see the potential. For example, local authorities in the UK have information in literally hundreds of different systems. We can bring that information into one place, data-cleanse it with products such as VisionWare’s MultiVue, and produce a ‘virtual citizen’, which is the aim of all councils at the moment. We’ve suddenly got a very easy way of showing how we can change these systems, with tools that are very rapid to use, because all the difficult bits, dealing with plumbing, security, error handling and so forth, are in there already.
We also wished that Microsoft had waited six months for the XML Schema specification to solidify, and not given us XDRs, and with them the XML Schema to XDR conversion issue. BizTalk 2004 supports XSD, as promised, but it was very close to going into BizTalk 2002 as a point release.
We looked at different ways to use the message-oriented architecture in BizTalk 2002, and it became obvious that publish/subscribe was a very flexible model. If I publish some data, such as a purchase order, to a central message box, and my orchestration subscribes to that message, then that business process can see the message arrive, pick it up, and process it. BizTalk 2004 uses publish and subscribe, and orchestration is an integral part of the process. Furthermore, a business process can be exposed as a web service; giving you a service-oriented architecture.
BizTalk 2004 gives you a much more powerful engine, with all the benefits from earlier versions, such as analyst view. There’s also new features such as a rules engine which allows you to build rules outside the business process, and in languages and vocabularies that people understand (e.g. ‘if purchase order greater than £1,000 then don’t sell it’). This means that the company can change the rules in the rules engine without touching the actual plumbing. The rules engine is very much as orchestration was in BizTalk 2000, in that you can take it out and use it without BizTalk.
The second big problem with earlier versions of BizTalk is that there was no way of doing anything if something went wrong. You set up a schema, validate the input, and if it fails it goes into the suspend queue - and that’s it. Now in BizTalk 2004 we have HAT (Health and Activity Tracking) which means that developers can see what’s gone wrong with messages, and even debug business processes with breakpoints. That increases the complexity of what you can do.
Techies like Biztalk, but we sell business solutions. When business people realize that all their data is passing through BizTalk’s messaging architecture, they want to see a picture of it at a business level. For example, when French Connection and House of Frazer were sending data to each other, one executive wanted to know how well a particular style of jean was selling. That was difficult in BizTalk 2002, but now we have the BAM (Business Activity Monitor) which will spin up an OLAP cube. It’s the corollary of HAT, but on the business side. It can send the result to an Office tool such as Excel, and it can do it in static or live mode.
One thing people started to do with BizTalk 2002 was leveraging Office; for example using Excel as an input device. According to urban myth, the heads of the Office and BizTalk teams met up at a Redmond coffee counter. BizTalk’s Scott Woodgate asked, “What are you doing in Office with XML?” His Office counterpart replied, “We’ve got this cool form design tool called InfoPath that can design web services.” This gave Scott the idea of using InfoPath with BizTalk. Then he had another idea: to use it with human workflow.
People think of BizTalk’s Orchestration as a human workflow tool, but it’s not: it’s business process management. You can make it into one, but it’s not the best one around.
One major item that’s been addressed is the issue of transport adapters. When we encounter a new legacy system we need to construct an adaptor: an interface between BizTalk and the legacy API which will allow us to get data in and out. BizTalk 2004 has a toolkit for that which comes with all the bells and whistles, and is .NET-oriented.
I was at an early BizTalk 2004 conference where ISVs were complaining that Microsoft were making it so easy to create adapters that companies like SolidSoft would end up creating PeopleSoft adaptors! I told them it wasn’t going to happen, because PeopleSoft still have the specialist knowledge of what the adaptor would have to adapt to.
Compared to earlier versions, what we have now is a chocolate box. The downside is complexity, which is an order of magnitude greater. It’s still a rapid integration toolset though, and Microsoft is still selling it that way. The biggest downside is the abysmal documentation, but Microsoft is addressing that now with quarterly updates.
The InfoPath form is creating a proposal document which will be dropped into SharePoint so that it can go through the approval system. The user doesn’t care where the data came from, while the back-ends systems don’t care who’s asking for it. The middle tier is running in a message-oriented way synchronously or asynchronously) filling in the data that’s missing, and BizTalk is gluing it all together. And if, say, the ERP system is replaced, then it’s just a small bit of pipework to hook the new one into the process. That’s what makes BizTalk 2004 so important.
SolidSoft started working with the BizTalk Framework Jumpstart Kit in 2000, and then with the first alpha versions. By the time BizTalk 2000 and 2002 were available, SolidSoft’s biggest project was the UK government’s Gateway, of which Liam Kelly was one of the chief architects. Other major clients include Securicor, Tescos and the Ministry of Defence.
Click here for our Privacy Statement. Copyright © Matt Publishing. All rights reserved. No part of this site may be reproduced without the prior consent of the copyright holder.
