TechEd 99 - IMDB

High speed data access with IMDB
(David Gristwood)

July 9 - In-memory databases (IMDB) are an interesting new service expected to ship with Windows 2000 as part of the COM+ infrastructure. Basically, IMDB is the realisation of what lots of developers and software companies have been doing for a long time - implementing a proprietary caching scheme to get faster access to read-only data. Designing and coding this is not straightforward and can turn out to be really costly in terms of time and effort. With Windows 2000, Microsoft introduces a system level, highly integrated service that provides such a caching mechanism. An explanation and demonstration of IMDB formed the backbone of David Gristwood's presentation.
      IMDB is essentially an OLE DB provider that loads in memory a data source and makes it accessible without direct disk accesses. To take advantage of IMDB you need to write COM+ components that will host the IMDB proxy DLLs in their address space. Such modules make data visible through global shared memory. To access such in-memory data from a COM+ component you can exploit a variety of data access components, from raw OLE DB calls to ADO.
      Using ADO you can read, write and seek data, as well as apply filters. To update data you need to enter a transaction (which is not required if you just want to read data). When handling a transaction, IMDB automatically uses DTC (Distributed Transaction Coordinator). IMDB can also transparently access 64-bit memory and is supported on DEC Alpha machines.
      To register a data source for availability through IMDB you use the COM+ administration tools. You can set up security properties and start-up parameters; start and stop the IMDB as a service; and have problems logged out to the Event Viewer message repository. IMDB security is based on the COM+ security model and the IMDB access is controlled by the access rights of the COM+ application using IMDB.
      There are lots of advantages to using IMDB: less trips to the database, more scalable components, high-performance (especially for read-only data), reduction of out-of-proc calls, and reduction of proprietary and non-interoperating caching schemes.
      A point to consider carefully about IMDB is that it doesn't have a query processor so you can't load just a specific recordset into memory. You can create temporary tables, but you can't load just the records that result from a 'select * from Employees' SQL command. To work around this limitation, you can use the ADO filter command or - better yet - a specialised OLE DB provider to create exactly the dataset you want to keep in memory for high-speed access.
      David Gristwood's presentation included an interesting demo based on a 3-tier application where an IMDB is used to make sensitive data easily accessible. This database is subject to nightly updates, ready for new accesses the next morning.
      To obtain greater performance, careful planning is required. You need to figure out which tables to load, and whether the loading must occur when the service is started or - more simply - on demand from the application. You should also consider using indexes and reducing the amount of data to be loaded. From the application's point of view, an IMDB is as easy to use as calling an OLE DB provider.
      The IMDB provider is called MSIMDB and the name of the loaded table is just a parameter:

Dim cn As New ADODB.Connection
Dim rs As New ADODB.Recordset
cn.Provider = "MSIMDB"
cn.ConnectionString = "datasource=…"
cn.Open
rs.ActiveConnection = cn
rs.Source = "test"
rs.Open

Dino Esposito

Related Links

Article: OLE DB 2.5

Article: Universal Data Access

High speed data access with IMDB (David Gristwood)

High speed data access with IMDB
(David Gristwood)