Dataworks Enterprise Cache Configurations

Introduction

The basis of Dataworks Enterprise is the Real-Time Cache component, RTCache. The Cache exposes a set of programmable objects used by the other high-level application components of the system. The various components use these objects to provide application specific functionality.

The cache is configured using a combination of configuration files and registry entries.

The Registry and Directory Structure

Dataworks Enterprise is installed using a standard installation program. During installation, the user specifies an installation directory (by default �C:\Program Files\Primark\Platform�). All of the software, with the exception of the Cache and its help files, are copied to this directory tree. In larger sites, this directory can be a shared network drive to reduce disk overhead. All installed components (including the Cache) are optimised to run in a network installation. When a host loads the binaries, the image is automatically copied to the local swap space of the host reducing network bandwidth caused by swapping.

As with standard system level DLLs, the Cache and its help files are installed in the system directory of each host running the system. To reduce the cost of administration and management, the install program supports both silent installs and SMS.

The content of the installed directory depends on the installation options selected during install. In most cases, only a few components will be installed on any individual host. The install program installs the appropriate files and updates the registry with configuration information. The install program also stores what has been installed and where. The install program can be run more than once for each individual host. This mechanism is used to update an individual host configuration or to upgrade the software. The install program also uses the information it has stored to uninstall the software if required. For these mechanisms to work, it is important that the files in the installed directory are neither amended nor additional files added. The install directory should be seen as a read-only repository of the installed software. Customer configurations and binaries need to be kept elsewhere (and backed up). The software provides mechanisms through the registry to allow the software to locate these configurations (see below). The distribution CD contains the entire set of software as unpacked binaries to assist in the isolation of installation problems.

Under the standard home directory there are two key directories, Config and Bin. The Config directory is used to store default configuration files and the Bin directory the executable code of installed components.

The Registry

The software uses a number of registry entries for the most basic configuration information (typically what is required to boot the system). The location of these entries depends on how the system is to be used. Whenever the software needs an entry from the registry, it first looks in HKEY_CURRENT_USER (HKCU) and if this fails in HKEY_LOCAL_MACHINE (HKLM). In both searches fail, the software has a built-in default. This allows per user configurations to override those of the host. All Thomson Financial registry settings are relative to a sub-key �Software\Primark\Platform�, i.e. either HKLM\Software\Primark\Platform or HKCU\Software\Primark\Platform. We will refer to this key as PLATFORM_REGISTRY. The installation installs the registry hive under HKLM by default.

The following registry entries are available under the PLATFORM_REGISTRY hive:

Launch This key is legacy and related to the original launcher program, which has been superseded by a revised launcher. It contains the list of scripts to be run by the launcher with an indication of the order to perform them in. See the section on the launcher software for more details.
Settings
Bin The semi-colon-separated path list used by Dataworks Enterprise to locate binary files, described in detail below.
Config The semi-colon -separated path list used by Dataworks Enterprise to locate configuration and script files, described in detail below.
ShMemSize The maximum size of the shared memory segment in pages (default around 16000 or 64MB).
MaxMsgSize The maximum size of space for pre-allocated messages in pages (default 126, which is equivalent to around 38000 messages).
RouterTimeout The maximum time in milliseconds to allow a process to be connected to the system without processing messages (default 60000).
Statistics If set to one, Dataworks Enterprise stores detailed operation statistics. These are very CPU intensive and should not be enabled by default. The default value, if the key is missing, is zero.
SystemDB This provides the name of the Cache system configuration file (by default, RTCache.dat).

There is some myth and magic surrounding these values. In order for the cache to communicate between processes, it uses shared memory. There are two pools of this memory, one used for message objects and the other for data. The sizes of these areas has to be configured and cannot be changed while there is an instance of the cache in memory. The consequence of this is that if these values change, you must restart everything.

If there is a slow client or handler in the system, messages can be left in its internal work queues for quite long periods of time. Consequently, it is possible that the cache runs out of memory in either shared pool. If this happens, it will block the thread until space becomes available.

As a rule of thumb these settings should be left alone unless you are expecting very high data rates and large amounts of data. If this is true, then increase the values. We have systems running at 400MB of shared memory and . You should always ensure that the total number of messages is less than the number of pages of data.

The RouterTimeout is the time that the cache will allow a client or a server to be busy before forcefully disconnecting them. In some rare cases, this value needs to be increased. In development scenarios, the checked build of the cache supplied with the SDK disables this behaviour to assist with debugging.

Directory Structure

The software locates files using the Bin and Config values under the �Settings� key of PLATFORM_REGISTRY. Each value is an ordered collection of path entries separated by a semi-colon (the same syntax as the PATH environmental variable). The config values are collectively known as BIN_PATH and CONFIG_PATH. Whenever the software attempts to run a binary or open a configuration file, it will search BIN_PATH and CONFIG_PATH respectively. By default, these paths are set to:

C:\Primark\Platform\Bin;<InstallationDirectory>\Bin
C:\Primark\Platform\Config;<InstallationDirectory>\Config.

The directories C:\Primark\Platform\Bin and C:\Primark\Platform\Config are intended as repositories for users to place their own configuration and binary files. Since these directories are listed before the installation directories, the Cache will search these folders first for component configuration files. This allows users to override any component default settings by copying the default configuration file from the installation directory into the C:\Primark\Platform\Config folder. In this way the configuration for one user is isolated from those of another.

In some cases, it may be better to specify a network drive for these custom folders so that all users are configured using one central configuration file. This can be specified during installation or when creating a silent install script. The installation program is not aware of these custom directories and will not attempt to update or remove any files from them.

Most Dataworks Enterprise components provide a default configuration file that define the settings that users can modify. For example, the RemoteSvr module provides the file RemoteSvr.Dat in the <InstallationDirectory>\Config folder. This configuration file describes and gives examples of how each of the user configurable settings can be modified. Each of the examples are commented out using the "!" character at the beginning of each line. At the end of file there is a configuration entry #include <RemoteSvr.usr>. This instructs the cache to look for user modified configuration entries in this file. Thus users should create the file RemoteSvr.usr in the C:\Primark\Platform\Config folder and copy and amend the desired configuration examples from the .Dat file (not forgetting to remove the "!" character to uncomment the directive) in order to override the component defaults.

The RTConfig.exe component, installed in the <InstallationDirectory>\Bin folder, can be used to modify the BIN_PATH and CONFIG_PATH settings.

Cache Configuration File (RTCache.dat & RTCache.usr)

Cache configurations are read from standard configuration files. The default standard configuration file is "RTCache.dat" although this can be configured in the registry (see above).

The main Cache supports the notion of field name aliases. Aliases hold static configuration associated with particular common fields. For each alias there are a number of general field names which map to the alias. These are presented in the object model as generic field names. Many individual field properties are derived from the generic field name data. Generic field names are configured in the Cache configuration file. The format for an individual configuration is:

	Tosca.Fields.<generic name>.<property>:<value>

For instance:

	Tosca.Fields.BID.HighlightStyle:1

indicates that there is a generic field name called BID whose highlight style property has the value 1.

The AliasNames property of a generic field provides a list of alternative incoming names which use the properties defined for the given generic name. For instance, the property:

	Tosca.Fields.BID.AliasNames:QBP

indicates that fields called "BID" or "QBP" share the same generic properties.

The HighlightStyle property identifies how the field should treat highlight colouring and is defined as:

  • 0 = No Highlight - Don�t do highlighting
  • 1 = Change Last - Highlight using a comparison with last
  • 2 = Change Zero - Highlight using a comparison with zero

    The Symbol attribute defines whether a field is a tick field or not. Tick fields are automatically displayed as arrows when rendered using the Paint method on the RTField object. Source developers should use the value "U", "D" and <space> to represent up, down and no change within tick fields.

    The DisplayName and Abbreviation are static data used to provide displayable names for end users. In many cases the actual field name is non-meaningful.

    The FieldInfo is the RTFieldInfo constant value given to the FieldInfo object. This allows common fields to be tested for some relationship to the record they are contained in such as a "Next Field" link, the "Last" value for the record, and so on. See the definition of RTFieldInfo in the exposed object model for more information.

    The

    	Tosca.Fields.LinkNames
    

    configuration contains a comma-separated list of prefix words that begin the names of link fields in a record. This is used to both identify the link fields of a record for RTFieldInfo properties and to number them if they are chain fields (the prefix is assumed to be immediately followed by the field number).

    The

    	Tosca.Sources
    

    key is used to define static attributes of sources. The format of the configuration is:

    	Tosca.Sources.<source name>.<attribute>:<value>
    

    For instance, the:

    	Tosca.Sources.SelectFeed.Font:ReutersIDN
    

    configuration states that the standard alternate font for the source SelectFeed is called "ReutersIDN".

    The PhysicalNames attribute provides alternative names for the sources in much the same way as the AliasNames property does for fields.

    The Font property provides a source-specific font name. The Font2 property provides the second source-specific font name.

    The Cache has a notion of highlighted updates. These highlights remain until either the update list is replaced or a length of time elapses. The:

    	Tosca.Field.UpdatePeriod
    

    specifies this length of time (in milliseconds). The other:

    	Tosca.Field.<key>
    

    entries provide the default colour for painting. The key values can be:

  • UpTicColor The up tic field colour (default Blue).

  • DnTicColor The down tic field colour (default Red).

  • NCTicColor The no change tic field colour (default Green).

  • FgTicColor The foreground colour (default Black).

  • BkTicColor The background colour (default White).

    The Cache also has a notion of user custom fonts that can be selected using standard escape sequences. The name of these fonts is configured using the:

     
    	Tosca.Encoding.Custom<n>
    

    where n is a number from 1 to 4.