High Performance OPC UA Server SDK  1.4.1.263
Address Space Generation

The SDK contains two tools for address space generation from XML NodeSet file.

  • xml2bin: Generates a binary file that can be loaded at runtime.
  • xml2c: Generates C code which can be compiled into the application.

Background

To understand the possibilities and restrictions if the SDK's address space model, it is necessary to first understand the OPC UA model and the implemented data structures used to store this information.

The OPC UA Address Space Model consists of an undirected graph of nodes which are connected using typed references. Beside the type information references contain additional meta information like IsAbstract and Symmetric. Also references that have a direction from the OPC UA point of view (Symmetric=true), can be browsed in reverse direction, so from the graph point of view this is still a undirected edge. It is also important to note that this graph can contain cycles.

Important properties of the graph:

  • Every node can contain one or more references which connect it with other nodes.
  • Each reference has exactly one source node and one destination node.

Important properties made by the SDK:

  • The source node owns the reference. This means of the source node gets deleted also its references get deleted. This is important for memory management.
  • The destination node is referenced by a reference, but does not own the reference.
  • Every node contains two single linked lists of references. One for the outgoing references, and one for the incoming references. The first list contains the references that are owned by the node, the second list is used for inverse browsing.
  • Adding a reference means:
    • adding the reference in the source node's outgoing reference list
    • adding the reference in the destination node's incoming reference list
  • Removing a reference means:
    • removing the reference from node's outgoing reference list
    • removing the reference from the destination node's incoming reference list

This can be best shown if the following simple node model:

refmodel.png
Address Space Model

Nodes and references are allocated in object pools which are managed by ua_addressspace. For each namespace index there exists a separate pool. This allows to eliminate redundant nsidx information.

A pool can be seen as an array (or table) of a structure (e.g. node or reference). No pointers are used, instead indices into those table entries create the "pointers" to other elements. For this reason the tables are address independent and relocatable in memory.

It is technically possible to store those tables in memory mapped files, or to generate C containing the table information, like it is done by xml2c.

When loading a binary file the tables are created at runtime with information from the file, thus the tables exit in RAM.

Generating Binary Files

Binary address space files are very compact files compared to their XML counterparts and way more efficient to load. This means it uses just a fraction of the memory required for XML parsing and it is much faster. The file format is described in OPC UA Binary File Format.

Invoking xml2bin

The possible command line arguments can be listed by calling the tool with the option -h as usual: ./xml2bin -h. The help also gives you one simple example.

Usage: xml2bin [-h] [-o <outputfile>] [<base xml files...>] <xml file...>
-h: print this help
-V: print version information
-s: strip unused type nodes (experimental)
-o: output file name
-a: export all namespace (except ns0)
-e <val>: select extensions to export where <val> can be 'none', 'default', or 'all'.
none: no node extensions will be exported.
default: all known Unified Automation extensions will be exported.
all: All extensions will be exported. Unkown will be stored as XML extension also in the binary file
so that they can be handled on application level.
-i: specifies a SDK nsidx to export
-u: specifies an namspace URI to export
-v: more verbose output
-b: byte string limit in byte (default = 4 MB = 4096 KB = 4194304 Byte)
-p: print addressspace
-P: placeholder file name
-l: default locale
<base xml files>: must be specified in correct order, starting with the base model,
followed by models built on top of the previous model.
<xml file>: The information model which gets processed.
Example: Convert the DI model to binary.
xml2bin -i2 -v -o di.bin Opc.Ua.NodeSet2.xml Opc.Ua.Di.NodeSet2.xml

Important Notes:

  • You need to list all NodeSet XML file in correct order which are required to load the XML file which you want to convert. In the example above this was the DI model, which is based on standard UA model (which is always required). E.g. PLCOpen whould require DI, which again requireds UA, so the list of files would be: Opc.Ua.NodeSet2.xml Opc.Ua.Di.NodeSet2.xml Opc.Ua.Plc.NodeSet2.xml
  • -o: Specifies the output filename e.g. 'di.bin'.
  • -i select this namespace to be exported. The namspaces are normally numbered that way:
    • 0=UA (always)
    • 1=UA Server Diagnostics (always)
    • 2=First custom model (e.g. DI)
    • 3=Second custom model (e.g. PLCOpen) If unsure just call the command without -i. It will print the namespace table.
$> ./xml2bin Opc.Ua.NodeSet2.xml Opc.Ua.Di.NodeSet2.xml Opc.Ua.Plc.NodeSet2.xml
SDK Index XML Index Prefix URI
0 0 ua http://opcfoundation.org/UA/
2 2 ns2 http://opcfoundation.org/UA/DI/
3 1 ns3 http://PLCopen.org/OpcUa/IEC61131-3/

Important here is the SDK Index, which is the index that will be used in the SDK and must be unique. The columne XML Index, shows the numeric index used in the XML file which is not unique across files. Prefix is used only for code generation. URI is the unique URI of the information model. Instead of selecting the SDK Index using -i you can also select the model using the option -u and the URI string.

Inspecting Binary Files

The uafileformat library (src/uafileformat) contains some test applications.

  • fileinfo: Simply output information from the file header to get the number of nodes, references, etc.
  • filetest: Is an example application, which loads the complete file and can print its contents to console. Because this is no UA Server application the contents doesn't need to make sense. E.g. referenced types may not exist. This can be used to inspect the binary file contents which maybe cannot be loaded in real servers. By default it will print any error in loading the file to stderr, if the file could not be parsed properly, but not the complete file contents. By using the option '-v' (verbose output) it prints the complete file dump.

FileInfo Example:

$> ./fileinfo ns0.bin
File version: 1.5
File statistics:
number of global extensions: 0
number of required namespaces: 0
number of provided namespaces: 1
number of string tables: 1
number of data types: 130
number of variable types: 37
number of object types: 121
number of reference types: 27
number of variables: 1284
number of objects: 265
number of methods: 144
number of views: 0
number of references: 4813
number of strings: 0
number of guids: 0

FileTest Example:

$> ./filetest -v ns0.bin
Adding extension namespace 0:'extension://unifiedautomation'
Adding stringtable 0 for locale '' with 1309 strings.
Adding string for table 0: ''
Adding string for table 0: '<AdditionalGroup>'
Adding string for table 0: '<ClientName>'
Adding string for table 0: '<FileDirectoryName>'
Adding string for table 0: '<FileName>'
Adding string for table 0: '<NamespaceIdentifier>'
Adding string for table 0: '<VendorCapability>'
Adding string for table 0: 'A URI that uniquely identifies the dictionary.'
Adding string for table 0: 'A base type for a user identity token.'
...
Adding node id=i=11939, nodeclass=DATATYPE, bn=0:OpenFileMode, dn=OpenFileMode, desc=
datatype attributes:
is_abstract: false
definition: Enum
num_fields: 4
fields:
Read -> 1
Write -> 2
EraseExisting -> 4
Append -> 8
...
Adding reference i=30 -> i=2001 : i=45
Adding reference i=30 -> i=2002 : i=45
...
File statistics:
header: 37 bytes
extensions: 22 bytes
stringtables: 36926 bytes
namespaces: 50 bytes
datatypes: 5779 bytes
referencetypes: 425 bytes
variabletypes: 696 bytes
objecttypes: 1689 bytes
variables: 377587 bytes
objects: 4000 bytes
methods: 2010 bytes
views: 0 bytes
references: 35729 bytes
# of strings: 1309
Successfully loaded file.

Testing Binary Files

To see if a binary file works and looks like expected in a real server you can use the provided demo server. By default the demo server loads the "demo" model, but by using one or more -m commandline switches you can load other models instead of the "demo" model.

Example: Loading DI Model and another model which requires DI.

$> ./uaserverhp -d32 -m di.bin -m mymodel.bin -t0
N|8|18:06:28.976115|6213| uaapplication_load_certificates: loaded cert file://hpsdk4096.der
N|8|18:06:28.976228|6213| uaapplication_load_certificates: loaded cert file://hpsdk2048.der
N|11|18:06:28.980476|6213| Registering dynamic address space: http://opcfoundation.org/UA/
N|11|18:06:29.016278|6213| Registering dynamic address space: urn:UnifiedAutomation:DemoServer:ws-gergap
N|11|18:06:29.019105|6213| Registering dynamic address space: http://opcfoundation.org/UA/DI/
N|11|18:06:29.023444|6213| Registering dynamic address space: http://www.acme.com/mymodel/
N|9|18:06:29.023488|6213| Node 00000006 contains the Unified Automation extension XML(8).
N|9|18:06:29.023499|6213| XML extension: '<myextension xmlns="http://www.acme.com/Foobar"><foo>bar</foo></myextension>'.
Server is up and running.
Listening on opc.tcp://ws-gergap.ascolab.com:4840

Options explained:

Option Description
-m Load given information model.
-d 32 Enable Notice trace level. This is useful to see what namespaces were loaded.
-t 0 Don't wait before shutdown when CTRL-C is pressed.

Loading Binary Files

At runtime you can load binary files simply calling ua_addressspace_load_file. See Sensor Model Server for an example.

Generating C Code

The purpose of generating C code is to reduce the memory requirements of OPC UA Server applications. UA address space can get huge and because they contain a lot of string information, the required memory is too big for many embedded applications.

By generating C code we can move the address space from RAM into ROM, thus reducing the RAM usage.

Note that this is only true for systems that can run code from ROM (e.g. flash memory). Systems that copy all data to RAM before execution cannot benefit from this solution for obvious reasons.

Value Stores

The process data (value attribute) is separated from other node information. This allows to connect dynamic data with static nodes. However it is also possible to define values already in the XML file for constant information like input- and output-arguments of a method.

xml2c generates a static values store for the variables which contain a value in the XML file. The generated node is connected with the correct store entry at compile time by assigning the according store index and value index.

For all other variables an alternative store index is assigned that you can specify as an command line argument. At runtime you need to create any kind of value store and register it for this store index. This way you can connect static nodes with dynamic data.

See Lesson 2: Custom Provider with Value Stores for a detailed explanation of value stores.

Homogeneous Address Space

If the whole address space is generated the table model just works as in RAM, because everything is index based. The only difference is that you cannot modify the address space, because it consists of C constants.

addressspace_1.png
Homogeneous Address Space

Mixed Address Space

There is the typical use case that you have a compiled-in static address spaces for type models like NS0 (OPC UA) and other models like e.g. DI or PLCOpen, but you need to add also nodes at runtime (e.g. instance of such types).

This leads to the problem that you need to add references from compiled-in static nodes to nodes that exist in RAM. But the node tables and reference tables of the source nodes are constants, so this is not possible.

To fix this problem we created the so-called placeholder concept. A static address space can contain - in addition to the static node table and reference table - a further dynamic reference table with placeholder references. Nodes which should be extendable at runtime contain a placeholder reference as last element in the reference list. All the indices from nodes to references, and from the last static reference to the place holder reference are static, and don't need to be changed. The placeholder reference does not have a type or destination node, which marks it as a placeholder. When adding the reference from a static node to a dynamic node the placeholder reference becomes a real reference which points to the newly created dynamic node.

The following image illustrates how the static node "Objects", which contains already a reference to the static node "Server", is connected to the dynamic node "DemoFolder" from NS2 using such a placeholder.

addressspace_2.png
Mixed Address Space

Limitations: When adding references via placeholders this works only for outgoing (forward) references, not for incoming (inverse references). This means if you add e.g. a HasTypeDefinition reference to a static type, this will not be added to the type's inverse reference list. Thus reverse browsing of a type to get all the instances will not work. Because this is not a typical use case must people can live with this limitation.

Invoking xml2c

Calling xml2c works very similar to invoking xml2bin, but requires some additional arguments.

Example:

$> mkdir -p output
$> ./xml2c -n di -i2:di:12 -o output Opc.Ua.NodeSet2.xml Opc.Ua.Di.NodeSet2.xml

The main difference to xml2bin is that this tool not only generates the address spaces, but also C Header and Source files for all types defined in the information model. It also creates a CMakeLists.txt which will create a complete C library for that information model. If the model contains datatypes this will get registered automatically at the SDK's generic encoder/decoder, so that the SDK is able to encode/decode those datatypes. This works out-of-the-box, the only thing you need to to in your application is to linked against this library and call the generated function <prefix>_register_static_addressspace.

Options:

Option Description
-n Specifies the library name used in the generated CMakeLists.txt
-i Selects the SDK Index to export, the prefix, and the alternate store index
-o Selects the output folder where the source files should be generated.

The example command above will generate the following files.

$> tree output
output
├── CMakeLists.txt
└── di
├── devicehealthenumeration.c
├── devicehealthenumeration.h
├── fetchresultdatadatatype.c
├── fetchresultdatadatatype.h
├── fetchresulterrordatatype.c
├── fetchresulterrordatatype.h
├── identifier.h
├── ns2.c
├── parameterresultdatatype.c
├── parameterresultdatatype.h
├── type_identifier.h
├── type_table.c
└── type_table.h

The table below describes the purpose of the different generated files.

File/Folder Description
CMakeLists.txt CMake project to compile the 'DI' library.
di/ Folder containing all sources for the DI model.
ns2.c The constant address space tables.
identifier.h Defines with NodeId identifiers.
type_* Type table code for generic encoder/decoder.
*type.[c,h] Generated code for DI datatypes.

The library will contain all the code to for datatypes and the generic encoder/decoder, but does not include ns2.c! This will needs to be included directly by your provider code as shown in the example Sensor Model Server.

The library gets linked with the application and can be used with static address spaces as well as with dynamic address spaces loaded from file.

Using Custom Datatypes

Because xml2c can generate the C datatypes based on the UA type information and the according type tables for the generic encoder/decoder it makes it very easy for to use custom datatypes.

All you need to do is:

  1. #include the generated type_table.h header file
  2. Call ua_type_table_register_const_table
    /* register custom types */
    ua_type_table_register_const_table(uaprovider_sensor_nsidx, g_type_table_sensor, countof(g_type_table_sensor), UA_TYPE_TABLE_SORTING_ASC);
    Note that xml2c will try to sort the type table by type_id to allow fast lookups using a binary search. However you can always pass UA_TYPE_TABLE_SORTING_NONE to ua_type_table_register_const_table. The function will check the table and detects sorting automatically.

Of course the application needs to link against th generated information model library, to be able to use the generates types and type tables.

Note: This works with dynamic and static type systems.

Limitations: Currently only numeric TypeIds are supported.