File Transfer

Table of Contents

• Background
• Creating File Instances
  • Creating a FileSystem File Instance
  • Creating a Memory File Instance
• Using File Transfer with Existing Nodes
  • Existing Node Example:
  • Using FileTransfer with Existing Nodes of Static Address Spaces
• Adding new File Implementations

The SDK implements the UA FileType as defined by "Annex C File Transfer" - "OPC UA Specification - Part 5: Information Model".

Background

The purpose of the FileType is to be able to transfer file contents, that might be exceeding the maximum message size of UA by adding methods which allow to read and write parts of files. In addition it provides functionality for locking files and read file meta data. See specification for more details on the FileType itself.

To understand the SDK architecture of the FileType you must first understand that the FileType that is visible via OPC UA address space is independent from the technical filetype used in the code. So it is possible that you have multiple instances of the same UA FileType, which use different file implementations. So one instance might be connected with a real filesystem file, another instance might be an in-memory file.

To achieve this the SDK defines a ua_filetype_interface, which is a calltable to the actual file implementation, and a ua_file_context, which represents one instance of a real file. This ua_file_context holds file meta data as well as the ua_filetype_interface to use for this instance. The UA node in the address space, which represents a file, is connected to such a ua_file_context. The type ua_file_handle represents and opened file with its meta data like e.g. the file position.

FileSystem Model

The SDK infrastructure takes care of handling UA file method invocations, locking and cleanup of file resources. E.g. if a session gets deleted or a file is not used for a configurable time (file_timeout), then the file gets closed automatically and the associated file handle gets removed.

Creating File Instances

The SDK provides two default implementations of a FileType.

1. ua_filesystem_implementation : This uses real files and implements access to those files using the platform layer (PL) file API. This means, if your PL provides a working file API, then also ua_filesystem_implementation is working. Because this API depends on the option SUPPORT_FILE_IO, also the ua_filesystem_implementation SUPPORT_FILE_IO depends on this option and is unavailable if this option was disabled.
2. ua_memoryfile_implementation : Represents an in-memory file based on a ua_bytestring.

Creating a FileSystem File Instance

The function ua_filetype_create_ex does two things in one step. It creates the instance of the specified file type (UA_NODE_FILETYPE in this example), and it creates a underlying file context and connects both.

Creating the instance in address space works the same way is ua_instance_new.

1. Create a ua_filetype_ctx instance using ua_filetype_ctx_create, which is used to control the instance creation. Therefor you can register callbacks at this context (ua_filetype_ctx_set*cb), and you can set the file's MIME type using ua_filetype_ctx_set_mime_type.
2. You call ua_filetype_create_ex and specify the file's new NodeId, its type and its implementation calltable. Depending on the used implementation you must specify additional data as the last argument. For the filesystem implementation this is the filename (const char*).
3. You can get the node handle of the new instance by calling ua_filetype_ctx_get_new_node
4. You cleanup the ua_filetype_ctx by calling ua_filetype_ctx_delete.
5. You can add a reference to this node by using ua_reference_add.
6. Optionally you can register callbacks to get notifications when a file is opened or closed. This may be useful if you need to do something after a file was updated.

Code snippet from the demoprovider:

    /* create nodeid for file instance */
    ret = ua_nodeid_attach_string_const(&id, g_uaprovider_demo_dynamic_nsidx, "testfile");
    if (ret != UA_EGOOD) goto error;

    /* create filetype context for ua_filetype_create_ex invocation */
    ctx = ua_filetype_ctx_create();
    if (ctx == NULL) goto error;

    /* optionally, set the file's MIME type */
    ua_filetype_ctx_set_mime_type(ctx, "application/text");

    /* Create the file instance. */
    ret = ua_filetype_create_ex(ctx, &id, UA_NODE_FILETYPE, 0, NULL, "Testfile", &ua_filesystem_implementation, UA_DEMOPROVIDER_TESTFILE);
    if (ret != UA_EGOOD) {
        TRACE_ERROR(TRACE_FAC_PROVIDER, "Creation of filetype for Testfile failed.\n");
        goto error;
    }

    /* get the newly create node handle of the file */
    g_file = ua_filetype_ctx_get_new_node(ctx);
    /* cleanup */
    ua_filetype_ctx_delete(ctx);
    ctx = NULL;
    ua_nodeid_clear(&id);

    /* reference node from existing folde node */
    ref = ua_reference_add(folder_node, g_file, UA_NODE_ORGANIZES);
    if (ref == UA_REF_INVALID) {
        ret = UA_EBAD;
        TRACE_ERROR(TRACE_FAC_PROVIDER, "Creation of reference to 014_Files folder failed.\n");
        goto error;
    }

    /* register callbacks to get notifications for file changes */
    ret = ua_filetype_set_callbacks(g_file, &cb);
    if (ret != 0) {
        TRACE_ERROR(TRACE_FAC_PROVIDER, "ua_filetype_set_callbacks failed.\n");
        goto error;
    }

Creating a Memory File Instance

Creating an in-memory file works essentially the same as the example above, with the following differences:

• You use ua_memoryfile_implementation instead of ua_filesystem_implementation when calling ua_filetype_create_ex.
• The last argument is a pointer to a ua_bytestring, which represents the file data, instead of the filename. You must create this bytestring beforehand.

The following example uses ua_filetype_create instead of ua_filetype_create_ex to demonstrate this. This is just a convenience wrapper function, so that you don't need to create the ua_filetype_ctx yourself. This function is sufficient for most cases, but if you need control over the created NodeIds you must use ua_filetype_create_ex and register the according callbacks.

    /* Create bytestring for in-memory file. */
    ret = ua_bytestring_create(&g_memoryfiledata, 1024);
    if (ret < 0) {
        TRACE_ERROR(TRACE_FAC_PROVIDER, "Bytestring generation for memory file failed.\n");
        goto error;
    }
    /* initialize bytestring with zeros */
    ua_bytestring_clear_data(&g_memoryfiledata);
    /* add some test data */
    ua_memcpy(g_memoryfiledata.data, "Hello World", 11);

    /* Set the string node id for "memoryfile". */
    ret = ua_nodeid_attach_string_const(&id, g_uaprovider_demo_dynamic_nsidx, "memoryfile");
    if (ret != UA_EGOOD) {
        goto error;
    }

    /* Create the memory file. */
    g_memoryfile = ua_filetype_create(&id, UA_NODE_FILETYPE, "Memoryfile", &ua_memoryfile_implementation, &g_memoryfiledata);
    if (g_memoryfile == UA_NODE_INVALID) {
        ret = UA_EBAD;
        TRACE_ERROR(TRACE_FAC_PROVIDER, "Creation memory file failed.\n");
        goto error;
    }

    /* cleanup */
    ua_nodeid_clear(&id);

    ref = ua_reference_add(folder_node, g_memoryfile, UA_NODE_ORGANIZES);
    if (ref == UA_REF_INVALID) {
        ret = UA_EBAD;
        TRACE_ERROR(TRACE_FAC_PROVIDER, "Creation of reference to 014_Files folder failed.\n");
        goto error;
    }

    ret = ua_filetype_set_callbacks(g_memoryfile, &cb);

Using File Transfer with Existing Nodes

The previous examples have shown how to create FileType instances dynamically. But it may be necessary to connect existing nodes with a ua_file_context. This is the case when loading information models from file which contain already such FileType nodes.

In this case you can use the function ua_filetype_create_with_existing_node as shown in the following example.

Existing Node Example:

    ret = ua_nodeid_attach_string_const(&id, 2, "ExistingFileNode");
    if (ret != UA_EGOOD) {
        goto error;
    }
    g_existing_file = ua_filetype_create_with_existing_node(&id, &ua_filesystem_implementation, UA_DEMOPROVIDER_EXISTINGNODE_FILE);
    if (g_existing_file == UA_NODE_INVALID) {
        ua_nodeid_clear(&id);
        TRACE_ERROR(TRACE_FAC_PROVIDER, "ua_filetype_create_with_existing_node failed.\n");
        goto error;
    }
    ua_nodeid_clear(&id);

The full example code can be found in examples/server_demo/demoprovider/uaprovider_demo_file.c.

Using FileTransfer with Existing Nodes of Static Address Spaces

Using File Transfer with nodes from static address spaces is possible too, but a little bit more complicated. You need to know some implementation details of FileTransfer to fully understand how this works.

Issues:

1. Normally, the FileType instances store the ua_file_context in the node's userdata property, because this is simple and efficient. However, this is not possible with static nodes in ROM (Read Only Memory).
2. The FileType methods can be found in a generic way using your method framework which respects the inheritance hierarchy. But the method implementations need to retrieve the ua_file_context from the FileType instance to work.
3. The FileType properties need to be connected with the ua_filestore so that the property values of the file can be read. These properties cannot be connected at runtime when they are in ROM.

Luckily, we have solutions for these issues as well:

1. The SDK can associate static nodes with file contexts by using a separate data structure in RAM. This does not change anything for dynamic nodes, but for static nodes you need to ensure that this data structure is large enough. To change the size you need to edit the file src/uaserver/filetransfer/assoarray.h and change the value of the define UA_MAX_FILETYPE_ASSO_ENTRIES. Normally, you don't need many FileType instances so the default value is 10.
2. This problem is resolved automatically with the solution of 1).
3. Associating a static variable with a specific store implementation is a general problem, that is resolved by using the address mapping feature of xml2c.

For this to work you need to know that the ua_filestore implementation uses the reserved store-id 244 (see define UA_VALUESTORE_IDX_FILETYPESTORE in src/uaserver/valuestore/valuestore.h). An example of such an address mapping can be found in the file examples/server_demo/addrmapping.txt and is shown below:

# Address Mapping for NodeIds to Store/ValueIndex
# Syntax: <NodeId>\t<StoreIndex>:<ValueIndex>\n
# <NodeId>: NodeId String encoding like generated by ua_nodeid_to_string()
# <StoreIndex>: Numeric StoreIndex that is used as-is
# <ValueIndex>: Numeric ValueIndex that is used as-is, or '*' which will be used to auto-increment values.
# Note: Nsidx values must be SDK nsidx values, not XML nsidx values.
#
# The demoserver.xml namespace uses these store indeces:
# 3: default static store (is always nsidx+1)
# 244: SDK FileStore
# 12: alternate store index (for memory store)
# 13: pointer store
ns=2;s=Demo.SimulationSpeed     13:*
ns=2;s=Demo.SimulationActive    13:*
# Trigger_BaseEvent
ns=2;i=6254                     13:*
# Trigger_SampleEvent
ns=2;i=6255                     13:*
# ExistingFileNode
ns=2;s=ExistingFileNode.OpenCount    244:*
ns=2;s=ExistingFileNode.Size         244:*
ns=2;s=ExistingFileNode.UserWritable 244:*
ns=2;s=ExistingFileNode.Writable     244:*
# TrustlistFile
ns=0;i=12643 244:*
ns=0;i=14157 244:*
ns=0;i=14158 244:*
ns=0;i=12646 244:*
# Trustlist Lastupdatetime (not part of base filetype)
ns=0;i=12662 245:*
# PUBSUB
ns=0;i=25452 244:*
ns=0;i=25453 244:*
ns=0;i=25454 244:*
ns=0;i=25455 244:*

With this in mind the same code of the example above works also for static address spaces as demonstrated by our DemoServer.

Adding new File Implementations

If you need other kinds of files you can add your own implementation of the file interface. All you need to do is creating an instance of ua_filetype_interface and set the function pointers of your implementation. Then use this calltable in ua_filetype_create like for the examples above.

The args argument of ua_filetype_create_ex will be forwarded to the create function of ua_filetype_interface. This way you can pass implementation specific data to your implementation. This private implementation details can be stored in the pimpl field of the ua_file_context. So this is available in all the function calls.

To understand this it's best to look at the existing two implementations which are shown in the UML diagrams below.

FileSystem Model

MemoryFile Model