Disclaimer

This document is only collection of author’s notes, experiences and point of views. It is not, in any meaning, either complete description of the topic nor official RTB documentation. It may be inaccurate, incomplete, obsolete, misleading or completely wrong. It may even cause loss of data or damage system integrity. It may not comply with company codex, values, presentation style or economic interests or may reveal company secrets.

As such, it is absolutely prohibited to distribute this document outside of RTB & Co. GmbH. Such an action may result into legal acts against both the sender and the company. It is only intended, after review of technical and presentational correctness and accuracy, to be used as an information source for official documentation.

Always contact documentation department for information about current presentation style and allowed formats before creating customer’s documentation.

1. Overview

This feature is designed to allow transfer of arbitrary data from PDM to PDM.control. It can be used for example to request current configuration, log files, printer logos, fonts …​ It is based on real time request. It allows us to clearly read request number instead of guessing request from mnemonic combination.

Remember that characters in text field which are used as control characters (e.g. ;) must be URL-encoded. E.g. text/plain; charset=UTF-8 must be send from PDM as text/plain%3B charset=UTF-8.

2. Specification of file to download

User express intention to transfer a file by sending a real time request to a PDM.

TBD: @Dragan?

3. Data transfer

All data are transferred inside real time request frame. Therefore, standard mnemonics which are expected outside of the real time request frame like PSA, GAC are not explicitly mentioned by every example.

Data transfer is split to multiple fragments which are send individually until whole transfer is completed. Such a transfer may be suspended and resumed over time and may interleave with other PDM messages.

Parameters to control the transfers are described in chapters below.

3.1. Request mnemonics

Despite of standard mnemonics like PSA, GAC, CRC following additional JSON fields are used.

3.1.1. File name FTN (required)

Name of the file to be presented to a user. Name can be prefixed by forward slash (/) to add category. E.g. name logs/payment.txt means file payment.txt in category logs.

PDM must provide file name in each message. Message without file name will be ignored.

File name must be unique within the PDM.

Group is only meant to distinguish different file groups in UI for better overview. It is recommended to utilize them.

3.1.2. File open mode FTO (optional)

Defines how to open a file to store this fragment. Following modes are defined:

/**
 * Defines how to open a file for writing of a data part.
 */
enum OpenMode {
	/** Append data to existing file or create new one. */
	APPEND(0),
	/** Create new file or truncate existing file and append data. */
	CREATE(1);
}

If the value is not provided mode 0 is used.

Please not that if a mode 1 is used the part number must be 1. Other part numbers will be rejected without storing it. See chapter Errors.

3.1.3. Transfer status FTA (optional)

Update status of transfer.

Following modes are defined:

/**
 * State of data transfer.
 */
enum TransferStatus {
	/**
	 * Transfer is in progress. PDM must use this value to indicate that there
	 *  will another parts of this transfer follow.
	 */
	IN_PROGRESS(0),
	/**
	 * Transfer was successfully completed. All data has been sent.
	 * PDM must use this value to indicate, that there will be no more parts.
	 */
	FINISHED(1),
	/**
	 * Transfer was aborted on user request.
	 * PDM must send this value to indicate end of transfer.
	 */
	ABORTED_USER(2),
	/**
	 * Transfer was aborted by PDM, because of unrecoverable error.
	 * PDM must send this value in case of protocol error
	 * or in case when it is not possible to continue with the transfer.
	 */
	ABORTED_ERROR(3),
	/**
	 * General abort for all transfers.
	 * All of them which are in state {@link #IN_PROGRESS} must transit to {@link #ABORTED_ERROR}
	 */
	ABORT_ALL(4);
}

If the value is not provided mode 0 is used.

Please note that it is possible only to add data to active transfers (0). As soon as PDM marks transfers as complete != 0 it becomes read only and it is not possible to modify it. In such a case an error will be reported to PDM. See chapter Errors.

3.1.4. Content type FTC (optional)

Defines what kind of data are present. This helps a user to correctly indemnify and open the content.

Default value is application/octet-stream which means unknown data. For text data the value text/plain should be used. In case that encoding is known use more specific variant like text/plain; charset=UTF-8.

PDM must use proper mime type based on sent content.

It is enough to send it in the firs message from the series. Subsequent send will modify existing value.

3.1.5. Creation time FTT (optional)

ISO date time which will be used as file creation timestamp. If not send current date and time is used.

It is enough to send it in the firs message from the series. Subsequent send will modify existing value.

Examples: 2022-04-01T10:10, 2022-04-01T10:10:11.745

3.1.6. Total file size FTS (optional)

Total size of transferred file in bytes. It is only used in UI to show download progress and estimated time to finish. If not provided UI element is not available.

It is enough to send it in the first message from the series. Subsequent send will modify existing value.

3.1.7. Transfer encoding FTE (optional)

Defines what encoding is used to transfer binary data. It may have following values:

/**
 * Type of encoding of transferred data.
 */
enum TransferEncoding {
	/**
	 * No encoding is used.
	 * High data density (~0% overhead).
	 * Data are considered to be <b>UTF-8</b> encoded text.
	 * <p>
	 * Be sure that protocol sensitive characters ({@code ;}, {@code 0x01}, {@code 0x03}) are replaced by URL encoded equivalent.
	 * </p>
	 *
	 * @see https://en.wikipedia.org/wiki/Percent-encoding
	 */
	TEXT_UTF8("text"),

	/**
	 * Standard WEB encoding. Safe for all network elements.
	 * Lower data density (~35% overhead) Base64.
	 *
	 * @see https://en.wikipedia.org/wiki/Base64
	 */
	BASE_64("base64");
}

If the value is not provided mode base64 is used.

It is enough to send it in the firs message from the series. Subsequent send will modify existing value.

3.1.8. Data FTD (optional)

The data to transfer. If data is not provided, only file attributes will be modified.

3.1.9. Total fragments FTY

Total number of fragments which will PDM eventually send. It is not necessary to repeat this in every fragment. Indicating it at the first fragment is enough.

If PDM finds out that it needs totally to send more or less packets it can adjust this value by providing new one any time.

3.1.10. Fragment number FTZ

Serial number of the fragment. Number must go in series from 1 to value of FTY. Any packet that comes in unexpected order will be reported as error.

See chapter Expected fragment number to see how the expected fragment number is reported.

PDM must follow required part number. If PDM can not continue by requested part it can either

3.2. Examples

Following examples documents only real time request part. Standard stuff like

PSA1027;GAC11;MRQ:{...};CRC123

will not be repeated. Only the part …​ is described in examples bellow.

It is not necessary to quote json keys, although examples below use them.

3.2.1. Transfer of unknown data of unknown size

Smallest possible request can define only file name and fragment number. This indicates binary transfer of unknown size. PDM control receives data as long as transfer status FTA != 0`.

Undefined parameters get their default values. Content type is application/octet-stream and transfer encoding is base64.

Smallest possible request
{"REQ":30, "FTN": "cfg/config1.pdm", "FTZ": 1, "FTD":"base64 data"}

Eventually the transfer must be closed either successfully or unsuccessfully.

Successful transfer
{"REQ":30, "FTN": "cfg/config1.pdm", "FTZ": 7, "FTA": 1, "FTD":"base64 data"}
Unsuccessful transfer
{"REQ":30, "FTN": "cfg/config1.pdm", "FTZ": 7, "FTA": 3}

See chapter transfer status for all possible values.

3.2.2. Transfer of known size binary with base64 encoding

PDM may indicate size of data transfer either in fragments of bytes or both. Encoding FTE is not mentioned because base64 is default value as well as application/octet-stream is default value for FTC.

Start of transfer of 480 B long binary data split to 5 fragments
{"REQ":30, "FTN": "cfg/config1.pdm", "FTZ": 1, "FTY": 5, "FTS": 480, "FTD":"base64 data"}

Subsequent fragments do not need to repeat size information. It can be only like this.

Subsequent request
{"REQ":30, "FTN": "cfg/config1.pdm", "FTZ": 2, "FTD":"base64 data"}

3.2.3. Text data

Text data can be transferred in two modes.

Encoded

Transfer encoding FTE must be set to base64. Text data such as log files should set properly content type to text/plain for unknown text encoding. If text encoding is known it must be specified after semicolon, e.g. text/plain; charset=UTF-8.

It is also possible to let content type set to application/octet-stream (default). In such a case the file will be normally transferred however some information like text encoding may be lost. Additionally, WEB browser may handle such a content incorrectly.

Text data encode base64
{"REQ":30, "FTN": "cfg/file1.txt", "FTZ": 1, "FTY": 1, "FTS": 7, "FTA": 1, "FTC":text/plain; charset=UTF-8, "FTD":"UMWZw63FoWVybmE="}

3.2.4. Plain

Plain text transfers data without additional overhead. It may by used for log files and other UTF-8 text files.

This mode supports only content type text/plain; charset=UTF-8. Setting value of parameter FTC has no effect. It is due to limitation of transport layer, which expects all text as UTF-8 encoded.

Be sure that data inside parameter FTD which can break communication protocol like ;, 0x01, 0x03 are properly URL-encoded and quotes are escaped.

If there is a need to transfer text file with UTF-8 text encoding, transfer encoding may be set to text. It brings no transfer overhead and achieves higher throughput.

Text data without encoding
{"REQ":30, "FTN": "cfg/file1.txt", "FTZ": 1, "FTY": 1, "FTS": 7, "FTA": 1, "FTD":"Příšera"}

3.2.5. Transfer abort

PDM may decide to abort the transfer. If there is a request to abort from PDM.control user PDM must send value of FTA = 2. If it is not possible to continue from technical reasons PDM must send value of FTA = 3.

See chapter transfer status for more details.

3.3. Transfer recovery

Sometimes it happens the response from PDM.control is not properly delivered to PDM. It may be caused by modem error or unreliability of mobile network.

To handle this case PDM may repeat last fragment. PDM.control will verify fragment number and its hash. If fragment is exactly the same as the one received last time, it will be silently discarded.

Diagram
Figure 1. Suppressing of repetition of last packet

PDM may repeat last fragment as many times as it wants as long as it is the same.

3.4. Response mnemonics

Despite of standard mnemonics like PDM, GAC, CRC following additional once are used.

3.4.1. User cancel

User may cancel transfer from PDM.control side. It is indicated by field abort. Its value represents reason why transfer is aborted. See Errors for all possible values. E.g. abort: 2 means that transfer is aborted because of user request.

In such a case, PDM must send only closing fragment with FTA = 2.

3.5. Response

3.5.1. Errors

Errors are reported in -TFR- mnemonic.

Please see following error definition:

	/** Mnemonic for transfer abort . */
	public static final String ABORT_MNEM = "abort";
	/** Abort reason: non-recoverable error. */
	public static final int ABORT_ERROR = 1;
	/** Abort reason: user canceled the transfer. */
	public static final int ABORT_USER = 2;
	/** Abort reason, file is already closed. */
	public static final int ABORT_CLOSED = 3;
	/** Abort reason, PDM sent unexpected part number. See {@value #NEXT_PART_MNEM} for expected part. */
	public static final int ABORT_UNEXPECTED_PART = 4;
	/** Abort reason, declared transfer encoding failed to decode the data. */
	public static final int ABORT_DATA_DECODING_FAILED = 5;

3.5.2. Expected fragment number

PDM.control always send back number of part which is expected to be received.

Please see following definition:

	/** Mnemonic to indicate which part must be send as the next one. */
	public static final String NEXT_PART_MNEM = "next";

4. Visualization

Go to PDM page and open software section. If there are some user files available, they are displayed there. Only files which was completely transferred are available for download.

During download a progress bar can be shown. Its availability depends on the fact whether the PDM provide a total size of file to be transferred.

It shows all available files, for which the download was already closed. Use can download or delete such a file.

5. Examples

Following section shows example of PDM messages

5.1. Base64

5.1.1. UTF-8 text file.

Transfer of text file split into 3 parts.

UTF-8 text with base64 encoding.
PSA10027;GAC00011;DTM2022-04-06T09:26:03;TRC023766;MRQ[{REQ:30,FTN:"test.txt",FTO:0,FTC:"text/plain%3B charset=UTF-8",FTT:"2022-04-06T09:23",FTS:27,FTE:"base64",FTD:"UMWZw63FoWVybsSbIA==",FTY:1,FTZ:3}];
PSA10027;GAC00011;DTM2022-04-06T09:26:03;TRC023767;MRQ[{REQ:30,FTN:"test.txt",FTE:"base64",FTD:"xb5sdXTDvSA=",FTY:2}];
PSA10027;GAC00011;DTM2022-04-06T09:26:03;TRC023768;MRQ[{REQ:30,FTN:"test.txt",FTA:1,FTE:"base64",FTD:"a8WvxYgu",FTY:3}];

Result should be a file test.txt with UTF-8 encoded text "Příšerně žlutý kůň.".