The reporting system

Several Data Factory tasks generate at the end of their execution a report in the form of an xml file. This is available as an output of the task and may be downloaded from settings.product-live.com application.

Key elements

• The report is generated at the end of the execution of a task.
• The only format available today is xml.
• The produced XML is always non indented.
• Every log has the following properties:
  • A type (info, warning, error)
  • A code (a string that can be used to identify the error)

Limitations

• A maximum of 100 000 logs can be generated per reports
  • If the maximum is reached, no more logs will be added to the report. This will have no effect on the task being executed.

Structure of the report

<Report task="task-name" start-at="iso date" end-at="iso date" duration-ms="" task-cid="" job-cid=""> 
    <!-- List of the task inputs -->
    <Input name="input1">...</Input>
    <Input name="input2">...</Input>
    
    <!-- List of logs -->
    <Log type="info" code="SOME_CODE">
        <!-- List of metadata / metadata collection-->
        <Metadata name="xpath">/foo</Metadata>
        <Metadata-Collection name="identifier">
            <Metadata name="ean">1234567890123</Metadata>
            <Metadata name="sku">00034-red-34</Metadata>
        </Metadata-Collection>
        <!-- user friendly message, useful for debug, but should not be used in automated systems -->
        <Message>here is your error</Message>
    </Log>
</Report>

XPath	Description	Occurrence
Report	root of the xml	1
Report/@task	task name	1
Report/@start-at	Task start date (see)	1
Report/@end-at	Task end date (see)	1
Report/@duration-ms	Task duration in milliseconds	1
Report/@task-cid	The task unique id	1
Report/@job-cid	The job execution unique id	1
Report/Input	Job input values	0..n
Report/Input@name	Job input name	1
Report/Log	A log	0..n
Report/Log@type	The log severity (info, warn error ...)	1
Report/Log@code	The log code	1
Report/Log/Metadata	Contextual info related to the log	0..n
Report/Log/Metadata@name	Name of the metadata	1
Report/Log/Metadata-Collection	A metadata collection is a group of metadata that are related to each other. A metadata collection can contain other metadata collections	0..n
Report/Log/Message	User friendly message	1

the provided message may change in the future

It should not be used in automated systems. Deducing information from the message is not recommended. Instead, use the metadata.

Logs severity

Error

Logs that indicate an error in the process. It does not mean that the task has failed, but that something went wrong. The task may continue to run.

Warning

Logs that indicate a potential error in the process. It does not mean that the task has failed, but that something went wrong. The task may continue to run.

Info

Logs that indicate a normal behavior of the task.

Debug

Use to log information that may be needed for diagnosing issues and troubleshooting.

Common metadata nodes

Code

A string that is used to identify the log type. The same code may be used along dirrerent tasks.

Xpath

The XPATh is provided when the log is related to a specific element in an XML file (usually an XML given as an input to the task).

Common logs

Code	severity	Description
OBJECT_CREATED	debug	An objected has been inserted in the database
OBJECT_UPDATED	debug	An objected has been updated in the database
UNKNOWN_ERROR	error	An unexpected error occured
INVALID_ARGUMENT	error	The inputs provided to the task are invalid. The task is stopped, and end with an error.