Data Factory > protocol-ftp-list

TL;DR;

This task allows listing the files present on a FTP/SFTP server.

Name:

Max execution time: 120 mins

Target

List the files on an FTP/SFTP server

Example of use in a job

1. Retrieve multiple image files and associate them with products

1. FTP List List the image files on the FTP
2. XSLT Define the files to recover
3. FTP Get Recover files
4. XSLT Build the product file
5. Import Items Import products

In order for this job not to go into error in case no file is available on the FTP server, we could have added a decision task to proceed as follows

1. FTP List List the image files present on the sFTP
2. Decision
    a. Case No
        i. Terminate Job Successful job termination
    b. Default
        i. XSLT Define files to recover
        ii. FTP Get Recover files
        iii. XSLT Build the product file
        iv. Import Items Import products

Inputs and Outputs

{
  "name": "protocol-ftp-list",
  "taskReferenceName": "ftp_list",
  "description": "The business description of the task",
  "type": "SUB_WORKFLOW",  
  "optional": false,
  "inputParameters": {
    "connection": "SFTP",
    "host": "",
    "username": "",
    "password": "",
    "port": 21,
    "remoteFolder": "/images/",
    "depth": 1,
  }
}

Inputs

TIP

If at least one task parameters (whether mandatory or not) is invalid, task execution is stopped and the returned status is FAILED. For example:

• The proposed value for the connection property is invalid.

Property	Description
connection	Mandatory - Enum - FTP, SFTP, FTPS
authenticationMethod	Enum - PASSWORD, PUBLIC_KEY (only available if connection = SFTP), Default: PASSWORD If mode != SFTP and authenticationMethod = PUBLIC_KEY, the task fails.
host	Mandatory – String
privateKey	Mandatory if authenticationMethod=PUBLIC_KEY - String It is strongly recommended to use a secret variable to store the private key.
passphrase	String Only taken into account if authenticationMethod=PUBLIC_KEY.
username	Mandatory - String
password	Mandatory if connection != SFTP, if connection == SFTP and password is set then an invalid input error is added - String
port	Mandatory - Number
depth	Mandatory – Number - Between 1 and 5 - Default: 1 The maximum depth of the subfolders you want to list.
remoteFolder	String Default value: /, the root folder The folder in which we want to list the files.

TIP

The supported private keys are the following :

• ---- BEGIN SSH2 PUBLIC KEY ----
• -----BEGIN RSA PRIVATE KEY-----
• -----BEGIN DSA PRIVATE KEY-----
• -----BEGIN EC PRIVATE KEY-----
• PuTTY-User-Key-File-2: (ssh-rsa)
• PuTTY-User-Key-File-2: (ssh-dss)
• -----BEGIN OPENSSH PRIVATE KEY-----

Outputs

TIP

The outputs of the task defined below are always available if the task completes its execution (status COMPLETED and FAILED). If the task is stopped before the end of its execution (if the task is canceled by the user for example - CANCELED status, or if it exceeds the authorized execution time - TIMED_OUT), the outputs are not available.

Property	Description
listing	File An XML file listing the files on the FTP server. See below for details.
noFile	Enum - YES, NO If at least one file is listed: YES otherwise NO.

Details regarding the listing document

<Files>
    <File>
        <Path>/file-1.jpg</Path>
        <Last-Modified>2020-04-10T13:40:23.83Z</Last-Modified>
        <Size>1954387</Size>
    </File>
    <File>
        <Path>/sub-folder/file-2.jpg</Path>
        <Last-Modified>2020-04-10T13:40:23.83Z</Last-Modified>
        <Size>1954387</Size>
    </File>
</Files>

XPath	Description	Occurrence
Files	Root	1
File	For each file listed	0..*
Path	The complete path. Subfolders are separated by the / character. Only files are listed, not directories.	1
Last-Modified	The date of the last modification of the file, format is UTC	1
Size	The size of the file in Kb	1

Limitations connues

• For SFTP connections, only the following ciphers are allowed: aes256-cbc, aes192-cbc, aes128-cbc, blowfish-cbc, 3des-cbc, aes128-gcm, aes256-gcm, arcfour256, arcfour128, cast128-cbc, arcfour.