Upload data from cloud folder

operators.sys.upload_data_from_cloud_folder(client, url='', folder_target='/folder_target', protocol='', username='', password='', port=21, worker_instance_type='x2large', manager_instance_type='small', extension_target='.laz', skip_existing_files=False)

upload_data_from_cloud_folder(client,
url=’’,
target=’/target’,
protocol=’’,
username=’’,
password=’’,
port=21,
worker_instance_type=’x2large’,
manager_instance_type=”small”,
extension_target=”.data”,
skip_existing_files = False )
Parameters:

  • url – destination URL

  • protocol – protocol: : automatically try to infer protocol, ftp: ftp, sftp: sftp

  • username – Username

  • password – Password

  • port – port

  • folder_target – Target location on host for upload. default folder: ./data

  • worker_instance_type – cloud instance type of worker nodes

  • manager_instance_type – cloud instance type of manager node

  • extension_target – File extension of files in folder for folder_target

  • skip_existing_files – skip files that already exist in the output folder