Download from s3 to aipha folder
- operators.sys.download_from_s3_to_aipha_folder(client, access_key_id='YOUR_KEY_ID', secret_access_key='YOUR_SECRET_KEY', aws_region='eu-central-1', location='file.laz', folder_destination='/folder_destination', bucket_name='Your S3 bucket', worker_instance_type='x2large', manager_instance_type='small', extension_destination='.laz', skip_existing_files=False)
- Download a path from a S3 bucket
- download_from_s3_to_aipha_folder(client,access_key_id=’YOUR_KEY_ID’,secret_access_key=’YOUR_SECRET_KEY’,aws_region=’eu-central-1’,location=’file.laz’,destination=’/destination’,bucket_name=’Your S3 bucket’,worker_instance_type=’x2large’,manager_instance_type=”small”,extension_destination=”.laz”,skip_existing_files = False )
- Parameters:
access_key_id – AWS access key ID
secret_access_key – AWS secret access key
aws_region – AWS region
location – Path to download from s3
bucket_name – S3 bucket name
folder_destination – Location to upload to aipha
worker_instance_type – cloud instance type of worker nodes
manager_instance_type – cloud instance type of manager node
extension_destination – File extension of files in folder for folder_destination
skip_existing_files – skip files that already exist in the output folder