Small simple Python FTP upload script
New here? Learn about Bountify and follow @bountify to get notified of new bounties! x

I need a small Python script which will:

  1. Fetch all files recursively from a target local path, unzipping any zipped files
  2. Upload all files/folders from above step to a target FTP server + FTP path, with FTP user name + password in parameters
do these local files need to get fetched to another temporary folder, or do you need them to be fetched and unzipped in the same folder? Also, do you need both versions of the same file to be uploaded (the zipped and unzipped ones)?
thelostt 10 months ago
To transfer to FTP, if there are zipped files then I only need the unzipped files + other non zipped files.
user856 10 months ago
what's the zipping format? Just a simple .zip?
thelostt 10 months ago
Yea just simple .zip files
user856 10 months ago
Hi i started working on this, do you need to upload the files in respective folders or just upload files directly without any folders etc ?
Subhash Dasyam 10 months ago
I have completed the solution, do check it :), also Happy new year :)
Subhash Dasyam 10 months ago
The script which i wrote uploads files, including folders.
Subhash Dasyam 10 months ago
awarded to thelostt
Tags
python

Crowdsource coding tasks.

2 Solutions

Winning solution

gist

https://gist.github.com/thelostt/849e4352709679e0b33d3769031cdb5b

usage

fetch_upload.py --local-path /target/local/path --remote-path /ftp/target/path --ftp-server SERVER --user USER --password PASSWORD

current behavior

First it tries to unzip all zipped files, and place them to /path/to/zippedfile_fetch. Then, it will upload all files (except for the zipped ones as wanted) to a specific directory in the ftp server, following the same path structure as in the local target.

Let me know if you get any issues. Cheers.

Sorry for the late reply, I ran the script and I got this error: ('Fetching all files from ', '/var/www/test', ' ...') ('Uploading fetched files to remote ', '/', ' ...') Traceback (most recent call last): File "static_script_ftp_upload.py", line 39, in <module> main(parser.parse_args()) File "static_script_ftp_upload.py", line 29, in main upload_fetched_files(args.local_path, args.remote_path, args.ftp_server, args.ftp_user, args.ftp_password) File "static_script_ftp_upload.py", line 17, in upload_fetched_files with ftplib.FTP(server, user, password) as ftp: AttributeError: FTP instance has no attribute '__exit__'
user856 10 months ago
It seems you are running python 2, can you try it and run with python3? You can just make the first line of the code be #!/bin/python3.
thelostt 10 months ago
I ran again with python3 with a single zip file in a local directory and the script finished, but nothing was uploaded. I ran again the script with another .xml file in local directory and the script gave me this error Traceback (most recent call last): File "static_script_ftp_upload.py", line 39, in <module> main(parser.parse_args()) File "static_script_ftp_upload.py", line 29, in main upload_fetched_files(args.local_path, args.remote_path, args.ftp_server, args.ftp_user, args.ftp_password) File "static_script_ftp_upload.py", line 23, in upload_fetched_files ftp.storbinary('STOR ' + os.path.join(remote_path, realpath_file), f)
user856 10 months ago
File "/usr/lib/python3.4/ftplib.py", line 503, in storbinary with self.transfercmd(cmd, rest) as conn: File "/usr/lib/python3.4/ftplib.py", line 398, in transfercmd return self.ntransfercmd(cmd, rest)[0] File "/usr/lib/python3.4/ftplib.py", line 364, in ntransfercmd resp = self.sendcmd(cmd) File "/usr/lib/python3.4/ftplib.py", line 272, in sendcmd return self.getresp() File "/usr/lib/python3.4/ftplib.py", line 245, in getresp raise error_perm(resp) ftplib.error_perm: 550 Filename invalid
user856 10 months ago
The error message was too long and I had to post twice above.
user856 10 months ago
Never mind, I got the issue. Do you have the ftputil python package in your system? I'll use it in the script if you don't mind.
thelostt 10 months ago
Okay, I got everything working in my end. I also added an option --ftp-port in case you're not using the default one. And it depends on ftputil, you may install it with pip.
thelostt 10 months ago
I notice that there seems to be a temporary folder created each time when zipping files, is it possible to have the script remove this folder afterwards?
user856 10 months ago
I also installed ftputil but I got this error when I ran the command: 502 Explicit TLS authentication not allowed Debugging info: ftputil 3.3.1, Python 3.4.3 (linux)
user856 10 months ago
I removed the TLS support and made the changes in order to remove _fetch files.
thelostt 10 months ago
Everything should work in your end now.
thelostt 10 months ago
I ran it again and still Fetching all files from /var/www/test ... Uploading fetched files to remote / ... 502 Explicit TLS authentication not allowed Debugging info: ftputil 3.3.1, Python 3.4.3 (linux)
user856 10 months ago
Okay, I'm forcing the script to not use a TLS connection, can you see if it works now?
thelostt 10 months ago
I got this error after running the new script, but I'm able to use an FTP client to transfer files to the target FTP directory Fetching all files from /var/www/test ... Uploading fetched files to remote / ... 550 Can't create directory. Permission denied Debugging info: ftputil 3.3.1, Python 3.4.3 (linux)
user856 10 months ago
Updated.
thelostt 10 months ago
Still same error
user856 10 months ago
Does this error occur when you try to upload to a directory that's not the root? Try not giving --remote-path something that starts with /
thelostt 10 months ago
Yes I tried other directories and same error. Can I send you an email with the FTP credentials and you can try from your end? I wasn't able to find an email I can send you with.
user856 10 months ago
I sent you an email with the server details, can you try on your end? :)
user856 10 months ago

Recursive FTP Upload Solution

Tested in Ubuntu

Instructions

Python 2.7

sudo apt-get install python-pip -y

sudo pip install configargparse zipfile

save the code in ftp_sol.py or some_file_name.py

python ftp_sol.py -u subhash_username -p password -s 192.168.5.5 -po 5555 -r /var/www/html/test

python ftp_sol.py --help

usage: ftp_sol.py [-h] -u [-p] -s [-po] -r

optional arguments:

-h, --help show this help message and exit

-u , --username FTP Username

-p , --password FTP Password

-s , --host FTP Host

-po , --port FTP Port Default 21

-r , --path local path

It will recursively create directories in FTP Server and upload the files
It also scan the zip files in local path unzip under same zip name then upload files by creating folders

How do I specify target FTP path?
user856 10 months ago
How do I specify target FTP path?
user856 10 months ago
you do not need to specify the target FTP path for example you have folders and subfolders and files including zip files in this path /var/www when you give the command python ftp_sol.py -u username -p password -s HOST_ADDRESS -po PORT -r /var/www the script first searches for all the zip files in the /var/www folder and unzip's them first lets say the script finds a zip file in this path /var/www/folder1/sometest.zip, it first unzip that sometest.zip into /var/www/folder1/some_test/ Next again it scans all the files and folders then start uploading into remote ftp recursively it creates a folder FTPSERVER:var/www/sometest etc , it uploads everything keeping the folder names intact
Subhash Dasyam 10 months ago
https://gist.github.com/subhashdasyam/1d51e8de6fc184055fd024b7ab363048 New GIST if you want to specify remote FTP path python ftp_sol.py -u subhash_username -p password -s SERVER_HOST -po PORT -r /var/www/html/test -rp remote_ftp_folder/another_sub_folder This will upload to specific remote folder :) PS: -rp = Remote Path :)
Subhash Dasyam 10 months ago
View Timeline