Download Sharepoint folders and files with Python
New here? Learn about Bountify and follow @bountify to get notified of new bounties! x

Write a Python script which is given a Sharepoint login and credentials. The http url which points to the top of the file hierarchy is also passed.

The script will downloads all files and folders under the address provided to the local disk. It will retain the same directory structures.

I can't give you the credentials to the actual site, but there are sandbox sharepoint sites out there. $25 bonus if completed in 24 hours.

You will want to specify if your sharepoint site is accessible using HTTP BASIC auth or requires a different authentication scheme. My company uses Office365 which requires non-BASIC auth making it much more complicated to write code against what you're looking for (and so I'm going to pass at this time). There are some python libraries out there for this.
skram 8 years ago
I believe it is HTTP BASIC auth
kusadasi 8 years ago
an idea: perhaps you could give a restricted login for testing of the app to verified folks. Just saying, I see bountify.co working best for public APIs and projects that do not require working with a black box such as this. There are many variables that have to do with Sharepoint auth in my (admittedly limited) experience.
skram 8 years ago
Hi skram. Thanks for your comments. I cannot give access to the app, but a generic script would be enough -- I could finish it off.
kusadasi 8 years ago
is python obligatory?
ochi 8 years ago
Hi ochi - what language did you have in mind to do the job?
kusadasi 8 years ago
just general question - python is nice but I know there are solutions using simple bash scripting and wget or even php
ochi 8 years ago
bash /wget or php are fine with me
kusadasi 8 years ago
Could you perhaps link to one of these sandboxed setups?
alixaxel 8 years ago
http://vreciss.codeplex.com/ - see the document library for an example of what the code would access.
kusadasi 8 years ago
@kusadasi: I found a SP 2010 demo service, I was able to login using basic authentication and get the list of files. However, I couldn't find a way to create folders in Sharepoint so I could test it after, I'm afraid I can't get past this point. =\
alixaxel 8 years ago
Hi @alixaxel post what you have - you might be the winner even with incomplete code
kusadasi 8 years ago
https://pypi.python.org/pypi/haufe.sharepoint This has some useful utilities
kusadasi 8 years ago
Can you post a url that you want it to accept as input?
tomtoump 8 years ago
awarded to tomtoump

Crowdsource coding tasks.

1 Solution

Winning solution

I wanted to get my hands down on Python lately, so this was a good chance for me. :)

Here is my first Python program:

import argparse
import urllib.parse
import os
import requests
from requests.auth import HTTPBasicAuth

parser = argparse.ArgumentParser()
parser.add_argument('server')
parser.add_argument('username')
parser.add_argument('password')
args = parser.parse_args()
server = urllib.parse.urljoin(args.server,'/')

auth = HTTPBasicAuth(args.username, args.password)
headers = {'accept': 'application/json;odata=verbose'}

def make_request(url):
    r = requests.get(url, auth=auth, headers=headers)
    return r.json()

def download_file(filename):
    filename = filename.strip('/')
    print('Downloading\t'+filename)
    r = requests.get(server+"_api/Web/GetFileByServerRelativeUrl('/"+filename+"')/$value", auth=auth, stream=True)
    with open(filename, 'wb+') as f:
        for chunk in r.iter_content(chunk_size=1024): 
            if chunk:
                f.write(chunk)
                f.flush()

def check_directory(path):
    path = path.strip('/')
    print('Checking\t'+path)
    if not os.path.exists(path):
        os.makedirs(path)
    folders = make_request(server+"_api/Web/GetFolderByServerRelativeUrl('/"+path+"')/Folders")['d']['results']
    for folder in folders:
        check_directory(folder['ServerRelativeUrl'])
    files = make_request(server+"_api/Web/GetFolderByServerRelativeUrl('/"+path+"')/Files")['d']['results']
    for file in files:
        download_file(file['ServerRelativeUrl'])

check_directory('/Shared Documents')

It takes server url, username and password as inputs and downloads every file on SharePoint.

I 've tested it under SharePoint 2013 and it works fine.

Thanks for your answer. I am trying it out... running into some minor issues: which version of python did you use? I am using python3 but am getting an error on - import requests
kusadasi 8 years ago
I am using Python 3 as well, but I also used the Requests library. Follow this instructions to install it.
tomtoump 8 years ago
I tried to launch your script under Ubuntu Linux 13.04, with Python3 installed manually, and I get an error. The requests lib can't be imported (even in the console) with Python3 after pip install (only Python2) so I embedded it. Here is the way I called it : ./sharepoint-python3.py https://my_domain/path/to/documents/ myemail mypassword These logins are used without any problem through the web interface. Here is the error: line 53, checkdirectory('/Shared Documents'); line 46, in checkdirectory, folders = makerequest(server+"api/Web/GetFolderByServerRelativeUrl('/"+path+"')/Folders")['d']['results']; line 29, in make_request, return r.json(); ValueError: Extra data: line 1 column 196 - line 1 column 249 (char 195 - 248)
Vincent14 8 years ago
i tried the code in python 2.7 version but its not working...could u tell what i do
suman almost 2 years ago