forked from geoffreyanderson/MyCourses-Scraper
-
Notifications
You must be signed in to change notification settings - Fork 2
/
Copy pathREADME
35 lines (26 loc) · 1.64 KB
/
README
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
Updated 5/13/2014 by ctrlaltdel121
Originally via zszafran/MyCourses-Scraper
[Purpose]
This python script will login to your MyCourses account and navigate its way through each course. When in a course, this script will dump all the news, course content and dropbox submissions to your output location. After running the script you should have a directory that represents each course which will contain all of that course's content.
NOTE 5/13/14: I never fixed news, only dropbox and some courses's content sections work.
[Usage]
python mycourses.py -h
python mycourses.py --output ~/mycourses_dump --crossdomain --news --content --dropbox --verbose 2 --delay 0 --zip
python mycourses.py --output ~/mycourses_dump --crossdomain --content --dropbox --verbose 1 --delay 0
python mycourses.py --output ~/mycourses_dump --content --dropbox --zip
python mycourses.py --output ~/mycourses_dump --content --dropbox
[Required Software & Packages]
Python 2.7.1
lxml
pyquery
demjson
[Ubuntu Server Installation]
sudo apt-get install libxml2-dev libxslt1-dev python-lxml python-pyquery python-httplib2 python-demjson
[Windows 7 64-bit Installation]
Download and install python 2.7.1 64-bit: http://www.python.org/ftp/python/2.7.1/python-2.7.1.amd64.msi
Download easy_install 64-bit to python directory: http://peak.telecommunity.com/dist/ez_setup.py
Install easy_install: C:\Python27>python ez_setup.py
Install lxml: C:\Python27\Scripts>easy_install.exe lxml
Install pyquery: C:\Python27\Scripts>easy_install.exe pyquery
Install demjson: C:\Python27\Scripts>easy_install.exe demjson
C:\Python27>python mycourses.py --output c:\mycourses --dropbox --content --verbose 2 --delay 0