Merge pull request #1013 from learningequality/release

Release -> Master (v0.2.0)

.buildkite/build_pex.sh

@@ -0,0 +1,8 @@
+#!/usr/bin/env bash
+
+set -euo pipefail
+
+pip install pex                 # pex is really the only thing we need here.
+buildkite-agent artifact download 'dist/*.whl' dist/
+make pex
+buildkite-agent artifact upload 'dist/*.pex'

.buildkite/build_whl.sh

@@ -0,0 +1,8 @@
+#!/usr/bin/env bash
+
+set -euo pipefail
+
+make dockerenvdist
+buildkite-agent artifact upload 'dist/*.whl'
+buildkite-agent artifact upload 'dist/*.zip'
+buildkite-agent artifact upload 'dist/*.tar.gz'

.buildkite/pipeline.yml

@@ -0,0 +1,13 @@
+steps:
+  - label: Build the docker environment
+    command: make dockerenvbuild
+
+  - wait
+
+  - label: Build the python packages
+    command: mkdir -p dist && .buildkite/build_whl.sh
+
+  - wait
+
+  - label: Build the pex file
+    command: mkdir -p dist && .buildkite/build_pex.sh

.dockerignore

@@ -0,0 +1,2 @@
+node_modules/*
+dist/*

.gitignore

@@ -92,3 +92,6 @@ Icon
 #ignore added files with DS_Store
 .DS_Store
 /crowdin.yaml
+
+# ignore the VERSION file we use to track the version for whl files
+VERSION

.travis.yml

@@ -8,6 +8,7 @@ services:
 sudo: false
 
 cache:
+  yarn: true
   directories:
     - node_modules
     - $HOME/.cache/pip
@@ -78,6 +79,8 @@ before_install:
   - . $HOME/.nvm/nvm.sh
   - nvm install $TRAVIS_NODE_VERSION
   - nvm use $TRAVIS_NODE_VERSION
+  - curl -o- -L https://yarnpkg.com/install.sh | bash
+  - export PATH=$HOME/.yarn/bin:$PATH
 
 before_script:
   - psql -c 'create database travis_ci_test;' -U postgres

CHANGELOG.rst

@@ -3,39 +3,98 @@
 Release Notes
 =============
 
-All notable changes to this project will be documented in this file.
-This project adheres to `Semantic Versioning <http://semver.org/>`_.
-
-*Don’t let your friends dump git logs into CHANGELOGs™*
-
-`http://keepachangelog.com/ <http://keepachangelog.com/>`_
-
-[0.0.1] - UNRELEASED
---------------------
-
-.. note ::
-    Please add new stuff chronologically, we should try chunking up the
-    list into components at some point
-
-Added
-^^^^^
-
- - Begin development of core auth app.
- - Add core user types (BaseUser, FacilityUser, DeviceOwner)
- - Add Collections and Roles, implemented using a special tree structure for efficient querying
- - Add authentication & authorization backends
- - Implement permissions for FacilityUsers by checking hierarchy relationships
- - Adds pipelining and integration for building frontend assets with webpack and dynamically serving them in Django.
- - Updates to  - Users, Collections, and Roles, mostly to account for multiple facilities in one database
- - Plugin API with hooks, documented and implemented
- - Adds Django JS Reverse and loads into kolibriGlobal object
- - Creates kolibri.auth API endpoint filtering
- - Adds management plugin for managing learners, classrooms, and groups
- - Automatic inclusion of requirements in a static build
- - Linting of client-side assets during webpack build process
+0.2.0
+-----
+
+ - HTML5 app renderer
+ - Vue2 refactor
+ - Webpack build process compatible with plugins outside the kolibri directory
+ - ES2015 transpilation now Bublé instead of Babel
+ - Pin dependencies with Yarn
+ - Improved docs
+ - Log filtering based on users and collections
+ - Wrap all user-facing strings for I18N
+ - Allow plugins to override core components
+ - Client heartbeat for usage tracking
+ - Versioning based on git tags
+ - User sign-up and profile-editing functionality
+ - New log-in page
+ - Update primary layout and navigation
+ - Begin using some keen-ui components
+ - Replace jeet grids with pure.css grids
+ - Add JS-based 'responsive mixin' as alternative to media queries
+ - Rename 'Learn/Explore' to 'Recommended/Topics'
+ - Temporarily remove 'search' functionality
+ - Refactor content renderer API interface
+ - Add authentication for tasks API
+
+
+0.1.1
+-----
+
+ - Coach reports
+ - Perseus exercise renderer
+ - Exercise mastery visualization
+ - SVG inlining
 
-Changed
-^^^^^^^
 
- - Webpack build process now compatible with plugins housed outside the kolibri directory.
- - ES2015 transpilation now uses Bublé instead of Babel.
+0.1.0 - MVP
+-----------
+
+ - Content downloading
+ - Documentation updates
+ - Setup wizard plugin
+ - Channel switching
+ - Session state and login widget
+ - Content import/export
+ - Task management
+ - User management UI
+ - Learn app styling changes
+ - Tab focus highlights
+ - A11Y updates
+ - Modal popups
+ - Channel switching bug fixes
+ - I18N string extraction
+ - Content interaction logging
+ - Drive enumeration
+ - Usage data export
+ - Fuzzy searching
+ - Make modals accessible
+ - Loading 'spinner'
+ - Resource layer smart cache busting
+ - Client-side router bug fixes
+ - Make plugins more self-contained
+ - Case-insensitive usernames
+ - Endpoint indexing into zip files
+ - Asset bundling performance improvements
+ - Conditional (cancelable) JS promises
+ - Improved documentation
+
+
+0.0.1 - MMVP
+------------
+
+ - Users, Collections, and Roles
+ - Authentication, authorization, permissions
+ - Webpack build pipeline, including linting
+ - Python plugin API with hooks
+ - Django JS Reverse with urls representation in kolibriGlobal object
+ - Automatic inclusion of requirements in a static build
+ - Content endpoints
+ - Learn app and content browsing
+ - Vue.js integration
+ - User management API
+ - Initial VueIntl integration
+ - Video, Document, and MP3 content renderers
+ - Content search
+ - Client-side routing
+ - Content recommendation endpoints
+ - API resource retrieval and caching
+ - Support for multiple content DBs
+ - Stylus/Jeet-based grids
+ - Vuex integration
+ - Cherrypy server
+ - A11Y updates
+ - Responsiveness updates
+ - Javascript logging module
+ - Page titles

Dockerfile

@@ -0,0 +1,19 @@
+FROM ubuntu:xenial
+
+# install latest python and nodejs
+RUN apt-get -y update
+RUN apt-get install -y software-properties-common curl
+RUN add-apt-repository ppa:voronov84/andreyv
+RUN curl -sL https://deb.nodesource.com/setup_6.x | bash -
+
+# add yarn ppa
+RUN curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add -
+RUN echo "deb https://dl.yarnpkg.com/debian/ stable main" | tee /etc/apt/sources.list.d/yarn.list
+
+RUN apt-get -y update
+RUN apt-get install -y python2.7 python3.6 python-pip git nodejs yarn gettext python-sphinx
+COPY . /kolibri
+
+VOLUME /kolibridist/  # for mounting the whl files into other docker containers
+CMD cd /kolibri && pip install -r requirements/dev.txt && pip install -r requirements/build.txt && yarn install && make dist && cp /kolibri/dist/* /kolibridist/
+

MANIFEST.in

@@ -4,6 +4,7 @@ include CHANGELOG.rst
 include LICENSE
 include README.rst
 include requirements.txt
+include kolibri/VERSION
 recursive-include requirements *.txt
 recursive-include kolibri/locale *.mo
 recursive-include kolibri/auth *
@@ -16,11 +17,11 @@ recursive-include kolibri/plugins *
 recursive-include kolibri/test *
 recursive-include kolibri/utils *
 recursive-include kolibri/tasks *
-recursive-exclude kolibri/* *pyc
 recursive-include kolibri/*/static *.*
 recursive-include kolibri/*/build/ *.json
 recursive-include kolibri/plugins/*/build *.json
 
+recursive-exclude kolibri/* *pyc
 recursive-exclude kolibri/core/assets *
 recursive-exclude kolibri/plugins/*/assets *
 recursive-exclude kolibri/plugins/*/node_modules *

Makefile

@@ -14,7 +14,7 @@ help:
 clean: clean-build clean-pyc clean-docs clean-static
 
 clean-static:
-	npm run clean
+	yarn run clean
 
 clean-build:
 	rm -fr build/
@@ -24,7 +24,8 @@ clean-build:
 	rm -fr *.egg-info
 	rm -fr .eggs
 	rm -fr .cache
-	git clean -X -d -f kolibri/dist
+	rm -r kolibri/dist/* || true # remove everything
+	git checkout -- kolibri/dist # restore __init__.py
 
 clean-pyc:
 	find . -name '*.pyc' -exec rm -f {} +
@@ -48,7 +49,7 @@ test-all:
 	tox
 
 assets: staticdeps
-	npm run build
+	yarn run build
 
 coverage:
 	coverage run --source kolibri setup.py test
@@ -63,15 +64,21 @@ release: clean assets
 	python setup.py bdist_wheel upload
 
 staticdeps: clean
-	DISABLE_SQLALCHEMY_CEXT=1 pip install -t kolibri/dist/ -r requirements.txt
+	pip install -t kolibri/dist -r requirements.txt
+	rm -r kolibri/dist/*.dist-info  # pip installs from PyPI will complain if we have more than one dist-info directory.
 
-dist: staticdeps assets compilemessages
+writeversion:
+	git describe --tags > kolibri/VERSION
+
+dist: writeversion staticdeps assets compilemessages
 	pip install -r requirements/build.txt
 	python setup.py sdist --format=gztar,zip --static > /dev/null # silence the sdist output! Too noisy!
 	python setup.py bdist_wheel --static
-	pex . --disable-cache -o dist/`python setup.py --fullname`.pex -m kolibri --python-shebang=/usr/bin/python
 	ls -l dist
 
+pex:
+	ls dist/*.whl | while read whlfile; do pex $$whlfile --disable-cache -o dist/kolibri-`unzip -p $$whlfile kolibri/VERSION`.pex -m kolibri --python-shebang=/usr/bin/python; done
+
 makedocsmessages:
 	make -C docs/ gettext
 	cd docs && sphinx-intl update -p _build/locale -l en
@@ -95,3 +102,31 @@ downloadmessages:
 
 distributefrontendmessages:
 	python ./utils/distribute_frontend_messages.py
+
+dockerenvclean:
+	docker container prune -f
+	docker image prune -f
+
+dockerenvbuild: writeversion
+	docker image build -t learningequality/kolibri:$$(cat kolibri/VERSION) -t learningequality/kolibri:latest .
+
+dockerenvdist: writeversion
+	docker run -v $$PWD/dist:/kolibridist learningequality/kolibri:$$(cat kolibri/VERSION)
+
+BUMPVERSION_CMD = bumpversion --current-version `python -m kolibri --version` $(PART_INCREMENT) --allow-dirty -m "new version" --no-commit --list
+
+minor_increment:
+	$(eval PART_INCREMENT = minor)
+	$(BUMPVERSION_CMD)
+
+patch_increment:
+	$(eval PART_INCREMENT = patch)
+	$(BUMPVERSION_CMD)
+
+release_phase_increment:
+	$(eval PART_INCREMENT = release_phase)
+	$(BUMPVERSION_CMD)
+
+release_number_increment:
+	$(eval PART_INCREMENT = release_number)
+	$(BUMPVERSION_CMD)

README.rst

@@ -40,6 +40,9 @@ Kolibri is under active development and is not yet ready to be used. In the mean
 How can I contribute?
 ---------------------
 
- * `Documentation <http://kolibri.readthedocs.org/en/latest/>`_ is available online, and in the ``docs/`` directory.
- * Mailing list: `Google groups <https://groups.google.com/a/learningequality.org/forum/#!forum/dev>`_.
- * IRC: #kolibri on Freenode
+.. warning::
+  We welcome new contributors but since **Kolibri** is still in development, the API is not yet completely ready to integrate external plugins. Please start by:
+
+* Reading our `Developer Documentation <http://kolibri.readthedocs.io/en/develop/dev/getting_started.html>`_ available online, and in the ``docs/`` directory.
+* Contacting us on the Mailing list: `Google groups <https://groups.google.com/a/learningequality.org/forum/#!forum/dev>`_.
+* or via IRC: #kolibri on Freenode.

docs/dev/conventions.rst

@@ -48,7 +48,7 @@ Note that the top-level tags of `Vue.js components <https://vuejs.org/guide/comp
 
 - Put child components inside the directory of a parent component if they are *only* used by the parent. Otherwise, put shared child components in the *vue* director.
 
-- Any user visisble interface text should be rendered translatable, see :doc:`/i18n` for details.
+- Any user visisble interface text should be rendered translatable, see :doc:`i18n` for details.
 
 
 JavaScript Code

docs/dev/frontend.rst

@@ -233,14 +233,12 @@ Vue.js components can also be tested. The management plugin contains an example
 Adding Dependencies
 -------------------
 
-Dependencies are tracked using ``npm shrinkwrap`` - `see the docs here <https://docs.npmjs.com/cli/shrinkwrap>`_.
+Dependencies are tracked using ``yarn`` - `see the docs here <https://yarnpkg.com/en/docs/>`_.
 
-We distinguish development dependencies from runtime dependencies, and these should be installed as such using ``npm install --save-dev [dep]`` or ``npm install --save [dep]``, respectively. Then you'll need to run ``npm shrinkwrap``. Your new dependency should now be recorded in *package.json*, and all of its dependencies should be recorded in *npm-shrinkwrap.json*.
+We distinguish development dependencies from runtime dependencies, and these should be installed as such using ``yarn add --dev [dep]`` or ``yarn add [dep]``, respectively. Your new dependency should now be recorded in *package.json*, and all of its dependencies should be recorded in *yarn.lock*.
 
-Note that we currently don't have a way of mapping dependencies to plugins - dependencies are installed globally.
+Individual plugins can also have their own package.json and yarn.lock for their own dependencies. Running ``yarn install`` will also install all the dependencies for each activated plugin (inside a node_modules folder inside the plugin itself). These dependencies will only be available to that plugin at build time. Dependencies for individual plugins should be added from within the root directory of that particular plugin.
 
-To assist in tracking the source of bloat in our codebase, the command ``npm run bundle-stats`` is available to give a full readout of the size that uglified packages take up in the final Javascript code.
-
-Individual plugins can also have their own package.json for their own dependencies. Running ``npm install`` will also install all the dependencies for each activated plugin (inside a node_modules folder inside the plugin itself). These dependencies will only be available to that plugin at build time.
+To assist in tracking the source of bloat in our codebase, the command ``yarn run bundle-stats`` is available to give a full readout of the size that uglified packages take up in the final Javascript code.
 
 In addition, a plugin can have its own webpack.config.js for plugin specific webpack configuration (loaders, plugins, etc.). These options will be merged with the base options using ``webpack-merge``.