EXERCISE TranSMART REST API V2 (2017).ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img style=\"float: right;\" src=\"files/thehyve_logo.png\">\n",
    "# TranSMART 17.1 REST API demonstration\n",
    "---------------\n",
    "Copyright (c) 2017 The Hyve B.V. This notebook is licensed under the GNU General Public License, version 3. Authors: Ward Weistra."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We start by importing the tranSMART Python library (https://pypi.python.org/pypi/transmart) and connecting to the tranSMART server."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "import getpass\n",
    "from transmart import TransmartApi\n",
    "\n",
    "api = TransmartApi(\n",
    "    host = 'http://transmart-test.thehyve.net',\n",
    "    user = input('Username:'),\n",
    "    password = getpass.getpass('Password:'),\n",
    "    apiversion = 2)\n",
    "\n",
    "api.access()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next we import and configure Pandas, a Python library to work with data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "from pandas.io.json import json_normalize\n",
    "\n",
    "pd.set_option('max_colwidth', 1000)\n",
    "pd.set_option(\"display.max_rows\",100)\n",
    "pd.set_option(\"display.max_columns\",100)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 1: Plotting blood pressure over time\n",
    "As a first REST API call it would be nice to see what studies are available in this tranSMART server.  \n",
    "  \n",
    "You will see a list of all studies, their name (studyId) and what dimensions are available for this study. Remember that tranSMART previously only supported the dimensions patients, concepts and studies. Now you should see studies with many more dimensions!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "studies = api.get_studies()\n",
    "json_normalize(studies['studies'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We choose the TRAINING study and ask for all patients in this study. You will get a list with their patient details and patient identifier."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "study_id = 'TRAINING'\n",
    "patients = api.get_patients(study = study_id)\n",
    "json_normalize(patients['patients'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next we ask for the full list of observations for this study. This list will include one row per observation, with information from all their dimensions. The columns will have headers like `<dimension name>.<field name>` and `numericValue` or `stringValue` for the actual observation value."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "obs = api.get_observations(study = study_id)\n",
    "obsDataframe = json_normalize(api.format_observations(obs))\n",
    "obsDataframe"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#DO STUFF WITH THE TRAINING STUDY HERE"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 2: Combining Glowing Bear and the Python client\n",
    "\n",
    "For the second part we will work with the Glowing Bear user interface that was developed at The Hyve, funded by IMI Translocation and BBMRI.\n",
    "\n",
    "An API is great to extract exactly the data you need and analyze that. But it is harder to get a nice overview of all data that is available and define the exact set to extract. That is where the Glowing Bear was built for.\n",
    "\n",
    "Please go to http://glowingbear2-head.thehyve.net and create a Patient Set on the Data Selection tab (under Select patients). Once you have saved your patient set, copy the patient set identifier and paste that below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "patient_set_id = 28733"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's return all patients for the patient set we made!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true,
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "patients = api.get_patients(patientSet = patient_set_id)\n",
    "json_normalize(patients['patients'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And do the same for all observations for this patient set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "obs = api.get_observations(study = study_id, patientSet = patient_set_id)\n",
    "obsDataframe = json_normalize(api.format_observations(obs))\n",
    "obsDataframe"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now you know exactly how to retrieve data from a tranSMART 2017 server and analyze this with Python! Please feel free to change this code in anyway you like. And if you have any questions, reach us at our public forum via development@thehyve.nl or https://groups.google.com/a/thehyve.nl/forum/#!forum/development."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}