Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ENH] Generate BIDS Derivatives-compatible outputs #691

Merged
merged 41 commits into from
May 5, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
21e479c
BIDS Derivatives-compatible outputs.
tsalo May 22, 2020
9dd2728
Fix outputs.
tsalo May 22, 2020
41c9512
Merge branch 'master' into ref/filenames-and-variables
tsalo Jul 13, 2020
66b0a0c
Fix filenames.
tsalo Jul 14, 2020
000997e
Merge branch 'master' into ref/filenames-and-variables
tsalo Sep 22, 2020
eb23f74
Merge branch 'master' into ref/filenames-and-variables
tsalo Oct 22, 2020
737b19c
Reformat MIR function with black.
tsalo Oct 22, 2020
0b82189
Update fiu_four_echo_outputs.txt
tsalo Oct 22, 2020
d00b03a
Merge branch 'main' into ref/filenames-and-variables
tsalo Feb 6, 2021
51c7a7b
Reorganize metric outputs.
tsalo Feb 6, 2021
65784d8
Write out PCA outputs.
tsalo Feb 7, 2021
7f88281
Fix tests.
tsalo Feb 7, 2021
141c1b1
Fix up gscontrol.
tsalo Feb 7, 2021
b6a375c
Fix row indexing.
tsalo Feb 7, 2021
06bd83f
Fix suffix.
tsalo Feb 7, 2021
de0526f
Simplify handling of component indices.
tsalo Feb 7, 2021
b5b69a1
Fix.
tsalo Feb 7, 2021
840b1c0
Fix the reports.
tsalo Feb 7, 2021
7e70a7b
Fix the bugs.
tsalo Feb 7, 2021
860bff3
Fix more bugs.
tsalo Feb 9, 2021
e587849
Fix everything!
tsalo Feb 9, 2021
bb15c2e
Fix the loading?
tsalo Feb 9, 2021
4303b97
Update fiu_four_echo_outputs.txt
tsalo Feb 9, 2021
1aa52bc
Fix outputs!
tsalo Feb 9, 2021
2b1524f
Ugh...
tsalo Feb 10, 2021
6b4ab31
Add dataset_description.json.
tsalo Feb 10, 2021
ed45069
Update outputs and workflow description.
tsalo Feb 10, 2021
f7a4103
Fix output list.
tsalo Feb 10, 2021
fb7e8f8
Fix more outputs.
tsalo Feb 10, 2021
39a18f7
Remove unused functions.
tsalo Feb 10, 2021
245c7bd
Fix imports.
tsalo Feb 10, 2021
ed85699
Change desc-ICA to desc-tedana.
tsalo Feb 12, 2021
cd1ce51
Update docs.
tsalo Feb 12, 2021
cc9296b
Update docs/outputs.rst
tsalo Feb 23, 2021
0789f81
Merge pull request #574 from tsalo/ref/filenames-and-variables
tsalo Feb 24, 2021
2b9a7ab
[REF] Modularize io (#692)
jbteves Apr 1, 2021
adebb32
Merge branch 'main' into bids-derivatives
tsalo Apr 1, 2021
c10505b
Revert accidental changes to cornell_three_echo test results
Apr 2, 2021
7c947b2
Actually revert those changes
Apr 2, 2021
aa499b5
Use class for managing outputs (#711)
tsalo May 4, 2021
713a076
Remove constants.py
May 5, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
91 changes: 63 additions & 28 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,92 +28,125 @@ jobs:
paths:
- /opt/conda/envs/tedana_py37

unittest_35:
unittest_36:
docker:
- image: continuumio/miniconda3
working_directory: /tmp/src/tedana
steps:
- checkout
- restore_cache:
key: conda-py35-v1-{{ checksum "tedana/info.py" }}
key: conda-py36-v1-{{ checksum "tedana/info.py" }}
- run:
name: Generate environment
command: |
apt-get install -yqq make
if [ ! -d /opt/conda/envs/tedana_py35 ]; then
conda create -yq -n tedana_py35 python=3.5
source activate tedana_py35
if [ ! -d /opt/conda/envs/tedana_py36 ]; then
conda create -yq -n tedana_py36 python=3.6
source activate tedana_py36
pip install .[tests]
fi
- run:
name: Running unit tests
command: |
source activate tedana_py35
source activate tedana_py36
make unittest
mkdir /tmp/src/coverage
mv /tmp/src/tedana/.coverage /tmp/src/coverage/.coverage.py35
mv /tmp/src/tedana/.coverage /tmp/src/coverage/.coverage.py36
- save_cache:
key: conda-py35-v1-{{ checksum "tedana/info.py" }}
key: conda-py36-v1-{{ checksum "tedana/info.py" }}
paths:
- /opt/conda/envs/tedana_py35
- /opt/conda/envs/tedana_py36
- persist_to_workspace:
root: /tmp
paths:
- src/coverage/.coverage.py35
- src/coverage/.coverage.py36

unittest_36:
unittest_37:
docker:
- image: continuumio/miniconda3
working_directory: /tmp/src/tedana
steps:
- checkout
- restore_cache:
key: conda-py36-v1-{{ checksum "tedana/info.py" }}
key: conda-py37-v1-{{ checksum "tedana/info.py" }}
- run:
name: Running unit tests
command: |
apt-get install -y make
source activate tedana_py37 # depends on makeenv_37
make unittest
mkdir /tmp/src/coverage
mv /tmp/src/tedana/.coverage /tmp/src/coverage/.coverage.py37
- persist_to_workspace:
root: /tmp
paths:
- src/coverage/.coverage.py37

unittest_38:
docker:
- image: continuumio/miniconda3
working_directory: /tmp/src/tedana
steps:
- checkout
- restore_cache:
key: conda-py38-v1-{{ checksum "tedana/info.py" }}
- run:
name: Generate environment
command: |
apt-get install -yqq make
if [ ! -d /opt/conda/envs/tedana_py36 ]; then
conda create -yq -n tedana_py36 python=3.6
source activate tedana_py36
if [ ! -d /opt/conda/envs/tedana_py38 ]; then
conda create -yq -n tedana_py38 python=3.8
source activate tedana_py38
pip install .[tests]
fi
- run:
name: Running unit tests
command: |
source activate tedana_py36
source activate tedana_py38
make unittest
mkdir /tmp/src/coverage
mv /tmp/src/tedana/.coverage /tmp/src/coverage/.coverage.py36
mv /tmp/src/tedana/.coverage /tmp/src/coverage/.coverage.py38
- save_cache:
key: conda-py36-v1-{{ checksum "tedana/info.py" }}
key: conda-py38-v1-{{ checksum "tedana/info.py" }}
paths:
- /opt/conda/envs/tedana_py36
- /opt/conda/envs/tedana_py38
- persist_to_workspace:
root: /tmp
paths:
- src/coverage/.coverage.py36
- src/coverage/.coverage.py38

unittest_37:
unittest_39:
docker:
- image: continuumio/miniconda3
working_directory: /tmp/src/tedana
steps:
- checkout
- restore_cache:
key: conda-py37-v1-{{ checksum "tedana/info.py" }}
key: conda-py39-v1-{{ checksum "tedana/info.py" }}
- run:
name: Generate environment
command: |
apt-get install -yqq make
if [ ! -d /opt/conda/envs/tedana_py39 ]; then
conda create -yq -n tedana_py39 python=3.9
source activate tedana_py39
pip install .[tests]
fi
- run:
name: Running unit tests
command: |
apt-get install -y make
source activate tedana_py37 # depends on makeenv_37
source activate tedana_py39
make unittest
mkdir /tmp/src/coverage
mv /tmp/src/tedana/.coverage /tmp/src/coverage/.coverage.py37
mv /tmp/src/tedana/.coverage /tmp/src/coverage/.coverage.py39
- save_cache:
key: conda-py39-v1-{{ checksum "tedana/info.py" }}
paths:
- /opt/conda/envs/tedana_py39
- persist_to_workspace:
root: /tmp
paths:
- src/coverage/.coverage.py37
- src/coverage/.coverage.py39

style_check:
docker:
Expand Down Expand Up @@ -254,7 +287,6 @@ workflows:
build_test:
jobs:
- makeenv_37
- unittest_35
- unittest_36
- unittest_37:
requires:
Expand All @@ -274,11 +306,14 @@ workflows:
- t2smap:
requires:
- makeenv_37
- unittest_38
- unittest_39
- merge_coverage:
requires:
- unittest_35
- unittest_36
- unittest_37
- unittest_38
- unittest_39
- three-echo
- four-echo
- five-echo
Expand Down
5 changes: 4 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -108,4 +108,7 @@ ENV/

# jupyter notebooks
.ipynb_checkpoints/
*.ipynb
*.ipynb

# vim swap files
*.swp
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ More information and documentation can be found at https://tedana.readthedocs.io
### Use `tedana` with your local Python environment

You'll need to set up a working development environment to use `tedana`.
To set up a local environment, you will need Python >=3.5 and the following packages will need to be installed:
To set up a local environment, you will need Python >=3.6 and the following packages will need to be installed:

* [numpy>=1.14](http://www.numpy.org/)
* [scipy](https://www.scipy.org/)
Expand Down
1 change: 0 additions & 1 deletion docs/api.rst
Original file line number Diff line number Diff line change
Expand Up @@ -161,7 +161,6 @@ API
tedana.io.add_decomp_prefix
tedana.io.split_ts
tedana.io.write_split_ts
tedana.io.writefeats
tedana.io.writeresults
tedana.io.writeresults_echoes

Expand Down
16 changes: 8 additions & 8 deletions docs/approach.rst
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ estimate voxel-wise :math:`T_{2}^*` and :math:`S_0`.
:math:`S_0` corresponds to the total signal in each voxel before decay and can reflect coil sensivity.
:math:`T_{2}^*` corresponds to the rate at which a voxel decays over time, which
is related to signal dropout and BOLD sensitivity.
Estimates of the parameters are saved as **t2sv.nii.gz** and **s0v.nii.gz**.
Estimates of the parameters are saved as **T2starmap.nii.gz** and **S0map.nii.gz**.

While :math:`T_{2}^*` and :math:`S_0` in fact fluctuate over time, estimating
them on a volume-by-volume basis with only a small number of echoes is not
Expand Down Expand Up @@ -153,7 +153,7 @@ between the distributions for other echoes.

The time series for the optimally combined data also looks like a combination
of the other echoes (which it is).
This optimally combined data is written out as **ts_OC.nii.gz**
This optimally combined data is written out as **desc-optcom_bold.nii.gz**

.. image:: /_static/a10_optimal_combination_timeseries.png

Expand Down Expand Up @@ -194,7 +194,7 @@ component analysis (PCA).
The goal of this step is to make it easier for the later ICA decomposition to converge.
Dimensionality reduction is a common step prior to ICA.
TEDPCA applies PCA to the optimally combined data in order to decompose it into component maps and
time series (saved as **mepca_mix.1D**).
time series (saved as **desc-PCA_mixing.tsv**).
Here we can see time series for some example components (we don't really care about the maps):

.. image:: /_static/a11_pca_component_timeseries.png
Expand Down Expand Up @@ -277,9 +277,9 @@ Next, ``tedana`` applies TE-dependent independent component analysis (ICA) in
order to identify and remove TE-independent (i.e., non-BOLD noise) components.
The dimensionally reduced optimally combined data are first subjected to ICA in
order to fit a mixing matrix to the whitened data.
This generates a number of independent timeseries (saved as **meica_mix.1D**),
as well as beta maps which show the spatial loading of these components on the
brain (**betas_OC.nii.gz**).
This generates a number of independent timeseries (saved as **desc-ICA_mixing.tsv**),
as well as parameter estimate maps which show the spatial loading of these components on the
brain (**desc-ICA_components.nii.gz**).

.. image:: /_static/a13_ica_component_timeseries.png

Expand Down Expand Up @@ -312,13 +312,13 @@ The blue and red lines show the predicted values for the :math:`S_0` and
A decision tree is applied to :math:`\kappa`, :math:`\rho`, and other metrics in order to
classify ICA components as TE-dependent (BOLD signal), TE-independent
(non-BOLD noise), or neither (to be ignored).
These classifications are saved in `comp_table_ica.txt`.
These classifications are saved in **desc-tedana_metrics.tsv**.
The actual decision tree is dependent on the component selection algorithm employed.
``tedana`` includes the option `kundu` (which uses hardcoded thresholds
applied to each of the metrics).

Components that are classified as noise are projected out of the optimally combined data,
yielding a denoised timeseries, which is saved as `dn_ts_OC.nii.gz`.
yielding a denoised timeseries, which is saved as **desc-optcomDenoised_bold.nii.gz**.

.. image:: /_static/a15_denoised_data_timeseries.png

Expand Down
2 changes: 1 addition & 1 deletion docs/installation.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ Installation
------------

You'll need to set up a working development environment to use ``tedana``.
To set up a local environment, you will need Python >=3.5 and the following
To set up a local environment, you will need Python >=3.6 and the following
packages will need to be installed:

- nilearn
Expand Down
Loading