Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: stack-viewer v2 using ndv as baseclass #299

Open
wants to merge 80 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
80 commits
Select commit Hold shift + click to select a range
0321817
wip
tlambert03 Apr 24, 2024
452c661
more progress
tlambert03 Apr 24, 2024
0d5b085
wip
tlambert03 Apr 24, 2024
1f42588
some fixes
tlambert03 Apr 25, 2024
1731f7c
wip
tlambert03 May 1, 2024
a332228
getting better
tlambert03 May 2, 2024
d066d77
some linting
tlambert03 May 2, 2024
be222a4
wip on composite mode
tlambert03 May 2, 2024
0c3edb5
starting pygfx
tlambert03 May 2, 2024
aa412e4
more updates
tlambert03 May 2, 2024
60101e3
more wip
tlambert03 May 3, 2024
d727dcd
wip
tlambert03 May 3, 2024
57187d2
wip
tlambert03 May 4, 2024
b066f3a
good progress
tlambert03 May 4, 2024
fb66297
more progress for xarray and numpy
tlambert03 May 4, 2024
f63328e
better pygfx
tlambert03 May 4, 2024
a7cb3e0
linting and cleanup
tlambert03 May 4, 2024
2215d50
save btn
tlambert03 May 4, 2024
4172741
remove qt backend
tlambert03 May 4, 2024
63e61d9
move and rename
tlambert03 May 4, 2024
91ecd0c
more renames
tlambert03 May 4, 2024
81758ec
remove bar color
tlambert03 May 6, 2024
b34b4df
bump superqt and add popup fps
tlambert03 May 6, 2024
d2ff45b
use async, add colors, start transform
tlambert03 May 6, 2024
c3ae898
fix typo
tlambert03 May 7, 2024
fa51abe
Merge branch 'main' into stack-viewer2
tlambert03 May 7, 2024
5ebdcc3
rename
tlambert03 May 7, 2024
d07876e
Merge branch 'stack-viewer2' of https://github.com/tlambert03/pymmcor…
tlambert03 May 7, 2024
ca4347d
hide sliders of size=1
tlambert03 May 7, 2024
f7cddf5
minor typing
tlambert03 May 7, 2024
5aae2af
additive mode
tlambert03 May 7, 2024
31cbf64
futures
tlambert03 May 7, 2024
fb66e29
more futurestuff
tlambert03 May 7, 2024
5eb5462
more cleanup
tlambert03 May 8, 2024
b0cb5da
style(pre-commit.ci): auto fixes [...]
pre-commit-ci[bot] May 8, 2024
f1fc4a2
changes to throttling
tlambert03 May 9, 2024
b4175ae
style(pre-commit.ci): auto fixes [...]
pre-commit-ci[bot] May 9, 2024
90827ff
change demo
tlambert03 May 10, 2024
cb9d031
remove print
tlambert03 May 10, 2024
c6e7ee1
add tensor store
tlambert03 May 10, 2024
5b6aed6
wip
tlambert03 May 10, 2024
f100cf9
use tensorstore backingh
tlambert03 May 11, 2024
120a47e
fix imports
tlambert03 May 11, 2024
7c895c4
channel names and docs
tlambert03 May 12, 2024
580cb20
style(pre-commit.ci): auto fixes [...]
pre-commit-ci[bot] May 12, 2024
f02d2fa
fix
tlambert03 May 12, 2024
155b45c
rearrange and fix delayed requests
tlambert03 May 12, 2024
5f84dde
style(pre-commit.ci): auto fixes [...]
pre-commit-ci[bot] May 12, 2024
5e21c23
Merge branch 'main' into stack-viewer2
tlambert03 May 24, 2024
a136f72
add xarray example
tlambert03 May 24, 2024
2efce9c
two fixes
tlambert03 May 29, 2024
857991b
tried to fix leaks. but failed
tlambert03 Jun 2, 2024
d3e3169
use data-wrapper
tlambert03 Jun 3, 2024
550d5d6
style(pre-commit.ci): auto fixes [...]
pre-commit-ci[bot] Jun 3, 2024
4441085
minor
tlambert03 Jun 3, 2024
0591afc
Merge branch 'stack-viewer2' of https://github.com/tlambert03/pymmcor…
tlambert03 Jun 3, 2024
759956d
misc
tlambert03 Jun 3, 2024
98b2717
fix one leak
tlambert03 Jun 3, 2024
18a6047
small changes
tlambert03 Jun 3, 2024
325a90e
more indexing
tlambert03 Jun 3, 2024
c7cfe6f
remove check
tlambert03 Jun 3, 2024
5bf478c
wip
tlambert03 Jun 3, 2024
88e93a6
style(pre-commit.ci): auto fixes [...]
pre-commit-ci[bot] Jun 3, 2024
2df4ce3
don't autoscale invisible
tlambert03 Jun 5, 2024
7a8d0cf
Merge branch 'stack-viewer2' of https://github.com/tlambert03/pymmcor…
tlambert03 Jun 5, 2024
527a737
start adding 3d
tlambert03 Jun 7, 2024
63c2c47
remove print
tlambert03 Jun 7, 2024
b26e0bf
better 3d
tlambert03 Jun 7, 2024
631d421
more tweaks
tlambert03 Jun 7, 2024
f3d3932
use ndv
tlambert03 Jun 8, 2024
fc2a792
add data wrapper
tlambert03 Jun 8, 2024
346e66e
add comment
tlambert03 Jun 8, 2024
064d72d
add dep
tlambert03 Jun 8, 2024
7d3632b
Merge branch 'main' into use-ndv
tlambert03 Jun 9, 2024
3d80b01
bump deps
tlambert03 Jun 9, 2024
0f359c2
lint
tlambert03 Jun 9, 2024
be728cd
remove test
tlambert03 Jun 9, 2024
8957e87
add test
tlambert03 Jun 9, 2024
ee879a9
style(pre-commit.ci): auto fixes [...]
pre-commit-ci[bot] Jun 9, 2024
9be8843
Merge branch 'main' into use-ndv
fdrgsp Jul 5, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -27,5 +27,5 @@ repos:
- id: mypy
files: "^src/"
additional_dependencies:
- pymmcore-plus >=0.9.0
- pymmcore-plus >=0.10.0
- useq-schema >=0.4.7
32 changes: 32 additions & 0 deletions examples/mda_viewer.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
from __future__ import annotations

from pymmcore_plus import CMMCorePlus, configure_logging
from qtpy import QtWidgets
from useq import MDASequence

from pymmcore_widgets._stack_viewer_v2._mda_viewer import MDAViewer

configure_logging(stderr_level="WARNING")

mmcore = CMMCorePlus.instance()
mmcore.loadSystemConfiguration()
mmcore.defineConfig("Channel", "DAPI", "Camera", "Mode", "Artificial Waves")
mmcore.defineConfig("Channel", "DAPI", "Camera", "StripeWidth", "1")
mmcore.defineConfig("Channel", "FITC", "Camera", "Mode", "Artificial Waves")
mmcore.defineConfig("Channel", "FITC", "Camera", "StripeWidth", "4")

sequence = MDASequence(
channels=({"config": "DAPI", "exposure": 1}, {"config": "FITC", "exposure": 1}),
stage_positions=[(0, 0), (1, 1)],
z_plan={"range": 9, "step": 0.4},
time_plan={"interval": 0.2, "loops": 4},
# grid_plan={"rows": 2, "columns": 1},
)


qapp = QtWidgets.QApplication([])
v = MDAViewer()
v.show()

mmcore.run_mda(sequence, output=v.data)
qapp.exec()
24 changes: 24 additions & 0 deletions examples/mda_viewer_queue.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
import sys
from queue import Queue

from pymmcore_plus import CMMCorePlus
from qtpy import QtWidgets
from useq import MDAEvent

from pymmcore_widgets._stack_viewer_v2._mda_viewer import MDAViewer

app = QtWidgets.QApplication(sys.argv)
mmcore = CMMCorePlus.instance()
mmcore.loadSystemConfiguration()

canvas = MDAViewer()
canvas.show()

q = Queue()
mmcore.run_mda(iter(q.get, None), output=canvas.data)
for i in range(10):
for c in range(2):
q.put(MDAEvent(index={"t": i, "c": c}, exposure=1))
q.put(None)

app.exec()
4 changes: 2 additions & 2 deletions examples/stack_viewer.py → examples/stack_viewer_v1.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
from qtpy import QtWidgets
from useq import MDASequence

from pymmcore_widgets.experimental import StackViewer
from pymmcore_widgets._stack_viewer_v1 import StackViewer

size = 1028

Expand All @@ -20,7 +20,7 @@

sequence = MDASequence(
channels=(
# {"config": "DAPI", "exposure": 10},
{"config": "DAPI", "exposure": 10},
# {"config": "FITC", "exposure": 1},
{"config": "Cy5", "exposure": 1},
),
Expand Down
16 changes: 13 additions & 3 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -50,14 +50,23 @@ dependencies = [
'fonticon-materialdesignicons6',
'pymmcore-plus[cli] >=0.10.2',
'qtpy >=2.0',
'superqt[quantity] >=0.5.3',
'superqt[quantity] >=0.6.5',
'useq-schema >=0.4.7',
'ndv >=0.0.3',
]

# extras
# https://peps.python.org/pep-0621/#dependencies-optional-dependencies
[project.optional-dependencies]
test = ["pytest>=6.0", "pytest-cov", "pytest-qt", "PyYAML", "vispy", "cmap", "zarr"]
test = [
"pytest>=6.0",
"pytest-cov",
"pytest-qt",
"PyYAML",
"vispy",
"cmap",
"zarr",
]
pyqt5 = ["PyQt5"]
pyside2 = ["PySide2"]
pyqt6 = ["PyQt6"]
Expand Down Expand Up @@ -129,6 +138,7 @@ docstring-code-format = true
[tool.pytest.ini_options]
minversion = "6.0"
testpaths = ["tests"]
markers = ["allow_leaks: mark test to allow widget leaks"]
filterwarnings = [
"error",
"ignore:distutils Version classes are deprecated",
Expand Down Expand Up @@ -174,4 +184,4 @@ ignore = [
]

[tool.typos.default]
extend-ignore-identifiers-re = ["(?i)ome"]
extend-ignore-identifiers-re = ["(?i)nd2?.*", "(?i)ome"]
Original file line number Diff line number Diff line change
Expand Up @@ -464,11 +464,9 @@ def _reload_position(self) -> None:
self.cmap_names = self.qt_settings.value("cmaps", ["gray", "cyan", "magenta"])

def _collapse_view(self) -> None:
w, h = self.img_size
view_rect = (
(
self.view_rect[0][0] - self.img_size[0] / 2,
self.view_rect[0][1] + self.img_size[1] / 2,
),
(self.view_rect[0][0] - w / 2, self.view_rect[0][1] + h / 2),
self.view_rect[1],
)
self.view.camera.rect = view_rect
Expand Down
3 changes: 3 additions & 0 deletions src/pymmcore_widgets/_stack_viewer_v2/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
from ._mda_viewer import MDAViewer

__all__ = ["MDAViewer"]
89 changes: 89 additions & 0 deletions src/pymmcore_widgets/_stack_viewer_v2/_data_wrapper.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
from __future__ import annotations

import warnings
from contextlib import suppress
from typing import TYPE_CHECKING, cast

from ndv import DataWrapper
from pymmcore_plus.mda.handlers import TensorStoreHandler

if TYPE_CHECKING:
from collections.abc import Hashable, Mapping
from pathlib import Path
from typing import Any, TypeGuard

import numpy as np
from ndv import Indices
from pymmcore_plus.mda.handlers._5d_writer_base import _5DWriterBase


class MMTensorStoreWrapper(DataWrapper["TensorStoreHandler"]):
def sizes(self) -> Mapping[Hashable, int]:
with suppress(Exception):
return self.data.current_sequence.sizes # type: ignore
return {}

def guess_channel_axis(self) -> Hashable | None:
return "c"

@classmethod
def supports(cls, obj: Any) -> TypeGuard[TensorStoreHandler]:
return isinstance(obj, TensorStoreHandler)

def isel(self, indexers: Indices) -> np.ndarray:
return self.data.isel({str(k): v for k, v in indexers.items()}) # type: ignore [no-any-return]

def save_as_zarr(self, save_loc: str | Path) -> None:
if (store := self.data.store) is None:
return
import tensorstore as ts

Check warning on line 39 in src/pymmcore_widgets/_stack_viewer_v2/_data_wrapper.py

View check run for this annotation

Codecov / codecov/patch

src/pymmcore_widgets/_stack_viewer_v2/_data_wrapper.py#L37-L39

Added lines #L37 - L39 were not covered by tests

new_spec = store.spec().to_json()
new_spec["kvstore"] = {"driver": "file", "path": str(save_loc)}
new_ts = ts.open(new_spec, create=True).result()
new_ts[:] = store.read().result()

Check warning on line 44 in src/pymmcore_widgets/_stack_viewer_v2/_data_wrapper.py

View check run for this annotation

Codecov / codecov/patch

src/pymmcore_widgets/_stack_viewer_v2/_data_wrapper.py#L41-L44

Added lines #L41 - L44 were not covered by tests


class MM5DWriter(DataWrapper["_5DWriterBase"]):
def guess_channel_axis(self) -> Hashable | None:
return "c"

Check warning on line 49 in src/pymmcore_widgets/_stack_viewer_v2/_data_wrapper.py

View check run for this annotation

Codecov / codecov/patch

src/pymmcore_widgets/_stack_viewer_v2/_data_wrapper.py#L49

Added line #L49 was not covered by tests

@classmethod
def supports(cls, obj: Any) -> TypeGuard[_5DWriterBase]:
try:
from pymmcore_plus.mda.handlers._5d_writer_base import _5DWriterBase

Check warning on line 54 in src/pymmcore_widgets/_stack_viewer_v2/_data_wrapper.py

View check run for this annotation

Codecov / codecov/patch

src/pymmcore_widgets/_stack_viewer_v2/_data_wrapper.py#L53-L54

Added lines #L53 - L54 were not covered by tests
except ImportError:
from pymmcore_plus.mda.handlers import OMETiffWriter, OMEZarrWriter

_5DWriterBase = (OMETiffWriter, OMEZarrWriter) # type: ignore
if isinstance(obj, _5DWriterBase):
return True
return False

Check warning on line 61 in src/pymmcore_widgets/_stack_viewer_v2/_data_wrapper.py

View check run for this annotation

Codecov / codecov/patch

src/pymmcore_widgets/_stack_viewer_v2/_data_wrapper.py#L59-L61

Added lines #L59 - L61 were not covered by tests

def save_as_zarr(self, save_loc: str | Path) -> None:
import zarr
from pymmcore_plus.mda.handlers import OMEZarrWriter

Check warning on line 65 in src/pymmcore_widgets/_stack_viewer_v2/_data_wrapper.py

View check run for this annotation

Codecov / codecov/patch

src/pymmcore_widgets/_stack_viewer_v2/_data_wrapper.py#L64-L65

Added lines #L64 - L65 were not covered by tests

if isinstance(self._data, OMEZarrWriter):
zarr.copy_store(self._data.group.store, zarr.DirectoryStore(save_loc))

Check warning on line 68 in src/pymmcore_widgets/_stack_viewer_v2/_data_wrapper.py

View check run for this annotation

Codecov / codecov/patch

src/pymmcore_widgets/_stack_viewer_v2/_data_wrapper.py#L67-L68

Added lines #L67 - L68 were not covered by tests
raise NotImplementedError(f"Cannot save {type(self._data)} data to Zarr.")

def isel(self, indexers: Indices) -> np.ndarray:
p_index = indexers.get("p", 0)
if isinstance(p_index, slice):
warnings.warn("Cannot slice over position index", stacklevel=2) # TODO
p_index = p_index.start
p_index = cast(int, p_index)

Check warning on line 76 in src/pymmcore_widgets/_stack_viewer_v2/_data_wrapper.py

View check run for this annotation

Codecov / codecov/patch

src/pymmcore_widgets/_stack_viewer_v2/_data_wrapper.py#L72-L76

Added lines #L72 - L76 were not covered by tests

try:
sizes = [*list(self._data.position_sizes[p_index]), "y", "x"]
except IndexError as e:
raise IndexError(

Check warning on line 81 in src/pymmcore_widgets/_stack_viewer_v2/_data_wrapper.py

View check run for this annotation

Codecov / codecov/patch

src/pymmcore_widgets/_stack_viewer_v2/_data_wrapper.py#L78-L81

Added lines #L78 - L81 were not covered by tests
f"Position index {p_index} out of range for "
f"{len(self._data.position_sizes)}"
) from e

data = self._data.position_arrays[self._data.get_position_key(p_index)]
full = slice(None, None)
index = tuple(indexers.get(k, full) for k in sizes)
return data[index] # type: ignore [no-any-return]

Check warning on line 89 in src/pymmcore_widgets/_stack_viewer_v2/_data_wrapper.py

View check run for this annotation

Codecov / codecov/patch

src/pymmcore_widgets/_stack_viewer_v2/_data_wrapper.py#L86-L89

Added lines #L86 - L89 were not covered by tests
95 changes: 95 additions & 0 deletions src/pymmcore_widgets/_stack_viewer_v2/_mda_viewer.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
from __future__ import annotations

import warnings
from pathlib import Path
from typing import TYPE_CHECKING, Any, Mapping

import superqt
import useq
from ndv import DataWrapper, NDViewer
from pymmcore_plus.mda.handlers import TensorStoreHandler
from qtpy.QtWidgets import QFileDialog, QPushButton, QWidget
from superqt.iconify import QIconifyIcon

# this import is necessary so that ndv can find our custom DataWrapper
from . import _data_wrapper # noqa: F401

if TYPE_CHECKING:
from pymmcore_plus.mda.handlers._5d_writer_base import _5DWriterBase
from qtpy.QtWidgets import QWidget


class MDAViewer(NDViewer):
"""StackViewer specialized for pymmcore-plus MDA acquisitions."""

_data: _5DWriterBase

def __init__(
self,
datastore: _5DWriterBase | TensorStoreHandler | None = None,
*,
parent: QWidget | None = None,
):
if datastore is None:
datastore = TensorStoreHandler()

# patch the frameReady method to call the superframeReady method
# AFTER handling the event
self._superframeReady = getattr(datastore, "frameReady", None)
if callable(self._superframeReady):
datastore.frameReady = self._patched_frame_ready # type: ignore
else: # pragma: no cover
warnings.warn(
"MDAViewer: datastore does not have a frameReady method to patch, "
"are you sure this is a valid data handler?",
stacklevel=2,
)

super().__init__(datastore, parent=parent, channel_axis="c")
self._save_btn = SaveButton(self._data_wrapper)
self._btns.addWidget(self._save_btn)
self.dims_sliders.set_locks_visible(True)
self._channel_names: dict[int, str] = {}

def _patched_frame_ready(self, *args: Any) -> None:
self._superframeReady(*args) # type: ignore
if len(args) >= 2 and isinstance(e := args[1], useq.MDAEvent):
self._on_frame_ready(e)

@superqt.ensure_main_thread # type: ignore
def _on_frame_ready(self, event: useq.MDAEvent) -> None:
c = event.index.get(self._channel_axis)
if c not in self._channel_names and c is not None and event.channel:
self._channel_names[c] = event.channel.config
self.set_current_index(event.index)

def _get_channel_name(self, index: Mapping) -> str:
if self._channel_axis in index:
if name := self._channel_names.get(index[self._channel_axis]):
return name
c = index.get(self._channel_axis, 0)
return f"Ch {c}"

Check warning on line 71 in src/pymmcore_widgets/_stack_viewer_v2/_mda_viewer.py

View check run for this annotation

Codecov / codecov/patch

src/pymmcore_widgets/_stack_viewer_v2/_mda_viewer.py#L67-L71

Added lines #L67 - L71 were not covered by tests


class SaveButton(QPushButton):
def __init__(
self,
data_wrapper: DataWrapper,
parent: QWidget | None = None,
):
super().__init__(parent=parent)
self.setIcon(QIconifyIcon("mdi:content-save"))
self.clicked.connect(self._on_click)

self._data_wrapper = data_wrapper
self._last_loc = str(Path.home())

def _on_click(self) -> None:
self._last_loc, _ = QFileDialog.getSaveFileName(

Check warning on line 88 in src/pymmcore_widgets/_stack_viewer_v2/_mda_viewer.py

View check run for this annotation

Codecov / codecov/patch

src/pymmcore_widgets/_stack_viewer_v2/_mda_viewer.py#L88

Added line #L88 was not covered by tests
self, "Choose destination", str(self._last_loc), ""
)
suffix = Path(self._last_loc).suffix
if suffix in (".zarr", ".ome.zarr", ""):
self._data_wrapper.save_as_zarr(self._last_loc)

Check warning on line 93 in src/pymmcore_widgets/_stack_viewer_v2/_mda_viewer.py

View check run for this annotation

Codecov / codecov/patch

src/pymmcore_widgets/_stack_viewer_v2/_mda_viewer.py#L91-L93

Added lines #L91 - L93 were not covered by tests
else:
raise ValueError(f"Unsupported file format: {self._last_loc}")

Check warning on line 95 in src/pymmcore_widgets/_stack_viewer_v2/_mda_viewer.py

View check run for this annotation

Codecov / codecov/patch

src/pymmcore_widgets/_stack_viewer_v2/_mda_viewer.py#L95

Added line #L95 was not covered by tests
5 changes: 3 additions & 2 deletions src/pymmcore_widgets/experimental.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
from ._stack_viewer import StackViewer
from ._stack_viewer_v1 import StackViewer
from ._stack_viewer_v2 import MDAViewer

__all__ = ["StackViewer"]
__all__ = ["StackViewer", "MDAViewer"]
Loading
Loading