Skip to content

Commit

Permalink
Used Ruff to reformat
Browse files Browse the repository at this point in the history
  • Loading branch information
wfondrie committed Apr 15, 2024
1 parent c910c4d commit 6730879
Show file tree
Hide file tree
Showing 22 changed files with 106 additions and 62 deletions.
24 changes: 0 additions & 24 deletions .github/workflows/black.yml

This file was deleted.

30 changes: 30 additions & 0 deletions .github/workflows/lint.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
name: Lint

on:
push:
branches: [ main ]
pull_request:
branches: [ main ]

jobs:
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Setup Python 3.10
uses: actions/setup-python@v4
with:
python-version: "3.10"

- name: Install Ruff
run: |
python -m pip install --upgrade pip
pip install ruff
- name: Lint with Ruff
run: |
ruff check . --output-format=github
- name: Check formatting with Ruff
run: |
ruff format --check .
28 changes: 15 additions & 13 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,17 @@
# Changelog for ppx

## [1.4.0]
### Changed
- Linting and formatting is now performed with Ruff.

### Fixed
- Fixed MassIVE FTP URLs using the MassIVE PROXI API.
- Fixed PRIDE FTP URLs... again.
- Updated unit tests with new PRIDE FTP URLs.

## [1.3.0] - 2022-04-39
### Added
- Support for cloud provider destinations. Thanks @sooheon!
- Support for cloud provider destinations. Thanks @sooheon!

## [1.2.6] - 2022-03-16
### Fixed
Expand Down Expand Up @@ -47,25 +50,25 @@

## [1.2.0] - 2021-09-14
### Added
- New `timeout` parameter for most functions and classes. This specifies the
- New `timeout` parameter for most functions and classes. This specifies the
maximum amount of time to wait for a response from the server.

### Changed
- The backend for MassIVE now uses the GNPS API to list files and projects,
only falling back to scraping the FTP server on failure. This should make
only falling back to scraping the FTP server on failure. This should make
it much faster. Thanks @mwang87!
- Files and projects are now returned in sorted order.

### Fixed
- Poor connections with PRIDE were leading to a number of occasional errors.
Multiple reconnect attempts are now tried for a wider variety of FTP
- Poor connections with PRIDE were leading to a number of occasional errors.
Multiple reconnect attempts are now tried for a wider variety of FTP
operations.

## [1.1.1] - 2021-07-02
### Fixed
- Downloading files is now more robust. ppx will now retry FTP connections up
to 10 times if the connection was dropped or refused.
- Partial downloads are now continued automatically by comparing the local
- Partial downloads are now continued automatically by comparing the local
file size to the remote file size.

## [1.1.0] - 2021-05-18
Expand All @@ -75,26 +78,25 @@
instead. **Note that this may change the number, identity, and order of the
file that were previously returned for PRIDE projects!**
- Small documentation updates.

### Added
- Caching of the remote files and directories found for a project. If a
- Caching of the remote files and directories found for a project. If a
project's `fetch` attribute is `False`, then we'll rely on this cached
data, so long as it is available. Setting `fetch=True` will always refresh
the data from the repository.

## [1.0.0] - 2021-05-14
### Changed
## [1.0.0] - 2021-05-14
### Changed
- **We did a complete rework of the API!** This will break nearly all previous
code using ppx, but greatly improves its versatility.
See the [docs](https://ppx.readthedocs.io) for more details
- Updated the build to align with
[PEP517](https://www.python.org/dev/peps/pep-0517/)

### Added
- A command line interface for downloading files from PRIDE and MassIVE
- A command line interface for downloading files from PRIDE and MassIVE
projects.
- Additional unit tests.
- A ppx logo
- This changelog.
- ppx is now available on bioconda!

1 change: 1 addition & 0 deletions ppx/__init__.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
"""See the README for detailed documentation and examples."""

try:
from importlib.metadata import PackageNotFoundError, version

Expand Down
4 changes: 3 additions & 1 deletion ppx/config.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
"""This module contains the configuration details for ppx"""

import logging
import os
from pathlib import Path
Expand Down Expand Up @@ -43,7 +44,8 @@ def path(self, path):
path = self._resolve_path(path)
if not path.exists():
raise FileNotFoundError(
f"The specified directory or bucket ({path}) " "does not exist."
f"The specified directory or bucket ({path}) "
"does not exist."
)

self._path = path
Expand Down
13 changes: 7 additions & 6 deletions ppx/factory.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
This is the foundation of the ppx package.
"""

import logging
import re
from urllib.parse import urlparse
Expand Down Expand Up @@ -78,11 +79,11 @@ def id(self):

def find(self):
"""Find the dataset at the partner repository"""
kwargs = dict(
local=self._local,
fetch=self._fetch,
timeout=self._timeout,
)
kwargs = {
"local": self._local,
"fetch": self._fetch,
"timeout": self._timeout,
}

if self._repo == "PRIDE":
return PrideProject(self._repo_id, **kwargs)
Expand Down Expand Up @@ -164,7 +165,7 @@ def find_project(identifier, local=None, repo=None, fetch=False, timeout=10.0):
repo = str(repo).lower()

# User-specified:
kwargs = dict(local=local, fetch=fetch, timeout=timeout)
kwargs = {"local": local, "fetch": fetch, "timeout": timeout}
if repo == "pride":
return PrideProject(identifier, **kwargs)

Expand Down
1 change: 1 addition & 0 deletions ppx/ftp.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
"""General utilities for working with the repository FTP sites."""

import logging
import re
import socket
Expand Down
19 changes: 11 additions & 8 deletions ppx/massive.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
"""MassIVE datasets."""

import logging
import re
import socket
Expand Down Expand Up @@ -50,12 +51,12 @@ class MassiveProject(BaseProject):
def __init__(self, msv_id, local=None, fetch=False, timeout=10.0):
"""Instantiate a MSVDataset object"""
super().__init__(msv_id, local, fetch, timeout)
self._params = dict(
_stream="on",
_sort="filepath",
dataset__exact=self.id,
_size="max",
)
self._params = {
"_stream": "on",
"_sort": "filepath",
"dataset__exact": self.id,
"_size": "max",
}

def _validate_id(self, identifier):
"""Validate a MassIVE identifier.
Expand Down Expand Up @@ -143,7 +144,9 @@ def remote_files(self, glob=None):
if self.fetch or self._remote_files is None:
try:
info = self.file_info().splitlines()[1:]
self._remote_files = [r.split(",")[0].split("/", 1)[1] for r in info]
self._remote_files = [
r.split(",")[0].split("/", 1)[1] for r in info
]
assert self._remote_files
except (
TimeoutError,
Expand Down Expand Up @@ -211,7 +214,7 @@ def list_projects(timeout=10.0):
"""
url = "https://gnps-datasetcache.ucsd.edu/datasette/database.csv"
params = dict(sql="select distinct dataset from filename", _size="max")
params = {"sql": "select distinct dataset from filename", "_size": "max"}
try:
res = requests.get(url, params, timeout=timeout).text.splitlines()[1:]
res.sort()
Expand Down
27 changes: 19 additions & 8 deletions ppx/ppx.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
"""The command line entry point for ppx"""

import logging
import sys
from argparse import ArgumentParser
Expand All @@ -11,17 +12,20 @@

def get_parser():
"""Parse the command line arguments"""
desc = """Use this command line utility to download files from the PRIDE and MassIVE
proteomics repositories. The paths to the downloaded files are written to
stdout."""
desc = """Use this command line utility to download files from the PRIDE
and MassIVE proteomics repositories. The paths to the downloaded files are
written to stdout."""

epilog = "More documentation and examples at: https://ppx.readthedocs.io"
parser = ArgumentParser(description=desc, epilog=epilog)

parser.add_argument(
"identifier",
type=str,
help=("The ProteomeXchange, PRIDE, or MassIVE identifier for the " "project."),
help=(
"The ProteomeXchange, PRIDE, or MassIVE identifier for the "
"project."
),
)

parser.add_argument(
Expand Down Expand Up @@ -79,7 +83,9 @@ def get_parser():

def main():
"""Run ppx"""
logging.basicConfig(level=logging.INFO, format="[%(levelname)s]: %(message)s")
logging.basicConfig(
level=logging.INFO, format="[%(levelname)s]: %(message)s"
)

parser = get_parser()
args = parser.parse_args()
Expand All @@ -95,16 +101,21 @@ def main():
matches.update(pat_match)

if not all(passed):
failed = " \n".join([f for f, p in zip(args.files, passed) if not p])
failed = " \n".join(
[f for f, p in zip(args.files, passed) if not p]
)

raise FileNotFoundError(
"Unable to find one or more of the files or patterns:" f"\n {failed}"
"Unable to find one or more of the files or patterns:"
f"\n {failed}"
)

else:
matches = remote_files

LOGGER.info("Downloading %i files from %s...", len(matches), args.identifier)
LOGGER.info(
"Downloading %i files from %s...", len(matches), args.identifier
)
downloaded = proj.download(matches)

for local_file in downloaded:
Expand Down
1 change: 1 addition & 0 deletions ppx/pride.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
"""A class for PRIDE datasets"""

import json
import re

Expand Down
5 changes: 4 additions & 1 deletion ppx/project.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
"""A base dataset class"""

from abc import ABC, abstractmethod
from pathlib import Path

Expand Down Expand Up @@ -239,7 +240,9 @@ def download(self, files, force_=False, silent=False):
f"{', '.join(missing)}"
)

return self._parser.download(files, self.local, force_=force_, silent=silent)
return self._parser.download(
files, self.local, force_=force_, silent=silent
)


def cache(files, cache_file, fetch):
Expand Down
1 change: 1 addition & 0 deletions setup.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
"""Setup ppx"""

import setuptools

setuptools.setup()
1 change: 1 addition & 0 deletions tests/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
- A mock FTP server response from PRIDE
- A mock FTP server response from MassIVE
"""

import ftplib
import json
import socket
Expand Down
1 change: 1 addition & 0 deletions tests/system_tests/test_cli.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
"""Test the command line interface"""

import subprocess

PXID = "PXD000001"
Expand Down
1 change: 1 addition & 0 deletions tests/unit_tests/test_cloud.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
"""Test using cloud locations"""

from cloudpathlib import CloudPath

import ppx
Expand Down
1 change: 1 addition & 0 deletions tests/unit_tests/test_config.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
"""Test the configuration"""

from pathlib import Path

import pytest
Expand Down
1 change: 1 addition & 0 deletions tests/unit_tests/test_download.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
These tests are in a separate file because they all require internet access.
"""

from ftplib import FTP, error_temp

import pytest
Expand Down
1 change: 1 addition & 0 deletions tests/unit_tests/test_find_project.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
"""Test finding projects"""

import pytest
from requests.exceptions import ConnectTimeout, ReadTimeout

Expand Down
1 change: 1 addition & 0 deletions tests/unit_tests/test_list_projects.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
"""Test that the list_projects() functions work"""

import pytest
import requests

Expand Down
1 change: 1 addition & 0 deletions tests/unit_tests/test_massive.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
"""Test MassIVE functionality w/o internet access"""

import shutil
from pathlib import Path

Expand Down
Loading

0 comments on commit 6730879

Please sign in to comment.