Skip to content

Commit

Permalink
Several minor improvements (#209)
Browse files Browse the repository at this point in the history
* Add github workflow
* Replace nose as pytest for py3.7+
* Apply code style to flake8 + black
  • Loading branch information
caesar0301 authored Mar 27, 2023
1 parent 226dd03 commit 70221d6
Show file tree
Hide file tree
Showing 21 changed files with 647 additions and 373 deletions.
59 changes: 59 additions & 0 deletions .github/workflows/python-package.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
# This workflow will install Python dependencies, run tests and lint with a variety of Python versions
# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python
name: Package testing

on:
push:
branches: [ "master", "dev" ]
pull_request:
branches: [ "master" ]

jobs:
build-pre37:

runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
python-version: ["2.7"]

steps:
- uses: actions/checkout@v3
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v3
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
python -m pip install --upgrade pip
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
if [ -f requirements-t-pre37.txt ]; then pip install -r requirements-t-pre37.txt; fi
- name: Test with nosetests
run: |
./scripts/testing.sh
build:

runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
python-version: ["3.7", "3.8", "3.9", "3.10", "3.11"]

steps:
- uses: actions/checkout@v3
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v3
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
python -m pip install --upgrade pip
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
if [ -f requirements-t.txt ]; then pip install -r requirements-t.txt; fi
- name: Lint with flake8
run: |
./scripts/flake8.sh
- name: Test with pytest
run: |
pytest
3 changes: 3 additions & 0 deletions HISTORY
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
Mar 27 2023 V1.6.3
Apply to black code style. Migrate future to six

Oct 13 2019 V1.5.6
Add ability to independently mutate multiple shallow copies of same initial tree.

Expand Down
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,8 @@ treelib

Tree implementation in python: simple for you to use.

[![Build Status](https://travis-ci.org/caesar0301/treelib.svg?branch=master)](https://travis-ci.org/caesar0301/treelib)
[![Build Status](https://github.com/caesar0301/treelib/actions/workflows/python-package.yml/badge.svg)](https://github.com/caesar0301/treelib/actions)
[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
[![Documentation Status](https://readthedocs.org/projects/treelib/badge/?version=latest)](http://treelib.readthedocs.io/en/latest/?badge=latest)
[![Status](https://img.shields.io/pypi/status/treelib.svg)](https://pypi.python.org/pypi/treelib)
[![Latest](https://img.shields.io/pypi/v/treelib.svg)](https://pypi.python.org/pypi/treelib)
Expand Down
22 changes: 11 additions & 11 deletions examples/family_tree.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
#
# Author: chenxm
#
__author__ = 'chenxm'
__author__ = "chenxm"

from treelib import Tree, Node

Expand All @@ -21,45 +21,45 @@ def create_family_tree():


def example(desp):
sep = "-"*20 + '\n'
sep = "-" * 20 + "\n"
print(sep + desp)


if __name__ == '__main__':
if __name__ == "__main__":
tree = create_family_tree()

example("Tree of the whole family:")
tree.show(key=lambda x: x.tag, reverse=True, line_type='ascii-em')
tree.show(key=lambda x: x.tag, reverse=True, line_type="ascii-em")

example("All family members in DEPTH mode:")
print(','.join([tree[node].tag for node in tree.expand_tree()]))
print(",".join([tree[node].tag for node in tree.expand_tree()]))

example("All family members (with identifiers) but Diane's sub-family:")
tree.show(idhidden=False, filter=lambda x: x.identifier != 'diane')
tree.show(idhidden=False, filter=lambda x: x.identifier != "diane")

example("Let me introduce Diane family only:")
sub_t = tree.subtree('diane')
sub_t = tree.subtree("diane")
sub_t.show()

example("Children of Diane:")
for child in tree.is_branch('diane'):
for child in tree.is_branch("diane"):
print(tree[child].tag)

example("New members join Jill's family:")
new_tree = Tree()
new_tree.create_node("n1", 1) # root node
new_tree.create_node("n2", 2, parent=1)
new_tree.create_node("n3", 3, parent=1)
tree.paste('bill', new_tree)
tree.paste("bill", new_tree)
tree.show()

example("They leave after a while:")
tree.remove_node(1)
tree.show()

example("Now Mary moves to live with grandfather Harry:")
tree.move_node('mary', 'harry')
tree.move_node("mary", "harry")
tree.show()

example("A big family for Mark to send message to the oldest Harry:")
print(','.join([tree[node].tag for node in tree.rsearch('mark')]))
print(",".join([tree[node].tag for node in tree.rsearch("mark")]))
97 changes: 57 additions & 40 deletions examples/folder_tree.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
# and pattern variables
#

__author__ = 'holger'
__author__ = "holger"

from treelib import tree

Expand Down Expand Up @@ -33,9 +33,13 @@
import cProfile


parser = argparse.ArgumentParser(description='Scan the given folder and print its structure in a tree.')
parser.add_argument('abspath', type=str, help='An absolute path to be scanned.')
parser.add_argument('pattern', type=str, help='File name pattern to filtered, e.g. *.pdf')
parser = argparse.ArgumentParser(
description="Scan the given folder and print its structure in a tree."
)
parser.add_argument("abspath", type=str, help="An absolute path to be scanned.")
parser.add_argument(
"pattern", type=str, help="File name pattern to filtered, e.g. *.pdf"
)

args = parser.parse_args()
rootPath = args.abspath
Expand All @@ -44,39 +48,46 @@
folder_blacklist = []

dir_tree = tree.Tree()
dir_tree.create_node('Root', rootPath) # root node
dir_tree.create_node("Root", rootPath) # root node


def crc32(data):
data = bytes(data, 'UTF-8')
data = bytes(data, "UTF-8")

if DEBUG:
print('++++++ CRC32 ++++++')
print('input: ' + str(data))
print('crc32: ' + hex(zlib.crc32(data) & 0xffffffff))
print('+++++++++++++++++++')
return hex(zlib.crc32(data) & 0xffffffff) # crc32 returns a signed value, &-ing it will match py3k
print("++++++ CRC32 ++++++")
print("input: " + str(data))
print("crc32: " + hex(zlib.crc32(data) & 0xFFFFFFFF))
print("+++++++++++++++++++")
return hex(
zlib.crc32(data) & 0xFFFFFFFF
) # crc32 returns a signed value, &-ing it will match py3k


parent = rootPath
i = 1

# calculating start depth
start_depth = rootPath.count('/')
start_depth = rootPath.count("/")


def get_noteid(depth, root, dir):
""" get_noteid returns
- depth contains the current depth of the folder hierarchy
- dir contains the current directory
"""get_noteid returns
- depth contains the current depth of the folder hierarchy
- dir contains the current directory
Function returns a string containing the current depth, the folder name and unique ID build by hashing the
absolute path of the directory. All spaces are replaced by '_'
Function returns a string containing the current depth, the folder name and unique ID build by hashing the
absolute path of the directory. All spaces are replaced by '_'
<depth>_<dirname>+++<crc32>
e.g. 2_Folder_XYZ_1+++<crc32>
<depth>_<dirname>+++<crc32>
e.g. 2_Folder_XYZ_1+++<crc32>
"""
return str(str(depth) + '_' + dir).replace(" ", "_") + '+++' + crc32(os.path.join(root, dir))
return (
str(str(depth) + "_" + dir).replace(" ", "_")
+ "+++"
+ crc32(os.path.join(root, dir))
)


# TODO: Verzeichnistiefe pruefen: Was ist mit sowas /mp3/

Expand All @@ -92,20 +103,26 @@ def get_parentid(current_depth, root, dir):
# get 'parent_folder'

search_string = os.path.join(root, dir)
pos2 = search_string.rfind('/')
pos1 = search_string.rfind('/', 0, pos2)
parent_dir = search_string[pos1 + 1:pos2]
parentid = str(current_depth - 1) + '_' + parent_dir.replace(" ", "_") + '+++' + crc32(root)
pos2 = search_string.rfind("/")
pos1 = search_string.rfind("/", 0, pos2)
parent_dir = search_string[pos1 + 1 : pos2]
parentid = (
str(current_depth - 1)
+ "_"
+ parent_dir.replace(" ", "_")
+ "+++"
+ crc32(root)
)
return parentid
# TODO: catch error


def print_node(dir, node_id, parent_id):
print('#############################')
print('node created')
print(' dir: ' + dir)
print(' note_id: ' + node_id)
print(' parent: ' + parent_id)
print("#############################")
print("node created")
print(" dir: " + dir)
print(" note_id: " + node_id)
print(" parent: " + parent_id)


def crawler():
Expand All @@ -118,10 +135,10 @@ def crawler():
for dir in dirs:

# calculating current depth
current_depth = os.path.join(root, dir).count('/') - start_depth
current_depth = os.path.join(root, dir).count("/") - start_depth

if DEBUG:
print('current: ' + os.path.join(root, dir))
print("current: " + os.path.join(root, dir))

node_id = get_noteid(current_depth, root, dir)
parent_id = str(get_parentid(current_depth, root, dir))
Expand All @@ -143,10 +160,10 @@ def crawler():
continue

# calculating current depth
current_depth = os.path.join(root, filename).count('/') - start_depth
current_depth = os.path.join(root, filename).count("/") - start_depth

if DEBUG:
print('current: ' + os.path.join(root, filename))
print("current: " + os.path.join(root, filename))

node_id = get_noteid(current_depth, root, filename)
parent_id = str(get_parentid(current_depth, root, filename))
Expand All @@ -166,28 +183,28 @@ def crawler():
crawler()
if PROFILING == 1:
t1 = timeit.Timer("crawler()", "from __main__ import crawler")
print('time: ' + str(t1.timeit(number=1)))
print("time: " + str(t1.timeit(number=1)))
if PROFILING == 2:
cProfile.run("crawler()")


print('filecount: ' + str(FILECOUNT))
print('dircount: ' + str(DIRCOUNT))
print("filecount: " + str(FILECOUNT))
print("dircount: " + str(DIRCOUNT))

if DIR_ERRORLIST:
for item in DIR_ERRORLIST:
print(item)
else:
print('no directory errors')
print("no directory errors")

print('\n\n\n')
print("\n\n\n")

if FILE_ERRORLIST:
for item in FILE_ERRORLIST:
print(item)
else:
print('no file errors')
print("no file errors")

print('nodes: ' + str(len(dir_tree.nodes)))
print("nodes: " + str(len(dir_tree.nodes)))

dir_tree.show()
16 changes: 11 additions & 5 deletions examples/recursive_dirtree_generator.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@


def get_random_string(length):
return ''.join(random.choice(digits + letters) for _ in range(length))
return "".join(random.choice(digits + letters) for _ in range(length))


def build_recursive_tree(tree, base, depth, width):
Expand All @@ -34,8 +34,12 @@ def build_recursive_tree(tree, base, depth, width):
depth -= 1
for i in xrange(width):
directory = Directory()
tree.create_node("{0}".format(directory.name), "{0}".format(hashlib.md5(directory.name)),
parent=base.identifier, data=directory) # node identifier is md5 hash of it's name
tree.create_node(
"{0}".format(directory.name),
"{0}".format(hashlib.md5(directory.name)),
parent=base.identifier,
data=directory,
) # node identifier is md5 hash of it's name
dirs_nodes = tree.children(base.identifier)
for dir in dirs_nodes:
newbase = tree.get_node(dir.identifier)
Expand All @@ -47,7 +51,9 @@ def build_recursive_tree(tree, base, depth, width):
class Directory(object):
def __init__(self):
self._name = get_random_string(64)
self._files = [File() for _ in xrange(MAX_FILES_PER_DIR)] # Each directory contains 1000 files
self._files = [
File() for _ in xrange(MAX_FILES_PER_DIR)
] # Each directory contains 1000 files

@property
def name(self):
Expand All @@ -68,7 +74,7 @@ def name(self):


tree = treelib.Tree()
base = tree.create_node('Root', 'root')
base = tree.create_node("Root", "root")
build_recursive_tree(tree, base, 2, 10)

tree.show()
2 changes: 1 addition & 1 deletion examples/save_tree_2_file.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,4 +9,4 @@
tree.create_node("Diane", "diane", parent="jane")
tree.create_node("Mary", "mary", parent="diane")
tree.create_node("Mark", "mark", parent="jane")
tree.save2file('tree.txt')
tree.save2file("tree.txt")
2 changes: 1 addition & 1 deletion requirements-testing.txt → requirements-t-pre37.txt
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
nose>=1.3.7
coverage>=4.4.1
nose-exclude>=0.5.0
coverage>=4.4.1
5 changes: 5 additions & 0 deletions requirements-t.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
pytest>=7.0.0
coverage>=4.4.1
flake8==5.0.0
flake8-black==0.3.6
black==23.1.0
1 change: 1 addition & 0 deletions requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
six>=1.13.0
3 changes: 3 additions & 0 deletions scripts/flake8.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
#!/bin/bash
flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
flake8 . --count --exit-zero --max-line-length=127 --statistics
File renamed without changes.
Loading

0 comments on commit 70221d6

Please sign in to comment.