Merge pull request #78 from EmbroidePy/tatarize-color+json

Adds color file formats and JSON, addresses #73 and #71

README.md

@@ -23,8 +23,8 @@ pyembroidery must to be small enough to be finished in short order and big enoug
 * pyembroidery must support and function in Python 2.7
 
 Pyembroidery fully meets and exceeds all of these requirements.
-* It writes 13 formats, including the mandated ones.
-* It reads 41 formats, including the mandated ones.
+* It writes 9 embroidery formats including the mandated ones. 19 different format in total.
+* It reads 40 embroidery formats including the mandated ones. 46 different formats in total.
 * It supports all the core commands where that format can use said command as well as FAST and SLOW for .u01.
 * SEQUINS work in all supported formats (.dst) that are known to support sequins. Further it supports SEQUIN to JUMP operations on the other formats.
   * It is currently fully compatable with Python 2.7 and Python 3.6
@@ -102,7 +102,7 @@ In most cases this information isn't going to matter, but it is provided because
 
 ## File I/O
 
-### Formats:
+### Embroidery Formats:
 
 Pyembroidery will write:
 * .pes (mandated)
@@ -113,10 +113,6 @@ Pyembroidery will write:
 * .u01
 * .pec
 * .xxx
-* .csv
-* .svg
-* .png
-* .txt
 * .gcode
 
 Pyembroidery will read:
@@ -158,31 +154,70 @@ Pyembroidery will read:
 * .u01
 * .xxx
 * .zxy
-* .csv
 * .gcode
 
+### Related Formats
+Pyembroidery includes some related formats like pure color formats that can be loaded as helpers for formats without colors. Or .pmv which is a stitch pattern format for Brother sewing machines.
+
+Pyembroidery will write:
+* .col : Color format.
+* .edr : Color format.
+* .inf : Color format.
+* .pmv : Brother Stitch Format.
+
+Pyembroidery will read:
+* .col : Color format.
+* .edr : Color format.
+* .inf : Color format.
+* .pmv : Brother Stitch Format.
+
+
+### Utility Formats:
+CSV and JSON are the two primary forms of lossless saving formats. PNG is an image format, txt output is sometimes used for easy parsing. And SVG is an open source interchange format for vectors.
+
+Pyembroidery will write:
+* .csv : comma-separated values 
+* .json : JavaScript Object Notation
+* .png : Portable Network Graphic
+* .txt : text file.
+* .svg : Scalable Vector Graphics
+
+Pyembroidery will read:
+* .csv : comma-separated values 
+* .json : JavaScript Object Notation
+
+
 #### Versions
-They .get_supported_formats() in pyembroidery provides an element called 'versions' this will contain a tuple of values which can be passed to the settings object as `{'version': <my version>}`. This provides much of the version controls for the types of outputs provided within each writer. For example, there is an extended header version of dst called `extended`, there's "6" and "6t" in the PesWriter which exports other and different versions of the file. 
+Formats within .get_supported_formats() in pyembroidery provides an element called 'versions' this will contain a tuple of values which can be passed to the settings object as `{'version': <my version>}`. This provides version controls for the types of outputs provided within each writer. For example, there is an extended header version of dst called `extended`, there's `6` and `6t` in the PesWriter which exports other and different versions of the file. 
 
-This is also intended as a good method to create new versions. For example, gcode can control a great many thing and varies greatly from one purpose to another and would be ideal for different versions. There's also different machines which may require different tweaks and these would be extended as versions.
+This is intended as a good method to create new versions. For example, gcode can control a many things, but varies greatly from one purpose to another and would be ideal for different versions. There are also different embroidery machines which may require different tweaks. Versions is the intended method to deliver machine specific files.
 
 #### Writing to CSV:
 Prints out a workable CSV file with the given data. Starting in 1.3 the csv patterns are written without being encoded. The CSV format is, in this form, lossless. If you wish to encode before you write the file you can set the encoder to True and override the default.
 
 `write_csv(pattern, "file.csv", {"encode": True})`
 
+
+#### Reading/Writing to JSON:
+Saves the pattern as a JSON object. This is intended to be useful as an interchange format since JSON is the most common data interchange format availible currently. 
+
+
 #### Writing to PNG:
 Writes to a image/png file.
 
+
 #### Writing to TXT:
 Writes to a text file. Generally lossy, it does not write threads or metadata, but certainly more easily parsed for a number of homebrew applications. The "mimic" option should mimic the embroidermodder functionality for exporting to txt files. By default it exports a bit less lossy giving the proper command indexes and their explicit names.
 
+
 #### Writing to Gcode:
-The Gcode is intended for a number of hobbiest projects that use a gcode controller to operate a sewing machine, usually X,Y for plotter and Z to turn the handwheel. However, if you have a hobbiest project and need a different command structure feel free to ask or discuss it by raising an issue.
+The Gcode is intended for a number of hobbiest projects that use a gcode controller to operate a sewing machine, usually X,Y for plotter and Z to turn the handwheel. However, if you have a hobbiest project and need a different command structure feel free to ask or discuss it by raising an issue. Other gcode versions should be able to be added through the versions method.
+
 
 #### Reading from HUS:
 The HUS format requires an obscure and defunct form of compression. The EmbCompress performs this decompression. It is written from the ground up in pure python. It does not require any compiled element or dll file. It has no obfuscation and is intended to be easily understood.
 
+
 ### Reading
 
 ```python
@@ -518,7 +553,7 @@ Sequins being written into files that do not support sequins can go several ways
 
 ### Tie On / Tie Off Contingency
 
-While there's only NONE, and THREE_SMALL for contingencies here, both the tie-on and tie-off contingencies are setup to be forward compatabile with other future potential tie-on and tie-off methods.
+While there's only NONE, and THREE_SMALL for contingencies currently, both the tie-on and tie-off contingencies are setup to be forward compatabile with other future potential tie-on and tie-off methods.
 
 ### Units
 

pyembroidery/ColReader.py

@@ -0,0 +1,14 @@
+from .EmbThread import *
+
+READ_FILE_IN_TEXT_MODE = True
+
+
+def read(f, out, settings=None):
+    count = int(f.readline())
+    for i in range(0,count):
+        line = f.readline()
+        splits = line.split(',')
+        thread = EmbThread()
+        thread.catalog_number = splits[0]
+        thread.set_color(int(splits[1]), int(splits[2]), int(splits[3]))
+        out.add_thread(thread)

pyembroidery/ColWriter.py

@@ -0,0 +1,15 @@
+from .WriteHelper import write_string_utf8
+
+ENCODE = False
+
+
+def write(pattern, f, settings=None):
+    write_string_utf8(f, "%d\r\n" % len(pattern.threadlist))
+    index = 0
+    for thread in pattern.threadlist:
+        write_string_utf8(f, "%d,%d,%d,%d\r\n" % (
+            index,
+            thread.get_red(),
+            thread.get_green(),
+            thread.get_blue()))
+        index += 1

pyembroidery/EdrReader.py

@@ -0,0 +1,15 @@
+from .EmbThread import EmbThread
+from .ReadHelper import read_int_8
+
+
+def read(f, out, settings=None):
+    while True:
+        red = read_int_8(f)
+        green = read_int_8(f)
+        blue = read_int_8(f)
+        if blue is None:
+            return
+        f.seek(1, 1)
+        thread = EmbThread()
+        thread.set_color(red, green, blue)
+        out.add_thread(thread)

pyembroidery/EdrWriter.py

@@ -0,0 +1,12 @@
+from .WriteHelper import write_int_8
+
+ENCODE = False
+
+
+def write(pattern, f, settings=None):
+    if len(pattern.threadlist) > 0:
+        for thread in pattern.threadlist:
+            write_int_8(f, thread.get_red())
+            write_int_8(f, thread.get_green())
+            write_int_8(f, thread.get_blue())
+            write_int_8(f, 0)

pyembroidery/InfReader.py

@@ -0,0 +1,32 @@
+from .EmbThread import EmbThread
+from .ReadHelper import read_int_32be, read_int_16be
+
+
+def read(f, out, settings=None):
+    u0 = read_int_32be(f)
+    u1 = read_int_32be(f)
+    u2 = read_int_32be(f)
+    number_of_colors = read_int_32be(f)
+    for j in range(0, number_of_colors):
+        length = read_int_16be(f) - 2  # 2 bytes of the length.
+        byte_data = bytearray(f.read(length))
+        if len(byte_data) != length:
+            break
+        red = byte_data[2]
+        green = byte_data[3]
+        blue = byte_data[4]
+        thread = EmbThread()
+        thread.set_color(red, green, blue)
+        byte_data = byte_data[7:]
+        for j in range(0, len(byte_data)):
+            b = byte_data[j]
+            if b == 0:
+                thread.description = byte_data[:j].decode('utf8')
+                byte_data = byte_data[j+1:]
+                break
+        for j in range(0, len(byte_data)):
+            b = byte_data[j]
+            if b == 0:
+                thread.chart = byte_data[:j].decode('utf8')
+                break
+        out.add_thread(thread)

pyembroidery/InfWriter.py

@@ -0,0 +1,39 @@
+from .WriteHelper import write_int_8, write_int_32be, write_int_16be, write_string_utf8
+
+ENCODE = False
+
+
+def patch_byte_offset(stream, offset):
+    current_pos = stream.tell()
+    stream.seek(offset, 0)  # Absolute position seek.
+    position = current_pos - offset - 4  # 4 bytes int32
+    write_int_32be(stream, position)
+    stream.seek(current_pos, 0)  # Absolute position seek.
+
+
+def write(pattern, f, settings=None):
+    write_int_32be(f, 1)
+    write_int_32be(f, 8)
+    placeholder = f.tell()
+    write_int_32be(f, 0)  # Placeholder.
+    write_int_32be(f, len(pattern.threadlist))
+    index = 0
+    for thread in pattern.threadlist:
+        details = thread.description
+        if details is None:
+            details = "Unknown"
+        chart = thread.chart
+        if chart is None:
+            chart = "Unknown"
+        write_int_16be(f, 11 + len(details) + len(chart))  # 2 + 2 + 1 + 1 + 1 + 2 + d + 1 + c + 1 = 11 + d + c
+        write_int_16be(f, index)  # record index
+        index += 1
+        write_int_8(f, thread.get_red())
+        write_int_8(f, thread.get_green())
+        write_int_8(f, thread.get_blue())
+        write_int_16be(f, index)  # needle number
+        write_string_utf8(f, details)
+        write_int_8(f, 0)
+        write_string_utf8(f, chart)
+        write_int_8(f, 0)
+    patch_byte_offset(f, placeholder)

pyembroidery/JsonReader.py

@@ -0,0 +1,46 @@
+from .EmbFunctions import *
+from .EmbThread import EmbThread
+
+
+def decoded_command(command_dict,name):
+    split = name.split(' ')
+    command = command_dict[split[0]]
+    for sp in split[1:]:
+        if sp[0] == "n":
+            needle = int(sp[1:])
+            command |= (needle + 1) << 16
+        if sp[0] == "o":
+            order = int(sp[1:])
+            command |= (order + 1) << 24
+        if sp[0] == "t":
+            thread = int(sp[1:])
+            command |= (thread + 1) << 8
+    return command
+
+
+def read(f, out, settings=None):
+    import json
+    json_object = json.load(f)
+    command_dict = get_command_dictionary()
+    stitches = json_object['stitches']
+    extras = json_object['extras']
+    threadlist = json_object['threadlist']
+    for t in threadlist:
+        color = t["color"]
+        thread = EmbThread(color)
+        thread.description = t["description"]
+        thread.catalog_number = t["catalog_number"]
+        thread.details = t["details"]
+        thread.brand = t["brand"]
+        thread.chart = t["chart"]
+        thread.weight = t["weight"]
+        out.add_thread(thread)
+    for s in stitches:
+        out.stitches.append(
+            [
+                s[0],
+                s[1],
+                decoded_command(command_dict, s[2])
+            ]
+        )
+    out.extras.update(extras)

pyembroidery/JsonWriter.py

@@ -0,0 +1,42 @@
+from .EmbFunctions import *
+
+ENCODE = False
+WRITE_FILE_IN_TEXT_MODE = True
+
+
+def decoded_name(names, data):
+    command = decode_embroidery_command(data)
+    try:
+        name = names[command[0]]
+        if command[1] is not None:
+            name = name + " t" + str(command[1])
+        if command[2] is not None:
+            name = name + " n" + str(command[2])
+        if command[3] is not None:
+            name = name + " o" + str(command[3])
+    except (IndexError, KeyError):
+        name = "UNKNOWN " + str(data)
+    return name
+
+
+def write(pattern, f, settings=None):
+    import json
+    names = get_common_name_dictionary()
+
+    json_normal = {
+        "threadlist": [
+            {
+                "color": thread.color,
+                "description": thread.description,
+                "catalog_number": thread.catalog_number,
+                "details": thread.details,
+                "brand": thread.brand,
+                "chart": thread.chart,
+                "weight": thread.weight
+            }
+            for thread in pattern.threadlist
+        ],
+        "stitches": [[s[0], s[1], str(decoded_name(names, s[2]))] for s in pattern.stitches],
+        "extras": pattern.extras
+    }
+    json.dump(json_normal, f, indent=4)