Skip to content

Latest commit

 

History

History
321 lines (293 loc) · 13.7 KB

TODO.md

File metadata and controls

321 lines (293 loc) · 13.7 KB

--- Post open-sourcing (to prioritise)

  • Improve test coverage
  • Add doxygen-style comments
  • Decuple UI from Engine (THE SPLIT)
  • Introduce mocks for AudioContext & Web Audio APIs

--- Refactoring

  • Rename ComboBox component to something better (SearchBox? PanelCreator? PanelSelector?)
  • Extract everything that can be extracted from panel factories

--- System integration

  • Main menu' commands
    • new
    • save
    • save as
    • open
    • import (loads file without wiping workarea)
  • Ask for confirmation before replacing current document in "new" and "open" actions
  • Ask for confirmation before overriding an existing file in "save as" action
  • Settings page
  • Plugin system
  • Add lateral scrolling to toolbar to be able to reach out-of-screen items
  • Main-thread service framework to avoid squashing everything into electron.js

--- Engine

  • THE SPLIT: Separate the execution code (the engine) from the rendering code (the react frontend)

    • Create a separate API daemon that can receive and process requests from the Electron app
    • Modify UI to work with API
    • Integrate CLI with API
    • The engine should take a representation of the network and a representation of the current state and store them
    • Once the state is set, it can be updated at any time
    • The engine can be queried for the next step, in which case it should execute one iteration and respond with the updated state
    • The engine should carry out any side effect
    • Make a CLI that can use the engine daemon to run a network in an headless way
    • Add types of connectors that can only transmit when in play
    • Transform all panel types to work on the engine

  • Stepper-type panels or endpoints or connections, still not sure which

  • Explicitly handle the "null/undefined" value in a more organic way

  • Needs more orthogonality: possibility to set both inputs and outputs from execute or onPulse, etc

  • Actual panels

    • VIDEO: Extract frame
    • VIDEO: Resize
    • VIDEO: Create from images
    • IMAGE: Image group
    • FLOW: Timeline (place events on a timeline and on play it will start triggering Pulses on the various eps
    • Web Audio nodes (experimental)
      • FFT/IFFT?
      • Mic input
      • Save audio
      • Panner
      • Channel Merger
      • Channel Splitter
      • Filter
      • Convolver
      • Gain
      • Integrate SamplePlay into SampleLoad
      • AudioOut: sink to which to connect sound producers
      • SampleLoad: load a sample
      • SamplePlay: config of a sample (loop etc)
      • Oscillator: play a frequency
    • P5.js integration
    • IMAGE: Transform (4 points)
    • IMAGE: Threshold
    • Add features to manipulate channels
    • 3d and vector graphics integration?
      • Three.js integration
    • Add turbulence input to generators
    • TILES: Tile
    • IMAGE: Path
    • IMAGE: Shape / Polygon
    • IMAGE: Polyline
    • IMAGE: FFT / IFFT
    • IMAGE: Pixelate
    • IMAGE: Mix?
    • IMAGE: Crop
    • IMAGE: Move
    • HASHES: Hash functions
      • Md5
      • Sha1
    • ALGEBRA: Unify scalar and dot product in a single panel
    • SOURCE: Glob
    • FLOW: For: (from:Value + to:Value + step:Value + Pulse -> Pulse + Pulse + Value) expose index variable and emit "execute" every loop until "exit"
    • ?????: (Value + Value -> Value) Date formatter according to pattern and separator
    • INPUTS: File info
    • Envelope visualizer (multi-input pin with value for each? Not sure)
    • TEXTURES: Noise generators
      • Gaussian noise
      • Simplex noise (take w/h + seed)
      • Anisotropic noise
    • ADJUSTMENTS
      • Posterize
      • Contrast
      • Brightness
      • Saturation
      • Hue
      • Lighten
      • Darken
      • Fade
    • TEXTURES: Add different types of generators
      • IMAGE: Stripes
      • IMAGE: Checkers
      • IMAGE: Rings
    • IMAGE: Unary Operators
      • [~] IMAGE: Canny
      • IMAGE: Invert colors
      • IMAGE: FlipX
      • IMAGE: FlipY
      • IMAGE: Grey
      • IMAGE: Sepia
    • IMAGE: Blend Modes
      • Dissolve
      • Darker Color
      • Lighter Color
      • Hue
      • Saturation
      • Color
      • Luminosity
      • Darken
      • Multiply
      • Color Burn
      • Linear Burn
      • Lighten
      • Screen
      • Color Dodge
      • Linear Dodge
      • Overlay
      • Soft Light
      • Vivid Light
      • Hard Light
      • Linear Light
      • Pin Light
      • Hard Mix
      • Difference
      • Exclusion
      • Subtract
      • Divide
    • Stdin/Stdout/Stderr
      • Stdin
      • Stderr?
      • Stdout
    • Operations with Arrays/Collections? How? What about objects? They are certainly needed for api calls etc
      • Basic Array panel (List)
        • Add Pop, Shift, Unshift pins
      • Merge
      • Intersection
      • Unique (Distinct)
      • Basic Object panel (Dictionary)
    • Color ramp
    • IMAGE: ImageSave
    • IMAGE: Heightmap
    • TEXTURES: Gradient
      • Linear Gradient
      • Circular Gradient
    • Hex2HSL introduce tinycolor
    • TEXTURES: Add angle to patterns
    • TEXTURES: Linear Gradient
    • INPUTS: Gradient Picker
    • Introduce tinycolor in components with hex input
    • IMAGE: Resize
    • IMAGE: Combine Channels
    • IMAGE: Replace Dilate and Erode with Morphology Operators
      • Dilate
      • Erode
      • Open
      • Close
      • Black Hat
      • Top Hat
    • IMAGE: Text
    • IMAGE: Erode
    • IMAGE: Dilate
    • IMAGE: CopyImage
    • IMAGE: Rescale
    • IMAGE: Create Empty Image (take w/h + background color, default transparent)
    • IMAGE: Get Channel
    • IMAGE: Rotate
    • IMAGE: Gaussian blur
    • IMAGE: imageView must have checkered background
    • IMAGE: imageView wrong panel size on first image load
    • IMAGE: Convolution
    • INPUT: Folder path
    • Add explicit support for different object types
      • Represent Matrices in TextView
      • Represent Matrices in DisplayLogger
      • Represent Matrices in FileLogger
    • ALGEBRA: Matrix summation
    • ALGEBRA: Normalize matrix
    • ALGEBRA: Matrix multiplication
    • ALGEBRA: Matrix
    • INPUT: time selector
    • INPUT: date selector
    • SET: item by item array
    • FLOW: All: (multiple Pulses -> single Pulse) waits for a pulse on each of the inputs and then sends a pulse through the output
    • TRANSFORMS: Date/time to Timestamp
    • TRANSFORMS: Timestamp to Date/time
    • ALGEBRA: Threshold: (Value -> Value) sets outputs (High, Equal, Low) to true depending on whether first input is <=> than second
    • INPUT: Toggle: (UI -> Value) set output to true or false depending on toggle
    • BASIC: Counter: (Pulse -> Value) count pulses (also has a Reset pin)
    • BASIC: Memory: (Pulse + Value -> Value) store content of value input when it receives a pulse
    • INPUT: Button: (UI -> Pulse) send a pulse when user presses button
    • FLOW: If: (Value -> Pulse) send pulse along Then or Else branches based on value of expression
    • FLOW: Delta: (Value -> Pulse) send a pulse when input changes
    • OUTPUT: Semaphore: (Value -> UI) turn on a led if the value is non-null, red or green depending on truthy value
    • OUTPUT: Logfile: (Value + Pulse -> OS) append message to file
    • INPUT: Deviator: (Value + Value -> Value) The output to 1 of N inputs based on the value of another input
    • FLOW: Delay: (Pulse -> X secs -> Pulse)
    • EVENTS: Clock: (every X secs -> Pulse)
    • INPUT: Color Picker
    • INPUT: Knob with inputs for min, max, and step
    • INPUT: Range with inputs for min, max, and step
    • OUTPUT: Logger: (a more complex mix of Display and LogFile that keeps N lines of text and adds a new line upon receiving a Pulse)
    • OUTPUT: Led: Similar to Semaphore but with color input
    • SOURCES: (Pulse -> Value) Fetch Current Date
    • SOURCES: (Pulse -> Value) Fetch Current Time
    • SOURCES: (Pulse -> Value) Fetch Random number between 0 and 1
    • SOURCES: (Pulse -> Value) Fetch Current timestamp
    • SOURCES: Text file, line by line
    • FLOW: Gate (pass transistor)
    • OUTPUTS: Color view
    • EVENTS: Emit on file change
    • EVENTS: Socket: socket listener, together with CLI tool to connect and send messages
    • EVENTS: Accumulator

  • Can we introduce jimp as well? Is it interoperable with image-js? what would it add?

  • Introduce Pulse-type signals

  • Introduce Sources panel palette

-- UI

  • Grouped panels should be surrounded by an outline
  • Grouped panels can be hidden and replaced by a placeholder?
  • Change editable property to become "editor" and change it from boolean to text where you specify the type of editor
  • Choose which eps to make editable
  • Make the context menu and combo box coordinates window-sensitive
  • Make width of collapsed panel a constant (currently the magic number 120)
  • Make magic numbers become constants
  • Make endpoint section in the panels collapsible
  • Editable eps' names have dotted underlining when connected
  • Editable eps should close on enter
  • Add scale input ep (editable) to range panel
  • Add scale input ep (editable) to knob panel
  • Pulling a connection from an endpoint to the workarea opens Panel search
    • Panel search is constrained by the initial endpoint being an input or output endpoint
    • Panel search is constrained by signal and type of the initial endpoint
  • Make selected panels get closer or farther away by rolling the scroller
  • Add tags to all panels
  • Double clicking on the workarea opens Panel search
  • Autofocus search box
  • Implement Panel Search box
  • [~] Move all visual state from panels to panelCoords
    • width and height, minWidth and minHeight
    • resizer
  • Review css of ep inline editor
  • Setting default values for primitive endpoints inline
  • Double clicking on title of panel to collapse a panel completely
  • Make Connector component require coordinates rather than elements, so that the logic can be made more flexible
  • Expose signal of endpoint throught color
  • Introduce white halo around focused panel
  • Lock children (and/or Lock parents?) of a panel OR lock selected panels so they always move together
  • Unlock a panel that had been locked with others
  • Expose type of endpoint throught symbols/letters
  • Expose type of endpoint in tooltip
  • Make background of text input panel darker
  • Introduce window manager

--- Bugs / Performance

  • select exclusive seems to select anything that is to the left of the selected area if the panel is vertically included

  • Review all components and apply change guards (don't process if inputs have not changed)

  • Introduce ndarray module in matrices etc.

  • State management seems to be the main bottleneck, general simplification of the state management seems to be necessary

    • Try transforming the "tuning" variables into local component state variables, so that they only redraw the component
    • Try reducing actual panel state to a single "redraw" property to be set to a random value when we want to update the component, the rest of the state can stay in a non-state array

  • Maintain and use endpoint offsets relative to panel position (only rely on screen state when updating the offsets)

  • Dilate and Erode block the UI while being applied?

  • Disallow Value-type signals from creating feedback loops

  • Sanitize topic input to listener (how?)

  • Stalls when trying to rewind file source from EOF signal without a delay

  • Values don't seem to propagate correctly when creating a panel from an endpoint

  • Window resize causes slowdown and stalling --- PROBABLY FIXED

  • Only render panels in view

  • Consider ways of reducing number of connections shown (bounding box overlapping view?)

  • Connectors losing contact with panels

  • Stopping clock connected to dice makes dice output go undefined

  • Deletion of panels must affect also panelCoords

  • Save-files should include position of viewport

  • Crash when deleting multiple panels

  • Crash when I pause with the Controls button

  • WatchFile panel should remove and replace watcher behind the scenes when file name changes, just like the Listener panel does

--- QA

  • Fix issue with tests
  • Improve test coverage

--- Context menu

  • Disconnect panel -> removes all connections
  • Duplicate panel -> creates an clone of the panel
  • Find origin -> moves view to --middle of the graph-- NO: it moves you back to [0, 0]
  • "Delete panel" becomes "Delete panels" when used on selected panel
  • Group/Ungroup panels: grouped panels move together

--- NOTES

  • If possible decouple UI panels from Pulse-activated panels

  • When having a UI panel that also has Pulse-type inputs pins, the two avenues of activation may end up overwriting each other's outputs causing bugs