Filesystem panel

[tobozo]

• Fixed Arduino IDE closing when closing the applet
• Restored FS/merge panel (work still in progress)

[serifpersia]

Let me know when you have working fs/merge working again. I would have to test both esptool and espota(wireless upload) before merge. If there is windows incompatibility will have to correct that.

[tobozo]

okay I have pushed what I've done so far, but more stuff will be added soon so please don't overwrite my fork yet :)

The fs/merge tools are working but there are requirements:

• Partition scheme should be exported first (Export CSV button) as partitions.csv in the sketch folder
• The sketch should be (re)compiled in the Arduino IDE before using upload/merge buttons, and after any change in the configuration

Cosmetic changes

• Platformio fixes

  • Rounded borders JTextField/JComboBox
  • Cells opacity
  • Scroll panel borders

• Import/export csv

  • File filter (.csv)
  • Try to keep the dialog on top
  • CSV dir defaults to sketch folder
  • CSV filename defaults to "partitions.csv"

TODO

• Block SPIFFS/Merge buttons usage when:
  • Partition scheme was modified but not exported to the sketch dir
  • Configuration changed but the sketch wasn't recompiled (dunno yet how to detect that)
• Add "Upload Merged Bin" button
• Add "Export Merged Bin" button
• Add "Save CSV" button (aliases exportCSV([sketch dir]/partitions.csv) )

[tobozo]

• new Merge & Upload Binary button
• clear build folder and change the window title (mimic 'unsaved document') when:
  • Partition scheme was modified
  • Configuration changed
• rebuild sketch if necessary
• show a progress bar while rebuilding sketch (not being refreshed though, dunno why)

[tobozo]

• console log window is restored, with a color hint for errors, and the ability to clear logs

[serifpersia]

Looking good there!

[tobozo]

• fix for serial port not selected from the right source
• fix for spiffs offset not being properly retrieved
• fixed repaint problem when re-opening the applet
• detached some actions from main event loop to prevent UI from freezing

don't trust the failed video capture, it is completely fluid on my desktop 😉

esp32partitiontool.mp4

[tobozo]

next task will be to add the missing "Save CSV" button, and I'll be finished with this iteration

this button is necessary because the current workflow exclusively relies on the temporary build folder; so if any partition editing occured and the user hasn't used "Export CSV" to explicitely overwrite "partitions.csv" in the sketch folder, the partition changes will be ignored by SPIFFS and Merge tools

this situation also applies when using the "Import CSV" button: the loaded custom partition scheme is only living in the GUI until it has been saved or exported

in order to prevent that, the partitions.csv file in the sketch folder (not the build folder) should be overwritten without prompt whenever necessary (e.g. on SPIFFS/Merge button press)

additionnally the "Save CSV" button can be grayed out when saving isn't necessary

[tobozo]

• Made CSV column names sticky (e.g. when a scrollbar appears)
• Adjusted Enable row: removed redundant label from checkboxes (also was glitchy with rollover)
• Added new Save CSV button: always overwrites {sketch-dir}/partitions.csv with arduino, and `args[0]' with platformio
• Grouped Load/Save/Export/Help/About buttons and replaced labels by images; some alternate icons are available in the resource folder
• Added tooltips (no delay) to buttons, partition graph items and csv file name
• Adjusted margins/borders for many panels

[serifpersia]

I pulled your pr changes for me it doesn't look the same as your images. Did you push those changes already? I'm not seeing the merge panel and the log panel. This is how it looks for me(Arduino IDE) I assume the fspanel should be visible only for Arduino IDE version but im not seeing it in Arduino IDE

[tobozo]

if the app falls back to standalone design, then one of the tests failed in AppSettingsArduino::init(), there may be more info printed in the console

my guess is it didn't like the concatenated / in /tools and failed at finding otatool or esptool

[serifpersia]

Hmm now it works. I just had AVR boards selected, went back to esp32 platform and restarted Arduino IDE and now it shows up. It did complain that it wasnt able to find tool for generating partitions.bin. Window size would have to be changed so the labels for the partitions is not stretched. I would have to test functionality like csv spiffs uploading and merge. Last time I think ota didn't work before merge all functions should work. I would have to install linux on my laptop to test linux as well, good old debian. Do you know if linux commands will work on MacOS? Are they similar enough?

[serifpersia]

Could you explain how the buttons work at least the workflow how should the app be used now with these new changes. At the moment it seems like it compiles the sketch each time, merge seems not to work complains about
[debug] settings.bootloader.path is null
Missing file #0: null. Forgot to compile the sketch?
And yes I did try to compile before trying merge still same

[tobozo]

Hmm now it works.

great! let's call this bug a feature 🤣

ota is the only feature I haven't tested, otherwise uploading spiffs and merged bin both work fine on linux Arch, you can skip linux testing anyway I can take care of that

I'was surprised you had to restart the Arduino IDE to see the board changes though, I'll push a fix.

Window size would have to be changed so the labels for the partitions is not stretched

the UI class can now access the frame object so in theory it could be resized, I just hope the stretched background image will not affect the window size

Do you know if linux commands will work on MacOS? Are they similar enough?

most bash commands should work on macos, but zsh is the default so you'll need to type "bash" before trying anything

[debug] settings.bootloader.path is null

the first message is a false positive, subsequent similar messages however are true positives

Missing file #0: null. Forgot to compile the sketch?

thanks for finding that bug, path is evaluated before the build occurs, I'll push a fix too

And yes I did try to compile before trying merge still same, I assume it can't find the generic boot bin used for esp32 boards located somwehere in sdk.

nope it doesn't look there, it relies only on the build folder, Arduino IDE does the compilation, upload spiffs uses that folder to figure out the partition offset, and mergebin just uses the existing binares

And yes I did try to compile before trying merge still same

it's useless to do that manually now, it triggers a compilation every time just to be safe :D

Could you explain how the buttons work

unless you can't see the tooltip text when hovering the buttons, I don't see what more I can explain than "click on the button to trigger the action"

[serifpersia]

Yeah I wasn't sure about how the buttons function now that you added auto compile before their respective actions.I do not see the toolt tips for those FSPanel buttons but they are self explanatory. Do we really need two merge buttons the old system ask user if they want to flash as well was nice but I don't mind this but feels redundant having two buttons separating build & upload. For SPIFFS button it does auto attempt to upload maybe users would like to build only so having an option to ask build and upload as options for both button cases would be better.
As long as you can get the bootloader path fixed and minor ui adjustment and lastly a quick test of the features in the app this looks good for merge. How does the PlatformIO look like it doesn't have FSPanel so its a bit simpler you said you added some more smart code to it as well(CSV location import/export save) I installed it just to test but seems like its better platform for dev than Arduino IDE? But I do like Arduino IDE even if it locks you out of the features of esp32 sdk. For simple projects it works just fine, but for serious projects I assume PlatformIO is better. For me having a tool like this in Arduino IDE makes life easy if FS is being used which is quite often when you use esp32 for projects that involve webserver and sockets.
The progress bar was nice addition now that we wait for compile a cue for the user is always nice, I hate when apps are in limbo and you are not sure if they are doing something or they crashed or froze. 👍

Thanks I don't really have time to test on linux if you can do it on your end great! I'm not gonna explicitly say its supported on Mac just in case unless someone chimes in for Mac testing we will not talk about it

[tobozo]

I do not see the toolt tips for those FSPanel buttons

oh crap, I'm not sure how to fix that, maybe they're disabled by some inherited os rules?

[edit] actually I misunderstood, there are no tooltips for the fsPanel buttons

Elements with tooltip text:

• Save/Export/Load/Help/About
• CSV graph components
• "partitions.csv" title bar

Do we really need two merge buttons

the Arduino IDE has a different button for compile and upload so it just made sense to inherit from that

moreover, a serial connection isn't always available, and a merged bin isn't always for flashing: it can be created for redistribution (e.g. publish to M5Burner)

adding back the create spiffs button would be redundant, the merge bin button already takes care of creating spiffs.bin and the logs show the build path where it has been created

bugfixes

• bootloader path not found
• applet not reloading board config when reopened
• applet not erroring when a non-ESP32 board is selected in the arduino ide boards menu
• applet size for standalone mode
• hscrollbar appearing in the logs panel

[serifpersia]

Okay tested merge works without issue I see the merge bin. I found out that if I add bottom alignment to the labels it makes it so they are always near the partitions and not in center. But the enable label is not part of the grid so it won't stay at the bottom but if we just make it empty it looks fine. Do you think we need to have label saying enable? If you want try to come up with the fix for it but I'm fine if it looks like this
line added after 265 in UI class

label.setVerticalAlignment(SwingConstants.BOTTOM);

before(with labels to bottom)

after(with labels to bottom)

without bottom the labels are not positioned near the partitions unless the window size height is small enough to make them look like they are (around 930 px)

[tobozo]

column titles are centered vertically with the standard line height, the problem is elsewhere

I suspect the Tahoma font used for the "Partition" title renders differently depending on the OS the applet is running on.

let's see if the latest changes sorted out the stretching problem:

Standalone:

• reduced the frame height to 564 to prevent column titles to stretch vertically
• removed top margin from the cvs scrollpanel

[serifpersia]

I have implemented working UI, let me know if you have anything other to commit so I can add my changes and merge it(after test of all features).

[tobozo]

incoming font support to prevent discrepancies depending on the OS

chosen font: DejaVu (lesser restrictive licence for redistribution, comes with monotype and sans)

[edit]

• frame now resizable (but stretch to fullscreen is ugly)
• column titles have a fixed height (unaffected by stretch)
• DejaVu fonts (sans, monotype, monotype bold) are assigned to all applicable components
• Initial frame size modified again for both builds

[serifpersia]

It wasn't the font it was the box layout you used. I changed layout that doesn't cause such issues. I'm thinking going with flatlaf it works on standalone but I think when I started this project flatlaf wasn't working it made the app not visible under tools
It would look like this but I won't implement it if it doesn't work inside Arduino IDE which it won't by default. I don't know where to start with trying to debug that. I will probably just emulate the look of it or just have separate looks for standalone and arduino ide.

[tobozo]

The fonts aren't responsible for the layout problem but they are creating discrepancies, just compare your screenshots with mine, the title doesn't have the same grease and my tahoma version is more condensed.

Adding a specific font bundled with the applet will prevent that, just remember not to use local or default fonts if you add more components.

[tobozo]

btw there's a quick way to test the FSPanel in the standalone version, just add this.hasFSPanel = true; in the AppSettingsStandalone() main func before the init(); buttons won't work but the UI will be drawn and you'll have a quickview of how it should look in Arduino IDE

flatlaf

never head of that but anything sounds better than GridBagLayout :-)

once the interface will have matured, it will be ready to receive l10n support

there won't be further commits for this iteration, I won't have time to code or test anything for a few weeks as I'll be busy with more urgent actitivies

[serifpersia]

I added minor ui fix changed the layout, looked at espota tool docs seems like it only can upload spiffs with -s -f (-s indicates spiffs -f indicates file) if -s is not used it will upload firmware ie app image, merged image can't be uploaded via ota port since we can't overwrite bootloader or any other non app or spiffs partition, so a warning is included to use serial ports instead.
I don't see any issues with this pr anymore its in acceptable state.
Thanks for contributing @tobozo

[tobozo]

tested on a mac by a friend and apparently working good

[serifpersia]

tested on a mac by a friend and apparently working good

Nicee!