Custom excite method for python interface

[tjdphd]

Am I correct in my impression that the custom excitation option available in the matlab interface is not available in the python interface? If so, is there any hope for one in the future? We're working on a project that would benefit strongly from being able to use python interface instead of matlab, but we also need to be able to have custom excitations.

[LubomirJagos]

Also today I was looking for python interface for coaxial and other ports and cannot find them, so there is more what is not there :D

I tried generate from octave .xml file with custom excitement and is defined inside so you can always write your script using gaussian or other excitement definition and then at the end of your script open .xml file and modified it using python and some xml lib, here is example how custom excitement looks like:

  <FDTD NumberOfTimesteps="2000000" endCriteria="0.0001" f_max="2400000000">
    <BoundaryCond xmin="PML_8" xmax="PML_8" ymin="PML_8" ymax="PML_8" zmin="PML_8" zmax="PML_8">
    </BoundaryCond>
    <Excitation Type="10" f0="2400000000" Function="4.2*sin(2*3.14*2400000000.0*t)">
    </Excitation>
  </FDTD>

I bet you can also using python access CSX underlying properties and use python over CSX to set custom excitation as python interface is generated from C++ files, look here CSXCAD.CSProperties.CSPropExcitation
https://docs.openems.de/python/CSXCAD/CSProperties/CSPropExcitation.html#cspropexcitation

[tjdphd]

Thanks! I'll have to study this though. I'm pretty new to trying to use openEMS at all.

[thliebig]

May I ask why you need a custom excitation. I always wonder why anybody would want that since the Gaussian excitation is always the most efficient...

[thliebig]

CSPropExcitation can not set a custom excitation!!
And NO, the python interface does not generate *.xml files like Matlab does, it generates the data using the C++ interface directly and starts openEMS directly. You can export the CSXCAD structurte into an *.xml file, but not the openEMS settings which is were a custom excitation would be defined.

[thliebig]

I can only repeat, the "SetCustomExcite" function is currently not available in python!

[tjdphd]

@thliebig thank you for the clarification.

[LubomirJagos42]

:D ok, but still like workaround if the xml structure is generated openEMS settings could be added as xml tags? Like it's just few tags and to handle xml file in python could not be problem

[thliebig]

I do not really see this as an option. Because the python interface cannot read/load the modified xml (with the openEMS settings). This means you would also need to start openEMS manually. Even the post-processing would not work nicely anymore. This would completely work-around the python interface. It is possible, but not really very nice... Fixing/adding the missing functions is the way to go...

[tjdphd]

@thliebig So I've been studying the project and the python interface as it currently exists, and I've tracked down the declaration of the C++ SetUpCustomExcite method of the Excitation class in the file excitation.h. At the moment I'm planning on adding something to the python interface myself that will allow me to use this with the rest of the interface, because it's absolutely crucial that my project have this functionality however rarely it may be used elsewhere.

Related to this I have some questions that I'd appreciate your help with if you don't mind:

Assuming I manage to figure out how to make SetUpCustomExcite available via the python interface how much rebuilding is involved? Is it just a matter of running python setup.py on the python interface itself or will I need to rebuild the entire project? (I've never used cython before so I'm having to learn all that as I go).

In case it's necessary can you please describe how to manually build the openEMS project under Windows 10? You make a point of not documenting a manual build under windows in your documentation, and I've spent a long time trying to figure out how to do that on my own in case I need to, but have not been successful.

thanks for considering this request.

[Sarajevo67]

That sounds like you are going to fade out Matlab/Octave interface and make openEMS (which is currently almost standalone) - a Python extension?

[thliebig]

No I have no intentions to drop or fade out any interface or make openEMS less standalone. I just want to improve the CSXCAD and openEMS library API's to make it even more flexible in how these two (independent) libraries can be used...

[Sarajevo67]

Great to hear so. I hope you will find workaround for that MSVC compiler dependency, because even Python extensions (i.e. NumPy), does not require MSVC.

BTW, is it possible to avoid Qt/vtk dependencies, for openEMS build?

[thliebig]

Great to hear so. I hope you will find workaround for that MSVC compiler dependency, because even Python extensions (i.e. NumPy), does not require MSVC.

I'm not sure what you mean by that, I'm pretty sure numpy is build on Windows with MSVC but like CSXCAD and openEMS once it is build and have e.g. a wheel you do not need anything. That said, the "normal" user does never know about the pain to setup the build-chain on Windows...

BTW, is it possible to avoid Qt/vtk dependencies, for openEMS build?

vtk is required by openEMS, but Qt is not. That is only necessary for the GUI parts (QCSXCAD etc.)

[Sarajevo67]

@thliebig
I'm not sure what you mean by that, I'm pretty sure numpy is build on Windows with MSVC
My fault, I blundered what pip3 actually did.

[thliebig]

I don't feel at liberty to describe any specifics of the project

That sounds very ominous, please make sure that your team checks that the license you use is compatible with openEMS as it is GPL and it requires every software using it to be open source too... There might be exceptions to this depending on how you use openEMS, but it was after all the intention behind my license choice many many years ago...

[tjdphd]

We will of course honor all license requirements. In any case, I will convey your concerns about the license to the rest of my team, and if any licensing questions arise I'll pass them along to you so we can be sure we're doing things right.

[tjdphd]

I've relayed your licensing concerns to my team, and I've got confirmation from them that we will not use openEMS in a way that violates the GPL license.

[tjdphd]

@thliebig By the way, while I'm waiting for someone on my end to help me with the Windows build (yes, we are bound and determined), I've been working on building under wsl (taking your advice to make my additions to the interface under linux and submit a pull request) and I ran into a little wrinkle while following the instructions here: https://docs.openems.de/install/manual.html . In these instructions for the openEMS build it says to use:

cmake ../ -DCMAKE_INSTALL_PREFIX=/home/user/openEMS -DFPARSER_ROOT_DIR=/tmp/opt -DCSXCAD_ROOT_DIR=/tmp/opt

If I do this the build fails at the make step because cmake can't find FPARSER_ROOT_DIR. So, I changed /tmp/opt to /home/tdennis/openEMS and that fixed it.

Point being, I'm wondering that if instruction is either outdated or a typo. Maybe in an older version you (or whoever your documentarian is) were setting CMAKE_INSTALL_PREFIX to /tmp/opt instead of /home/user/openEMS . If so, I just thought you might want to update it to prevent others from running into this error. Then again maybe this is wsl specific? I seem to recall you saying you hadn't tried building under wsl. Anyway best wishes.

[tjdphd]

@thliebig

Greetings Thorsten,

So, I've spent the last little while learning Cython and I took your suggestion to add the SetCustomExcite() interface method under Linux first since it's easier to build manually there.

So far, I've added the necessary code to include the SetCustomExcite() method, and I've tested it by using a waveform string (called waveform) that we've been using elsewhere via the Matlab interface, and replacing the line:

FDTD.SetGaussExcite(0.5*(f_start+f_stop), 0.5*(f_stop-f_start))

with

FDTD.SetCustomExcite(waveform, 0.5*(f_start+f_stop), 0.5*(f_stop-f_start))

and it runs just fine and produces some interesting output that is distinctly different but still sensible looking.

You said that after I've done some testing, I could put in a pull request for you to review. I'm at the point now where I could construct more of my own tests to make sure what I'm doing is producing the expected outcome - and of course, that's what I will do barring any input from you, but I'd prefer to know what your own standards are for what would constitute an acceptably rigorous test. I want to be sure that what I've produced is up to your standards. I would also need to know how to go about making the pull request once I'm ready. I've never done that on github before.

Meanwhile we're working on a way to automate as much as possible the process of building openEMS under windows and our PI here has expressed a desire to share our build system with you once we have it working, and I will be very happy to do just that for you when we have it up and running.

Please let me know what you think.

Thanks, Tim