Generalize to other languages/frameworks

[ceztko]

Hello! I have been using node-api-dotnet for quite some time, and I also made some suggestions that were nicely integrated. I have some relevant experience of marshalling native code in different languages/frameworks using .NET/C# as a common wrapper template, in particular Java/JNI (JDK and Android), ObjectiveC in Apple platforms and my custom Node.js solution. I also made a proposal to have a similar technology endorsed by Microsoft but understandably it didn't gather much attention, as I admit myself being too much a low level framework to work with, and the cost of packaging a module/library for any target language/platform is also high and not included in my original work. I have seen your approach with node-api-dotnet, which is more self contained and don't require a featureful C# transpiler, and I thought it could be generalized to other languages/frameworks using the Native AOT generation. Of course in such a general software solution the JS prefixed attributes would look quite out of context, and possibly the current feature set can't be fully ported to all languages/frameworks, but certainly a common feature set can be found (see some CodeBinder codegen examples). The question is: are you internally planning at Microsoft to extend the node-api-dotnet approach to a general tool that support more langauges/framework?

[ceztko]

@jasongin sorry for pinging you directly, maybe you missed/forgot this discussion. In short I would like to understand if you have a long term plan for generic marshalling in languages/frameworks other than NodeJS. My custom technology served me and is still serving me well but reached the limits of its evolution, and I'm trying to understand if I will be able to move to something else in a 2-5 years time frame.

[jasongin]

The long-term plan is to support multiple JavaScript engines/frameworks, not only Node.js. See the example project using the Hermes JavaScript engine, and a discussion about Deno support. This is possible because "Node API" is actually not specific to Node.js but is adopted by other JS engines as a consistent way to support native interop.

But there are no plans for this project to ever directly support interop with languages other than JavaScript (and variants like TypeScript).

There may be some parts of this project that could be re-usable for interop with other languages, if they were factored out. I can think of two parts which are both innovative (meaning I haven't seen the techniques anywhere else) and potentially useful interop with other languages:

1. The .NET runtime hosting code that is itself implemented in C# with Native AOT. Every other project that I've seen uses C++ for this purpose, which can work just as well but brings associated complexity in terms of toolchain requirements and build process. The use of C# for this part makes it easier to work with and allows sharing code between native and managed parts of the solution.
2. The expression-based marshaller that supports both build-time source-generation and runtime marshalling. I haven't seen another source-generator that works like this (and it wasn't simple to develop) and it works out well when different .NET applications can have conflicting requirements for AOT compilation versus dynamic invocation.

So I might be open to refactoring those components to make them reusable, but it hasn't been a priority so far.

[ceztko]

But there are no plans for this project to ever directly support interop with languages

It's clear that such work would not fit in a project named node-api-dotnet. The question was really if you thought in your team about enlarging the scope of attribute based marshaling to other platforms or if your heard about the topic from the .NET runtime team, as I'm sure they are interested in marshaling in other languages/platforms, just look at the increasing platform support for Native AOT.

The .NET runtime hosting code that is itself implemented in C# with Native AOT

That's interesting but it has no use in the mobile world, where the .NET runtime can be not allowed at all (iOS). Please note that all my comments are driven by what I see as the holy grail of software re-usability: write a class library once and consume it everywhere. From this perspective I really see a big potential about extending your approach to such platforms, and while you clarified such work is not exactly being pursued at moment (at least in your team), I still hope to see it in the future. If there would be a MS job opening for such task, probably I would apply right away to it.

[jasongin]

The question was really if you thought in your team about enlarging the scope of attribute based marshaling to other platforms or if your heard about the topic from the .NET runtime team, as I'm sure they are interested in marshaling in other languages/platforms, just look at the increasing platform support for Native AOT.

I think they made their stance clear in the response to the proposal you linked. And when I presented this project to the .NET team, I couldn't get them very interested because their focus on JavaScript interop is limited in scope to Blazor / WebASM scenarios for now, despite the reality of significant overlap between .NET and JS developers across broader scenarios.

If there would be a MS job opening for such task, probably I would apply right away to it.

So would I. :) But with the way the company works, it is very difficult to make a business case to fund a team to build bridges to other non-Microsoft platforms unless it's necessary to fill a gap in the Microsoft platform (as is the case of Blazor) or it has very broad applicability and contributes to revenue (such as tools/SDKs for running other platforms on Azure). I have firsthand experience with that, as I previously worked on the team that enabled Android apps to run on Windows, that was cancelled primarily for business reasons. And even this node-api-dotnet project is not really funded, it is merely a side project that Vladimir and I work on in our spare time (hence the slow progress). It is only slightly sponsored by Vladimir's team so that we can have some resources like official release pipelines.

[ceztko]

So would I. :) But with the way the company works, it is very difficult to make a business case to fund a team to build bridges to other non-Microsoft platforms unless it's necessary to fill a gap in the Microsoft platform (as is the case of Blazor) or it has very broad applicability and contributes to revenue (such as tools/SDKs for running other platforms on Azure)

Thank you for the insights! Though, at first glance, this seems a bit in contradiction with the amount of resources that are being spent in supporting Native AOT libraries in mobile platforms (iOS/Android) which is certainly relevant, and don't have a clear link to Azure.