Analyze the ops of style transfer model

[miaobin]

https://github.com/acerwebai/VangoghCrazyWorld

The TensorFlow Lite model:

Tflite ops	polyfill	tfjs	webnn	contents	workaround
Transpose	√	tf.transpose	builder.transpose	-	-
MirrorPad	x	x	x	Pads a input with mirrored values according to the paddings you specify.	tf.pad
Conv2D	√	tf.conv2d	builder.conv2d	-	-
Mean	x	tf.mean	builder.reduceMean	Computes the mean of elements across dimensions of a tensor.	-
Sub	x	tf.sub	builder.sub	Returns x x y elementxwise.	-
SquaredDifference	x	tf.squaredDifference	x	Returns (x x y)(x x y) elementxwise.	-
Add	√	tf.add	builder.add	-	-
Relu	√	tf.relu	builder.relu	-	-
Pow	x	tf.pow	x	Computes the power of one value to another.	-
Div	x	tf.div	builder.div	Returns x / y elementxwise.	-
Mul	x	tf.mul	builder.mul	Returns x * y elementxwise.	-
TransposeConv	x	tf.conv2dTranspose	x	Computes the gradients of convolution with respect to the input.	-
Tanh	√	tf.tanh	builder.tanh	-	-

The ONNX model:

ONNX ops	polyfill	tfjs	webnn	contents	workaround
Transpose	√	tf.transpose	builder.transpose	-	-
Pad	√	tf.pad	x	tfjs only support the CONSTANT mode	-
Conv	√	tf.conv2d	builder.conv2d	-	-
ReduceMean	x	tf.mean	builder.reduceMean	Computes the mean of the input tensor's element along the provided axes.	-
Sub	x	tf.sub	builder.sub	Performs elementxwise binary subtraction.	-
Mul	x	tf.mul	builder.mul	Performs elementxwise binary multiplication	-
Add	√	tf.add	builder.add	-	-
Pow	x	tf.pow	x	f(x) = x^exponent	-
Div	x	tf.div	builder.div	Performs elementxwise binary division.	-
Relu	√	tf.relu	builder.relu	-	-
ConvTranspose	x	tf.conv2dTranspose	x	Computes the transposed convolution of an image, also known as a deconvolution.	-
Tanh	√	tf.tanh	builder.tanh	-	-

[huningxin]

The tables are very helpful. Thanks @miaobin . @fujunwei @mingmingtasd @lisa0314 , we need to investigate the platform API support including DirectML, Android NNAPI and OpenVINO + DNNL (Lisha). I'll take care of webnn spec gap analysis. Thanks all!

[fujunwei]

Add DirectML column.

ONNX ops	polyfill	tfjs	DirectML	contents	workaround
Transpose	√	√	DML_OPERATOR_ELEMENT_WISE_IDENTITY	-	-
Pad	√	√	DML_OPERATOR_PADDING	-	-
Conv	√	√	Supported	-	-
ReduceMean	x	√	DML_REDUCE_FUNCTION_AVERAGE	Computes the mean of the input tensor's element along the provided axes.	-
Sub	x	√	DML_ELEMENT_WISE_SUBTRACT_OPERATOR_DESC	Performs elementxwise binary subtraction.	-
Mul	x	√	Supported	Performs elementxwise binary multiplication	-
Add	√	√	Supported	-	-
Pow	x	√	DML_OPERATOR_ELEMENT_WISE_POW	f(x) = x^exponent	-
Div	x	√	DML_ELEMENT_WISE_DIVIDE	Performs elementxwise binary division.	-
Relu	√	√	Supported	-	-
ConvTranspose	x	√	DML_CONVOLUTION_DIRECTION_BACKWARD	Computes the transposed convolution of an image, also known as a deconvolution.	-
Tanh	√	√	DML_ACTIVATION_TANH_OPERATOR_DESC	-	-

[huningxin]

la_muse_onnxv7.onnx

ONNX ops	webnn	remarks
Transpose	transpose	-
Conv	conv2d	-
ReduceMean	reduceMean	Computes the mean of the input tensor's element along the provided axes.
Sub	sub	Performs elementxwise binary subtraction.
Mul	mul	Performs elementxwise binary multiplication
Add	add	-
Pow(x, 0.5)	sqrt	f(x) = x^exponent, if exponent = 0.5, it can be supported by sqrt(x)
Div	div	Performs elementxwise binary division.
Relu	relu	-
ConvTranspose	x	Computes the transposed convolution of an image, also known as a deconvolution.
Tanh	tanh	-

[mingmingtasd]

ONNX ops	polyfill	Android NNAPI	remarks
Transpose	√	ANEURALNETWORKS_TRANSPOSE	-
Pad	√	ANEURALNETWORKS_PAD	-
Conv	√	ANEURALNETWORKS_CONV_2D	-
ReduceMean	x	ANEURALNETWORKS_MEAN	Computes the mean of the input tensor's element along the provided axes.
Sub	x	ANEURALNETWORKS_SUB	Performs elementxwise binary subtraction.
Mul	x	ANEURALNETWORKS_MUL	Performs elementxwise binary multiplication
Add	√	ANEURALNETWORKS_ADD	-
Pow	x	ANEURALNETWORKS_POW	f(x) = x^exponent
Div	x	ANEURALNETWORKS_DIV	Performs elementxwise binary division.
Relu	√	ANEURALNETWORKS_RELU	-
ConvTranspose	x	ANEURALNETWORKS_TRANSPOSE_CONV_2D	Computes the transposed convolution of an image, also known as a deconvolution.
Tanh	√	ANEURALNETWORKS_TANH	-

ONNX ops	polyfill	OpenVINO	remarks
Transpose	√	Transpose-1	-
Pad	√	Pad-1	-
Conv	√	Convolution-1	-
ReduceMean	x	ReduceMean-1	Computes the mean of the input tensor's element along the provided axes.
Sub	x	Subtract-1	Performs elementxwise binary subtraction.
Mul	x	Multiply-1	Performs elementxwise binary multiplication
Add	√	Add-1	-
Pow	x	Power-1	f(x) = x^exponent
Div	x	Divide-1	Performs elementxwise binary division.
Relu	√	ReLU-1	-
ConvTranspose	x	ConvolutionBackpropData-1	Computes the transposed convolution of an image, also known as a deconvolution.
Tanh	√	Tanh-1	-

[mingmingtasd]

Ops tables for Android NNAPI and OpenVINO have been added as above , please review, thanks! @huningxin

[lisa0314]

ONNX ops	polyfill	DNNL	remarks
Transpose	√	x	-
Pad	√	x	-
Conv	√	Supported	-
ReduceMean	x	x	Computes the mean of the input tensor's element along the provided axes.
Sub	x	x	Performs elementxwise binary subtraction.
Mul	x	dnnl_binary_mul	Performs elementxwise binary multiplication
Add	√	Supported	-
Pow	x	dnnl_eltwise_pow	f(x) = x^exponent
Div	x	x	Performs elementxwise binary division.
Relu	√	Supported	-
ConvTranspose	x	Deconvolution	Computes the transposed convolution of an image, also known as a deconvolution.
Tanh	√	dnnl_eltwise_tanh	-

[huningxin]

@miaobin , please remove the pb table and update the ONNX table according to https://github.com/acerwebai/VangoghCrazyWorld/blob/master/tf2onnx_models/la_muse_onnxv7.onnx.

To me, the onnx model doesn't have Pad op. It seems Pad op are folded into Conv op.

[huningxin]

And I think the Pow(x, 0.5) can be covered by sqrt in webnn spec.

[huningxin]

Ops tables for Android NNAPI

The ANEURALNETWORKS_MEAN op supports ReduceMean, does it?

[mingmingtasd]

Ops tables for Android NNAPI

The ANEURALNETWORKS_MEAN op supports ReduceMean, does it?

Yes, I added it, thanks! @huningxin

[miaobin]

@huningxin @ibelem I have updated the table and added the comparison with Web Neural Network API.

[huningxin]

Thanks @miaobin , it looks like webnn needs to add convTranspose, pad and pow operations for this use case.

[miaobin]

Thanks @miaobin , it looks like webnn needs to add convTranspose, pad and pow operations for this use case.

Yes, three are required in ONNX format, and one more (squaredDifference) is required in TFLite format.