Custom decoding for sealed traits with discriminator

[petervecera-jr]

Hi I would like to ask you for your opinion on problem I've encountered. I'll try to be as brief as possible.

sealed trait MyTrait
case class Foo(x: Int) extends MyTrait
case class Bar(y: String) extends MyTrait

case class ApiDTO(discriminator: String, payload: MyTrait)

I want to build a tooling that will be able to create a JsonValueCodec[ApiDTO]

I've come up with this solution which I can't make work because of the closing bracket at the end. (T <=> MyTrait, ? <=> {Foo, Bar, ...})

def makeJsonValueCodec[T](
    discriminatorKey: String,
    payloadKey: String,
    discToCodec: Map[String, (JsonValueCodec[? <: T], ClassTag[? <: T])]
  ): JsonValueCodec[T] =
    new JsonValueCodec[T]:
      val codecToDisc: Map[(JsonValueCodec[? <: T], ClassTag[? <: T]), String] = discToCodec.map(_.swap)
      assert(discToCodec.size == codecToDisc.size, "Duplicate codec or discriminator")

      override def decodeValue(in: JsonReader, default: T): T =

        assert(in.isNextToken('{'), "Expected '{' at the beginning of the object")
        assert(in.readKeyAsString() == discriminatorKey, "Expected discriminator key")
        val disc = in.readString(null)
        val (codec, _) = discToCodec.getOrElse(disc, throw new IllegalArgumentException(s"Unknown discriminator: $disc"))
        assert(in.isNextToken(','), "Expected ',' after discriminator")
        assert(in.readKeyAsString() == payloadKey, "Expected payload key")
        codec.decodeValue(in, default.asInstanceOf) // This decode function fails

      override def encodeValue(x: T, out: JsonWriter): Unit =
        val ((codec, _), discriminatorValue) = codecToDisc
          .find {
            case ((_, tag), _) => tag.runtimeClass.isInstance(x)
          }
          .getOrElse(throw new IllegalArgumentException(s"Unknown type: $x"))

        out.writeObjectStart()
        out.writeKey(discriminatorKey)
        out.writeVal(discriminatorValue)
        out.writeKey(payloadKey)
        codec.encodeValue(x.asInstanceOf, out)
        out.writeObjectEnd()

      override def nullValue: T = null.asInstanceOf[T]

[petervecera-jr]

Decode function fails since it's decoding invalid JSON because it has some trailing } at the end.

[plokhotnyuk]

Why do you need discriminator in your ApiDTO?
Do you have an ability to modify your data structures?
Why do you trade off performant code generated by macros to some definitely slow one?
Could you please share more context about your challenge without thinking about particular solution that you shared above?

[petervecera-jr]

1. To know which codec should be used e.g. if there is a discriminator for type A use JsonCodec[A] etc..
2. Structures are arbitrary, I just receive JSON in this format and need to emit it in the same...
3. The API we want to use is using this format therefore we need to conform to it.
4. I need a JsonValueCodec for the case class with a generic payload determined by a discriminator.
   a. I have a payload of type T and when I serialize it I want the codec to add a discriminator with appropriate value to the JSON automatically
   b. When I receive a JSON from a server I want the codec to parse out the payload appropriately

[plokhotnyuk]

Have you tried to derive codecs with

JsonCodecMaker.make(CodecMakerConfig.withDiscriminatorFieldName(Some("discriminator")).withRequireDiscriminatorFirst(false))

and check if works for you?

[petervecera-jr]

I'll use an example I have that works:

sealed trait MyTrait {
  def payload: Payload
}

sealed trait Payload
case class Foo(x: BigDecimal) extends Payload
case object Bar extends Payload

object MyTrait:
  implicit class FooWrapper(val payload: Foo) extends MyTrait
  implicit class BarWrapper(val payload: Heartbeat.type) extends MyTrait

  given codec: JsonValueCodec[MyTrait] =
    JsonCodecMaker.make {
      CodecMakerConfig
        .withDiscriminatorFieldName(Some("discriminator"))
        .withAdtLeafClassNameMapper {
          case "MyTrait$.FooWrapper" => "Foo"
          case "MyTrait$.BarWrapper" => "Bar"
          case invalid => sys.error(s"Invalid request type: $invalid")
        }
    }

the problem is that I would like to build a tooling around it which I tried to show an example that's nearly working in the first post.

[plokhotnyuk]

Why do you need tooling?
Could you please share JSON samples and related data structures that need to be decoded from it?

[petervecera-jr]

I want to make a more high-level API that supports my use case, and my fellow programmers can just use that tooling which does the heavy lifting for them.

JSON structure
{
  "typeKey": "typeValue",
  "payloadKey": {}
}

examples

JSON example 1:
  {
    "type": "HEARTBEAT",
    "payload": {}
  }

JSON example 2:
  {
    "type": "DATA",
    "payload": {
      "age": 20,
      "name": "John",
      "surname": "Doe"
    }
  }

or there may be additional fields for example:

Format example:
{
  "typeKey": "typeValue",
  "payloadKey": {},
  "revision": "revisionValue"
}

examples

JSON example 3:
  {
    "type": "HEARTBEAT",
    "payload": {},
    "revision": 1
  }

JSON example 4:
  {
    "type": "DATA",
    "payload": {
      "age": 20,
      "name": "John",
      "surname": "Doe"
    },
    "revision": 10
  }

[plokhotnyuk]

Am I understand right that your JSON representation cannot be changed to something like:

{
  "DATA": {
    "age": 20,
    "name": "John",
    "surname": "Doe"
  },
  "revision": 10
}

or

{
  "payload": {
    "type": "DATA",
    "age": 20,
    "name": "John",
    "surname": "Doe"
  },
  "revision": 10
}

?

[petervecera-jr]

Yes, that's right since I'm not the one creating them.

[plokhotnyuk]

I'm strongly recommend switching to 1st (the most safe and performant) or 2nd option.

It will reduce runtime and maintenance costs, eliminate present and future challenges.

[petervecera-jr]

I agree with you, but I can't just go and rewrite all legacy codebase to the 1st variant, and since I want to support it I lack the tooling around the case I've described above.

Side note I'm using the 1st option in new applications.