eth_estimateGas underestimates tx with function that catches reverts

[MartinquaXD]

System information

Erigon version: v2.60.6
Also tested it later with v3-beta1 which had the same issue

OS & Version: Linux

Commit hash:

Erigon Command (with flags/config):
Copied from kubectl describe

  erigon:
    Container ID:  containerd://8711b62e06349d25a7471fafc82b2774a132bf6f01e83110ef58459f62c3fecf
    Image:         thorax/erigon:v2.60.6
    Image ID:      docker.io/thorax/erigon@sha256:2602265a1c0f3d3a41862cb07d13525d99a12200062d6710c922066ddccfafc2
    Ports:         8545/TCP, 8546/TCP, 9090/TCP, 30303/TCP, 30303/UDP, 8551/TCP
    Host Ports:    0/TCP, 0/TCP, 0/TCP, 0/TCP, 0/UDP, 0/TCP
    Args:
      --datadir=/data/gnosis
      --chain=gnosis
      --authrpc.addr=0.0.0.0
      --authrpc.port=8551
      --authrpc.jwtsecret=/jwt/jwt.hex
      --authrpc.vhosts=*
      --http
      --http.addr=0.0.0.0
      --http.port=8545
      --http.vhosts=*
      --http.corsdomain=*
      --http.api=eth,debug,net,trace,web3,erigon,txpool
      --ws
      --metrics
      --metrics.addr=0.0.0.0
      --metrics.port=9090
      --private.api.addr=127.0.0.1:9001
      --nat=any
      --torrent.download.rate=1000mb

Consensus Layer: lighthouse

Consensus Layer Command (with flags/config):
copied from kubectl describe

  lighthouse:
    Container ID:  containerd://d2f10276dded520f4a45b8dd0e373da5501d7b88319ea86d28bd0373c267aba9
    Image:         sigp/lighthouse:v5.3.0
    Image ID:      docker.io/sigp/lighthouse@sha256:f9b97e018d549a7fa8c742a6cb5b6c7d4780d1cdce707d823dd378aaf3dd34c5
    Ports:         5052/TCP, 9091/TCP, 9000/TCP, 9000/UDP
    Host Ports:    0/TCP, 0/TCP, 0/TCP, 0/UDP
    Args:
      lighthouse
      bn
      --network
      gnosis
      --datadir
      /data
      --execution-jwt
      /jwt/jwt.hex
      --execution-endpoint
      http://127.0.0.1:8551
      --metrics
      --metrics-address
      0.0.0.0
      --metrics-port
      9091
      --checkpoint-sync-url
      https://checkpoint.gnosischain.com/
      --http
      --http-address
      0.0.0.

Chain/Network: gnosis chain

Expected behaviour

eth_estimateGas computes the maximum amount of gas a transaction can consume.

Actual behaviour

eth_estimateGas under estimates the gas A LOT.
Here is some context in case it's helpful. Users on CoW Protocol can specify post-hooks which get executed after their order. These user provided calls get executed via a trampoline contract to have them run in an unprivileged context. Additionally the contract enforces a maximum amount of gas that may be used by the call.
Before submitting a CoW Protocol transaction we estimate the gas for it and set this (+ a buffer) as the gas limit. Since eth_estimateGas underestimated the gas usage a lot the post hook reverts because it runs out of gas.

I suspect the issue is that because the transaction internally catches a revert caused by too little gas the minimum amount of gas a transaction consumes without reverting completely is relatively low. However, if we allow the transaction to consume more gas it would because the call that catches the revert gets to do more overall.
So instead of returning the minimum amount a tx can consume without reverting completely eth_estimateGas should probably return the maximum amount of gas the tx can actually consume to ensure that all the logic that could run as part of the tx actually gets to run.

Steps to reproduce the behaviour

To reproduce the behavior I took one of the transactions where the underlying trade was executed successfully but the post hook reverted. From this I stripped out the underlying trade such that the transaction would only execute the post hook.
On tenderly this produces a gas estimate of ~4.8M while eth_estimateGas only reports ~181K:

curl -X POST --header "Content-Type: application/json" --data '{ "jsonrpc": "2.0", "id": 1, "method": "eth_estimateGas", "params": [ { "from": "0x056545021b39790efb0a074827e7fddcb751a179", "to": "0x9008D19f58AAbD9eD0D60971565AA8510560ab41", "data": "0x13d79a0b000000000000000000000000000000000000000000000000000000000000008000000000000000000000000000000000000000000000000000000000000000a000000000000000000000000000000000000000000000000000000000000000c000000000000000000000000000000000000000000000000000000000000000e00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000060000000000000000000000000000000000000000000000000000000000000008000000000000000000000000000000000000000000000000000000000000000a0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000002000000000000000000000000001dcb88678aedd0c4cc9552b20f4718550250574000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000600000000000000000000000000000000000000000000000000000000000000104760f2a0b000000000000000000000000000000000000000000000000000000000000002000000000000000000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000020000000000000000000000000a73e2339797af9446dc0a16594d44f100a03eeb1000000000000000000000000000000000000000000000000000000000000006000000000000000000000000000000000000000000000000000000000005b8d80000000000000000000000000000000000000000000000000000000000000000413e8f89f0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000" } ] }' http://localhost:8545/

Interestingly if I only estimate the hook without the settlement and trampoline contract around it tenderly and erigon agree on the estimate:

curl -X POST --header "Content-Type: application/json" --data '{ "jsonrpc": "2.0", "id": 1, "method": "eth_estimateGas", "params": [ { "from": "0x056545021b39790efb0a074827e7fddcb751a179", "to": "0xa73e2339797af9446dc0a16594d44f100a03eeb1", "data": "0x13e8f89f" } ] }' http://localhost:8545/

[AskAlexSharov]

eth_estimateGas - does support historical blocks
let's use precise blockNum (to reduce chain-tip variability).
let's use block: 38300000 (0x2486960)

curl -X POST --header "Content-Type: application/json" --data '{ "jsonrpc": "2.0", "id": 1, "method": "eth_estimateGas", "params": [ { "from": "0x056545021b39790efb0a074827e7fddcb751a179", "to": "0x9008D19f58AAbD9eD0D60971565AA8510560ab41", "data": "0x13d79a0b000000000000000000000000000000000000000000000000000000000000008000000000000000000000000000000000000000000000000000000000000000a000000000000000000000000000000000000000000000000000000000000000c000000000000000000000000000000000000000000000000000000000000000e00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000060000000000000000000000000000000000000000000000000000000000000008000000000000000000000000000000000000000000000000000000000000000a0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000002000000000000000000000000001dcb88678aedd0c4cc9552b20f4718550250574000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000600000000000000000000000000000000000000000000000000000000000000104760f2a0b000000000000000000000000000000000000000000000000000000000000002000000000000000000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000020000000000000000000000000a73e2339797af9446dc0a16594d44f100a03eeb1000000000000000000000000000000000000000000000000000000000000006000000000000000000000000000000000000000000000000000000000005b8d80000000000000000000000000000000000000000000000000000000000000000413e8f89f0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000" } , "0x2486960"] }' http://localhost:8546/

{"jsonrpc":"2.0","id":1,"result":"0xeb51"}

So, it's 60241 (60K).

Oke, somebody will take a look.

[MartinquaXD]

eth_estimateGas - does support historical blocks
let's use precise blockNum (to reduce chain-tip variability).

From my previous tests around this it seems like eth_estimateGas only uses the passed in block for fetching the gas limit of that block to cap the gas limit when no limit was provided in the request parameters. But AFAICT it does not compute the gas estimate based on the chain state of that block.
So while it reduces some variability the main source of variability would not be addressed.

[yperbasis]

Phillippe requested to prioritize it because it's a considerable blocker for CoW.

[Giulio2002]

76904e7#diff-872622599a17b0ea9ec04867ff56e5daad4d3636fc4d855857141ecf366e1ae0

@AskAlexSharov related?

[Giulio2002]

also can we get an example rpc call? so that we can reproduce it? also I think it is fixed in main. would you mind trying the main branch?

[Giulio2002]

due to lack of repsonsiveness. I will move this milestone to the next beta release

[MartinquaXD]

Sorry for the late response. Didn't notice the notification in my inbox.

also can we get an example rpc call? so that we can reproduce it?

If erigon can now simulate on historic blocks this request should be able to reproduce the bug (if it's still there):

curl -X POST --header "Content-Type: application/json" --data '{ "jsonrpc": "2.0", "id": 1, "method": "eth_estimateGas", "params": [ { "from": "0x056545021b39790efb0a074827e7fddcb751a179", "to": "0x9008D19f58AAbD9eD0D60971565AA8510560ab41", "data": "0x13d79a0b000000000000000000000000000000000000000000000000000000000000008000000000000000000000000000000000000000000000000000000000000000a000000000000000000000000000000000000000000000000000000000000000c000000000000000000000000000000000000000000000000000000000000000e00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000060000000000000000000000000000000000000000000000000000000000000008000000000000000000000000000000000000000000000000000000000000000a0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000002000000000000000000000000001dcb88678aedd0c4cc9552b20f4718550250574000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000600000000000000000000000000000000000000000000000000000000000000104760f2a0b000000000000000000000000000000000000000000000000000000000000002000000000000000000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000020000000000000000000000000a73e2339797af9446dc0a16594d44f100a03eeb1000000000000000000000000000000000000000000000000000000000000006000000000000000000000000000000000000000000000000000000000005b8d80000000000000000000000000000000000000000000000000000000000000000413e8f89f0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000" } , "0x248bd51"] }' http://localhost:8546/

It's the same call as the tenderly simulation linked in the description and also tied to the same block. So if erigon now correctly estimates the gas costs this should also return ~4.8M.

also I think it is fixed in main. would you mind trying the main branch?

I don't have access to a synced main erigon node at the moment. But I can ask if our devops team can spin one up for testing..

[Giulio2002]

@somnathb1 attack

[somnathb1]

Probably need an archive node to simulate this, syncing a node. Still will be on hold for a day or two

[somnathb1]

@MartinquaXD I do see that when I send the request with 8.5M gas, it consumes around 4.8M gas. But it also doesn't throw any issues when passed around 180K gas.
I am not familiar with the internals of thos contract. Could you clarify

• Actual gas used on a similar txn that i can see on the chain
• The trace of execution of the other tool you're using to estimate gas

At this point, i have a small fix for the API, but i would need to understand more about your requirements, and will take a call on whether this API should be fine tuned for that

[Giulio2002]

@somnathb1 I would just compare with geth and copy the mechanism