Proposal for Addressing ION's Bottleneck Due to Bitcoin Core's Block Time

[vlgsteve]

Hi everyone,

Problem Statement

While experimenting with ION, I noticed a significant bottleneck caused by Bitcoin Core's block time for transaction processing. The current mainchain block time ranges from 40 minutes to 1 hour and 30 minutes. This is a major issue for use-cases that require low-latency operations.

Proposed Solution: Temporary Verification Flagging

To address this, I propose introducing a mechanism called "Temporary Verification Flagging."

How It Works

1. Garbage Collection Queue: Create a garbage collection queue populated with DID operation hashes at the time of the request.
2. Batching and Anchoring: Operations would still be batched and anchored on the mainchain as usual.
3. Temporary Flagging: Until the operations are actually anchored, they would be flagged as 'temporary.'
4. Timed Checks: The garbage collection queue will run timed checks (max 2 hours) to account for block time and propagation delay.
5. Flag Removal: If an operation is confirmed to be anchored and propagated on the network, its 'temporary' flag will be removed. Otherwise, the flag will be retained, and/or any further operations will be rejected.

Benefits

• This approach allows for more immediate verification without waiting for the mainchain to catch up.
• It provides a fail-safe mechanism to ensure that only verified operations are finalised.

Potential Implementation Areas

To make this proposal actionable, I've considered two primary areas where this could be implemented:

ION Node Level
Best for: Decentralised nature, quick local decisions.
Challenges: Nodes might have inconsistent states; requires updates to all nodes.

Client SDK/Library
Best for: Quick implementation, less impact on existing network.
Challenges: Requires updates to all client applications; may not be suitable for all use-cases.

Challenges of Implementing Temporary Verification Flagging

While the proposed solution aims to address the latency issues in ION due to Bitcoin Core's block time, there are several challenges to consider:

1. Data Consistency: With multiple nodes flagging operations as 'temporary,' there's a risk of data inconsistency across the network.

2. Security Risks: The 'temporary' flag could potentially be exploited for malicious activities, such as double-spending or replay attacks.

3. Resource Utilisation: The garbage collection queue and timed checks could consume additional computational and memory resources, affecting the node's performance.

4. Network Propagation: The time it takes for the 'temporary' flags to propagate across the network could introduce additional latency.

5. User Experience: Users might find the 'temporary' status confusing or unsettling, affecting adoption and trust.

6. Backward Compatibility: Existing applications and services built on ION may require updates to handle the new 'temporary' flag, leading to potential compatibility issues.

7. Governance and Standardisation: Implementing this feature would require community consensus and possibly changes to the ION protocol, which could be a lengthy process.

8. Rollback Mechanisms: There needs to be a robust system to handle cases where a 'temporary' operation is eventually not confirmed on the mainchain.

9. Error Handling: Comprehensive error handling mechanisms must be in place to deal with various edge cases that could arise from this new feature.

I welcome any thoughts on how these challenges could be mitigated or if there are additional challenges to consider.

I'd love to hear your thoughts on this proposal. Is this something we can consider implementing?

CC: @csuwildcat

Best regards,
Steve