Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: replicate the persistent cache task when delete host #3787

Merged
merged 1 commit into from
Jan 27, 2025

Conversation

gaius-qi
Copy link
Member

Description

This pull request includes changes to the persistent cache task management and scheduling logic, as well as updates to mock implementations and dependency versions. The most important changes focus on adding new methods for updating persistent cache tasks, improving the handling of peer states, and refining the scheduling logic.

Persistent Cache Task Management:

  • Added UpdatePersistentCacheTask method to V2 interface and its implementation in client_v2.go to update persistent cache task information. [1] [2]
  • Modified peer_manager.go to correctly set task joint-set TTL and handle peer deletion more robustly by using the task ID instead of host ID. [1] [2] [3] [4] [5]

Scheduling Logic:

  • Updated FindReplicatePersistentCacheHosts method to return both cached and non-cached replicate hosts, improving the scheduling process for persistent cache tasks. [1] [2] [3] [4] [5] [6] [7]

Mock Implementations:

  • Added mock methods for UpdatePersistentCacheTask in client_v2_mock.go and seed_peer_client_mock.go. [1] [2]
  • Added new mock methods for loading peer IDs by host and task in peer_manager_mock.go.

Dependency Updates:

  • Updated d7y.io/api/v2 dependency version from v2.1.18 to v2.1.21 in go.mod.

Peer State Handling:

  • Enhanced peer state handling in service_v2.go to set peer state to failed if various peer operations fail, ensuring more accurate state tracking.

Related Issue

Motivation and Context

Screenshots (if appropriate)

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation Update (if none of the other choices apply)

Checklist

  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have read the CONTRIBUTING document.
  • I have added tests to cover my changes.

Sorry, something went wrong.

@gaius-qi gaius-qi added the enhancement New feature or request label Jan 26, 2025
@gaius-qi gaius-qi added this to the v2.3.0 milestone Jan 26, 2025
@gaius-qi gaius-qi self-assigned this Jan 26, 2025
@gaius-qi gaius-qi requested a review from a team as a code owner January 26, 2025 11:41
Copy link

codecov bot commented Jan 26, 2025

Codecov Report

Attention: Patch coverage is 3.31950% with 233 lines in your changes missing coverage. Please review.

Project coverage is 34.19%. Comparing base (a1cb0a4) to head (25ab410).
Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
scheduler/service/service_v2.go 5.88% 128 Missing ⚠️
scheduler/scheduling/scheduling.go 0.00% 37 Missing ⚠️
...uler/resource/persistentcache/peer_manager_mock.go 0.00% 18 Missing ⚠️
pkg/rpc/dfdaemon/client/mocks/client_v2_mock.go 0.00% 13 Missing ⚠️
...heduler/resource/standard/seed_peer_client_mock.go 0.00% 13 Missing ⚠️
scheduler/resource/persistentcache/peer_manager.go 0.00% 12 Missing ⚠️
pkg/rpc/dfdaemon/client/client_v2.go 0.00% 6 Missing ⚠️
scheduler/scheduling/mocks/scheduling_mock.go 0.00% 5 Missing ⚠️
scheduler/resource/persistentcache/peer.go 0.00% 1 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #3787      +/-   ##
==========================================
- Coverage   34.32%   34.19%   -0.13%     
==========================================
  Files         337      337              
  Lines       38831    38982     +151     
==========================================
+ Hits        13327    13330       +3     
- Misses      24663    24810     +147     
- Partials      841      842       +1     
Flag Coverage Δ
unittests 34.19% <3.31%> (-0.13%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
scheduler/resource/persistentcache/peer.go 0.00% <0.00%> (ø)
scheduler/scheduling/mocks/scheduling_mock.go 43.93% <0.00%> (-0.68%) ⬇️
pkg/rpc/dfdaemon/client/client_v2.go 29.36% <0.00%> (-1.47%) ⬇️
scheduler/resource/persistentcache/peer_manager.go 0.00% <0.00%> (ø)
pkg/rpc/dfdaemon/client/mocks/client_v2_mock.go 0.00% <0.00%> (ø)
...heduler/resource/standard/seed_peer_client_mock.go 13.93% <0.00%> (-0.97%) ⬇️
...uler/resource/persistentcache/peer_manager_mock.go 0.00% <0.00%> (ø)
scheduler/scheduling/scheduling.go 63.40% <0.00%> (-0.21%) ⬇️
scheduler/service/service_v2.go 46.69% <5.88%> (-1.85%) ⬇️

... and 1 file with indirect coverage changes

@gaius-qi gaius-qi force-pushed the feature/replicate branch 2 times, most recently from b4734d5 to 3cb90e4 Compare January 26, 2025 14:15

Unverified

This user has not yet uploaded their public signing key.
Signed-off-by: Gaius <gaius.qi@gmail.com>
@gaius-qi gaius-qi merged commit 20926e7 into main Jan 27, 2025
27 checks passed
@gaius-qi gaius-qi deleted the feature/replicate branch January 27, 2025 02:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant