Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix raft repl dev ut #543

Merged
merged 2 commits into from
Sep 11, 2024
Merged

Conversation

JacksonYao287
Copy link
Contributor

@JacksonYao287 JacksonYao287 commented Sep 10, 2024

before install snapshot completes, raft_server()->get_committed_log_idx() will be the same if write_snapshot_data is called several times, so that snp_data->offset can not be updated to new cursor.
this PR use a separate last_data_committed_lsn to maintain the last committed lsn for writing data, which is use for updating
snp_data->offset in write_snapshot_data

// last_data_committed_lsn to notify leader to transfer new data to follower.
// actually, leader will transfer all the data in one shot, so this is will not trigger any error, but we should
// make the logic correct.
snp_data->offset = last_data_committed_lsn + 1;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggesting snapshot_write_to_lsn instead of last_data_committed_lsn

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is will be updated in on_commit

xiaoxichen
xiaoxichen previously approved these changes Sep 10, 2024
@codecov-commenter
Copy link

codecov-commenter commented Sep 10, 2024

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 67.92%. Comparing base (1a0cef8) to head (074161d).
Report is 57 commits behind head on master.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@             Coverage Diff             @@
##           master     #543       +/-   ##
===========================================
+ Coverage   56.51%   67.92%   +11.41%     
===========================================
  Files         108      109        +1     
  Lines       10300    10432      +132     
  Branches     1402     1400        -2     
===========================================
+ Hits         5821     7086     +1265     
+ Misses       3894     2679     -1215     
- Partials      585      667       +82     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

@sanebay sanebay left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please run with -num_io=5000/10000

@@ -269,7 +269,10 @@ class TestReplicatedDB : public homestore::ReplDevListener {
"[Replica={}] Save logical snapshot callback obj_id={} term={} idx={} is_last={} num_items={}",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Move log one line after updating the snp_data->offset.

Copy link
Contributor Author

@JacksonYao287 JacksonYao287 Sep 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this log wants to print out what we receive from leader , so I think we should keep it as this

@JacksonYao287
Copy link
Contributor Author

@sanebay I have run it with 5000 and it works

@JacksonYao287 JacksonYao287 merged commit 55f68a0 into eBay:master Sep 11, 2024
21 checks passed
@JacksonYao287 JacksonYao287 deleted the fix-raft-repl-dev-ut branch September 11, 2024 04:07
xiaoxichen pushed a commit to xiaoxichen/HomeStore that referenced this pull request Sep 17, 2024
before install snapshot completes, raft_server()->get_committed_log_idx() will be the same if write_snapshot_data is called several times, so that snp_data->offset can not be updated to new cursor.
this PR use a separate last_data_committed_lsn to maintain the last committed lsn for writing data, which is use for updating
snp_data->offset in write_snapshot_data
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants