Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RFC: multi-generation log cache #854

Open
killme2008 opened this issue Jun 23, 2022 · 9 comments
Open

RFC: multi-generation log cache #854

killme2008 opened this issue Jun 23, 2022 · 9 comments
Labels
enhancement New feature or request help wanted Extra attention is needed

Comments

@killme2008
Copy link
Contributor

RFC: multi-generation log cache

As discussion in #840 , jraft keeps a cache for latest log entries, it's unlimited and may consume too much memory under write-heavy workload or catching up replication when node restarts.

I think we can abstracts a cache interface and provide a multi-generation cache to replace current one:

  • Gen-1: totaly in memory with limited capacity.
  • Gen-2: Off-heap cache.When Gen-1 overflows, move the LRU(least recently used) item into off-heap cache such as OHC.

Main goals:

  • Abstracts log cache interface.
  • Provide a multi-generation cache implementation.
@killme2008 killme2008 added enhancement New feature or request help wanted Extra attention is needed labels Jun 23, 2022
@sollhui
Copy link

sollhui commented Jul 14, 2022

@killme2008 hi,I have some Off-heap cache experience before ,and I accept issue #856 ,but the programme wait to discuss,so I want to do this first,and I have time do them together,can you assign me?

@killme2008
Copy link
Contributor Author

@HHoflittlefish777 Of course, it's yours, feel free to try and discuss with us in slack.

@sollhui
Copy link

sollhui commented Jul 14, 2022

@HHoflittlefish777 Of course, it's yours, feel free to try and discuss with us in slack.

Thanks ,I will discuss in slack later

@sollhui
Copy link

sollhui commented Jul 16, 2022

@killme2008 Hi,I ask some question in slack yesterday,hope to be given a answer if have time.

@killme2008
Copy link
Contributor Author

@HHoflittlefish777 do not have time to impl this feature, it's open for any developers that are interested in it.

@killme2008 killme2008 self-assigned this Aug 26, 2022
@RockRockWhite
Copy link

@killme2008 @hzh0425 Hello, I'm learning MIT 6.824 Course now, and I've passed the Lab2 in which I implemented Raft. I am not so familiar with JVM. But whatever, I want to have a try. Can you assign me?

@hzh0425 hzh0425 assigned RockRockWhite and unassigned killme2008 Sep 19, 2022
@hzh0425
Copy link
Member

hzh0425 commented Sep 19, 2022

@killme2008 @hzh0425 Hello, I'm learning MIT 6.824 Course now, and I've passed the Lab2 in which I implemented Raft. I am not so familiar with JVM. But whatever, I want to have a try. Can you assign me?

Sure, already assigned to you @RockRockWhite

@hzh0425
Copy link
Member

hzh0425 commented Sep 19, 2022

@RockRockWhite You can submit a RFC first

@shihuili1218
Copy link
Collaborator

@RockRockWhite May I ask this task is still being implemented?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

5 participants