Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

new feature: Add more checksum support for S3 #5550

Open
1 of 7 tasks
Xuanwo opened this issue Jan 15, 2025 · 1 comment
Open
1 of 7 tasks

new feature: Add more checksum support for S3 #5550

Xuanwo opened this issue Jan 15, 2025 · 1 comment
Labels
enhancement New feature or request services/s3

Comments

@Xuanwo
Copy link
Member

Xuanwo commented Jan 15, 2025 •
edited
Loading

Feature Description

crc64-nvme is the new crc algo that supported by s3: https://docs.aws.amazon.com/AmazonS3/latest/userguide/checking-object-integrity.html

Problem and Solution

  • CRC64-NVME (Recommended)
    • crc64-nvme is the default algo used by s3. Adding it in opendal can make sure the data we upload is correct.
  • CRC-32
  • CRC-32C
  • SHA-1
  • SHA-256
  • Also, content-md5.

Additional Context

No response

Are you willing to contribute to the development of this feature?

  • Yes, I am willing to contribute to the development of this feature.
@Xuanwo Xuanwo added the enhancement New feature or request label Jan 15, 2025
@Xuanwo Xuanwo changed the title new feature: Add crc64-nvme checksum support for S3 new feature: Add more checksum support for S3 Jan 15, 2025
@geetanshjuneja
Copy link
Contributor

Hi, I would like to work on this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request services/s3
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@messense @Xuanwo @geetanshjuneja