Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add option to ignore single artifact downloading error #6

Open
ensean opened this issue Feb 18, 2020 · 2 comments
Open

Add option to ignore single artifact downloading error #6

ensean opened this issue Feb 18, 2020 · 2 comments

Comments

@ensean
Copy link

ensean commented Feb 18, 2020

We set https://pub.flutter-io.cn/api as upstream due to network connection consideration, but recently we find that some corrupt artifact url we cause the mirroring process stopped.

Unhandled exception:
failed with 478 when visiting https://pub.flutter-io.cn/packages/sqfentity_gen/versions/1.2.3%2B18.tar.gz: status code 478
package:pub_mirror/http.dart 62:5                                                        saveFileTo
package:pub_mirror/pub_mirror.dart 130:11                                                PubMirrorTool.saveArchiveFile
package:pub_mirror/pub_mirror.dart 97:15                                                 PubMirrorTool.downloadPackage
package:pub_mirror/pub_mirror.dart 161:15                                                PubMirrorTool.download.<fn>
root/.pub-cache/hosted/pub.dartlang.org/executor-2.1.2/lib/src/executor_impl.dart 61:27  _Executor.scheduleTask
package:pub_mirror/pub_mirror.dart 159:30                                                PubMirrorTool.download
root/.pub-cache/hosted/pub.dartlang.org/pub_mirror-1.0.7/bin/pub_mirror.dart 46:8        main
dart:isolate/runtime/libisolate_patch.dart 287:32                                        _startIsolate.<fn>
dart:isolate/runtime/libisolate_patch.dart 171:12                                        _RawReceivePortImpl._handleMessage
===== asynchronous gap ===========================
package:pub_mirror/pub_mirror.dart 159:30                                                PubMirrorTool.download
root/.pub-cache/hosted/pub.dartlang.org/pub_mirror-1.0.7/bin/pub_mirror.dart 46:8        main
dart:isolate/runtime/libisolate_patch.dart 287:32                                        _startIsolate.<fn>
dart:isolate/runtime/libisolate_patch.dart 171:12                                        _RawReceivePortImpl._handleMessage

So is it possible to provide an option to ignore such kind of error? So we can sync most of the artifacts with the https://pub.flutter-io.cn/api upstream and sync the corrupt files from pub.dev

@chenglu
Copy link

chenglu commented Mar 6, 2020

The "HTTP 478 error" might be caused by the CN mirrors provider at that time,
this issue might not happen again.

But like I mentioned in this issue, if the package owner deletes the package versions, could we detect and delete that on our cached server? This might be a #TODO in the next version.
cc/ @huiyiqun, @jiegec

@z4yx
Copy link
Member

z4yx commented Mar 22, 2020

The "HTTP 478 error" might be caused by the CN mirrors provider at that time,
this issue might not happen again.

But like I mentioned in this issue, if the package owner deletes the package versions, could we detect and delete that on our cached server? This might be a #TODO in the next version.
cc/ @huiyiqun, @jiegec

Deleting old versions are now supported on the dev branch of https://github.com/tuna/pub-mirror/ , enabling with --delete option.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants