Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
2.044934694s
FAIL (2025-08-07 16:16:47 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== entMakeInline Aug 7 15:57:10.271829415 (6h17m9.861174403s) [upload=6533]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 15:57:10.587098885 (6h17m10.176443872s) [upload=6533]: Done uploading segments... Aug 7 15:57:10.587115857 (6h17m10.176460835s) [upload=6533]: Upload finished. err: <nil> Aug 7 15:57:48.918901442 (6h17m48.508246429s) [upload=6545]: Starting upload Aug 7 15:57:48.918948701 (6h17m48.508293688s) [upload=6545]: Done starting upload Aug 7 15:57:48.919013486 (6h17m48.508358473s) [upload=6545]: Uploading segments... Aug 7 15:57:48.919055005 (6h17m48.508399982s) [upload=6545]: Got next segment. Inline: true Aug 7 15:57:48.919059032 (6h17m48.508404009s) [upload=6545]: Waiting for error group managing segments... Aug 7 15:57:48.919091585 (6h17m48.508436572s) [upload=6545]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 15:57:48.919095493 (6h17m48.50844048s) [upload=6545]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 15:57:48.919097998 (6h17m48.508442975s) [upload=6545]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 15:57:49.200378977 (6h17m48.789723964s) [upload=6545]: Done uploading segments... Aug 7 15:57:49.200397332 (6h17m48.789742319s) [upload=6545]: Upload finished. err: <nil> Aug 7 15:58:50.898935036 (6h18m50.488280023s) [upload=6562]: Starting upload Aug 7 15:58:50.898979992 (6h18m50.488324979s) [upload=6562]: Done starting upload Aug 7 15:58:50.899026682 (6h18m50.488371669s) [upload=6562]: Uploading segments... Aug 7 15:58:50.899065355 (6h18m50.488410343s) [upload=6562]: Got next segment. Inline: true Aug 7 15:58:50.899070335 (6h18m50.488415322s) [upload=6562]: Waiting for error group managing segments... Aug 7 15:58:50.899108268 (6h18m50.488453255s) [upload=6562]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 15:58:50.899112326 (6h18m50.488457313s) [upload=6562]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 15:58:50.899114820 (6h18m50.488459807s) [upload=6562]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 15:58:51.203289783 (6h18m50.79263478s) [upload=6562]: Done uploading segments... Aug 7 15:58:51.203312587 (6h18m50.792657624s) [upload=6562]: Upload finished. err: <nil> Aug 7 16:00:00.470970125 (6h20m0.060315112s) [upload=6580]: Starting upload Aug 7 16:00:00.471017334 (6h20m0.060362321s) [upload=6580]: Done starting upload Aug 7 16:00:00.471072199 (6h20m0.060417186s) [upload=6580]: Uploading segments... Aug 7 16:00:00.471112327 (6h20m0.060457314s) [upload=6580]: Got next segment. Inline: true Aug 7 16:00:00.471116364 (6h20m0.060461351s) [upload=6580]: Waiting for error group managing segments... Aug 7 16:00:00.471149427 (6h20m0.060494414s) [upload=6580]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:00:00.471153305 (6h20m0.060498292s) [upload=6580]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:00:00.471155819 (6h20m0.060500806s) [upload=6580]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:00:00.739624026 (6h20m0.328969023s) [upload=6580]: Done uploading segments... Aug 7 16:00:00.739645636 (6h20m0.328990623s) [upload=6580]: Upload finished. err: <nil> Aug 7 16:00:49.882359165 (6h20m49.471704152s) [upload=6598]: Starting upload Aug 7 16:00:49.882404682 (6h20m49.471749659s) [upload=6598]: Done starting upload Aug 7 16:00:49.882456812 (6h20m49.471801799s) [upload=6598]: Uploading segments... Aug 7 16:00:49.882489865 (6h20m49.471834852s) [upload=6598]: Got next segment. Inline: true Aug 7 16:00:49.882493702 (6h20m49.47183869s) [upload=6598]: Waiting for error group managing segments... Aug 7 16:00:49.882525894 (6h20m49.471870881s) [upload=6598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:00:49.882529422 (6h20m49.471874399s) [upload=6598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:00:49.882531676 (6h20m49.471876663s) [upload=6598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:00:50.213720013 (6h20m49.803065s) [upload=6598]: Done uploading segments... Aug 7 16:00:50.213739550 (6h20m49.803084537s) [upload=6598]: Upload finished. err: <nil> Aug 7 16:01:51.762778733 (6h21m51.35212372s) [upload=6617]: Starting upload Aug 7 16:01:51.762856161 (6h21m51.352201148s) [upload=6617]: Done starting upload Aug 7 16:01:51.762914883 (6h21m51.35225987s) [upload=6617]: Uploading segments... Aug 7 16:01:51.762952445 (6h21m51.352297432s) [upload=6617]: Got next segment. Inline: true Aug 7 16:01:51.762957344 (6h21m51.352302331s) [upload=6617]: Waiting for error group managing segments... Aug 7 16:01:51.762994485 (6h21m51.352339473s) [upload=6617]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:01:51.762998313 (6h21m51.3523433s) [upload=6617]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:01:51.763001278 (6h21m51.352346265s) [upload=6617]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:01:52.034377407 (6h21m51.623722394s) [upload=6617]: Done uploading segments... Aug 7 16:01:52.034395451 (6h21m51.623740439s) [upload=6617]: Upload finished. err: <nil> Aug 7 16:02:11.154406473 (6h22m10.74375146s) [upload=6618]: Starting upload Aug 7 16:02:11.154456328 (6h22m10.743801315s) [upload=6618]: Done starting upload Aug 7 16:02:11.154503859 (6h22m10.743848846s) [upload=6618]: Uploading segments... Aug 7 16:02:11.154545308 (6h22m10.743890295s) [upload=6618]: Got next segment. Inline: true Aug 7 16:02:11.154548574 (6h22m10.743893561s) [upload=6618]: Waiting for error group managing segments... Aug 7 16:02:11.154580566 (6h22m10.743925553s) [upload=6618]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:02:11.154584283 (6h22m10.74392927s) [upload=6618]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:02:11.154586828 (6h22m10.743931815s) [upload=6618]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:02:11.475329609 (6h22m11.064674596s) [upload=6618]: Done uploading segments... Aug 7 16:02:11.475484125 (6h22m11.064829112s) [upload=6618]: Upload finished. err: <nil> Aug 7 16:02:42.941029250 (6h22m42.530374238s) [upload=6628]: Starting upload Aug 7 16:02:42.941096740 (6h22m42.530441727s) [upload=6628]: Done starting upload Aug 7 16:02:42.941158728 (6h22m42.530503716s) [upload=6628]: Uploading segments... Aug 7 16:02:42.941210478 (6h22m42.530555465s) [upload=6628]: Got next segment. Inline: true Aug 7 16:02:42.941216419 (6h22m42.530561416s) [upload=6628]: Waiting for error group managing segments... Aug 7 16:02:42.941281664 (6h22m42.530626651s) [upload=6628]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:02:42.941286824 (6h22m42.530631811s) [upload=6628]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:02:42.941289880 (6h22m42.530634867s) [upload=6628]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:02:43.233814342 (6h22m42.823159339s) [upload=6628]: Done uploading segments... Aug 7 16:02:43.233838148 (6h22m42.823183145s) [upload=6628]: Upload finished. err: <nil> Aug 7 16:03:45.201690833 (6h23m44.79103582s) [upload=6651]: Starting upload Aug 7 16:03:45.201738173 (6h23m44.79108316s) [upload=6651]: Done starting upload Aug 7 16:03:45.201785534 (6h23m44.791130511s) [upload=6651]: Uploading segments... Aug 7 16:03:45.201855618 (6h23m44.791200605s) [upload=6651]: Got next segment. Inline: true Aug 7 16:03:45.201859616 (6h23m44.791204603s) [upload=6651]: Waiting for error group managing segments... Aug 7 16:03:45.201894122 (6h23m44.791239109s) [upload=6651]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:03:45.201897799 (6h23m44.791242786s) [upload=6651]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:03:45.201900023 (6h23m44.79124501s) [upload=6651]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:03:45.644503676 (6h23m45.233848664s) [upload=6651]: Done uploading segments... Aug 7 16:03:45.644521010 (6h23m45.233865987s) [upload=6651]: Upload finished. err: <nil> Aug 7 16:04:31.274936212 (6h24m30.864281199s) [upload=6663]: Starting upload Aug 7 16:04:31.274977170 (6h24m30.864322157s) [upload=6663]: Done starting upload Aug 7 16:04:31.275026655 (6h24m30.864371642s) [upload=6663]: Uploading segments... Aug 7 16:04:31.275054187 (6h24m30.864399174s) [upload=6663]: Got next segment. Inline: true Aug 7 16:04:31.275057754 (6h24m30.864402731s) [upload=6663]: Waiting for error group managing segments... Aug 7 16:04:31.275091278 (6h24m30.864436265s) [upload=6663]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:04:31.275095255 (6h24m30.864440242s) [upload=6663]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:04:31.275097651 (6h24m30.864442638s) [upload=6663]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:04:31.554258512 (6h24m31.143603499s) [upload=6663]: Done uploading segments... Aug 7 16:04:31.554277108 (6h24m31.143622095s) [upload=6663]: Upload finished. err: <nil> Aug 7 16:05:14.730035713 (6h25m14.3193807s) [upload=6674]: Starting upload Aug 7 16:05:14.730082813 (6h25m14.3194278s) [upload=6674]: Done starting upload Aug 7 16:05:14.730159079 (6h25m14.319504056s) [upload=6674]: Uploading segments... Aug 7 16:05:14.730249532 (6h25m14.319594519s) [upload=6674]: Got next segment. Inline: true Aug 7 16:05:14.730254050 (6h25m14.319599037s) [upload=6674]: Waiting for error group managing segments... Aug 7 16:05:14.730294949 (6h25m14.319639936s) [upload=6674]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:05:14.730299728 (6h25m14.319644715s) [upload=6674]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:05:14.730302243 (6h25m14.31964723s) [upload=6674]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:05:14.942773689 (6h25m14.532118666s) [upload=6674]: Done uploading segments... Aug 7 16:05:14.942790141 (6h25m14.532135118s) [upload=6674]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.044955655s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
1.966875146s
FAIL (2025-08-07 16:16:17 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== entMakeInline Aug 7 15:57:10.271829415 (6h17m9.861174403s) [upload=6533]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 15:57:10.587098885 (6h17m10.176443872s) [upload=6533]: Done uploading segments... Aug 7 15:57:10.587115857 (6h17m10.176460835s) [upload=6533]: Upload finished. err: <nil> Aug 7 15:57:48.918901442 (6h17m48.508246429s) [upload=6545]: Starting upload Aug 7 15:57:48.918948701 (6h17m48.508293688s) [upload=6545]: Done starting upload Aug 7 15:57:48.919013486 (6h17m48.508358473s) [upload=6545]: Uploading segments... Aug 7 15:57:48.919055005 (6h17m48.508399982s) [upload=6545]: Got next segment. Inline: true Aug 7 15:57:48.919059032 (6h17m48.508404009s) [upload=6545]: Waiting for error group managing segments... Aug 7 15:57:48.919091585 (6h17m48.508436572s) [upload=6545]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 15:57:48.919095493 (6h17m48.50844048s) [upload=6545]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 15:57:48.919097998 (6h17m48.508442975s) [upload=6545]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 15:57:49.200378977 (6h17m48.789723964s) [upload=6545]: Done uploading segments... Aug 7 15:57:49.200397332 (6h17m48.789742319s) [upload=6545]: Upload finished. err: <nil> Aug 7 15:58:50.898935036 (6h18m50.488280023s) [upload=6562]: Starting upload Aug 7 15:58:50.898979992 (6h18m50.488324979s) [upload=6562]: Done starting upload Aug 7 15:58:50.899026682 (6h18m50.488371669s) [upload=6562]: Uploading segments... Aug 7 15:58:50.899065355 (6h18m50.488410343s) [upload=6562]: Got next segment. Inline: true Aug 7 15:58:50.899070335 (6h18m50.488415322s) [upload=6562]: Waiting for error group managing segments... Aug 7 15:58:50.899108268 (6h18m50.488453255s) [upload=6562]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 15:58:50.899112326 (6h18m50.488457313s) [upload=6562]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 15:58:50.899114820 (6h18m50.488459807s) [upload=6562]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 15:58:51.203289783 (6h18m50.79263478s) [upload=6562]: Done uploading segments... Aug 7 15:58:51.203312587 (6h18m50.792657624s) [upload=6562]: Upload finished. err: <nil> Aug 7 16:00:00.470970125 (6h20m0.060315112s) [upload=6580]: Starting upload Aug 7 16:00:00.471017334 (6h20m0.060362321s) [upload=6580]: Done starting upload Aug 7 16:00:00.471072199 (6h20m0.060417186s) [upload=6580]: Uploading segments... Aug 7 16:00:00.471112327 (6h20m0.060457314s) [upload=6580]: Got next segment. Inline: true Aug 7 16:00:00.471116364 (6h20m0.060461351s) [upload=6580]: Waiting for error group managing segments... Aug 7 16:00:00.471149427 (6h20m0.060494414s) [upload=6580]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:00:00.471153305 (6h20m0.060498292s) [upload=6580]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:00:00.471155819 (6h20m0.060500806s) [upload=6580]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:00:00.739624026 (6h20m0.328969023s) [upload=6580]: Done uploading segments... Aug 7 16:00:00.739645636 (6h20m0.328990623s) [upload=6580]: Upload finished. err: <nil> Aug 7 16:00:49.882359165 (6h20m49.471704152s) [upload=6598]: Starting upload Aug 7 16:00:49.882404682 (6h20m49.471749659s) [upload=6598]: Done starting upload Aug 7 16:00:49.882456812 (6h20m49.471801799s) [upload=6598]: Uploading segments... Aug 7 16:00:49.882489865 (6h20m49.471834852s) [upload=6598]: Got next segment. Inline: true Aug 7 16:00:49.882493702 (6h20m49.47183869s) [upload=6598]: Waiting for error group managing segments... Aug 7 16:00:49.882525894 (6h20m49.471870881s) [upload=6598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:00:49.882529422 (6h20m49.471874399s) [upload=6598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:00:49.882531676 (6h20m49.471876663s) [upload=6598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:00:50.213720013 (6h20m49.803065s) [upload=6598]: Done uploading segments... Aug 7 16:00:50.213739550 (6h20m49.803084537s) [upload=6598]: Upload finished. err: <nil> Aug 7 16:01:51.762778733 (6h21m51.35212372s) [upload=6617]: Starting upload Aug 7 16:01:51.762856161 (6h21m51.352201148s) [upload=6617]: Done starting upload Aug 7 16:01:51.762914883 (6h21m51.35225987s) [upload=6617]: Uploading segments... Aug 7 16:01:51.762952445 (6h21m51.352297432s) [upload=6617]: Got next segment. Inline: true Aug 7 16:01:51.762957344 (6h21m51.352302331s) [upload=6617]: Waiting for error group managing segments... Aug 7 16:01:51.762994485 (6h21m51.352339473s) [upload=6617]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:01:51.762998313 (6h21m51.3523433s) [upload=6617]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:01:51.763001278 (6h21m51.352346265s) [upload=6617]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:01:52.034377407 (6h21m51.623722394s) [upload=6617]: Done uploading segments... Aug 7 16:01:52.034395451 (6h21m51.623740439s) [upload=6617]: Upload finished. err: <nil> Aug 7 16:02:11.154406473 (6h22m10.74375146s) [upload=6618]: Starting upload Aug 7 16:02:11.154456328 (6h22m10.743801315s) [upload=6618]: Done starting upload Aug 7 16:02:11.154503859 (6h22m10.743848846s) [upload=6618]: Uploading segments... Aug 7 16:02:11.154545308 (6h22m10.743890295s) [upload=6618]: Got next segment. Inline: true Aug 7 16:02:11.154548574 (6h22m10.743893561s) [upload=6618]: Waiting for error group managing segments... Aug 7 16:02:11.154580566 (6h22m10.743925553s) [upload=6618]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:02:11.154584283 (6h22m10.74392927s) [upload=6618]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:02:11.154586828 (6h22m10.743931815s) [upload=6618]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:02:11.475329609 (6h22m11.064674596s) [upload=6618]: Done uploading segments... Aug 7 16:02:11.475484125 (6h22m11.064829112s) [upload=6618]: Upload finished. err: <nil> Aug 7 16:02:42.941029250 (6h22m42.530374238s) [upload=6628]: Starting upload Aug 7 16:02:42.941096740 (6h22m42.530441727s) [upload=6628]: Done starting upload Aug 7 16:02:42.941158728 (6h22m42.530503716s) [upload=6628]: Uploading segments... Aug 7 16:02:42.941210478 (6h22m42.530555465s) [upload=6628]: Got next segment. Inline: true Aug 7 16:02:42.941216419 (6h22m42.530561416s) [upload=6628]: Waiting for error group managing segments... Aug 7 16:02:42.941281664 (6h22m42.530626651s) [upload=6628]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:02:42.941286824 (6h22m42.530631811s) [upload=6628]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:02:42.941289880 (6h22m42.530634867s) [upload=6628]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:02:43.233814342 (6h22m42.823159339s) [upload=6628]: Done uploading segments... Aug 7 16:02:43.233838148 (6h22m42.823183145s) [upload=6628]: Upload finished. err: <nil> Aug 7 16:03:45.201690833 (6h23m44.79103582s) [upload=6651]: Starting upload Aug 7 16:03:45.201738173 (6h23m44.79108316s) [upload=6651]: Done starting upload Aug 7 16:03:45.201785534 (6h23m44.791130511s) [upload=6651]: Uploading segments... Aug 7 16:03:45.201855618 (6h23m44.791200605s) [upload=6651]: Got next segment. Inline: true Aug 7 16:03:45.201859616 (6h23m44.791204603s) [upload=6651]: Waiting for error group managing segments... Aug 7 16:03:45.201894122 (6h23m44.791239109s) [upload=6651]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:03:45.201897799 (6h23m44.791242786s) [upload=6651]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:03:45.201900023 (6h23m44.79124501s) [upload=6651]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:03:45.644503676 (6h23m45.233848664s) [upload=6651]: Done uploading segments... Aug 7 16:03:45.644521010 (6h23m45.233865987s) [upload=6651]: Upload finished. err: <nil> Aug 7 16:04:31.274936212 (6h24m30.864281199s) [upload=6663]: Starting upload Aug 7 16:04:31.274977170 (6h24m30.864322157s) [upload=6663]: Done starting upload Aug 7 16:04:31.275026655 (6h24m30.864371642s) [upload=6663]: Uploading segments... Aug 7 16:04:31.275054187 (6h24m30.864399174s) [upload=6663]: Got next segment. Inline: true Aug 7 16:04:31.275057754 (6h24m30.864402731s) [upload=6663]: Waiting for error group managing segments... Aug 7 16:04:31.275091278 (6h24m30.864436265s) [upload=6663]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:04:31.275095255 (6h24m30.864440242s) [upload=6663]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:04:31.275097651 (6h24m30.864442638s) [upload=6663]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:04:31.554258512 (6h24m31.143603499s) [upload=6663]: Done uploading segments... Aug 7 16:04:31.554277108 (6h24m31.143622095s) [upload=6663]: Upload finished. err: <nil> Aug 7 16:05:14.730035713 (6h25m14.3193807s) [upload=6674]: Starting upload Aug 7 16:05:14.730082813 (6h25m14.3194278s) [upload=6674]: Done starting upload Aug 7 16:05:14.730159079 (6h25m14.319504056s) [upload=6674]: Uploading segments... Aug 7 16:05:14.730249532 (6h25m14.319594519s) [upload=6674]: Got next segment. Inline: true Aug 7 16:05:14.730254050 (6h25m14.319599037s) [upload=6674]: Waiting for error group managing segments... Aug 7 16:05:14.730294949 (6h25m14.319639936s) [upload=6674]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:05:14.730299728 (6h25m14.319644715s) [upload=6674]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:05:14.730302243 (6h25m14.31964723s) [upload=6674]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:05:14.942773689 (6h25m14.532118666s) [upload=6674]: Done uploading segments... Aug 7 16:05:14.942790141 (6h25m14.532135118s) [upload=6674]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.966905354s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [...............................X.......X....................] duration:
5.851342625s
FAIL (2025-08-07 16:15:54 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
e begin upload segment.
Aug 7 16:01:39.748945470 (6h21m39.338290447s) [noise=true , address=104.36.82.42:4820 , piece_id=XJIHI2E35NJKDFYFTW4V5EIF3CL7GDJ5EKVCTEV2KL26X43A3CYA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.748969657 (6h21m39.338314644s) [noise=true , address=104.36.82.60:4345 , piece_id=VPYOQHGEP6E257WI7IGIWXRO5WOLCNGDJYNPFPC56WUMOYDUS6RA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749368329 (6h21m39.338713316s) [noise=true , address=45.91.116.9:4267 , piece_id=MWAYSXFIIYQGDPTLWZSAHZP3UO72EXYDSSULHBJJU54H5OWG36NA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749375964 (6h21m39.338720951s) [noise=true , address=104.36.82.10:4110 , piece_id=WAUEFOZXH3SULGTXTWXWLEPYHZJWJDDQRRWWXRBXVWF7WCWPYGYQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749660398 (6h21m39.339005385s) [noise=true , address=79.127.205.235:4666 , piece_id=WA6GZKNO2UUAC5VKBNOZ7UO25RQR7QDT5HVNQYZDIWJEB656WMGA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749701216 (6h21m39.339046203s) [noise=true , address=79.127.201.215:4345 , piece_id=MAAFKAPO4OCQXU5WJ6DAP72KEHIM4AO5JJFJ4ZEFNWKS5M4OV43Q, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749938690 (6h21m39.339283677s) [noise=true , address=104.36.82.59:4313 , piece_id=KDH6AVALUMDXB6USO7WG77DVOSRBTGSZA6VTJFFQJR7QFC2PE35A, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750024905 (6h21m39.339369892s) [noise=true , address=104.36.82.23:4455 , piece_id=UJOJ7P3VET4XIM257B3SHAN6SJG2H64QJHV6PYU7OEB7PPBZVDAA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750215089 (6h21m39.339560076s) [noise=true , address=104.36.82.61:4387 , piece_id=MAWGNSJTYYH7EVPDIXQEG6KP3MKACSMWYVMMLYUASUDJBIX5POOA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750282639 (6h21m39.339627626s) [noise=true , address=45.91.116.10:4092 , piece_id=BUVKJC3K3RKZZTW2CQPEQM7ECQOLIBX4CU66P26JZDTOQZLYGMGA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750481038 (6h21m39.339826025s) [noise=true , address=79.127.220.97:4396 , piece_id=X2CCRNYWY5ZQOAODHNJE7VLYQIHD37PEE7MQXI5WIYV36ICONILQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750533900 (6h21m39.339878887s) [noise=true , address=104.36.82.60:4369 , piece_id=Y6SYJOB7SYBJX7STC25OJJPD4FSK5TZ5KXI7WN7QS42UNTOF5BCQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750725967 (6h21m39.340070954s) [noise=true , address=104.36.82.9:4039 , piece_id=PRBG5EVKAZ3LGDRX4MCI57T4RSNFVKZVUNOAIRHBM6SNS5XXUTPQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750795189 (6h21m39.340140176s) [noise=true , address=104.36.82.22:4525 , piece_id=NKD4EXV774KNXLEHICE4MNKOQS5QKPIUTJL2PPCNYUMYPMQEF75A, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750954023 (6h21m39.34029901s) [noise=true , address=104.36.82.45:4912 , piece_id=B6HTS6ORSGMUKZO5G2ALW4LVUSIBTCEHPX2YJTYPPCH3XROIO3HA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.751098079 (6h21m39.340443066s) [noise=true , address=104.36.82.8:4992 , piece_id=226HGROQ3DBKTJB3U6TG2OF4JQRYAGTLMSIL2JIL4NY5PISP4RUQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.751175397 (6h21m39.340520384s) [noise=true , address=79.127.205.233:4631 , piece_id=77WNKENAMS4Q75KYK4F4VNRP4DD4MKGEJIU5K7WZ3EHVQLEW6Q5A, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.751392602 (6h21m39.340737579s) [noise=true , address=104.36.82.5:4845 , piece_id=JKN4VFS7ET2LVGHGE4LIBPCGG33GNY4SGSOXAWB2MXFYPJVO4FCQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:40.878402060 (6h21m40.467747047s) [upload=6610]: Next returned nil segment
Aug 7 16:01:40.878416708 (6h21m40.467761695s) [upload=6610]: Waiting for error group managing segments...
Aug 7 16:01:41.195597254 (6h21m40.784942241s) [noise=true , address=79.127.205.235:4666 , piece_id=WA6GZKNO2UUAC5VKBNOZ7UO25RQR7QDT5HVNQYZDIWJEB656WMGA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.261081133 (6h21m40.85042612s) [noise=true , address=79.127.205.233:4631 , piece_id=77WNKENAMS4Q75KYK4F4VNRP4DD4MKGEJIU5K7WZ3EHVQLEW6Q5A, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.420961069 (6h21m41.010306046s) [noise=true , address=79.127.220.97:4396 , piece_id=X2CCRNYWY5ZQOAODHNJE7VLYQIHD37PEE7MQXI5WIYV36ICONILQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.638341479 (6h21m41.227686486s) [noise=true , address=79.127.201.215:4345 , piece_id=MAAFKAPO4OCQXU5WJ6DAP72KEHIM4AO5JJFJ4ZEFNWKS5M4OV43Q, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.658966771 (6h21m41.248311758s) [noise=true , address=45.91.116.10:4092 , piece_id=BUVKJC3K3RKZZTW2CQPEQM7ECQOLIBX4CU66P26JZDTOQZLYGMGA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.684178459 (6h21m41.273523446s) [noise=true , address=104.36.82.9:4039 , piece_id=PRBG5EVKAZ3LGDRX4MCI57T4RSNFVKZVUNOAIRHBM6SNS5XXUTPQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.691448508 (6h21m41.280793485s) [noise=true , address=104.36.82.22:4525 , piece_id=NKD4EXV774KNXLEHICE4MNKOQS5QKPIUTJL2PPCNYUMYPMQEF75A, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.717147267 (6h21m41.306492254s) [noise=true , address=104.36.82.59:4313 , piece_id=KDH6AVALUMDXB6USO7WG77DVOSRBTGSZA6VTJFFQJR7QFC2PE35A, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.719612885 (6h21m41.308957872s) [noise=true , address=104.36.82.45:4912 , piece_id=B6HTS6ORSGMUKZO5G2ALW4LVUSIBTCEHPX2YJTYPPCH3XROIO3HA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.733973160 (6h21m41.323318147s) [noise=true , address=104.36.82.60:4369 , piece_id=Y6SYJOB7SYBJX7STC25OJJPD4FSK5TZ5KXI7WN7QS42UNTOF5BCQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.737980718 (6h21m41.327325705s) [noise=true , address=104.36.82.5:4845 , piece_id=JKN4VFS7ET2LVGHGE4LIBPCGG33GNY4SGSOXAWB2MXFYPJVO4FCQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.745434729 (6h21m41.334779717s) [noise=true , address=104.36.82.42:4820 , piece_id=XJIHI2E35NJKDFYFTW4V5EIF3CL7GDJ5EKVCTEV2KL26X43A3CYA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.754624732 (6h21m41.34396972s) [noise=true , address=104.36.82.8:4992 , piece_id=226HGROQ3DBKTJB3U6TG2OF4JQRYAGTLMSIL2JIL4NY5PISP4RUQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.763605976 (6h21m41.352950963s) [noise=true , address=104.36.82.60:4345 , piece_id=VPYOQHGEP6E257WI7IGIWXRO5WOLCNGDJYNPFPC56WUMOYDUS6RA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.763623339 (6h21m41.352968336s) [seg_pos={0 0}, upload=6610]: Segment reached optimal threshold of 14 pieces.
Aug 7 16:01:41.763741525 (6h21m41.353086512s) [noise=true , address=104.36.82.10:4110 , piece_id=WAUEFOZXH3SULGTXTWXWLEPYHZJWJDDQRRWWXRBXVWF7WCWPYGYQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ADbnPJGRTw2NqRzhy7ptojGYoNdJgqmjuWBSjcrQK53Wm7LXi): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763803123 (6h21m41.35314811s) [noise=true , address=104.36.82.61:4387 , piece_id=MAWGNSJTYYH7EVPDIXQEG6KP3MKACSMWYVMMLYUASUDJBIX5POOA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P4wAXSyaRY9poPRK5q4td8DJM69xWioG38TTwLTHKHA7QFxLp): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763857136 (6h21m41.353202113s) [noise=true , address=45.91.116.9:4267 , piece_id=MWAYSXFIIYQGDPTLWZSAHZP3UO72EXYDSSULHBJJU54H5OWG36NA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6M5TgGqUwjXyM25uqv5hsh67mS81ByzMM4kWFUNaFvisDN9d): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763904968 (6h21m41.353249955s) [noise=true , address=104.36.82.23:4455 , piece_id=UJOJ7P3VET4XIM257B3SHAN6SJG2H64QJHV6PYU7OEB7PPBZVDAA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wsyjF5X3iPrJsezUjakUArxc8K88T36yT4rE3yu6q9BMKfycm): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763945405 (6h21m41.353290392s) [seg_pos={0 0}, upload=6610]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 16:01:41.764033824 (6h21m41.353378811s) [upload=6610]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 16:01:41.764038263 (6h21m41.35338325s) [upload=6610]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 16:01:42.043459384 (6h21m41.632804371s) [upload=6610]: Done uploading segments...
Aug 7 16:01:42.043475946 (6h21m41.632820933s) [upload=6610]: Upload finished. err: <nil>
Aug 7 16:02:11.915996790 (6h22m11.505341777s) [upload=6619]: Starting upload
Aug 7 16:02:11.916045183 (6h22m11.50539017s) [upload=6619]: Done starting upload
Aug 7 16:02:11.916152489 (6h22m11.505497476s) [upload=6619]: Uploading segments...
Aug 7 16:02:11.916287706 (6h22m11.505632693s) [upload=6619]: Got next segment. Inline: false
Aug 7 16:02:11.916319939 (6h22m11.505664926s) [upload=6619]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 16:02:11.916324708 (6h22m11.505669695s) [upload=6619]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 16:02:12.129590885 (6h22m11.718935892s) [upload=6619]: Next segment err: metaclient: Too Many Requests
Aug 7 16:02:12.129623196 (6h22m11.718968194s) [upload=6619]: Done uploading segments...
Aug 7 16:02:12.129640310 (6h22m11.718985307s) [upload=6619]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.851371641s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [................................X.......X...................] duration:
6.095168729s
FAIL (2025-08-07 16:15:26 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
e begin upload segment.
Aug 7 16:01:39.748945470 (6h21m39.338290447s) [noise=true , address=104.36.82.42:4820 , piece_id=XJIHI2E35NJKDFYFTW4V5EIF3CL7GDJ5EKVCTEV2KL26X43A3CYA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.748969657 (6h21m39.338314644s) [noise=true , address=104.36.82.60:4345 , piece_id=VPYOQHGEP6E257WI7IGIWXRO5WOLCNGDJYNPFPC56WUMOYDUS6RA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749368329 (6h21m39.338713316s) [noise=true , address=45.91.116.9:4267 , piece_id=MWAYSXFIIYQGDPTLWZSAHZP3UO72EXYDSSULHBJJU54H5OWG36NA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749375964 (6h21m39.338720951s) [noise=true , address=104.36.82.10:4110 , piece_id=WAUEFOZXH3SULGTXTWXWLEPYHZJWJDDQRRWWXRBXVWF7WCWPYGYQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749660398 (6h21m39.339005385s) [noise=true , address=79.127.205.235:4666 , piece_id=WA6GZKNO2UUAC5VKBNOZ7UO25RQR7QDT5HVNQYZDIWJEB656WMGA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749701216 (6h21m39.339046203s) [noise=true , address=79.127.201.215:4345 , piece_id=MAAFKAPO4OCQXU5WJ6DAP72KEHIM4AO5JJFJ4ZEFNWKS5M4OV43Q, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749938690 (6h21m39.339283677s) [noise=true , address=104.36.82.59:4313 , piece_id=KDH6AVALUMDXB6USO7WG77DVOSRBTGSZA6VTJFFQJR7QFC2PE35A, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750024905 (6h21m39.339369892s) [noise=true , address=104.36.82.23:4455 , piece_id=UJOJ7P3VET4XIM257B3SHAN6SJG2H64QJHV6PYU7OEB7PPBZVDAA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750215089 (6h21m39.339560076s) [noise=true , address=104.36.82.61:4387 , piece_id=MAWGNSJTYYH7EVPDIXQEG6KP3MKACSMWYVMMLYUASUDJBIX5POOA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750282639 (6h21m39.339627626s) [noise=true , address=45.91.116.10:4092 , piece_id=BUVKJC3K3RKZZTW2CQPEQM7ECQOLIBX4CU66P26JZDTOQZLYGMGA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750481038 (6h21m39.339826025s) [noise=true , address=79.127.220.97:4396 , piece_id=X2CCRNYWY5ZQOAODHNJE7VLYQIHD37PEE7MQXI5WIYV36ICONILQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750533900 (6h21m39.339878887s) [noise=true , address=104.36.82.60:4369 , piece_id=Y6SYJOB7SYBJX7STC25OJJPD4FSK5TZ5KXI7WN7QS42UNTOF5BCQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750725967 (6h21m39.340070954s) [noise=true , address=104.36.82.9:4039 , piece_id=PRBG5EVKAZ3LGDRX4MCI57T4RSNFVKZVUNOAIRHBM6SNS5XXUTPQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750795189 (6h21m39.340140176s) [noise=true , address=104.36.82.22:4525 , piece_id=NKD4EXV774KNXLEHICE4MNKOQS5QKPIUTJL2PPCNYUMYPMQEF75A, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750954023 (6h21m39.34029901s) [noise=true , address=104.36.82.45:4912 , piece_id=B6HTS6ORSGMUKZO5G2ALW4LVUSIBTCEHPX2YJTYPPCH3XROIO3HA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.751098079 (6h21m39.340443066s) [noise=true , address=104.36.82.8:4992 , piece_id=226HGROQ3DBKTJB3U6TG2OF4JQRYAGTLMSIL2JIL4NY5PISP4RUQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.751175397 (6h21m39.340520384s) [noise=true , address=79.127.205.233:4631 , piece_id=77WNKENAMS4Q75KYK4F4VNRP4DD4MKGEJIU5K7WZ3EHVQLEW6Q5A, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.751392602 (6h21m39.340737579s) [noise=true , address=104.36.82.5:4845 , piece_id=JKN4VFS7ET2LVGHGE4LIBPCGG33GNY4SGSOXAWB2MXFYPJVO4FCQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:40.878402060 (6h21m40.467747047s) [upload=6610]: Next returned nil segment
Aug 7 16:01:40.878416708 (6h21m40.467761695s) [upload=6610]: Waiting for error group managing segments...
Aug 7 16:01:41.195597254 (6h21m40.784942241s) [noise=true , address=79.127.205.235:4666 , piece_id=WA6GZKNO2UUAC5VKBNOZ7UO25RQR7QDT5HVNQYZDIWJEB656WMGA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.261081133 (6h21m40.85042612s) [noise=true , address=79.127.205.233:4631 , piece_id=77WNKENAMS4Q75KYK4F4VNRP4DD4MKGEJIU5K7WZ3EHVQLEW6Q5A, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.420961069 (6h21m41.010306046s) [noise=true , address=79.127.220.97:4396 , piece_id=X2CCRNYWY5ZQOAODHNJE7VLYQIHD37PEE7MQXI5WIYV36ICONILQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.638341479 (6h21m41.227686486s) [noise=true , address=79.127.201.215:4345 , piece_id=MAAFKAPO4OCQXU5WJ6DAP72KEHIM4AO5JJFJ4ZEFNWKS5M4OV43Q, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.658966771 (6h21m41.248311758s) [noise=true , address=45.91.116.10:4092 , piece_id=BUVKJC3K3RKZZTW2CQPEQM7ECQOLIBX4CU66P26JZDTOQZLYGMGA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.684178459 (6h21m41.273523446s) [noise=true , address=104.36.82.9:4039 , piece_id=PRBG5EVKAZ3LGDRX4MCI57T4RSNFVKZVUNOAIRHBM6SNS5XXUTPQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.691448508 (6h21m41.280793485s) [noise=true , address=104.36.82.22:4525 , piece_id=NKD4EXV774KNXLEHICE4MNKOQS5QKPIUTJL2PPCNYUMYPMQEF75A, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.717147267 (6h21m41.306492254s) [noise=true , address=104.36.82.59:4313 , piece_id=KDH6AVALUMDXB6USO7WG77DVOSRBTGSZA6VTJFFQJR7QFC2PE35A, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.719612885 (6h21m41.308957872s) [noise=true , address=104.36.82.45:4912 , piece_id=B6HTS6ORSGMUKZO5G2ALW4LVUSIBTCEHPX2YJTYPPCH3XROIO3HA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.733973160 (6h21m41.323318147s) [noise=true , address=104.36.82.60:4369 , piece_id=Y6SYJOB7SYBJX7STC25OJJPD4FSK5TZ5KXI7WN7QS42UNTOF5BCQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.737980718 (6h21m41.327325705s) [noise=true , address=104.36.82.5:4845 , piece_id=JKN4VFS7ET2LVGHGE4LIBPCGG33GNY4SGSOXAWB2MXFYPJVO4FCQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.745434729 (6h21m41.334779717s) [noise=true , address=104.36.82.42:4820 , piece_id=XJIHI2E35NJKDFYFTW4V5EIF3CL7GDJ5EKVCTEV2KL26X43A3CYA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.754624732 (6h21m41.34396972s) [noise=true , address=104.36.82.8:4992 , piece_id=226HGROQ3DBKTJB3U6TG2OF4JQRYAGTLMSIL2JIL4NY5PISP4RUQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.763605976 (6h21m41.352950963s) [noise=true , address=104.36.82.60:4345 , piece_id=VPYOQHGEP6E257WI7IGIWXRO5WOLCNGDJYNPFPC56WUMOYDUS6RA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.763623339 (6h21m41.352968336s) [seg_pos={0 0}, upload=6610]: Segment reached optimal threshold of 14 pieces.
Aug 7 16:01:41.763741525 (6h21m41.353086512s) [noise=true , address=104.36.82.10:4110 , piece_id=WAUEFOZXH3SULGTXTWXWLEPYHZJWJDDQRRWWXRBXVWF7WCWPYGYQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ADbnPJGRTw2NqRzhy7ptojGYoNdJgqmjuWBSjcrQK53Wm7LXi): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763803123 (6h21m41.35314811s) [noise=true , address=104.36.82.61:4387 , piece_id=MAWGNSJTYYH7EVPDIXQEG6KP3MKACSMWYVMMLYUASUDJBIX5POOA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P4wAXSyaRY9poPRK5q4td8DJM69xWioG38TTwLTHKHA7QFxLp): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763857136 (6h21m41.353202113s) [noise=true , address=45.91.116.9:4267 , piece_id=MWAYSXFIIYQGDPTLWZSAHZP3UO72EXYDSSULHBJJU54H5OWG36NA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6M5TgGqUwjXyM25uqv5hsh67mS81ByzMM4kWFUNaFvisDN9d): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763904968 (6h21m41.353249955s) [noise=true , address=104.36.82.23:4455 , piece_id=UJOJ7P3VET4XIM257B3SHAN6SJG2H64QJHV6PYU7OEB7PPBZVDAA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wsyjF5X3iPrJsezUjakUArxc8K88T36yT4rE3yu6q9BMKfycm): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763945405 (6h21m41.353290392s) [seg_pos={0 0}, upload=6610]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 16:01:41.764033824 (6h21m41.353378811s) [upload=6610]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 16:01:41.764038263 (6h21m41.35338325s) [upload=6610]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 16:01:42.043459384 (6h21m41.632804371s) [upload=6610]: Done uploading segments...
Aug 7 16:01:42.043475946 (6h21m41.632820933s) [upload=6610]: Upload finished. err: <nil>
Aug 7 16:02:11.915996790 (6h22m11.505341777s) [upload=6619]: Starting upload
Aug 7 16:02:11.916045183 (6h22m11.50539017s) [upload=6619]: Done starting upload
Aug 7 16:02:11.916152489 (6h22m11.505497476s) [upload=6619]: Uploading segments...
Aug 7 16:02:11.916287706 (6h22m11.505632693s) [upload=6619]: Got next segment. Inline: false
Aug 7 16:02:11.916319939 (6h22m11.505664926s) [upload=6619]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 16:02:11.916324708 (6h22m11.505669695s) [upload=6619]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 16:02:12.129590885 (6h22m11.718935892s) [upload=6619]: Next segment err: metaclient: Too Many Requests
Aug 7 16:02:12.129623196 (6h22m11.718968194s) [upload=6619]: Done uploading segments...
Aug 7 16:02:12.129640310 (6h22m11.718985307s) [upload=6619]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.095192186s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
2.137681055s
FAIL (2025-08-07 16:15:25 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== entMakeInline Aug 7 15:57:10.271829415 (6h17m9.861174403s) [upload=6533]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 15:57:10.587098885 (6h17m10.176443872s) [upload=6533]: Done uploading segments... Aug 7 15:57:10.587115857 (6h17m10.176460835s) [upload=6533]: Upload finished. err: <nil> Aug 7 15:57:48.918901442 (6h17m48.508246429s) [upload=6545]: Starting upload Aug 7 15:57:48.918948701 (6h17m48.508293688s) [upload=6545]: Done starting upload Aug 7 15:57:48.919013486 (6h17m48.508358473s) [upload=6545]: Uploading segments... Aug 7 15:57:48.919055005 (6h17m48.508399982s) [upload=6545]: Got next segment. Inline: true Aug 7 15:57:48.919059032 (6h17m48.508404009s) [upload=6545]: Waiting for error group managing segments... Aug 7 15:57:48.919091585 (6h17m48.508436572s) [upload=6545]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 15:57:48.919095493 (6h17m48.50844048s) [upload=6545]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 15:57:48.919097998 (6h17m48.508442975s) [upload=6545]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 15:57:49.200378977 (6h17m48.789723964s) [upload=6545]: Done uploading segments... Aug 7 15:57:49.200397332 (6h17m48.789742319s) [upload=6545]: Upload finished. err: <nil> Aug 7 15:58:50.898935036 (6h18m50.488280023s) [upload=6562]: Starting upload Aug 7 15:58:50.898979992 (6h18m50.488324979s) [upload=6562]: Done starting upload Aug 7 15:58:50.899026682 (6h18m50.488371669s) [upload=6562]: Uploading segments... Aug 7 15:58:50.899065355 (6h18m50.488410343s) [upload=6562]: Got next segment. Inline: true Aug 7 15:58:50.899070335 (6h18m50.488415322s) [upload=6562]: Waiting for error group managing segments... Aug 7 15:58:50.899108268 (6h18m50.488453255s) [upload=6562]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 15:58:50.899112326 (6h18m50.488457313s) [upload=6562]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 15:58:50.899114820 (6h18m50.488459807s) [upload=6562]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 15:58:51.203289783 (6h18m50.79263478s) [upload=6562]: Done uploading segments... Aug 7 15:58:51.203312587 (6h18m50.792657624s) [upload=6562]: Upload finished. err: <nil> Aug 7 16:00:00.470970125 (6h20m0.060315112s) [upload=6580]: Starting upload Aug 7 16:00:00.471017334 (6h20m0.060362321s) [upload=6580]: Done starting upload Aug 7 16:00:00.471072199 (6h20m0.060417186s) [upload=6580]: Uploading segments... Aug 7 16:00:00.471112327 (6h20m0.060457314s) [upload=6580]: Got next segment. Inline: true Aug 7 16:00:00.471116364 (6h20m0.060461351s) [upload=6580]: Waiting for error group managing segments... Aug 7 16:00:00.471149427 (6h20m0.060494414s) [upload=6580]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:00:00.471153305 (6h20m0.060498292s) [upload=6580]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:00:00.471155819 (6h20m0.060500806s) [upload=6580]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:00:00.739624026 (6h20m0.328969023s) [upload=6580]: Done uploading segments... Aug 7 16:00:00.739645636 (6h20m0.328990623s) [upload=6580]: Upload finished. err: <nil> Aug 7 16:00:49.882359165 (6h20m49.471704152s) [upload=6598]: Starting upload Aug 7 16:00:49.882404682 (6h20m49.471749659s) [upload=6598]: Done starting upload Aug 7 16:00:49.882456812 (6h20m49.471801799s) [upload=6598]: Uploading segments... Aug 7 16:00:49.882489865 (6h20m49.471834852s) [upload=6598]: Got next segment. Inline: true Aug 7 16:00:49.882493702 (6h20m49.47183869s) [upload=6598]: Waiting for error group managing segments... Aug 7 16:00:49.882525894 (6h20m49.471870881s) [upload=6598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:00:49.882529422 (6h20m49.471874399s) [upload=6598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:00:49.882531676 (6h20m49.471876663s) [upload=6598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:00:50.213720013 (6h20m49.803065s) [upload=6598]: Done uploading segments... Aug 7 16:00:50.213739550 (6h20m49.803084537s) [upload=6598]: Upload finished. err: <nil> Aug 7 16:01:51.762778733 (6h21m51.35212372s) [upload=6617]: Starting upload Aug 7 16:01:51.762856161 (6h21m51.352201148s) [upload=6617]: Done starting upload Aug 7 16:01:51.762914883 (6h21m51.35225987s) [upload=6617]: Uploading segments... Aug 7 16:01:51.762952445 (6h21m51.352297432s) [upload=6617]: Got next segment. Inline: true Aug 7 16:01:51.762957344 (6h21m51.352302331s) [upload=6617]: Waiting for error group managing segments... Aug 7 16:01:51.762994485 (6h21m51.352339473s) [upload=6617]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:01:51.762998313 (6h21m51.3523433s) [upload=6617]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:01:51.763001278 (6h21m51.352346265s) [upload=6617]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:01:52.034377407 (6h21m51.623722394s) [upload=6617]: Done uploading segments... Aug 7 16:01:52.034395451 (6h21m51.623740439s) [upload=6617]: Upload finished. err: <nil> Aug 7 16:02:11.154406473 (6h22m10.74375146s) [upload=6618]: Starting upload Aug 7 16:02:11.154456328 (6h22m10.743801315s) [upload=6618]: Done starting upload Aug 7 16:02:11.154503859 (6h22m10.743848846s) [upload=6618]: Uploading segments... Aug 7 16:02:11.154545308 (6h22m10.743890295s) [upload=6618]: Got next segment. Inline: true Aug 7 16:02:11.154548574 (6h22m10.743893561s) [upload=6618]: Waiting for error group managing segments... Aug 7 16:02:11.154580566 (6h22m10.743925553s) [upload=6618]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:02:11.154584283 (6h22m10.74392927s) [upload=6618]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:02:11.154586828 (6h22m10.743931815s) [upload=6618]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:02:11.475329609 (6h22m11.064674596s) [upload=6618]: Done uploading segments... Aug 7 16:02:11.475484125 (6h22m11.064829112s) [upload=6618]: Upload finished. err: <nil> Aug 7 16:02:42.941029250 (6h22m42.530374238s) [upload=6628]: Starting upload Aug 7 16:02:42.941096740 (6h22m42.530441727s) [upload=6628]: Done starting upload Aug 7 16:02:42.941158728 (6h22m42.530503716s) [upload=6628]: Uploading segments... Aug 7 16:02:42.941210478 (6h22m42.530555465s) [upload=6628]: Got next segment. Inline: true Aug 7 16:02:42.941216419 (6h22m42.530561416s) [upload=6628]: Waiting for error group managing segments... Aug 7 16:02:42.941281664 (6h22m42.530626651s) [upload=6628]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:02:42.941286824 (6h22m42.530631811s) [upload=6628]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:02:42.941289880 (6h22m42.530634867s) [upload=6628]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:02:43.233814342 (6h22m42.823159339s) [upload=6628]: Done uploading segments... Aug 7 16:02:43.233838148 (6h22m42.823183145s) [upload=6628]: Upload finished. err: <nil> Aug 7 16:03:45.201690833 (6h23m44.79103582s) [upload=6651]: Starting upload Aug 7 16:03:45.201738173 (6h23m44.79108316s) [upload=6651]: Done starting upload Aug 7 16:03:45.201785534 (6h23m44.791130511s) [upload=6651]: Uploading segments... Aug 7 16:03:45.201855618 (6h23m44.791200605s) [upload=6651]: Got next segment. Inline: true Aug 7 16:03:45.201859616 (6h23m44.791204603s) [upload=6651]: Waiting for error group managing segments... Aug 7 16:03:45.201894122 (6h23m44.791239109s) [upload=6651]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:03:45.201897799 (6h23m44.791242786s) [upload=6651]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:03:45.201900023 (6h23m44.79124501s) [upload=6651]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:03:45.644503676 (6h23m45.233848664s) [upload=6651]: Done uploading segments... Aug 7 16:03:45.644521010 (6h23m45.233865987s) [upload=6651]: Upload finished. err: <nil> Aug 7 16:04:31.274936212 (6h24m30.864281199s) [upload=6663]: Starting upload Aug 7 16:04:31.274977170 (6h24m30.864322157s) [upload=6663]: Done starting upload Aug 7 16:04:31.275026655 (6h24m30.864371642s) [upload=6663]: Uploading segments... Aug 7 16:04:31.275054187 (6h24m30.864399174s) [upload=6663]: Got next segment. Inline: true Aug 7 16:04:31.275057754 (6h24m30.864402731s) [upload=6663]: Waiting for error group managing segments... Aug 7 16:04:31.275091278 (6h24m30.864436265s) [upload=6663]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:04:31.275095255 (6h24m30.864440242s) [upload=6663]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:04:31.275097651 (6h24m30.864442638s) [upload=6663]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:04:31.554258512 (6h24m31.143603499s) [upload=6663]: Done uploading segments... Aug 7 16:04:31.554277108 (6h24m31.143622095s) [upload=6663]: Upload finished. err: <nil> Aug 7 16:05:14.730035713 (6h25m14.3193807s) [upload=6674]: Starting upload Aug 7 16:05:14.730082813 (6h25m14.3194278s) [upload=6674]: Done starting upload Aug 7 16:05:14.730159079 (6h25m14.319504056s) [upload=6674]: Uploading segments... Aug 7 16:05:14.730249532 (6h25m14.319594519s) [upload=6674]: Got next segment. Inline: true Aug 7 16:05:14.730254050 (6h25m14.319599037s) [upload=6674]: Waiting for error group managing segments... Aug 7 16:05:14.730294949 (6h25m14.319639936s) [upload=6674]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:05:14.730299728 (6h25m14.319644715s) [upload=6674]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:05:14.730302243 (6h25m14.31964723s) [upload=6674]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:05:14.942773689 (6h25m14.532118666s) [upload=6674]: Done uploading segments... Aug 7 16:05:14.942790141 (6h25m14.532135118s) [upload=6674]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.13770997s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
1.840390415s
FAIL (2025-08-07 16:14:51 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== entMakeInline Aug 7 15:57:10.271829415 (6h17m9.861174403s) [upload=6533]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 15:57:10.587098885 (6h17m10.176443872s) [upload=6533]: Done uploading segments... Aug 7 15:57:10.587115857 (6h17m10.176460835s) [upload=6533]: Upload finished. err: <nil> Aug 7 15:57:48.918901442 (6h17m48.508246429s) [upload=6545]: Starting upload Aug 7 15:57:48.918948701 (6h17m48.508293688s) [upload=6545]: Done starting upload Aug 7 15:57:48.919013486 (6h17m48.508358473s) [upload=6545]: Uploading segments... Aug 7 15:57:48.919055005 (6h17m48.508399982s) [upload=6545]: Got next segment. Inline: true Aug 7 15:57:48.919059032 (6h17m48.508404009s) [upload=6545]: Waiting for error group managing segments... Aug 7 15:57:48.919091585 (6h17m48.508436572s) [upload=6545]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 15:57:48.919095493 (6h17m48.50844048s) [upload=6545]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 15:57:48.919097998 (6h17m48.508442975s) [upload=6545]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 15:57:49.200378977 (6h17m48.789723964s) [upload=6545]: Done uploading segments... Aug 7 15:57:49.200397332 (6h17m48.789742319s) [upload=6545]: Upload finished. err: <nil> Aug 7 15:58:50.898935036 (6h18m50.488280023s) [upload=6562]: Starting upload Aug 7 15:58:50.898979992 (6h18m50.488324979s) [upload=6562]: Done starting upload Aug 7 15:58:50.899026682 (6h18m50.488371669s) [upload=6562]: Uploading segments... Aug 7 15:58:50.899065355 (6h18m50.488410343s) [upload=6562]: Got next segment. Inline: true Aug 7 15:58:50.899070335 (6h18m50.488415322s) [upload=6562]: Waiting for error group managing segments... Aug 7 15:58:50.899108268 (6h18m50.488453255s) [upload=6562]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 15:58:50.899112326 (6h18m50.488457313s) [upload=6562]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 15:58:50.899114820 (6h18m50.488459807s) [upload=6562]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 15:58:51.203289783 (6h18m50.79263478s) [upload=6562]: Done uploading segments... Aug 7 15:58:51.203312587 (6h18m50.792657624s) [upload=6562]: Upload finished. err: <nil> Aug 7 16:00:00.470970125 (6h20m0.060315112s) [upload=6580]: Starting upload Aug 7 16:00:00.471017334 (6h20m0.060362321s) [upload=6580]: Done starting upload Aug 7 16:00:00.471072199 (6h20m0.060417186s) [upload=6580]: Uploading segments... Aug 7 16:00:00.471112327 (6h20m0.060457314s) [upload=6580]: Got next segment. Inline: true Aug 7 16:00:00.471116364 (6h20m0.060461351s) [upload=6580]: Waiting for error group managing segments... Aug 7 16:00:00.471149427 (6h20m0.060494414s) [upload=6580]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:00:00.471153305 (6h20m0.060498292s) [upload=6580]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:00:00.471155819 (6h20m0.060500806s) [upload=6580]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:00:00.739624026 (6h20m0.328969023s) [upload=6580]: Done uploading segments... Aug 7 16:00:00.739645636 (6h20m0.328990623s) [upload=6580]: Upload finished. err: <nil> Aug 7 16:00:49.882359165 (6h20m49.471704152s) [upload=6598]: Starting upload Aug 7 16:00:49.882404682 (6h20m49.471749659s) [upload=6598]: Done starting upload Aug 7 16:00:49.882456812 (6h20m49.471801799s) [upload=6598]: Uploading segments... Aug 7 16:00:49.882489865 (6h20m49.471834852s) [upload=6598]: Got next segment. Inline: true Aug 7 16:00:49.882493702 (6h20m49.47183869s) [upload=6598]: Waiting for error group managing segments... Aug 7 16:00:49.882525894 (6h20m49.471870881s) [upload=6598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:00:49.882529422 (6h20m49.471874399s) [upload=6598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:00:49.882531676 (6h20m49.471876663s) [upload=6598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:00:50.213720013 (6h20m49.803065s) [upload=6598]: Done uploading segments... Aug 7 16:00:50.213739550 (6h20m49.803084537s) [upload=6598]: Upload finished. err: <nil> Aug 7 16:01:51.762778733 (6h21m51.35212372s) [upload=6617]: Starting upload Aug 7 16:01:51.762856161 (6h21m51.352201148s) [upload=6617]: Done starting upload Aug 7 16:01:51.762914883 (6h21m51.35225987s) [upload=6617]: Uploading segments... Aug 7 16:01:51.762952445 (6h21m51.352297432s) [upload=6617]: Got next segment. Inline: true Aug 7 16:01:51.762957344 (6h21m51.352302331s) [upload=6617]: Waiting for error group managing segments... Aug 7 16:01:51.762994485 (6h21m51.352339473s) [upload=6617]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:01:51.762998313 (6h21m51.3523433s) [upload=6617]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:01:51.763001278 (6h21m51.352346265s) [upload=6617]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:01:52.034377407 (6h21m51.623722394s) [upload=6617]: Done uploading segments... Aug 7 16:01:52.034395451 (6h21m51.623740439s) [upload=6617]: Upload finished. err: <nil> Aug 7 16:02:11.154406473 (6h22m10.74375146s) [upload=6618]: Starting upload Aug 7 16:02:11.154456328 (6h22m10.743801315s) [upload=6618]: Done starting upload Aug 7 16:02:11.154503859 (6h22m10.743848846s) [upload=6618]: Uploading segments... Aug 7 16:02:11.154545308 (6h22m10.743890295s) [upload=6618]: Got next segment. Inline: true Aug 7 16:02:11.154548574 (6h22m10.743893561s) [upload=6618]: Waiting for error group managing segments... Aug 7 16:02:11.154580566 (6h22m10.743925553s) [upload=6618]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:02:11.154584283 (6h22m10.74392927s) [upload=6618]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:02:11.154586828 (6h22m10.743931815s) [upload=6618]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:02:11.475329609 (6h22m11.064674596s) [upload=6618]: Done uploading segments... Aug 7 16:02:11.475484125 (6h22m11.064829112s) [upload=6618]: Upload finished. err: <nil> Aug 7 16:02:42.941029250 (6h22m42.530374238s) [upload=6628]: Starting upload Aug 7 16:02:42.941096740 (6h22m42.530441727s) [upload=6628]: Done starting upload Aug 7 16:02:42.941158728 (6h22m42.530503716s) [upload=6628]: Uploading segments... Aug 7 16:02:42.941210478 (6h22m42.530555465s) [upload=6628]: Got next segment. Inline: true Aug 7 16:02:42.941216419 (6h22m42.530561416s) [upload=6628]: Waiting for error group managing segments... Aug 7 16:02:42.941281664 (6h22m42.530626651s) [upload=6628]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:02:42.941286824 (6h22m42.530631811s) [upload=6628]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:02:42.941289880 (6h22m42.530634867s) [upload=6628]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:02:43.233814342 (6h22m42.823159339s) [upload=6628]: Done uploading segments... Aug 7 16:02:43.233838148 (6h22m42.823183145s) [upload=6628]: Upload finished. err: <nil> Aug 7 16:03:45.201690833 (6h23m44.79103582s) [upload=6651]: Starting upload Aug 7 16:03:45.201738173 (6h23m44.79108316s) [upload=6651]: Done starting upload Aug 7 16:03:45.201785534 (6h23m44.791130511s) [upload=6651]: Uploading segments... Aug 7 16:03:45.201855618 (6h23m44.791200605s) [upload=6651]: Got next segment. Inline: true Aug 7 16:03:45.201859616 (6h23m44.791204603s) [upload=6651]: Waiting for error group managing segments... Aug 7 16:03:45.201894122 (6h23m44.791239109s) [upload=6651]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:03:45.201897799 (6h23m44.791242786s) [upload=6651]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:03:45.201900023 (6h23m44.79124501s) [upload=6651]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:03:45.644503676 (6h23m45.233848664s) [upload=6651]: Done uploading segments... Aug 7 16:03:45.644521010 (6h23m45.233865987s) [upload=6651]: Upload finished. err: <nil> Aug 7 16:04:31.274936212 (6h24m30.864281199s) [upload=6663]: Starting upload Aug 7 16:04:31.274977170 (6h24m30.864322157s) [upload=6663]: Done starting upload Aug 7 16:04:31.275026655 (6h24m30.864371642s) [upload=6663]: Uploading segments... Aug 7 16:04:31.275054187 (6h24m30.864399174s) [upload=6663]: Got next segment. Inline: true Aug 7 16:04:31.275057754 (6h24m30.864402731s) [upload=6663]: Waiting for error group managing segments... Aug 7 16:04:31.275091278 (6h24m30.864436265s) [upload=6663]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:04:31.275095255 (6h24m30.864440242s) [upload=6663]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:04:31.275097651 (6h24m30.864442638s) [upload=6663]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:04:31.554258512 (6h24m31.143603499s) [upload=6663]: Done uploading segments... Aug 7 16:04:31.554277108 (6h24m31.143622095s) [upload=6663]: Upload finished. err: <nil> Aug 7 16:05:14.730035713 (6h25m14.3193807s) [upload=6674]: Starting upload Aug 7 16:05:14.730082813 (6h25m14.3194278s) [upload=6674]: Done starting upload Aug 7 16:05:14.730159079 (6h25m14.319504056s) [upload=6674]: Uploading segments... Aug 7 16:05:14.730249532 (6h25m14.319594519s) [upload=6674]: Got next segment. Inline: true Aug 7 16:05:14.730254050 (6h25m14.319599037s) [upload=6674]: Waiting for error group managing segments... Aug 7 16:05:14.730294949 (6h25m14.319639936s) [upload=6674]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:05:14.730299728 (6h25m14.319644715s) [upload=6674]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:05:14.730302243 (6h25m14.31964723s) [upload=6674]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:05:14.942773689 (6h25m14.532118666s) [upload=6674]: Done uploading segments... Aug 7 16:05:14.942790141 (6h25m14.532135118s) [upload=6674]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.840432045s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [.................................X.......X..................] duration:
5.969068875s
FAIL (2025-08-07 16:14:33 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
e begin upload segment.
Aug 7 16:01:39.748945470 (6h21m39.338290447s) [noise=true , address=104.36.82.42:4820 , piece_id=XJIHI2E35NJKDFYFTW4V5EIF3CL7GDJ5EKVCTEV2KL26X43A3CYA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.748969657 (6h21m39.338314644s) [noise=true , address=104.36.82.60:4345 , piece_id=VPYOQHGEP6E257WI7IGIWXRO5WOLCNGDJYNPFPC56WUMOYDUS6RA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749368329 (6h21m39.338713316s) [noise=true , address=45.91.116.9:4267 , piece_id=MWAYSXFIIYQGDPTLWZSAHZP3UO72EXYDSSULHBJJU54H5OWG36NA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749375964 (6h21m39.338720951s) [noise=true , address=104.36.82.10:4110 , piece_id=WAUEFOZXH3SULGTXTWXWLEPYHZJWJDDQRRWWXRBXVWF7WCWPYGYQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749660398 (6h21m39.339005385s) [noise=true , address=79.127.205.235:4666 , piece_id=WA6GZKNO2UUAC5VKBNOZ7UO25RQR7QDT5HVNQYZDIWJEB656WMGA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749701216 (6h21m39.339046203s) [noise=true , address=79.127.201.215:4345 , piece_id=MAAFKAPO4OCQXU5WJ6DAP72KEHIM4AO5JJFJ4ZEFNWKS5M4OV43Q, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749938690 (6h21m39.339283677s) [noise=true , address=104.36.82.59:4313 , piece_id=KDH6AVALUMDXB6USO7WG77DVOSRBTGSZA6VTJFFQJR7QFC2PE35A, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750024905 (6h21m39.339369892s) [noise=true , address=104.36.82.23:4455 , piece_id=UJOJ7P3VET4XIM257B3SHAN6SJG2H64QJHV6PYU7OEB7PPBZVDAA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750215089 (6h21m39.339560076s) [noise=true , address=104.36.82.61:4387 , piece_id=MAWGNSJTYYH7EVPDIXQEG6KP3MKACSMWYVMMLYUASUDJBIX5POOA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750282639 (6h21m39.339627626s) [noise=true , address=45.91.116.10:4092 , piece_id=BUVKJC3K3RKZZTW2CQPEQM7ECQOLIBX4CU66P26JZDTOQZLYGMGA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750481038 (6h21m39.339826025s) [noise=true , address=79.127.220.97:4396 , piece_id=X2CCRNYWY5ZQOAODHNJE7VLYQIHD37PEE7MQXI5WIYV36ICONILQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750533900 (6h21m39.339878887s) [noise=true , address=104.36.82.60:4369 , piece_id=Y6SYJOB7SYBJX7STC25OJJPD4FSK5TZ5KXI7WN7QS42UNTOF5BCQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750725967 (6h21m39.340070954s) [noise=true , address=104.36.82.9:4039 , piece_id=PRBG5EVKAZ3LGDRX4MCI57T4RSNFVKZVUNOAIRHBM6SNS5XXUTPQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750795189 (6h21m39.340140176s) [noise=true , address=104.36.82.22:4525 , piece_id=NKD4EXV774KNXLEHICE4MNKOQS5QKPIUTJL2PPCNYUMYPMQEF75A, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750954023 (6h21m39.34029901s) [noise=true , address=104.36.82.45:4912 , piece_id=B6HTS6ORSGMUKZO5G2ALW4LVUSIBTCEHPX2YJTYPPCH3XROIO3HA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.751098079 (6h21m39.340443066s) [noise=true , address=104.36.82.8:4992 , piece_id=226HGROQ3DBKTJB3U6TG2OF4JQRYAGTLMSIL2JIL4NY5PISP4RUQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.751175397 (6h21m39.340520384s) [noise=true , address=79.127.205.233:4631 , piece_id=77WNKENAMS4Q75KYK4F4VNRP4DD4MKGEJIU5K7WZ3EHVQLEW6Q5A, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.751392602 (6h21m39.340737579s) [noise=true , address=104.36.82.5:4845 , piece_id=JKN4VFS7ET2LVGHGE4LIBPCGG33GNY4SGSOXAWB2MXFYPJVO4FCQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:40.878402060 (6h21m40.467747047s) [upload=6610]: Next returned nil segment
Aug 7 16:01:40.878416708 (6h21m40.467761695s) [upload=6610]: Waiting for error group managing segments...
Aug 7 16:01:41.195597254 (6h21m40.784942241s) [noise=true , address=79.127.205.235:4666 , piece_id=WA6GZKNO2UUAC5VKBNOZ7UO25RQR7QDT5HVNQYZDIWJEB656WMGA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.261081133 (6h21m40.85042612s) [noise=true , address=79.127.205.233:4631 , piece_id=77WNKENAMS4Q75KYK4F4VNRP4DD4MKGEJIU5K7WZ3EHVQLEW6Q5A, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.420961069 (6h21m41.010306046s) [noise=true , address=79.127.220.97:4396 , piece_id=X2CCRNYWY5ZQOAODHNJE7VLYQIHD37PEE7MQXI5WIYV36ICONILQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.638341479 (6h21m41.227686486s) [noise=true , address=79.127.201.215:4345 , piece_id=MAAFKAPO4OCQXU5WJ6DAP72KEHIM4AO5JJFJ4ZEFNWKS5M4OV43Q, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.658966771 (6h21m41.248311758s) [noise=true , address=45.91.116.10:4092 , piece_id=BUVKJC3K3RKZZTW2CQPEQM7ECQOLIBX4CU66P26JZDTOQZLYGMGA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.684178459 (6h21m41.273523446s) [noise=true , address=104.36.82.9:4039 , piece_id=PRBG5EVKAZ3LGDRX4MCI57T4RSNFVKZVUNOAIRHBM6SNS5XXUTPQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.691448508 (6h21m41.280793485s) [noise=true , address=104.36.82.22:4525 , piece_id=NKD4EXV774KNXLEHICE4MNKOQS5QKPIUTJL2PPCNYUMYPMQEF75A, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.717147267 (6h21m41.306492254s) [noise=true , address=104.36.82.59:4313 , piece_id=KDH6AVALUMDXB6USO7WG77DVOSRBTGSZA6VTJFFQJR7QFC2PE35A, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.719612885 (6h21m41.308957872s) [noise=true , address=104.36.82.45:4912 , piece_id=B6HTS6ORSGMUKZO5G2ALW4LVUSIBTCEHPX2YJTYPPCH3XROIO3HA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.733973160 (6h21m41.323318147s) [noise=true , address=104.36.82.60:4369 , piece_id=Y6SYJOB7SYBJX7STC25OJJPD4FSK5TZ5KXI7WN7QS42UNTOF5BCQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.737980718 (6h21m41.327325705s) [noise=true , address=104.36.82.5:4845 , piece_id=JKN4VFS7ET2LVGHGE4LIBPCGG33GNY4SGSOXAWB2MXFYPJVO4FCQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.745434729 (6h21m41.334779717s) [noise=true , address=104.36.82.42:4820 , piece_id=XJIHI2E35NJKDFYFTW4V5EIF3CL7GDJ5EKVCTEV2KL26X43A3CYA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.754624732 (6h21m41.34396972s) [noise=true , address=104.36.82.8:4992 , piece_id=226HGROQ3DBKTJB3U6TG2OF4JQRYAGTLMSIL2JIL4NY5PISP4RUQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.763605976 (6h21m41.352950963s) [noise=true , address=104.36.82.60:4345 , piece_id=VPYOQHGEP6E257WI7IGIWXRO5WOLCNGDJYNPFPC56WUMOYDUS6RA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.763623339 (6h21m41.352968336s) [seg_pos={0 0}, upload=6610]: Segment reached optimal threshold of 14 pieces.
Aug 7 16:01:41.763741525 (6h21m41.353086512s) [noise=true , address=104.36.82.10:4110 , piece_id=WAUEFOZXH3SULGTXTWXWLEPYHZJWJDDQRRWWXRBXVWF7WCWPYGYQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ADbnPJGRTw2NqRzhy7ptojGYoNdJgqmjuWBSjcrQK53Wm7LXi): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763803123 (6h21m41.35314811s) [noise=true , address=104.36.82.61:4387 , piece_id=MAWGNSJTYYH7EVPDIXQEG6KP3MKACSMWYVMMLYUASUDJBIX5POOA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P4wAXSyaRY9poPRK5q4td8DJM69xWioG38TTwLTHKHA7QFxLp): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763857136 (6h21m41.353202113s) [noise=true , address=45.91.116.9:4267 , piece_id=MWAYSXFIIYQGDPTLWZSAHZP3UO72EXYDSSULHBJJU54H5OWG36NA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6M5TgGqUwjXyM25uqv5hsh67mS81ByzMM4kWFUNaFvisDN9d): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763904968 (6h21m41.353249955s) [noise=true , address=104.36.82.23:4455 , piece_id=UJOJ7P3VET4XIM257B3SHAN6SJG2H64QJHV6PYU7OEB7PPBZVDAA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wsyjF5X3iPrJsezUjakUArxc8K88T36yT4rE3yu6q9BMKfycm): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763945405 (6h21m41.353290392s) [seg_pos={0 0}, upload=6610]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 16:01:41.764033824 (6h21m41.353378811s) [upload=6610]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 16:01:41.764038263 (6h21m41.35338325s) [upload=6610]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 16:01:42.043459384 (6h21m41.632804371s) [upload=6610]: Done uploading segments...
Aug 7 16:01:42.043475946 (6h21m41.632820933s) [upload=6610]: Upload finished. err: <nil>
Aug 7 16:02:11.915996790 (6h22m11.505341777s) [upload=6619]: Starting upload
Aug 7 16:02:11.916045183 (6h22m11.50539017s) [upload=6619]: Done starting upload
Aug 7 16:02:11.916152489 (6h22m11.505497476s) [upload=6619]: Uploading segments...
Aug 7 16:02:11.916287706 (6h22m11.505632693s) [upload=6619]: Got next segment. Inline: false
Aug 7 16:02:11.916319939 (6h22m11.505664926s) [upload=6619]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 16:02:11.916324708 (6h22m11.505669695s) [upload=6619]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 16:02:12.129590885 (6h22m11.718935892s) [upload=6619]: Next segment err: metaclient: Too Many Requests
Aug 7 16:02:12.129623196 (6h22m11.718968194s) [upload=6619]: Done uploading segments...
Aug 7 16:02:12.129640310 (6h22m11.718985307s) [upload=6619]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.96909244s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [..................................X.......X.................] duration:
5.907857221s
FAIL (2025-08-07 16:13:42 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
e begin upload segment.
Aug 7 16:01:39.748945470 (6h21m39.338290447s) [noise=true , address=104.36.82.42:4820 , piece_id=XJIHI2E35NJKDFYFTW4V5EIF3CL7GDJ5EKVCTEV2KL26X43A3CYA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.748969657 (6h21m39.338314644s) [noise=true , address=104.36.82.60:4345 , piece_id=VPYOQHGEP6E257WI7IGIWXRO5WOLCNGDJYNPFPC56WUMOYDUS6RA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749368329 (6h21m39.338713316s) [noise=true , address=45.91.116.9:4267 , piece_id=MWAYSXFIIYQGDPTLWZSAHZP3UO72EXYDSSULHBJJU54H5OWG36NA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749375964 (6h21m39.338720951s) [noise=true , address=104.36.82.10:4110 , piece_id=WAUEFOZXH3SULGTXTWXWLEPYHZJWJDDQRRWWXRBXVWF7WCWPYGYQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749660398 (6h21m39.339005385s) [noise=true , address=79.127.205.235:4666 , piece_id=WA6GZKNO2UUAC5VKBNOZ7UO25RQR7QDT5HVNQYZDIWJEB656WMGA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749701216 (6h21m39.339046203s) [noise=true , address=79.127.201.215:4345 , piece_id=MAAFKAPO4OCQXU5WJ6DAP72KEHIM4AO5JJFJ4ZEFNWKS5M4OV43Q, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749938690 (6h21m39.339283677s) [noise=true , address=104.36.82.59:4313 , piece_id=KDH6AVALUMDXB6USO7WG77DVOSRBTGSZA6VTJFFQJR7QFC2PE35A, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750024905 (6h21m39.339369892s) [noise=true , address=104.36.82.23:4455 , piece_id=UJOJ7P3VET4XIM257B3SHAN6SJG2H64QJHV6PYU7OEB7PPBZVDAA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750215089 (6h21m39.339560076s) [noise=true , address=104.36.82.61:4387 , piece_id=MAWGNSJTYYH7EVPDIXQEG6KP3MKACSMWYVMMLYUASUDJBIX5POOA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750282639 (6h21m39.339627626s) [noise=true , address=45.91.116.10:4092 , piece_id=BUVKJC3K3RKZZTW2CQPEQM7ECQOLIBX4CU66P26JZDTOQZLYGMGA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750481038 (6h21m39.339826025s) [noise=true , address=79.127.220.97:4396 , piece_id=X2CCRNYWY5ZQOAODHNJE7VLYQIHD37PEE7MQXI5WIYV36ICONILQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750533900 (6h21m39.339878887s) [noise=true , address=104.36.82.60:4369 , piece_id=Y6SYJOB7SYBJX7STC25OJJPD4FSK5TZ5KXI7WN7QS42UNTOF5BCQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750725967 (6h21m39.340070954s) [noise=true , address=104.36.82.9:4039 , piece_id=PRBG5EVKAZ3LGDRX4MCI57T4RSNFVKZVUNOAIRHBM6SNS5XXUTPQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750795189 (6h21m39.340140176s) [noise=true , address=104.36.82.22:4525 , piece_id=NKD4EXV774KNXLEHICE4MNKOQS5QKPIUTJL2PPCNYUMYPMQEF75A, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750954023 (6h21m39.34029901s) [noise=true , address=104.36.82.45:4912 , piece_id=B6HTS6ORSGMUKZO5G2ALW4LVUSIBTCEHPX2YJTYPPCH3XROIO3HA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.751098079 (6h21m39.340443066s) [noise=true , address=104.36.82.8:4992 , piece_id=226HGROQ3DBKTJB3U6TG2OF4JQRYAGTLMSIL2JIL4NY5PISP4RUQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.751175397 (6h21m39.340520384s) [noise=true , address=79.127.205.233:4631 , piece_id=77WNKENAMS4Q75KYK4F4VNRP4DD4MKGEJIU5K7WZ3EHVQLEW6Q5A, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.751392602 (6h21m39.340737579s) [noise=true , address=104.36.82.5:4845 , piece_id=JKN4VFS7ET2LVGHGE4LIBPCGG33GNY4SGSOXAWB2MXFYPJVO4FCQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:40.878402060 (6h21m40.467747047s) [upload=6610]: Next returned nil segment
Aug 7 16:01:40.878416708 (6h21m40.467761695s) [upload=6610]: Waiting for error group managing segments...
Aug 7 16:01:41.195597254 (6h21m40.784942241s) [noise=true , address=79.127.205.235:4666 , piece_id=WA6GZKNO2UUAC5VKBNOZ7UO25RQR7QDT5HVNQYZDIWJEB656WMGA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.261081133 (6h21m40.85042612s) [noise=true , address=79.127.205.233:4631 , piece_id=77WNKENAMS4Q75KYK4F4VNRP4DD4MKGEJIU5K7WZ3EHVQLEW6Q5A, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.420961069 (6h21m41.010306046s) [noise=true , address=79.127.220.97:4396 , piece_id=X2CCRNYWY5ZQOAODHNJE7VLYQIHD37PEE7MQXI5WIYV36ICONILQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.638341479 (6h21m41.227686486s) [noise=true , address=79.127.201.215:4345 , piece_id=MAAFKAPO4OCQXU5WJ6DAP72KEHIM4AO5JJFJ4ZEFNWKS5M4OV43Q, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.658966771 (6h21m41.248311758s) [noise=true , address=45.91.116.10:4092 , piece_id=BUVKJC3K3RKZZTW2CQPEQM7ECQOLIBX4CU66P26JZDTOQZLYGMGA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.684178459 (6h21m41.273523446s) [noise=true , address=104.36.82.9:4039 , piece_id=PRBG5EVKAZ3LGDRX4MCI57T4RSNFVKZVUNOAIRHBM6SNS5XXUTPQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.691448508 (6h21m41.280793485s) [noise=true , address=104.36.82.22:4525 , piece_id=NKD4EXV774KNXLEHICE4MNKOQS5QKPIUTJL2PPCNYUMYPMQEF75A, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.717147267 (6h21m41.306492254s) [noise=true , address=104.36.82.59:4313 , piece_id=KDH6AVALUMDXB6USO7WG77DVOSRBTGSZA6VTJFFQJR7QFC2PE35A, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.719612885 (6h21m41.308957872s) [noise=true , address=104.36.82.45:4912 , piece_id=B6HTS6ORSGMUKZO5G2ALW4LVUSIBTCEHPX2YJTYPPCH3XROIO3HA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.733973160 (6h21m41.323318147s) [noise=true , address=104.36.82.60:4369 , piece_id=Y6SYJOB7SYBJX7STC25OJJPD4FSK5TZ5KXI7WN7QS42UNTOF5BCQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.737980718 (6h21m41.327325705s) [noise=true , address=104.36.82.5:4845 , piece_id=JKN4VFS7ET2LVGHGE4LIBPCGG33GNY4SGSOXAWB2MXFYPJVO4FCQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.745434729 (6h21m41.334779717s) [noise=true , address=104.36.82.42:4820 , piece_id=XJIHI2E35NJKDFYFTW4V5EIF3CL7GDJ5EKVCTEV2KL26X43A3CYA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.754624732 (6h21m41.34396972s) [noise=true , address=104.36.82.8:4992 , piece_id=226HGROQ3DBKTJB3U6TG2OF4JQRYAGTLMSIL2JIL4NY5PISP4RUQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.763605976 (6h21m41.352950963s) [noise=true , address=104.36.82.60:4345 , piece_id=VPYOQHGEP6E257WI7IGIWXRO5WOLCNGDJYNPFPC56WUMOYDUS6RA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.763623339 (6h21m41.352968336s) [seg_pos={0 0}, upload=6610]: Segment reached optimal threshold of 14 pieces.
Aug 7 16:01:41.763741525 (6h21m41.353086512s) [noise=true , address=104.36.82.10:4110 , piece_id=WAUEFOZXH3SULGTXTWXWLEPYHZJWJDDQRRWWXRBXVWF7WCWPYGYQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ADbnPJGRTw2NqRzhy7ptojGYoNdJgqmjuWBSjcrQK53Wm7LXi): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763803123 (6h21m41.35314811s) [noise=true , address=104.36.82.61:4387 , piece_id=MAWGNSJTYYH7EVPDIXQEG6KP3MKACSMWYVMMLYUASUDJBIX5POOA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P4wAXSyaRY9poPRK5q4td8DJM69xWioG38TTwLTHKHA7QFxLp): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763857136 (6h21m41.353202113s) [noise=true , address=45.91.116.9:4267 , piece_id=MWAYSXFIIYQGDPTLWZSAHZP3UO72EXYDSSULHBJJU54H5OWG36NA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6M5TgGqUwjXyM25uqv5hsh67mS81ByzMM4kWFUNaFvisDN9d): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763904968 (6h21m41.353249955s) [noise=true , address=104.36.82.23:4455 , piece_id=UJOJ7P3VET4XIM257B3SHAN6SJG2H64QJHV6PYU7OEB7PPBZVDAA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wsyjF5X3iPrJsezUjakUArxc8K88T36yT4rE3yu6q9BMKfycm): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763945405 (6h21m41.353290392s) [seg_pos={0 0}, upload=6610]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 16:01:41.764033824 (6h21m41.353378811s) [upload=6610]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 16:01:41.764038263 (6h21m41.35338325s) [upload=6610]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 16:01:42.043459384 (6h21m41.632804371s) [upload=6610]: Done uploading segments...
Aug 7 16:01:42.043475946 (6h21m41.632820933s) [upload=6610]: Upload finished. err: <nil>
Aug 7 16:02:11.915996790 (6h22m11.505341777s) [upload=6619]: Starting upload
Aug 7 16:02:11.916045183 (6h22m11.50539017s) [upload=6619]: Done starting upload
Aug 7 16:02:11.916152489 (6h22m11.505497476s) [upload=6619]: Uploading segments...
Aug 7 16:02:11.916287706 (6h22m11.505632693s) [upload=6619]: Got next segment. Inline: false
Aug 7 16:02:11.916319939 (6h22m11.505664926s) [upload=6619]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 16:02:11.916324708 (6h22m11.505669695s) [upload=6619]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 16:02:12.129590885 (6h22m11.718935892s) [upload=6619]: Next segment err: metaclient: Too Many Requests
Aug 7 16:02:12.129623196 (6h22m11.718968194s) [upload=6619]: Done uploading segments...
Aug 7 16:02:12.129640310 (6h22m11.718985307s) [upload=6619]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.907886087s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
1.830840654s
FAIL (2025-08-07 16:13:37 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== entMakeInline Aug 7 15:57:10.271829415 (6h17m9.861174403s) [upload=6533]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 15:57:10.587098885 (6h17m10.176443872s) [upload=6533]: Done uploading segments... Aug 7 15:57:10.587115857 (6h17m10.176460835s) [upload=6533]: Upload finished. err: <nil> Aug 7 15:57:48.918901442 (6h17m48.508246429s) [upload=6545]: Starting upload Aug 7 15:57:48.918948701 (6h17m48.508293688s) [upload=6545]: Done starting upload Aug 7 15:57:48.919013486 (6h17m48.508358473s) [upload=6545]: Uploading segments... Aug 7 15:57:48.919055005 (6h17m48.508399982s) [upload=6545]: Got next segment. Inline: true Aug 7 15:57:48.919059032 (6h17m48.508404009s) [upload=6545]: Waiting for error group managing segments... Aug 7 15:57:48.919091585 (6h17m48.508436572s) [upload=6545]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 15:57:48.919095493 (6h17m48.50844048s) [upload=6545]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 15:57:48.919097998 (6h17m48.508442975s) [upload=6545]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 15:57:49.200378977 (6h17m48.789723964s) [upload=6545]: Done uploading segments... Aug 7 15:57:49.200397332 (6h17m48.789742319s) [upload=6545]: Upload finished. err: <nil> Aug 7 15:58:50.898935036 (6h18m50.488280023s) [upload=6562]: Starting upload Aug 7 15:58:50.898979992 (6h18m50.488324979s) [upload=6562]: Done starting upload Aug 7 15:58:50.899026682 (6h18m50.488371669s) [upload=6562]: Uploading segments... Aug 7 15:58:50.899065355 (6h18m50.488410343s) [upload=6562]: Got next segment. Inline: true Aug 7 15:58:50.899070335 (6h18m50.488415322s) [upload=6562]: Waiting for error group managing segments... Aug 7 15:58:50.899108268 (6h18m50.488453255s) [upload=6562]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 15:58:50.899112326 (6h18m50.488457313s) [upload=6562]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 15:58:50.899114820 (6h18m50.488459807s) [upload=6562]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 15:58:51.203289783 (6h18m50.79263478s) [upload=6562]: Done uploading segments... Aug 7 15:58:51.203312587 (6h18m50.792657624s) [upload=6562]: Upload finished. err: <nil> Aug 7 16:00:00.470970125 (6h20m0.060315112s) [upload=6580]: Starting upload Aug 7 16:00:00.471017334 (6h20m0.060362321s) [upload=6580]: Done starting upload Aug 7 16:00:00.471072199 (6h20m0.060417186s) [upload=6580]: Uploading segments... Aug 7 16:00:00.471112327 (6h20m0.060457314s) [upload=6580]: Got next segment. Inline: true Aug 7 16:00:00.471116364 (6h20m0.060461351s) [upload=6580]: Waiting for error group managing segments... Aug 7 16:00:00.471149427 (6h20m0.060494414s) [upload=6580]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:00:00.471153305 (6h20m0.060498292s) [upload=6580]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:00:00.471155819 (6h20m0.060500806s) [upload=6580]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:00:00.739624026 (6h20m0.328969023s) [upload=6580]: Done uploading segments... Aug 7 16:00:00.739645636 (6h20m0.328990623s) [upload=6580]: Upload finished. err: <nil> Aug 7 16:00:49.882359165 (6h20m49.471704152s) [upload=6598]: Starting upload Aug 7 16:00:49.882404682 (6h20m49.471749659s) [upload=6598]: Done starting upload Aug 7 16:00:49.882456812 (6h20m49.471801799s) [upload=6598]: Uploading segments... Aug 7 16:00:49.882489865 (6h20m49.471834852s) [upload=6598]: Got next segment. Inline: true Aug 7 16:00:49.882493702 (6h20m49.47183869s) [upload=6598]: Waiting for error group managing segments... Aug 7 16:00:49.882525894 (6h20m49.471870881s) [upload=6598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:00:49.882529422 (6h20m49.471874399s) [upload=6598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:00:49.882531676 (6h20m49.471876663s) [upload=6598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:00:50.213720013 (6h20m49.803065s) [upload=6598]: Done uploading segments... Aug 7 16:00:50.213739550 (6h20m49.803084537s) [upload=6598]: Upload finished. err: <nil> Aug 7 16:01:51.762778733 (6h21m51.35212372s) [upload=6617]: Starting upload Aug 7 16:01:51.762856161 (6h21m51.352201148s) [upload=6617]: Done starting upload Aug 7 16:01:51.762914883 (6h21m51.35225987s) [upload=6617]: Uploading segments... Aug 7 16:01:51.762952445 (6h21m51.352297432s) [upload=6617]: Got next segment. Inline: true Aug 7 16:01:51.762957344 (6h21m51.352302331s) [upload=6617]: Waiting for error group managing segments... Aug 7 16:01:51.762994485 (6h21m51.352339473s) [upload=6617]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:01:51.762998313 (6h21m51.3523433s) [upload=6617]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:01:51.763001278 (6h21m51.352346265s) [upload=6617]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:01:52.034377407 (6h21m51.623722394s) [upload=6617]: Done uploading segments... Aug 7 16:01:52.034395451 (6h21m51.623740439s) [upload=6617]: Upload finished. err: <nil> Aug 7 16:02:11.154406473 (6h22m10.74375146s) [upload=6618]: Starting upload Aug 7 16:02:11.154456328 (6h22m10.743801315s) [upload=6618]: Done starting upload Aug 7 16:02:11.154503859 (6h22m10.743848846s) [upload=6618]: Uploading segments... Aug 7 16:02:11.154545308 (6h22m10.743890295s) [upload=6618]: Got next segment. Inline: true Aug 7 16:02:11.154548574 (6h22m10.743893561s) [upload=6618]: Waiting for error group managing segments... Aug 7 16:02:11.154580566 (6h22m10.743925553s) [upload=6618]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:02:11.154584283 (6h22m10.74392927s) [upload=6618]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:02:11.154586828 (6h22m10.743931815s) [upload=6618]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:02:11.475329609 (6h22m11.064674596s) [upload=6618]: Done uploading segments... Aug 7 16:02:11.475484125 (6h22m11.064829112s) [upload=6618]: Upload finished. err: <nil> Aug 7 16:02:42.941029250 (6h22m42.530374238s) [upload=6628]: Starting upload Aug 7 16:02:42.941096740 (6h22m42.530441727s) [upload=6628]: Done starting upload Aug 7 16:02:42.941158728 (6h22m42.530503716s) [upload=6628]: Uploading segments... Aug 7 16:02:42.941210478 (6h22m42.530555465s) [upload=6628]: Got next segment. Inline: true Aug 7 16:02:42.941216419 (6h22m42.530561416s) [upload=6628]: Waiting for error group managing segments... Aug 7 16:02:42.941281664 (6h22m42.530626651s) [upload=6628]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:02:42.941286824 (6h22m42.530631811s) [upload=6628]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:02:42.941289880 (6h22m42.530634867s) [upload=6628]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:02:43.233814342 (6h22m42.823159339s) [upload=6628]: Done uploading segments... Aug 7 16:02:43.233838148 (6h22m42.823183145s) [upload=6628]: Upload finished. err: <nil> Aug 7 16:03:45.201690833 (6h23m44.79103582s) [upload=6651]: Starting upload Aug 7 16:03:45.201738173 (6h23m44.79108316s) [upload=6651]: Done starting upload Aug 7 16:03:45.201785534 (6h23m44.791130511s) [upload=6651]: Uploading segments... Aug 7 16:03:45.201855618 (6h23m44.791200605s) [upload=6651]: Got next segment. Inline: true Aug 7 16:03:45.201859616 (6h23m44.791204603s) [upload=6651]: Waiting for error group managing segments... Aug 7 16:03:45.201894122 (6h23m44.791239109s) [upload=6651]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:03:45.201897799 (6h23m44.791242786s) [upload=6651]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:03:45.201900023 (6h23m44.79124501s) [upload=6651]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:03:45.644503676 (6h23m45.233848664s) [upload=6651]: Done uploading segments... Aug 7 16:03:45.644521010 (6h23m45.233865987s) [upload=6651]: Upload finished. err: <nil> Aug 7 16:04:31.274936212 (6h24m30.864281199s) [upload=6663]: Starting upload Aug 7 16:04:31.274977170 (6h24m30.864322157s) [upload=6663]: Done starting upload Aug 7 16:04:31.275026655 (6h24m30.864371642s) [upload=6663]: Uploading segments... Aug 7 16:04:31.275054187 (6h24m30.864399174s) [upload=6663]: Got next segment. Inline: true Aug 7 16:04:31.275057754 (6h24m30.864402731s) [upload=6663]: Waiting for error group managing segments... Aug 7 16:04:31.275091278 (6h24m30.864436265s) [upload=6663]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:04:31.275095255 (6h24m30.864440242s) [upload=6663]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:04:31.275097651 (6h24m30.864442638s) [upload=6663]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:04:31.554258512 (6h24m31.143603499s) [upload=6663]: Done uploading segments... Aug 7 16:04:31.554277108 (6h24m31.143622095s) [upload=6663]: Upload finished. err: <nil> Aug 7 16:05:14.730035713 (6h25m14.3193807s) [upload=6674]: Starting upload Aug 7 16:05:14.730082813 (6h25m14.3194278s) [upload=6674]: Done starting upload Aug 7 16:05:14.730159079 (6h25m14.319504056s) [upload=6674]: Uploading segments... Aug 7 16:05:14.730249532 (6h25m14.319594519s) [upload=6674]: Got next segment. Inline: true Aug 7 16:05:14.730254050 (6h25m14.319599037s) [upload=6674]: Waiting for error group managing segments... Aug 7 16:05:14.730294949 (6h25m14.319639936s) [upload=6674]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:05:14.730299728 (6h25m14.319644715s) [upload=6674]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:05:14.730302243 (6h25m14.31964723s) [upload=6674]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:05:14.942773689 (6h25m14.532118666s) [upload=6674]: Done uploading segments... Aug 7 16:05:14.942790141 (6h25m14.532135118s) [upload=6674]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.830865671s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [...................................X.......X................] duration:
5.725796087s
FAIL (2025-08-07 16:13:23 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
e begin upload segment.
Aug 7 16:01:39.748945470 (6h21m39.338290447s) [noise=true , address=104.36.82.42:4820 , piece_id=XJIHI2E35NJKDFYFTW4V5EIF3CL7GDJ5EKVCTEV2KL26X43A3CYA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.748969657 (6h21m39.338314644s) [noise=true , address=104.36.82.60:4345 , piece_id=VPYOQHGEP6E257WI7IGIWXRO5WOLCNGDJYNPFPC56WUMOYDUS6RA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749368329 (6h21m39.338713316s) [noise=true , address=45.91.116.9:4267 , piece_id=MWAYSXFIIYQGDPTLWZSAHZP3UO72EXYDSSULHBJJU54H5OWG36NA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749375964 (6h21m39.338720951s) [noise=true , address=104.36.82.10:4110 , piece_id=WAUEFOZXH3SULGTXTWXWLEPYHZJWJDDQRRWWXRBXVWF7WCWPYGYQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749660398 (6h21m39.339005385s) [noise=true , address=79.127.205.235:4666 , piece_id=WA6GZKNO2UUAC5VKBNOZ7UO25RQR7QDT5HVNQYZDIWJEB656WMGA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749701216 (6h21m39.339046203s) [noise=true , address=79.127.201.215:4345 , piece_id=MAAFKAPO4OCQXU5WJ6DAP72KEHIM4AO5JJFJ4ZEFNWKS5M4OV43Q, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749938690 (6h21m39.339283677s) [noise=true , address=104.36.82.59:4313 , piece_id=KDH6AVALUMDXB6USO7WG77DVOSRBTGSZA6VTJFFQJR7QFC2PE35A, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750024905 (6h21m39.339369892s) [noise=true , address=104.36.82.23:4455 , piece_id=UJOJ7P3VET4XIM257B3SHAN6SJG2H64QJHV6PYU7OEB7PPBZVDAA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750215089 (6h21m39.339560076s) [noise=true , address=104.36.82.61:4387 , piece_id=MAWGNSJTYYH7EVPDIXQEG6KP3MKACSMWYVMMLYUASUDJBIX5POOA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750282639 (6h21m39.339627626s) [noise=true , address=45.91.116.10:4092 , piece_id=BUVKJC3K3RKZZTW2CQPEQM7ECQOLIBX4CU66P26JZDTOQZLYGMGA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750481038 (6h21m39.339826025s) [noise=true , address=79.127.220.97:4396 , piece_id=X2CCRNYWY5ZQOAODHNJE7VLYQIHD37PEE7MQXI5WIYV36ICONILQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750533900 (6h21m39.339878887s) [noise=true , address=104.36.82.60:4369 , piece_id=Y6SYJOB7SYBJX7STC25OJJPD4FSK5TZ5KXI7WN7QS42UNTOF5BCQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750725967 (6h21m39.340070954s) [noise=true , address=104.36.82.9:4039 , piece_id=PRBG5EVKAZ3LGDRX4MCI57T4RSNFVKZVUNOAIRHBM6SNS5XXUTPQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750795189 (6h21m39.340140176s) [noise=true , address=104.36.82.22:4525 , piece_id=NKD4EXV774KNXLEHICE4MNKOQS5QKPIUTJL2PPCNYUMYPMQEF75A, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750954023 (6h21m39.34029901s) [noise=true , address=104.36.82.45:4912 , piece_id=B6HTS6ORSGMUKZO5G2ALW4LVUSIBTCEHPX2YJTYPPCH3XROIO3HA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.751098079 (6h21m39.340443066s) [noise=true , address=104.36.82.8:4992 , piece_id=226HGROQ3DBKTJB3U6TG2OF4JQRYAGTLMSIL2JIL4NY5PISP4RUQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.751175397 (6h21m39.340520384s) [noise=true , address=79.127.205.233:4631 , piece_id=77WNKENAMS4Q75KYK4F4VNRP4DD4MKGEJIU5K7WZ3EHVQLEW6Q5A, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.751392602 (6h21m39.340737579s) [noise=true , address=104.36.82.5:4845 , piece_id=JKN4VFS7ET2LVGHGE4LIBPCGG33GNY4SGSOXAWB2MXFYPJVO4FCQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:40.878402060 (6h21m40.467747047s) [upload=6610]: Next returned nil segment
Aug 7 16:01:40.878416708 (6h21m40.467761695s) [upload=6610]: Waiting for error group managing segments...
Aug 7 16:01:41.195597254 (6h21m40.784942241s) [noise=true , address=79.127.205.235:4666 , piece_id=WA6GZKNO2UUAC5VKBNOZ7UO25RQR7QDT5HVNQYZDIWJEB656WMGA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.261081133 (6h21m40.85042612s) [noise=true , address=79.127.205.233:4631 , piece_id=77WNKENAMS4Q75KYK4F4VNRP4DD4MKGEJIU5K7WZ3EHVQLEW6Q5A, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.420961069 (6h21m41.010306046s) [noise=true , address=79.127.220.97:4396 , piece_id=X2CCRNYWY5ZQOAODHNJE7VLYQIHD37PEE7MQXI5WIYV36ICONILQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.638341479 (6h21m41.227686486s) [noise=true , address=79.127.201.215:4345 , piece_id=MAAFKAPO4OCQXU5WJ6DAP72KEHIM4AO5JJFJ4ZEFNWKS5M4OV43Q, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.658966771 (6h21m41.248311758s) [noise=true , address=45.91.116.10:4092 , piece_id=BUVKJC3K3RKZZTW2CQPEQM7ECQOLIBX4CU66P26JZDTOQZLYGMGA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.684178459 (6h21m41.273523446s) [noise=true , address=104.36.82.9:4039 , piece_id=PRBG5EVKAZ3LGDRX4MCI57T4RSNFVKZVUNOAIRHBM6SNS5XXUTPQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.691448508 (6h21m41.280793485s) [noise=true , address=104.36.82.22:4525 , piece_id=NKD4EXV774KNXLEHICE4MNKOQS5QKPIUTJL2PPCNYUMYPMQEF75A, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.717147267 (6h21m41.306492254s) [noise=true , address=104.36.82.59:4313 , piece_id=KDH6AVALUMDXB6USO7WG77DVOSRBTGSZA6VTJFFQJR7QFC2PE35A, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.719612885 (6h21m41.308957872s) [noise=true , address=104.36.82.45:4912 , piece_id=B6HTS6ORSGMUKZO5G2ALW4LVUSIBTCEHPX2YJTYPPCH3XROIO3HA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.733973160 (6h21m41.323318147s) [noise=true , address=104.36.82.60:4369 , piece_id=Y6SYJOB7SYBJX7STC25OJJPD4FSK5TZ5KXI7WN7QS42UNTOF5BCQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.737980718 (6h21m41.327325705s) [noise=true , address=104.36.82.5:4845 , piece_id=JKN4VFS7ET2LVGHGE4LIBPCGG33GNY4SGSOXAWB2MXFYPJVO4FCQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.745434729 (6h21m41.334779717s) [noise=true , address=104.36.82.42:4820 , piece_id=XJIHI2E35NJKDFYFTW4V5EIF3CL7GDJ5EKVCTEV2KL26X43A3CYA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.754624732 (6h21m41.34396972s) [noise=true , address=104.36.82.8:4992 , piece_id=226HGROQ3DBKTJB3U6TG2OF4JQRYAGTLMSIL2JIL4NY5PISP4RUQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.763605976 (6h21m41.352950963s) [noise=true , address=104.36.82.60:4345 , piece_id=VPYOQHGEP6E257WI7IGIWXRO5WOLCNGDJYNPFPC56WUMOYDUS6RA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.763623339 (6h21m41.352968336s) [seg_pos={0 0}, upload=6610]: Segment reached optimal threshold of 14 pieces.
Aug 7 16:01:41.763741525 (6h21m41.353086512s) [noise=true , address=104.36.82.10:4110 , piece_id=WAUEFOZXH3SULGTXTWXWLEPYHZJWJDDQRRWWXRBXVWF7WCWPYGYQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ADbnPJGRTw2NqRzhy7ptojGYoNdJgqmjuWBSjcrQK53Wm7LXi): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763803123 (6h21m41.35314811s) [noise=true , address=104.36.82.61:4387 , piece_id=MAWGNSJTYYH7EVPDIXQEG6KP3MKACSMWYVMMLYUASUDJBIX5POOA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P4wAXSyaRY9poPRK5q4td8DJM69xWioG38TTwLTHKHA7QFxLp): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763857136 (6h21m41.353202113s) [noise=true , address=45.91.116.9:4267 , piece_id=MWAYSXFIIYQGDPTLWZSAHZP3UO72EXYDSSULHBJJU54H5OWG36NA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6M5TgGqUwjXyM25uqv5hsh67mS81ByzMM4kWFUNaFvisDN9d): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763904968 (6h21m41.353249955s) [noise=true , address=104.36.82.23:4455 , piece_id=UJOJ7P3VET4XIM257B3SHAN6SJG2H64QJHV6PYU7OEB7PPBZVDAA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wsyjF5X3iPrJsezUjakUArxc8K88T36yT4rE3yu6q9BMKfycm): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763945405 (6h21m41.353290392s) [seg_pos={0 0}, upload=6610]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 16:01:41.764033824 (6h21m41.353378811s) [upload=6610]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 16:01:41.764038263 (6h21m41.35338325s) [upload=6610]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 16:01:42.043459384 (6h21m41.632804371s) [upload=6610]: Done uploading segments...
Aug 7 16:01:42.043475946 (6h21m41.632820933s) [upload=6610]: Upload finished. err: <nil>
Aug 7 16:02:11.915996790 (6h22m11.505341777s) [upload=6619]: Starting upload
Aug 7 16:02:11.916045183 (6h22m11.50539017s) [upload=6619]: Done starting upload
Aug 7 16:02:11.916152489 (6h22m11.505497476s) [upload=6619]: Uploading segments...
Aug 7 16:02:11.916287706 (6h22m11.505632693s) [upload=6619]: Got next segment. Inline: false
Aug 7 16:02:11.916319939 (6h22m11.505664926s) [upload=6619]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 16:02:11.916324708 (6h22m11.505669695s) [upload=6619]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 16:02:12.129590885 (6h22m11.718935892s) [upload=6619]: Next segment err: metaclient: Too Many Requests
Aug 7 16:02:12.129623196 (6h22m11.718968194s) [upload=6619]: Done uploading segments...
Aug 7 16:02:12.129640310 (6h22m11.718985307s) [upload=6619]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.725820174s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
1.929504194s
FAIL (2025-08-07 16:13:17 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== entMakeInline Aug 7 15:57:10.271829415 (6h17m9.861174403s) [upload=6533]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 15:57:10.587098885 (6h17m10.176443872s) [upload=6533]: Done uploading segments... Aug 7 15:57:10.587115857 (6h17m10.176460835s) [upload=6533]: Upload finished. err: <nil> Aug 7 15:57:48.918901442 (6h17m48.508246429s) [upload=6545]: Starting upload Aug 7 15:57:48.918948701 (6h17m48.508293688s) [upload=6545]: Done starting upload Aug 7 15:57:48.919013486 (6h17m48.508358473s) [upload=6545]: Uploading segments... Aug 7 15:57:48.919055005 (6h17m48.508399982s) [upload=6545]: Got next segment. Inline: true Aug 7 15:57:48.919059032 (6h17m48.508404009s) [upload=6545]: Waiting for error group managing segments... Aug 7 15:57:48.919091585 (6h17m48.508436572s) [upload=6545]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 15:57:48.919095493 (6h17m48.50844048s) [upload=6545]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 15:57:48.919097998 (6h17m48.508442975s) [upload=6545]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 15:57:49.200378977 (6h17m48.789723964s) [upload=6545]: Done uploading segments... Aug 7 15:57:49.200397332 (6h17m48.789742319s) [upload=6545]: Upload finished. err: <nil> Aug 7 15:58:50.898935036 (6h18m50.488280023s) [upload=6562]: Starting upload Aug 7 15:58:50.898979992 (6h18m50.488324979s) [upload=6562]: Done starting upload Aug 7 15:58:50.899026682 (6h18m50.488371669s) [upload=6562]: Uploading segments... Aug 7 15:58:50.899065355 (6h18m50.488410343s) [upload=6562]: Got next segment. Inline: true Aug 7 15:58:50.899070335 (6h18m50.488415322s) [upload=6562]: Waiting for error group managing segments... Aug 7 15:58:50.899108268 (6h18m50.488453255s) [upload=6562]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 15:58:50.899112326 (6h18m50.488457313s) [upload=6562]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 15:58:50.899114820 (6h18m50.488459807s) [upload=6562]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 15:58:51.203289783 (6h18m50.79263478s) [upload=6562]: Done uploading segments... Aug 7 15:58:51.203312587 (6h18m50.792657624s) [upload=6562]: Upload finished. err: <nil> Aug 7 16:00:00.470970125 (6h20m0.060315112s) [upload=6580]: Starting upload Aug 7 16:00:00.471017334 (6h20m0.060362321s) [upload=6580]: Done starting upload Aug 7 16:00:00.471072199 (6h20m0.060417186s) [upload=6580]: Uploading segments... Aug 7 16:00:00.471112327 (6h20m0.060457314s) [upload=6580]: Got next segment. Inline: true Aug 7 16:00:00.471116364 (6h20m0.060461351s) [upload=6580]: Waiting for error group managing segments... Aug 7 16:00:00.471149427 (6h20m0.060494414s) [upload=6580]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:00:00.471153305 (6h20m0.060498292s) [upload=6580]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:00:00.471155819 (6h20m0.060500806s) [upload=6580]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:00:00.739624026 (6h20m0.328969023s) [upload=6580]: Done uploading segments... Aug 7 16:00:00.739645636 (6h20m0.328990623s) [upload=6580]: Upload finished. err: <nil> Aug 7 16:00:49.882359165 (6h20m49.471704152s) [upload=6598]: Starting upload Aug 7 16:00:49.882404682 (6h20m49.471749659s) [upload=6598]: Done starting upload Aug 7 16:00:49.882456812 (6h20m49.471801799s) [upload=6598]: Uploading segments... Aug 7 16:00:49.882489865 (6h20m49.471834852s) [upload=6598]: Got next segment. Inline: true Aug 7 16:00:49.882493702 (6h20m49.47183869s) [upload=6598]: Waiting for error group managing segments... Aug 7 16:00:49.882525894 (6h20m49.471870881s) [upload=6598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:00:49.882529422 (6h20m49.471874399s) [upload=6598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:00:49.882531676 (6h20m49.471876663s) [upload=6598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:00:50.213720013 (6h20m49.803065s) [upload=6598]: Done uploading segments... Aug 7 16:00:50.213739550 (6h20m49.803084537s) [upload=6598]: Upload finished. err: <nil> Aug 7 16:01:51.762778733 (6h21m51.35212372s) [upload=6617]: Starting upload Aug 7 16:01:51.762856161 (6h21m51.352201148s) [upload=6617]: Done starting upload Aug 7 16:01:51.762914883 (6h21m51.35225987s) [upload=6617]: Uploading segments... Aug 7 16:01:51.762952445 (6h21m51.352297432s) [upload=6617]: Got next segment. Inline: true Aug 7 16:01:51.762957344 (6h21m51.352302331s) [upload=6617]: Waiting for error group managing segments... Aug 7 16:01:51.762994485 (6h21m51.352339473s) [upload=6617]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:01:51.762998313 (6h21m51.3523433s) [upload=6617]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:01:51.763001278 (6h21m51.352346265s) [upload=6617]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:01:52.034377407 (6h21m51.623722394s) [upload=6617]: Done uploading segments... Aug 7 16:01:52.034395451 (6h21m51.623740439s) [upload=6617]: Upload finished. err: <nil> Aug 7 16:02:11.154406473 (6h22m10.74375146s) [upload=6618]: Starting upload Aug 7 16:02:11.154456328 (6h22m10.743801315s) [upload=6618]: Done starting upload Aug 7 16:02:11.154503859 (6h22m10.743848846s) [upload=6618]: Uploading segments... Aug 7 16:02:11.154545308 (6h22m10.743890295s) [upload=6618]: Got next segment. Inline: true Aug 7 16:02:11.154548574 (6h22m10.743893561s) [upload=6618]: Waiting for error group managing segments... Aug 7 16:02:11.154580566 (6h22m10.743925553s) [upload=6618]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:02:11.154584283 (6h22m10.74392927s) [upload=6618]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:02:11.154586828 (6h22m10.743931815s) [upload=6618]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:02:11.475329609 (6h22m11.064674596s) [upload=6618]: Done uploading segments... Aug 7 16:02:11.475484125 (6h22m11.064829112s) [upload=6618]: Upload finished. err: <nil> Aug 7 16:02:42.941029250 (6h22m42.530374238s) [upload=6628]: Starting upload Aug 7 16:02:42.941096740 (6h22m42.530441727s) [upload=6628]: Done starting upload Aug 7 16:02:42.941158728 (6h22m42.530503716s) [upload=6628]: Uploading segments... Aug 7 16:02:42.941210478 (6h22m42.530555465s) [upload=6628]: Got next segment. Inline: true Aug 7 16:02:42.941216419 (6h22m42.530561416s) [upload=6628]: Waiting for error group managing segments... Aug 7 16:02:42.941281664 (6h22m42.530626651s) [upload=6628]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:02:42.941286824 (6h22m42.530631811s) [upload=6628]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:02:42.941289880 (6h22m42.530634867s) [upload=6628]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:02:43.233814342 (6h22m42.823159339s) [upload=6628]: Done uploading segments... Aug 7 16:02:43.233838148 (6h22m42.823183145s) [upload=6628]: Upload finished. err: <nil> Aug 7 16:03:45.201690833 (6h23m44.79103582s) [upload=6651]: Starting upload Aug 7 16:03:45.201738173 (6h23m44.79108316s) [upload=6651]: Done starting upload Aug 7 16:03:45.201785534 (6h23m44.791130511s) [upload=6651]: Uploading segments... Aug 7 16:03:45.201855618 (6h23m44.791200605s) [upload=6651]: Got next segment. Inline: true Aug 7 16:03:45.201859616 (6h23m44.791204603s) [upload=6651]: Waiting for error group managing segments... Aug 7 16:03:45.201894122 (6h23m44.791239109s) [upload=6651]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:03:45.201897799 (6h23m44.791242786s) [upload=6651]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:03:45.201900023 (6h23m44.79124501s) [upload=6651]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:03:45.644503676 (6h23m45.233848664s) [upload=6651]: Done uploading segments... Aug 7 16:03:45.644521010 (6h23m45.233865987s) [upload=6651]: Upload finished. err: <nil> Aug 7 16:04:31.274936212 (6h24m30.864281199s) [upload=6663]: Starting upload Aug 7 16:04:31.274977170 (6h24m30.864322157s) [upload=6663]: Done starting upload Aug 7 16:04:31.275026655 (6h24m30.864371642s) [upload=6663]: Uploading segments... Aug 7 16:04:31.275054187 (6h24m30.864399174s) [upload=6663]: Got next segment. Inline: true Aug 7 16:04:31.275057754 (6h24m30.864402731s) [upload=6663]: Waiting for error group managing segments... Aug 7 16:04:31.275091278 (6h24m30.864436265s) [upload=6663]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:04:31.275095255 (6h24m30.864440242s) [upload=6663]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:04:31.275097651 (6h24m30.864442638s) [upload=6663]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:04:31.554258512 (6h24m31.143603499s) [upload=6663]: Done uploading segments... Aug 7 16:04:31.554277108 (6h24m31.143622095s) [upload=6663]: Upload finished. err: <nil> Aug 7 16:05:14.730035713 (6h25m14.3193807s) [upload=6674]: Starting upload Aug 7 16:05:14.730082813 (6h25m14.3194278s) [upload=6674]: Done starting upload Aug 7 16:05:14.730159079 (6h25m14.319504056s) [upload=6674]: Uploading segments... Aug 7 16:05:14.730249532 (6h25m14.319594519s) [upload=6674]: Got next segment. Inline: true Aug 7 16:05:14.730254050 (6h25m14.319599037s) [upload=6674]: Waiting for error group managing segments... Aug 7 16:05:14.730294949 (6h25m14.319639936s) [upload=6674]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:05:14.730299728 (6h25m14.319644715s) [upload=6674]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:05:14.730302243 (6h25m14.31964723s) [upload=6674]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:05:14.942773689 (6h25m14.532118666s) [upload=6674]: Done uploading segments... Aug 7 16:05:14.942790141 (6h25m14.532135118s) [upload=6674]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.929527359s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
1.887613263s
FAIL (2025-08-07 16:12:58 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== entMakeInline Aug 7 15:57:10.271829415 (6h17m9.861174403s) [upload=6533]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 15:57:10.587098885 (6h17m10.176443872s) [upload=6533]: Done uploading segments... Aug 7 15:57:10.587115857 (6h17m10.176460835s) [upload=6533]: Upload finished. err: <nil> Aug 7 15:57:48.918901442 (6h17m48.508246429s) [upload=6545]: Starting upload Aug 7 15:57:48.918948701 (6h17m48.508293688s) [upload=6545]: Done starting upload Aug 7 15:57:48.919013486 (6h17m48.508358473s) [upload=6545]: Uploading segments... Aug 7 15:57:48.919055005 (6h17m48.508399982s) [upload=6545]: Got next segment. Inline: true Aug 7 15:57:48.919059032 (6h17m48.508404009s) [upload=6545]: Waiting for error group managing segments... Aug 7 15:57:48.919091585 (6h17m48.508436572s) [upload=6545]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 15:57:48.919095493 (6h17m48.50844048s) [upload=6545]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 15:57:48.919097998 (6h17m48.508442975s) [upload=6545]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 15:57:49.200378977 (6h17m48.789723964s) [upload=6545]: Done uploading segments... Aug 7 15:57:49.200397332 (6h17m48.789742319s) [upload=6545]: Upload finished. err: <nil> Aug 7 15:58:50.898935036 (6h18m50.488280023s) [upload=6562]: Starting upload Aug 7 15:58:50.898979992 (6h18m50.488324979s) [upload=6562]: Done starting upload Aug 7 15:58:50.899026682 (6h18m50.488371669s) [upload=6562]: Uploading segments... Aug 7 15:58:50.899065355 (6h18m50.488410343s) [upload=6562]: Got next segment. Inline: true Aug 7 15:58:50.899070335 (6h18m50.488415322s) [upload=6562]: Waiting for error group managing segments... Aug 7 15:58:50.899108268 (6h18m50.488453255s) [upload=6562]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 15:58:50.899112326 (6h18m50.488457313s) [upload=6562]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 15:58:50.899114820 (6h18m50.488459807s) [upload=6562]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 15:58:51.203289783 (6h18m50.79263478s) [upload=6562]: Done uploading segments... Aug 7 15:58:51.203312587 (6h18m50.792657624s) [upload=6562]: Upload finished. err: <nil> Aug 7 16:00:00.470970125 (6h20m0.060315112s) [upload=6580]: Starting upload Aug 7 16:00:00.471017334 (6h20m0.060362321s) [upload=6580]: Done starting upload Aug 7 16:00:00.471072199 (6h20m0.060417186s) [upload=6580]: Uploading segments... Aug 7 16:00:00.471112327 (6h20m0.060457314s) [upload=6580]: Got next segment. Inline: true Aug 7 16:00:00.471116364 (6h20m0.060461351s) [upload=6580]: Waiting for error group managing segments... Aug 7 16:00:00.471149427 (6h20m0.060494414s) [upload=6580]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:00:00.471153305 (6h20m0.060498292s) [upload=6580]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:00:00.471155819 (6h20m0.060500806s) [upload=6580]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:00:00.739624026 (6h20m0.328969023s) [upload=6580]: Done uploading segments... Aug 7 16:00:00.739645636 (6h20m0.328990623s) [upload=6580]: Upload finished. err: <nil> Aug 7 16:00:49.882359165 (6h20m49.471704152s) [upload=6598]: Starting upload Aug 7 16:00:49.882404682 (6h20m49.471749659s) [upload=6598]: Done starting upload Aug 7 16:00:49.882456812 (6h20m49.471801799s) [upload=6598]: Uploading segments... Aug 7 16:00:49.882489865 (6h20m49.471834852s) [upload=6598]: Got next segment. Inline: true Aug 7 16:00:49.882493702 (6h20m49.47183869s) [upload=6598]: Waiting for error group managing segments... Aug 7 16:00:49.882525894 (6h20m49.471870881s) [upload=6598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:00:49.882529422 (6h20m49.471874399s) [upload=6598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:00:49.882531676 (6h20m49.471876663s) [upload=6598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:00:50.213720013 (6h20m49.803065s) [upload=6598]: Done uploading segments... Aug 7 16:00:50.213739550 (6h20m49.803084537s) [upload=6598]: Upload finished. err: <nil> Aug 7 16:01:51.762778733 (6h21m51.35212372s) [upload=6617]: Starting upload Aug 7 16:01:51.762856161 (6h21m51.352201148s) [upload=6617]: Done starting upload Aug 7 16:01:51.762914883 (6h21m51.35225987s) [upload=6617]: Uploading segments... Aug 7 16:01:51.762952445 (6h21m51.352297432s) [upload=6617]: Got next segment. Inline: true Aug 7 16:01:51.762957344 (6h21m51.352302331s) [upload=6617]: Waiting for error group managing segments... Aug 7 16:01:51.762994485 (6h21m51.352339473s) [upload=6617]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:01:51.762998313 (6h21m51.3523433s) [upload=6617]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:01:51.763001278 (6h21m51.352346265s) [upload=6617]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:01:52.034377407 (6h21m51.623722394s) [upload=6617]: Done uploading segments... Aug 7 16:01:52.034395451 (6h21m51.623740439s) [upload=6617]: Upload finished. err: <nil> Aug 7 16:02:11.154406473 (6h22m10.74375146s) [upload=6618]: Starting upload Aug 7 16:02:11.154456328 (6h22m10.743801315s) [upload=6618]: Done starting upload Aug 7 16:02:11.154503859 (6h22m10.743848846s) [upload=6618]: Uploading segments... Aug 7 16:02:11.154545308 (6h22m10.743890295s) [upload=6618]: Got next segment. Inline: true Aug 7 16:02:11.154548574 (6h22m10.743893561s) [upload=6618]: Waiting for error group managing segments... Aug 7 16:02:11.154580566 (6h22m10.743925553s) [upload=6618]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:02:11.154584283 (6h22m10.74392927s) [upload=6618]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:02:11.154586828 (6h22m10.743931815s) [upload=6618]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:02:11.475329609 (6h22m11.064674596s) [upload=6618]: Done uploading segments... Aug 7 16:02:11.475484125 (6h22m11.064829112s) [upload=6618]: Upload finished. err: <nil> Aug 7 16:02:42.941029250 (6h22m42.530374238s) [upload=6628]: Starting upload Aug 7 16:02:42.941096740 (6h22m42.530441727s) [upload=6628]: Done starting upload Aug 7 16:02:42.941158728 (6h22m42.530503716s) [upload=6628]: Uploading segments... Aug 7 16:02:42.941210478 (6h22m42.530555465s) [upload=6628]: Got next segment. Inline: true Aug 7 16:02:42.941216419 (6h22m42.530561416s) [upload=6628]: Waiting for error group managing segments... Aug 7 16:02:42.941281664 (6h22m42.530626651s) [upload=6628]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:02:42.941286824 (6h22m42.530631811s) [upload=6628]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:02:42.941289880 (6h22m42.530634867s) [upload=6628]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:02:43.233814342 (6h22m42.823159339s) [upload=6628]: Done uploading segments... Aug 7 16:02:43.233838148 (6h22m42.823183145s) [upload=6628]: Upload finished. err: <nil> Aug 7 16:03:45.201690833 (6h23m44.79103582s) [upload=6651]: Starting upload Aug 7 16:03:45.201738173 (6h23m44.79108316s) [upload=6651]: Done starting upload Aug 7 16:03:45.201785534 (6h23m44.791130511s) [upload=6651]: Uploading segments... Aug 7 16:03:45.201855618 (6h23m44.791200605s) [upload=6651]: Got next segment. Inline: true Aug 7 16:03:45.201859616 (6h23m44.791204603s) [upload=6651]: Waiting for error group managing segments... Aug 7 16:03:45.201894122 (6h23m44.791239109s) [upload=6651]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:03:45.201897799 (6h23m44.791242786s) [upload=6651]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:03:45.201900023 (6h23m44.79124501s) [upload=6651]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:03:45.644503676 (6h23m45.233848664s) [upload=6651]: Done uploading segments... Aug 7 16:03:45.644521010 (6h23m45.233865987s) [upload=6651]: Upload finished. err: <nil> Aug 7 16:04:31.274936212 (6h24m30.864281199s) [upload=6663]: Starting upload Aug 7 16:04:31.274977170 (6h24m30.864322157s) [upload=6663]: Done starting upload Aug 7 16:04:31.275026655 (6h24m30.864371642s) [upload=6663]: Uploading segments... Aug 7 16:04:31.275054187 (6h24m30.864399174s) [upload=6663]: Got next segment. Inline: true Aug 7 16:04:31.275057754 (6h24m30.864402731s) [upload=6663]: Waiting for error group managing segments... Aug 7 16:04:31.275091278 (6h24m30.864436265s) [upload=6663]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:04:31.275095255 (6h24m30.864440242s) [upload=6663]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:04:31.275097651 (6h24m30.864442638s) [upload=6663]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:04:31.554258512 (6h24m31.143603499s) [upload=6663]: Done uploading segments... Aug 7 16:04:31.554277108 (6h24m31.143622095s) [upload=6663]: Upload finished. err: <nil> Aug 7 16:05:14.730035713 (6h25m14.3193807s) [upload=6674]: Starting upload Aug 7 16:05:14.730082813 (6h25m14.3194278s) [upload=6674]: Done starting upload Aug 7 16:05:14.730159079 (6h25m14.319504056s) [upload=6674]: Uploading segments... Aug 7 16:05:14.730249532 (6h25m14.319594519s) [upload=6674]: Got next segment. Inline: true Aug 7 16:05:14.730254050 (6h25m14.319599037s) [upload=6674]: Waiting for error group managing segments... Aug 7 16:05:14.730294949 (6h25m14.319639936s) [upload=6674]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:05:14.730299728 (6h25m14.319644715s) [upload=6674]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:05:14.730302243 (6h25m14.31964723s) [upload=6674]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:05:14.942773689 (6h25m14.532118666s) [upload=6674]: Done uploading segments... Aug 7 16:05:14.942790141 (6h25m14.532135118s) [upload=6674]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.887639545s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
1.878837546s
FAIL (2025-08-07 16:12:18 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== entMakeInline Aug 7 15:57:10.271829415 (6h17m9.861174403s) [upload=6533]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 15:57:10.587098885 (6h17m10.176443872s) [upload=6533]: Done uploading segments... Aug 7 15:57:10.587115857 (6h17m10.176460835s) [upload=6533]: Upload finished. err: <nil> Aug 7 15:57:48.918901442 (6h17m48.508246429s) [upload=6545]: Starting upload Aug 7 15:57:48.918948701 (6h17m48.508293688s) [upload=6545]: Done starting upload Aug 7 15:57:48.919013486 (6h17m48.508358473s) [upload=6545]: Uploading segments... Aug 7 15:57:48.919055005 (6h17m48.508399982s) [upload=6545]: Got next segment. Inline: true Aug 7 15:57:48.919059032 (6h17m48.508404009s) [upload=6545]: Waiting for error group managing segments... Aug 7 15:57:48.919091585 (6h17m48.508436572s) [upload=6545]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 15:57:48.919095493 (6h17m48.50844048s) [upload=6545]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 15:57:48.919097998 (6h17m48.508442975s) [upload=6545]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 15:57:49.200378977 (6h17m48.789723964s) [upload=6545]: Done uploading segments... Aug 7 15:57:49.200397332 (6h17m48.789742319s) [upload=6545]: Upload finished. err: <nil> Aug 7 15:58:50.898935036 (6h18m50.488280023s) [upload=6562]: Starting upload Aug 7 15:58:50.898979992 (6h18m50.488324979s) [upload=6562]: Done starting upload Aug 7 15:58:50.899026682 (6h18m50.488371669s) [upload=6562]: Uploading segments... Aug 7 15:58:50.899065355 (6h18m50.488410343s) [upload=6562]: Got next segment. Inline: true Aug 7 15:58:50.899070335 (6h18m50.488415322s) [upload=6562]: Waiting for error group managing segments... Aug 7 15:58:50.899108268 (6h18m50.488453255s) [upload=6562]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 15:58:50.899112326 (6h18m50.488457313s) [upload=6562]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 15:58:50.899114820 (6h18m50.488459807s) [upload=6562]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 15:58:51.203289783 (6h18m50.79263478s) [upload=6562]: Done uploading segments... Aug 7 15:58:51.203312587 (6h18m50.792657624s) [upload=6562]: Upload finished. err: <nil> Aug 7 16:00:00.470970125 (6h20m0.060315112s) [upload=6580]: Starting upload Aug 7 16:00:00.471017334 (6h20m0.060362321s) [upload=6580]: Done starting upload Aug 7 16:00:00.471072199 (6h20m0.060417186s) [upload=6580]: Uploading segments... Aug 7 16:00:00.471112327 (6h20m0.060457314s) [upload=6580]: Got next segment. Inline: true Aug 7 16:00:00.471116364 (6h20m0.060461351s) [upload=6580]: Waiting for error group managing segments... Aug 7 16:00:00.471149427 (6h20m0.060494414s) [upload=6580]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:00:00.471153305 (6h20m0.060498292s) [upload=6580]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:00:00.471155819 (6h20m0.060500806s) [upload=6580]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:00:00.739624026 (6h20m0.328969023s) [upload=6580]: Done uploading segments... Aug 7 16:00:00.739645636 (6h20m0.328990623s) [upload=6580]: Upload finished. err: <nil> Aug 7 16:00:49.882359165 (6h20m49.471704152s) [upload=6598]: Starting upload Aug 7 16:00:49.882404682 (6h20m49.471749659s) [upload=6598]: Done starting upload Aug 7 16:00:49.882456812 (6h20m49.471801799s) [upload=6598]: Uploading segments... Aug 7 16:00:49.882489865 (6h20m49.471834852s) [upload=6598]: Got next segment. Inline: true Aug 7 16:00:49.882493702 (6h20m49.47183869s) [upload=6598]: Waiting for error group managing segments... Aug 7 16:00:49.882525894 (6h20m49.471870881s) [upload=6598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:00:49.882529422 (6h20m49.471874399s) [upload=6598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:00:49.882531676 (6h20m49.471876663s) [upload=6598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:00:50.213720013 (6h20m49.803065s) [upload=6598]: Done uploading segments... Aug 7 16:00:50.213739550 (6h20m49.803084537s) [upload=6598]: Upload finished. err: <nil> Aug 7 16:01:51.762778733 (6h21m51.35212372s) [upload=6617]: Starting upload Aug 7 16:01:51.762856161 (6h21m51.352201148s) [upload=6617]: Done starting upload Aug 7 16:01:51.762914883 (6h21m51.35225987s) [upload=6617]: Uploading segments... Aug 7 16:01:51.762952445 (6h21m51.352297432s) [upload=6617]: Got next segment. Inline: true Aug 7 16:01:51.762957344 (6h21m51.352302331s) [upload=6617]: Waiting for error group managing segments... Aug 7 16:01:51.762994485 (6h21m51.352339473s) [upload=6617]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:01:51.762998313 (6h21m51.3523433s) [upload=6617]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:01:51.763001278 (6h21m51.352346265s) [upload=6617]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:01:52.034377407 (6h21m51.623722394s) [upload=6617]: Done uploading segments... Aug 7 16:01:52.034395451 (6h21m51.623740439s) [upload=6617]: Upload finished. err: <nil> Aug 7 16:02:11.154406473 (6h22m10.74375146s) [upload=6618]: Starting upload Aug 7 16:02:11.154456328 (6h22m10.743801315s) [upload=6618]: Done starting upload Aug 7 16:02:11.154503859 (6h22m10.743848846s) [upload=6618]: Uploading segments... Aug 7 16:02:11.154545308 (6h22m10.743890295s) [upload=6618]: Got next segment. Inline: true Aug 7 16:02:11.154548574 (6h22m10.743893561s) [upload=6618]: Waiting for error group managing segments... Aug 7 16:02:11.154580566 (6h22m10.743925553s) [upload=6618]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:02:11.154584283 (6h22m10.74392927s) [upload=6618]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:02:11.154586828 (6h22m10.743931815s) [upload=6618]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:02:11.475329609 (6h22m11.064674596s) [upload=6618]: Done uploading segments... Aug 7 16:02:11.475484125 (6h22m11.064829112s) [upload=6618]: Upload finished. err: <nil> Aug 7 16:02:42.941029250 (6h22m42.530374238s) [upload=6628]: Starting upload Aug 7 16:02:42.941096740 (6h22m42.530441727s) [upload=6628]: Done starting upload Aug 7 16:02:42.941158728 (6h22m42.530503716s) [upload=6628]: Uploading segments... Aug 7 16:02:42.941210478 (6h22m42.530555465s) [upload=6628]: Got next segment. Inline: true Aug 7 16:02:42.941216419 (6h22m42.530561416s) [upload=6628]: Waiting for error group managing segments... Aug 7 16:02:42.941281664 (6h22m42.530626651s) [upload=6628]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:02:42.941286824 (6h22m42.530631811s) [upload=6628]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:02:42.941289880 (6h22m42.530634867s) [upload=6628]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:02:43.233814342 (6h22m42.823159339s) [upload=6628]: Done uploading segments... Aug 7 16:02:43.233838148 (6h22m42.823183145s) [upload=6628]: Upload finished. err: <nil> Aug 7 16:03:45.201690833 (6h23m44.79103582s) [upload=6651]: Starting upload Aug 7 16:03:45.201738173 (6h23m44.79108316s) [upload=6651]: Done starting upload Aug 7 16:03:45.201785534 (6h23m44.791130511s) [upload=6651]: Uploading segments... Aug 7 16:03:45.201855618 (6h23m44.791200605s) [upload=6651]: Got next segment. Inline: true Aug 7 16:03:45.201859616 (6h23m44.791204603s) [upload=6651]: Waiting for error group managing segments... Aug 7 16:03:45.201894122 (6h23m44.791239109s) [upload=6651]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:03:45.201897799 (6h23m44.791242786s) [upload=6651]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:03:45.201900023 (6h23m44.79124501s) [upload=6651]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:03:45.644503676 (6h23m45.233848664s) [upload=6651]: Done uploading segments... Aug 7 16:03:45.644521010 (6h23m45.233865987s) [upload=6651]: Upload finished. err: <nil> Aug 7 16:04:31.274936212 (6h24m30.864281199s) [upload=6663]: Starting upload Aug 7 16:04:31.274977170 (6h24m30.864322157s) [upload=6663]: Done starting upload Aug 7 16:04:31.275026655 (6h24m30.864371642s) [upload=6663]: Uploading segments... Aug 7 16:04:31.275054187 (6h24m30.864399174s) [upload=6663]: Got next segment. Inline: true Aug 7 16:04:31.275057754 (6h24m30.864402731s) [upload=6663]: Waiting for error group managing segments... Aug 7 16:04:31.275091278 (6h24m30.864436265s) [upload=6663]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:04:31.275095255 (6h24m30.864440242s) [upload=6663]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:04:31.275097651 (6h24m30.864442638s) [upload=6663]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:04:31.554258512 (6h24m31.143603499s) [upload=6663]: Done uploading segments... Aug 7 16:04:31.554277108 (6h24m31.143622095s) [upload=6663]: Upload finished. err: <nil> Aug 7 16:05:14.730035713 (6h25m14.3193807s) [upload=6674]: Starting upload Aug 7 16:05:14.730082813 (6h25m14.3194278s) [upload=6674]: Done starting upload Aug 7 16:05:14.730159079 (6h25m14.319504056s) [upload=6674]: Uploading segments... Aug 7 16:05:14.730249532 (6h25m14.319594519s) [upload=6674]: Got next segment. Inline: true Aug 7 16:05:14.730254050 (6h25m14.319599037s) [upload=6674]: Waiting for error group managing segments... Aug 7 16:05:14.730294949 (6h25m14.319639936s) [upload=6674]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:05:14.730299728 (6h25m14.319644715s) [upload=6674]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:05:14.730302243 (6h25m14.31964723s) [upload=6674]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:05:14.942773689 (6h25m14.532118666s) [upload=6674]: Done uploading segments... Aug 7 16:05:14.942790141 (6h25m14.532135118s) [upload=6674]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.878862212s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [....................................X.......X...............] duration:
5.878868988s
FAIL (2025-08-07 16:12:09 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
e begin upload segment.
Aug 7 16:01:39.748945470 (6h21m39.338290447s) [noise=true , address=104.36.82.42:4820 , piece_id=XJIHI2E35NJKDFYFTW4V5EIF3CL7GDJ5EKVCTEV2KL26X43A3CYA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.748969657 (6h21m39.338314644s) [noise=true , address=104.36.82.60:4345 , piece_id=VPYOQHGEP6E257WI7IGIWXRO5WOLCNGDJYNPFPC56WUMOYDUS6RA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749368329 (6h21m39.338713316s) [noise=true , address=45.91.116.9:4267 , piece_id=MWAYSXFIIYQGDPTLWZSAHZP3UO72EXYDSSULHBJJU54H5OWG36NA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749375964 (6h21m39.338720951s) [noise=true , address=104.36.82.10:4110 , piece_id=WAUEFOZXH3SULGTXTWXWLEPYHZJWJDDQRRWWXRBXVWF7WCWPYGYQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749660398 (6h21m39.339005385s) [noise=true , address=79.127.205.235:4666 , piece_id=WA6GZKNO2UUAC5VKBNOZ7UO25RQR7QDT5HVNQYZDIWJEB656WMGA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749701216 (6h21m39.339046203s) [noise=true , address=79.127.201.215:4345 , piece_id=MAAFKAPO4OCQXU5WJ6DAP72KEHIM4AO5JJFJ4ZEFNWKS5M4OV43Q, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749938690 (6h21m39.339283677s) [noise=true , address=104.36.82.59:4313 , piece_id=KDH6AVALUMDXB6USO7WG77DVOSRBTGSZA6VTJFFQJR7QFC2PE35A, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750024905 (6h21m39.339369892s) [noise=true , address=104.36.82.23:4455 , piece_id=UJOJ7P3VET4XIM257B3SHAN6SJG2H64QJHV6PYU7OEB7PPBZVDAA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750215089 (6h21m39.339560076s) [noise=true , address=104.36.82.61:4387 , piece_id=MAWGNSJTYYH7EVPDIXQEG6KP3MKACSMWYVMMLYUASUDJBIX5POOA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750282639 (6h21m39.339627626s) [noise=true , address=45.91.116.10:4092 , piece_id=BUVKJC3K3RKZZTW2CQPEQM7ECQOLIBX4CU66P26JZDTOQZLYGMGA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750481038 (6h21m39.339826025s) [noise=true , address=79.127.220.97:4396 , piece_id=X2CCRNYWY5ZQOAODHNJE7VLYQIHD37PEE7MQXI5WIYV36ICONILQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750533900 (6h21m39.339878887s) [noise=true , address=104.36.82.60:4369 , piece_id=Y6SYJOB7SYBJX7STC25OJJPD4FSK5TZ5KXI7WN7QS42UNTOF5BCQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750725967 (6h21m39.340070954s) [noise=true , address=104.36.82.9:4039 , piece_id=PRBG5EVKAZ3LGDRX4MCI57T4RSNFVKZVUNOAIRHBM6SNS5XXUTPQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750795189 (6h21m39.340140176s) [noise=true , address=104.36.82.22:4525 , piece_id=NKD4EXV774KNXLEHICE4MNKOQS5QKPIUTJL2PPCNYUMYPMQEF75A, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750954023 (6h21m39.34029901s) [noise=true , address=104.36.82.45:4912 , piece_id=B6HTS6ORSGMUKZO5G2ALW4LVUSIBTCEHPX2YJTYPPCH3XROIO3HA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.751098079 (6h21m39.340443066s) [noise=true , address=104.36.82.8:4992 , piece_id=226HGROQ3DBKTJB3U6TG2OF4JQRYAGTLMSIL2JIL4NY5PISP4RUQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.751175397 (6h21m39.340520384s) [noise=true , address=79.127.205.233:4631 , piece_id=77WNKENAMS4Q75KYK4F4VNRP4DD4MKGEJIU5K7WZ3EHVQLEW6Q5A, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.751392602 (6h21m39.340737579s) [noise=true , address=104.36.82.5:4845 , piece_id=JKN4VFS7ET2LVGHGE4LIBPCGG33GNY4SGSOXAWB2MXFYPJVO4FCQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:40.878402060 (6h21m40.467747047s) [upload=6610]: Next returned nil segment
Aug 7 16:01:40.878416708 (6h21m40.467761695s) [upload=6610]: Waiting for error group managing segments...
Aug 7 16:01:41.195597254 (6h21m40.784942241s) [noise=true , address=79.127.205.235:4666 , piece_id=WA6GZKNO2UUAC5VKBNOZ7UO25RQR7QDT5HVNQYZDIWJEB656WMGA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.261081133 (6h21m40.85042612s) [noise=true , address=79.127.205.233:4631 , piece_id=77WNKENAMS4Q75KYK4F4VNRP4DD4MKGEJIU5K7WZ3EHVQLEW6Q5A, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.420961069 (6h21m41.010306046s) [noise=true , address=79.127.220.97:4396 , piece_id=X2CCRNYWY5ZQOAODHNJE7VLYQIHD37PEE7MQXI5WIYV36ICONILQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.638341479 (6h21m41.227686486s) [noise=true , address=79.127.201.215:4345 , piece_id=MAAFKAPO4OCQXU5WJ6DAP72KEHIM4AO5JJFJ4ZEFNWKS5M4OV43Q, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.658966771 (6h21m41.248311758s) [noise=true , address=45.91.116.10:4092 , piece_id=BUVKJC3K3RKZZTW2CQPEQM7ECQOLIBX4CU66P26JZDTOQZLYGMGA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.684178459 (6h21m41.273523446s) [noise=true , address=104.36.82.9:4039 , piece_id=PRBG5EVKAZ3LGDRX4MCI57T4RSNFVKZVUNOAIRHBM6SNS5XXUTPQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.691448508 (6h21m41.280793485s) [noise=true , address=104.36.82.22:4525 , piece_id=NKD4EXV774KNXLEHICE4MNKOQS5QKPIUTJL2PPCNYUMYPMQEF75A, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.717147267 (6h21m41.306492254s) [noise=true , address=104.36.82.59:4313 , piece_id=KDH6AVALUMDXB6USO7WG77DVOSRBTGSZA6VTJFFQJR7QFC2PE35A, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.719612885 (6h21m41.308957872s) [noise=true , address=104.36.82.45:4912 , piece_id=B6HTS6ORSGMUKZO5G2ALW4LVUSIBTCEHPX2YJTYPPCH3XROIO3HA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.733973160 (6h21m41.323318147s) [noise=true , address=104.36.82.60:4369 , piece_id=Y6SYJOB7SYBJX7STC25OJJPD4FSK5TZ5KXI7WN7QS42UNTOF5BCQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.737980718 (6h21m41.327325705s) [noise=true , address=104.36.82.5:4845 , piece_id=JKN4VFS7ET2LVGHGE4LIBPCGG33GNY4SGSOXAWB2MXFYPJVO4FCQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.745434729 (6h21m41.334779717s) [noise=true , address=104.36.82.42:4820 , piece_id=XJIHI2E35NJKDFYFTW4V5EIF3CL7GDJ5EKVCTEV2KL26X43A3CYA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.754624732 (6h21m41.34396972s) [noise=true , address=104.36.82.8:4992 , piece_id=226HGROQ3DBKTJB3U6TG2OF4JQRYAGTLMSIL2JIL4NY5PISP4RUQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.763605976 (6h21m41.352950963s) [noise=true , address=104.36.82.60:4345 , piece_id=VPYOQHGEP6E257WI7IGIWXRO5WOLCNGDJYNPFPC56WUMOYDUS6RA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.763623339 (6h21m41.352968336s) [seg_pos={0 0}, upload=6610]: Segment reached optimal threshold of 14 pieces.
Aug 7 16:01:41.763741525 (6h21m41.353086512s) [noise=true , address=104.36.82.10:4110 , piece_id=WAUEFOZXH3SULGTXTWXWLEPYHZJWJDDQRRWWXRBXVWF7WCWPYGYQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ADbnPJGRTw2NqRzhy7ptojGYoNdJgqmjuWBSjcrQK53Wm7LXi): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763803123 (6h21m41.35314811s) [noise=true , address=104.36.82.61:4387 , piece_id=MAWGNSJTYYH7EVPDIXQEG6KP3MKACSMWYVMMLYUASUDJBIX5POOA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P4wAXSyaRY9poPRK5q4td8DJM69xWioG38TTwLTHKHA7QFxLp): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763857136 (6h21m41.353202113s) [noise=true , address=45.91.116.9:4267 , piece_id=MWAYSXFIIYQGDPTLWZSAHZP3UO72EXYDSSULHBJJU54H5OWG36NA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6M5TgGqUwjXyM25uqv5hsh67mS81ByzMM4kWFUNaFvisDN9d): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763904968 (6h21m41.353249955s) [noise=true , address=104.36.82.23:4455 , piece_id=UJOJ7P3VET4XIM257B3SHAN6SJG2H64QJHV6PYU7OEB7PPBZVDAA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wsyjF5X3iPrJsezUjakUArxc8K88T36yT4rE3yu6q9BMKfycm): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763945405 (6h21m41.353290392s) [seg_pos={0 0}, upload=6610]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 16:01:41.764033824 (6h21m41.353378811s) [upload=6610]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 16:01:41.764038263 (6h21m41.35338325s) [upload=6610]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 16:01:42.043459384 (6h21m41.632804371s) [upload=6610]: Done uploading segments...
Aug 7 16:01:42.043475946 (6h21m41.632820933s) [upload=6610]: Upload finished. err: <nil>
Aug 7 16:02:11.915996790 (6h22m11.505341777s) [upload=6619]: Starting upload
Aug 7 16:02:11.916045183 (6h22m11.50539017s) [upload=6619]: Done starting upload
Aug 7 16:02:11.916152489 (6h22m11.505497476s) [upload=6619]: Uploading segments...
Aug 7 16:02:11.916287706 (6h22m11.505632693s) [upload=6619]: Got next segment. Inline: false
Aug 7 16:02:11.916319939 (6h22m11.505664926s) [upload=6619]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 16:02:11.916324708 (6h22m11.505669695s) [upload=6619]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 16:02:12.129590885 (6h22m11.718935892s) [upload=6619]: Next segment err: metaclient: Too Many Requests
Aug 7 16:02:12.129623196 (6h22m11.718968194s) [upload=6619]: Done uploading segments...
Aug 7 16:02:12.129640310 (6h22m11.718985307s) [upload=6619]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.878889997s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
1.870685488s
FAIL (2025-08-07 16:12:01 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== entMakeInline Aug 7 15:57:10.271829415 (6h17m9.861174403s) [upload=6533]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 15:57:10.587098885 (6h17m10.176443872s) [upload=6533]: Done uploading segments... Aug 7 15:57:10.587115857 (6h17m10.176460835s) [upload=6533]: Upload finished. err: <nil> Aug 7 15:57:48.918901442 (6h17m48.508246429s) [upload=6545]: Starting upload Aug 7 15:57:48.918948701 (6h17m48.508293688s) [upload=6545]: Done starting upload Aug 7 15:57:48.919013486 (6h17m48.508358473s) [upload=6545]: Uploading segments... Aug 7 15:57:48.919055005 (6h17m48.508399982s) [upload=6545]: Got next segment. Inline: true Aug 7 15:57:48.919059032 (6h17m48.508404009s) [upload=6545]: Waiting for error group managing segments... Aug 7 15:57:48.919091585 (6h17m48.508436572s) [upload=6545]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 15:57:48.919095493 (6h17m48.50844048s) [upload=6545]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 15:57:48.919097998 (6h17m48.508442975s) [upload=6545]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 15:57:49.200378977 (6h17m48.789723964s) [upload=6545]: Done uploading segments... Aug 7 15:57:49.200397332 (6h17m48.789742319s) [upload=6545]: Upload finished. err: <nil> Aug 7 15:58:50.898935036 (6h18m50.488280023s) [upload=6562]: Starting upload Aug 7 15:58:50.898979992 (6h18m50.488324979s) [upload=6562]: Done starting upload Aug 7 15:58:50.899026682 (6h18m50.488371669s) [upload=6562]: Uploading segments... Aug 7 15:58:50.899065355 (6h18m50.488410343s) [upload=6562]: Got next segment. Inline: true Aug 7 15:58:50.899070335 (6h18m50.488415322s) [upload=6562]: Waiting for error group managing segments... Aug 7 15:58:50.899108268 (6h18m50.488453255s) [upload=6562]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 15:58:50.899112326 (6h18m50.488457313s) [upload=6562]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 15:58:50.899114820 (6h18m50.488459807s) [upload=6562]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 15:58:51.203289783 (6h18m50.79263478s) [upload=6562]: Done uploading segments... Aug 7 15:58:51.203312587 (6h18m50.792657624s) [upload=6562]: Upload finished. err: <nil> Aug 7 16:00:00.470970125 (6h20m0.060315112s) [upload=6580]: Starting upload Aug 7 16:00:00.471017334 (6h20m0.060362321s) [upload=6580]: Done starting upload Aug 7 16:00:00.471072199 (6h20m0.060417186s) [upload=6580]: Uploading segments... Aug 7 16:00:00.471112327 (6h20m0.060457314s) [upload=6580]: Got next segment. Inline: true Aug 7 16:00:00.471116364 (6h20m0.060461351s) [upload=6580]: Waiting for error group managing segments... Aug 7 16:00:00.471149427 (6h20m0.060494414s) [upload=6580]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:00:00.471153305 (6h20m0.060498292s) [upload=6580]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:00:00.471155819 (6h20m0.060500806s) [upload=6580]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:00:00.739624026 (6h20m0.328969023s) [upload=6580]: Done uploading segments... Aug 7 16:00:00.739645636 (6h20m0.328990623s) [upload=6580]: Upload finished. err: <nil> Aug 7 16:00:49.882359165 (6h20m49.471704152s) [upload=6598]: Starting upload Aug 7 16:00:49.882404682 (6h20m49.471749659s) [upload=6598]: Done starting upload Aug 7 16:00:49.882456812 (6h20m49.471801799s) [upload=6598]: Uploading segments... Aug 7 16:00:49.882489865 (6h20m49.471834852s) [upload=6598]: Got next segment. Inline: true Aug 7 16:00:49.882493702 (6h20m49.47183869s) [upload=6598]: Waiting for error group managing segments... Aug 7 16:00:49.882525894 (6h20m49.471870881s) [upload=6598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:00:49.882529422 (6h20m49.471874399s) [upload=6598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:00:49.882531676 (6h20m49.471876663s) [upload=6598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:00:50.213720013 (6h20m49.803065s) [upload=6598]: Done uploading segments... Aug 7 16:00:50.213739550 (6h20m49.803084537s) [upload=6598]: Upload finished. err: <nil> Aug 7 16:01:51.762778733 (6h21m51.35212372s) [upload=6617]: Starting upload Aug 7 16:01:51.762856161 (6h21m51.352201148s) [upload=6617]: Done starting upload Aug 7 16:01:51.762914883 (6h21m51.35225987s) [upload=6617]: Uploading segments... Aug 7 16:01:51.762952445 (6h21m51.352297432s) [upload=6617]: Got next segment. Inline: true Aug 7 16:01:51.762957344 (6h21m51.352302331s) [upload=6617]: Waiting for error group managing segments... Aug 7 16:01:51.762994485 (6h21m51.352339473s) [upload=6617]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:01:51.762998313 (6h21m51.3523433s) [upload=6617]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:01:51.763001278 (6h21m51.352346265s) [upload=6617]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:01:52.034377407 (6h21m51.623722394s) [upload=6617]: Done uploading segments... Aug 7 16:01:52.034395451 (6h21m51.623740439s) [upload=6617]: Upload finished. err: <nil> Aug 7 16:02:11.154406473 (6h22m10.74375146s) [upload=6618]: Starting upload Aug 7 16:02:11.154456328 (6h22m10.743801315s) [upload=6618]: Done starting upload Aug 7 16:02:11.154503859 (6h22m10.743848846s) [upload=6618]: Uploading segments... Aug 7 16:02:11.154545308 (6h22m10.743890295s) [upload=6618]: Got next segment. Inline: true Aug 7 16:02:11.154548574 (6h22m10.743893561s) [upload=6618]: Waiting for error group managing segments... Aug 7 16:02:11.154580566 (6h22m10.743925553s) [upload=6618]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:02:11.154584283 (6h22m10.74392927s) [upload=6618]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:02:11.154586828 (6h22m10.743931815s) [upload=6618]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:02:11.475329609 (6h22m11.064674596s) [upload=6618]: Done uploading segments... Aug 7 16:02:11.475484125 (6h22m11.064829112s) [upload=6618]: Upload finished. err: <nil> Aug 7 16:02:42.941029250 (6h22m42.530374238s) [upload=6628]: Starting upload Aug 7 16:02:42.941096740 (6h22m42.530441727s) [upload=6628]: Done starting upload Aug 7 16:02:42.941158728 (6h22m42.530503716s) [upload=6628]: Uploading segments... Aug 7 16:02:42.941210478 (6h22m42.530555465s) [upload=6628]: Got next segment. Inline: true Aug 7 16:02:42.941216419 (6h22m42.530561416s) [upload=6628]: Waiting for error group managing segments... Aug 7 16:02:42.941281664 (6h22m42.530626651s) [upload=6628]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:02:42.941286824 (6h22m42.530631811s) [upload=6628]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:02:42.941289880 (6h22m42.530634867s) [upload=6628]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:02:43.233814342 (6h22m42.823159339s) [upload=6628]: Done uploading segments... Aug 7 16:02:43.233838148 (6h22m42.823183145s) [upload=6628]: Upload finished. err: <nil> Aug 7 16:03:45.201690833 (6h23m44.79103582s) [upload=6651]: Starting upload Aug 7 16:03:45.201738173 (6h23m44.79108316s) [upload=6651]: Done starting upload Aug 7 16:03:45.201785534 (6h23m44.791130511s) [upload=6651]: Uploading segments... Aug 7 16:03:45.201855618 (6h23m44.791200605s) [upload=6651]: Got next segment. Inline: true Aug 7 16:03:45.201859616 (6h23m44.791204603s) [upload=6651]: Waiting for error group managing segments... Aug 7 16:03:45.201894122 (6h23m44.791239109s) [upload=6651]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:03:45.201897799 (6h23m44.791242786s) [upload=6651]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:03:45.201900023 (6h23m44.79124501s) [upload=6651]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:03:45.644503676 (6h23m45.233848664s) [upload=6651]: Done uploading segments... Aug 7 16:03:45.644521010 (6h23m45.233865987s) [upload=6651]: Upload finished. err: <nil> Aug 7 16:04:31.274936212 (6h24m30.864281199s) [upload=6663]: Starting upload Aug 7 16:04:31.274977170 (6h24m30.864322157s) [upload=6663]: Done starting upload Aug 7 16:04:31.275026655 (6h24m30.864371642s) [upload=6663]: Uploading segments... Aug 7 16:04:31.275054187 (6h24m30.864399174s) [upload=6663]: Got next segment. Inline: true Aug 7 16:04:31.275057754 (6h24m30.864402731s) [upload=6663]: Waiting for error group managing segments... Aug 7 16:04:31.275091278 (6h24m30.864436265s) [upload=6663]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:04:31.275095255 (6h24m30.864440242s) [upload=6663]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:04:31.275097651 (6h24m30.864442638s) [upload=6663]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:04:31.554258512 (6h24m31.143603499s) [upload=6663]: Done uploading segments... Aug 7 16:04:31.554277108 (6h24m31.143622095s) [upload=6663]: Upload finished. err: <nil> Aug 7 16:05:14.730035713 (6h25m14.3193807s) [upload=6674]: Starting upload Aug 7 16:05:14.730082813 (6h25m14.3194278s) [upload=6674]: Done starting upload Aug 7 16:05:14.730159079 (6h25m14.319504056s) [upload=6674]: Uploading segments... Aug 7 16:05:14.730249532 (6h25m14.319594519s) [upload=6674]: Got next segment. Inline: true Aug 7 16:05:14.730254050 (6h25m14.319599037s) [upload=6674]: Waiting for error group managing segments... Aug 7 16:05:14.730294949 (6h25m14.319639936s) [upload=6674]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:05:14.730299728 (6h25m14.319644715s) [upload=6674]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:05:14.730302243 (6h25m14.31964723s) [upload=6674]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:05:14.942773689 (6h25m14.532118666s) [upload=6674]: Done uploading segments... Aug 7 16:05:14.942790141 (6h25m14.532135118s) [upload=6674]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.870710325s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [.....................................X.......X..............] duration:
5.749068533s
FAIL (2025-08-07 16:10:56 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
e begin upload segment.
Aug 7 16:01:39.748945470 (6h21m39.338290447s) [noise=true , address=104.36.82.42:4820 , piece_id=XJIHI2E35NJKDFYFTW4V5EIF3CL7GDJ5EKVCTEV2KL26X43A3CYA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.748969657 (6h21m39.338314644s) [noise=true , address=104.36.82.60:4345 , piece_id=VPYOQHGEP6E257WI7IGIWXRO5WOLCNGDJYNPFPC56WUMOYDUS6RA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749368329 (6h21m39.338713316s) [noise=true , address=45.91.116.9:4267 , piece_id=MWAYSXFIIYQGDPTLWZSAHZP3UO72EXYDSSULHBJJU54H5OWG36NA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749375964 (6h21m39.338720951s) [noise=true , address=104.36.82.10:4110 , piece_id=WAUEFOZXH3SULGTXTWXWLEPYHZJWJDDQRRWWXRBXVWF7WCWPYGYQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749660398 (6h21m39.339005385s) [noise=true , address=79.127.205.235:4666 , piece_id=WA6GZKNO2UUAC5VKBNOZ7UO25RQR7QDT5HVNQYZDIWJEB656WMGA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749701216 (6h21m39.339046203s) [noise=true , address=79.127.201.215:4345 , piece_id=MAAFKAPO4OCQXU5WJ6DAP72KEHIM4AO5JJFJ4ZEFNWKS5M4OV43Q, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749938690 (6h21m39.339283677s) [noise=true , address=104.36.82.59:4313 , piece_id=KDH6AVALUMDXB6USO7WG77DVOSRBTGSZA6VTJFFQJR7QFC2PE35A, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750024905 (6h21m39.339369892s) [noise=true , address=104.36.82.23:4455 , piece_id=UJOJ7P3VET4XIM257B3SHAN6SJG2H64QJHV6PYU7OEB7PPBZVDAA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750215089 (6h21m39.339560076s) [noise=true , address=104.36.82.61:4387 , piece_id=MAWGNSJTYYH7EVPDIXQEG6KP3MKACSMWYVMMLYUASUDJBIX5POOA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750282639 (6h21m39.339627626s) [noise=true , address=45.91.116.10:4092 , piece_id=BUVKJC3K3RKZZTW2CQPEQM7ECQOLIBX4CU66P26JZDTOQZLYGMGA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750481038 (6h21m39.339826025s) [noise=true , address=79.127.220.97:4396 , piece_id=X2CCRNYWY5ZQOAODHNJE7VLYQIHD37PEE7MQXI5WIYV36ICONILQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750533900 (6h21m39.339878887s) [noise=true , address=104.36.82.60:4369 , piece_id=Y6SYJOB7SYBJX7STC25OJJPD4FSK5TZ5KXI7WN7QS42UNTOF5BCQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750725967 (6h21m39.340070954s) [noise=true , address=104.36.82.9:4039 , piece_id=PRBG5EVKAZ3LGDRX4MCI57T4RSNFVKZVUNOAIRHBM6SNS5XXUTPQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750795189 (6h21m39.340140176s) [noise=true , address=104.36.82.22:4525 , piece_id=NKD4EXV774KNXLEHICE4MNKOQS5QKPIUTJL2PPCNYUMYPMQEF75A, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750954023 (6h21m39.34029901s) [noise=true , address=104.36.82.45:4912 , piece_id=B6HTS6ORSGMUKZO5G2ALW4LVUSIBTCEHPX2YJTYPPCH3XROIO3HA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.751098079 (6h21m39.340443066s) [noise=true , address=104.36.82.8:4992 , piece_id=226HGROQ3DBKTJB3U6TG2OF4JQRYAGTLMSIL2JIL4NY5PISP4RUQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.751175397 (6h21m39.340520384s) [noise=true , address=79.127.205.233:4631 , piece_id=77WNKENAMS4Q75KYK4F4VNRP4DD4MKGEJIU5K7WZ3EHVQLEW6Q5A, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.751392602 (6h21m39.340737579s) [noise=true , address=104.36.82.5:4845 , piece_id=JKN4VFS7ET2LVGHGE4LIBPCGG33GNY4SGSOXAWB2MXFYPJVO4FCQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:40.878402060 (6h21m40.467747047s) [upload=6610]: Next returned nil segment
Aug 7 16:01:40.878416708 (6h21m40.467761695s) [upload=6610]: Waiting for error group managing segments...
Aug 7 16:01:41.195597254 (6h21m40.784942241s) [noise=true , address=79.127.205.235:4666 , piece_id=WA6GZKNO2UUAC5VKBNOZ7UO25RQR7QDT5HVNQYZDIWJEB656WMGA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.261081133 (6h21m40.85042612s) [noise=true , address=79.127.205.233:4631 , piece_id=77WNKENAMS4Q75KYK4F4VNRP4DD4MKGEJIU5K7WZ3EHVQLEW6Q5A, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.420961069 (6h21m41.010306046s) [noise=true , address=79.127.220.97:4396 , piece_id=X2CCRNYWY5ZQOAODHNJE7VLYQIHD37PEE7MQXI5WIYV36ICONILQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.638341479 (6h21m41.227686486s) [noise=true , address=79.127.201.215:4345 , piece_id=MAAFKAPO4OCQXU5WJ6DAP72KEHIM4AO5JJFJ4ZEFNWKS5M4OV43Q, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.658966771 (6h21m41.248311758s) [noise=true , address=45.91.116.10:4092 , piece_id=BUVKJC3K3RKZZTW2CQPEQM7ECQOLIBX4CU66P26JZDTOQZLYGMGA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.684178459 (6h21m41.273523446s) [noise=true , address=104.36.82.9:4039 , piece_id=PRBG5EVKAZ3LGDRX4MCI57T4RSNFVKZVUNOAIRHBM6SNS5XXUTPQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.691448508 (6h21m41.280793485s) [noise=true , address=104.36.82.22:4525 , piece_id=NKD4EXV774KNXLEHICE4MNKOQS5QKPIUTJL2PPCNYUMYPMQEF75A, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.717147267 (6h21m41.306492254s) [noise=true , address=104.36.82.59:4313 , piece_id=KDH6AVALUMDXB6USO7WG77DVOSRBTGSZA6VTJFFQJR7QFC2PE35A, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.719612885 (6h21m41.308957872s) [noise=true , address=104.36.82.45:4912 , piece_id=B6HTS6ORSGMUKZO5G2ALW4LVUSIBTCEHPX2YJTYPPCH3XROIO3HA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.733973160 (6h21m41.323318147s) [noise=true , address=104.36.82.60:4369 , piece_id=Y6SYJOB7SYBJX7STC25OJJPD4FSK5TZ5KXI7WN7QS42UNTOF5BCQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.737980718 (6h21m41.327325705s) [noise=true , address=104.36.82.5:4845 , piece_id=JKN4VFS7ET2LVGHGE4LIBPCGG33GNY4SGSOXAWB2MXFYPJVO4FCQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.745434729 (6h21m41.334779717s) [noise=true , address=104.36.82.42:4820 , piece_id=XJIHI2E35NJKDFYFTW4V5EIF3CL7GDJ5EKVCTEV2KL26X43A3CYA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.754624732 (6h21m41.34396972s) [noise=true , address=104.36.82.8:4992 , piece_id=226HGROQ3DBKTJB3U6TG2OF4JQRYAGTLMSIL2JIL4NY5PISP4RUQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.763605976 (6h21m41.352950963s) [noise=true , address=104.36.82.60:4345 , piece_id=VPYOQHGEP6E257WI7IGIWXRO5WOLCNGDJYNPFPC56WUMOYDUS6RA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.763623339 (6h21m41.352968336s) [seg_pos={0 0}, upload=6610]: Segment reached optimal threshold of 14 pieces.
Aug 7 16:01:41.763741525 (6h21m41.353086512s) [noise=true , address=104.36.82.10:4110 , piece_id=WAUEFOZXH3SULGTXTWXWLEPYHZJWJDDQRRWWXRBXVWF7WCWPYGYQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ADbnPJGRTw2NqRzhy7ptojGYoNdJgqmjuWBSjcrQK53Wm7LXi): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763803123 (6h21m41.35314811s) [noise=true , address=104.36.82.61:4387 , piece_id=MAWGNSJTYYH7EVPDIXQEG6KP3MKACSMWYVMMLYUASUDJBIX5POOA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P4wAXSyaRY9poPRK5q4td8DJM69xWioG38TTwLTHKHA7QFxLp): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763857136 (6h21m41.353202113s) [noise=true , address=45.91.116.9:4267 , piece_id=MWAYSXFIIYQGDPTLWZSAHZP3UO72EXYDSSULHBJJU54H5OWG36NA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6M5TgGqUwjXyM25uqv5hsh67mS81ByzMM4kWFUNaFvisDN9d): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763904968 (6h21m41.353249955s) [noise=true , address=104.36.82.23:4455 , piece_id=UJOJ7P3VET4XIM257B3SHAN6SJG2H64QJHV6PYU7OEB7PPBZVDAA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wsyjF5X3iPrJsezUjakUArxc8K88T36yT4rE3yu6q9BMKfycm): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763945405 (6h21m41.353290392s) [seg_pos={0 0}, upload=6610]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 16:01:41.764033824 (6h21m41.353378811s) [upload=6610]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 16:01:41.764038263 (6h21m41.35338325s) [upload=6610]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 16:01:42.043459384 (6h21m41.632804371s) [upload=6610]: Done uploading segments...
Aug 7 16:01:42.043475946 (6h21m41.632820933s) [upload=6610]: Upload finished. err: <nil>
Aug 7 16:02:11.915996790 (6h22m11.505341777s) [upload=6619]: Starting upload
Aug 7 16:02:11.916045183 (6h22m11.50539017s) [upload=6619]: Done starting upload
Aug 7 16:02:11.916152489 (6h22m11.505497476s) [upload=6619]: Uploading segments...
Aug 7 16:02:11.916287706 (6h22m11.505632693s) [upload=6619]: Got next segment. Inline: false
Aug 7 16:02:11.916319939 (6h22m11.505664926s) [upload=6619]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 16:02:11.916324708 (6h22m11.505669695s) [upload=6619]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 16:02:12.129590885 (6h22m11.718935892s) [upload=6619]: Next segment err: metaclient: Too Many Requests
Aug 7 16:02:12.129623196 (6h22m11.718968194s) [upload=6619]: Done uploading segments...
Aug 7 16:02:12.129640310 (6h22m11.718985307s) [upload=6619]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.749091977s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
1.958871571s
FAIL (2025-08-07 16:10:53 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== entMakeInline Aug 7 15:57:10.271829415 (6h17m9.861174403s) [upload=6533]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 15:57:10.587098885 (6h17m10.176443872s) [upload=6533]: Done uploading segments... Aug 7 15:57:10.587115857 (6h17m10.176460835s) [upload=6533]: Upload finished. err: <nil> Aug 7 15:57:48.918901442 (6h17m48.508246429s) [upload=6545]: Starting upload Aug 7 15:57:48.918948701 (6h17m48.508293688s) [upload=6545]: Done starting upload Aug 7 15:57:48.919013486 (6h17m48.508358473s) [upload=6545]: Uploading segments... Aug 7 15:57:48.919055005 (6h17m48.508399982s) [upload=6545]: Got next segment. Inline: true Aug 7 15:57:48.919059032 (6h17m48.508404009s) [upload=6545]: Waiting for error group managing segments... Aug 7 15:57:48.919091585 (6h17m48.508436572s) [upload=6545]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 15:57:48.919095493 (6h17m48.50844048s) [upload=6545]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 15:57:48.919097998 (6h17m48.508442975s) [upload=6545]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 15:57:49.200378977 (6h17m48.789723964s) [upload=6545]: Done uploading segments... Aug 7 15:57:49.200397332 (6h17m48.789742319s) [upload=6545]: Upload finished. err: <nil> Aug 7 15:58:50.898935036 (6h18m50.488280023s) [upload=6562]: Starting upload Aug 7 15:58:50.898979992 (6h18m50.488324979s) [upload=6562]: Done starting upload Aug 7 15:58:50.899026682 (6h18m50.488371669s) [upload=6562]: Uploading segments... Aug 7 15:58:50.899065355 (6h18m50.488410343s) [upload=6562]: Got next segment. Inline: true Aug 7 15:58:50.899070335 (6h18m50.488415322s) [upload=6562]: Waiting for error group managing segments... Aug 7 15:58:50.899108268 (6h18m50.488453255s) [upload=6562]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 15:58:50.899112326 (6h18m50.488457313s) [upload=6562]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 15:58:50.899114820 (6h18m50.488459807s) [upload=6562]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 15:58:51.203289783 (6h18m50.79263478s) [upload=6562]: Done uploading segments... Aug 7 15:58:51.203312587 (6h18m50.792657624s) [upload=6562]: Upload finished. err: <nil> Aug 7 16:00:00.470970125 (6h20m0.060315112s) [upload=6580]: Starting upload Aug 7 16:00:00.471017334 (6h20m0.060362321s) [upload=6580]: Done starting upload Aug 7 16:00:00.471072199 (6h20m0.060417186s) [upload=6580]: Uploading segments... Aug 7 16:00:00.471112327 (6h20m0.060457314s) [upload=6580]: Got next segment. Inline: true Aug 7 16:00:00.471116364 (6h20m0.060461351s) [upload=6580]: Waiting for error group managing segments... Aug 7 16:00:00.471149427 (6h20m0.060494414s) [upload=6580]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:00:00.471153305 (6h20m0.060498292s) [upload=6580]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:00:00.471155819 (6h20m0.060500806s) [upload=6580]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:00:00.739624026 (6h20m0.328969023s) [upload=6580]: Done uploading segments... Aug 7 16:00:00.739645636 (6h20m0.328990623s) [upload=6580]: Upload finished. err: <nil> Aug 7 16:00:49.882359165 (6h20m49.471704152s) [upload=6598]: Starting upload Aug 7 16:00:49.882404682 (6h20m49.471749659s) [upload=6598]: Done starting upload Aug 7 16:00:49.882456812 (6h20m49.471801799s) [upload=6598]: Uploading segments... Aug 7 16:00:49.882489865 (6h20m49.471834852s) [upload=6598]: Got next segment. Inline: true Aug 7 16:00:49.882493702 (6h20m49.47183869s) [upload=6598]: Waiting for error group managing segments... Aug 7 16:00:49.882525894 (6h20m49.471870881s) [upload=6598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:00:49.882529422 (6h20m49.471874399s) [upload=6598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:00:49.882531676 (6h20m49.471876663s) [upload=6598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:00:50.213720013 (6h20m49.803065s) [upload=6598]: Done uploading segments... Aug 7 16:00:50.213739550 (6h20m49.803084537s) [upload=6598]: Upload finished. err: <nil> Aug 7 16:01:51.762778733 (6h21m51.35212372s) [upload=6617]: Starting upload Aug 7 16:01:51.762856161 (6h21m51.352201148s) [upload=6617]: Done starting upload Aug 7 16:01:51.762914883 (6h21m51.35225987s) [upload=6617]: Uploading segments... Aug 7 16:01:51.762952445 (6h21m51.352297432s) [upload=6617]: Got next segment. Inline: true Aug 7 16:01:51.762957344 (6h21m51.352302331s) [upload=6617]: Waiting for error group managing segments... Aug 7 16:01:51.762994485 (6h21m51.352339473s) [upload=6617]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:01:51.762998313 (6h21m51.3523433s) [upload=6617]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:01:51.763001278 (6h21m51.352346265s) [upload=6617]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:01:52.034377407 (6h21m51.623722394s) [upload=6617]: Done uploading segments... Aug 7 16:01:52.034395451 (6h21m51.623740439s) [upload=6617]: Upload finished. err: <nil> Aug 7 16:02:11.154406473 (6h22m10.74375146s) [upload=6618]: Starting upload Aug 7 16:02:11.154456328 (6h22m10.743801315s) [upload=6618]: Done starting upload Aug 7 16:02:11.154503859 (6h22m10.743848846s) [upload=6618]: Uploading segments... Aug 7 16:02:11.154545308 (6h22m10.743890295s) [upload=6618]: Got next segment. Inline: true Aug 7 16:02:11.154548574 (6h22m10.743893561s) [upload=6618]: Waiting for error group managing segments... Aug 7 16:02:11.154580566 (6h22m10.743925553s) [upload=6618]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:02:11.154584283 (6h22m10.74392927s) [upload=6618]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:02:11.154586828 (6h22m10.743931815s) [upload=6618]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:02:11.475329609 (6h22m11.064674596s) [upload=6618]: Done uploading segments... Aug 7 16:02:11.475484125 (6h22m11.064829112s) [upload=6618]: Upload finished. err: <nil> Aug 7 16:02:42.941029250 (6h22m42.530374238s) [upload=6628]: Starting upload Aug 7 16:02:42.941096740 (6h22m42.530441727s) [upload=6628]: Done starting upload Aug 7 16:02:42.941158728 (6h22m42.530503716s) [upload=6628]: Uploading segments... Aug 7 16:02:42.941210478 (6h22m42.530555465s) [upload=6628]: Got next segment. Inline: true Aug 7 16:02:42.941216419 (6h22m42.530561416s) [upload=6628]: Waiting for error group managing segments... Aug 7 16:02:42.941281664 (6h22m42.530626651s) [upload=6628]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:02:42.941286824 (6h22m42.530631811s) [upload=6628]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:02:42.941289880 (6h22m42.530634867s) [upload=6628]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:02:43.233814342 (6h22m42.823159339s) [upload=6628]: Done uploading segments... Aug 7 16:02:43.233838148 (6h22m42.823183145s) [upload=6628]: Upload finished. err: <nil> Aug 7 16:03:45.201690833 (6h23m44.79103582s) [upload=6651]: Starting upload Aug 7 16:03:45.201738173 (6h23m44.79108316s) [upload=6651]: Done starting upload Aug 7 16:03:45.201785534 (6h23m44.791130511s) [upload=6651]: Uploading segments... Aug 7 16:03:45.201855618 (6h23m44.791200605s) [upload=6651]: Got next segment. Inline: true Aug 7 16:03:45.201859616 (6h23m44.791204603s) [upload=6651]: Waiting for error group managing segments... Aug 7 16:03:45.201894122 (6h23m44.791239109s) [upload=6651]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:03:45.201897799 (6h23m44.791242786s) [upload=6651]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:03:45.201900023 (6h23m44.79124501s) [upload=6651]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:03:45.644503676 (6h23m45.233848664s) [upload=6651]: Done uploading segments... Aug 7 16:03:45.644521010 (6h23m45.233865987s) [upload=6651]: Upload finished. err: <nil> Aug 7 16:04:31.274936212 (6h24m30.864281199s) [upload=6663]: Starting upload Aug 7 16:04:31.274977170 (6h24m30.864322157s) [upload=6663]: Done starting upload Aug 7 16:04:31.275026655 (6h24m30.864371642s) [upload=6663]: Uploading segments... Aug 7 16:04:31.275054187 (6h24m30.864399174s) [upload=6663]: Got next segment. Inline: true Aug 7 16:04:31.275057754 (6h24m30.864402731s) [upload=6663]: Waiting for error group managing segments... Aug 7 16:04:31.275091278 (6h24m30.864436265s) [upload=6663]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:04:31.275095255 (6h24m30.864440242s) [upload=6663]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:04:31.275097651 (6h24m30.864442638s) [upload=6663]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:04:31.554258512 (6h24m31.143603499s) [upload=6663]: Done uploading segments... Aug 7 16:04:31.554277108 (6h24m31.143622095s) [upload=6663]: Upload finished. err: <nil> Aug 7 16:05:14.730035713 (6h25m14.3193807s) [upload=6674]: Starting upload Aug 7 16:05:14.730082813 (6h25m14.3194278s) [upload=6674]: Done starting upload Aug 7 16:05:14.730159079 (6h25m14.319504056s) [upload=6674]: Uploading segments... Aug 7 16:05:14.730249532 (6h25m14.319594519s) [upload=6674]: Got next segment. Inline: true Aug 7 16:05:14.730254050 (6h25m14.319599037s) [upload=6674]: Waiting for error group managing segments... Aug 7 16:05:14.730294949 (6h25m14.319639936s) [upload=6674]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:05:14.730299728 (6h25m14.319644715s) [upload=6674]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:05:14.730302243 (6h25m14.31964723s) [upload=6674]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:05:14.942773689 (6h25m14.532118666s) [upload=6674]: Done uploading segments... Aug 7 16:05:14.942790141 (6h25m14.532135118s) [upload=6674]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.958899185s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [......................................X.......X.............] duration:
6.010846388s
FAIL (2025-08-07 16:10:30 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
e begin upload segment.
Aug 7 16:01:39.748945470 (6h21m39.338290447s) [noise=true , address=104.36.82.42:4820 , piece_id=XJIHI2E35NJKDFYFTW4V5EIF3CL7GDJ5EKVCTEV2KL26X43A3CYA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.748969657 (6h21m39.338314644s) [noise=true , address=104.36.82.60:4345 , piece_id=VPYOQHGEP6E257WI7IGIWXRO5WOLCNGDJYNPFPC56WUMOYDUS6RA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749368329 (6h21m39.338713316s) [noise=true , address=45.91.116.9:4267 , piece_id=MWAYSXFIIYQGDPTLWZSAHZP3UO72EXYDSSULHBJJU54H5OWG36NA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749375964 (6h21m39.338720951s) [noise=true , address=104.36.82.10:4110 , piece_id=WAUEFOZXH3SULGTXTWXWLEPYHZJWJDDQRRWWXRBXVWF7WCWPYGYQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749660398 (6h21m39.339005385s) [noise=true , address=79.127.205.235:4666 , piece_id=WA6GZKNO2UUAC5VKBNOZ7UO25RQR7QDT5HVNQYZDIWJEB656WMGA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749701216 (6h21m39.339046203s) [noise=true , address=79.127.201.215:4345 , piece_id=MAAFKAPO4OCQXU5WJ6DAP72KEHIM4AO5JJFJ4ZEFNWKS5M4OV43Q, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749938690 (6h21m39.339283677s) [noise=true , address=104.36.82.59:4313 , piece_id=KDH6AVALUMDXB6USO7WG77DVOSRBTGSZA6VTJFFQJR7QFC2PE35A, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750024905 (6h21m39.339369892s) [noise=true , address=104.36.82.23:4455 , piece_id=UJOJ7P3VET4XIM257B3SHAN6SJG2H64QJHV6PYU7OEB7PPBZVDAA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750215089 (6h21m39.339560076s) [noise=true , address=104.36.82.61:4387 , piece_id=MAWGNSJTYYH7EVPDIXQEG6KP3MKACSMWYVMMLYUASUDJBIX5POOA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750282639 (6h21m39.339627626s) [noise=true , address=45.91.116.10:4092 , piece_id=BUVKJC3K3RKZZTW2CQPEQM7ECQOLIBX4CU66P26JZDTOQZLYGMGA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750481038 (6h21m39.339826025s) [noise=true , address=79.127.220.97:4396 , piece_id=X2CCRNYWY5ZQOAODHNJE7VLYQIHD37PEE7MQXI5WIYV36ICONILQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750533900 (6h21m39.339878887s) [noise=true , address=104.36.82.60:4369 , piece_id=Y6SYJOB7SYBJX7STC25OJJPD4FSK5TZ5KXI7WN7QS42UNTOF5BCQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750725967 (6h21m39.340070954s) [noise=true , address=104.36.82.9:4039 , piece_id=PRBG5EVKAZ3LGDRX4MCI57T4RSNFVKZVUNOAIRHBM6SNS5XXUTPQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750795189 (6h21m39.340140176s) [noise=true , address=104.36.82.22:4525 , piece_id=NKD4EXV774KNXLEHICE4MNKOQS5QKPIUTJL2PPCNYUMYPMQEF75A, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750954023 (6h21m39.34029901s) [noise=true , address=104.36.82.45:4912 , piece_id=B6HTS6ORSGMUKZO5G2ALW4LVUSIBTCEHPX2YJTYPPCH3XROIO3HA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.751098079 (6h21m39.340443066s) [noise=true , address=104.36.82.8:4992 , piece_id=226HGROQ3DBKTJB3U6TG2OF4JQRYAGTLMSIL2JIL4NY5PISP4RUQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.751175397 (6h21m39.340520384s) [noise=true , address=79.127.205.233:4631 , piece_id=77WNKENAMS4Q75KYK4F4VNRP4DD4MKGEJIU5K7WZ3EHVQLEW6Q5A, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.751392602 (6h21m39.340737579s) [noise=true , address=104.36.82.5:4845 , piece_id=JKN4VFS7ET2LVGHGE4LIBPCGG33GNY4SGSOXAWB2MXFYPJVO4FCQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:40.878402060 (6h21m40.467747047s) [upload=6610]: Next returned nil segment
Aug 7 16:01:40.878416708 (6h21m40.467761695s) [upload=6610]: Waiting for error group managing segments...
Aug 7 16:01:41.195597254 (6h21m40.784942241s) [noise=true , address=79.127.205.235:4666 , piece_id=WA6GZKNO2UUAC5VKBNOZ7UO25RQR7QDT5HVNQYZDIWJEB656WMGA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.261081133 (6h21m40.85042612s) [noise=true , address=79.127.205.233:4631 , piece_id=77WNKENAMS4Q75KYK4F4VNRP4DD4MKGEJIU5K7WZ3EHVQLEW6Q5A, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.420961069 (6h21m41.010306046s) [noise=true , address=79.127.220.97:4396 , piece_id=X2CCRNYWY5ZQOAODHNJE7VLYQIHD37PEE7MQXI5WIYV36ICONILQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.638341479 (6h21m41.227686486s) [noise=true , address=79.127.201.215:4345 , piece_id=MAAFKAPO4OCQXU5WJ6DAP72KEHIM4AO5JJFJ4ZEFNWKS5M4OV43Q, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.658966771 (6h21m41.248311758s) [noise=true , address=45.91.116.10:4092 , piece_id=BUVKJC3K3RKZZTW2CQPEQM7ECQOLIBX4CU66P26JZDTOQZLYGMGA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.684178459 (6h21m41.273523446s) [noise=true , address=104.36.82.9:4039 , piece_id=PRBG5EVKAZ3LGDRX4MCI57T4RSNFVKZVUNOAIRHBM6SNS5XXUTPQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.691448508 (6h21m41.280793485s) [noise=true , address=104.36.82.22:4525 , piece_id=NKD4EXV774KNXLEHICE4MNKOQS5QKPIUTJL2PPCNYUMYPMQEF75A, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.717147267 (6h21m41.306492254s) [noise=true , address=104.36.82.59:4313 , piece_id=KDH6AVALUMDXB6USO7WG77DVOSRBTGSZA6VTJFFQJR7QFC2PE35A, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.719612885 (6h21m41.308957872s) [noise=true , address=104.36.82.45:4912 , piece_id=B6HTS6ORSGMUKZO5G2ALW4LVUSIBTCEHPX2YJTYPPCH3XROIO3HA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.733973160 (6h21m41.323318147s) [noise=true , address=104.36.82.60:4369 , piece_id=Y6SYJOB7SYBJX7STC25OJJPD4FSK5TZ5KXI7WN7QS42UNTOF5BCQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.737980718 (6h21m41.327325705s) [noise=true , address=104.36.82.5:4845 , piece_id=JKN4VFS7ET2LVGHGE4LIBPCGG33GNY4SGSOXAWB2MXFYPJVO4FCQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.745434729 (6h21m41.334779717s) [noise=true , address=104.36.82.42:4820 , piece_id=XJIHI2E35NJKDFYFTW4V5EIF3CL7GDJ5EKVCTEV2KL26X43A3CYA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.754624732 (6h21m41.34396972s) [noise=true , address=104.36.82.8:4992 , piece_id=226HGROQ3DBKTJB3U6TG2OF4JQRYAGTLMSIL2JIL4NY5PISP4RUQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.763605976 (6h21m41.352950963s) [noise=true , address=104.36.82.60:4345 , piece_id=VPYOQHGEP6E257WI7IGIWXRO5WOLCNGDJYNPFPC56WUMOYDUS6RA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.763623339 (6h21m41.352968336s) [seg_pos={0 0}, upload=6610]: Segment reached optimal threshold of 14 pieces.
Aug 7 16:01:41.763741525 (6h21m41.353086512s) [noise=true , address=104.36.82.10:4110 , piece_id=WAUEFOZXH3SULGTXTWXWLEPYHZJWJDDQRRWWXRBXVWF7WCWPYGYQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ADbnPJGRTw2NqRzhy7ptojGYoNdJgqmjuWBSjcrQK53Wm7LXi): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763803123 (6h21m41.35314811s) [noise=true , address=104.36.82.61:4387 , piece_id=MAWGNSJTYYH7EVPDIXQEG6KP3MKACSMWYVMMLYUASUDJBIX5POOA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P4wAXSyaRY9poPRK5q4td8DJM69xWioG38TTwLTHKHA7QFxLp): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763857136 (6h21m41.353202113s) [noise=true , address=45.91.116.9:4267 , piece_id=MWAYSXFIIYQGDPTLWZSAHZP3UO72EXYDSSULHBJJU54H5OWG36NA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6M5TgGqUwjXyM25uqv5hsh67mS81ByzMM4kWFUNaFvisDN9d): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763904968 (6h21m41.353249955s) [noise=true , address=104.36.82.23:4455 , piece_id=UJOJ7P3VET4XIM257B3SHAN6SJG2H64QJHV6PYU7OEB7PPBZVDAA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wsyjF5X3iPrJsezUjakUArxc8K88T36yT4rE3yu6q9BMKfycm): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763945405 (6h21m41.353290392s) [seg_pos={0 0}, upload=6610]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 16:01:41.764033824 (6h21m41.353378811s) [upload=6610]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 16:01:41.764038263 (6h21m41.35338325s) [upload=6610]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 16:01:42.043459384 (6h21m41.632804371s) [upload=6610]: Done uploading segments...
Aug 7 16:01:42.043475946 (6h21m41.632820933s) [upload=6610]: Upload finished. err: <nil>
Aug 7 16:02:11.915996790 (6h22m11.505341777s) [upload=6619]: Starting upload
Aug 7 16:02:11.916045183 (6h22m11.50539017s) [upload=6619]: Done starting upload
Aug 7 16:02:11.916152489 (6h22m11.505497476s) [upload=6619]: Uploading segments...
Aug 7 16:02:11.916287706 (6h22m11.505632693s) [upload=6619]: Got next segment. Inline: false
Aug 7 16:02:11.916319939 (6h22m11.505664926s) [upload=6619]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 16:02:11.916324708 (6h22m11.505669695s) [upload=6619]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 16:02:12.129590885 (6h22m11.718935892s) [upload=6619]: Next segment err: metaclient: Too Many Requests
Aug 7 16:02:12.129623196 (6h22m11.718968194s) [upload=6619]: Done uploading segments...
Aug 7 16:02:12.129640310 (6h22m11.718985307s) [upload=6619]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.010872717s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
2.518689321s
FAIL (2025-08-07 16:10:21 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== entMakeInline Aug 7 15:57:10.271829415 (6h17m9.861174403s) [upload=6533]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 15:57:10.587098885 (6h17m10.176443872s) [upload=6533]: Done uploading segments... Aug 7 15:57:10.587115857 (6h17m10.176460835s) [upload=6533]: Upload finished. err: <nil> Aug 7 15:57:48.918901442 (6h17m48.508246429s) [upload=6545]: Starting upload Aug 7 15:57:48.918948701 (6h17m48.508293688s) [upload=6545]: Done starting upload Aug 7 15:57:48.919013486 (6h17m48.508358473s) [upload=6545]: Uploading segments... Aug 7 15:57:48.919055005 (6h17m48.508399982s) [upload=6545]: Got next segment. Inline: true Aug 7 15:57:48.919059032 (6h17m48.508404009s) [upload=6545]: Waiting for error group managing segments... Aug 7 15:57:48.919091585 (6h17m48.508436572s) [upload=6545]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 15:57:48.919095493 (6h17m48.50844048s) [upload=6545]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 15:57:48.919097998 (6h17m48.508442975s) [upload=6545]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 15:57:49.200378977 (6h17m48.789723964s) [upload=6545]: Done uploading segments... Aug 7 15:57:49.200397332 (6h17m48.789742319s) [upload=6545]: Upload finished. err: <nil> Aug 7 15:58:50.898935036 (6h18m50.488280023s) [upload=6562]: Starting upload Aug 7 15:58:50.898979992 (6h18m50.488324979s) [upload=6562]: Done starting upload Aug 7 15:58:50.899026682 (6h18m50.488371669s) [upload=6562]: Uploading segments... Aug 7 15:58:50.899065355 (6h18m50.488410343s) [upload=6562]: Got next segment. Inline: true Aug 7 15:58:50.899070335 (6h18m50.488415322s) [upload=6562]: Waiting for error group managing segments... Aug 7 15:58:50.899108268 (6h18m50.488453255s) [upload=6562]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 15:58:50.899112326 (6h18m50.488457313s) [upload=6562]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 15:58:50.899114820 (6h18m50.488459807s) [upload=6562]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 15:58:51.203289783 (6h18m50.79263478s) [upload=6562]: Done uploading segments... Aug 7 15:58:51.203312587 (6h18m50.792657624s) [upload=6562]: Upload finished. err: <nil> Aug 7 16:00:00.470970125 (6h20m0.060315112s) [upload=6580]: Starting upload Aug 7 16:00:00.471017334 (6h20m0.060362321s) [upload=6580]: Done starting upload Aug 7 16:00:00.471072199 (6h20m0.060417186s) [upload=6580]: Uploading segments... Aug 7 16:00:00.471112327 (6h20m0.060457314s) [upload=6580]: Got next segment. Inline: true Aug 7 16:00:00.471116364 (6h20m0.060461351s) [upload=6580]: Waiting for error group managing segments... Aug 7 16:00:00.471149427 (6h20m0.060494414s) [upload=6580]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:00:00.471153305 (6h20m0.060498292s) [upload=6580]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:00:00.471155819 (6h20m0.060500806s) [upload=6580]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:00:00.739624026 (6h20m0.328969023s) [upload=6580]: Done uploading segments... Aug 7 16:00:00.739645636 (6h20m0.328990623s) [upload=6580]: Upload finished. err: <nil> Aug 7 16:00:49.882359165 (6h20m49.471704152s) [upload=6598]: Starting upload Aug 7 16:00:49.882404682 (6h20m49.471749659s) [upload=6598]: Done starting upload Aug 7 16:00:49.882456812 (6h20m49.471801799s) [upload=6598]: Uploading segments... Aug 7 16:00:49.882489865 (6h20m49.471834852s) [upload=6598]: Got next segment. Inline: true Aug 7 16:00:49.882493702 (6h20m49.47183869s) [upload=6598]: Waiting for error group managing segments... Aug 7 16:00:49.882525894 (6h20m49.471870881s) [upload=6598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:00:49.882529422 (6h20m49.471874399s) [upload=6598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:00:49.882531676 (6h20m49.471876663s) [upload=6598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:00:50.213720013 (6h20m49.803065s) [upload=6598]: Done uploading segments... Aug 7 16:00:50.213739550 (6h20m49.803084537s) [upload=6598]: Upload finished. err: <nil> Aug 7 16:01:51.762778733 (6h21m51.35212372s) [upload=6617]: Starting upload Aug 7 16:01:51.762856161 (6h21m51.352201148s) [upload=6617]: Done starting upload Aug 7 16:01:51.762914883 (6h21m51.35225987s) [upload=6617]: Uploading segments... Aug 7 16:01:51.762952445 (6h21m51.352297432s) [upload=6617]: Got next segment. Inline: true Aug 7 16:01:51.762957344 (6h21m51.352302331s) [upload=6617]: Waiting for error group managing segments... Aug 7 16:01:51.762994485 (6h21m51.352339473s) [upload=6617]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:01:51.762998313 (6h21m51.3523433s) [upload=6617]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:01:51.763001278 (6h21m51.352346265s) [upload=6617]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:01:52.034377407 (6h21m51.623722394s) [upload=6617]: Done uploading segments... Aug 7 16:01:52.034395451 (6h21m51.623740439s) [upload=6617]: Upload finished. err: <nil> Aug 7 16:02:11.154406473 (6h22m10.74375146s) [upload=6618]: Starting upload Aug 7 16:02:11.154456328 (6h22m10.743801315s) [upload=6618]: Done starting upload Aug 7 16:02:11.154503859 (6h22m10.743848846s) [upload=6618]: Uploading segments... Aug 7 16:02:11.154545308 (6h22m10.743890295s) [upload=6618]: Got next segment. Inline: true Aug 7 16:02:11.154548574 (6h22m10.743893561s) [upload=6618]: Waiting for error group managing segments... Aug 7 16:02:11.154580566 (6h22m10.743925553s) [upload=6618]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:02:11.154584283 (6h22m10.74392927s) [upload=6618]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:02:11.154586828 (6h22m10.743931815s) [upload=6618]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:02:11.475329609 (6h22m11.064674596s) [upload=6618]: Done uploading segments... Aug 7 16:02:11.475484125 (6h22m11.064829112s) [upload=6618]: Upload finished. err: <nil> Aug 7 16:02:42.941029250 (6h22m42.530374238s) [upload=6628]: Starting upload Aug 7 16:02:42.941096740 (6h22m42.530441727s) [upload=6628]: Done starting upload Aug 7 16:02:42.941158728 (6h22m42.530503716s) [upload=6628]: Uploading segments... Aug 7 16:02:42.941210478 (6h22m42.530555465s) [upload=6628]: Got next segment. Inline: true Aug 7 16:02:42.941216419 (6h22m42.530561416s) [upload=6628]: Waiting for error group managing segments... Aug 7 16:02:42.941281664 (6h22m42.530626651s) [upload=6628]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:02:42.941286824 (6h22m42.530631811s) [upload=6628]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:02:42.941289880 (6h22m42.530634867s) [upload=6628]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:02:43.233814342 (6h22m42.823159339s) [upload=6628]: Done uploading segments... Aug 7 16:02:43.233838148 (6h22m42.823183145s) [upload=6628]: Upload finished. err: <nil> Aug 7 16:03:45.201690833 (6h23m44.79103582s) [upload=6651]: Starting upload Aug 7 16:03:45.201738173 (6h23m44.79108316s) [upload=6651]: Done starting upload Aug 7 16:03:45.201785534 (6h23m44.791130511s) [upload=6651]: Uploading segments... Aug 7 16:03:45.201855618 (6h23m44.791200605s) [upload=6651]: Got next segment. Inline: true Aug 7 16:03:45.201859616 (6h23m44.791204603s) [upload=6651]: Waiting for error group managing segments... Aug 7 16:03:45.201894122 (6h23m44.791239109s) [upload=6651]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:03:45.201897799 (6h23m44.791242786s) [upload=6651]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:03:45.201900023 (6h23m44.79124501s) [upload=6651]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:03:45.644503676 (6h23m45.233848664s) [upload=6651]: Done uploading segments... Aug 7 16:03:45.644521010 (6h23m45.233865987s) [upload=6651]: Upload finished. err: <nil> Aug 7 16:04:31.274936212 (6h24m30.864281199s) [upload=6663]: Starting upload Aug 7 16:04:31.274977170 (6h24m30.864322157s) [upload=6663]: Done starting upload Aug 7 16:04:31.275026655 (6h24m30.864371642s) [upload=6663]: Uploading segments... Aug 7 16:04:31.275054187 (6h24m30.864399174s) [upload=6663]: Got next segment. Inline: true Aug 7 16:04:31.275057754 (6h24m30.864402731s) [upload=6663]: Waiting for error group managing segments... Aug 7 16:04:31.275091278 (6h24m30.864436265s) [upload=6663]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:04:31.275095255 (6h24m30.864440242s) [upload=6663]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:04:31.275097651 (6h24m30.864442638s) [upload=6663]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:04:31.554258512 (6h24m31.143603499s) [upload=6663]: Done uploading segments... Aug 7 16:04:31.554277108 (6h24m31.143622095s) [upload=6663]: Upload finished. err: <nil> Aug 7 16:05:14.730035713 (6h25m14.3193807s) [upload=6674]: Starting upload Aug 7 16:05:14.730082813 (6h25m14.3194278s) [upload=6674]: Done starting upload Aug 7 16:05:14.730159079 (6h25m14.319504056s) [upload=6674]: Uploading segments... Aug 7 16:05:14.730249532 (6h25m14.319594519s) [upload=6674]: Got next segment. Inline: true Aug 7 16:05:14.730254050 (6h25m14.319599037s) [upload=6674]: Waiting for error group managing segments... Aug 7 16:05:14.730294949 (6h25m14.319639936s) [upload=6674]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:05:14.730299728 (6h25m14.319644715s) [upload=6674]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:05:14.730302243 (6h25m14.31964723s) [upload=6674]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:05:14.942773689 (6h25m14.532118666s) [upload=6674]: Done uploading segments... Aug 7 16:05:14.942790141 (6h25m14.532135118s) [upload=6674]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.518721162s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [.......................................X.......X............] duration:
6.006626234s
FAIL (2025-08-07 16:09:55 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
e begin upload segment.
Aug 7 16:01:39.748945470 (6h21m39.338290447s) [noise=true , address=104.36.82.42:4820 , piece_id=XJIHI2E35NJKDFYFTW4V5EIF3CL7GDJ5EKVCTEV2KL26X43A3CYA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.748969657 (6h21m39.338314644s) [noise=true , address=104.36.82.60:4345 , piece_id=VPYOQHGEP6E257WI7IGIWXRO5WOLCNGDJYNPFPC56WUMOYDUS6RA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749368329 (6h21m39.338713316s) [noise=true , address=45.91.116.9:4267 , piece_id=MWAYSXFIIYQGDPTLWZSAHZP3UO72EXYDSSULHBJJU54H5OWG36NA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749375964 (6h21m39.338720951s) [noise=true , address=104.36.82.10:4110 , piece_id=WAUEFOZXH3SULGTXTWXWLEPYHZJWJDDQRRWWXRBXVWF7WCWPYGYQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749660398 (6h21m39.339005385s) [noise=true , address=79.127.205.235:4666 , piece_id=WA6GZKNO2UUAC5VKBNOZ7UO25RQR7QDT5HVNQYZDIWJEB656WMGA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749701216 (6h21m39.339046203s) [noise=true , address=79.127.201.215:4345 , piece_id=MAAFKAPO4OCQXU5WJ6DAP72KEHIM4AO5JJFJ4ZEFNWKS5M4OV43Q, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749938690 (6h21m39.339283677s) [noise=true , address=104.36.82.59:4313 , piece_id=KDH6AVALUMDXB6USO7WG77DVOSRBTGSZA6VTJFFQJR7QFC2PE35A, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750024905 (6h21m39.339369892s) [noise=true , address=104.36.82.23:4455 , piece_id=UJOJ7P3VET4XIM257B3SHAN6SJG2H64QJHV6PYU7OEB7PPBZVDAA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750215089 (6h21m39.339560076s) [noise=true , address=104.36.82.61:4387 , piece_id=MAWGNSJTYYH7EVPDIXQEG6KP3MKACSMWYVMMLYUASUDJBIX5POOA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750282639 (6h21m39.339627626s) [noise=true , address=45.91.116.10:4092 , piece_id=BUVKJC3K3RKZZTW2CQPEQM7ECQOLIBX4CU66P26JZDTOQZLYGMGA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750481038 (6h21m39.339826025s) [noise=true , address=79.127.220.97:4396 , piece_id=X2CCRNYWY5ZQOAODHNJE7VLYQIHD37PEE7MQXI5WIYV36ICONILQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750533900 (6h21m39.339878887s) [noise=true , address=104.36.82.60:4369 , piece_id=Y6SYJOB7SYBJX7STC25OJJPD4FSK5TZ5KXI7WN7QS42UNTOF5BCQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750725967 (6h21m39.340070954s) [noise=true , address=104.36.82.9:4039 , piece_id=PRBG5EVKAZ3LGDRX4MCI57T4RSNFVKZVUNOAIRHBM6SNS5XXUTPQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750795189 (6h21m39.340140176s) [noise=true , address=104.36.82.22:4525 , piece_id=NKD4EXV774KNXLEHICE4MNKOQS5QKPIUTJL2PPCNYUMYPMQEF75A, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750954023 (6h21m39.34029901s) [noise=true , address=104.36.82.45:4912 , piece_id=B6HTS6ORSGMUKZO5G2ALW4LVUSIBTCEHPX2YJTYPPCH3XROIO3HA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.751098079 (6h21m39.340443066s) [noise=true , address=104.36.82.8:4992 , piece_id=226HGROQ3DBKTJB3U6TG2OF4JQRYAGTLMSIL2JIL4NY5PISP4RUQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.751175397 (6h21m39.340520384s) [noise=true , address=79.127.205.233:4631 , piece_id=77WNKENAMS4Q75KYK4F4VNRP4DD4MKGEJIU5K7WZ3EHVQLEW6Q5A, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.751392602 (6h21m39.340737579s) [noise=true , address=104.36.82.5:4845 , piece_id=JKN4VFS7ET2LVGHGE4LIBPCGG33GNY4SGSOXAWB2MXFYPJVO4FCQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:40.878402060 (6h21m40.467747047s) [upload=6610]: Next returned nil segment
Aug 7 16:01:40.878416708 (6h21m40.467761695s) [upload=6610]: Waiting for error group managing segments...
Aug 7 16:01:41.195597254 (6h21m40.784942241s) [noise=true , address=79.127.205.235:4666 , piece_id=WA6GZKNO2UUAC5VKBNOZ7UO25RQR7QDT5HVNQYZDIWJEB656WMGA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.261081133 (6h21m40.85042612s) [noise=true , address=79.127.205.233:4631 , piece_id=77WNKENAMS4Q75KYK4F4VNRP4DD4MKGEJIU5K7WZ3EHVQLEW6Q5A, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.420961069 (6h21m41.010306046s) [noise=true , address=79.127.220.97:4396 , piece_id=X2CCRNYWY5ZQOAODHNJE7VLYQIHD37PEE7MQXI5WIYV36ICONILQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.638341479 (6h21m41.227686486s) [noise=true , address=79.127.201.215:4345 , piece_id=MAAFKAPO4OCQXU5WJ6DAP72KEHIM4AO5JJFJ4ZEFNWKS5M4OV43Q, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.658966771 (6h21m41.248311758s) [noise=true , address=45.91.116.10:4092 , piece_id=BUVKJC3K3RKZZTW2CQPEQM7ECQOLIBX4CU66P26JZDTOQZLYGMGA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.684178459 (6h21m41.273523446s) [noise=true , address=104.36.82.9:4039 , piece_id=PRBG5EVKAZ3LGDRX4MCI57T4RSNFVKZVUNOAIRHBM6SNS5XXUTPQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.691448508 (6h21m41.280793485s) [noise=true , address=104.36.82.22:4525 , piece_id=NKD4EXV774KNXLEHICE4MNKOQS5QKPIUTJL2PPCNYUMYPMQEF75A, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.717147267 (6h21m41.306492254s) [noise=true , address=104.36.82.59:4313 , piece_id=KDH6AVALUMDXB6USO7WG77DVOSRBTGSZA6VTJFFQJR7QFC2PE35A, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.719612885 (6h21m41.308957872s) [noise=true , address=104.36.82.45:4912 , piece_id=B6HTS6ORSGMUKZO5G2ALW4LVUSIBTCEHPX2YJTYPPCH3XROIO3HA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.733973160 (6h21m41.323318147s) [noise=true , address=104.36.82.60:4369 , piece_id=Y6SYJOB7SYBJX7STC25OJJPD4FSK5TZ5KXI7WN7QS42UNTOF5BCQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.737980718 (6h21m41.327325705s) [noise=true , address=104.36.82.5:4845 , piece_id=JKN4VFS7ET2LVGHGE4LIBPCGG33GNY4SGSOXAWB2MXFYPJVO4FCQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.745434729 (6h21m41.334779717s) [noise=true , address=104.36.82.42:4820 , piece_id=XJIHI2E35NJKDFYFTW4V5EIF3CL7GDJ5EKVCTEV2KL26X43A3CYA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.754624732 (6h21m41.34396972s) [noise=true , address=104.36.82.8:4992 , piece_id=226HGROQ3DBKTJB3U6TG2OF4JQRYAGTLMSIL2JIL4NY5PISP4RUQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.763605976 (6h21m41.352950963s) [noise=true , address=104.36.82.60:4345 , piece_id=VPYOQHGEP6E257WI7IGIWXRO5WOLCNGDJYNPFPC56WUMOYDUS6RA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.763623339 (6h21m41.352968336s) [seg_pos={0 0}, upload=6610]: Segment reached optimal threshold of 14 pieces.
Aug 7 16:01:41.763741525 (6h21m41.353086512s) [noise=true , address=104.36.82.10:4110 , piece_id=WAUEFOZXH3SULGTXTWXWLEPYHZJWJDDQRRWWXRBXVWF7WCWPYGYQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ADbnPJGRTw2NqRzhy7ptojGYoNdJgqmjuWBSjcrQK53Wm7LXi): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763803123 (6h21m41.35314811s) [noise=true , address=104.36.82.61:4387 , piece_id=MAWGNSJTYYH7EVPDIXQEG6KP3MKACSMWYVMMLYUASUDJBIX5POOA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P4wAXSyaRY9poPRK5q4td8DJM69xWioG38TTwLTHKHA7QFxLp): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763857136 (6h21m41.353202113s) [noise=true , address=45.91.116.9:4267 , piece_id=MWAYSXFIIYQGDPTLWZSAHZP3UO72EXYDSSULHBJJU54H5OWG36NA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6M5TgGqUwjXyM25uqv5hsh67mS81ByzMM4kWFUNaFvisDN9d): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763904968 (6h21m41.353249955s) [noise=true , address=104.36.82.23:4455 , piece_id=UJOJ7P3VET4XIM257B3SHAN6SJG2H64QJHV6PYU7OEB7PPBZVDAA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wsyjF5X3iPrJsezUjakUArxc8K88T36yT4rE3yu6q9BMKfycm): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763945405 (6h21m41.353290392s) [seg_pos={0 0}, upload=6610]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 16:01:41.764033824 (6h21m41.353378811s) [upload=6610]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 16:01:41.764038263 (6h21m41.35338325s) [upload=6610]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 16:01:42.043459384 (6h21m41.632804371s) [upload=6610]: Done uploading segments...
Aug 7 16:01:42.043475946 (6h21m41.632820933s) [upload=6610]: Upload finished. err: <nil>
Aug 7 16:02:11.915996790 (6h22m11.505341777s) [upload=6619]: Starting upload
Aug 7 16:02:11.916045183 (6h22m11.50539017s) [upload=6619]: Done starting upload
Aug 7 16:02:11.916152489 (6h22m11.505497476s) [upload=6619]: Uploading segments...
Aug 7 16:02:11.916287706 (6h22m11.505632693s) [upload=6619]: Got next segment. Inline: false
Aug 7 16:02:11.916319939 (6h22m11.505664926s) [upload=6619]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 16:02:11.916324708 (6h22m11.505669695s) [upload=6619]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 16:02:12.129590885 (6h22m11.718935892s) [upload=6619]: Next segment err: metaclient: Too Many Requests
Aug 7 16:02:12.129623196 (6h22m11.718968194s) [upload=6619]: Done uploading segments...
Aug 7 16:02:12.129640310 (6h22m11.718985307s) [upload=6619]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.006646873s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
2.017854531s
FAIL (2025-08-07 16:09:16 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== entMakeInline Aug 7 15:57:10.271829415 (6h17m9.861174403s) [upload=6533]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 15:57:10.587098885 (6h17m10.176443872s) [upload=6533]: Done uploading segments... Aug 7 15:57:10.587115857 (6h17m10.176460835s) [upload=6533]: Upload finished. err: <nil> Aug 7 15:57:48.918901442 (6h17m48.508246429s) [upload=6545]: Starting upload Aug 7 15:57:48.918948701 (6h17m48.508293688s) [upload=6545]: Done starting upload Aug 7 15:57:48.919013486 (6h17m48.508358473s) [upload=6545]: Uploading segments... Aug 7 15:57:48.919055005 (6h17m48.508399982s) [upload=6545]: Got next segment. Inline: true Aug 7 15:57:48.919059032 (6h17m48.508404009s) [upload=6545]: Waiting for error group managing segments... Aug 7 15:57:48.919091585 (6h17m48.508436572s) [upload=6545]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 15:57:48.919095493 (6h17m48.50844048s) [upload=6545]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 15:57:48.919097998 (6h17m48.508442975s) [upload=6545]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 15:57:49.200378977 (6h17m48.789723964s) [upload=6545]: Done uploading segments... Aug 7 15:57:49.200397332 (6h17m48.789742319s) [upload=6545]: Upload finished. err: <nil> Aug 7 15:58:50.898935036 (6h18m50.488280023s) [upload=6562]: Starting upload Aug 7 15:58:50.898979992 (6h18m50.488324979s) [upload=6562]: Done starting upload Aug 7 15:58:50.899026682 (6h18m50.488371669s) [upload=6562]: Uploading segments... Aug 7 15:58:50.899065355 (6h18m50.488410343s) [upload=6562]: Got next segment. Inline: true Aug 7 15:58:50.899070335 (6h18m50.488415322s) [upload=6562]: Waiting for error group managing segments... Aug 7 15:58:50.899108268 (6h18m50.488453255s) [upload=6562]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 15:58:50.899112326 (6h18m50.488457313s) [upload=6562]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 15:58:50.899114820 (6h18m50.488459807s) [upload=6562]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 15:58:51.203289783 (6h18m50.79263478s) [upload=6562]: Done uploading segments... Aug 7 15:58:51.203312587 (6h18m50.792657624s) [upload=6562]: Upload finished. err: <nil> Aug 7 16:00:00.470970125 (6h20m0.060315112s) [upload=6580]: Starting upload Aug 7 16:00:00.471017334 (6h20m0.060362321s) [upload=6580]: Done starting upload Aug 7 16:00:00.471072199 (6h20m0.060417186s) [upload=6580]: Uploading segments... Aug 7 16:00:00.471112327 (6h20m0.060457314s) [upload=6580]: Got next segment. Inline: true Aug 7 16:00:00.471116364 (6h20m0.060461351s) [upload=6580]: Waiting for error group managing segments... Aug 7 16:00:00.471149427 (6h20m0.060494414s) [upload=6580]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:00:00.471153305 (6h20m0.060498292s) [upload=6580]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:00:00.471155819 (6h20m0.060500806s) [upload=6580]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:00:00.739624026 (6h20m0.328969023s) [upload=6580]: Done uploading segments... Aug 7 16:00:00.739645636 (6h20m0.328990623s) [upload=6580]: Upload finished. err: <nil> Aug 7 16:00:49.882359165 (6h20m49.471704152s) [upload=6598]: Starting upload Aug 7 16:00:49.882404682 (6h20m49.471749659s) [upload=6598]: Done starting upload Aug 7 16:00:49.882456812 (6h20m49.471801799s) [upload=6598]: Uploading segments... Aug 7 16:00:49.882489865 (6h20m49.471834852s) [upload=6598]: Got next segment. Inline: true Aug 7 16:00:49.882493702 (6h20m49.47183869s) [upload=6598]: Waiting for error group managing segments... Aug 7 16:00:49.882525894 (6h20m49.471870881s) [upload=6598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:00:49.882529422 (6h20m49.471874399s) [upload=6598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:00:49.882531676 (6h20m49.471876663s) [upload=6598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:00:50.213720013 (6h20m49.803065s) [upload=6598]: Done uploading segments... Aug 7 16:00:50.213739550 (6h20m49.803084537s) [upload=6598]: Upload finished. err: <nil> Aug 7 16:01:51.762778733 (6h21m51.35212372s) [upload=6617]: Starting upload Aug 7 16:01:51.762856161 (6h21m51.352201148s) [upload=6617]: Done starting upload Aug 7 16:01:51.762914883 (6h21m51.35225987s) [upload=6617]: Uploading segments... Aug 7 16:01:51.762952445 (6h21m51.352297432s) [upload=6617]: Got next segment. Inline: true Aug 7 16:01:51.762957344 (6h21m51.352302331s) [upload=6617]: Waiting for error group managing segments... Aug 7 16:01:51.762994485 (6h21m51.352339473s) [upload=6617]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:01:51.762998313 (6h21m51.3523433s) [upload=6617]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:01:51.763001278 (6h21m51.352346265s) [upload=6617]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:01:52.034377407 (6h21m51.623722394s) [upload=6617]: Done uploading segments... Aug 7 16:01:52.034395451 (6h21m51.623740439s) [upload=6617]: Upload finished. err: <nil> Aug 7 16:02:11.154406473 (6h22m10.74375146s) [upload=6618]: Starting upload Aug 7 16:02:11.154456328 (6h22m10.743801315s) [upload=6618]: Done starting upload Aug 7 16:02:11.154503859 (6h22m10.743848846s) [upload=6618]: Uploading segments... Aug 7 16:02:11.154545308 (6h22m10.743890295s) [upload=6618]: Got next segment. Inline: true Aug 7 16:02:11.154548574 (6h22m10.743893561s) [upload=6618]: Waiting for error group managing segments... Aug 7 16:02:11.154580566 (6h22m10.743925553s) [upload=6618]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:02:11.154584283 (6h22m10.74392927s) [upload=6618]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:02:11.154586828 (6h22m10.743931815s) [upload=6618]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:02:11.475329609 (6h22m11.064674596s) [upload=6618]: Done uploading segments... Aug 7 16:02:11.475484125 (6h22m11.064829112s) [upload=6618]: Upload finished. err: <nil> Aug 7 16:02:42.941029250 (6h22m42.530374238s) [upload=6628]: Starting upload Aug 7 16:02:42.941096740 (6h22m42.530441727s) [upload=6628]: Done starting upload Aug 7 16:02:42.941158728 (6h22m42.530503716s) [upload=6628]: Uploading segments... Aug 7 16:02:42.941210478 (6h22m42.530555465s) [upload=6628]: Got next segment. Inline: true Aug 7 16:02:42.941216419 (6h22m42.530561416s) [upload=6628]: Waiting for error group managing segments... Aug 7 16:02:42.941281664 (6h22m42.530626651s) [upload=6628]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:02:42.941286824 (6h22m42.530631811s) [upload=6628]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:02:42.941289880 (6h22m42.530634867s) [upload=6628]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:02:43.233814342 (6h22m42.823159339s) [upload=6628]: Done uploading segments... Aug 7 16:02:43.233838148 (6h22m42.823183145s) [upload=6628]: Upload finished. err: <nil> Aug 7 16:03:45.201690833 (6h23m44.79103582s) [upload=6651]: Starting upload Aug 7 16:03:45.201738173 (6h23m44.79108316s) [upload=6651]: Done starting upload Aug 7 16:03:45.201785534 (6h23m44.791130511s) [upload=6651]: Uploading segments... Aug 7 16:03:45.201855618 (6h23m44.791200605s) [upload=6651]: Got next segment. Inline: true Aug 7 16:03:45.201859616 (6h23m44.791204603s) [upload=6651]: Waiting for error group managing segments... Aug 7 16:03:45.201894122 (6h23m44.791239109s) [upload=6651]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:03:45.201897799 (6h23m44.791242786s) [upload=6651]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:03:45.201900023 (6h23m44.79124501s) [upload=6651]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:03:45.644503676 (6h23m45.233848664s) [upload=6651]: Done uploading segments... Aug 7 16:03:45.644521010 (6h23m45.233865987s) [upload=6651]: Upload finished. err: <nil> Aug 7 16:04:31.274936212 (6h24m30.864281199s) [upload=6663]: Starting upload Aug 7 16:04:31.274977170 (6h24m30.864322157s) [upload=6663]: Done starting upload Aug 7 16:04:31.275026655 (6h24m30.864371642s) [upload=6663]: Uploading segments... Aug 7 16:04:31.275054187 (6h24m30.864399174s) [upload=6663]: Got next segment. Inline: true Aug 7 16:04:31.275057754 (6h24m30.864402731s) [upload=6663]: Waiting for error group managing segments... Aug 7 16:04:31.275091278 (6h24m30.864436265s) [upload=6663]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:04:31.275095255 (6h24m30.864440242s) [upload=6663]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:04:31.275097651 (6h24m30.864442638s) [upload=6663]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:04:31.554258512 (6h24m31.143603499s) [upload=6663]: Done uploading segments... Aug 7 16:04:31.554277108 (6h24m31.143622095s) [upload=6663]: Upload finished. err: <nil> Aug 7 16:05:14.730035713 (6h25m14.3193807s) [upload=6674]: Starting upload Aug 7 16:05:14.730082813 (6h25m14.3194278s) [upload=6674]: Done starting upload Aug 7 16:05:14.730159079 (6h25m14.319504056s) [upload=6674]: Uploading segments... Aug 7 16:05:14.730249532 (6h25m14.319594519s) [upload=6674]: Got next segment. Inline: true Aug 7 16:05:14.730254050 (6h25m14.319599037s) [upload=6674]: Waiting for error group managing segments... Aug 7 16:05:14.730294949 (6h25m14.319639936s) [upload=6674]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:05:14.730299728 (6h25m14.319644715s) [upload=6674]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:05:14.730302243 (6h25m14.31964723s) [upload=6674]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:05:14.942773689 (6h25m14.532118666s) [upload=6674]: Done uploading segments... Aug 7 16:05:14.942790141 (6h25m14.532135118s) [upload=6674]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.017878418s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [........................................X.......X...........] duration:
5.630922927s
FAIL (2025-08-07 16:09:04 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
e begin upload segment.
Aug 7 16:01:39.748945470 (6h21m39.338290447s) [noise=true , address=104.36.82.42:4820 , piece_id=XJIHI2E35NJKDFYFTW4V5EIF3CL7GDJ5EKVCTEV2KL26X43A3CYA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.748969657 (6h21m39.338314644s) [noise=true , address=104.36.82.60:4345 , piece_id=VPYOQHGEP6E257WI7IGIWXRO5WOLCNGDJYNPFPC56WUMOYDUS6RA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749368329 (6h21m39.338713316s) [noise=true , address=45.91.116.9:4267 , piece_id=MWAYSXFIIYQGDPTLWZSAHZP3UO72EXYDSSULHBJJU54H5OWG36NA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749375964 (6h21m39.338720951s) [noise=true , address=104.36.82.10:4110 , piece_id=WAUEFOZXH3SULGTXTWXWLEPYHZJWJDDQRRWWXRBXVWF7WCWPYGYQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749660398 (6h21m39.339005385s) [noise=true , address=79.127.205.235:4666 , piece_id=WA6GZKNO2UUAC5VKBNOZ7UO25RQR7QDT5HVNQYZDIWJEB656WMGA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749701216 (6h21m39.339046203s) [noise=true , address=79.127.201.215:4345 , piece_id=MAAFKAPO4OCQXU5WJ6DAP72KEHIM4AO5JJFJ4ZEFNWKS5M4OV43Q, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749938690 (6h21m39.339283677s) [noise=true , address=104.36.82.59:4313 , piece_id=KDH6AVALUMDXB6USO7WG77DVOSRBTGSZA6VTJFFQJR7QFC2PE35A, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750024905 (6h21m39.339369892s) [noise=true , address=104.36.82.23:4455 , piece_id=UJOJ7P3VET4XIM257B3SHAN6SJG2H64QJHV6PYU7OEB7PPBZVDAA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750215089 (6h21m39.339560076s) [noise=true , address=104.36.82.61:4387 , piece_id=MAWGNSJTYYH7EVPDIXQEG6KP3MKACSMWYVMMLYUASUDJBIX5POOA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750282639 (6h21m39.339627626s) [noise=true , address=45.91.116.10:4092 , piece_id=BUVKJC3K3RKZZTW2CQPEQM7ECQOLIBX4CU66P26JZDTOQZLYGMGA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750481038 (6h21m39.339826025s) [noise=true , address=79.127.220.97:4396 , piece_id=X2CCRNYWY5ZQOAODHNJE7VLYQIHD37PEE7MQXI5WIYV36ICONILQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750533900 (6h21m39.339878887s) [noise=true , address=104.36.82.60:4369 , piece_id=Y6SYJOB7SYBJX7STC25OJJPD4FSK5TZ5KXI7WN7QS42UNTOF5BCQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750725967 (6h21m39.340070954s) [noise=true , address=104.36.82.9:4039 , piece_id=PRBG5EVKAZ3LGDRX4MCI57T4RSNFVKZVUNOAIRHBM6SNS5XXUTPQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750795189 (6h21m39.340140176s) [noise=true , address=104.36.82.22:4525 , piece_id=NKD4EXV774KNXLEHICE4MNKOQS5QKPIUTJL2PPCNYUMYPMQEF75A, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750954023 (6h21m39.34029901s) [noise=true , address=104.36.82.45:4912 , piece_id=B6HTS6ORSGMUKZO5G2ALW4LVUSIBTCEHPX2YJTYPPCH3XROIO3HA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.751098079 (6h21m39.340443066s) [noise=true , address=104.36.82.8:4992 , piece_id=226HGROQ3DBKTJB3U6TG2OF4JQRYAGTLMSIL2JIL4NY5PISP4RUQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.751175397 (6h21m39.340520384s) [noise=true , address=79.127.205.233:4631 , piece_id=77WNKENAMS4Q75KYK4F4VNRP4DD4MKGEJIU5K7WZ3EHVQLEW6Q5A, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.751392602 (6h21m39.340737579s) [noise=true , address=104.36.82.5:4845 , piece_id=JKN4VFS7ET2LVGHGE4LIBPCGG33GNY4SGSOXAWB2MXFYPJVO4FCQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:40.878402060 (6h21m40.467747047s) [upload=6610]: Next returned nil segment
Aug 7 16:01:40.878416708 (6h21m40.467761695s) [upload=6610]: Waiting for error group managing segments...
Aug 7 16:01:41.195597254 (6h21m40.784942241s) [noise=true , address=79.127.205.235:4666 , piece_id=WA6GZKNO2UUAC5VKBNOZ7UO25RQR7QDT5HVNQYZDIWJEB656WMGA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.261081133 (6h21m40.85042612s) [noise=true , address=79.127.205.233:4631 , piece_id=77WNKENAMS4Q75KYK4F4VNRP4DD4MKGEJIU5K7WZ3EHVQLEW6Q5A, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.420961069 (6h21m41.010306046s) [noise=true , address=79.127.220.97:4396 , piece_id=X2CCRNYWY5ZQOAODHNJE7VLYQIHD37PEE7MQXI5WIYV36ICONILQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.638341479 (6h21m41.227686486s) [noise=true , address=79.127.201.215:4345 , piece_id=MAAFKAPO4OCQXU5WJ6DAP72KEHIM4AO5JJFJ4ZEFNWKS5M4OV43Q, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.658966771 (6h21m41.248311758s) [noise=true , address=45.91.116.10:4092 , piece_id=BUVKJC3K3RKZZTW2CQPEQM7ECQOLIBX4CU66P26JZDTOQZLYGMGA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.684178459 (6h21m41.273523446s) [noise=true , address=104.36.82.9:4039 , piece_id=PRBG5EVKAZ3LGDRX4MCI57T4RSNFVKZVUNOAIRHBM6SNS5XXUTPQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.691448508 (6h21m41.280793485s) [noise=true , address=104.36.82.22:4525 , piece_id=NKD4EXV774KNXLEHICE4MNKOQS5QKPIUTJL2PPCNYUMYPMQEF75A, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.717147267 (6h21m41.306492254s) [noise=true , address=104.36.82.59:4313 , piece_id=KDH6AVALUMDXB6USO7WG77DVOSRBTGSZA6VTJFFQJR7QFC2PE35A, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.719612885 (6h21m41.308957872s) [noise=true , address=104.36.82.45:4912 , piece_id=B6HTS6ORSGMUKZO5G2ALW4LVUSIBTCEHPX2YJTYPPCH3XROIO3HA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.733973160 (6h21m41.323318147s) [noise=true , address=104.36.82.60:4369 , piece_id=Y6SYJOB7SYBJX7STC25OJJPD4FSK5TZ5KXI7WN7QS42UNTOF5BCQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.737980718 (6h21m41.327325705s) [noise=true , address=104.36.82.5:4845 , piece_id=JKN4VFS7ET2LVGHGE4LIBPCGG33GNY4SGSOXAWB2MXFYPJVO4FCQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.745434729 (6h21m41.334779717s) [noise=true , address=104.36.82.42:4820 , piece_id=XJIHI2E35NJKDFYFTW4V5EIF3CL7GDJ5EKVCTEV2KL26X43A3CYA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.754624732 (6h21m41.34396972s) [noise=true , address=104.36.82.8:4992 , piece_id=226HGROQ3DBKTJB3U6TG2OF4JQRYAGTLMSIL2JIL4NY5PISP4RUQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.763605976 (6h21m41.352950963s) [noise=true , address=104.36.82.60:4345 , piece_id=VPYOQHGEP6E257WI7IGIWXRO5WOLCNGDJYNPFPC56WUMOYDUS6RA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.763623339 (6h21m41.352968336s) [seg_pos={0 0}, upload=6610]: Segment reached optimal threshold of 14 pieces.
Aug 7 16:01:41.763741525 (6h21m41.353086512s) [noise=true , address=104.36.82.10:4110 , piece_id=WAUEFOZXH3SULGTXTWXWLEPYHZJWJDDQRRWWXRBXVWF7WCWPYGYQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ADbnPJGRTw2NqRzhy7ptojGYoNdJgqmjuWBSjcrQK53Wm7LXi): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763803123 (6h21m41.35314811s) [noise=true , address=104.36.82.61:4387 , piece_id=MAWGNSJTYYH7EVPDIXQEG6KP3MKACSMWYVMMLYUASUDJBIX5POOA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P4wAXSyaRY9poPRK5q4td8DJM69xWioG38TTwLTHKHA7QFxLp): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763857136 (6h21m41.353202113s) [noise=true , address=45.91.116.9:4267 , piece_id=MWAYSXFIIYQGDPTLWZSAHZP3UO72EXYDSSULHBJJU54H5OWG36NA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6M5TgGqUwjXyM25uqv5hsh67mS81ByzMM4kWFUNaFvisDN9d): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763904968 (6h21m41.353249955s) [noise=true , address=104.36.82.23:4455 , piece_id=UJOJ7P3VET4XIM257B3SHAN6SJG2H64QJHV6PYU7OEB7PPBZVDAA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wsyjF5X3iPrJsezUjakUArxc8K88T36yT4rE3yu6q9BMKfycm): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763945405 (6h21m41.353290392s) [seg_pos={0 0}, upload=6610]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 16:01:41.764033824 (6h21m41.353378811s) [upload=6610]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 16:01:41.764038263 (6h21m41.35338325s) [upload=6610]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 16:01:42.043459384 (6h21m41.632804371s) [upload=6610]: Done uploading segments...
Aug 7 16:01:42.043475946 (6h21m41.632820933s) [upload=6610]: Upload finished. err: <nil>
Aug 7 16:02:11.915996790 (6h22m11.505341777s) [upload=6619]: Starting upload
Aug 7 16:02:11.916045183 (6h22m11.50539017s) [upload=6619]: Done starting upload
Aug 7 16:02:11.916152489 (6h22m11.505497476s) [upload=6619]: Uploading segments...
Aug 7 16:02:11.916287706 (6h22m11.505632693s) [upload=6619]: Got next segment. Inline: false
Aug 7 16:02:11.916319939 (6h22m11.505664926s) [upload=6619]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 16:02:11.916324708 (6h22m11.505669695s) [upload=6619]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 16:02:12.129590885 (6h22m11.718935892s) [upload=6619]: Next segment err: metaclient: Too Many Requests
Aug 7 16:02:12.129623196 (6h22m11.718968194s) [upload=6619]: Done uploading segments...
Aug 7 16:02:12.129640310 (6h22m11.718985307s) [upload=6619]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.630958886s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
1.855079308s
FAIL (2025-08-07 16:08:28 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== entMakeInline Aug 7 15:57:10.271829415 (6h17m9.861174403s) [upload=6533]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 15:57:10.587098885 (6h17m10.176443872s) [upload=6533]: Done uploading segments... Aug 7 15:57:10.587115857 (6h17m10.176460835s) [upload=6533]: Upload finished. err: <nil> Aug 7 15:57:48.918901442 (6h17m48.508246429s) [upload=6545]: Starting upload Aug 7 15:57:48.918948701 (6h17m48.508293688s) [upload=6545]: Done starting upload Aug 7 15:57:48.919013486 (6h17m48.508358473s) [upload=6545]: Uploading segments... Aug 7 15:57:48.919055005 (6h17m48.508399982s) [upload=6545]: Got next segment. Inline: true Aug 7 15:57:48.919059032 (6h17m48.508404009s) [upload=6545]: Waiting for error group managing segments... Aug 7 15:57:48.919091585 (6h17m48.508436572s) [upload=6545]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 15:57:48.919095493 (6h17m48.50844048s) [upload=6545]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 15:57:48.919097998 (6h17m48.508442975s) [upload=6545]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 15:57:49.200378977 (6h17m48.789723964s) [upload=6545]: Done uploading segments... Aug 7 15:57:49.200397332 (6h17m48.789742319s) [upload=6545]: Upload finished. err: <nil> Aug 7 15:58:50.898935036 (6h18m50.488280023s) [upload=6562]: Starting upload Aug 7 15:58:50.898979992 (6h18m50.488324979s) [upload=6562]: Done starting upload Aug 7 15:58:50.899026682 (6h18m50.488371669s) [upload=6562]: Uploading segments... Aug 7 15:58:50.899065355 (6h18m50.488410343s) [upload=6562]: Got next segment. Inline: true Aug 7 15:58:50.899070335 (6h18m50.488415322s) [upload=6562]: Waiting for error group managing segments... Aug 7 15:58:50.899108268 (6h18m50.488453255s) [upload=6562]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 15:58:50.899112326 (6h18m50.488457313s) [upload=6562]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 15:58:50.899114820 (6h18m50.488459807s) [upload=6562]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 15:58:51.203289783 (6h18m50.79263478s) [upload=6562]: Done uploading segments... Aug 7 15:58:51.203312587 (6h18m50.792657624s) [upload=6562]: Upload finished. err: <nil> Aug 7 16:00:00.470970125 (6h20m0.060315112s) [upload=6580]: Starting upload Aug 7 16:00:00.471017334 (6h20m0.060362321s) [upload=6580]: Done starting upload Aug 7 16:00:00.471072199 (6h20m0.060417186s) [upload=6580]: Uploading segments... Aug 7 16:00:00.471112327 (6h20m0.060457314s) [upload=6580]: Got next segment. Inline: true Aug 7 16:00:00.471116364 (6h20m0.060461351s) [upload=6580]: Waiting for error group managing segments... Aug 7 16:00:00.471149427 (6h20m0.060494414s) [upload=6580]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:00:00.471153305 (6h20m0.060498292s) [upload=6580]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:00:00.471155819 (6h20m0.060500806s) [upload=6580]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:00:00.739624026 (6h20m0.328969023s) [upload=6580]: Done uploading segments... Aug 7 16:00:00.739645636 (6h20m0.328990623s) [upload=6580]: Upload finished. err: <nil> Aug 7 16:00:49.882359165 (6h20m49.471704152s) [upload=6598]: Starting upload Aug 7 16:00:49.882404682 (6h20m49.471749659s) [upload=6598]: Done starting upload Aug 7 16:00:49.882456812 (6h20m49.471801799s) [upload=6598]: Uploading segments... Aug 7 16:00:49.882489865 (6h20m49.471834852s) [upload=6598]: Got next segment. Inline: true Aug 7 16:00:49.882493702 (6h20m49.47183869s) [upload=6598]: Waiting for error group managing segments... Aug 7 16:00:49.882525894 (6h20m49.471870881s) [upload=6598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:00:49.882529422 (6h20m49.471874399s) [upload=6598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:00:49.882531676 (6h20m49.471876663s) [upload=6598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:00:50.213720013 (6h20m49.803065s) [upload=6598]: Done uploading segments... Aug 7 16:00:50.213739550 (6h20m49.803084537s) [upload=6598]: Upload finished. err: <nil> Aug 7 16:01:51.762778733 (6h21m51.35212372s) [upload=6617]: Starting upload Aug 7 16:01:51.762856161 (6h21m51.352201148s) [upload=6617]: Done starting upload Aug 7 16:01:51.762914883 (6h21m51.35225987s) [upload=6617]: Uploading segments... Aug 7 16:01:51.762952445 (6h21m51.352297432s) [upload=6617]: Got next segment. Inline: true Aug 7 16:01:51.762957344 (6h21m51.352302331s) [upload=6617]: Waiting for error group managing segments... Aug 7 16:01:51.762994485 (6h21m51.352339473s) [upload=6617]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:01:51.762998313 (6h21m51.3523433s) [upload=6617]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:01:51.763001278 (6h21m51.352346265s) [upload=6617]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:01:52.034377407 (6h21m51.623722394s) [upload=6617]: Done uploading segments... Aug 7 16:01:52.034395451 (6h21m51.623740439s) [upload=6617]: Upload finished. err: <nil> Aug 7 16:02:11.154406473 (6h22m10.74375146s) [upload=6618]: Starting upload Aug 7 16:02:11.154456328 (6h22m10.743801315s) [upload=6618]: Done starting upload Aug 7 16:02:11.154503859 (6h22m10.743848846s) [upload=6618]: Uploading segments... Aug 7 16:02:11.154545308 (6h22m10.743890295s) [upload=6618]: Got next segment. Inline: true Aug 7 16:02:11.154548574 (6h22m10.743893561s) [upload=6618]: Waiting for error group managing segments... Aug 7 16:02:11.154580566 (6h22m10.743925553s) [upload=6618]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:02:11.154584283 (6h22m10.74392927s) [upload=6618]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:02:11.154586828 (6h22m10.743931815s) [upload=6618]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:02:11.475329609 (6h22m11.064674596s) [upload=6618]: Done uploading segments... Aug 7 16:02:11.475484125 (6h22m11.064829112s) [upload=6618]: Upload finished. err: <nil> Aug 7 16:02:42.941029250 (6h22m42.530374238s) [upload=6628]: Starting upload Aug 7 16:02:42.941096740 (6h22m42.530441727s) [upload=6628]: Done starting upload Aug 7 16:02:42.941158728 (6h22m42.530503716s) [upload=6628]: Uploading segments... Aug 7 16:02:42.941210478 (6h22m42.530555465s) [upload=6628]: Got next segment. Inline: true Aug 7 16:02:42.941216419 (6h22m42.530561416s) [upload=6628]: Waiting for error group managing segments... Aug 7 16:02:42.941281664 (6h22m42.530626651s) [upload=6628]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:02:42.941286824 (6h22m42.530631811s) [upload=6628]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:02:42.941289880 (6h22m42.530634867s) [upload=6628]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:02:43.233814342 (6h22m42.823159339s) [upload=6628]: Done uploading segments... Aug 7 16:02:43.233838148 (6h22m42.823183145s) [upload=6628]: Upload finished. err: <nil> Aug 7 16:03:45.201690833 (6h23m44.79103582s) [upload=6651]: Starting upload Aug 7 16:03:45.201738173 (6h23m44.79108316s) [upload=6651]: Done starting upload Aug 7 16:03:45.201785534 (6h23m44.791130511s) [upload=6651]: Uploading segments... Aug 7 16:03:45.201855618 (6h23m44.791200605s) [upload=6651]: Got next segment. Inline: true Aug 7 16:03:45.201859616 (6h23m44.791204603s) [upload=6651]: Waiting for error group managing segments... Aug 7 16:03:45.201894122 (6h23m44.791239109s) [upload=6651]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:03:45.201897799 (6h23m44.791242786s) [upload=6651]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:03:45.201900023 (6h23m44.79124501s) [upload=6651]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:03:45.644503676 (6h23m45.233848664s) [upload=6651]: Done uploading segments... Aug 7 16:03:45.644521010 (6h23m45.233865987s) [upload=6651]: Upload finished. err: <nil> Aug 7 16:04:31.274936212 (6h24m30.864281199s) [upload=6663]: Starting upload Aug 7 16:04:31.274977170 (6h24m30.864322157s) [upload=6663]: Done starting upload Aug 7 16:04:31.275026655 (6h24m30.864371642s) [upload=6663]: Uploading segments... Aug 7 16:04:31.275054187 (6h24m30.864399174s) [upload=6663]: Got next segment. Inline: true Aug 7 16:04:31.275057754 (6h24m30.864402731s) [upload=6663]: Waiting for error group managing segments... Aug 7 16:04:31.275091278 (6h24m30.864436265s) [upload=6663]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:04:31.275095255 (6h24m30.864440242s) [upload=6663]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:04:31.275097651 (6h24m30.864442638s) [upload=6663]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:04:31.554258512 (6h24m31.143603499s) [upload=6663]: Done uploading segments... Aug 7 16:04:31.554277108 (6h24m31.143622095s) [upload=6663]: Upload finished. err: <nil> Aug 7 16:05:14.730035713 (6h25m14.3193807s) [upload=6674]: Starting upload Aug 7 16:05:14.730082813 (6h25m14.3194278s) [upload=6674]: Done starting upload Aug 7 16:05:14.730159079 (6h25m14.319504056s) [upload=6674]: Uploading segments... Aug 7 16:05:14.730249532 (6h25m14.319594519s) [upload=6674]: Got next segment. Inline: true Aug 7 16:05:14.730254050 (6h25m14.319599037s) [upload=6674]: Waiting for error group managing segments... Aug 7 16:05:14.730294949 (6h25m14.319639936s) [upload=6674]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:05:14.730299728 (6h25m14.319644715s) [upload=6674]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:05:14.730302243 (6h25m14.31964723s) [upload=6674]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:05:14.942773689 (6h25m14.532118666s) [upload=6674]: Done uploading segments... Aug 7 16:05:14.942790141 (6h25m14.532135118s) [upload=6674]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.855112942s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [.........................................X.......X..........] duration:
6.114488074s
FAIL (2025-08-07 16:08:17 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
e begin upload segment.
Aug 7 16:01:39.748945470 (6h21m39.338290447s) [noise=true , address=104.36.82.42:4820 , piece_id=XJIHI2E35NJKDFYFTW4V5EIF3CL7GDJ5EKVCTEV2KL26X43A3CYA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.748969657 (6h21m39.338314644s) [noise=true , address=104.36.82.60:4345 , piece_id=VPYOQHGEP6E257WI7IGIWXRO5WOLCNGDJYNPFPC56WUMOYDUS6RA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749368329 (6h21m39.338713316s) [noise=true , address=45.91.116.9:4267 , piece_id=MWAYSXFIIYQGDPTLWZSAHZP3UO72EXYDSSULHBJJU54H5OWG36NA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749375964 (6h21m39.338720951s) [noise=true , address=104.36.82.10:4110 , piece_id=WAUEFOZXH3SULGTXTWXWLEPYHZJWJDDQRRWWXRBXVWF7WCWPYGYQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749660398 (6h21m39.339005385s) [noise=true , address=79.127.205.235:4666 , piece_id=WA6GZKNO2UUAC5VKBNOZ7UO25RQR7QDT5HVNQYZDIWJEB656WMGA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749701216 (6h21m39.339046203s) [noise=true , address=79.127.201.215:4345 , piece_id=MAAFKAPO4OCQXU5WJ6DAP72KEHIM4AO5JJFJ4ZEFNWKS5M4OV43Q, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749938690 (6h21m39.339283677s) [noise=true , address=104.36.82.59:4313 , piece_id=KDH6AVALUMDXB6USO7WG77DVOSRBTGSZA6VTJFFQJR7QFC2PE35A, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750024905 (6h21m39.339369892s) [noise=true , address=104.36.82.23:4455 , piece_id=UJOJ7P3VET4XIM257B3SHAN6SJG2H64QJHV6PYU7OEB7PPBZVDAA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750215089 (6h21m39.339560076s) [noise=true , address=104.36.82.61:4387 , piece_id=MAWGNSJTYYH7EVPDIXQEG6KP3MKACSMWYVMMLYUASUDJBIX5POOA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750282639 (6h21m39.339627626s) [noise=true , address=45.91.116.10:4092 , piece_id=BUVKJC3K3RKZZTW2CQPEQM7ECQOLIBX4CU66P26JZDTOQZLYGMGA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750481038 (6h21m39.339826025s) [noise=true , address=79.127.220.97:4396 , piece_id=X2CCRNYWY5ZQOAODHNJE7VLYQIHD37PEE7MQXI5WIYV36ICONILQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750533900 (6h21m39.339878887s) [noise=true , address=104.36.82.60:4369 , piece_id=Y6SYJOB7SYBJX7STC25OJJPD4FSK5TZ5KXI7WN7QS42UNTOF5BCQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750725967 (6h21m39.340070954s) [noise=true , address=104.36.82.9:4039 , piece_id=PRBG5EVKAZ3LGDRX4MCI57T4RSNFVKZVUNOAIRHBM6SNS5XXUTPQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750795189 (6h21m39.340140176s) [noise=true , address=104.36.82.22:4525 , piece_id=NKD4EXV774KNXLEHICE4MNKOQS5QKPIUTJL2PPCNYUMYPMQEF75A, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750954023 (6h21m39.34029901s) [noise=true , address=104.36.82.45:4912 , piece_id=B6HTS6ORSGMUKZO5G2ALW4LVUSIBTCEHPX2YJTYPPCH3XROIO3HA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.751098079 (6h21m39.340443066s) [noise=true , address=104.36.82.8:4992 , piece_id=226HGROQ3DBKTJB3U6TG2OF4JQRYAGTLMSIL2JIL4NY5PISP4RUQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.751175397 (6h21m39.340520384s) [noise=true , address=79.127.205.233:4631 , piece_id=77WNKENAMS4Q75KYK4F4VNRP4DD4MKGEJIU5K7WZ3EHVQLEW6Q5A, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.751392602 (6h21m39.340737579s) [noise=true , address=104.36.82.5:4845 , piece_id=JKN4VFS7ET2LVGHGE4LIBPCGG33GNY4SGSOXAWB2MXFYPJVO4FCQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:40.878402060 (6h21m40.467747047s) [upload=6610]: Next returned nil segment
Aug 7 16:01:40.878416708 (6h21m40.467761695s) [upload=6610]: Waiting for error group managing segments...
Aug 7 16:01:41.195597254 (6h21m40.784942241s) [noise=true , address=79.127.205.235:4666 , piece_id=WA6GZKNO2UUAC5VKBNOZ7UO25RQR7QDT5HVNQYZDIWJEB656WMGA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.261081133 (6h21m40.85042612s) [noise=true , address=79.127.205.233:4631 , piece_id=77WNKENAMS4Q75KYK4F4VNRP4DD4MKGEJIU5K7WZ3EHVQLEW6Q5A, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.420961069 (6h21m41.010306046s) [noise=true , address=79.127.220.97:4396 , piece_id=X2CCRNYWY5ZQOAODHNJE7VLYQIHD37PEE7MQXI5WIYV36ICONILQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.638341479 (6h21m41.227686486s) [noise=true , address=79.127.201.215:4345 , piece_id=MAAFKAPO4OCQXU5WJ6DAP72KEHIM4AO5JJFJ4ZEFNWKS5M4OV43Q, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.658966771 (6h21m41.248311758s) [noise=true , address=45.91.116.10:4092 , piece_id=BUVKJC3K3RKZZTW2CQPEQM7ECQOLIBX4CU66P26JZDTOQZLYGMGA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.684178459 (6h21m41.273523446s) [noise=true , address=104.36.82.9:4039 , piece_id=PRBG5EVKAZ3LGDRX4MCI57T4RSNFVKZVUNOAIRHBM6SNS5XXUTPQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.691448508 (6h21m41.280793485s) [noise=true , address=104.36.82.22:4525 , piece_id=NKD4EXV774KNXLEHICE4MNKOQS5QKPIUTJL2PPCNYUMYPMQEF75A, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.717147267 (6h21m41.306492254s) [noise=true , address=104.36.82.59:4313 , piece_id=KDH6AVALUMDXB6USO7WG77DVOSRBTGSZA6VTJFFQJR7QFC2PE35A, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.719612885 (6h21m41.308957872s) [noise=true , address=104.36.82.45:4912 , piece_id=B6HTS6ORSGMUKZO5G2ALW4LVUSIBTCEHPX2YJTYPPCH3XROIO3HA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.733973160 (6h21m41.323318147s) [noise=true , address=104.36.82.60:4369 , piece_id=Y6SYJOB7SYBJX7STC25OJJPD4FSK5TZ5KXI7WN7QS42UNTOF5BCQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.737980718 (6h21m41.327325705s) [noise=true , address=104.36.82.5:4845 , piece_id=JKN4VFS7ET2LVGHGE4LIBPCGG33GNY4SGSOXAWB2MXFYPJVO4FCQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.745434729 (6h21m41.334779717s) [noise=true , address=104.36.82.42:4820 , piece_id=XJIHI2E35NJKDFYFTW4V5EIF3CL7GDJ5EKVCTEV2KL26X43A3CYA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.754624732 (6h21m41.34396972s) [noise=true , address=104.36.82.8:4992 , piece_id=226HGROQ3DBKTJB3U6TG2OF4JQRYAGTLMSIL2JIL4NY5PISP4RUQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.763605976 (6h21m41.352950963s) [noise=true , address=104.36.82.60:4345 , piece_id=VPYOQHGEP6E257WI7IGIWXRO5WOLCNGDJYNPFPC56WUMOYDUS6RA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.763623339 (6h21m41.352968336s) [seg_pos={0 0}, upload=6610]: Segment reached optimal threshold of 14 pieces.
Aug 7 16:01:41.763741525 (6h21m41.353086512s) [noise=true , address=104.36.82.10:4110 , piece_id=WAUEFOZXH3SULGTXTWXWLEPYHZJWJDDQRRWWXRBXVWF7WCWPYGYQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ADbnPJGRTw2NqRzhy7ptojGYoNdJgqmjuWBSjcrQK53Wm7LXi): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763803123 (6h21m41.35314811s) [noise=true , address=104.36.82.61:4387 , piece_id=MAWGNSJTYYH7EVPDIXQEG6KP3MKACSMWYVMMLYUASUDJBIX5POOA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P4wAXSyaRY9poPRK5q4td8DJM69xWioG38TTwLTHKHA7QFxLp): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763857136 (6h21m41.353202113s) [noise=true , address=45.91.116.9:4267 , piece_id=MWAYSXFIIYQGDPTLWZSAHZP3UO72EXYDSSULHBJJU54H5OWG36NA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6M5TgGqUwjXyM25uqv5hsh67mS81ByzMM4kWFUNaFvisDN9d): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763904968 (6h21m41.353249955s) [noise=true , address=104.36.82.23:4455 , piece_id=UJOJ7P3VET4XIM257B3SHAN6SJG2H64QJHV6PYU7OEB7PPBZVDAA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wsyjF5X3iPrJsezUjakUArxc8K88T36yT4rE3yu6q9BMKfycm): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763945405 (6h21m41.353290392s) [seg_pos={0 0}, upload=6610]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 16:01:41.764033824 (6h21m41.353378811s) [upload=6610]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 16:01:41.764038263 (6h21m41.35338325s) [upload=6610]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 16:01:42.043459384 (6h21m41.632804371s) [upload=6610]: Done uploading segments...
Aug 7 16:01:42.043475946 (6h21m41.632820933s) [upload=6610]: Upload finished. err: <nil>
Aug 7 16:02:11.915996790 (6h22m11.505341777s) [upload=6619]: Starting upload
Aug 7 16:02:11.916045183 (6h22m11.50539017s) [upload=6619]: Done starting upload
Aug 7 16:02:11.916152489 (6h22m11.505497476s) [upload=6619]: Uploading segments...
Aug 7 16:02:11.916287706 (6h22m11.505632693s) [upload=6619]: Got next segment. Inline: false
Aug 7 16:02:11.916319939 (6h22m11.505664926s) [upload=6619]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 16:02:11.916324708 (6h22m11.505669695s) [upload=6619]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 16:02:12.129590885 (6h22m11.718935892s) [upload=6619]: Next segment err: metaclient: Too Many Requests
Aug 7 16:02:12.129623196 (6h22m11.718968194s) [upload=6619]: Done uploading segments...
Aug 7 16:02:12.129640310 (6h22m11.718985307s) [upload=6619]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.114517921s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
1.844802009s
FAIL (2025-08-07 16:07:21 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== entMakeInline Aug 7 15:57:10.271829415 (6h17m9.861174403s) [upload=6533]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 15:57:10.587098885 (6h17m10.176443872s) [upload=6533]: Done uploading segments... Aug 7 15:57:10.587115857 (6h17m10.176460835s) [upload=6533]: Upload finished. err: <nil> Aug 7 15:57:48.918901442 (6h17m48.508246429s) [upload=6545]: Starting upload Aug 7 15:57:48.918948701 (6h17m48.508293688s) [upload=6545]: Done starting upload Aug 7 15:57:48.919013486 (6h17m48.508358473s) [upload=6545]: Uploading segments... Aug 7 15:57:48.919055005 (6h17m48.508399982s) [upload=6545]: Got next segment. Inline: true Aug 7 15:57:48.919059032 (6h17m48.508404009s) [upload=6545]: Waiting for error group managing segments... Aug 7 15:57:48.919091585 (6h17m48.508436572s) [upload=6545]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 15:57:48.919095493 (6h17m48.50844048s) [upload=6545]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 15:57:48.919097998 (6h17m48.508442975s) [upload=6545]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 15:57:49.200378977 (6h17m48.789723964s) [upload=6545]: Done uploading segments... Aug 7 15:57:49.200397332 (6h17m48.789742319s) [upload=6545]: Upload finished. err: <nil> Aug 7 15:58:50.898935036 (6h18m50.488280023s) [upload=6562]: Starting upload Aug 7 15:58:50.898979992 (6h18m50.488324979s) [upload=6562]: Done starting upload Aug 7 15:58:50.899026682 (6h18m50.488371669s) [upload=6562]: Uploading segments... Aug 7 15:58:50.899065355 (6h18m50.488410343s) [upload=6562]: Got next segment. Inline: true Aug 7 15:58:50.899070335 (6h18m50.488415322s) [upload=6562]: Waiting for error group managing segments... Aug 7 15:58:50.899108268 (6h18m50.488453255s) [upload=6562]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 15:58:50.899112326 (6h18m50.488457313s) [upload=6562]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 15:58:50.899114820 (6h18m50.488459807s) [upload=6562]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 15:58:51.203289783 (6h18m50.79263478s) [upload=6562]: Done uploading segments... Aug 7 15:58:51.203312587 (6h18m50.792657624s) [upload=6562]: Upload finished. err: <nil> Aug 7 16:00:00.470970125 (6h20m0.060315112s) [upload=6580]: Starting upload Aug 7 16:00:00.471017334 (6h20m0.060362321s) [upload=6580]: Done starting upload Aug 7 16:00:00.471072199 (6h20m0.060417186s) [upload=6580]: Uploading segments... Aug 7 16:00:00.471112327 (6h20m0.060457314s) [upload=6580]: Got next segment. Inline: true Aug 7 16:00:00.471116364 (6h20m0.060461351s) [upload=6580]: Waiting for error group managing segments... Aug 7 16:00:00.471149427 (6h20m0.060494414s) [upload=6580]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:00:00.471153305 (6h20m0.060498292s) [upload=6580]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:00:00.471155819 (6h20m0.060500806s) [upload=6580]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:00:00.739624026 (6h20m0.328969023s) [upload=6580]: Done uploading segments... Aug 7 16:00:00.739645636 (6h20m0.328990623s) [upload=6580]: Upload finished. err: <nil> Aug 7 16:00:49.882359165 (6h20m49.471704152s) [upload=6598]: Starting upload Aug 7 16:00:49.882404682 (6h20m49.471749659s) [upload=6598]: Done starting upload Aug 7 16:00:49.882456812 (6h20m49.471801799s) [upload=6598]: Uploading segments... Aug 7 16:00:49.882489865 (6h20m49.471834852s) [upload=6598]: Got next segment. Inline: true Aug 7 16:00:49.882493702 (6h20m49.47183869s) [upload=6598]: Waiting for error group managing segments... Aug 7 16:00:49.882525894 (6h20m49.471870881s) [upload=6598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:00:49.882529422 (6h20m49.471874399s) [upload=6598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:00:49.882531676 (6h20m49.471876663s) [upload=6598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:00:50.213720013 (6h20m49.803065s) [upload=6598]: Done uploading segments... Aug 7 16:00:50.213739550 (6h20m49.803084537s) [upload=6598]: Upload finished. err: <nil> Aug 7 16:01:51.762778733 (6h21m51.35212372s) [upload=6617]: Starting upload Aug 7 16:01:51.762856161 (6h21m51.352201148s) [upload=6617]: Done starting upload Aug 7 16:01:51.762914883 (6h21m51.35225987s) [upload=6617]: Uploading segments... Aug 7 16:01:51.762952445 (6h21m51.352297432s) [upload=6617]: Got next segment. Inline: true Aug 7 16:01:51.762957344 (6h21m51.352302331s) [upload=6617]: Waiting for error group managing segments... Aug 7 16:01:51.762994485 (6h21m51.352339473s) [upload=6617]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:01:51.762998313 (6h21m51.3523433s) [upload=6617]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:01:51.763001278 (6h21m51.352346265s) [upload=6617]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:01:52.034377407 (6h21m51.623722394s) [upload=6617]: Done uploading segments... Aug 7 16:01:52.034395451 (6h21m51.623740439s) [upload=6617]: Upload finished. err: <nil> Aug 7 16:02:11.154406473 (6h22m10.74375146s) [upload=6618]: Starting upload Aug 7 16:02:11.154456328 (6h22m10.743801315s) [upload=6618]: Done starting upload Aug 7 16:02:11.154503859 (6h22m10.743848846s) [upload=6618]: Uploading segments... Aug 7 16:02:11.154545308 (6h22m10.743890295s) [upload=6618]: Got next segment. Inline: true Aug 7 16:02:11.154548574 (6h22m10.743893561s) [upload=6618]: Waiting for error group managing segments... Aug 7 16:02:11.154580566 (6h22m10.743925553s) [upload=6618]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:02:11.154584283 (6h22m10.74392927s) [upload=6618]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:02:11.154586828 (6h22m10.743931815s) [upload=6618]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:02:11.475329609 (6h22m11.064674596s) [upload=6618]: Done uploading segments... Aug 7 16:02:11.475484125 (6h22m11.064829112s) [upload=6618]: Upload finished. err: <nil> Aug 7 16:02:42.941029250 (6h22m42.530374238s) [upload=6628]: Starting upload Aug 7 16:02:42.941096740 (6h22m42.530441727s) [upload=6628]: Done starting upload Aug 7 16:02:42.941158728 (6h22m42.530503716s) [upload=6628]: Uploading segments... Aug 7 16:02:42.941210478 (6h22m42.530555465s) [upload=6628]: Got next segment. Inline: true Aug 7 16:02:42.941216419 (6h22m42.530561416s) [upload=6628]: Waiting for error group managing segments... Aug 7 16:02:42.941281664 (6h22m42.530626651s) [upload=6628]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:02:42.941286824 (6h22m42.530631811s) [upload=6628]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:02:42.941289880 (6h22m42.530634867s) [upload=6628]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:02:43.233814342 (6h22m42.823159339s) [upload=6628]: Done uploading segments... Aug 7 16:02:43.233838148 (6h22m42.823183145s) [upload=6628]: Upload finished. err: <nil> Aug 7 16:03:45.201690833 (6h23m44.79103582s) [upload=6651]: Starting upload Aug 7 16:03:45.201738173 (6h23m44.79108316s) [upload=6651]: Done starting upload Aug 7 16:03:45.201785534 (6h23m44.791130511s) [upload=6651]: Uploading segments... Aug 7 16:03:45.201855618 (6h23m44.791200605s) [upload=6651]: Got next segment. Inline: true Aug 7 16:03:45.201859616 (6h23m44.791204603s) [upload=6651]: Waiting for error group managing segments... Aug 7 16:03:45.201894122 (6h23m44.791239109s) [upload=6651]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:03:45.201897799 (6h23m44.791242786s) [upload=6651]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:03:45.201900023 (6h23m44.79124501s) [upload=6651]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:03:45.644503676 (6h23m45.233848664s) [upload=6651]: Done uploading segments... Aug 7 16:03:45.644521010 (6h23m45.233865987s) [upload=6651]: Upload finished. err: <nil> Aug 7 16:04:31.274936212 (6h24m30.864281199s) [upload=6663]: Starting upload Aug 7 16:04:31.274977170 (6h24m30.864322157s) [upload=6663]: Done starting upload Aug 7 16:04:31.275026655 (6h24m30.864371642s) [upload=6663]: Uploading segments... Aug 7 16:04:31.275054187 (6h24m30.864399174s) [upload=6663]: Got next segment. Inline: true Aug 7 16:04:31.275057754 (6h24m30.864402731s) [upload=6663]: Waiting for error group managing segments... Aug 7 16:04:31.275091278 (6h24m30.864436265s) [upload=6663]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:04:31.275095255 (6h24m30.864440242s) [upload=6663]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:04:31.275097651 (6h24m30.864442638s) [upload=6663]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:04:31.554258512 (6h24m31.143603499s) [upload=6663]: Done uploading segments... Aug 7 16:04:31.554277108 (6h24m31.143622095s) [upload=6663]: Upload finished. err: <nil> Aug 7 16:05:14.730035713 (6h25m14.3193807s) [upload=6674]: Starting upload Aug 7 16:05:14.730082813 (6h25m14.3194278s) [upload=6674]: Done starting upload Aug 7 16:05:14.730159079 (6h25m14.319504056s) [upload=6674]: Uploading segments... Aug 7 16:05:14.730249532 (6h25m14.319594519s) [upload=6674]: Got next segment. Inline: true Aug 7 16:05:14.730254050 (6h25m14.319599037s) [upload=6674]: Waiting for error group managing segments... Aug 7 16:05:14.730294949 (6h25m14.319639936s) [upload=6674]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:05:14.730299728 (6h25m14.319644715s) [upload=6674]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:05:14.730302243 (6h25m14.31964723s) [upload=6674]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:05:14.942773689 (6h25m14.532118666s) [upload=6674]: Done uploading segments... Aug 7 16:05:14.942790141 (6h25m14.532135118s) [upload=6674]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.844815995s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [..........................................X.......X.........] duration:
5.830530025s
FAIL (2025-08-07 16:07:11 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
e begin upload segment.
Aug 7 16:01:39.748945470 (6h21m39.338290447s) [noise=true , address=104.36.82.42:4820 , piece_id=XJIHI2E35NJKDFYFTW4V5EIF3CL7GDJ5EKVCTEV2KL26X43A3CYA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.748969657 (6h21m39.338314644s) [noise=true , address=104.36.82.60:4345 , piece_id=VPYOQHGEP6E257WI7IGIWXRO5WOLCNGDJYNPFPC56WUMOYDUS6RA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749368329 (6h21m39.338713316s) [noise=true , address=45.91.116.9:4267 , piece_id=MWAYSXFIIYQGDPTLWZSAHZP3UO72EXYDSSULHBJJU54H5OWG36NA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749375964 (6h21m39.338720951s) [noise=true , address=104.36.82.10:4110 , piece_id=WAUEFOZXH3SULGTXTWXWLEPYHZJWJDDQRRWWXRBXVWF7WCWPYGYQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749660398 (6h21m39.339005385s) [noise=true , address=79.127.205.235:4666 , piece_id=WA6GZKNO2UUAC5VKBNOZ7UO25RQR7QDT5HVNQYZDIWJEB656WMGA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749701216 (6h21m39.339046203s) [noise=true , address=79.127.201.215:4345 , piece_id=MAAFKAPO4OCQXU5WJ6DAP72KEHIM4AO5JJFJ4ZEFNWKS5M4OV43Q, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749938690 (6h21m39.339283677s) [noise=true , address=104.36.82.59:4313 , piece_id=KDH6AVALUMDXB6USO7WG77DVOSRBTGSZA6VTJFFQJR7QFC2PE35A, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750024905 (6h21m39.339369892s) [noise=true , address=104.36.82.23:4455 , piece_id=UJOJ7P3VET4XIM257B3SHAN6SJG2H64QJHV6PYU7OEB7PPBZVDAA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750215089 (6h21m39.339560076s) [noise=true , address=104.36.82.61:4387 , piece_id=MAWGNSJTYYH7EVPDIXQEG6KP3MKACSMWYVMMLYUASUDJBIX5POOA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750282639 (6h21m39.339627626s) [noise=true , address=45.91.116.10:4092 , piece_id=BUVKJC3K3RKZZTW2CQPEQM7ECQOLIBX4CU66P26JZDTOQZLYGMGA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750481038 (6h21m39.339826025s) [noise=true , address=79.127.220.97:4396 , piece_id=X2CCRNYWY5ZQOAODHNJE7VLYQIHD37PEE7MQXI5WIYV36ICONILQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750533900 (6h21m39.339878887s) [noise=true , address=104.36.82.60:4369 , piece_id=Y6SYJOB7SYBJX7STC25OJJPD4FSK5TZ5KXI7WN7QS42UNTOF5BCQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750725967 (6h21m39.340070954s) [noise=true , address=104.36.82.9:4039 , piece_id=PRBG5EVKAZ3LGDRX4MCI57T4RSNFVKZVUNOAIRHBM6SNS5XXUTPQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750795189 (6h21m39.340140176s) [noise=true , address=104.36.82.22:4525 , piece_id=NKD4EXV774KNXLEHICE4MNKOQS5QKPIUTJL2PPCNYUMYPMQEF75A, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750954023 (6h21m39.34029901s) [noise=true , address=104.36.82.45:4912 , piece_id=B6HTS6ORSGMUKZO5G2ALW4LVUSIBTCEHPX2YJTYPPCH3XROIO3HA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.751098079 (6h21m39.340443066s) [noise=true , address=104.36.82.8:4992 , piece_id=226HGROQ3DBKTJB3U6TG2OF4JQRYAGTLMSIL2JIL4NY5PISP4RUQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.751175397 (6h21m39.340520384s) [noise=true , address=79.127.205.233:4631 , piece_id=77WNKENAMS4Q75KYK4F4VNRP4DD4MKGEJIU5K7WZ3EHVQLEW6Q5A, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.751392602 (6h21m39.340737579s) [noise=true , address=104.36.82.5:4845 , piece_id=JKN4VFS7ET2LVGHGE4LIBPCGG33GNY4SGSOXAWB2MXFYPJVO4FCQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:40.878402060 (6h21m40.467747047s) [upload=6610]: Next returned nil segment
Aug 7 16:01:40.878416708 (6h21m40.467761695s) [upload=6610]: Waiting for error group managing segments...
Aug 7 16:01:41.195597254 (6h21m40.784942241s) [noise=true , address=79.127.205.235:4666 , piece_id=WA6GZKNO2UUAC5VKBNOZ7UO25RQR7QDT5HVNQYZDIWJEB656WMGA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.261081133 (6h21m40.85042612s) [noise=true , address=79.127.205.233:4631 , piece_id=77WNKENAMS4Q75KYK4F4VNRP4DD4MKGEJIU5K7WZ3EHVQLEW6Q5A, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.420961069 (6h21m41.010306046s) [noise=true , address=79.127.220.97:4396 , piece_id=X2CCRNYWY5ZQOAODHNJE7VLYQIHD37PEE7MQXI5WIYV36ICONILQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.638341479 (6h21m41.227686486s) [noise=true , address=79.127.201.215:4345 , piece_id=MAAFKAPO4OCQXU5WJ6DAP72KEHIM4AO5JJFJ4ZEFNWKS5M4OV43Q, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.658966771 (6h21m41.248311758s) [noise=true , address=45.91.116.10:4092 , piece_id=BUVKJC3K3RKZZTW2CQPEQM7ECQOLIBX4CU66P26JZDTOQZLYGMGA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.684178459 (6h21m41.273523446s) [noise=true , address=104.36.82.9:4039 , piece_id=PRBG5EVKAZ3LGDRX4MCI57T4RSNFVKZVUNOAIRHBM6SNS5XXUTPQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.691448508 (6h21m41.280793485s) [noise=true , address=104.36.82.22:4525 , piece_id=NKD4EXV774KNXLEHICE4MNKOQS5QKPIUTJL2PPCNYUMYPMQEF75A, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.717147267 (6h21m41.306492254s) [noise=true , address=104.36.82.59:4313 , piece_id=KDH6AVALUMDXB6USO7WG77DVOSRBTGSZA6VTJFFQJR7QFC2PE35A, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.719612885 (6h21m41.308957872s) [noise=true , address=104.36.82.45:4912 , piece_id=B6HTS6ORSGMUKZO5G2ALW4LVUSIBTCEHPX2YJTYPPCH3XROIO3HA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.733973160 (6h21m41.323318147s) [noise=true , address=104.36.82.60:4369 , piece_id=Y6SYJOB7SYBJX7STC25OJJPD4FSK5TZ5KXI7WN7QS42UNTOF5BCQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.737980718 (6h21m41.327325705s) [noise=true , address=104.36.82.5:4845 , piece_id=JKN4VFS7ET2LVGHGE4LIBPCGG33GNY4SGSOXAWB2MXFYPJVO4FCQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.745434729 (6h21m41.334779717s) [noise=true , address=104.36.82.42:4820 , piece_id=XJIHI2E35NJKDFYFTW4V5EIF3CL7GDJ5EKVCTEV2KL26X43A3CYA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.754624732 (6h21m41.34396972s) [noise=true , address=104.36.82.8:4992 , piece_id=226HGROQ3DBKTJB3U6TG2OF4JQRYAGTLMSIL2JIL4NY5PISP4RUQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.763605976 (6h21m41.352950963s) [noise=true , address=104.36.82.60:4345 , piece_id=VPYOQHGEP6E257WI7IGIWXRO5WOLCNGDJYNPFPC56WUMOYDUS6RA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.763623339 (6h21m41.352968336s) [seg_pos={0 0}, upload=6610]: Segment reached optimal threshold of 14 pieces.
Aug 7 16:01:41.763741525 (6h21m41.353086512s) [noise=true , address=104.36.82.10:4110 , piece_id=WAUEFOZXH3SULGTXTWXWLEPYHZJWJDDQRRWWXRBXVWF7WCWPYGYQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ADbnPJGRTw2NqRzhy7ptojGYoNdJgqmjuWBSjcrQK53Wm7LXi): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763803123 (6h21m41.35314811s) [noise=true , address=104.36.82.61:4387 , piece_id=MAWGNSJTYYH7EVPDIXQEG6KP3MKACSMWYVMMLYUASUDJBIX5POOA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P4wAXSyaRY9poPRK5q4td8DJM69xWioG38TTwLTHKHA7QFxLp): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763857136 (6h21m41.353202113s) [noise=true , address=45.91.116.9:4267 , piece_id=MWAYSXFIIYQGDPTLWZSAHZP3UO72EXYDSSULHBJJU54H5OWG36NA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6M5TgGqUwjXyM25uqv5hsh67mS81ByzMM4kWFUNaFvisDN9d): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763904968 (6h21m41.353249955s) [noise=true , address=104.36.82.23:4455 , piece_id=UJOJ7P3VET4XIM257B3SHAN6SJG2H64QJHV6PYU7OEB7PPBZVDAA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wsyjF5X3iPrJsezUjakUArxc8K88T36yT4rE3yu6q9BMKfycm): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763945405 (6h21m41.353290392s) [seg_pos={0 0}, upload=6610]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 16:01:41.764033824 (6h21m41.353378811s) [upload=6610]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 16:01:41.764038263 (6h21m41.35338325s) [upload=6610]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 16:01:42.043459384 (6h21m41.632804371s) [upload=6610]: Done uploading segments...
Aug 7 16:01:42.043475946 (6h21m41.632820933s) [upload=6610]: Upload finished. err: <nil>
Aug 7 16:02:11.915996790 (6h22m11.505341777s) [upload=6619]: Starting upload
Aug 7 16:02:11.916045183 (6h22m11.50539017s) [upload=6619]: Done starting upload
Aug 7 16:02:11.916152489 (6h22m11.505497476s) [upload=6619]: Uploading segments...
Aug 7 16:02:11.916287706 (6h22m11.505632693s) [upload=6619]: Got next segment. Inline: false
Aug 7 16:02:11.916319939 (6h22m11.505664926s) [upload=6619]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 16:02:11.916324708 (6h22m11.505669695s) [upload=6619]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 16:02:12.129590885 (6h22m11.718935892s) [upload=6619]: Next segment err: metaclient: Too Many Requests
Aug 7 16:02:12.129623196 (6h22m11.718968194s) [upload=6619]: Done uploading segments...
Aug 7 16:02:12.129640310 (6h22m11.718985307s) [upload=6619]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.83055935s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
1.874387743s
FAIL (2025-08-07 16:07:02 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== entMakeInline Aug 7 15:57:10.271829415 (6h17m9.861174403s) [upload=6533]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 15:57:10.587098885 (6h17m10.176443872s) [upload=6533]: Done uploading segments... Aug 7 15:57:10.587115857 (6h17m10.176460835s) [upload=6533]: Upload finished. err: <nil> Aug 7 15:57:48.918901442 (6h17m48.508246429s) [upload=6545]: Starting upload Aug 7 15:57:48.918948701 (6h17m48.508293688s) [upload=6545]: Done starting upload Aug 7 15:57:48.919013486 (6h17m48.508358473s) [upload=6545]: Uploading segments... Aug 7 15:57:48.919055005 (6h17m48.508399982s) [upload=6545]: Got next segment. Inline: true Aug 7 15:57:48.919059032 (6h17m48.508404009s) [upload=6545]: Waiting for error group managing segments... Aug 7 15:57:48.919091585 (6h17m48.508436572s) [upload=6545]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 15:57:48.919095493 (6h17m48.50844048s) [upload=6545]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 15:57:48.919097998 (6h17m48.508442975s) [upload=6545]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 15:57:49.200378977 (6h17m48.789723964s) [upload=6545]: Done uploading segments... Aug 7 15:57:49.200397332 (6h17m48.789742319s) [upload=6545]: Upload finished. err: <nil> Aug 7 15:58:50.898935036 (6h18m50.488280023s) [upload=6562]: Starting upload Aug 7 15:58:50.898979992 (6h18m50.488324979s) [upload=6562]: Done starting upload Aug 7 15:58:50.899026682 (6h18m50.488371669s) [upload=6562]: Uploading segments... Aug 7 15:58:50.899065355 (6h18m50.488410343s) [upload=6562]: Got next segment. Inline: true Aug 7 15:58:50.899070335 (6h18m50.488415322s) [upload=6562]: Waiting for error group managing segments... Aug 7 15:58:50.899108268 (6h18m50.488453255s) [upload=6562]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 15:58:50.899112326 (6h18m50.488457313s) [upload=6562]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 15:58:50.899114820 (6h18m50.488459807s) [upload=6562]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 15:58:51.203289783 (6h18m50.79263478s) [upload=6562]: Done uploading segments... Aug 7 15:58:51.203312587 (6h18m50.792657624s) [upload=6562]: Upload finished. err: <nil> Aug 7 16:00:00.470970125 (6h20m0.060315112s) [upload=6580]: Starting upload Aug 7 16:00:00.471017334 (6h20m0.060362321s) [upload=6580]: Done starting upload Aug 7 16:00:00.471072199 (6h20m0.060417186s) [upload=6580]: Uploading segments... Aug 7 16:00:00.471112327 (6h20m0.060457314s) [upload=6580]: Got next segment. Inline: true Aug 7 16:00:00.471116364 (6h20m0.060461351s) [upload=6580]: Waiting for error group managing segments... Aug 7 16:00:00.471149427 (6h20m0.060494414s) [upload=6580]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:00:00.471153305 (6h20m0.060498292s) [upload=6580]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:00:00.471155819 (6h20m0.060500806s) [upload=6580]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:00:00.739624026 (6h20m0.328969023s) [upload=6580]: Done uploading segments... Aug 7 16:00:00.739645636 (6h20m0.328990623s) [upload=6580]: Upload finished. err: <nil> Aug 7 16:00:49.882359165 (6h20m49.471704152s) [upload=6598]: Starting upload Aug 7 16:00:49.882404682 (6h20m49.471749659s) [upload=6598]: Done starting upload Aug 7 16:00:49.882456812 (6h20m49.471801799s) [upload=6598]: Uploading segments... Aug 7 16:00:49.882489865 (6h20m49.471834852s) [upload=6598]: Got next segment. Inline: true Aug 7 16:00:49.882493702 (6h20m49.47183869s) [upload=6598]: Waiting for error group managing segments... Aug 7 16:00:49.882525894 (6h20m49.471870881s) [upload=6598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:00:49.882529422 (6h20m49.471874399s) [upload=6598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:00:49.882531676 (6h20m49.471876663s) [upload=6598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:00:50.213720013 (6h20m49.803065s) [upload=6598]: Done uploading segments... Aug 7 16:00:50.213739550 (6h20m49.803084537s) [upload=6598]: Upload finished. err: <nil> Aug 7 16:01:51.762778733 (6h21m51.35212372s) [upload=6617]: Starting upload Aug 7 16:01:51.762856161 (6h21m51.352201148s) [upload=6617]: Done starting upload Aug 7 16:01:51.762914883 (6h21m51.35225987s) [upload=6617]: Uploading segments... Aug 7 16:01:51.762952445 (6h21m51.352297432s) [upload=6617]: Got next segment. Inline: true Aug 7 16:01:51.762957344 (6h21m51.352302331s) [upload=6617]: Waiting for error group managing segments... Aug 7 16:01:51.762994485 (6h21m51.352339473s) [upload=6617]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:01:51.762998313 (6h21m51.3523433s) [upload=6617]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:01:51.763001278 (6h21m51.352346265s) [upload=6617]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:01:52.034377407 (6h21m51.623722394s) [upload=6617]: Done uploading segments... Aug 7 16:01:52.034395451 (6h21m51.623740439s) [upload=6617]: Upload finished. err: <nil> Aug 7 16:02:11.154406473 (6h22m10.74375146s) [upload=6618]: Starting upload Aug 7 16:02:11.154456328 (6h22m10.743801315s) [upload=6618]: Done starting upload Aug 7 16:02:11.154503859 (6h22m10.743848846s) [upload=6618]: Uploading segments... Aug 7 16:02:11.154545308 (6h22m10.743890295s) [upload=6618]: Got next segment. Inline: true Aug 7 16:02:11.154548574 (6h22m10.743893561s) [upload=6618]: Waiting for error group managing segments... Aug 7 16:02:11.154580566 (6h22m10.743925553s) [upload=6618]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:02:11.154584283 (6h22m10.74392927s) [upload=6618]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:02:11.154586828 (6h22m10.743931815s) [upload=6618]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:02:11.475329609 (6h22m11.064674596s) [upload=6618]: Done uploading segments... Aug 7 16:02:11.475484125 (6h22m11.064829112s) [upload=6618]: Upload finished. err: <nil> Aug 7 16:02:42.941029250 (6h22m42.530374238s) [upload=6628]: Starting upload Aug 7 16:02:42.941096740 (6h22m42.530441727s) [upload=6628]: Done starting upload Aug 7 16:02:42.941158728 (6h22m42.530503716s) [upload=6628]: Uploading segments... Aug 7 16:02:42.941210478 (6h22m42.530555465s) [upload=6628]: Got next segment. Inline: true Aug 7 16:02:42.941216419 (6h22m42.530561416s) [upload=6628]: Waiting for error group managing segments... Aug 7 16:02:42.941281664 (6h22m42.530626651s) [upload=6628]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:02:42.941286824 (6h22m42.530631811s) [upload=6628]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:02:42.941289880 (6h22m42.530634867s) [upload=6628]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:02:43.233814342 (6h22m42.823159339s) [upload=6628]: Done uploading segments... Aug 7 16:02:43.233838148 (6h22m42.823183145s) [upload=6628]: Upload finished. err: <nil> Aug 7 16:03:45.201690833 (6h23m44.79103582s) [upload=6651]: Starting upload Aug 7 16:03:45.201738173 (6h23m44.79108316s) [upload=6651]: Done starting upload Aug 7 16:03:45.201785534 (6h23m44.791130511s) [upload=6651]: Uploading segments... Aug 7 16:03:45.201855618 (6h23m44.791200605s) [upload=6651]: Got next segment. Inline: true Aug 7 16:03:45.201859616 (6h23m44.791204603s) [upload=6651]: Waiting for error group managing segments... Aug 7 16:03:45.201894122 (6h23m44.791239109s) [upload=6651]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:03:45.201897799 (6h23m44.791242786s) [upload=6651]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:03:45.201900023 (6h23m44.79124501s) [upload=6651]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:03:45.644503676 (6h23m45.233848664s) [upload=6651]: Done uploading segments... Aug 7 16:03:45.644521010 (6h23m45.233865987s) [upload=6651]: Upload finished. err: <nil> Aug 7 16:04:31.274936212 (6h24m30.864281199s) [upload=6663]: Starting upload Aug 7 16:04:31.274977170 (6h24m30.864322157s) [upload=6663]: Done starting upload Aug 7 16:04:31.275026655 (6h24m30.864371642s) [upload=6663]: Uploading segments... Aug 7 16:04:31.275054187 (6h24m30.864399174s) [upload=6663]: Got next segment. Inline: true Aug 7 16:04:31.275057754 (6h24m30.864402731s) [upload=6663]: Waiting for error group managing segments... Aug 7 16:04:31.275091278 (6h24m30.864436265s) [upload=6663]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:04:31.275095255 (6h24m30.864440242s) [upload=6663]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:04:31.275097651 (6h24m30.864442638s) [upload=6663]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:04:31.554258512 (6h24m31.143603499s) [upload=6663]: Done uploading segments... Aug 7 16:04:31.554277108 (6h24m31.143622095s) [upload=6663]: Upload finished. err: <nil> Aug 7 16:05:14.730035713 (6h25m14.3193807s) [upload=6674]: Starting upload Aug 7 16:05:14.730082813 (6h25m14.3194278s) [upload=6674]: Done starting upload Aug 7 16:05:14.730159079 (6h25m14.319504056s) [upload=6674]: Uploading segments... Aug 7 16:05:14.730249532 (6h25m14.319594519s) [upload=6674]: Got next segment. Inline: true Aug 7 16:05:14.730254050 (6h25m14.319599037s) [upload=6674]: Waiting for error group managing segments... Aug 7 16:05:14.730294949 (6h25m14.319639936s) [upload=6674]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:05:14.730299728 (6h25m14.319644715s) [upload=6674]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:05:14.730302243 (6h25m14.31964723s) [upload=6674]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:05:14.942773689 (6h25m14.532118666s) [upload=6674]: Done uploading segments... Aug 7 16:05:14.942790141 (6h25m14.532135118s) [upload=6674]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.874411188s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
1.874124099s
FAIL (2025-08-07 16:06:27 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== entMakeInline Aug 7 15:57:10.271829415 (6h17m9.861174403s) [upload=6533]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 15:57:10.587098885 (6h17m10.176443872s) [upload=6533]: Done uploading segments... Aug 7 15:57:10.587115857 (6h17m10.176460835s) [upload=6533]: Upload finished. err: <nil> Aug 7 15:57:48.918901442 (6h17m48.508246429s) [upload=6545]: Starting upload Aug 7 15:57:48.918948701 (6h17m48.508293688s) [upload=6545]: Done starting upload Aug 7 15:57:48.919013486 (6h17m48.508358473s) [upload=6545]: Uploading segments... Aug 7 15:57:48.919055005 (6h17m48.508399982s) [upload=6545]: Got next segment. Inline: true Aug 7 15:57:48.919059032 (6h17m48.508404009s) [upload=6545]: Waiting for error group managing segments... Aug 7 15:57:48.919091585 (6h17m48.508436572s) [upload=6545]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 15:57:48.919095493 (6h17m48.50844048s) [upload=6545]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 15:57:48.919097998 (6h17m48.508442975s) [upload=6545]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 15:57:49.200378977 (6h17m48.789723964s) [upload=6545]: Done uploading segments... Aug 7 15:57:49.200397332 (6h17m48.789742319s) [upload=6545]: Upload finished. err: <nil> Aug 7 15:58:50.898935036 (6h18m50.488280023s) [upload=6562]: Starting upload Aug 7 15:58:50.898979992 (6h18m50.488324979s) [upload=6562]: Done starting upload Aug 7 15:58:50.899026682 (6h18m50.488371669s) [upload=6562]: Uploading segments... Aug 7 15:58:50.899065355 (6h18m50.488410343s) [upload=6562]: Got next segment. Inline: true Aug 7 15:58:50.899070335 (6h18m50.488415322s) [upload=6562]: Waiting for error group managing segments... Aug 7 15:58:50.899108268 (6h18m50.488453255s) [upload=6562]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 15:58:50.899112326 (6h18m50.488457313s) [upload=6562]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 15:58:50.899114820 (6h18m50.488459807s) [upload=6562]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 15:58:51.203289783 (6h18m50.79263478s) [upload=6562]: Done uploading segments... Aug 7 15:58:51.203312587 (6h18m50.792657624s) [upload=6562]: Upload finished. err: <nil> Aug 7 16:00:00.470970125 (6h20m0.060315112s) [upload=6580]: Starting upload Aug 7 16:00:00.471017334 (6h20m0.060362321s) [upload=6580]: Done starting upload Aug 7 16:00:00.471072199 (6h20m0.060417186s) [upload=6580]: Uploading segments... Aug 7 16:00:00.471112327 (6h20m0.060457314s) [upload=6580]: Got next segment. Inline: true Aug 7 16:00:00.471116364 (6h20m0.060461351s) [upload=6580]: Waiting for error group managing segments... Aug 7 16:00:00.471149427 (6h20m0.060494414s) [upload=6580]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:00:00.471153305 (6h20m0.060498292s) [upload=6580]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:00:00.471155819 (6h20m0.060500806s) [upload=6580]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:00:00.739624026 (6h20m0.328969023s) [upload=6580]: Done uploading segments... Aug 7 16:00:00.739645636 (6h20m0.328990623s) [upload=6580]: Upload finished. err: <nil> Aug 7 16:00:49.882359165 (6h20m49.471704152s) [upload=6598]: Starting upload Aug 7 16:00:49.882404682 (6h20m49.471749659s) [upload=6598]: Done starting upload Aug 7 16:00:49.882456812 (6h20m49.471801799s) [upload=6598]: Uploading segments... Aug 7 16:00:49.882489865 (6h20m49.471834852s) [upload=6598]: Got next segment. Inline: true Aug 7 16:00:49.882493702 (6h20m49.47183869s) [upload=6598]: Waiting for error group managing segments... Aug 7 16:00:49.882525894 (6h20m49.471870881s) [upload=6598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:00:49.882529422 (6h20m49.471874399s) [upload=6598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:00:49.882531676 (6h20m49.471876663s) [upload=6598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:00:50.213720013 (6h20m49.803065s) [upload=6598]: Done uploading segments... Aug 7 16:00:50.213739550 (6h20m49.803084537s) [upload=6598]: Upload finished. err: <nil> Aug 7 16:01:51.762778733 (6h21m51.35212372s) [upload=6617]: Starting upload Aug 7 16:01:51.762856161 (6h21m51.352201148s) [upload=6617]: Done starting upload Aug 7 16:01:51.762914883 (6h21m51.35225987s) [upload=6617]: Uploading segments... Aug 7 16:01:51.762952445 (6h21m51.352297432s) [upload=6617]: Got next segment. Inline: true Aug 7 16:01:51.762957344 (6h21m51.352302331s) [upload=6617]: Waiting for error group managing segments... Aug 7 16:01:51.762994485 (6h21m51.352339473s) [upload=6617]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:01:51.762998313 (6h21m51.3523433s) [upload=6617]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:01:51.763001278 (6h21m51.352346265s) [upload=6617]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:01:52.034377407 (6h21m51.623722394s) [upload=6617]: Done uploading segments... Aug 7 16:01:52.034395451 (6h21m51.623740439s) [upload=6617]: Upload finished. err: <nil> Aug 7 16:02:11.154406473 (6h22m10.74375146s) [upload=6618]: Starting upload Aug 7 16:02:11.154456328 (6h22m10.743801315s) [upload=6618]: Done starting upload Aug 7 16:02:11.154503859 (6h22m10.743848846s) [upload=6618]: Uploading segments... Aug 7 16:02:11.154545308 (6h22m10.743890295s) [upload=6618]: Got next segment. Inline: true Aug 7 16:02:11.154548574 (6h22m10.743893561s) [upload=6618]: Waiting for error group managing segments... Aug 7 16:02:11.154580566 (6h22m10.743925553s) [upload=6618]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:02:11.154584283 (6h22m10.74392927s) [upload=6618]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:02:11.154586828 (6h22m10.743931815s) [upload=6618]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:02:11.475329609 (6h22m11.064674596s) [upload=6618]: Done uploading segments... Aug 7 16:02:11.475484125 (6h22m11.064829112s) [upload=6618]: Upload finished. err: <nil> Aug 7 16:02:42.941029250 (6h22m42.530374238s) [upload=6628]: Starting upload Aug 7 16:02:42.941096740 (6h22m42.530441727s) [upload=6628]: Done starting upload Aug 7 16:02:42.941158728 (6h22m42.530503716s) [upload=6628]: Uploading segments... Aug 7 16:02:42.941210478 (6h22m42.530555465s) [upload=6628]: Got next segment. Inline: true Aug 7 16:02:42.941216419 (6h22m42.530561416s) [upload=6628]: Waiting for error group managing segments... Aug 7 16:02:42.941281664 (6h22m42.530626651s) [upload=6628]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:02:42.941286824 (6h22m42.530631811s) [upload=6628]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:02:42.941289880 (6h22m42.530634867s) [upload=6628]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:02:43.233814342 (6h22m42.823159339s) [upload=6628]: Done uploading segments... Aug 7 16:02:43.233838148 (6h22m42.823183145s) [upload=6628]: Upload finished. err: <nil> Aug 7 16:03:45.201690833 (6h23m44.79103582s) [upload=6651]: Starting upload Aug 7 16:03:45.201738173 (6h23m44.79108316s) [upload=6651]: Done starting upload Aug 7 16:03:45.201785534 (6h23m44.791130511s) [upload=6651]: Uploading segments... Aug 7 16:03:45.201855618 (6h23m44.791200605s) [upload=6651]: Got next segment. Inline: true Aug 7 16:03:45.201859616 (6h23m44.791204603s) [upload=6651]: Waiting for error group managing segments... Aug 7 16:03:45.201894122 (6h23m44.791239109s) [upload=6651]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:03:45.201897799 (6h23m44.791242786s) [upload=6651]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:03:45.201900023 (6h23m44.79124501s) [upload=6651]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:03:45.644503676 (6h23m45.233848664s) [upload=6651]: Done uploading segments... Aug 7 16:03:45.644521010 (6h23m45.233865987s) [upload=6651]: Upload finished. err: <nil> Aug 7 16:04:31.274936212 (6h24m30.864281199s) [upload=6663]: Starting upload Aug 7 16:04:31.274977170 (6h24m30.864322157s) [upload=6663]: Done starting upload Aug 7 16:04:31.275026655 (6h24m30.864371642s) [upload=6663]: Uploading segments... Aug 7 16:04:31.275054187 (6h24m30.864399174s) [upload=6663]: Got next segment. Inline: true Aug 7 16:04:31.275057754 (6h24m30.864402731s) [upload=6663]: Waiting for error group managing segments... Aug 7 16:04:31.275091278 (6h24m30.864436265s) [upload=6663]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:04:31.275095255 (6h24m30.864440242s) [upload=6663]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:04:31.275097651 (6h24m30.864442638s) [upload=6663]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:04:31.554258512 (6h24m31.143603499s) [upload=6663]: Done uploading segments... Aug 7 16:04:31.554277108 (6h24m31.143622095s) [upload=6663]: Upload finished. err: <nil> Aug 7 16:05:14.730035713 (6h25m14.3193807s) [upload=6674]: Starting upload Aug 7 16:05:14.730082813 (6h25m14.3194278s) [upload=6674]: Done starting upload Aug 7 16:05:14.730159079 (6h25m14.319504056s) [upload=6674]: Uploading segments... Aug 7 16:05:14.730249532 (6h25m14.319594519s) [upload=6674]: Got next segment. Inline: true Aug 7 16:05:14.730254050 (6h25m14.319599037s) [upload=6674]: Waiting for error group managing segments... Aug 7 16:05:14.730294949 (6h25m14.319639936s) [upload=6674]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:05:14.730299728 (6h25m14.319644715s) [upload=6674]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:05:14.730302243 (6h25m14.31964723s) [upload=6674]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:05:14.942773689 (6h25m14.532118666s) [upload=6674]: Done uploading segments... Aug 7 16:05:14.942790141 (6h25m14.532135118s) [upload=6674]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.874148988s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [...........................................X.......X........] duration:
5.894047764s
FAIL (2025-08-07 16:06:10 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
e begin upload segment.
Aug 7 16:01:39.748945470 (6h21m39.338290447s) [noise=true , address=104.36.82.42:4820 , piece_id=XJIHI2E35NJKDFYFTW4V5EIF3CL7GDJ5EKVCTEV2KL26X43A3CYA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.748969657 (6h21m39.338314644s) [noise=true , address=104.36.82.60:4345 , piece_id=VPYOQHGEP6E257WI7IGIWXRO5WOLCNGDJYNPFPC56WUMOYDUS6RA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749368329 (6h21m39.338713316s) [noise=true , address=45.91.116.9:4267 , piece_id=MWAYSXFIIYQGDPTLWZSAHZP3UO72EXYDSSULHBJJU54H5OWG36NA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749375964 (6h21m39.338720951s) [noise=true , address=104.36.82.10:4110 , piece_id=WAUEFOZXH3SULGTXTWXWLEPYHZJWJDDQRRWWXRBXVWF7WCWPYGYQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749660398 (6h21m39.339005385s) [noise=true , address=79.127.205.235:4666 , piece_id=WA6GZKNO2UUAC5VKBNOZ7UO25RQR7QDT5HVNQYZDIWJEB656WMGA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749701216 (6h21m39.339046203s) [noise=true , address=79.127.201.215:4345 , piece_id=MAAFKAPO4OCQXU5WJ6DAP72KEHIM4AO5JJFJ4ZEFNWKS5M4OV43Q, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749938690 (6h21m39.339283677s) [noise=true , address=104.36.82.59:4313 , piece_id=KDH6AVALUMDXB6USO7WG77DVOSRBTGSZA6VTJFFQJR7QFC2PE35A, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750024905 (6h21m39.339369892s) [noise=true , address=104.36.82.23:4455 , piece_id=UJOJ7P3VET4XIM257B3SHAN6SJG2H64QJHV6PYU7OEB7PPBZVDAA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750215089 (6h21m39.339560076s) [noise=true , address=104.36.82.61:4387 , piece_id=MAWGNSJTYYH7EVPDIXQEG6KP3MKACSMWYVMMLYUASUDJBIX5POOA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750282639 (6h21m39.339627626s) [noise=true , address=45.91.116.10:4092 , piece_id=BUVKJC3K3RKZZTW2CQPEQM7ECQOLIBX4CU66P26JZDTOQZLYGMGA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750481038 (6h21m39.339826025s) [noise=true , address=79.127.220.97:4396 , piece_id=X2CCRNYWY5ZQOAODHNJE7VLYQIHD37PEE7MQXI5WIYV36ICONILQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750533900 (6h21m39.339878887s) [noise=true , address=104.36.82.60:4369 , piece_id=Y6SYJOB7SYBJX7STC25OJJPD4FSK5TZ5KXI7WN7QS42UNTOF5BCQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750725967 (6h21m39.340070954s) [noise=true , address=104.36.82.9:4039 , piece_id=PRBG5EVKAZ3LGDRX4MCI57T4RSNFVKZVUNOAIRHBM6SNS5XXUTPQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750795189 (6h21m39.340140176s) [noise=true , address=104.36.82.22:4525 , piece_id=NKD4EXV774KNXLEHICE4MNKOQS5QKPIUTJL2PPCNYUMYPMQEF75A, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750954023 (6h21m39.34029901s) [noise=true , address=104.36.82.45:4912 , piece_id=B6HTS6ORSGMUKZO5G2ALW4LVUSIBTCEHPX2YJTYPPCH3XROIO3HA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.751098079 (6h21m39.340443066s) [noise=true , address=104.36.82.8:4992 , piece_id=226HGROQ3DBKTJB3U6TG2OF4JQRYAGTLMSIL2JIL4NY5PISP4RUQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.751175397 (6h21m39.340520384s) [noise=true , address=79.127.205.233:4631 , piece_id=77WNKENAMS4Q75KYK4F4VNRP4DD4MKGEJIU5K7WZ3EHVQLEW6Q5A, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.751392602 (6h21m39.340737579s) [noise=true , address=104.36.82.5:4845 , piece_id=JKN4VFS7ET2LVGHGE4LIBPCGG33GNY4SGSOXAWB2MXFYPJVO4FCQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:40.878402060 (6h21m40.467747047s) [upload=6610]: Next returned nil segment
Aug 7 16:01:40.878416708 (6h21m40.467761695s) [upload=6610]: Waiting for error group managing segments...
Aug 7 16:01:41.195597254 (6h21m40.784942241s) [noise=true , address=79.127.205.235:4666 , piece_id=WA6GZKNO2UUAC5VKBNOZ7UO25RQR7QDT5HVNQYZDIWJEB656WMGA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.261081133 (6h21m40.85042612s) [noise=true , address=79.127.205.233:4631 , piece_id=77WNKENAMS4Q75KYK4F4VNRP4DD4MKGEJIU5K7WZ3EHVQLEW6Q5A, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.420961069 (6h21m41.010306046s) [noise=true , address=79.127.220.97:4396 , piece_id=X2CCRNYWY5ZQOAODHNJE7VLYQIHD37PEE7MQXI5WIYV36ICONILQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.638341479 (6h21m41.227686486s) [noise=true , address=79.127.201.215:4345 , piece_id=MAAFKAPO4OCQXU5WJ6DAP72KEHIM4AO5JJFJ4ZEFNWKS5M4OV43Q, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.658966771 (6h21m41.248311758s) [noise=true , address=45.91.116.10:4092 , piece_id=BUVKJC3K3RKZZTW2CQPEQM7ECQOLIBX4CU66P26JZDTOQZLYGMGA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.684178459 (6h21m41.273523446s) [noise=true , address=104.36.82.9:4039 , piece_id=PRBG5EVKAZ3LGDRX4MCI57T4RSNFVKZVUNOAIRHBM6SNS5XXUTPQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.691448508 (6h21m41.280793485s) [noise=true , address=104.36.82.22:4525 , piece_id=NKD4EXV774KNXLEHICE4MNKOQS5QKPIUTJL2PPCNYUMYPMQEF75A, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.717147267 (6h21m41.306492254s) [noise=true , address=104.36.82.59:4313 , piece_id=KDH6AVALUMDXB6USO7WG77DVOSRBTGSZA6VTJFFQJR7QFC2PE35A, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.719612885 (6h21m41.308957872s) [noise=true , address=104.36.82.45:4912 , piece_id=B6HTS6ORSGMUKZO5G2ALW4LVUSIBTCEHPX2YJTYPPCH3XROIO3HA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.733973160 (6h21m41.323318147s) [noise=true , address=104.36.82.60:4369 , piece_id=Y6SYJOB7SYBJX7STC25OJJPD4FSK5TZ5KXI7WN7QS42UNTOF5BCQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.737980718 (6h21m41.327325705s) [noise=true , address=104.36.82.5:4845 , piece_id=JKN4VFS7ET2LVGHGE4LIBPCGG33GNY4SGSOXAWB2MXFYPJVO4FCQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.745434729 (6h21m41.334779717s) [noise=true , address=104.36.82.42:4820 , piece_id=XJIHI2E35NJKDFYFTW4V5EIF3CL7GDJ5EKVCTEV2KL26X43A3CYA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.754624732 (6h21m41.34396972s) [noise=true , address=104.36.82.8:4992 , piece_id=226HGROQ3DBKTJB3U6TG2OF4JQRYAGTLMSIL2JIL4NY5PISP4RUQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.763605976 (6h21m41.352950963s) [noise=true , address=104.36.82.60:4345 , piece_id=VPYOQHGEP6E257WI7IGIWXRO5WOLCNGDJYNPFPC56WUMOYDUS6RA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.763623339 (6h21m41.352968336s) [seg_pos={0 0}, upload=6610]: Segment reached optimal threshold of 14 pieces.
Aug 7 16:01:41.763741525 (6h21m41.353086512s) [noise=true , address=104.36.82.10:4110 , piece_id=WAUEFOZXH3SULGTXTWXWLEPYHZJWJDDQRRWWXRBXVWF7WCWPYGYQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ADbnPJGRTw2NqRzhy7ptojGYoNdJgqmjuWBSjcrQK53Wm7LXi): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763803123 (6h21m41.35314811s) [noise=true , address=104.36.82.61:4387 , piece_id=MAWGNSJTYYH7EVPDIXQEG6KP3MKACSMWYVMMLYUASUDJBIX5POOA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P4wAXSyaRY9poPRK5q4td8DJM69xWioG38TTwLTHKHA7QFxLp): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763857136 (6h21m41.353202113s) [noise=true , address=45.91.116.9:4267 , piece_id=MWAYSXFIIYQGDPTLWZSAHZP3UO72EXYDSSULHBJJU54H5OWG36NA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6M5TgGqUwjXyM25uqv5hsh67mS81ByzMM4kWFUNaFvisDN9d): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763904968 (6h21m41.353249955s) [noise=true , address=104.36.82.23:4455 , piece_id=UJOJ7P3VET4XIM257B3SHAN6SJG2H64QJHV6PYU7OEB7PPBZVDAA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wsyjF5X3iPrJsezUjakUArxc8K88T36yT4rE3yu6q9BMKfycm): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763945405 (6h21m41.353290392s) [seg_pos={0 0}, upload=6610]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 16:01:41.764033824 (6h21m41.353378811s) [upload=6610]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 16:01:41.764038263 (6h21m41.35338325s) [upload=6610]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 16:01:42.043459384 (6h21m41.632804371s) [upload=6610]: Done uploading segments...
Aug 7 16:01:42.043475946 (6h21m41.632820933s) [upload=6610]: Upload finished. err: <nil>
Aug 7 16:02:11.915996790 (6h22m11.505341777s) [upload=6619]: Starting upload
Aug 7 16:02:11.916045183 (6h22m11.50539017s) [upload=6619]: Done starting upload
Aug 7 16:02:11.916152489 (6h22m11.505497476s) [upload=6619]: Uploading segments...
Aug 7 16:02:11.916287706 (6h22m11.505632693s) [upload=6619]: Got next segment. Inline: false
Aug 7 16:02:11.916319939 (6h22m11.505664926s) [upload=6619]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 16:02:11.916324708 (6h22m11.505669695s) [upload=6619]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 16:02:12.129590885 (6h22m11.718935892s) [upload=6619]: Next segment err: metaclient: Too Many Requests
Aug 7 16:02:12.129623196 (6h22m11.718968194s) [upload=6619]: Done uploading segments...
Aug 7 16:02:12.129640310 (6h22m11.718985307s) [upload=6619]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.894085235s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [............................................X.......X.......] duration:
5.824077949s
FAIL (2025-08-07 16:05:55 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
e begin upload segment.
Aug 7 16:01:39.748945470 (6h21m39.338290447s) [noise=true , address=104.36.82.42:4820 , piece_id=XJIHI2E35NJKDFYFTW4V5EIF3CL7GDJ5EKVCTEV2KL26X43A3CYA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.748969657 (6h21m39.338314644s) [noise=true , address=104.36.82.60:4345 , piece_id=VPYOQHGEP6E257WI7IGIWXRO5WOLCNGDJYNPFPC56WUMOYDUS6RA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749368329 (6h21m39.338713316s) [noise=true , address=45.91.116.9:4267 , piece_id=MWAYSXFIIYQGDPTLWZSAHZP3UO72EXYDSSULHBJJU54H5OWG36NA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749375964 (6h21m39.338720951s) [noise=true , address=104.36.82.10:4110 , piece_id=WAUEFOZXH3SULGTXTWXWLEPYHZJWJDDQRRWWXRBXVWF7WCWPYGYQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749660398 (6h21m39.339005385s) [noise=true , address=79.127.205.235:4666 , piece_id=WA6GZKNO2UUAC5VKBNOZ7UO25RQR7QDT5HVNQYZDIWJEB656WMGA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749701216 (6h21m39.339046203s) [noise=true , address=79.127.201.215:4345 , piece_id=MAAFKAPO4OCQXU5WJ6DAP72KEHIM4AO5JJFJ4ZEFNWKS5M4OV43Q, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749938690 (6h21m39.339283677s) [noise=true , address=104.36.82.59:4313 , piece_id=KDH6AVALUMDXB6USO7WG77DVOSRBTGSZA6VTJFFQJR7QFC2PE35A, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750024905 (6h21m39.339369892s) [noise=true , address=104.36.82.23:4455 , piece_id=UJOJ7P3VET4XIM257B3SHAN6SJG2H64QJHV6PYU7OEB7PPBZVDAA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750215089 (6h21m39.339560076s) [noise=true , address=104.36.82.61:4387 , piece_id=MAWGNSJTYYH7EVPDIXQEG6KP3MKACSMWYVMMLYUASUDJBIX5POOA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750282639 (6h21m39.339627626s) [noise=true , address=45.91.116.10:4092 , piece_id=BUVKJC3K3RKZZTW2CQPEQM7ECQOLIBX4CU66P26JZDTOQZLYGMGA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750481038 (6h21m39.339826025s) [noise=true , address=79.127.220.97:4396 , piece_id=X2CCRNYWY5ZQOAODHNJE7VLYQIHD37PEE7MQXI5WIYV36ICONILQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750533900 (6h21m39.339878887s) [noise=true , address=104.36.82.60:4369 , piece_id=Y6SYJOB7SYBJX7STC25OJJPD4FSK5TZ5KXI7WN7QS42UNTOF5BCQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750725967 (6h21m39.340070954s) [noise=true , address=104.36.82.9:4039 , piece_id=PRBG5EVKAZ3LGDRX4MCI57T4RSNFVKZVUNOAIRHBM6SNS5XXUTPQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750795189 (6h21m39.340140176s) [noise=true , address=104.36.82.22:4525 , piece_id=NKD4EXV774KNXLEHICE4MNKOQS5QKPIUTJL2PPCNYUMYPMQEF75A, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750954023 (6h21m39.34029901s) [noise=true , address=104.36.82.45:4912 , piece_id=B6HTS6ORSGMUKZO5G2ALW4LVUSIBTCEHPX2YJTYPPCH3XROIO3HA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.751098079 (6h21m39.340443066s) [noise=true , address=104.36.82.8:4992 , piece_id=226HGROQ3DBKTJB3U6TG2OF4JQRYAGTLMSIL2JIL4NY5PISP4RUQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.751175397 (6h21m39.340520384s) [noise=true , address=79.127.205.233:4631 , piece_id=77WNKENAMS4Q75KYK4F4VNRP4DD4MKGEJIU5K7WZ3EHVQLEW6Q5A, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.751392602 (6h21m39.340737579s) [noise=true , address=104.36.82.5:4845 , piece_id=JKN4VFS7ET2LVGHGE4LIBPCGG33GNY4SGSOXAWB2MXFYPJVO4FCQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:40.878402060 (6h21m40.467747047s) [upload=6610]: Next returned nil segment
Aug 7 16:01:40.878416708 (6h21m40.467761695s) [upload=6610]: Waiting for error group managing segments...
Aug 7 16:01:41.195597254 (6h21m40.784942241s) [noise=true , address=79.127.205.235:4666 , piece_id=WA6GZKNO2UUAC5VKBNOZ7UO25RQR7QDT5HVNQYZDIWJEB656WMGA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.261081133 (6h21m40.85042612s) [noise=true , address=79.127.205.233:4631 , piece_id=77WNKENAMS4Q75KYK4F4VNRP4DD4MKGEJIU5K7WZ3EHVQLEW6Q5A, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.420961069 (6h21m41.010306046s) [noise=true , address=79.127.220.97:4396 , piece_id=X2CCRNYWY5ZQOAODHNJE7VLYQIHD37PEE7MQXI5WIYV36ICONILQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.638341479 (6h21m41.227686486s) [noise=true , address=79.127.201.215:4345 , piece_id=MAAFKAPO4OCQXU5WJ6DAP72KEHIM4AO5JJFJ4ZEFNWKS5M4OV43Q, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.658966771 (6h21m41.248311758s) [noise=true , address=45.91.116.10:4092 , piece_id=BUVKJC3K3RKZZTW2CQPEQM7ECQOLIBX4CU66P26JZDTOQZLYGMGA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.684178459 (6h21m41.273523446s) [noise=true , address=104.36.82.9:4039 , piece_id=PRBG5EVKAZ3LGDRX4MCI57T4RSNFVKZVUNOAIRHBM6SNS5XXUTPQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.691448508 (6h21m41.280793485s) [noise=true , address=104.36.82.22:4525 , piece_id=NKD4EXV774KNXLEHICE4MNKOQS5QKPIUTJL2PPCNYUMYPMQEF75A, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.717147267 (6h21m41.306492254s) [noise=true , address=104.36.82.59:4313 , piece_id=KDH6AVALUMDXB6USO7WG77DVOSRBTGSZA6VTJFFQJR7QFC2PE35A, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.719612885 (6h21m41.308957872s) [noise=true , address=104.36.82.45:4912 , piece_id=B6HTS6ORSGMUKZO5G2ALW4LVUSIBTCEHPX2YJTYPPCH3XROIO3HA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.733973160 (6h21m41.323318147s) [noise=true , address=104.36.82.60:4369 , piece_id=Y6SYJOB7SYBJX7STC25OJJPD4FSK5TZ5KXI7WN7QS42UNTOF5BCQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.737980718 (6h21m41.327325705s) [noise=true , address=104.36.82.5:4845 , piece_id=JKN4VFS7ET2LVGHGE4LIBPCGG33GNY4SGSOXAWB2MXFYPJVO4FCQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.745434729 (6h21m41.334779717s) [noise=true , address=104.36.82.42:4820 , piece_id=XJIHI2E35NJKDFYFTW4V5EIF3CL7GDJ5EKVCTEV2KL26X43A3CYA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.754624732 (6h21m41.34396972s) [noise=true , address=104.36.82.8:4992 , piece_id=226HGROQ3DBKTJB3U6TG2OF4JQRYAGTLMSIL2JIL4NY5PISP4RUQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.763605976 (6h21m41.352950963s) [noise=true , address=104.36.82.60:4345 , piece_id=VPYOQHGEP6E257WI7IGIWXRO5WOLCNGDJYNPFPC56WUMOYDUS6RA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.763623339 (6h21m41.352968336s) [seg_pos={0 0}, upload=6610]: Segment reached optimal threshold of 14 pieces.
Aug 7 16:01:41.763741525 (6h21m41.353086512s) [noise=true , address=104.36.82.10:4110 , piece_id=WAUEFOZXH3SULGTXTWXWLEPYHZJWJDDQRRWWXRBXVWF7WCWPYGYQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ADbnPJGRTw2NqRzhy7ptojGYoNdJgqmjuWBSjcrQK53Wm7LXi): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763803123 (6h21m41.35314811s) [noise=true , address=104.36.82.61:4387 , piece_id=MAWGNSJTYYH7EVPDIXQEG6KP3MKACSMWYVMMLYUASUDJBIX5POOA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P4wAXSyaRY9poPRK5q4td8DJM69xWioG38TTwLTHKHA7QFxLp): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763857136 (6h21m41.353202113s) [noise=true , address=45.91.116.9:4267 , piece_id=MWAYSXFIIYQGDPTLWZSAHZP3UO72EXYDSSULHBJJU54H5OWG36NA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6M5TgGqUwjXyM25uqv5hsh67mS81ByzMM4kWFUNaFvisDN9d): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763904968 (6h21m41.353249955s) [noise=true , address=104.36.82.23:4455 , piece_id=UJOJ7P3VET4XIM257B3SHAN6SJG2H64QJHV6PYU7OEB7PPBZVDAA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wsyjF5X3iPrJsezUjakUArxc8K88T36yT4rE3yu6q9BMKfycm): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763945405 (6h21m41.353290392s) [seg_pos={0 0}, upload=6610]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 16:01:41.764033824 (6h21m41.353378811s) [upload=6610]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 16:01:41.764038263 (6h21m41.35338325s) [upload=6610]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 16:01:42.043459384 (6h21m41.632804371s) [upload=6610]: Done uploading segments...
Aug 7 16:01:42.043475946 (6h21m41.632820933s) [upload=6610]: Upload finished. err: <nil>
Aug 7 16:02:11.915996790 (6h22m11.505341777s) [upload=6619]: Starting upload
Aug 7 16:02:11.916045183 (6h22m11.50539017s) [upload=6619]: Done starting upload
Aug 7 16:02:11.916152489 (6h22m11.505497476s) [upload=6619]: Uploading segments...
Aug 7 16:02:11.916287706 (6h22m11.505632693s) [upload=6619]: Got next segment. Inline: false
Aug 7 16:02:11.916319939 (6h22m11.505664926s) [upload=6619]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 16:02:11.916324708 (6h22m11.505669695s) [upload=6619]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 16:02:12.129590885 (6h22m11.718935892s) [upload=6619]: Next segment err: metaclient: Too Many Requests
Aug 7 16:02:12.129623196 (6h22m11.718968194s) [upload=6619]: Done uploading segments...
Aug 7 16:02:12.129640310 (6h22m11.718985307s) [upload=6619]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.824101715s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
1.973119106s
FAIL (2025-08-07 16:05:48 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== entMakeInline Aug 7 15:57:10.271829415 (6h17m9.861174403s) [upload=6533]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 15:57:10.587098885 (6h17m10.176443872s) [upload=6533]: Done uploading segments... Aug 7 15:57:10.587115857 (6h17m10.176460835s) [upload=6533]: Upload finished. err: <nil> Aug 7 15:57:48.918901442 (6h17m48.508246429s) [upload=6545]: Starting upload Aug 7 15:57:48.918948701 (6h17m48.508293688s) [upload=6545]: Done starting upload Aug 7 15:57:48.919013486 (6h17m48.508358473s) [upload=6545]: Uploading segments... Aug 7 15:57:48.919055005 (6h17m48.508399982s) [upload=6545]: Got next segment. Inline: true Aug 7 15:57:48.919059032 (6h17m48.508404009s) [upload=6545]: Waiting for error group managing segments... Aug 7 15:57:48.919091585 (6h17m48.508436572s) [upload=6545]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 15:57:48.919095493 (6h17m48.50844048s) [upload=6545]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 15:57:48.919097998 (6h17m48.508442975s) [upload=6545]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 15:57:49.200378977 (6h17m48.789723964s) [upload=6545]: Done uploading segments... Aug 7 15:57:49.200397332 (6h17m48.789742319s) [upload=6545]: Upload finished. err: <nil> Aug 7 15:58:50.898935036 (6h18m50.488280023s) [upload=6562]: Starting upload Aug 7 15:58:50.898979992 (6h18m50.488324979s) [upload=6562]: Done starting upload Aug 7 15:58:50.899026682 (6h18m50.488371669s) [upload=6562]: Uploading segments... Aug 7 15:58:50.899065355 (6h18m50.488410343s) [upload=6562]: Got next segment. Inline: true Aug 7 15:58:50.899070335 (6h18m50.488415322s) [upload=6562]: Waiting for error group managing segments... Aug 7 15:58:50.899108268 (6h18m50.488453255s) [upload=6562]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 15:58:50.899112326 (6h18m50.488457313s) [upload=6562]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 15:58:50.899114820 (6h18m50.488459807s) [upload=6562]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 15:58:51.203289783 (6h18m50.79263478s) [upload=6562]: Done uploading segments... Aug 7 15:58:51.203312587 (6h18m50.792657624s) [upload=6562]: Upload finished. err: <nil> Aug 7 16:00:00.470970125 (6h20m0.060315112s) [upload=6580]: Starting upload Aug 7 16:00:00.471017334 (6h20m0.060362321s) [upload=6580]: Done starting upload Aug 7 16:00:00.471072199 (6h20m0.060417186s) [upload=6580]: Uploading segments... Aug 7 16:00:00.471112327 (6h20m0.060457314s) [upload=6580]: Got next segment. Inline: true Aug 7 16:00:00.471116364 (6h20m0.060461351s) [upload=6580]: Waiting for error group managing segments... Aug 7 16:00:00.471149427 (6h20m0.060494414s) [upload=6580]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:00:00.471153305 (6h20m0.060498292s) [upload=6580]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:00:00.471155819 (6h20m0.060500806s) [upload=6580]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:00:00.739624026 (6h20m0.328969023s) [upload=6580]: Done uploading segments... Aug 7 16:00:00.739645636 (6h20m0.328990623s) [upload=6580]: Upload finished. err: <nil> Aug 7 16:00:49.882359165 (6h20m49.471704152s) [upload=6598]: Starting upload Aug 7 16:00:49.882404682 (6h20m49.471749659s) [upload=6598]: Done starting upload Aug 7 16:00:49.882456812 (6h20m49.471801799s) [upload=6598]: Uploading segments... Aug 7 16:00:49.882489865 (6h20m49.471834852s) [upload=6598]: Got next segment. Inline: true Aug 7 16:00:49.882493702 (6h20m49.47183869s) [upload=6598]: Waiting for error group managing segments... Aug 7 16:00:49.882525894 (6h20m49.471870881s) [upload=6598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:00:49.882529422 (6h20m49.471874399s) [upload=6598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:00:49.882531676 (6h20m49.471876663s) [upload=6598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:00:50.213720013 (6h20m49.803065s) [upload=6598]: Done uploading segments... Aug 7 16:00:50.213739550 (6h20m49.803084537s) [upload=6598]: Upload finished. err: <nil> Aug 7 16:01:51.762778733 (6h21m51.35212372s) [upload=6617]: Starting upload Aug 7 16:01:51.762856161 (6h21m51.352201148s) [upload=6617]: Done starting upload Aug 7 16:01:51.762914883 (6h21m51.35225987s) [upload=6617]: Uploading segments... Aug 7 16:01:51.762952445 (6h21m51.352297432s) [upload=6617]: Got next segment. Inline: true Aug 7 16:01:51.762957344 (6h21m51.352302331s) [upload=6617]: Waiting for error group managing segments... Aug 7 16:01:51.762994485 (6h21m51.352339473s) [upload=6617]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:01:51.762998313 (6h21m51.3523433s) [upload=6617]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:01:51.763001278 (6h21m51.352346265s) [upload=6617]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:01:52.034377407 (6h21m51.623722394s) [upload=6617]: Done uploading segments... Aug 7 16:01:52.034395451 (6h21m51.623740439s) [upload=6617]: Upload finished. err: <nil> Aug 7 16:02:11.154406473 (6h22m10.74375146s) [upload=6618]: Starting upload Aug 7 16:02:11.154456328 (6h22m10.743801315s) [upload=6618]: Done starting upload Aug 7 16:02:11.154503859 (6h22m10.743848846s) [upload=6618]: Uploading segments... Aug 7 16:02:11.154545308 (6h22m10.743890295s) [upload=6618]: Got next segment. Inline: true Aug 7 16:02:11.154548574 (6h22m10.743893561s) [upload=6618]: Waiting for error group managing segments... Aug 7 16:02:11.154580566 (6h22m10.743925553s) [upload=6618]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:02:11.154584283 (6h22m10.74392927s) [upload=6618]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:02:11.154586828 (6h22m10.743931815s) [upload=6618]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:02:11.475329609 (6h22m11.064674596s) [upload=6618]: Done uploading segments... Aug 7 16:02:11.475484125 (6h22m11.064829112s) [upload=6618]: Upload finished. err: <nil> Aug 7 16:02:42.941029250 (6h22m42.530374238s) [upload=6628]: Starting upload Aug 7 16:02:42.941096740 (6h22m42.530441727s) [upload=6628]: Done starting upload Aug 7 16:02:42.941158728 (6h22m42.530503716s) [upload=6628]: Uploading segments... Aug 7 16:02:42.941210478 (6h22m42.530555465s) [upload=6628]: Got next segment. Inline: true Aug 7 16:02:42.941216419 (6h22m42.530561416s) [upload=6628]: Waiting for error group managing segments... Aug 7 16:02:42.941281664 (6h22m42.530626651s) [upload=6628]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:02:42.941286824 (6h22m42.530631811s) [upload=6628]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:02:42.941289880 (6h22m42.530634867s) [upload=6628]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:02:43.233814342 (6h22m42.823159339s) [upload=6628]: Done uploading segments... Aug 7 16:02:43.233838148 (6h22m42.823183145s) [upload=6628]: Upload finished. err: <nil> Aug 7 16:03:45.201690833 (6h23m44.79103582s) [upload=6651]: Starting upload Aug 7 16:03:45.201738173 (6h23m44.79108316s) [upload=6651]: Done starting upload Aug 7 16:03:45.201785534 (6h23m44.791130511s) [upload=6651]: Uploading segments... Aug 7 16:03:45.201855618 (6h23m44.791200605s) [upload=6651]: Got next segment. Inline: true Aug 7 16:03:45.201859616 (6h23m44.791204603s) [upload=6651]: Waiting for error group managing segments... Aug 7 16:03:45.201894122 (6h23m44.791239109s) [upload=6651]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:03:45.201897799 (6h23m44.791242786s) [upload=6651]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:03:45.201900023 (6h23m44.79124501s) [upload=6651]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:03:45.644503676 (6h23m45.233848664s) [upload=6651]: Done uploading segments... Aug 7 16:03:45.644521010 (6h23m45.233865987s) [upload=6651]: Upload finished. err: <nil> Aug 7 16:04:31.274936212 (6h24m30.864281199s) [upload=6663]: Starting upload Aug 7 16:04:31.274977170 (6h24m30.864322157s) [upload=6663]: Done starting upload Aug 7 16:04:31.275026655 (6h24m30.864371642s) [upload=6663]: Uploading segments... Aug 7 16:04:31.275054187 (6h24m30.864399174s) [upload=6663]: Got next segment. Inline: true Aug 7 16:04:31.275057754 (6h24m30.864402731s) [upload=6663]: Waiting for error group managing segments... Aug 7 16:04:31.275091278 (6h24m30.864436265s) [upload=6663]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:04:31.275095255 (6h24m30.864440242s) [upload=6663]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:04:31.275097651 (6h24m30.864442638s) [upload=6663]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:04:31.554258512 (6h24m31.143603499s) [upload=6663]: Done uploading segments... Aug 7 16:04:31.554277108 (6h24m31.143622095s) [upload=6663]: Upload finished. err: <nil> Aug 7 16:05:14.730035713 (6h25m14.3193807s) [upload=6674]: Starting upload Aug 7 16:05:14.730082813 (6h25m14.3194278s) [upload=6674]: Done starting upload Aug 7 16:05:14.730159079 (6h25m14.319504056s) [upload=6674]: Uploading segments... Aug 7 16:05:14.730249532 (6h25m14.319594519s) [upload=6674]: Got next segment. Inline: true Aug 7 16:05:14.730254050 (6h25m14.319599037s) [upload=6674]: Waiting for error group managing segments... Aug 7 16:05:14.730294949 (6h25m14.319639936s) [upload=6674]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:05:14.730299728 (6h25m14.319644715s) [upload=6674]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:05:14.730302243 (6h25m14.31964723s) [upload=6674]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:05:14.942773689 (6h25m14.532118666s) [upload=6674]: Done uploading segments... Aug 7 16:05:14.942790141 (6h25m14.532135118s) [upload=6674]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.97314204s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
858.055594ms
FAIL (2025-08-07 16:05:14 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== entMakeInline Aug 7 15:57:10.271829415 (6h17m9.861174403s) [upload=6533]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 15:57:10.587098885 (6h17m10.176443872s) [upload=6533]: Done uploading segments... Aug 7 15:57:10.587115857 (6h17m10.176460835s) [upload=6533]: Upload finished. err: <nil> Aug 7 15:57:48.918901442 (6h17m48.508246429s) [upload=6545]: Starting upload Aug 7 15:57:48.918948701 (6h17m48.508293688s) [upload=6545]: Done starting upload Aug 7 15:57:48.919013486 (6h17m48.508358473s) [upload=6545]: Uploading segments... Aug 7 15:57:48.919055005 (6h17m48.508399982s) [upload=6545]: Got next segment. Inline: true Aug 7 15:57:48.919059032 (6h17m48.508404009s) [upload=6545]: Waiting for error group managing segments... Aug 7 15:57:48.919091585 (6h17m48.508436572s) [upload=6545]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 15:57:48.919095493 (6h17m48.50844048s) [upload=6545]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 15:57:48.919097998 (6h17m48.508442975s) [upload=6545]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 15:57:49.200378977 (6h17m48.789723964s) [upload=6545]: Done uploading segments... Aug 7 15:57:49.200397332 (6h17m48.789742319s) [upload=6545]: Upload finished. err: <nil> Aug 7 15:58:50.898935036 (6h18m50.488280023s) [upload=6562]: Starting upload Aug 7 15:58:50.898979992 (6h18m50.488324979s) [upload=6562]: Done starting upload Aug 7 15:58:50.899026682 (6h18m50.488371669s) [upload=6562]: Uploading segments... Aug 7 15:58:50.899065355 (6h18m50.488410343s) [upload=6562]: Got next segment. Inline: true Aug 7 15:58:50.899070335 (6h18m50.488415322s) [upload=6562]: Waiting for error group managing segments... Aug 7 15:58:50.899108268 (6h18m50.488453255s) [upload=6562]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 15:58:50.899112326 (6h18m50.488457313s) [upload=6562]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 15:58:50.899114820 (6h18m50.488459807s) [upload=6562]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 15:58:51.203289783 (6h18m50.79263478s) [upload=6562]: Done uploading segments... Aug 7 15:58:51.203312587 (6h18m50.792657624s) [upload=6562]: Upload finished. err: <nil> Aug 7 16:00:00.470970125 (6h20m0.060315112s) [upload=6580]: Starting upload Aug 7 16:00:00.471017334 (6h20m0.060362321s) [upload=6580]: Done starting upload Aug 7 16:00:00.471072199 (6h20m0.060417186s) [upload=6580]: Uploading segments... Aug 7 16:00:00.471112327 (6h20m0.060457314s) [upload=6580]: Got next segment. Inline: true Aug 7 16:00:00.471116364 (6h20m0.060461351s) [upload=6580]: Waiting for error group managing segments... Aug 7 16:00:00.471149427 (6h20m0.060494414s) [upload=6580]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:00:00.471153305 (6h20m0.060498292s) [upload=6580]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:00:00.471155819 (6h20m0.060500806s) [upload=6580]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:00:00.739624026 (6h20m0.328969023s) [upload=6580]: Done uploading segments... Aug 7 16:00:00.739645636 (6h20m0.328990623s) [upload=6580]: Upload finished. err: <nil> Aug 7 16:00:49.882359165 (6h20m49.471704152s) [upload=6598]: Starting upload Aug 7 16:00:49.882404682 (6h20m49.471749659s) [upload=6598]: Done starting upload Aug 7 16:00:49.882456812 (6h20m49.471801799s) [upload=6598]: Uploading segments... Aug 7 16:00:49.882489865 (6h20m49.471834852s) [upload=6598]: Got next segment. Inline: true Aug 7 16:00:49.882493702 (6h20m49.47183869s) [upload=6598]: Waiting for error group managing segments... Aug 7 16:00:49.882525894 (6h20m49.471870881s) [upload=6598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:00:49.882529422 (6h20m49.471874399s) [upload=6598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:00:49.882531676 (6h20m49.471876663s) [upload=6598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:00:50.213720013 (6h20m49.803065s) [upload=6598]: Done uploading segments... Aug 7 16:00:50.213739550 (6h20m49.803084537s) [upload=6598]: Upload finished. err: <nil> Aug 7 16:01:51.762778733 (6h21m51.35212372s) [upload=6617]: Starting upload Aug 7 16:01:51.762856161 (6h21m51.352201148s) [upload=6617]: Done starting upload Aug 7 16:01:51.762914883 (6h21m51.35225987s) [upload=6617]: Uploading segments... Aug 7 16:01:51.762952445 (6h21m51.352297432s) [upload=6617]: Got next segment. Inline: true Aug 7 16:01:51.762957344 (6h21m51.352302331s) [upload=6617]: Waiting for error group managing segments... Aug 7 16:01:51.762994485 (6h21m51.352339473s) [upload=6617]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:01:51.762998313 (6h21m51.3523433s) [upload=6617]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:01:51.763001278 (6h21m51.352346265s) [upload=6617]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:01:52.034377407 (6h21m51.623722394s) [upload=6617]: Done uploading segments... Aug 7 16:01:52.034395451 (6h21m51.623740439s) [upload=6617]: Upload finished. err: <nil> Aug 7 16:02:11.154406473 (6h22m10.74375146s) [upload=6618]: Starting upload Aug 7 16:02:11.154456328 (6h22m10.743801315s) [upload=6618]: Done starting upload Aug 7 16:02:11.154503859 (6h22m10.743848846s) [upload=6618]: Uploading segments... Aug 7 16:02:11.154545308 (6h22m10.743890295s) [upload=6618]: Got next segment. Inline: true Aug 7 16:02:11.154548574 (6h22m10.743893561s) [upload=6618]: Waiting for error group managing segments... Aug 7 16:02:11.154580566 (6h22m10.743925553s) [upload=6618]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:02:11.154584283 (6h22m10.74392927s) [upload=6618]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:02:11.154586828 (6h22m10.743931815s) [upload=6618]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:02:11.475329609 (6h22m11.064674596s) [upload=6618]: Done uploading segments... Aug 7 16:02:11.475484125 (6h22m11.064829112s) [upload=6618]: Upload finished. err: <nil> Aug 7 16:02:42.941029250 (6h22m42.530374238s) [upload=6628]: Starting upload Aug 7 16:02:42.941096740 (6h22m42.530441727s) [upload=6628]: Done starting upload Aug 7 16:02:42.941158728 (6h22m42.530503716s) [upload=6628]: Uploading segments... Aug 7 16:02:42.941210478 (6h22m42.530555465s) [upload=6628]: Got next segment. Inline: true Aug 7 16:02:42.941216419 (6h22m42.530561416s) [upload=6628]: Waiting for error group managing segments... Aug 7 16:02:42.941281664 (6h22m42.530626651s) [upload=6628]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:02:42.941286824 (6h22m42.530631811s) [upload=6628]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:02:42.941289880 (6h22m42.530634867s) [upload=6628]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:02:43.233814342 (6h22m42.823159339s) [upload=6628]: Done uploading segments... Aug 7 16:02:43.233838148 (6h22m42.823183145s) [upload=6628]: Upload finished. err: <nil> Aug 7 16:03:45.201690833 (6h23m44.79103582s) [upload=6651]: Starting upload Aug 7 16:03:45.201738173 (6h23m44.79108316s) [upload=6651]: Done starting upload Aug 7 16:03:45.201785534 (6h23m44.791130511s) [upload=6651]: Uploading segments... Aug 7 16:03:45.201855618 (6h23m44.791200605s) [upload=6651]: Got next segment. Inline: true Aug 7 16:03:45.201859616 (6h23m44.791204603s) [upload=6651]: Waiting for error group managing segments... Aug 7 16:03:45.201894122 (6h23m44.791239109s) [upload=6651]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:03:45.201897799 (6h23m44.791242786s) [upload=6651]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:03:45.201900023 (6h23m44.79124501s) [upload=6651]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:03:45.644503676 (6h23m45.233848664s) [upload=6651]: Done uploading segments... Aug 7 16:03:45.644521010 (6h23m45.233865987s) [upload=6651]: Upload finished. err: <nil> Aug 7 16:04:31.274936212 (6h24m30.864281199s) [upload=6663]: Starting upload Aug 7 16:04:31.274977170 (6h24m30.864322157s) [upload=6663]: Done starting upload Aug 7 16:04:31.275026655 (6h24m30.864371642s) [upload=6663]: Uploading segments... Aug 7 16:04:31.275054187 (6h24m30.864399174s) [upload=6663]: Got next segment. Inline: true Aug 7 16:04:31.275057754 (6h24m30.864402731s) [upload=6663]: Waiting for error group managing segments... Aug 7 16:04:31.275091278 (6h24m30.864436265s) [upload=6663]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:04:31.275095255 (6h24m30.864440242s) [upload=6663]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:04:31.275097651 (6h24m30.864442638s) [upload=6663]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:04:31.554258512 (6h24m31.143603499s) [upload=6663]: Done uploading segments... Aug 7 16:04:31.554277108 (6h24m31.143622095s) [upload=6663]: Upload finished. err: <nil> Aug 7 16:05:14.730035713 (6h25m14.3193807s) [upload=6674]: Starting upload Aug 7 16:05:14.730082813 (6h25m14.3194278s) [upload=6674]: Done starting upload Aug 7 16:05:14.730159079 (6h25m14.319504056s) [upload=6674]: Uploading segments... Aug 7 16:05:14.730249532 (6h25m14.319594519s) [upload=6674]: Got next segment. Inline: true Aug 7 16:05:14.730254050 (6h25m14.319599037s) [upload=6674]: Waiting for error group managing segments... Aug 7 16:05:14.730294949 (6h25m14.319639936s) [upload=6674]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 16:05:14.730299728 (6h25m14.319644715s) [upload=6674]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 16:05:14.730302243 (6h25m14.31964723s) [upload=6674]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 16:05:14.942773689 (6h25m14.532118666s) [upload=6674]: Done uploading segments... Aug 7 16:05:14.942790141 (6h25m14.532135118s) [upload=6674]: Upload finished. err: metaclient: Too Many Requests
Duration: 858.078047ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [.............................................X.......X......] duration:
5.686208275s
FAIL (2025-08-07 16:05:04 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
e begin upload segment.
Aug 7 16:01:39.748945470 (6h21m39.338290447s) [noise=true , address=104.36.82.42:4820 , piece_id=XJIHI2E35NJKDFYFTW4V5EIF3CL7GDJ5EKVCTEV2KL26X43A3CYA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.748969657 (6h21m39.338314644s) [noise=true , address=104.36.82.60:4345 , piece_id=VPYOQHGEP6E257WI7IGIWXRO5WOLCNGDJYNPFPC56WUMOYDUS6RA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749368329 (6h21m39.338713316s) [noise=true , address=45.91.116.9:4267 , piece_id=MWAYSXFIIYQGDPTLWZSAHZP3UO72EXYDSSULHBJJU54H5OWG36NA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749375964 (6h21m39.338720951s) [noise=true , address=104.36.82.10:4110 , piece_id=WAUEFOZXH3SULGTXTWXWLEPYHZJWJDDQRRWWXRBXVWF7WCWPYGYQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749660398 (6h21m39.339005385s) [noise=true , address=79.127.205.235:4666 , piece_id=WA6GZKNO2UUAC5VKBNOZ7UO25RQR7QDT5HVNQYZDIWJEB656WMGA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749701216 (6h21m39.339046203s) [noise=true , address=79.127.201.215:4345 , piece_id=MAAFKAPO4OCQXU5WJ6DAP72KEHIM4AO5JJFJ4ZEFNWKS5M4OV43Q, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749938690 (6h21m39.339283677s) [noise=true , address=104.36.82.59:4313 , piece_id=KDH6AVALUMDXB6USO7WG77DVOSRBTGSZA6VTJFFQJR7QFC2PE35A, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750024905 (6h21m39.339369892s) [noise=true , address=104.36.82.23:4455 , piece_id=UJOJ7P3VET4XIM257B3SHAN6SJG2H64QJHV6PYU7OEB7PPBZVDAA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750215089 (6h21m39.339560076s) [noise=true , address=104.36.82.61:4387 , piece_id=MAWGNSJTYYH7EVPDIXQEG6KP3MKACSMWYVMMLYUASUDJBIX5POOA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750282639 (6h21m39.339627626s) [noise=true , address=45.91.116.10:4092 , piece_id=BUVKJC3K3RKZZTW2CQPEQM7ECQOLIBX4CU66P26JZDTOQZLYGMGA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750481038 (6h21m39.339826025s) [noise=true , address=79.127.220.97:4396 , piece_id=X2CCRNYWY5ZQOAODHNJE7VLYQIHD37PEE7MQXI5WIYV36ICONILQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750533900 (6h21m39.339878887s) [noise=true , address=104.36.82.60:4369 , piece_id=Y6SYJOB7SYBJX7STC25OJJPD4FSK5TZ5KXI7WN7QS42UNTOF5BCQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750725967 (6h21m39.340070954s) [noise=true , address=104.36.82.9:4039 , piece_id=PRBG5EVKAZ3LGDRX4MCI57T4RSNFVKZVUNOAIRHBM6SNS5XXUTPQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750795189 (6h21m39.340140176s) [noise=true , address=104.36.82.22:4525 , piece_id=NKD4EXV774KNXLEHICE4MNKOQS5QKPIUTJL2PPCNYUMYPMQEF75A, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750954023 (6h21m39.34029901s) [noise=true , address=104.36.82.45:4912 , piece_id=B6HTS6ORSGMUKZO5G2ALW4LVUSIBTCEHPX2YJTYPPCH3XROIO3HA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.751098079 (6h21m39.340443066s) [noise=true , address=104.36.82.8:4992 , piece_id=226HGROQ3DBKTJB3U6TG2OF4JQRYAGTLMSIL2JIL4NY5PISP4RUQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.751175397 (6h21m39.340520384s) [noise=true , address=79.127.205.233:4631 , piece_id=77WNKENAMS4Q75KYK4F4VNRP4DD4MKGEJIU5K7WZ3EHVQLEW6Q5A, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.751392602 (6h21m39.340737579s) [noise=true , address=104.36.82.5:4845 , piece_id=JKN4VFS7ET2LVGHGE4LIBPCGG33GNY4SGSOXAWB2MXFYPJVO4FCQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:40.878402060 (6h21m40.467747047s) [upload=6610]: Next returned nil segment
Aug 7 16:01:40.878416708 (6h21m40.467761695s) [upload=6610]: Waiting for error group managing segments...
Aug 7 16:01:41.195597254 (6h21m40.784942241s) [noise=true , address=79.127.205.235:4666 , piece_id=WA6GZKNO2UUAC5VKBNOZ7UO25RQR7QDT5HVNQYZDIWJEB656WMGA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.261081133 (6h21m40.85042612s) [noise=true , address=79.127.205.233:4631 , piece_id=77WNKENAMS4Q75KYK4F4VNRP4DD4MKGEJIU5K7WZ3EHVQLEW6Q5A, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.420961069 (6h21m41.010306046s) [noise=true , address=79.127.220.97:4396 , piece_id=X2CCRNYWY5ZQOAODHNJE7VLYQIHD37PEE7MQXI5WIYV36ICONILQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.638341479 (6h21m41.227686486s) [noise=true , address=79.127.201.215:4345 , piece_id=MAAFKAPO4OCQXU5WJ6DAP72KEHIM4AO5JJFJ4ZEFNWKS5M4OV43Q, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.658966771 (6h21m41.248311758s) [noise=true , address=45.91.116.10:4092 , piece_id=BUVKJC3K3RKZZTW2CQPEQM7ECQOLIBX4CU66P26JZDTOQZLYGMGA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.684178459 (6h21m41.273523446s) [noise=true , address=104.36.82.9:4039 , piece_id=PRBG5EVKAZ3LGDRX4MCI57T4RSNFVKZVUNOAIRHBM6SNS5XXUTPQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.691448508 (6h21m41.280793485s) [noise=true , address=104.36.82.22:4525 , piece_id=NKD4EXV774KNXLEHICE4MNKOQS5QKPIUTJL2PPCNYUMYPMQEF75A, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.717147267 (6h21m41.306492254s) [noise=true , address=104.36.82.59:4313 , piece_id=KDH6AVALUMDXB6USO7WG77DVOSRBTGSZA6VTJFFQJR7QFC2PE35A, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.719612885 (6h21m41.308957872s) [noise=true , address=104.36.82.45:4912 , piece_id=B6HTS6ORSGMUKZO5G2ALW4LVUSIBTCEHPX2YJTYPPCH3XROIO3HA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.733973160 (6h21m41.323318147s) [noise=true , address=104.36.82.60:4369 , piece_id=Y6SYJOB7SYBJX7STC25OJJPD4FSK5TZ5KXI7WN7QS42UNTOF5BCQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.737980718 (6h21m41.327325705s) [noise=true , address=104.36.82.5:4845 , piece_id=JKN4VFS7ET2LVGHGE4LIBPCGG33GNY4SGSOXAWB2MXFYPJVO4FCQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.745434729 (6h21m41.334779717s) [noise=true , address=104.36.82.42:4820 , piece_id=XJIHI2E35NJKDFYFTW4V5EIF3CL7GDJ5EKVCTEV2KL26X43A3CYA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.754624732 (6h21m41.34396972s) [noise=true , address=104.36.82.8:4992 , piece_id=226HGROQ3DBKTJB3U6TG2OF4JQRYAGTLMSIL2JIL4NY5PISP4RUQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.763605976 (6h21m41.352950963s) [noise=true , address=104.36.82.60:4345 , piece_id=VPYOQHGEP6E257WI7IGIWXRO5WOLCNGDJYNPFPC56WUMOYDUS6RA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.763623339 (6h21m41.352968336s) [seg_pos={0 0}, upload=6610]: Segment reached optimal threshold of 14 pieces.
Aug 7 16:01:41.763741525 (6h21m41.353086512s) [noise=true , address=104.36.82.10:4110 , piece_id=WAUEFOZXH3SULGTXTWXWLEPYHZJWJDDQRRWWXRBXVWF7WCWPYGYQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ADbnPJGRTw2NqRzhy7ptojGYoNdJgqmjuWBSjcrQK53Wm7LXi): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763803123 (6h21m41.35314811s) [noise=true , address=104.36.82.61:4387 , piece_id=MAWGNSJTYYH7EVPDIXQEG6KP3MKACSMWYVMMLYUASUDJBIX5POOA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P4wAXSyaRY9poPRK5q4td8DJM69xWioG38TTwLTHKHA7QFxLp): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763857136 (6h21m41.353202113s) [noise=true , address=45.91.116.9:4267 , piece_id=MWAYSXFIIYQGDPTLWZSAHZP3UO72EXYDSSULHBJJU54H5OWG36NA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6M5TgGqUwjXyM25uqv5hsh67mS81ByzMM4kWFUNaFvisDN9d): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763904968 (6h21m41.353249955s) [noise=true , address=104.36.82.23:4455 , piece_id=UJOJ7P3VET4XIM257B3SHAN6SJG2H64QJHV6PYU7OEB7PPBZVDAA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wsyjF5X3iPrJsezUjakUArxc8K88T36yT4rE3yu6q9BMKfycm): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763945405 (6h21m41.353290392s) [seg_pos={0 0}, upload=6610]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 16:01:41.764033824 (6h21m41.353378811s) [upload=6610]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 16:01:41.764038263 (6h21m41.35338325s) [upload=6610]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 16:01:42.043459384 (6h21m41.632804371s) [upload=6610]: Done uploading segments...
Aug 7 16:01:42.043475946 (6h21m41.632820933s) [upload=6610]: Upload finished. err: <nil>
Aug 7 16:02:11.915996790 (6h22m11.505341777s) [upload=6619]: Starting upload
Aug 7 16:02:11.916045183 (6h22m11.50539017s) [upload=6619]: Done starting upload
Aug 7 16:02:11.916152489 (6h22m11.505497476s) [upload=6619]: Uploading segments...
Aug 7 16:02:11.916287706 (6h22m11.505632693s) [upload=6619]: Got next segment. Inline: false
Aug 7 16:02:11.916319939 (6h22m11.505664926s) [upload=6619]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 16:02:11.916324708 (6h22m11.505669695s) [upload=6619]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 16:02:12.129590885 (6h22m11.718935892s) [upload=6619]: Next segment err: metaclient: Too Many Requests
Aug 7 16:02:12.129623196 (6h22m11.718968194s) [upload=6619]: Done uploading segments...
Aug 7 16:02:12.129640310 (6h22m11.718985307s) [upload=6619]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.686233042s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [..............................................X.......X.....] duration:
5.878537585s
FAIL (2025-08-07 16:04:26 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
e begin upload segment.
Aug 7 16:01:39.748945470 (6h21m39.338290447s) [noise=true , address=104.36.82.42:4820 , piece_id=XJIHI2E35NJKDFYFTW4V5EIF3CL7GDJ5EKVCTEV2KL26X43A3CYA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.748969657 (6h21m39.338314644s) [noise=true , address=104.36.82.60:4345 , piece_id=VPYOQHGEP6E257WI7IGIWXRO5WOLCNGDJYNPFPC56WUMOYDUS6RA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749368329 (6h21m39.338713316s) [noise=true , address=45.91.116.9:4267 , piece_id=MWAYSXFIIYQGDPTLWZSAHZP3UO72EXYDSSULHBJJU54H5OWG36NA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749375964 (6h21m39.338720951s) [noise=true , address=104.36.82.10:4110 , piece_id=WAUEFOZXH3SULGTXTWXWLEPYHZJWJDDQRRWWXRBXVWF7WCWPYGYQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749660398 (6h21m39.339005385s) [noise=true , address=79.127.205.235:4666 , piece_id=WA6GZKNO2UUAC5VKBNOZ7UO25RQR7QDT5HVNQYZDIWJEB656WMGA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749701216 (6h21m39.339046203s) [noise=true , address=79.127.201.215:4345 , piece_id=MAAFKAPO4OCQXU5WJ6DAP72KEHIM4AO5JJFJ4ZEFNWKS5M4OV43Q, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749938690 (6h21m39.339283677s) [noise=true , address=104.36.82.59:4313 , piece_id=KDH6AVALUMDXB6USO7WG77DVOSRBTGSZA6VTJFFQJR7QFC2PE35A, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750024905 (6h21m39.339369892s) [noise=true , address=104.36.82.23:4455 , piece_id=UJOJ7P3VET4XIM257B3SHAN6SJG2H64QJHV6PYU7OEB7PPBZVDAA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750215089 (6h21m39.339560076s) [noise=true , address=104.36.82.61:4387 , piece_id=MAWGNSJTYYH7EVPDIXQEG6KP3MKACSMWYVMMLYUASUDJBIX5POOA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750282639 (6h21m39.339627626s) [noise=true , address=45.91.116.10:4092 , piece_id=BUVKJC3K3RKZZTW2CQPEQM7ECQOLIBX4CU66P26JZDTOQZLYGMGA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750481038 (6h21m39.339826025s) [noise=true , address=79.127.220.97:4396 , piece_id=X2CCRNYWY5ZQOAODHNJE7VLYQIHD37PEE7MQXI5WIYV36ICONILQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750533900 (6h21m39.339878887s) [noise=true , address=104.36.82.60:4369 , piece_id=Y6SYJOB7SYBJX7STC25OJJPD4FSK5TZ5KXI7WN7QS42UNTOF5BCQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750725967 (6h21m39.340070954s) [noise=true , address=104.36.82.9:4039 , piece_id=PRBG5EVKAZ3LGDRX4MCI57T4RSNFVKZVUNOAIRHBM6SNS5XXUTPQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750795189 (6h21m39.340140176s) [noise=true , address=104.36.82.22:4525 , piece_id=NKD4EXV774KNXLEHICE4MNKOQS5QKPIUTJL2PPCNYUMYPMQEF75A, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750954023 (6h21m39.34029901s) [noise=true , address=104.36.82.45:4912 , piece_id=B6HTS6ORSGMUKZO5G2ALW4LVUSIBTCEHPX2YJTYPPCH3XROIO3HA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.751098079 (6h21m39.340443066s) [noise=true , address=104.36.82.8:4992 , piece_id=226HGROQ3DBKTJB3U6TG2OF4JQRYAGTLMSIL2JIL4NY5PISP4RUQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.751175397 (6h21m39.340520384s) [noise=true , address=79.127.205.233:4631 , piece_id=77WNKENAMS4Q75KYK4F4VNRP4DD4MKGEJIU5K7WZ3EHVQLEW6Q5A, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.751392602 (6h21m39.340737579s) [noise=true , address=104.36.82.5:4845 , piece_id=JKN4VFS7ET2LVGHGE4LIBPCGG33GNY4SGSOXAWB2MXFYPJVO4FCQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:40.878402060 (6h21m40.467747047s) [upload=6610]: Next returned nil segment
Aug 7 16:01:40.878416708 (6h21m40.467761695s) [upload=6610]: Waiting for error group managing segments...
Aug 7 16:01:41.195597254 (6h21m40.784942241s) [noise=true , address=79.127.205.235:4666 , piece_id=WA6GZKNO2UUAC5VKBNOZ7UO25RQR7QDT5HVNQYZDIWJEB656WMGA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.261081133 (6h21m40.85042612s) [noise=true , address=79.127.205.233:4631 , piece_id=77WNKENAMS4Q75KYK4F4VNRP4DD4MKGEJIU5K7WZ3EHVQLEW6Q5A, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.420961069 (6h21m41.010306046s) [noise=true , address=79.127.220.97:4396 , piece_id=X2CCRNYWY5ZQOAODHNJE7VLYQIHD37PEE7MQXI5WIYV36ICONILQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.638341479 (6h21m41.227686486s) [noise=true , address=79.127.201.215:4345 , piece_id=MAAFKAPO4OCQXU5WJ6DAP72KEHIM4AO5JJFJ4ZEFNWKS5M4OV43Q, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.658966771 (6h21m41.248311758s) [noise=true , address=45.91.116.10:4092 , piece_id=BUVKJC3K3RKZZTW2CQPEQM7ECQOLIBX4CU66P26JZDTOQZLYGMGA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.684178459 (6h21m41.273523446s) [noise=true , address=104.36.82.9:4039 , piece_id=PRBG5EVKAZ3LGDRX4MCI57T4RSNFVKZVUNOAIRHBM6SNS5XXUTPQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.691448508 (6h21m41.280793485s) [noise=true , address=104.36.82.22:4525 , piece_id=NKD4EXV774KNXLEHICE4MNKOQS5QKPIUTJL2PPCNYUMYPMQEF75A, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.717147267 (6h21m41.306492254s) [noise=true , address=104.36.82.59:4313 , piece_id=KDH6AVALUMDXB6USO7WG77DVOSRBTGSZA6VTJFFQJR7QFC2PE35A, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.719612885 (6h21m41.308957872s) [noise=true , address=104.36.82.45:4912 , piece_id=B6HTS6ORSGMUKZO5G2ALW4LVUSIBTCEHPX2YJTYPPCH3XROIO3HA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.733973160 (6h21m41.323318147s) [noise=true , address=104.36.82.60:4369 , piece_id=Y6SYJOB7SYBJX7STC25OJJPD4FSK5TZ5KXI7WN7QS42UNTOF5BCQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.737980718 (6h21m41.327325705s) [noise=true , address=104.36.82.5:4845 , piece_id=JKN4VFS7ET2LVGHGE4LIBPCGG33GNY4SGSOXAWB2MXFYPJVO4FCQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.745434729 (6h21m41.334779717s) [noise=true , address=104.36.82.42:4820 , piece_id=XJIHI2E35NJKDFYFTW4V5EIF3CL7GDJ5EKVCTEV2KL26X43A3CYA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.754624732 (6h21m41.34396972s) [noise=true , address=104.36.82.8:4992 , piece_id=226HGROQ3DBKTJB3U6TG2OF4JQRYAGTLMSIL2JIL4NY5PISP4RUQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.763605976 (6h21m41.352950963s) [noise=true , address=104.36.82.60:4345 , piece_id=VPYOQHGEP6E257WI7IGIWXRO5WOLCNGDJYNPFPC56WUMOYDUS6RA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.763623339 (6h21m41.352968336s) [seg_pos={0 0}, upload=6610]: Segment reached optimal threshold of 14 pieces.
Aug 7 16:01:41.763741525 (6h21m41.353086512s) [noise=true , address=104.36.82.10:4110 , piece_id=WAUEFOZXH3SULGTXTWXWLEPYHZJWJDDQRRWWXRBXVWF7WCWPYGYQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ADbnPJGRTw2NqRzhy7ptojGYoNdJgqmjuWBSjcrQK53Wm7LXi): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763803123 (6h21m41.35314811s) [noise=true , address=104.36.82.61:4387 , piece_id=MAWGNSJTYYH7EVPDIXQEG6KP3MKACSMWYVMMLYUASUDJBIX5POOA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P4wAXSyaRY9poPRK5q4td8DJM69xWioG38TTwLTHKHA7QFxLp): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763857136 (6h21m41.353202113s) [noise=true , address=45.91.116.9:4267 , piece_id=MWAYSXFIIYQGDPTLWZSAHZP3UO72EXYDSSULHBJJU54H5OWG36NA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6M5TgGqUwjXyM25uqv5hsh67mS81ByzMM4kWFUNaFvisDN9d): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763904968 (6h21m41.353249955s) [noise=true , address=104.36.82.23:4455 , piece_id=UJOJ7P3VET4XIM257B3SHAN6SJG2H64QJHV6PYU7OEB7PPBZVDAA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wsyjF5X3iPrJsezUjakUArxc8K88T36yT4rE3yu6q9BMKfycm): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763945405 (6h21m41.353290392s) [seg_pos={0 0}, upload=6610]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 16:01:41.764033824 (6h21m41.353378811s) [upload=6610]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 16:01:41.764038263 (6h21m41.35338325s) [upload=6610]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 16:01:42.043459384 (6h21m41.632804371s) [upload=6610]: Done uploading segments...
Aug 7 16:01:42.043475946 (6h21m41.632820933s) [upload=6610]: Upload finished. err: <nil>
Aug 7 16:02:11.915996790 (6h22m11.505341777s) [upload=6619]: Starting upload
Aug 7 16:02:11.916045183 (6h22m11.50539017s) [upload=6619]: Done starting upload
Aug 7 16:02:11.916152489 (6h22m11.505497476s) [upload=6619]: Uploading segments...
Aug 7 16:02:11.916287706 (6h22m11.505632693s) [upload=6619]: Got next segment. Inline: false
Aug 7 16:02:11.916319939 (6h22m11.505664926s) [upload=6619]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 16:02:11.916324708 (6h22m11.505669695s) [upload=6619]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 16:02:12.129590885 (6h22m11.718935892s) [upload=6619]: Next segment err: metaclient: Too Many Requests
Aug 7 16:02:12.129623196 (6h22m11.718968194s) [upload=6619]: Done uploading segments...
Aug 7 16:02:12.129640310 (6h22m11.718985307s) [upload=6619]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.878562453s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [...............................................X.......X....] duration:
6.114042685s
FAIL (2025-08-07 16:03:37 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
e begin upload segment.
Aug 7 16:01:39.748945470 (6h21m39.338290447s) [noise=true , address=104.36.82.42:4820 , piece_id=XJIHI2E35NJKDFYFTW4V5EIF3CL7GDJ5EKVCTEV2KL26X43A3CYA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.748969657 (6h21m39.338314644s) [noise=true , address=104.36.82.60:4345 , piece_id=VPYOQHGEP6E257WI7IGIWXRO5WOLCNGDJYNPFPC56WUMOYDUS6RA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749368329 (6h21m39.338713316s) [noise=true , address=45.91.116.9:4267 , piece_id=MWAYSXFIIYQGDPTLWZSAHZP3UO72EXYDSSULHBJJU54H5OWG36NA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749375964 (6h21m39.338720951s) [noise=true , address=104.36.82.10:4110 , piece_id=WAUEFOZXH3SULGTXTWXWLEPYHZJWJDDQRRWWXRBXVWF7WCWPYGYQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749660398 (6h21m39.339005385s) [noise=true , address=79.127.205.235:4666 , piece_id=WA6GZKNO2UUAC5VKBNOZ7UO25RQR7QDT5HVNQYZDIWJEB656WMGA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749701216 (6h21m39.339046203s) [noise=true , address=79.127.201.215:4345 , piece_id=MAAFKAPO4OCQXU5WJ6DAP72KEHIM4AO5JJFJ4ZEFNWKS5M4OV43Q, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749938690 (6h21m39.339283677s) [noise=true , address=104.36.82.59:4313 , piece_id=KDH6AVALUMDXB6USO7WG77DVOSRBTGSZA6VTJFFQJR7QFC2PE35A, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750024905 (6h21m39.339369892s) [noise=true , address=104.36.82.23:4455 , piece_id=UJOJ7P3VET4XIM257B3SHAN6SJG2H64QJHV6PYU7OEB7PPBZVDAA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750215089 (6h21m39.339560076s) [noise=true , address=104.36.82.61:4387 , piece_id=MAWGNSJTYYH7EVPDIXQEG6KP3MKACSMWYVMMLYUASUDJBIX5POOA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750282639 (6h21m39.339627626s) [noise=true , address=45.91.116.10:4092 , piece_id=BUVKJC3K3RKZZTW2CQPEQM7ECQOLIBX4CU66P26JZDTOQZLYGMGA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750481038 (6h21m39.339826025s) [noise=true , address=79.127.220.97:4396 , piece_id=X2CCRNYWY5ZQOAODHNJE7VLYQIHD37PEE7MQXI5WIYV36ICONILQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750533900 (6h21m39.339878887s) [noise=true , address=104.36.82.60:4369 , piece_id=Y6SYJOB7SYBJX7STC25OJJPD4FSK5TZ5KXI7WN7QS42UNTOF5BCQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750725967 (6h21m39.340070954s) [noise=true , address=104.36.82.9:4039 , piece_id=PRBG5EVKAZ3LGDRX4MCI57T4RSNFVKZVUNOAIRHBM6SNS5XXUTPQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750795189 (6h21m39.340140176s) [noise=true , address=104.36.82.22:4525 , piece_id=NKD4EXV774KNXLEHICE4MNKOQS5QKPIUTJL2PPCNYUMYPMQEF75A, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750954023 (6h21m39.34029901s) [noise=true , address=104.36.82.45:4912 , piece_id=B6HTS6ORSGMUKZO5G2ALW4LVUSIBTCEHPX2YJTYPPCH3XROIO3HA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.751098079 (6h21m39.340443066s) [noise=true , address=104.36.82.8:4992 , piece_id=226HGROQ3DBKTJB3U6TG2OF4JQRYAGTLMSIL2JIL4NY5PISP4RUQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.751175397 (6h21m39.340520384s) [noise=true , address=79.127.205.233:4631 , piece_id=77WNKENAMS4Q75KYK4F4VNRP4DD4MKGEJIU5K7WZ3EHVQLEW6Q5A, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.751392602 (6h21m39.340737579s) [noise=true , address=104.36.82.5:4845 , piece_id=JKN4VFS7ET2LVGHGE4LIBPCGG33GNY4SGSOXAWB2MXFYPJVO4FCQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:40.878402060 (6h21m40.467747047s) [upload=6610]: Next returned nil segment
Aug 7 16:01:40.878416708 (6h21m40.467761695s) [upload=6610]: Waiting for error group managing segments...
Aug 7 16:01:41.195597254 (6h21m40.784942241s) [noise=true , address=79.127.205.235:4666 , piece_id=WA6GZKNO2UUAC5VKBNOZ7UO25RQR7QDT5HVNQYZDIWJEB656WMGA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.261081133 (6h21m40.85042612s) [noise=true , address=79.127.205.233:4631 , piece_id=77WNKENAMS4Q75KYK4F4VNRP4DD4MKGEJIU5K7WZ3EHVQLEW6Q5A, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.420961069 (6h21m41.010306046s) [noise=true , address=79.127.220.97:4396 , piece_id=X2CCRNYWY5ZQOAODHNJE7VLYQIHD37PEE7MQXI5WIYV36ICONILQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.638341479 (6h21m41.227686486s) [noise=true , address=79.127.201.215:4345 , piece_id=MAAFKAPO4OCQXU5WJ6DAP72KEHIM4AO5JJFJ4ZEFNWKS5M4OV43Q, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.658966771 (6h21m41.248311758s) [noise=true , address=45.91.116.10:4092 , piece_id=BUVKJC3K3RKZZTW2CQPEQM7ECQOLIBX4CU66P26JZDTOQZLYGMGA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.684178459 (6h21m41.273523446s) [noise=true , address=104.36.82.9:4039 , piece_id=PRBG5EVKAZ3LGDRX4MCI57T4RSNFVKZVUNOAIRHBM6SNS5XXUTPQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.691448508 (6h21m41.280793485s) [noise=true , address=104.36.82.22:4525 , piece_id=NKD4EXV774KNXLEHICE4MNKOQS5QKPIUTJL2PPCNYUMYPMQEF75A, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.717147267 (6h21m41.306492254s) [noise=true , address=104.36.82.59:4313 , piece_id=KDH6AVALUMDXB6USO7WG77DVOSRBTGSZA6VTJFFQJR7QFC2PE35A, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.719612885 (6h21m41.308957872s) [noise=true , address=104.36.82.45:4912 , piece_id=B6HTS6ORSGMUKZO5G2ALW4LVUSIBTCEHPX2YJTYPPCH3XROIO3HA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.733973160 (6h21m41.323318147s) [noise=true , address=104.36.82.60:4369 , piece_id=Y6SYJOB7SYBJX7STC25OJJPD4FSK5TZ5KXI7WN7QS42UNTOF5BCQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.737980718 (6h21m41.327325705s) [noise=true , address=104.36.82.5:4845 , piece_id=JKN4VFS7ET2LVGHGE4LIBPCGG33GNY4SGSOXAWB2MXFYPJVO4FCQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.745434729 (6h21m41.334779717s) [noise=true , address=104.36.82.42:4820 , piece_id=XJIHI2E35NJKDFYFTW4V5EIF3CL7GDJ5EKVCTEV2KL26X43A3CYA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.754624732 (6h21m41.34396972s) [noise=true , address=104.36.82.8:4992 , piece_id=226HGROQ3DBKTJB3U6TG2OF4JQRYAGTLMSIL2JIL4NY5PISP4RUQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.763605976 (6h21m41.352950963s) [noise=true , address=104.36.82.60:4345 , piece_id=VPYOQHGEP6E257WI7IGIWXRO5WOLCNGDJYNPFPC56WUMOYDUS6RA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.763623339 (6h21m41.352968336s) [seg_pos={0 0}, upload=6610]: Segment reached optimal threshold of 14 pieces.
Aug 7 16:01:41.763741525 (6h21m41.353086512s) [noise=true , address=104.36.82.10:4110 , piece_id=WAUEFOZXH3SULGTXTWXWLEPYHZJWJDDQRRWWXRBXVWF7WCWPYGYQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ADbnPJGRTw2NqRzhy7ptojGYoNdJgqmjuWBSjcrQK53Wm7LXi): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763803123 (6h21m41.35314811s) [noise=true , address=104.36.82.61:4387 , piece_id=MAWGNSJTYYH7EVPDIXQEG6KP3MKACSMWYVMMLYUASUDJBIX5POOA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P4wAXSyaRY9poPRK5q4td8DJM69xWioG38TTwLTHKHA7QFxLp): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763857136 (6h21m41.353202113s) [noise=true , address=45.91.116.9:4267 , piece_id=MWAYSXFIIYQGDPTLWZSAHZP3UO72EXYDSSULHBJJU54H5OWG36NA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6M5TgGqUwjXyM25uqv5hsh67mS81ByzMM4kWFUNaFvisDN9d): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763904968 (6h21m41.353249955s) [noise=true , address=104.36.82.23:4455 , piece_id=UJOJ7P3VET4XIM257B3SHAN6SJG2H64QJHV6PYU7OEB7PPBZVDAA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wsyjF5X3iPrJsezUjakUArxc8K88T36yT4rE3yu6q9BMKfycm): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763945405 (6h21m41.353290392s) [seg_pos={0 0}, upload=6610]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 16:01:41.764033824 (6h21m41.353378811s) [upload=6610]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 16:01:41.764038263 (6h21m41.35338325s) [upload=6610]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 16:01:42.043459384 (6h21m41.632804371s) [upload=6610]: Done uploading segments...
Aug 7 16:01:42.043475946 (6h21m41.632820933s) [upload=6610]: Upload finished. err: <nil>
Aug 7 16:02:11.915996790 (6h22m11.505341777s) [upload=6619]: Starting upload
Aug 7 16:02:11.916045183 (6h22m11.50539017s) [upload=6619]: Done starting upload
Aug 7 16:02:11.916152489 (6h22m11.505497476s) [upload=6619]: Uploading segments...
Aug 7 16:02:11.916287706 (6h22m11.505632693s) [upload=6619]: Got next segment. Inline: false
Aug 7 16:02:11.916319939 (6h22m11.505664926s) [upload=6619]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 16:02:11.916324708 (6h22m11.505669695s) [upload=6619]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 16:02:12.129590885 (6h22m11.718935892s) [upload=6619]: Next segment err: metaclient: Too Many Requests
Aug 7 16:02:12.129623196 (6h22m11.718968194s) [upload=6619]: Done uploading segments...
Aug 7 16:02:12.129640310 (6h22m11.718985307s) [upload=6619]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.114076078s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [................................................X.......X...] duration:
5.885585429s
FAIL (2025-08-07 16:03:19 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
e begin upload segment.
Aug 7 16:01:39.748945470 (6h21m39.338290447s) [noise=true , address=104.36.82.42:4820 , piece_id=XJIHI2E35NJKDFYFTW4V5EIF3CL7GDJ5EKVCTEV2KL26X43A3CYA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.748969657 (6h21m39.338314644s) [noise=true , address=104.36.82.60:4345 , piece_id=VPYOQHGEP6E257WI7IGIWXRO5WOLCNGDJYNPFPC56WUMOYDUS6RA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749368329 (6h21m39.338713316s) [noise=true , address=45.91.116.9:4267 , piece_id=MWAYSXFIIYQGDPTLWZSAHZP3UO72EXYDSSULHBJJU54H5OWG36NA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749375964 (6h21m39.338720951s) [noise=true , address=104.36.82.10:4110 , piece_id=WAUEFOZXH3SULGTXTWXWLEPYHZJWJDDQRRWWXRBXVWF7WCWPYGYQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749660398 (6h21m39.339005385s) [noise=true , address=79.127.205.235:4666 , piece_id=WA6GZKNO2UUAC5VKBNOZ7UO25RQR7QDT5HVNQYZDIWJEB656WMGA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749701216 (6h21m39.339046203s) [noise=true , address=79.127.201.215:4345 , piece_id=MAAFKAPO4OCQXU5WJ6DAP72KEHIM4AO5JJFJ4ZEFNWKS5M4OV43Q, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749938690 (6h21m39.339283677s) [noise=true , address=104.36.82.59:4313 , piece_id=KDH6AVALUMDXB6USO7WG77DVOSRBTGSZA6VTJFFQJR7QFC2PE35A, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750024905 (6h21m39.339369892s) [noise=true , address=104.36.82.23:4455 , piece_id=UJOJ7P3VET4XIM257B3SHAN6SJG2H64QJHV6PYU7OEB7PPBZVDAA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750215089 (6h21m39.339560076s) [noise=true , address=104.36.82.61:4387 , piece_id=MAWGNSJTYYH7EVPDIXQEG6KP3MKACSMWYVMMLYUASUDJBIX5POOA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750282639 (6h21m39.339627626s) [noise=true , address=45.91.116.10:4092 , piece_id=BUVKJC3K3RKZZTW2CQPEQM7ECQOLIBX4CU66P26JZDTOQZLYGMGA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750481038 (6h21m39.339826025s) [noise=true , address=79.127.220.97:4396 , piece_id=X2CCRNYWY5ZQOAODHNJE7VLYQIHD37PEE7MQXI5WIYV36ICONILQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750533900 (6h21m39.339878887s) [noise=true , address=104.36.82.60:4369 , piece_id=Y6SYJOB7SYBJX7STC25OJJPD4FSK5TZ5KXI7WN7QS42UNTOF5BCQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750725967 (6h21m39.340070954s) [noise=true , address=104.36.82.9:4039 , piece_id=PRBG5EVKAZ3LGDRX4MCI57T4RSNFVKZVUNOAIRHBM6SNS5XXUTPQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750795189 (6h21m39.340140176s) [noise=true , address=104.36.82.22:4525 , piece_id=NKD4EXV774KNXLEHICE4MNKOQS5QKPIUTJL2PPCNYUMYPMQEF75A, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750954023 (6h21m39.34029901s) [noise=true , address=104.36.82.45:4912 , piece_id=B6HTS6ORSGMUKZO5G2ALW4LVUSIBTCEHPX2YJTYPPCH3XROIO3HA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.751098079 (6h21m39.340443066s) [noise=true , address=104.36.82.8:4992 , piece_id=226HGROQ3DBKTJB3U6TG2OF4JQRYAGTLMSIL2JIL4NY5PISP4RUQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.751175397 (6h21m39.340520384s) [noise=true , address=79.127.205.233:4631 , piece_id=77WNKENAMS4Q75KYK4F4VNRP4DD4MKGEJIU5K7WZ3EHVQLEW6Q5A, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.751392602 (6h21m39.340737579s) [noise=true , address=104.36.82.5:4845 , piece_id=JKN4VFS7ET2LVGHGE4LIBPCGG33GNY4SGSOXAWB2MXFYPJVO4FCQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:40.878402060 (6h21m40.467747047s) [upload=6610]: Next returned nil segment
Aug 7 16:01:40.878416708 (6h21m40.467761695s) [upload=6610]: Waiting for error group managing segments...
Aug 7 16:01:41.195597254 (6h21m40.784942241s) [noise=true , address=79.127.205.235:4666 , piece_id=WA6GZKNO2UUAC5VKBNOZ7UO25RQR7QDT5HVNQYZDIWJEB656WMGA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.261081133 (6h21m40.85042612s) [noise=true , address=79.127.205.233:4631 , piece_id=77WNKENAMS4Q75KYK4F4VNRP4DD4MKGEJIU5K7WZ3EHVQLEW6Q5A, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.420961069 (6h21m41.010306046s) [noise=true , address=79.127.220.97:4396 , piece_id=X2CCRNYWY5ZQOAODHNJE7VLYQIHD37PEE7MQXI5WIYV36ICONILQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.638341479 (6h21m41.227686486s) [noise=true , address=79.127.201.215:4345 , piece_id=MAAFKAPO4OCQXU5WJ6DAP72KEHIM4AO5JJFJ4ZEFNWKS5M4OV43Q, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.658966771 (6h21m41.248311758s) [noise=true , address=45.91.116.10:4092 , piece_id=BUVKJC3K3RKZZTW2CQPEQM7ECQOLIBX4CU66P26JZDTOQZLYGMGA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.684178459 (6h21m41.273523446s) [noise=true , address=104.36.82.9:4039 , piece_id=PRBG5EVKAZ3LGDRX4MCI57T4RSNFVKZVUNOAIRHBM6SNS5XXUTPQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.691448508 (6h21m41.280793485s) [noise=true , address=104.36.82.22:4525 , piece_id=NKD4EXV774KNXLEHICE4MNKOQS5QKPIUTJL2PPCNYUMYPMQEF75A, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.717147267 (6h21m41.306492254s) [noise=true , address=104.36.82.59:4313 , piece_id=KDH6AVALUMDXB6USO7WG77DVOSRBTGSZA6VTJFFQJR7QFC2PE35A, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.719612885 (6h21m41.308957872s) [noise=true , address=104.36.82.45:4912 , piece_id=B6HTS6ORSGMUKZO5G2ALW4LVUSIBTCEHPX2YJTYPPCH3XROIO3HA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.733973160 (6h21m41.323318147s) [noise=true , address=104.36.82.60:4369 , piece_id=Y6SYJOB7SYBJX7STC25OJJPD4FSK5TZ5KXI7WN7QS42UNTOF5BCQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.737980718 (6h21m41.327325705s) [noise=true , address=104.36.82.5:4845 , piece_id=JKN4VFS7ET2LVGHGE4LIBPCGG33GNY4SGSOXAWB2MXFYPJVO4FCQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.745434729 (6h21m41.334779717s) [noise=true , address=104.36.82.42:4820 , piece_id=XJIHI2E35NJKDFYFTW4V5EIF3CL7GDJ5EKVCTEV2KL26X43A3CYA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.754624732 (6h21m41.34396972s) [noise=true , address=104.36.82.8:4992 , piece_id=226HGROQ3DBKTJB3U6TG2OF4JQRYAGTLMSIL2JIL4NY5PISP4RUQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.763605976 (6h21m41.352950963s) [noise=true , address=104.36.82.60:4345 , piece_id=VPYOQHGEP6E257WI7IGIWXRO5WOLCNGDJYNPFPC56WUMOYDUS6RA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.763623339 (6h21m41.352968336s) [seg_pos={0 0}, upload=6610]: Segment reached optimal threshold of 14 pieces.
Aug 7 16:01:41.763741525 (6h21m41.353086512s) [noise=true , address=104.36.82.10:4110 , piece_id=WAUEFOZXH3SULGTXTWXWLEPYHZJWJDDQRRWWXRBXVWF7WCWPYGYQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ADbnPJGRTw2NqRzhy7ptojGYoNdJgqmjuWBSjcrQK53Wm7LXi): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763803123 (6h21m41.35314811s) [noise=true , address=104.36.82.61:4387 , piece_id=MAWGNSJTYYH7EVPDIXQEG6KP3MKACSMWYVMMLYUASUDJBIX5POOA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P4wAXSyaRY9poPRK5q4td8DJM69xWioG38TTwLTHKHA7QFxLp): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763857136 (6h21m41.353202113s) [noise=true , address=45.91.116.9:4267 , piece_id=MWAYSXFIIYQGDPTLWZSAHZP3UO72EXYDSSULHBJJU54H5OWG36NA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6M5TgGqUwjXyM25uqv5hsh67mS81ByzMM4kWFUNaFvisDN9d): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763904968 (6h21m41.353249955s) [noise=true , address=104.36.82.23:4455 , piece_id=UJOJ7P3VET4XIM257B3SHAN6SJG2H64QJHV6PYU7OEB7PPBZVDAA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wsyjF5X3iPrJsezUjakUArxc8K88T36yT4rE3yu6q9BMKfycm): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763945405 (6h21m41.353290392s) [seg_pos={0 0}, upload=6610]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 16:01:41.764033824 (6h21m41.353378811s) [upload=6610]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 16:01:41.764038263 (6h21m41.35338325s) [upload=6610]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 16:01:42.043459384 (6h21m41.632804371s) [upload=6610]: Done uploading segments...
Aug 7 16:01:42.043475946 (6h21m41.632820933s) [upload=6610]: Upload finished. err: <nil>
Aug 7 16:02:11.915996790 (6h22m11.505341777s) [upload=6619]: Starting upload
Aug 7 16:02:11.916045183 (6h22m11.50539017s) [upload=6619]: Done starting upload
Aug 7 16:02:11.916152489 (6h22m11.505497476s) [upload=6619]: Uploading segments...
Aug 7 16:02:11.916287706 (6h22m11.505632693s) [upload=6619]: Got next segment. Inline: false
Aug 7 16:02:11.916319939 (6h22m11.505664926s) [upload=6619]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 16:02:11.916324708 (6h22m11.505669695s) [upload=6619]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 16:02:12.129590885 (6h22m11.718935892s) [upload=6619]: Next segment err: metaclient: Too Many Requests
Aug 7 16:02:12.129623196 (6h22m11.718968194s) [upload=6619]: Done uploading segments...
Aug 7 16:02:12.129640310 (6h22m11.718985307s) [upload=6619]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.88560149s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [.................................................X.......X..] duration:
5.679903373s
FAIL (2025-08-07 16:02:59 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
e begin upload segment.
Aug 7 16:01:39.748945470 (6h21m39.338290447s) [noise=true , address=104.36.82.42:4820 , piece_id=XJIHI2E35NJKDFYFTW4V5EIF3CL7GDJ5EKVCTEV2KL26X43A3CYA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.748969657 (6h21m39.338314644s) [noise=true , address=104.36.82.60:4345 , piece_id=VPYOQHGEP6E257WI7IGIWXRO5WOLCNGDJYNPFPC56WUMOYDUS6RA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749368329 (6h21m39.338713316s) [noise=true , address=45.91.116.9:4267 , piece_id=MWAYSXFIIYQGDPTLWZSAHZP3UO72EXYDSSULHBJJU54H5OWG36NA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749375964 (6h21m39.338720951s) [noise=true , address=104.36.82.10:4110 , piece_id=WAUEFOZXH3SULGTXTWXWLEPYHZJWJDDQRRWWXRBXVWF7WCWPYGYQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749660398 (6h21m39.339005385s) [noise=true , address=79.127.205.235:4666 , piece_id=WA6GZKNO2UUAC5VKBNOZ7UO25RQR7QDT5HVNQYZDIWJEB656WMGA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749701216 (6h21m39.339046203s) [noise=true , address=79.127.201.215:4345 , piece_id=MAAFKAPO4OCQXU5WJ6DAP72KEHIM4AO5JJFJ4ZEFNWKS5M4OV43Q, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749938690 (6h21m39.339283677s) [noise=true , address=104.36.82.59:4313 , piece_id=KDH6AVALUMDXB6USO7WG77DVOSRBTGSZA6VTJFFQJR7QFC2PE35A, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750024905 (6h21m39.339369892s) [noise=true , address=104.36.82.23:4455 , piece_id=UJOJ7P3VET4XIM257B3SHAN6SJG2H64QJHV6PYU7OEB7PPBZVDAA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750215089 (6h21m39.339560076s) [noise=true , address=104.36.82.61:4387 , piece_id=MAWGNSJTYYH7EVPDIXQEG6KP3MKACSMWYVMMLYUASUDJBIX5POOA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750282639 (6h21m39.339627626s) [noise=true , address=45.91.116.10:4092 , piece_id=BUVKJC3K3RKZZTW2CQPEQM7ECQOLIBX4CU66P26JZDTOQZLYGMGA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750481038 (6h21m39.339826025s) [noise=true , address=79.127.220.97:4396 , piece_id=X2CCRNYWY5ZQOAODHNJE7VLYQIHD37PEE7MQXI5WIYV36ICONILQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750533900 (6h21m39.339878887s) [noise=true , address=104.36.82.60:4369 , piece_id=Y6SYJOB7SYBJX7STC25OJJPD4FSK5TZ5KXI7WN7QS42UNTOF5BCQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750725967 (6h21m39.340070954s) [noise=true , address=104.36.82.9:4039 , piece_id=PRBG5EVKAZ3LGDRX4MCI57T4RSNFVKZVUNOAIRHBM6SNS5XXUTPQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750795189 (6h21m39.340140176s) [noise=true , address=104.36.82.22:4525 , piece_id=NKD4EXV774KNXLEHICE4MNKOQS5QKPIUTJL2PPCNYUMYPMQEF75A, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750954023 (6h21m39.34029901s) [noise=true , address=104.36.82.45:4912 , piece_id=B6HTS6ORSGMUKZO5G2ALW4LVUSIBTCEHPX2YJTYPPCH3XROIO3HA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.751098079 (6h21m39.340443066s) [noise=true , address=104.36.82.8:4992 , piece_id=226HGROQ3DBKTJB3U6TG2OF4JQRYAGTLMSIL2JIL4NY5PISP4RUQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.751175397 (6h21m39.340520384s) [noise=true , address=79.127.205.233:4631 , piece_id=77WNKENAMS4Q75KYK4F4VNRP4DD4MKGEJIU5K7WZ3EHVQLEW6Q5A, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.751392602 (6h21m39.340737579s) [noise=true , address=104.36.82.5:4845 , piece_id=JKN4VFS7ET2LVGHGE4LIBPCGG33GNY4SGSOXAWB2MXFYPJVO4FCQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:40.878402060 (6h21m40.467747047s) [upload=6610]: Next returned nil segment
Aug 7 16:01:40.878416708 (6h21m40.467761695s) [upload=6610]: Waiting for error group managing segments...
Aug 7 16:01:41.195597254 (6h21m40.784942241s) [noise=true , address=79.127.205.235:4666 , piece_id=WA6GZKNO2UUAC5VKBNOZ7UO25RQR7QDT5HVNQYZDIWJEB656WMGA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.261081133 (6h21m40.85042612s) [noise=true , address=79.127.205.233:4631 , piece_id=77WNKENAMS4Q75KYK4F4VNRP4DD4MKGEJIU5K7WZ3EHVQLEW6Q5A, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.420961069 (6h21m41.010306046s) [noise=true , address=79.127.220.97:4396 , piece_id=X2CCRNYWY5ZQOAODHNJE7VLYQIHD37PEE7MQXI5WIYV36ICONILQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.638341479 (6h21m41.227686486s) [noise=true , address=79.127.201.215:4345 , piece_id=MAAFKAPO4OCQXU5WJ6DAP72KEHIM4AO5JJFJ4ZEFNWKS5M4OV43Q, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.658966771 (6h21m41.248311758s) [noise=true , address=45.91.116.10:4092 , piece_id=BUVKJC3K3RKZZTW2CQPEQM7ECQOLIBX4CU66P26JZDTOQZLYGMGA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.684178459 (6h21m41.273523446s) [noise=true , address=104.36.82.9:4039 , piece_id=PRBG5EVKAZ3LGDRX4MCI57T4RSNFVKZVUNOAIRHBM6SNS5XXUTPQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.691448508 (6h21m41.280793485s) [noise=true , address=104.36.82.22:4525 , piece_id=NKD4EXV774KNXLEHICE4MNKOQS5QKPIUTJL2PPCNYUMYPMQEF75A, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.717147267 (6h21m41.306492254s) [noise=true , address=104.36.82.59:4313 , piece_id=KDH6AVALUMDXB6USO7WG77DVOSRBTGSZA6VTJFFQJR7QFC2PE35A, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.719612885 (6h21m41.308957872s) [noise=true , address=104.36.82.45:4912 , piece_id=B6HTS6ORSGMUKZO5G2ALW4LVUSIBTCEHPX2YJTYPPCH3XROIO3HA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.733973160 (6h21m41.323318147s) [noise=true , address=104.36.82.60:4369 , piece_id=Y6SYJOB7SYBJX7STC25OJJPD4FSK5TZ5KXI7WN7QS42UNTOF5BCQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.737980718 (6h21m41.327325705s) [noise=true , address=104.36.82.5:4845 , piece_id=JKN4VFS7ET2LVGHGE4LIBPCGG33GNY4SGSOXAWB2MXFYPJVO4FCQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.745434729 (6h21m41.334779717s) [noise=true , address=104.36.82.42:4820 , piece_id=XJIHI2E35NJKDFYFTW4V5EIF3CL7GDJ5EKVCTEV2KL26X43A3CYA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.754624732 (6h21m41.34396972s) [noise=true , address=104.36.82.8:4992 , piece_id=226HGROQ3DBKTJB3U6TG2OF4JQRYAGTLMSIL2JIL4NY5PISP4RUQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.763605976 (6h21m41.352950963s) [noise=true , address=104.36.82.60:4345 , piece_id=VPYOQHGEP6E257WI7IGIWXRO5WOLCNGDJYNPFPC56WUMOYDUS6RA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.763623339 (6h21m41.352968336s) [seg_pos={0 0}, upload=6610]: Segment reached optimal threshold of 14 pieces.
Aug 7 16:01:41.763741525 (6h21m41.353086512s) [noise=true , address=104.36.82.10:4110 , piece_id=WAUEFOZXH3SULGTXTWXWLEPYHZJWJDDQRRWWXRBXVWF7WCWPYGYQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ADbnPJGRTw2NqRzhy7ptojGYoNdJgqmjuWBSjcrQK53Wm7LXi): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763803123 (6h21m41.35314811s) [noise=true , address=104.36.82.61:4387 , piece_id=MAWGNSJTYYH7EVPDIXQEG6KP3MKACSMWYVMMLYUASUDJBIX5POOA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P4wAXSyaRY9poPRK5q4td8DJM69xWioG38TTwLTHKHA7QFxLp): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763857136 (6h21m41.353202113s) [noise=true , address=45.91.116.9:4267 , piece_id=MWAYSXFIIYQGDPTLWZSAHZP3UO72EXYDSSULHBJJU54H5OWG36NA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6M5TgGqUwjXyM25uqv5hsh67mS81ByzMM4kWFUNaFvisDN9d): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763904968 (6h21m41.353249955s) [noise=true , address=104.36.82.23:4455 , piece_id=UJOJ7P3VET4XIM257B3SHAN6SJG2H64QJHV6PYU7OEB7PPBZVDAA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wsyjF5X3iPrJsezUjakUArxc8K88T36yT4rE3yu6q9BMKfycm): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763945405 (6h21m41.353290392s) [seg_pos={0 0}, upload=6610]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 16:01:41.764033824 (6h21m41.353378811s) [upload=6610]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 16:01:41.764038263 (6h21m41.35338325s) [upload=6610]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 16:01:42.043459384 (6h21m41.632804371s) [upload=6610]: Done uploading segments...
Aug 7 16:01:42.043475946 (6h21m41.632820933s) [upload=6610]: Upload finished. err: <nil>
Aug 7 16:02:11.915996790 (6h22m11.505341777s) [upload=6619]: Starting upload
Aug 7 16:02:11.916045183 (6h22m11.50539017s) [upload=6619]: Done starting upload
Aug 7 16:02:11.916152489 (6h22m11.505497476s) [upload=6619]: Uploading segments...
Aug 7 16:02:11.916287706 (6h22m11.505632693s) [upload=6619]: Got next segment. Inline: false
Aug 7 16:02:11.916319939 (6h22m11.505664926s) [upload=6619]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 16:02:11.916324708 (6h22m11.505669695s) [upload=6619]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 16:02:12.129590885 (6h22m11.718935892s) [upload=6619]: Next segment err: metaclient: Too Many Requests
Aug 7 16:02:12.129623196 (6h22m11.718968194s) [upload=6619]: Done uploading segments...
Aug 7 16:02:12.129640310 (6h22m11.718985307s) [upload=6619]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.679920646s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [..................................................X.......X.] duration:
6.263154888s
FAIL (2025-08-07 16:02:35 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
e begin upload segment.
Aug 7 16:01:39.748945470 (6h21m39.338290447s) [noise=true , address=104.36.82.42:4820 , piece_id=XJIHI2E35NJKDFYFTW4V5EIF3CL7GDJ5EKVCTEV2KL26X43A3CYA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.748969657 (6h21m39.338314644s) [noise=true , address=104.36.82.60:4345 , piece_id=VPYOQHGEP6E257WI7IGIWXRO5WOLCNGDJYNPFPC56WUMOYDUS6RA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749368329 (6h21m39.338713316s) [noise=true , address=45.91.116.9:4267 , piece_id=MWAYSXFIIYQGDPTLWZSAHZP3UO72EXYDSSULHBJJU54H5OWG36NA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749375964 (6h21m39.338720951s) [noise=true , address=104.36.82.10:4110 , piece_id=WAUEFOZXH3SULGTXTWXWLEPYHZJWJDDQRRWWXRBXVWF7WCWPYGYQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749660398 (6h21m39.339005385s) [noise=true , address=79.127.205.235:4666 , piece_id=WA6GZKNO2UUAC5VKBNOZ7UO25RQR7QDT5HVNQYZDIWJEB656WMGA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749701216 (6h21m39.339046203s) [noise=true , address=79.127.201.215:4345 , piece_id=MAAFKAPO4OCQXU5WJ6DAP72KEHIM4AO5JJFJ4ZEFNWKS5M4OV43Q, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749938690 (6h21m39.339283677s) [noise=true , address=104.36.82.59:4313 , piece_id=KDH6AVALUMDXB6USO7WG77DVOSRBTGSZA6VTJFFQJR7QFC2PE35A, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750024905 (6h21m39.339369892s) [noise=true , address=104.36.82.23:4455 , piece_id=UJOJ7P3VET4XIM257B3SHAN6SJG2H64QJHV6PYU7OEB7PPBZVDAA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750215089 (6h21m39.339560076s) [noise=true , address=104.36.82.61:4387 , piece_id=MAWGNSJTYYH7EVPDIXQEG6KP3MKACSMWYVMMLYUASUDJBIX5POOA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750282639 (6h21m39.339627626s) [noise=true , address=45.91.116.10:4092 , piece_id=BUVKJC3K3RKZZTW2CQPEQM7ECQOLIBX4CU66P26JZDTOQZLYGMGA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750481038 (6h21m39.339826025s) [noise=true , address=79.127.220.97:4396 , piece_id=X2CCRNYWY5ZQOAODHNJE7VLYQIHD37PEE7MQXI5WIYV36ICONILQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750533900 (6h21m39.339878887s) [noise=true , address=104.36.82.60:4369 , piece_id=Y6SYJOB7SYBJX7STC25OJJPD4FSK5TZ5KXI7WN7QS42UNTOF5BCQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750725967 (6h21m39.340070954s) [noise=true , address=104.36.82.9:4039 , piece_id=PRBG5EVKAZ3LGDRX4MCI57T4RSNFVKZVUNOAIRHBM6SNS5XXUTPQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750795189 (6h21m39.340140176s) [noise=true , address=104.36.82.22:4525 , piece_id=NKD4EXV774KNXLEHICE4MNKOQS5QKPIUTJL2PPCNYUMYPMQEF75A, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750954023 (6h21m39.34029901s) [noise=true , address=104.36.82.45:4912 , piece_id=B6HTS6ORSGMUKZO5G2ALW4LVUSIBTCEHPX2YJTYPPCH3XROIO3HA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.751098079 (6h21m39.340443066s) [noise=true , address=104.36.82.8:4992 , piece_id=226HGROQ3DBKTJB3U6TG2OF4JQRYAGTLMSIL2JIL4NY5PISP4RUQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.751175397 (6h21m39.340520384s) [noise=true , address=79.127.205.233:4631 , piece_id=77WNKENAMS4Q75KYK4F4VNRP4DD4MKGEJIU5K7WZ3EHVQLEW6Q5A, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.751392602 (6h21m39.340737579s) [noise=true , address=104.36.82.5:4845 , piece_id=JKN4VFS7ET2LVGHGE4LIBPCGG33GNY4SGSOXAWB2MXFYPJVO4FCQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:40.878402060 (6h21m40.467747047s) [upload=6610]: Next returned nil segment
Aug 7 16:01:40.878416708 (6h21m40.467761695s) [upload=6610]: Waiting for error group managing segments...
Aug 7 16:01:41.195597254 (6h21m40.784942241s) [noise=true , address=79.127.205.235:4666 , piece_id=WA6GZKNO2UUAC5VKBNOZ7UO25RQR7QDT5HVNQYZDIWJEB656WMGA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.261081133 (6h21m40.85042612s) [noise=true , address=79.127.205.233:4631 , piece_id=77WNKENAMS4Q75KYK4F4VNRP4DD4MKGEJIU5K7WZ3EHVQLEW6Q5A, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.420961069 (6h21m41.010306046s) [noise=true , address=79.127.220.97:4396 , piece_id=X2CCRNYWY5ZQOAODHNJE7VLYQIHD37PEE7MQXI5WIYV36ICONILQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.638341479 (6h21m41.227686486s) [noise=true , address=79.127.201.215:4345 , piece_id=MAAFKAPO4OCQXU5WJ6DAP72KEHIM4AO5JJFJ4ZEFNWKS5M4OV43Q, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.658966771 (6h21m41.248311758s) [noise=true , address=45.91.116.10:4092 , piece_id=BUVKJC3K3RKZZTW2CQPEQM7ECQOLIBX4CU66P26JZDTOQZLYGMGA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.684178459 (6h21m41.273523446s) [noise=true , address=104.36.82.9:4039 , piece_id=PRBG5EVKAZ3LGDRX4MCI57T4RSNFVKZVUNOAIRHBM6SNS5XXUTPQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.691448508 (6h21m41.280793485s) [noise=true , address=104.36.82.22:4525 , piece_id=NKD4EXV774KNXLEHICE4MNKOQS5QKPIUTJL2PPCNYUMYPMQEF75A, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.717147267 (6h21m41.306492254s) [noise=true , address=104.36.82.59:4313 , piece_id=KDH6AVALUMDXB6USO7WG77DVOSRBTGSZA6VTJFFQJR7QFC2PE35A, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.719612885 (6h21m41.308957872s) [noise=true , address=104.36.82.45:4912 , piece_id=B6HTS6ORSGMUKZO5G2ALW4LVUSIBTCEHPX2YJTYPPCH3XROIO3HA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.733973160 (6h21m41.323318147s) [noise=true , address=104.36.82.60:4369 , piece_id=Y6SYJOB7SYBJX7STC25OJJPD4FSK5TZ5KXI7WN7QS42UNTOF5BCQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.737980718 (6h21m41.327325705s) [noise=true , address=104.36.82.5:4845 , piece_id=JKN4VFS7ET2LVGHGE4LIBPCGG33GNY4SGSOXAWB2MXFYPJVO4FCQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.745434729 (6h21m41.334779717s) [noise=true , address=104.36.82.42:4820 , piece_id=XJIHI2E35NJKDFYFTW4V5EIF3CL7GDJ5EKVCTEV2KL26X43A3CYA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.754624732 (6h21m41.34396972s) [noise=true , address=104.36.82.8:4992 , piece_id=226HGROQ3DBKTJB3U6TG2OF4JQRYAGTLMSIL2JIL4NY5PISP4RUQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.763605976 (6h21m41.352950963s) [noise=true , address=104.36.82.60:4345 , piece_id=VPYOQHGEP6E257WI7IGIWXRO5WOLCNGDJYNPFPC56WUMOYDUS6RA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.763623339 (6h21m41.352968336s) [seg_pos={0 0}, upload=6610]: Segment reached optimal threshold of 14 pieces.
Aug 7 16:01:41.763741525 (6h21m41.353086512s) [noise=true , address=104.36.82.10:4110 , piece_id=WAUEFOZXH3SULGTXTWXWLEPYHZJWJDDQRRWWXRBXVWF7WCWPYGYQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ADbnPJGRTw2NqRzhy7ptojGYoNdJgqmjuWBSjcrQK53Wm7LXi): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763803123 (6h21m41.35314811s) [noise=true , address=104.36.82.61:4387 , piece_id=MAWGNSJTYYH7EVPDIXQEG6KP3MKACSMWYVMMLYUASUDJBIX5POOA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P4wAXSyaRY9poPRK5q4td8DJM69xWioG38TTwLTHKHA7QFxLp): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763857136 (6h21m41.353202113s) [noise=true , address=45.91.116.9:4267 , piece_id=MWAYSXFIIYQGDPTLWZSAHZP3UO72EXYDSSULHBJJU54H5OWG36NA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6M5TgGqUwjXyM25uqv5hsh67mS81ByzMM4kWFUNaFvisDN9d): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763904968 (6h21m41.353249955s) [noise=true , address=104.36.82.23:4455 , piece_id=UJOJ7P3VET4XIM257B3SHAN6SJG2H64QJHV6PYU7OEB7PPBZVDAA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wsyjF5X3iPrJsezUjakUArxc8K88T36yT4rE3yu6q9BMKfycm): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763945405 (6h21m41.353290392s) [seg_pos={0 0}, upload=6610]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 16:01:41.764033824 (6h21m41.353378811s) [upload=6610]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 16:01:41.764038263 (6h21m41.35338325s) [upload=6610]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 16:01:42.043459384 (6h21m41.632804371s) [upload=6610]: Done uploading segments...
Aug 7 16:01:42.043475946 (6h21m41.632820933s) [upload=6610]: Upload finished. err: <nil>
Aug 7 16:02:11.915996790 (6h22m11.505341777s) [upload=6619]: Starting upload
Aug 7 16:02:11.916045183 (6h22m11.50539017s) [upload=6619]: Done starting upload
Aug 7 16:02:11.916152489 (6h22m11.505497476s) [upload=6619]: Uploading segments...
Aug 7 16:02:11.916287706 (6h22m11.505632693s) [upload=6619]: Got next segment. Inline: false
Aug 7 16:02:11.916319939 (6h22m11.505664926s) [upload=6619]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 16:02:11.916324708 (6h22m11.505669695s) [upload=6619]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 16:02:12.129590885 (6h22m11.718935892s) [upload=6619]: Next segment err: metaclient: Too Many Requests
Aug 7 16:02:12.129623196 (6h22m11.718968194s) [upload=6619]: Done uploading segments...
Aug 7 16:02:12.129640310 (6h22m11.718985307s) [upload=6619]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.263178553s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [...................................................X.......X] duration:
862.282157ms
FAIL (2025-08-07 16:02:12 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
e begin upload segment.
Aug 7 16:01:39.748945470 (6h21m39.338290447s) [noise=true , address=104.36.82.42:4820 , piece_id=XJIHI2E35NJKDFYFTW4V5EIF3CL7GDJ5EKVCTEV2KL26X43A3CYA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.748969657 (6h21m39.338314644s) [noise=true , address=104.36.82.60:4345 , piece_id=VPYOQHGEP6E257WI7IGIWXRO5WOLCNGDJYNPFPC56WUMOYDUS6RA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749368329 (6h21m39.338713316s) [noise=true , address=45.91.116.9:4267 , piece_id=MWAYSXFIIYQGDPTLWZSAHZP3UO72EXYDSSULHBJJU54H5OWG36NA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749375964 (6h21m39.338720951s) [noise=true , address=104.36.82.10:4110 , piece_id=WAUEFOZXH3SULGTXTWXWLEPYHZJWJDDQRRWWXRBXVWF7WCWPYGYQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749660398 (6h21m39.339005385s) [noise=true , address=79.127.205.235:4666 , piece_id=WA6GZKNO2UUAC5VKBNOZ7UO25RQR7QDT5HVNQYZDIWJEB656WMGA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749701216 (6h21m39.339046203s) [noise=true , address=79.127.201.215:4345 , piece_id=MAAFKAPO4OCQXU5WJ6DAP72KEHIM4AO5JJFJ4ZEFNWKS5M4OV43Q, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.749938690 (6h21m39.339283677s) [noise=true , address=104.36.82.59:4313 , piece_id=KDH6AVALUMDXB6USO7WG77DVOSRBTGSZA6VTJFFQJR7QFC2PE35A, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750024905 (6h21m39.339369892s) [noise=true , address=104.36.82.23:4455 , piece_id=UJOJ7P3VET4XIM257B3SHAN6SJG2H64QJHV6PYU7OEB7PPBZVDAA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750215089 (6h21m39.339560076s) [noise=true , address=104.36.82.61:4387 , piece_id=MAWGNSJTYYH7EVPDIXQEG6KP3MKACSMWYVMMLYUASUDJBIX5POOA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750282639 (6h21m39.339627626s) [noise=true , address=45.91.116.10:4092 , piece_id=BUVKJC3K3RKZZTW2CQPEQM7ECQOLIBX4CU66P26JZDTOQZLYGMGA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750481038 (6h21m39.339826025s) [noise=true , address=79.127.220.97:4396 , piece_id=X2CCRNYWY5ZQOAODHNJE7VLYQIHD37PEE7MQXI5WIYV36ICONILQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750533900 (6h21m39.339878887s) [noise=true , address=104.36.82.60:4369 , piece_id=Y6SYJOB7SYBJX7STC25OJJPD4FSK5TZ5KXI7WN7QS42UNTOF5BCQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750725967 (6h21m39.340070954s) [noise=true , address=104.36.82.9:4039 , piece_id=PRBG5EVKAZ3LGDRX4MCI57T4RSNFVKZVUNOAIRHBM6SNS5XXUTPQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750795189 (6h21m39.340140176s) [noise=true , address=104.36.82.22:4525 , piece_id=NKD4EXV774KNXLEHICE4MNKOQS5QKPIUTJL2PPCNYUMYPMQEF75A, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.750954023 (6h21m39.34029901s) [noise=true , address=104.36.82.45:4912 , piece_id=B6HTS6ORSGMUKZO5G2ALW4LVUSIBTCEHPX2YJTYPPCH3XROIO3HA, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.751098079 (6h21m39.340443066s) [noise=true , address=104.36.82.8:4992 , piece_id=226HGROQ3DBKTJB3U6TG2OF4JQRYAGTLMSIL2JIL4NY5PISP4RUQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.751175397 (6h21m39.340520384s) [noise=true , address=79.127.205.233:4631 , piece_id=77WNKENAMS4Q75KYK4F4VNRP4DD4MKGEJIU5K7WZ3EHVQLEW6Q5A, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:39.751392602 (6h21m39.340737579s) [noise=true , address=104.36.82.5:4845 , piece_id=JKN4VFS7ET2LVGHGE4LIBPCGG33GNY4SGSOXAWB2MXFYPJVO4FCQ, seg_pos={0 0}, upload=6610]: Uploading piece...
Aug 7 16:01:40.878402060 (6h21m40.467747047s) [upload=6610]: Next returned nil segment
Aug 7 16:01:40.878416708 (6h21m40.467761695s) [upload=6610]: Waiting for error group managing segments...
Aug 7 16:01:41.195597254 (6h21m40.784942241s) [noise=true , address=79.127.205.235:4666 , piece_id=WA6GZKNO2UUAC5VKBNOZ7UO25RQR7QDT5HVNQYZDIWJEB656WMGA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.261081133 (6h21m40.85042612s) [noise=true , address=79.127.205.233:4631 , piece_id=77WNKENAMS4Q75KYK4F4VNRP4DD4MKGEJIU5K7WZ3EHVQLEW6Q5A, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.420961069 (6h21m41.010306046s) [noise=true , address=79.127.220.97:4396 , piece_id=X2CCRNYWY5ZQOAODHNJE7VLYQIHD37PEE7MQXI5WIYV36ICONILQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.638341479 (6h21m41.227686486s) [noise=true , address=79.127.201.215:4345 , piece_id=MAAFKAPO4OCQXU5WJ6DAP72KEHIM4AO5JJFJ4ZEFNWKS5M4OV43Q, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.658966771 (6h21m41.248311758s) [noise=true , address=45.91.116.10:4092 , piece_id=BUVKJC3K3RKZZTW2CQPEQM7ECQOLIBX4CU66P26JZDTOQZLYGMGA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.684178459 (6h21m41.273523446s) [noise=true , address=104.36.82.9:4039 , piece_id=PRBG5EVKAZ3LGDRX4MCI57T4RSNFVKZVUNOAIRHBM6SNS5XXUTPQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.691448508 (6h21m41.280793485s) [noise=true , address=104.36.82.22:4525 , piece_id=NKD4EXV774KNXLEHICE4MNKOQS5QKPIUTJL2PPCNYUMYPMQEF75A, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.717147267 (6h21m41.306492254s) [noise=true , address=104.36.82.59:4313 , piece_id=KDH6AVALUMDXB6USO7WG77DVOSRBTGSZA6VTJFFQJR7QFC2PE35A, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.719612885 (6h21m41.308957872s) [noise=true , address=104.36.82.45:4912 , piece_id=B6HTS6ORSGMUKZO5G2ALW4LVUSIBTCEHPX2YJTYPPCH3XROIO3HA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.733973160 (6h21m41.323318147s) [noise=true , address=104.36.82.60:4369 , piece_id=Y6SYJOB7SYBJX7STC25OJJPD4FSK5TZ5KXI7WN7QS42UNTOF5BCQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.737980718 (6h21m41.327325705s) [noise=true , address=104.36.82.5:4845 , piece_id=JKN4VFS7ET2LVGHGE4LIBPCGG33GNY4SGSOXAWB2MXFYPJVO4FCQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.745434729 (6h21m41.334779717s) [noise=true , address=104.36.82.42:4820 , piece_id=XJIHI2E35NJKDFYFTW4V5EIF3CL7GDJ5EKVCTEV2KL26X43A3CYA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.754624732 (6h21m41.34396972s) [noise=true , address=104.36.82.8:4992 , piece_id=226HGROQ3DBKTJB3U6TG2OF4JQRYAGTLMSIL2JIL4NY5PISP4RUQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.763605976 (6h21m41.352950963s) [noise=true , address=104.36.82.60:4345 , piece_id=VPYOQHGEP6E257WI7IGIWXRO5WOLCNGDJYNPFPC56WUMOYDUS6RA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: <nil>
Aug 7 16:01:41.763623339 (6h21m41.352968336s) [seg_pos={0 0}, upload=6610]: Segment reached optimal threshold of 14 pieces.
Aug 7 16:01:41.763741525 (6h21m41.353086512s) [noise=true , address=104.36.82.10:4110 , piece_id=WAUEFOZXH3SULGTXTWXWLEPYHZJWJDDQRRWWXRBXVWF7WCWPYGYQ, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ADbnPJGRTw2NqRzhy7ptojGYoNdJgqmjuWBSjcrQK53Wm7LXi): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763803123 (6h21m41.35314811s) [noise=true , address=104.36.82.61:4387 , piece_id=MAWGNSJTYYH7EVPDIXQEG6KP3MKACSMWYVMMLYUASUDJBIX5POOA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P4wAXSyaRY9poPRK5q4td8DJM69xWioG38TTwLTHKHA7QFxLp): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763857136 (6h21m41.353202113s) [noise=true , address=45.91.116.9:4267 , piece_id=MWAYSXFIIYQGDPTLWZSAHZP3UO72EXYDSSULHBJJU54H5OWG36NA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6M5TgGqUwjXyM25uqv5hsh67mS81ByzMM4kWFUNaFvisDN9d): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763904968 (6h21m41.353249955s) [noise=true , address=104.36.82.23:4455 , piece_id=UJOJ7P3VET4XIM257B3SHAN6SJG2H64QJHV6PYU7OEB7PPBZVDAA, seg_pos={0 0}, upload=6610]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wsyjF5X3iPrJsezUjakUArxc8K88T36yT4rE3yu6q9BMKfycm): protocol: expected piece hash; context canceled; EOF
Aug 7 16:01:41.763945405 (6h21m41.353290392s) [seg_pos={0 0}, upload=6610]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 16:01:41.764033824 (6h21m41.353378811s) [upload=6610]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 16:01:41.764038263 (6h21m41.35338325s) [upload=6610]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 16:01:42.043459384 (6h21m41.632804371s) [upload=6610]: Done uploading segments...
Aug 7 16:01:42.043475946 (6h21m41.632820933s) [upload=6610]: Upload finished. err: <nil>
Aug 7 16:02:11.915996790 (6h22m11.505341777s) [upload=6619]: Starting upload
Aug 7 16:02:11.916045183 (6h22m11.50539017s) [upload=6619]: Done starting upload
Aug 7 16:02:11.916152489 (6h22m11.505497476s) [upload=6619]: Uploading segments...
Aug 7 16:02:11.916287706 (6h22m11.505632693s) [upload=6619]: Got next segment. Inline: false
Aug 7 16:02:11.916319939 (6h22m11.505664926s) [upload=6619]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 16:02:11.916324708 (6h22m11.505669695s) [upload=6619]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 16:02:12.129590885 (6h22m11.718935892s) [upload=6619]: Next segment err: metaclient: Too Many Requests
Aug 7 16:02:12.129623196 (6h22m11.718968194s) [upload=6619]: Done uploading segments...
Aug 7 16:02:12.129640310 (6h22m11.718985307s) [upload=6619]: Upload finished. err: metaclient: Too Many Requests
Duration: 862.307728ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
5.698592129s
FAIL (2025-08-07 16:01:44 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 3200fe2951d5c40fb8761116f48f2f99d16073e37121b9d866fc562ea15cb16d != before c94b9c1515c346fea3820a839d33c97ee1d3d72c7bec0256717cc21a4fac6aee
======LOGS======
) [upload=6534]: Done uploading segments...
Aug 7 15:57:19.948735937 (6h17m19.538080924s) [upload=6534]: Upload finished. err: <nil>
Aug 7 15:57:57.857972392 (6h17m57.447317379s) [upload=6548]: Starting upload
Aug 7 15:57:57.858018270 (6h17m57.447363257s) [upload=6548]: Done starting upload
Aug 7 15:57:57.858179137 (6h17m57.447524124s) [upload=6548]: Uploading segments...
Aug 7 15:57:57.858211270 (6h17m57.447556257s) [upload=6548]: Got next segment. Inline: false
Aug 7 15:57:57.858233912 (6h17m57.447578899s) [upload=6548]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 15:57:57.858238311 (6h17m57.447583298s) [upload=6548]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 15:57:58.302772381 (6h17m57.892117368s) [seg_pos={0 0}, upload=6548]: Begin upload segment...
Aug 7 15:57:58.302817418 (6h17m57.892162405s) [seg_pos={0 0}, upload=6548]: Done begin upload segment.
Aug 7 15:57:58.302827557 (6h17m57.892172544s) [noise=true , address=104.36.82.58:4282 , piece_id=EZRUNKFZDJQ3VTF5B7KXT2GHWYHTRC4H66DAUZAW5K6A7ZVB7YYA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.302836495 (6h17m57.892181482s) [noise=true , address=104.36.82.55:4206 , piece_id=QFSCVEEEMATXZWSY44VQ4I7HMGHJ63W2ULCGSATOGDUAGU3FHWGA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.303047227 (6h17m57.892392204s) [noise=true , address=104.36.82.62:4414 , piece_id=NVTRYTXJPGK2TONVR2XVANHYEWKCYFFXXYHUZGNVA6L4RM2YM5BQ, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.303152960 (6h17m57.892497947s) [noise=true , address=104.36.82.9:4040 , piece_id=ONBH7LWEZGVYG5LFTGGPMYCOXUVJA72WJULSE3E2NB5M7KOXBW3A, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.303394782 (6h17m57.892739769s) [noise=true , address=104.36.82.59:4304 , piece_id=VDKH25FKVGXQRQC47OYLYXNRC4NFPM6M3QME7SD235PFRWNXCYGA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.303635603 (6h17m57.89298058s) [noise=true , address=104.36.82.60:4339 , piece_id=ODENHZJO7X5ZQTJ2JIXLEPCJJZAIIR3K2ZCUXB25EKISOPWEFCZA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.303845374 (6h17m57.893190361s) [noise=true , address=104.36.82.5:4849 , piece_id=JPNMX6MSMTXRZOJERFQ56Z5DTRAZVGENT5AX5WX3DPIZEBME34WA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.304169585 (6h17m57.893514572s) [noise=true , address=104.36.82.35:4597 , piece_id=TXU5UA6PTEZ7SQMOL3Y3D4NT5KZJPWOZGJRZGMUHH243YS2QCNYA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.304490920 (6h17m57.893835907s) [noise=true , address=104.36.82.47:4521 , piece_id=WYHU6FTKFQSOUP4EYAAMQ5XTNXCB553GYDWK6NBYOB35EGGDMJCQ, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.304732652 (6h17m57.89407764s) [noise=true , address=104.36.82.45:4932 , piece_id=7KPJBELZNUXKSPNNHKJBEDEGNR65QHAN6OXHEV2M66V4VSFGOOYQ, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.304761858 (6h17m57.894106835s) [noise=true , address=104.36.82.61:4380 , piece_id=VF5R77WCIK2YWTVJRYXLPV3JCLH5M7MSPDKGZN74HIGV6FMUXROA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.305070939 (6h17m57.894415926s) [noise=true , address=104.36.82.58:4278 , piece_id=MFX2JYFRYTHSI5YPO2O7CIU2TKBLRCUABHRN6NCBOHMEFKNTHV5A, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.305273066 (6h17m57.894618053s) [noise=true , address=79.127.201.215:4350 , piece_id=ICC6W43RAFLNMP7JO7WJAHYQB4BQWG6DIXYEQP4R7GE6WIXE34CQ, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.305467959 (6h17m57.894812946s) [noise=true , address=104.36.82.23:4447 , piece_id=X5R2UHGAP62QXFD6SNOLKKECMJCSQCAKSOQL3B3AMC7LLC3ATKMA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.305665046 (6h17m57.895010043s) [noise=true , address=45.91.116.7:4013 , piece_id=I4EE6EKA4BIRJ3YAF3QBEYEESPYXPMPG5BYN4KPBH64CILDE7GJA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.305892882 (6h17m57.895237859s) [noise=true , address=104.36.82.44:4885 , piece_id=C7O5U5KHQHECH32AFFEV2WFYZHQIK6MNSBYW2E4OFUFJO7DVYW5A, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.306083567 (6h17m57.895428554s) [noise=true , address=45.91.116.10:4101 , piece_id=ZGOGKEWHNMQ4TS6FPL4XPX6RPAO53ZDQLENVJ2C6WSD7USLPBHCA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.306272689 (6h17m57.895617676s) [noise=true , address=64.49.13.140:4514 , piece_id=FYZL6C4DD6XAH7KLJ4GITQQ7PSJSVGGRQB6FQX2BLV3GUDVGW5VQ, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:59.460161214 (6h17m59.049506191s) [upload=6548]: Next returned nil segment
Aug 7 15:57:59.460176714 (6h17m59.049521701s) [upload=6548]: Waiting for error group managing segments...
Aug 7 15:57:59.783757965 (6h17m59.373102953s) [noise=true , address=64.49.13.140:4514 , piece_id=FYZL6C4DD6XAH7KLJ4GITQQ7PSJSVGGRQB6FQX2BLV3GUDVGW5VQ, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.212453512 (6h17m59.801798499s) [noise=true , address=104.36.82.58:4282 , piece_id=EZRUNKFZDJQ3VTF5B7KXT2GHWYHTRC4H66DAUZAW5K6A7ZVB7YYA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.212849750 (6h17m59.802194737s) [noise=true , address=104.36.82.62:4414 , piece_id=NVTRYTXJPGK2TONVR2XVANHYEWKCYFFXXYHUZGNVA6L4RM2YM5BQ, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.227864147 (6h17m59.817209164s) [noise=true , address=45.91.116.7:4013 , piece_id=I4EE6EKA4BIRJ3YAF3QBEYEESPYXPMPG5BYN4KPBH64CILDE7GJA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.235873902 (6h17m59.825218889s) [noise=true , address=79.127.201.215:4350 , piece_id=ICC6W43RAFLNMP7JO7WJAHYQB4BQWG6DIXYEQP4R7GE6WIXE34CQ, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.238157331 (6h17m59.827502318s) [noise=true , address=104.36.82.45:4932 , piece_id=7KPJBELZNUXKSPNNHKJBEDEGNR65QHAN6OXHEV2M66V4VSFGOOYQ, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.292806129 (6h17m59.882151126s) [noise=true , address=104.36.82.61:4380 , piece_id=VF5R77WCIK2YWTVJRYXLPV3JCLH5M7MSPDKGZN74HIGV6FMUXROA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.302174494 (6h17m59.891519481s) [noise=true , address=104.36.82.59:4304 , piece_id=VDKH25FKVGXQRQC47OYLYXNRC4NFPM6M3QME7SD235PFRWNXCYGA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.306753885 (6h17m59.896098872s) [noise=true , address=104.36.82.47:4521 , piece_id=WYHU6FTKFQSOUP4EYAAMQ5XTNXCB553GYDWK6NBYOB35EGGDMJCQ, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.308138284 (6h17m59.897483271s) [noise=true , address=104.36.82.35:4597 , piece_id=TXU5UA6PTEZ7SQMOL3Y3D4NT5KZJPWOZGJRZGMUHH243YS2QCNYA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.320733332 (6h17m59.910078319s) [noise=true , address=104.36.82.5:4849 , piece_id=JPNMX6MSMTXRZOJERFQ56Z5DTRAZVGENT5AX5WX3DPIZEBME34WA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.323232033 (6h17m59.91257701s) [noise=true , address=104.36.82.55:4206 , piece_id=QFSCVEEEMATXZWSY44VQ4I7HMGHJ63W2ULCGSATOGDUAGU3FHWGA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.329549890 (6h17m59.918894877s) [noise=true , address=104.36.82.23:4447 , piece_id=X5R2UHGAP62QXFD6SNOLKKECMJCSQCAKSOQL3B3AMC7LLC3ATKMA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.403643443 (6h17m59.99298843s) [noise=true , address=104.36.82.9:4040 , piece_id=ONBH7LWEZGVYG5LFTGGPMYCOXUVJA72WJULSE3E2NB5M7KOXBW3A, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.403667438 (6h17m59.993012425s) [seg_pos={0 0}, upload=6548]: Segment reached optimal threshold of 14 pieces.
Aug 7 15:58:00.403768472 (6h17m59.993113459s) [noise=true , address=104.36.82.60:4339 , piece_id=ODENHZJO7X5ZQTJ2JIXLEPCJJZAIIR3K2ZCUXB25EKISOPWEFCZA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R): protocol: expected piece hash; context canceled; EOF
Aug 7 15:58:00.403822285 (6h17m59.993167272s) [noise=true , address=104.36.82.44:4885 , piece_id=C7O5U5KHQHECH32AFFEV2WFYZHQIK6MNSBYW2E4OFUFJO7DVYW5A, seg_pos={0 0}, upload=6548]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:118qGQLFSNbsYsvFrKjTXXWbFYu2sRaRHrUNkKNCrNVpzVQRtG): protocol: expected piece hash; context canceled; EOF
Aug 7 15:58:00.403875677 (6h17m59.993220664s) [noise=true , address=104.36.82.58:4278 , piece_id=MFX2JYFRYTHSI5YPO2O7CIU2TKBLRCUABHRN6NCBOHMEFKNTHV5A, seg_pos={0 0}, upload=6548]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A56U9nehknX9qwMC9j4h2AjfAvEGbjwFFt648E7haU3jusW76): protocol: expected piece hash; context canceled; EOF
Aug 7 15:58:00.403941923 (6h17m59.9932869s) [noise=true , address=45.91.116.10:4101 , piece_id=ZGOGKEWHNMQ4TS6FPL4XPX6RPAO53ZDQLENVJ2C6WSD7USLPBHCA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CoUo7z4dqTHpL9eRTd8aetc3jvdgo8Xc6r2GozFTHkY6vqFjs): protocol: expected piece hash; context canceled; EOF
Aug 7 15:58:00.403989855 (6h17m59.993334842s) [seg_pos={0 0}, upload=6548]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 15:58:00.404093945 (6h17m59.993438932s) [upload=6548]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 15:58:00.404099565 (6h17m59.993444552s) [upload=6548]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 15:58:00.701205833 (6h18m0.29055082s) [upload=6548]: Done uploading segments...
Aug 7 15:58:00.701222535 (6h18m0.290567523s) [upload=6548]: Upload finished. err: <nil>
Duration: 5.698620604s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
5.97074296s
FAIL (2025-08-07 16:01:19 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 3200fe2951d5c40fb8761116f48f2f99d16073e37121b9d866fc562ea15cb16d != before c94b9c1515c346fea3820a839d33c97ee1d3d72c7bec0256717cc21a4fac6aee
======LOGS======
) [upload=6534]: Done uploading segments...
Aug 7 15:57:19.948735937 (6h17m19.538080924s) [upload=6534]: Upload finished. err: <nil>
Aug 7 15:57:57.857972392 (6h17m57.447317379s) [upload=6548]: Starting upload
Aug 7 15:57:57.858018270 (6h17m57.447363257s) [upload=6548]: Done starting upload
Aug 7 15:57:57.858179137 (6h17m57.447524124s) [upload=6548]: Uploading segments...
Aug 7 15:57:57.858211270 (6h17m57.447556257s) [upload=6548]: Got next segment. Inline: false
Aug 7 15:57:57.858233912 (6h17m57.447578899s) [upload=6548]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 15:57:57.858238311 (6h17m57.447583298s) [upload=6548]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 15:57:58.302772381 (6h17m57.892117368s) [seg_pos={0 0}, upload=6548]: Begin upload segment...
Aug 7 15:57:58.302817418 (6h17m57.892162405s) [seg_pos={0 0}, upload=6548]: Done begin upload segment.
Aug 7 15:57:58.302827557 (6h17m57.892172544s) [noise=true , address=104.36.82.58:4282 , piece_id=EZRUNKFZDJQ3VTF5B7KXT2GHWYHTRC4H66DAUZAW5K6A7ZVB7YYA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.302836495 (6h17m57.892181482s) [noise=true , address=104.36.82.55:4206 , piece_id=QFSCVEEEMATXZWSY44VQ4I7HMGHJ63W2ULCGSATOGDUAGU3FHWGA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.303047227 (6h17m57.892392204s) [noise=true , address=104.36.82.62:4414 , piece_id=NVTRYTXJPGK2TONVR2XVANHYEWKCYFFXXYHUZGNVA6L4RM2YM5BQ, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.303152960 (6h17m57.892497947s) [noise=true , address=104.36.82.9:4040 , piece_id=ONBH7LWEZGVYG5LFTGGPMYCOXUVJA72WJULSE3E2NB5M7KOXBW3A, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.303394782 (6h17m57.892739769s) [noise=true , address=104.36.82.59:4304 , piece_id=VDKH25FKVGXQRQC47OYLYXNRC4NFPM6M3QME7SD235PFRWNXCYGA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.303635603 (6h17m57.89298058s) [noise=true , address=104.36.82.60:4339 , piece_id=ODENHZJO7X5ZQTJ2JIXLEPCJJZAIIR3K2ZCUXB25EKISOPWEFCZA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.303845374 (6h17m57.893190361s) [noise=true , address=104.36.82.5:4849 , piece_id=JPNMX6MSMTXRZOJERFQ56Z5DTRAZVGENT5AX5WX3DPIZEBME34WA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.304169585 (6h17m57.893514572s) [noise=true , address=104.36.82.35:4597 , piece_id=TXU5UA6PTEZ7SQMOL3Y3D4NT5KZJPWOZGJRZGMUHH243YS2QCNYA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.304490920 (6h17m57.893835907s) [noise=true , address=104.36.82.47:4521 , piece_id=WYHU6FTKFQSOUP4EYAAMQ5XTNXCB553GYDWK6NBYOB35EGGDMJCQ, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.304732652 (6h17m57.89407764s) [noise=true , address=104.36.82.45:4932 , piece_id=7KPJBELZNUXKSPNNHKJBEDEGNR65QHAN6OXHEV2M66V4VSFGOOYQ, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.304761858 (6h17m57.894106835s) [noise=true , address=104.36.82.61:4380 , piece_id=VF5R77WCIK2YWTVJRYXLPV3JCLH5M7MSPDKGZN74HIGV6FMUXROA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.305070939 (6h17m57.894415926s) [noise=true , address=104.36.82.58:4278 , piece_id=MFX2JYFRYTHSI5YPO2O7CIU2TKBLRCUABHRN6NCBOHMEFKNTHV5A, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.305273066 (6h17m57.894618053s) [noise=true , address=79.127.201.215:4350 , piece_id=ICC6W43RAFLNMP7JO7WJAHYQB4BQWG6DIXYEQP4R7GE6WIXE34CQ, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.305467959 (6h17m57.894812946s) [noise=true , address=104.36.82.23:4447 , piece_id=X5R2UHGAP62QXFD6SNOLKKECMJCSQCAKSOQL3B3AMC7LLC3ATKMA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.305665046 (6h17m57.895010043s) [noise=true , address=45.91.116.7:4013 , piece_id=I4EE6EKA4BIRJ3YAF3QBEYEESPYXPMPG5BYN4KPBH64CILDE7GJA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.305892882 (6h17m57.895237859s) [noise=true , address=104.36.82.44:4885 , piece_id=C7O5U5KHQHECH32AFFEV2WFYZHQIK6MNSBYW2E4OFUFJO7DVYW5A, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.306083567 (6h17m57.895428554s) [noise=true , address=45.91.116.10:4101 , piece_id=ZGOGKEWHNMQ4TS6FPL4XPX6RPAO53ZDQLENVJ2C6WSD7USLPBHCA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.306272689 (6h17m57.895617676s) [noise=true , address=64.49.13.140:4514 , piece_id=FYZL6C4DD6XAH7KLJ4GITQQ7PSJSVGGRQB6FQX2BLV3GUDVGW5VQ, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:59.460161214 (6h17m59.049506191s) [upload=6548]: Next returned nil segment
Aug 7 15:57:59.460176714 (6h17m59.049521701s) [upload=6548]: Waiting for error group managing segments...
Aug 7 15:57:59.783757965 (6h17m59.373102953s) [noise=true , address=64.49.13.140:4514 , piece_id=FYZL6C4DD6XAH7KLJ4GITQQ7PSJSVGGRQB6FQX2BLV3GUDVGW5VQ, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.212453512 (6h17m59.801798499s) [noise=true , address=104.36.82.58:4282 , piece_id=EZRUNKFZDJQ3VTF5B7KXT2GHWYHTRC4H66DAUZAW5K6A7ZVB7YYA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.212849750 (6h17m59.802194737s) [noise=true , address=104.36.82.62:4414 , piece_id=NVTRYTXJPGK2TONVR2XVANHYEWKCYFFXXYHUZGNVA6L4RM2YM5BQ, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.227864147 (6h17m59.817209164s) [noise=true , address=45.91.116.7:4013 , piece_id=I4EE6EKA4BIRJ3YAF3QBEYEESPYXPMPG5BYN4KPBH64CILDE7GJA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.235873902 (6h17m59.825218889s) [noise=true , address=79.127.201.215:4350 , piece_id=ICC6W43RAFLNMP7JO7WJAHYQB4BQWG6DIXYEQP4R7GE6WIXE34CQ, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.238157331 (6h17m59.827502318s) [noise=true , address=104.36.82.45:4932 , piece_id=7KPJBELZNUXKSPNNHKJBEDEGNR65QHAN6OXHEV2M66V4VSFGOOYQ, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.292806129 (6h17m59.882151126s) [noise=true , address=104.36.82.61:4380 , piece_id=VF5R77WCIK2YWTVJRYXLPV3JCLH5M7MSPDKGZN74HIGV6FMUXROA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.302174494 (6h17m59.891519481s) [noise=true , address=104.36.82.59:4304 , piece_id=VDKH25FKVGXQRQC47OYLYXNRC4NFPM6M3QME7SD235PFRWNXCYGA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.306753885 (6h17m59.896098872s) [noise=true , address=104.36.82.47:4521 , piece_id=WYHU6FTKFQSOUP4EYAAMQ5XTNXCB553GYDWK6NBYOB35EGGDMJCQ, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.308138284 (6h17m59.897483271s) [noise=true , address=104.36.82.35:4597 , piece_id=TXU5UA6PTEZ7SQMOL3Y3D4NT5KZJPWOZGJRZGMUHH243YS2QCNYA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.320733332 (6h17m59.910078319s) [noise=true , address=104.36.82.5:4849 , piece_id=JPNMX6MSMTXRZOJERFQ56Z5DTRAZVGENT5AX5WX3DPIZEBME34WA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.323232033 (6h17m59.91257701s) [noise=true , address=104.36.82.55:4206 , piece_id=QFSCVEEEMATXZWSY44VQ4I7HMGHJ63W2ULCGSATOGDUAGU3FHWGA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.329549890 (6h17m59.918894877s) [noise=true , address=104.36.82.23:4447 , piece_id=X5R2UHGAP62QXFD6SNOLKKECMJCSQCAKSOQL3B3AMC7LLC3ATKMA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.403643443 (6h17m59.99298843s) [noise=true , address=104.36.82.9:4040 , piece_id=ONBH7LWEZGVYG5LFTGGPMYCOXUVJA72WJULSE3E2NB5M7KOXBW3A, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.403667438 (6h17m59.993012425s) [seg_pos={0 0}, upload=6548]: Segment reached optimal threshold of 14 pieces.
Aug 7 15:58:00.403768472 (6h17m59.993113459s) [noise=true , address=104.36.82.60:4339 , piece_id=ODENHZJO7X5ZQTJ2JIXLEPCJJZAIIR3K2ZCUXB25EKISOPWEFCZA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R): protocol: expected piece hash; context canceled; EOF
Aug 7 15:58:00.403822285 (6h17m59.993167272s) [noise=true , address=104.36.82.44:4885 , piece_id=C7O5U5KHQHECH32AFFEV2WFYZHQIK6MNSBYW2E4OFUFJO7DVYW5A, seg_pos={0 0}, upload=6548]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:118qGQLFSNbsYsvFrKjTXXWbFYu2sRaRHrUNkKNCrNVpzVQRtG): protocol: expected piece hash; context canceled; EOF
Aug 7 15:58:00.403875677 (6h17m59.993220664s) [noise=true , address=104.36.82.58:4278 , piece_id=MFX2JYFRYTHSI5YPO2O7CIU2TKBLRCUABHRN6NCBOHMEFKNTHV5A, seg_pos={0 0}, upload=6548]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A56U9nehknX9qwMC9j4h2AjfAvEGbjwFFt648E7haU3jusW76): protocol: expected piece hash; context canceled; EOF
Aug 7 15:58:00.403941923 (6h17m59.9932869s) [noise=true , address=45.91.116.10:4101 , piece_id=ZGOGKEWHNMQ4TS6FPL4XPX6RPAO53ZDQLENVJ2C6WSD7USLPBHCA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CoUo7z4dqTHpL9eRTd8aetc3jvdgo8Xc6r2GozFTHkY6vqFjs): protocol: expected piece hash; context canceled; EOF
Aug 7 15:58:00.403989855 (6h17m59.993334842s) [seg_pos={0 0}, upload=6548]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 15:58:00.404093945 (6h17m59.993438932s) [upload=6548]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 15:58:00.404099565 (6h17m59.993444552s) [upload=6548]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 15:58:00.701205833 (6h18m0.29055082s) [upload=6548]: Done uploading segments...
Aug 7 15:58:00.701222535 (6h18m0.290567523s) [upload=6548]: Upload finished. err: <nil>
Duration: 5.970783727s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
6.112081721s
FAIL (2025-08-07 16:00:56 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 3200fe2951d5c40fb8761116f48f2f99d16073e37121b9d866fc562ea15cb16d != before c94b9c1515c346fea3820a839d33c97ee1d3d72c7bec0256717cc21a4fac6aee
======LOGS======
) [upload=6534]: Done uploading segments...
Aug 7 15:57:19.948735937 (6h17m19.538080924s) [upload=6534]: Upload finished. err: <nil>
Aug 7 15:57:57.857972392 (6h17m57.447317379s) [upload=6548]: Starting upload
Aug 7 15:57:57.858018270 (6h17m57.447363257s) [upload=6548]: Done starting upload
Aug 7 15:57:57.858179137 (6h17m57.447524124s) [upload=6548]: Uploading segments...
Aug 7 15:57:57.858211270 (6h17m57.447556257s) [upload=6548]: Got next segment. Inline: false
Aug 7 15:57:57.858233912 (6h17m57.447578899s) [upload=6548]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 15:57:57.858238311 (6h17m57.447583298s) [upload=6548]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 15:57:58.302772381 (6h17m57.892117368s) [seg_pos={0 0}, upload=6548]: Begin upload segment...
Aug 7 15:57:58.302817418 (6h17m57.892162405s) [seg_pos={0 0}, upload=6548]: Done begin upload segment.
Aug 7 15:57:58.302827557 (6h17m57.892172544s) [noise=true , address=104.36.82.58:4282 , piece_id=EZRUNKFZDJQ3VTF5B7KXT2GHWYHTRC4H66DAUZAW5K6A7ZVB7YYA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.302836495 (6h17m57.892181482s) [noise=true , address=104.36.82.55:4206 , piece_id=QFSCVEEEMATXZWSY44VQ4I7HMGHJ63W2ULCGSATOGDUAGU3FHWGA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.303047227 (6h17m57.892392204s) [noise=true , address=104.36.82.62:4414 , piece_id=NVTRYTXJPGK2TONVR2XVANHYEWKCYFFXXYHUZGNVA6L4RM2YM5BQ, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.303152960 (6h17m57.892497947s) [noise=true , address=104.36.82.9:4040 , piece_id=ONBH7LWEZGVYG5LFTGGPMYCOXUVJA72WJULSE3E2NB5M7KOXBW3A, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.303394782 (6h17m57.892739769s) [noise=true , address=104.36.82.59:4304 , piece_id=VDKH25FKVGXQRQC47OYLYXNRC4NFPM6M3QME7SD235PFRWNXCYGA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.303635603 (6h17m57.89298058s) [noise=true , address=104.36.82.60:4339 , piece_id=ODENHZJO7X5ZQTJ2JIXLEPCJJZAIIR3K2ZCUXB25EKISOPWEFCZA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.303845374 (6h17m57.893190361s) [noise=true , address=104.36.82.5:4849 , piece_id=JPNMX6MSMTXRZOJERFQ56Z5DTRAZVGENT5AX5WX3DPIZEBME34WA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.304169585 (6h17m57.893514572s) [noise=true , address=104.36.82.35:4597 , piece_id=TXU5UA6PTEZ7SQMOL3Y3D4NT5KZJPWOZGJRZGMUHH243YS2QCNYA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.304490920 (6h17m57.893835907s) [noise=true , address=104.36.82.47:4521 , piece_id=WYHU6FTKFQSOUP4EYAAMQ5XTNXCB553GYDWK6NBYOB35EGGDMJCQ, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.304732652 (6h17m57.89407764s) [noise=true , address=104.36.82.45:4932 , piece_id=7KPJBELZNUXKSPNNHKJBEDEGNR65QHAN6OXHEV2M66V4VSFGOOYQ, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.304761858 (6h17m57.894106835s) [noise=true , address=104.36.82.61:4380 , piece_id=VF5R77WCIK2YWTVJRYXLPV3JCLH5M7MSPDKGZN74HIGV6FMUXROA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.305070939 (6h17m57.894415926s) [noise=true , address=104.36.82.58:4278 , piece_id=MFX2JYFRYTHSI5YPO2O7CIU2TKBLRCUABHRN6NCBOHMEFKNTHV5A, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.305273066 (6h17m57.894618053s) [noise=true , address=79.127.201.215:4350 , piece_id=ICC6W43RAFLNMP7JO7WJAHYQB4BQWG6DIXYEQP4R7GE6WIXE34CQ, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.305467959 (6h17m57.894812946s) [noise=true , address=104.36.82.23:4447 , piece_id=X5R2UHGAP62QXFD6SNOLKKECMJCSQCAKSOQL3B3AMC7LLC3ATKMA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.305665046 (6h17m57.895010043s) [noise=true , address=45.91.116.7:4013 , piece_id=I4EE6EKA4BIRJ3YAF3QBEYEESPYXPMPG5BYN4KPBH64CILDE7GJA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.305892882 (6h17m57.895237859s) [noise=true , address=104.36.82.44:4885 , piece_id=C7O5U5KHQHECH32AFFEV2WFYZHQIK6MNSBYW2E4OFUFJO7DVYW5A, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.306083567 (6h17m57.895428554s) [noise=true , address=45.91.116.10:4101 , piece_id=ZGOGKEWHNMQ4TS6FPL4XPX6RPAO53ZDQLENVJ2C6WSD7USLPBHCA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.306272689 (6h17m57.895617676s) [noise=true , address=64.49.13.140:4514 , piece_id=FYZL6C4DD6XAH7KLJ4GITQQ7PSJSVGGRQB6FQX2BLV3GUDVGW5VQ, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:59.460161214 (6h17m59.049506191s) [upload=6548]: Next returned nil segment
Aug 7 15:57:59.460176714 (6h17m59.049521701s) [upload=6548]: Waiting for error group managing segments...
Aug 7 15:57:59.783757965 (6h17m59.373102953s) [noise=true , address=64.49.13.140:4514 , piece_id=FYZL6C4DD6XAH7KLJ4GITQQ7PSJSVGGRQB6FQX2BLV3GUDVGW5VQ, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.212453512 (6h17m59.801798499s) [noise=true , address=104.36.82.58:4282 , piece_id=EZRUNKFZDJQ3VTF5B7KXT2GHWYHTRC4H66DAUZAW5K6A7ZVB7YYA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.212849750 (6h17m59.802194737s) [noise=true , address=104.36.82.62:4414 , piece_id=NVTRYTXJPGK2TONVR2XVANHYEWKCYFFXXYHUZGNVA6L4RM2YM5BQ, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.227864147 (6h17m59.817209164s) [noise=true , address=45.91.116.7:4013 , piece_id=I4EE6EKA4BIRJ3YAF3QBEYEESPYXPMPG5BYN4KPBH64CILDE7GJA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.235873902 (6h17m59.825218889s) [noise=true , address=79.127.201.215:4350 , piece_id=ICC6W43RAFLNMP7JO7WJAHYQB4BQWG6DIXYEQP4R7GE6WIXE34CQ, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.238157331 (6h17m59.827502318s) [noise=true , address=104.36.82.45:4932 , piece_id=7KPJBELZNUXKSPNNHKJBEDEGNR65QHAN6OXHEV2M66V4VSFGOOYQ, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.292806129 (6h17m59.882151126s) [noise=true , address=104.36.82.61:4380 , piece_id=VF5R77WCIK2YWTVJRYXLPV3JCLH5M7MSPDKGZN74HIGV6FMUXROA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.302174494 (6h17m59.891519481s) [noise=true , address=104.36.82.59:4304 , piece_id=VDKH25FKVGXQRQC47OYLYXNRC4NFPM6M3QME7SD235PFRWNXCYGA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.306753885 (6h17m59.896098872s) [noise=true , address=104.36.82.47:4521 , piece_id=WYHU6FTKFQSOUP4EYAAMQ5XTNXCB553GYDWK6NBYOB35EGGDMJCQ, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.308138284 (6h17m59.897483271s) [noise=true , address=104.36.82.35:4597 , piece_id=TXU5UA6PTEZ7SQMOL3Y3D4NT5KZJPWOZGJRZGMUHH243YS2QCNYA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.320733332 (6h17m59.910078319s) [noise=true , address=104.36.82.5:4849 , piece_id=JPNMX6MSMTXRZOJERFQ56Z5DTRAZVGENT5AX5WX3DPIZEBME34WA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.323232033 (6h17m59.91257701s) [noise=true , address=104.36.82.55:4206 , piece_id=QFSCVEEEMATXZWSY44VQ4I7HMGHJ63W2ULCGSATOGDUAGU3FHWGA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.329549890 (6h17m59.918894877s) [noise=true , address=104.36.82.23:4447 , piece_id=X5R2UHGAP62QXFD6SNOLKKECMJCSQCAKSOQL3B3AMC7LLC3ATKMA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.403643443 (6h17m59.99298843s) [noise=true , address=104.36.82.9:4040 , piece_id=ONBH7LWEZGVYG5LFTGGPMYCOXUVJA72WJULSE3E2NB5M7KOXBW3A, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.403667438 (6h17m59.993012425s) [seg_pos={0 0}, upload=6548]: Segment reached optimal threshold of 14 pieces.
Aug 7 15:58:00.403768472 (6h17m59.993113459s) [noise=true , address=104.36.82.60:4339 , piece_id=ODENHZJO7X5ZQTJ2JIXLEPCJJZAIIR3K2ZCUXB25EKISOPWEFCZA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R): protocol: expected piece hash; context canceled; EOF
Aug 7 15:58:00.403822285 (6h17m59.993167272s) [noise=true , address=104.36.82.44:4885 , piece_id=C7O5U5KHQHECH32AFFEV2WFYZHQIK6MNSBYW2E4OFUFJO7DVYW5A, seg_pos={0 0}, upload=6548]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:118qGQLFSNbsYsvFrKjTXXWbFYu2sRaRHrUNkKNCrNVpzVQRtG): protocol: expected piece hash; context canceled; EOF
Aug 7 15:58:00.403875677 (6h17m59.993220664s) [noise=true , address=104.36.82.58:4278 , piece_id=MFX2JYFRYTHSI5YPO2O7CIU2TKBLRCUABHRN6NCBOHMEFKNTHV5A, seg_pos={0 0}, upload=6548]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A56U9nehknX9qwMC9j4h2AjfAvEGbjwFFt648E7haU3jusW76): protocol: expected piece hash; context canceled; EOF
Aug 7 15:58:00.403941923 (6h17m59.9932869s) [noise=true , address=45.91.116.10:4101 , piece_id=ZGOGKEWHNMQ4TS6FPL4XPX6RPAO53ZDQLENVJ2C6WSD7USLPBHCA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CoUo7z4dqTHpL9eRTd8aetc3jvdgo8Xc6r2GozFTHkY6vqFjs): protocol: expected piece hash; context canceled; EOF
Aug 7 15:58:00.403989855 (6h17m59.993334842s) [seg_pos={0 0}, upload=6548]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 15:58:00.404093945 (6h17m59.993438932s) [upload=6548]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 15:58:00.404099565 (6h17m59.993444552s) [upload=6548]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 15:58:00.701205833 (6h18m0.29055082s) [upload=6548]: Done uploading segments...
Aug 7 15:58:00.701222535 (6h18m0.290567523s) [upload=6548]: Upload finished. err: <nil>
Duration: 6.112101089s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
5.8372855s
FAIL (2025-08-07 16:00:22 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 3200fe2951d5c40fb8761116f48f2f99d16073e37121b9d866fc562ea15cb16d != before c94b9c1515c346fea3820a839d33c97ee1d3d72c7bec0256717cc21a4fac6aee
======LOGS======
) [upload=6534]: Done uploading segments...
Aug 7 15:57:19.948735937 (6h17m19.538080924s) [upload=6534]: Upload finished. err: <nil>
Aug 7 15:57:57.857972392 (6h17m57.447317379s) [upload=6548]: Starting upload
Aug 7 15:57:57.858018270 (6h17m57.447363257s) [upload=6548]: Done starting upload
Aug 7 15:57:57.858179137 (6h17m57.447524124s) [upload=6548]: Uploading segments...
Aug 7 15:57:57.858211270 (6h17m57.447556257s) [upload=6548]: Got next segment. Inline: false
Aug 7 15:57:57.858233912 (6h17m57.447578899s) [upload=6548]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 15:57:57.858238311 (6h17m57.447583298s) [upload=6548]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 15:57:58.302772381 (6h17m57.892117368s) [seg_pos={0 0}, upload=6548]: Begin upload segment...
Aug 7 15:57:58.302817418 (6h17m57.892162405s) [seg_pos={0 0}, upload=6548]: Done begin upload segment.
Aug 7 15:57:58.302827557 (6h17m57.892172544s) [noise=true , address=104.36.82.58:4282 , piece_id=EZRUNKFZDJQ3VTF5B7KXT2GHWYHTRC4H66DAUZAW5K6A7ZVB7YYA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.302836495 (6h17m57.892181482s) [noise=true , address=104.36.82.55:4206 , piece_id=QFSCVEEEMATXZWSY44VQ4I7HMGHJ63W2ULCGSATOGDUAGU3FHWGA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.303047227 (6h17m57.892392204s) [noise=true , address=104.36.82.62:4414 , piece_id=NVTRYTXJPGK2TONVR2XVANHYEWKCYFFXXYHUZGNVA6L4RM2YM5BQ, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.303152960 (6h17m57.892497947s) [noise=true , address=104.36.82.9:4040 , piece_id=ONBH7LWEZGVYG5LFTGGPMYCOXUVJA72WJULSE3E2NB5M7KOXBW3A, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.303394782 (6h17m57.892739769s) [noise=true , address=104.36.82.59:4304 , piece_id=VDKH25FKVGXQRQC47OYLYXNRC4NFPM6M3QME7SD235PFRWNXCYGA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.303635603 (6h17m57.89298058s) [noise=true , address=104.36.82.60:4339 , piece_id=ODENHZJO7X5ZQTJ2JIXLEPCJJZAIIR3K2ZCUXB25EKISOPWEFCZA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.303845374 (6h17m57.893190361s) [noise=true , address=104.36.82.5:4849 , piece_id=JPNMX6MSMTXRZOJERFQ56Z5DTRAZVGENT5AX5WX3DPIZEBME34WA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.304169585 (6h17m57.893514572s) [noise=true , address=104.36.82.35:4597 , piece_id=TXU5UA6PTEZ7SQMOL3Y3D4NT5KZJPWOZGJRZGMUHH243YS2QCNYA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.304490920 (6h17m57.893835907s) [noise=true , address=104.36.82.47:4521 , piece_id=WYHU6FTKFQSOUP4EYAAMQ5XTNXCB553GYDWK6NBYOB35EGGDMJCQ, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.304732652 (6h17m57.89407764s) [noise=true , address=104.36.82.45:4932 , piece_id=7KPJBELZNUXKSPNNHKJBEDEGNR65QHAN6OXHEV2M66V4VSFGOOYQ, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.304761858 (6h17m57.894106835s) [noise=true , address=104.36.82.61:4380 , piece_id=VF5R77WCIK2YWTVJRYXLPV3JCLH5M7MSPDKGZN74HIGV6FMUXROA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.305070939 (6h17m57.894415926s) [noise=true , address=104.36.82.58:4278 , piece_id=MFX2JYFRYTHSI5YPO2O7CIU2TKBLRCUABHRN6NCBOHMEFKNTHV5A, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.305273066 (6h17m57.894618053s) [noise=true , address=79.127.201.215:4350 , piece_id=ICC6W43RAFLNMP7JO7WJAHYQB4BQWG6DIXYEQP4R7GE6WIXE34CQ, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.305467959 (6h17m57.894812946s) [noise=true , address=104.36.82.23:4447 , piece_id=X5R2UHGAP62QXFD6SNOLKKECMJCSQCAKSOQL3B3AMC7LLC3ATKMA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.305665046 (6h17m57.895010043s) [noise=true , address=45.91.116.7:4013 , piece_id=I4EE6EKA4BIRJ3YAF3QBEYEESPYXPMPG5BYN4KPBH64CILDE7GJA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.305892882 (6h17m57.895237859s) [noise=true , address=104.36.82.44:4885 , piece_id=C7O5U5KHQHECH32AFFEV2WFYZHQIK6MNSBYW2E4OFUFJO7DVYW5A, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.306083567 (6h17m57.895428554s) [noise=true , address=45.91.116.10:4101 , piece_id=ZGOGKEWHNMQ4TS6FPL4XPX6RPAO53ZDQLENVJ2C6WSD7USLPBHCA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.306272689 (6h17m57.895617676s) [noise=true , address=64.49.13.140:4514 , piece_id=FYZL6C4DD6XAH7KLJ4GITQQ7PSJSVGGRQB6FQX2BLV3GUDVGW5VQ, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:59.460161214 (6h17m59.049506191s) [upload=6548]: Next returned nil segment
Aug 7 15:57:59.460176714 (6h17m59.049521701s) [upload=6548]: Waiting for error group managing segments...
Aug 7 15:57:59.783757965 (6h17m59.373102953s) [noise=true , address=64.49.13.140:4514 , piece_id=FYZL6C4DD6XAH7KLJ4GITQQ7PSJSVGGRQB6FQX2BLV3GUDVGW5VQ, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.212453512 (6h17m59.801798499s) [noise=true , address=104.36.82.58:4282 , piece_id=EZRUNKFZDJQ3VTF5B7KXT2GHWYHTRC4H66DAUZAW5K6A7ZVB7YYA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.212849750 (6h17m59.802194737s) [noise=true , address=104.36.82.62:4414 , piece_id=NVTRYTXJPGK2TONVR2XVANHYEWKCYFFXXYHUZGNVA6L4RM2YM5BQ, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.227864147 (6h17m59.817209164s) [noise=true , address=45.91.116.7:4013 , piece_id=I4EE6EKA4BIRJ3YAF3QBEYEESPYXPMPG5BYN4KPBH64CILDE7GJA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.235873902 (6h17m59.825218889s) [noise=true , address=79.127.201.215:4350 , piece_id=ICC6W43RAFLNMP7JO7WJAHYQB4BQWG6DIXYEQP4R7GE6WIXE34CQ, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.238157331 (6h17m59.827502318s) [noise=true , address=104.36.82.45:4932 , piece_id=7KPJBELZNUXKSPNNHKJBEDEGNR65QHAN6OXHEV2M66V4VSFGOOYQ, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.292806129 (6h17m59.882151126s) [noise=true , address=104.36.82.61:4380 , piece_id=VF5R77WCIK2YWTVJRYXLPV3JCLH5M7MSPDKGZN74HIGV6FMUXROA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.302174494 (6h17m59.891519481s) [noise=true , address=104.36.82.59:4304 , piece_id=VDKH25FKVGXQRQC47OYLYXNRC4NFPM6M3QME7SD235PFRWNXCYGA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.306753885 (6h17m59.896098872s) [noise=true , address=104.36.82.47:4521 , piece_id=WYHU6FTKFQSOUP4EYAAMQ5XTNXCB553GYDWK6NBYOB35EGGDMJCQ, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.308138284 (6h17m59.897483271s) [noise=true , address=104.36.82.35:4597 , piece_id=TXU5UA6PTEZ7SQMOL3Y3D4NT5KZJPWOZGJRZGMUHH243YS2QCNYA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.320733332 (6h17m59.910078319s) [noise=true , address=104.36.82.5:4849 , piece_id=JPNMX6MSMTXRZOJERFQ56Z5DTRAZVGENT5AX5WX3DPIZEBME34WA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.323232033 (6h17m59.91257701s) [noise=true , address=104.36.82.55:4206 , piece_id=QFSCVEEEMATXZWSY44VQ4I7HMGHJ63W2ULCGSATOGDUAGU3FHWGA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.329549890 (6h17m59.918894877s) [noise=true , address=104.36.82.23:4447 , piece_id=X5R2UHGAP62QXFD6SNOLKKECMJCSQCAKSOQL3B3AMC7LLC3ATKMA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.403643443 (6h17m59.99298843s) [noise=true , address=104.36.82.9:4040 , piece_id=ONBH7LWEZGVYG5LFTGGPMYCOXUVJA72WJULSE3E2NB5M7KOXBW3A, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.403667438 (6h17m59.993012425s) [seg_pos={0 0}, upload=6548]: Segment reached optimal threshold of 14 pieces.
Aug 7 15:58:00.403768472 (6h17m59.993113459s) [noise=true , address=104.36.82.60:4339 , piece_id=ODENHZJO7X5ZQTJ2JIXLEPCJJZAIIR3K2ZCUXB25EKISOPWEFCZA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R): protocol: expected piece hash; context canceled; EOF
Aug 7 15:58:00.403822285 (6h17m59.993167272s) [noise=true , address=104.36.82.44:4885 , piece_id=C7O5U5KHQHECH32AFFEV2WFYZHQIK6MNSBYW2E4OFUFJO7DVYW5A, seg_pos={0 0}, upload=6548]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:118qGQLFSNbsYsvFrKjTXXWbFYu2sRaRHrUNkKNCrNVpzVQRtG): protocol: expected piece hash; context canceled; EOF
Aug 7 15:58:00.403875677 (6h17m59.993220664s) [noise=true , address=104.36.82.58:4278 , piece_id=MFX2JYFRYTHSI5YPO2O7CIU2TKBLRCUABHRN6NCBOHMEFKNTHV5A, seg_pos={0 0}, upload=6548]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A56U9nehknX9qwMC9j4h2AjfAvEGbjwFFt648E7haU3jusW76): protocol: expected piece hash; context canceled; EOF
Aug 7 15:58:00.403941923 (6h17m59.9932869s) [noise=true , address=45.91.116.10:4101 , piece_id=ZGOGKEWHNMQ4TS6FPL4XPX6RPAO53ZDQLENVJ2C6WSD7USLPBHCA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CoUo7z4dqTHpL9eRTd8aetc3jvdgo8Xc6r2GozFTHkY6vqFjs): protocol: expected piece hash; context canceled; EOF
Aug 7 15:58:00.403989855 (6h17m59.993334842s) [seg_pos={0 0}, upload=6548]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 15:58:00.404093945 (6h17m59.993438932s) [upload=6548]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 15:58:00.404099565 (6h17m59.993444552s) [upload=6548]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 15:58:00.701205833 (6h18m0.29055082s) [upload=6548]: Done uploading segments...
Aug 7 15:58:00.701222535 (6h18m0.290567523s) [upload=6548]: Upload finished. err: <nil>
Duration: 5.837312702s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
5.634511796s
FAIL (2025-08-07 15:59:29 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 3200fe2951d5c40fb8761116f48f2f99d16073e37121b9d866fc562ea15cb16d != before c94b9c1515c346fea3820a839d33c97ee1d3d72c7bec0256717cc21a4fac6aee
======LOGS======
) [upload=6534]: Done uploading segments...
Aug 7 15:57:19.948735937 (6h17m19.538080924s) [upload=6534]: Upload finished. err: <nil>
Aug 7 15:57:57.857972392 (6h17m57.447317379s) [upload=6548]: Starting upload
Aug 7 15:57:57.858018270 (6h17m57.447363257s) [upload=6548]: Done starting upload
Aug 7 15:57:57.858179137 (6h17m57.447524124s) [upload=6548]: Uploading segments...
Aug 7 15:57:57.858211270 (6h17m57.447556257s) [upload=6548]: Got next segment. Inline: false
Aug 7 15:57:57.858233912 (6h17m57.447578899s) [upload=6548]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 15:57:57.858238311 (6h17m57.447583298s) [upload=6548]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 15:57:58.302772381 (6h17m57.892117368s) [seg_pos={0 0}, upload=6548]: Begin upload segment...
Aug 7 15:57:58.302817418 (6h17m57.892162405s) [seg_pos={0 0}, upload=6548]: Done begin upload segment.
Aug 7 15:57:58.302827557 (6h17m57.892172544s) [noise=true , address=104.36.82.58:4282 , piece_id=EZRUNKFZDJQ3VTF5B7KXT2GHWYHTRC4H66DAUZAW5K6A7ZVB7YYA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.302836495 (6h17m57.892181482s) [noise=true , address=104.36.82.55:4206 , piece_id=QFSCVEEEMATXZWSY44VQ4I7HMGHJ63W2ULCGSATOGDUAGU3FHWGA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.303047227 (6h17m57.892392204s) [noise=true , address=104.36.82.62:4414 , piece_id=NVTRYTXJPGK2TONVR2XVANHYEWKCYFFXXYHUZGNVA6L4RM2YM5BQ, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.303152960 (6h17m57.892497947s) [noise=true , address=104.36.82.9:4040 , piece_id=ONBH7LWEZGVYG5LFTGGPMYCOXUVJA72WJULSE3E2NB5M7KOXBW3A, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.303394782 (6h17m57.892739769s) [noise=true , address=104.36.82.59:4304 , piece_id=VDKH25FKVGXQRQC47OYLYXNRC4NFPM6M3QME7SD235PFRWNXCYGA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.303635603 (6h17m57.89298058s) [noise=true , address=104.36.82.60:4339 , piece_id=ODENHZJO7X5ZQTJ2JIXLEPCJJZAIIR3K2ZCUXB25EKISOPWEFCZA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.303845374 (6h17m57.893190361s) [noise=true , address=104.36.82.5:4849 , piece_id=JPNMX6MSMTXRZOJERFQ56Z5DTRAZVGENT5AX5WX3DPIZEBME34WA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.304169585 (6h17m57.893514572s) [noise=true , address=104.36.82.35:4597 , piece_id=TXU5UA6PTEZ7SQMOL3Y3D4NT5KZJPWOZGJRZGMUHH243YS2QCNYA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.304490920 (6h17m57.893835907s) [noise=true , address=104.36.82.47:4521 , piece_id=WYHU6FTKFQSOUP4EYAAMQ5XTNXCB553GYDWK6NBYOB35EGGDMJCQ, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.304732652 (6h17m57.89407764s) [noise=true , address=104.36.82.45:4932 , piece_id=7KPJBELZNUXKSPNNHKJBEDEGNR65QHAN6OXHEV2M66V4VSFGOOYQ, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.304761858 (6h17m57.894106835s) [noise=true , address=104.36.82.61:4380 , piece_id=VF5R77WCIK2YWTVJRYXLPV3JCLH5M7MSPDKGZN74HIGV6FMUXROA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.305070939 (6h17m57.894415926s) [noise=true , address=104.36.82.58:4278 , piece_id=MFX2JYFRYTHSI5YPO2O7CIU2TKBLRCUABHRN6NCBOHMEFKNTHV5A, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.305273066 (6h17m57.894618053s) [noise=true , address=79.127.201.215:4350 , piece_id=ICC6W43RAFLNMP7JO7WJAHYQB4BQWG6DIXYEQP4R7GE6WIXE34CQ, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.305467959 (6h17m57.894812946s) [noise=true , address=104.36.82.23:4447 , piece_id=X5R2UHGAP62QXFD6SNOLKKECMJCSQCAKSOQL3B3AMC7LLC3ATKMA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.305665046 (6h17m57.895010043s) [noise=true , address=45.91.116.7:4013 , piece_id=I4EE6EKA4BIRJ3YAF3QBEYEESPYXPMPG5BYN4KPBH64CILDE7GJA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.305892882 (6h17m57.895237859s) [noise=true , address=104.36.82.44:4885 , piece_id=C7O5U5KHQHECH32AFFEV2WFYZHQIK6MNSBYW2E4OFUFJO7DVYW5A, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.306083567 (6h17m57.895428554s) [noise=true , address=45.91.116.10:4101 , piece_id=ZGOGKEWHNMQ4TS6FPL4XPX6RPAO53ZDQLENVJ2C6WSD7USLPBHCA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.306272689 (6h17m57.895617676s) [noise=true , address=64.49.13.140:4514 , piece_id=FYZL6C4DD6XAH7KLJ4GITQQ7PSJSVGGRQB6FQX2BLV3GUDVGW5VQ, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:59.460161214 (6h17m59.049506191s) [upload=6548]: Next returned nil segment
Aug 7 15:57:59.460176714 (6h17m59.049521701s) [upload=6548]: Waiting for error group managing segments...
Aug 7 15:57:59.783757965 (6h17m59.373102953s) [noise=true , address=64.49.13.140:4514 , piece_id=FYZL6C4DD6XAH7KLJ4GITQQ7PSJSVGGRQB6FQX2BLV3GUDVGW5VQ, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.212453512 (6h17m59.801798499s) [noise=true , address=104.36.82.58:4282 , piece_id=EZRUNKFZDJQ3VTF5B7KXT2GHWYHTRC4H66DAUZAW5K6A7ZVB7YYA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.212849750 (6h17m59.802194737s) [noise=true , address=104.36.82.62:4414 , piece_id=NVTRYTXJPGK2TONVR2XVANHYEWKCYFFXXYHUZGNVA6L4RM2YM5BQ, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.227864147 (6h17m59.817209164s) [noise=true , address=45.91.116.7:4013 , piece_id=I4EE6EKA4BIRJ3YAF3QBEYEESPYXPMPG5BYN4KPBH64CILDE7GJA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.235873902 (6h17m59.825218889s) [noise=true , address=79.127.201.215:4350 , piece_id=ICC6W43RAFLNMP7JO7WJAHYQB4BQWG6DIXYEQP4R7GE6WIXE34CQ, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.238157331 (6h17m59.827502318s) [noise=true , address=104.36.82.45:4932 , piece_id=7KPJBELZNUXKSPNNHKJBEDEGNR65QHAN6OXHEV2M66V4VSFGOOYQ, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.292806129 (6h17m59.882151126s) [noise=true , address=104.36.82.61:4380 , piece_id=VF5R77WCIK2YWTVJRYXLPV3JCLH5M7MSPDKGZN74HIGV6FMUXROA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.302174494 (6h17m59.891519481s) [noise=true , address=104.36.82.59:4304 , piece_id=VDKH25FKVGXQRQC47OYLYXNRC4NFPM6M3QME7SD235PFRWNXCYGA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.306753885 (6h17m59.896098872s) [noise=true , address=104.36.82.47:4521 , piece_id=WYHU6FTKFQSOUP4EYAAMQ5XTNXCB553GYDWK6NBYOB35EGGDMJCQ, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.308138284 (6h17m59.897483271s) [noise=true , address=104.36.82.35:4597 , piece_id=TXU5UA6PTEZ7SQMOL3Y3D4NT5KZJPWOZGJRZGMUHH243YS2QCNYA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.320733332 (6h17m59.910078319s) [noise=true , address=104.36.82.5:4849 , piece_id=JPNMX6MSMTXRZOJERFQ56Z5DTRAZVGENT5AX5WX3DPIZEBME34WA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.323232033 (6h17m59.91257701s) [noise=true , address=104.36.82.55:4206 , piece_id=QFSCVEEEMATXZWSY44VQ4I7HMGHJ63W2ULCGSATOGDUAGU3FHWGA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.329549890 (6h17m59.918894877s) [noise=true , address=104.36.82.23:4447 , piece_id=X5R2UHGAP62QXFD6SNOLKKECMJCSQCAKSOQL3B3AMC7LLC3ATKMA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.403643443 (6h17m59.99298843s) [noise=true , address=104.36.82.9:4040 , piece_id=ONBH7LWEZGVYG5LFTGGPMYCOXUVJA72WJULSE3E2NB5M7KOXBW3A, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.403667438 (6h17m59.993012425s) [seg_pos={0 0}, upload=6548]: Segment reached optimal threshold of 14 pieces.
Aug 7 15:58:00.403768472 (6h17m59.993113459s) [noise=true , address=104.36.82.60:4339 , piece_id=ODENHZJO7X5ZQTJ2JIXLEPCJJZAIIR3K2ZCUXB25EKISOPWEFCZA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R): protocol: expected piece hash; context canceled; EOF
Aug 7 15:58:00.403822285 (6h17m59.993167272s) [noise=true , address=104.36.82.44:4885 , piece_id=C7O5U5KHQHECH32AFFEV2WFYZHQIK6MNSBYW2E4OFUFJO7DVYW5A, seg_pos={0 0}, upload=6548]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:118qGQLFSNbsYsvFrKjTXXWbFYu2sRaRHrUNkKNCrNVpzVQRtG): protocol: expected piece hash; context canceled; EOF
Aug 7 15:58:00.403875677 (6h17m59.993220664s) [noise=true , address=104.36.82.58:4278 , piece_id=MFX2JYFRYTHSI5YPO2O7CIU2TKBLRCUABHRN6NCBOHMEFKNTHV5A, seg_pos={0 0}, upload=6548]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A56U9nehknX9qwMC9j4h2AjfAvEGbjwFFt648E7haU3jusW76): protocol: expected piece hash; context canceled; EOF
Aug 7 15:58:00.403941923 (6h17m59.9932869s) [noise=true , address=45.91.116.10:4101 , piece_id=ZGOGKEWHNMQ4TS6FPL4XPX6RPAO53ZDQLENVJ2C6WSD7USLPBHCA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CoUo7z4dqTHpL9eRTd8aetc3jvdgo8Xc6r2GozFTHkY6vqFjs): protocol: expected piece hash; context canceled; EOF
Aug 7 15:58:00.403989855 (6h17m59.993334842s) [seg_pos={0 0}, upload=6548]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 15:58:00.404093945 (6h17m59.993438932s) [upload=6548]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 15:58:00.404099565 (6h17m59.993444552s) [upload=6548]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 15:58:00.701205833 (6h18m0.29055082s) [upload=6548]: Done uploading segments...
Aug 7 15:58:00.701222535 (6h18m0.290567523s) [upload=6548]: Upload finished. err: <nil>
Duration: 5.634539969s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
5.792154741s
FAIL (2025-08-07 15:59:05 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 3200fe2951d5c40fb8761116f48f2f99d16073e37121b9d866fc562ea15cb16d != before c94b9c1515c346fea3820a839d33c97ee1d3d72c7bec0256717cc21a4fac6aee
======LOGS======
) [upload=6534]: Done uploading segments...
Aug 7 15:57:19.948735937 (6h17m19.538080924s) [upload=6534]: Upload finished. err: <nil>
Aug 7 15:57:57.857972392 (6h17m57.447317379s) [upload=6548]: Starting upload
Aug 7 15:57:57.858018270 (6h17m57.447363257s) [upload=6548]: Done starting upload
Aug 7 15:57:57.858179137 (6h17m57.447524124s) [upload=6548]: Uploading segments...
Aug 7 15:57:57.858211270 (6h17m57.447556257s) [upload=6548]: Got next segment. Inline: false
Aug 7 15:57:57.858233912 (6h17m57.447578899s) [upload=6548]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 15:57:57.858238311 (6h17m57.447583298s) [upload=6548]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 15:57:58.302772381 (6h17m57.892117368s) [seg_pos={0 0}, upload=6548]: Begin upload segment...
Aug 7 15:57:58.302817418 (6h17m57.892162405s) [seg_pos={0 0}, upload=6548]: Done begin upload segment.
Aug 7 15:57:58.302827557 (6h17m57.892172544s) [noise=true , address=104.36.82.58:4282 , piece_id=EZRUNKFZDJQ3VTF5B7KXT2GHWYHTRC4H66DAUZAW5K6A7ZVB7YYA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.302836495 (6h17m57.892181482s) [noise=true , address=104.36.82.55:4206 , piece_id=QFSCVEEEMATXZWSY44VQ4I7HMGHJ63W2ULCGSATOGDUAGU3FHWGA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.303047227 (6h17m57.892392204s) [noise=true , address=104.36.82.62:4414 , piece_id=NVTRYTXJPGK2TONVR2XVANHYEWKCYFFXXYHUZGNVA6L4RM2YM5BQ, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.303152960 (6h17m57.892497947s) [noise=true , address=104.36.82.9:4040 , piece_id=ONBH7LWEZGVYG5LFTGGPMYCOXUVJA72WJULSE3E2NB5M7KOXBW3A, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.303394782 (6h17m57.892739769s) [noise=true , address=104.36.82.59:4304 , piece_id=VDKH25FKVGXQRQC47OYLYXNRC4NFPM6M3QME7SD235PFRWNXCYGA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.303635603 (6h17m57.89298058s) [noise=true , address=104.36.82.60:4339 , piece_id=ODENHZJO7X5ZQTJ2JIXLEPCJJZAIIR3K2ZCUXB25EKISOPWEFCZA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.303845374 (6h17m57.893190361s) [noise=true , address=104.36.82.5:4849 , piece_id=JPNMX6MSMTXRZOJERFQ56Z5DTRAZVGENT5AX5WX3DPIZEBME34WA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.304169585 (6h17m57.893514572s) [noise=true , address=104.36.82.35:4597 , piece_id=TXU5UA6PTEZ7SQMOL3Y3D4NT5KZJPWOZGJRZGMUHH243YS2QCNYA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.304490920 (6h17m57.893835907s) [noise=true , address=104.36.82.47:4521 , piece_id=WYHU6FTKFQSOUP4EYAAMQ5XTNXCB553GYDWK6NBYOB35EGGDMJCQ, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.304732652 (6h17m57.89407764s) [noise=true , address=104.36.82.45:4932 , piece_id=7KPJBELZNUXKSPNNHKJBEDEGNR65QHAN6OXHEV2M66V4VSFGOOYQ, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.304761858 (6h17m57.894106835s) [noise=true , address=104.36.82.61:4380 , piece_id=VF5R77WCIK2YWTVJRYXLPV3JCLH5M7MSPDKGZN74HIGV6FMUXROA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.305070939 (6h17m57.894415926s) [noise=true , address=104.36.82.58:4278 , piece_id=MFX2JYFRYTHSI5YPO2O7CIU2TKBLRCUABHRN6NCBOHMEFKNTHV5A, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.305273066 (6h17m57.894618053s) [noise=true , address=79.127.201.215:4350 , piece_id=ICC6W43RAFLNMP7JO7WJAHYQB4BQWG6DIXYEQP4R7GE6WIXE34CQ, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.305467959 (6h17m57.894812946s) [noise=true , address=104.36.82.23:4447 , piece_id=X5R2UHGAP62QXFD6SNOLKKECMJCSQCAKSOQL3B3AMC7LLC3ATKMA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.305665046 (6h17m57.895010043s) [noise=true , address=45.91.116.7:4013 , piece_id=I4EE6EKA4BIRJ3YAF3QBEYEESPYXPMPG5BYN4KPBH64CILDE7GJA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.305892882 (6h17m57.895237859s) [noise=true , address=104.36.82.44:4885 , piece_id=C7O5U5KHQHECH32AFFEV2WFYZHQIK6MNSBYW2E4OFUFJO7DVYW5A, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.306083567 (6h17m57.895428554s) [noise=true , address=45.91.116.10:4101 , piece_id=ZGOGKEWHNMQ4TS6FPL4XPX6RPAO53ZDQLENVJ2C6WSD7USLPBHCA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.306272689 (6h17m57.895617676s) [noise=true , address=64.49.13.140:4514 , piece_id=FYZL6C4DD6XAH7KLJ4GITQQ7PSJSVGGRQB6FQX2BLV3GUDVGW5VQ, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:59.460161214 (6h17m59.049506191s) [upload=6548]: Next returned nil segment
Aug 7 15:57:59.460176714 (6h17m59.049521701s) [upload=6548]: Waiting for error group managing segments...
Aug 7 15:57:59.783757965 (6h17m59.373102953s) [noise=true , address=64.49.13.140:4514 , piece_id=FYZL6C4DD6XAH7KLJ4GITQQ7PSJSVGGRQB6FQX2BLV3GUDVGW5VQ, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.212453512 (6h17m59.801798499s) [noise=true , address=104.36.82.58:4282 , piece_id=EZRUNKFZDJQ3VTF5B7KXT2GHWYHTRC4H66DAUZAW5K6A7ZVB7YYA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.212849750 (6h17m59.802194737s) [noise=true , address=104.36.82.62:4414 , piece_id=NVTRYTXJPGK2TONVR2XVANHYEWKCYFFXXYHUZGNVA6L4RM2YM5BQ, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.227864147 (6h17m59.817209164s) [noise=true , address=45.91.116.7:4013 , piece_id=I4EE6EKA4BIRJ3YAF3QBEYEESPYXPMPG5BYN4KPBH64CILDE7GJA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.235873902 (6h17m59.825218889s) [noise=true , address=79.127.201.215:4350 , piece_id=ICC6W43RAFLNMP7JO7WJAHYQB4BQWG6DIXYEQP4R7GE6WIXE34CQ, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.238157331 (6h17m59.827502318s) [noise=true , address=104.36.82.45:4932 , piece_id=7KPJBELZNUXKSPNNHKJBEDEGNR65QHAN6OXHEV2M66V4VSFGOOYQ, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.292806129 (6h17m59.882151126s) [noise=true , address=104.36.82.61:4380 , piece_id=VF5R77WCIK2YWTVJRYXLPV3JCLH5M7MSPDKGZN74HIGV6FMUXROA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.302174494 (6h17m59.891519481s) [noise=true , address=104.36.82.59:4304 , piece_id=VDKH25FKVGXQRQC47OYLYXNRC4NFPM6M3QME7SD235PFRWNXCYGA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.306753885 (6h17m59.896098872s) [noise=true , address=104.36.82.47:4521 , piece_id=WYHU6FTKFQSOUP4EYAAMQ5XTNXCB553GYDWK6NBYOB35EGGDMJCQ, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.308138284 (6h17m59.897483271s) [noise=true , address=104.36.82.35:4597 , piece_id=TXU5UA6PTEZ7SQMOL3Y3D4NT5KZJPWOZGJRZGMUHH243YS2QCNYA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.320733332 (6h17m59.910078319s) [noise=true , address=104.36.82.5:4849 , piece_id=JPNMX6MSMTXRZOJERFQ56Z5DTRAZVGENT5AX5WX3DPIZEBME34WA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.323232033 (6h17m59.91257701s) [noise=true , address=104.36.82.55:4206 , piece_id=QFSCVEEEMATXZWSY44VQ4I7HMGHJ63W2ULCGSATOGDUAGU3FHWGA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.329549890 (6h17m59.918894877s) [noise=true , address=104.36.82.23:4447 , piece_id=X5R2UHGAP62QXFD6SNOLKKECMJCSQCAKSOQL3B3AMC7LLC3ATKMA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.403643443 (6h17m59.99298843s) [noise=true , address=104.36.82.9:4040 , piece_id=ONBH7LWEZGVYG5LFTGGPMYCOXUVJA72WJULSE3E2NB5M7KOXBW3A, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.403667438 (6h17m59.993012425s) [seg_pos={0 0}, upload=6548]: Segment reached optimal threshold of 14 pieces.
Aug 7 15:58:00.403768472 (6h17m59.993113459s) [noise=true , address=104.36.82.60:4339 , piece_id=ODENHZJO7X5ZQTJ2JIXLEPCJJZAIIR3K2ZCUXB25EKISOPWEFCZA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R): protocol: expected piece hash; context canceled; EOF
Aug 7 15:58:00.403822285 (6h17m59.993167272s) [noise=true , address=104.36.82.44:4885 , piece_id=C7O5U5KHQHECH32AFFEV2WFYZHQIK6MNSBYW2E4OFUFJO7DVYW5A, seg_pos={0 0}, upload=6548]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:118qGQLFSNbsYsvFrKjTXXWbFYu2sRaRHrUNkKNCrNVpzVQRtG): protocol: expected piece hash; context canceled; EOF
Aug 7 15:58:00.403875677 (6h17m59.993220664s) [noise=true , address=104.36.82.58:4278 , piece_id=MFX2JYFRYTHSI5YPO2O7CIU2TKBLRCUABHRN6NCBOHMEFKNTHV5A, seg_pos={0 0}, upload=6548]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A56U9nehknX9qwMC9j4h2AjfAvEGbjwFFt648E7haU3jusW76): protocol: expected piece hash; context canceled; EOF
Aug 7 15:58:00.403941923 (6h17m59.9932869s) [noise=true , address=45.91.116.10:4101 , piece_id=ZGOGKEWHNMQ4TS6FPL4XPX6RPAO53ZDQLENVJ2C6WSD7USLPBHCA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CoUo7z4dqTHpL9eRTd8aetc3jvdgo8Xc6r2GozFTHkY6vqFjs): protocol: expected piece hash; context canceled; EOF
Aug 7 15:58:00.403989855 (6h17m59.993334842s) [seg_pos={0 0}, upload=6548]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 15:58:00.404093945 (6h17m59.993438932s) [upload=6548]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 15:58:00.404099565 (6h17m59.993444552s) [upload=6548]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 15:58:00.701205833 (6h18m0.29055082s) [upload=6548]: Done uploading segments...
Aug 7 15:58:00.701222535 (6h18m0.290567523s) [upload=6548]: Upload finished. err: <nil>
Duration: 5.792182404s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
5.841222743s
FAIL (2025-08-07 15:58:31 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 3200fe2951d5c40fb8761116f48f2f99d16073e37121b9d866fc562ea15cb16d != before c94b9c1515c346fea3820a839d33c97ee1d3d72c7bec0256717cc21a4fac6aee
======LOGS======
) [upload=6534]: Done uploading segments...
Aug 7 15:57:19.948735937 (6h17m19.538080924s) [upload=6534]: Upload finished. err: <nil>
Aug 7 15:57:57.857972392 (6h17m57.447317379s) [upload=6548]: Starting upload
Aug 7 15:57:57.858018270 (6h17m57.447363257s) [upload=6548]: Done starting upload
Aug 7 15:57:57.858179137 (6h17m57.447524124s) [upload=6548]: Uploading segments...
Aug 7 15:57:57.858211270 (6h17m57.447556257s) [upload=6548]: Got next segment. Inline: false
Aug 7 15:57:57.858233912 (6h17m57.447578899s) [upload=6548]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 15:57:57.858238311 (6h17m57.447583298s) [upload=6548]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 15:57:58.302772381 (6h17m57.892117368s) [seg_pos={0 0}, upload=6548]: Begin upload segment...
Aug 7 15:57:58.302817418 (6h17m57.892162405s) [seg_pos={0 0}, upload=6548]: Done begin upload segment.
Aug 7 15:57:58.302827557 (6h17m57.892172544s) [noise=true , address=104.36.82.58:4282 , piece_id=EZRUNKFZDJQ3VTF5B7KXT2GHWYHTRC4H66DAUZAW5K6A7ZVB7YYA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.302836495 (6h17m57.892181482s) [noise=true , address=104.36.82.55:4206 , piece_id=QFSCVEEEMATXZWSY44VQ4I7HMGHJ63W2ULCGSATOGDUAGU3FHWGA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.303047227 (6h17m57.892392204s) [noise=true , address=104.36.82.62:4414 , piece_id=NVTRYTXJPGK2TONVR2XVANHYEWKCYFFXXYHUZGNVA6L4RM2YM5BQ, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.303152960 (6h17m57.892497947s) [noise=true , address=104.36.82.9:4040 , piece_id=ONBH7LWEZGVYG5LFTGGPMYCOXUVJA72WJULSE3E2NB5M7KOXBW3A, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.303394782 (6h17m57.892739769s) [noise=true , address=104.36.82.59:4304 , piece_id=VDKH25FKVGXQRQC47OYLYXNRC4NFPM6M3QME7SD235PFRWNXCYGA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.303635603 (6h17m57.89298058s) [noise=true , address=104.36.82.60:4339 , piece_id=ODENHZJO7X5ZQTJ2JIXLEPCJJZAIIR3K2ZCUXB25EKISOPWEFCZA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.303845374 (6h17m57.893190361s) [noise=true , address=104.36.82.5:4849 , piece_id=JPNMX6MSMTXRZOJERFQ56Z5DTRAZVGENT5AX5WX3DPIZEBME34WA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.304169585 (6h17m57.893514572s) [noise=true , address=104.36.82.35:4597 , piece_id=TXU5UA6PTEZ7SQMOL3Y3D4NT5KZJPWOZGJRZGMUHH243YS2QCNYA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.304490920 (6h17m57.893835907s) [noise=true , address=104.36.82.47:4521 , piece_id=WYHU6FTKFQSOUP4EYAAMQ5XTNXCB553GYDWK6NBYOB35EGGDMJCQ, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.304732652 (6h17m57.89407764s) [noise=true , address=104.36.82.45:4932 , piece_id=7KPJBELZNUXKSPNNHKJBEDEGNR65QHAN6OXHEV2M66V4VSFGOOYQ, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.304761858 (6h17m57.894106835s) [noise=true , address=104.36.82.61:4380 , piece_id=VF5R77WCIK2YWTVJRYXLPV3JCLH5M7MSPDKGZN74HIGV6FMUXROA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.305070939 (6h17m57.894415926s) [noise=true , address=104.36.82.58:4278 , piece_id=MFX2JYFRYTHSI5YPO2O7CIU2TKBLRCUABHRN6NCBOHMEFKNTHV5A, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.305273066 (6h17m57.894618053s) [noise=true , address=79.127.201.215:4350 , piece_id=ICC6W43RAFLNMP7JO7WJAHYQB4BQWG6DIXYEQP4R7GE6WIXE34CQ, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.305467959 (6h17m57.894812946s) [noise=true , address=104.36.82.23:4447 , piece_id=X5R2UHGAP62QXFD6SNOLKKECMJCSQCAKSOQL3B3AMC7LLC3ATKMA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.305665046 (6h17m57.895010043s) [noise=true , address=45.91.116.7:4013 , piece_id=I4EE6EKA4BIRJ3YAF3QBEYEESPYXPMPG5BYN4KPBH64CILDE7GJA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.305892882 (6h17m57.895237859s) [noise=true , address=104.36.82.44:4885 , piece_id=C7O5U5KHQHECH32AFFEV2WFYZHQIK6MNSBYW2E4OFUFJO7DVYW5A, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.306083567 (6h17m57.895428554s) [noise=true , address=45.91.116.10:4101 , piece_id=ZGOGKEWHNMQ4TS6FPL4XPX6RPAO53ZDQLENVJ2C6WSD7USLPBHCA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.306272689 (6h17m57.895617676s) [noise=true , address=64.49.13.140:4514 , piece_id=FYZL6C4DD6XAH7KLJ4GITQQ7PSJSVGGRQB6FQX2BLV3GUDVGW5VQ, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:59.460161214 (6h17m59.049506191s) [upload=6548]: Next returned nil segment
Aug 7 15:57:59.460176714 (6h17m59.049521701s) [upload=6548]: Waiting for error group managing segments...
Aug 7 15:57:59.783757965 (6h17m59.373102953s) [noise=true , address=64.49.13.140:4514 , piece_id=FYZL6C4DD6XAH7KLJ4GITQQ7PSJSVGGRQB6FQX2BLV3GUDVGW5VQ, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.212453512 (6h17m59.801798499s) [noise=true , address=104.36.82.58:4282 , piece_id=EZRUNKFZDJQ3VTF5B7KXT2GHWYHTRC4H66DAUZAW5K6A7ZVB7YYA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.212849750 (6h17m59.802194737s) [noise=true , address=104.36.82.62:4414 , piece_id=NVTRYTXJPGK2TONVR2XVANHYEWKCYFFXXYHUZGNVA6L4RM2YM5BQ, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.227864147 (6h17m59.817209164s) [noise=true , address=45.91.116.7:4013 , piece_id=I4EE6EKA4BIRJ3YAF3QBEYEESPYXPMPG5BYN4KPBH64CILDE7GJA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.235873902 (6h17m59.825218889s) [noise=true , address=79.127.201.215:4350 , piece_id=ICC6W43RAFLNMP7JO7WJAHYQB4BQWG6DIXYEQP4R7GE6WIXE34CQ, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.238157331 (6h17m59.827502318s) [noise=true , address=104.36.82.45:4932 , piece_id=7KPJBELZNUXKSPNNHKJBEDEGNR65QHAN6OXHEV2M66V4VSFGOOYQ, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.292806129 (6h17m59.882151126s) [noise=true , address=104.36.82.61:4380 , piece_id=VF5R77WCIK2YWTVJRYXLPV3JCLH5M7MSPDKGZN74HIGV6FMUXROA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.302174494 (6h17m59.891519481s) [noise=true , address=104.36.82.59:4304 , piece_id=VDKH25FKVGXQRQC47OYLYXNRC4NFPM6M3QME7SD235PFRWNXCYGA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.306753885 (6h17m59.896098872s) [noise=true , address=104.36.82.47:4521 , piece_id=WYHU6FTKFQSOUP4EYAAMQ5XTNXCB553GYDWK6NBYOB35EGGDMJCQ, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.308138284 (6h17m59.897483271s) [noise=true , address=104.36.82.35:4597 , piece_id=TXU5UA6PTEZ7SQMOL3Y3D4NT5KZJPWOZGJRZGMUHH243YS2QCNYA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.320733332 (6h17m59.910078319s) [noise=true , address=104.36.82.5:4849 , piece_id=JPNMX6MSMTXRZOJERFQ56Z5DTRAZVGENT5AX5WX3DPIZEBME34WA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.323232033 (6h17m59.91257701s) [noise=true , address=104.36.82.55:4206 , piece_id=QFSCVEEEMATXZWSY44VQ4I7HMGHJ63W2ULCGSATOGDUAGU3FHWGA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.329549890 (6h17m59.918894877s) [noise=true , address=104.36.82.23:4447 , piece_id=X5R2UHGAP62QXFD6SNOLKKECMJCSQCAKSOQL3B3AMC7LLC3ATKMA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.403643443 (6h17m59.99298843s) [noise=true , address=104.36.82.9:4040 , piece_id=ONBH7LWEZGVYG5LFTGGPMYCOXUVJA72WJULSE3E2NB5M7KOXBW3A, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.403667438 (6h17m59.993012425s) [seg_pos={0 0}, upload=6548]: Segment reached optimal threshold of 14 pieces.
Aug 7 15:58:00.403768472 (6h17m59.993113459s) [noise=true , address=104.36.82.60:4339 , piece_id=ODENHZJO7X5ZQTJ2JIXLEPCJJZAIIR3K2ZCUXB25EKISOPWEFCZA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R): protocol: expected piece hash; context canceled; EOF
Aug 7 15:58:00.403822285 (6h17m59.993167272s) [noise=true , address=104.36.82.44:4885 , piece_id=C7O5U5KHQHECH32AFFEV2WFYZHQIK6MNSBYW2E4OFUFJO7DVYW5A, seg_pos={0 0}, upload=6548]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:118qGQLFSNbsYsvFrKjTXXWbFYu2sRaRHrUNkKNCrNVpzVQRtG): protocol: expected piece hash; context canceled; EOF
Aug 7 15:58:00.403875677 (6h17m59.993220664s) [noise=true , address=104.36.82.58:4278 , piece_id=MFX2JYFRYTHSI5YPO2O7CIU2TKBLRCUABHRN6NCBOHMEFKNTHV5A, seg_pos={0 0}, upload=6548]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A56U9nehknX9qwMC9j4h2AjfAvEGbjwFFt648E7haU3jusW76): protocol: expected piece hash; context canceled; EOF
Aug 7 15:58:00.403941923 (6h17m59.9932869s) [noise=true , address=45.91.116.10:4101 , piece_id=ZGOGKEWHNMQ4TS6FPL4XPX6RPAO53ZDQLENVJ2C6WSD7USLPBHCA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CoUo7z4dqTHpL9eRTd8aetc3jvdgo8Xc6r2GozFTHkY6vqFjs): protocol: expected piece hash; context canceled; EOF
Aug 7 15:58:00.403989855 (6h17m59.993334842s) [seg_pos={0 0}, upload=6548]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 15:58:00.404093945 (6h17m59.993438932s) [upload=6548]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 15:58:00.404099565 (6h17m59.993444552s) [upload=6548]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 15:58:00.701205833 (6h18m0.29055082s) [upload=6548]: Done uploading segments...
Aug 7 15:58:00.701222535 (6h18m0.290567523s) [upload=6548]: Upload finished. err: <nil>
Duration: 5.841257188s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
5.775730036s
FAIL (2025-08-07 15:58:02 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 3200fe2951d5c40fb8761116f48f2f99d16073e37121b9d866fc562ea15cb16d != before c94b9c1515c346fea3820a839d33c97ee1d3d72c7bec0256717cc21a4fac6aee
======LOGS======
) [upload=6534]: Done uploading segments...
Aug 7 15:57:19.948735937 (6h17m19.538080924s) [upload=6534]: Upload finished. err: <nil>
Aug 7 15:57:57.857972392 (6h17m57.447317379s) [upload=6548]: Starting upload
Aug 7 15:57:57.858018270 (6h17m57.447363257s) [upload=6548]: Done starting upload
Aug 7 15:57:57.858179137 (6h17m57.447524124s) [upload=6548]: Uploading segments...
Aug 7 15:57:57.858211270 (6h17m57.447556257s) [upload=6548]: Got next segment. Inline: false
Aug 7 15:57:57.858233912 (6h17m57.447578899s) [upload=6548]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 7 15:57:57.858238311 (6h17m57.447583298s) [upload=6548]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 7 15:57:58.302772381 (6h17m57.892117368s) [seg_pos={0 0}, upload=6548]: Begin upload segment...
Aug 7 15:57:58.302817418 (6h17m57.892162405s) [seg_pos={0 0}, upload=6548]: Done begin upload segment.
Aug 7 15:57:58.302827557 (6h17m57.892172544s) [noise=true , address=104.36.82.58:4282 , piece_id=EZRUNKFZDJQ3VTF5B7KXT2GHWYHTRC4H66DAUZAW5K6A7ZVB7YYA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.302836495 (6h17m57.892181482s) [noise=true , address=104.36.82.55:4206 , piece_id=QFSCVEEEMATXZWSY44VQ4I7HMGHJ63W2ULCGSATOGDUAGU3FHWGA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.303047227 (6h17m57.892392204s) [noise=true , address=104.36.82.62:4414 , piece_id=NVTRYTXJPGK2TONVR2XVANHYEWKCYFFXXYHUZGNVA6L4RM2YM5BQ, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.303152960 (6h17m57.892497947s) [noise=true , address=104.36.82.9:4040 , piece_id=ONBH7LWEZGVYG5LFTGGPMYCOXUVJA72WJULSE3E2NB5M7KOXBW3A, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.303394782 (6h17m57.892739769s) [noise=true , address=104.36.82.59:4304 , piece_id=VDKH25FKVGXQRQC47OYLYXNRC4NFPM6M3QME7SD235PFRWNXCYGA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.303635603 (6h17m57.89298058s) [noise=true , address=104.36.82.60:4339 , piece_id=ODENHZJO7X5ZQTJ2JIXLEPCJJZAIIR3K2ZCUXB25EKISOPWEFCZA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.303845374 (6h17m57.893190361s) [noise=true , address=104.36.82.5:4849 , piece_id=JPNMX6MSMTXRZOJERFQ56Z5DTRAZVGENT5AX5WX3DPIZEBME34WA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.304169585 (6h17m57.893514572s) [noise=true , address=104.36.82.35:4597 , piece_id=TXU5UA6PTEZ7SQMOL3Y3D4NT5KZJPWOZGJRZGMUHH243YS2QCNYA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.304490920 (6h17m57.893835907s) [noise=true , address=104.36.82.47:4521 , piece_id=WYHU6FTKFQSOUP4EYAAMQ5XTNXCB553GYDWK6NBYOB35EGGDMJCQ, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.304732652 (6h17m57.89407764s) [noise=true , address=104.36.82.45:4932 , piece_id=7KPJBELZNUXKSPNNHKJBEDEGNR65QHAN6OXHEV2M66V4VSFGOOYQ, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.304761858 (6h17m57.894106835s) [noise=true , address=104.36.82.61:4380 , piece_id=VF5R77WCIK2YWTVJRYXLPV3JCLH5M7MSPDKGZN74HIGV6FMUXROA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.305070939 (6h17m57.894415926s) [noise=true , address=104.36.82.58:4278 , piece_id=MFX2JYFRYTHSI5YPO2O7CIU2TKBLRCUABHRN6NCBOHMEFKNTHV5A, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.305273066 (6h17m57.894618053s) [noise=true , address=79.127.201.215:4350 , piece_id=ICC6W43RAFLNMP7JO7WJAHYQB4BQWG6DIXYEQP4R7GE6WIXE34CQ, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.305467959 (6h17m57.894812946s) [noise=true , address=104.36.82.23:4447 , piece_id=X5R2UHGAP62QXFD6SNOLKKECMJCSQCAKSOQL3B3AMC7LLC3ATKMA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.305665046 (6h17m57.895010043s) [noise=true , address=45.91.116.7:4013 , piece_id=I4EE6EKA4BIRJ3YAF3QBEYEESPYXPMPG5BYN4KPBH64CILDE7GJA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.305892882 (6h17m57.895237859s) [noise=true , address=104.36.82.44:4885 , piece_id=C7O5U5KHQHECH32AFFEV2WFYZHQIK6MNSBYW2E4OFUFJO7DVYW5A, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.306083567 (6h17m57.895428554s) [noise=true , address=45.91.116.10:4101 , piece_id=ZGOGKEWHNMQ4TS6FPL4XPX6RPAO53ZDQLENVJ2C6WSD7USLPBHCA, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:58.306272689 (6h17m57.895617676s) [noise=true , address=64.49.13.140:4514 , piece_id=FYZL6C4DD6XAH7KLJ4GITQQ7PSJSVGGRQB6FQX2BLV3GUDVGW5VQ, seg_pos={0 0}, upload=6548]: Uploading piece...
Aug 7 15:57:59.460161214 (6h17m59.049506191s) [upload=6548]: Next returned nil segment
Aug 7 15:57:59.460176714 (6h17m59.049521701s) [upload=6548]: Waiting for error group managing segments...
Aug 7 15:57:59.783757965 (6h17m59.373102953s) [noise=true , address=64.49.13.140:4514 , piece_id=FYZL6C4DD6XAH7KLJ4GITQQ7PSJSVGGRQB6FQX2BLV3GUDVGW5VQ, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.212453512 (6h17m59.801798499s) [noise=true , address=104.36.82.58:4282 , piece_id=EZRUNKFZDJQ3VTF5B7KXT2GHWYHTRC4H66DAUZAW5K6A7ZVB7YYA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.212849750 (6h17m59.802194737s) [noise=true , address=104.36.82.62:4414 , piece_id=NVTRYTXJPGK2TONVR2XVANHYEWKCYFFXXYHUZGNVA6L4RM2YM5BQ, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.227864147 (6h17m59.817209164s) [noise=true , address=45.91.116.7:4013 , piece_id=I4EE6EKA4BIRJ3YAF3QBEYEESPYXPMPG5BYN4KPBH64CILDE7GJA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.235873902 (6h17m59.825218889s) [noise=true , address=79.127.201.215:4350 , piece_id=ICC6W43RAFLNMP7JO7WJAHYQB4BQWG6DIXYEQP4R7GE6WIXE34CQ, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.238157331 (6h17m59.827502318s) [noise=true , address=104.36.82.45:4932 , piece_id=7KPJBELZNUXKSPNNHKJBEDEGNR65QHAN6OXHEV2M66V4VSFGOOYQ, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.292806129 (6h17m59.882151126s) [noise=true , address=104.36.82.61:4380 , piece_id=VF5R77WCIK2YWTVJRYXLPV3JCLH5M7MSPDKGZN74HIGV6FMUXROA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.302174494 (6h17m59.891519481s) [noise=true , address=104.36.82.59:4304 , piece_id=VDKH25FKVGXQRQC47OYLYXNRC4NFPM6M3QME7SD235PFRWNXCYGA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.306753885 (6h17m59.896098872s) [noise=true , address=104.36.82.47:4521 , piece_id=WYHU6FTKFQSOUP4EYAAMQ5XTNXCB553GYDWK6NBYOB35EGGDMJCQ, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.308138284 (6h17m59.897483271s) [noise=true , address=104.36.82.35:4597 , piece_id=TXU5UA6PTEZ7SQMOL3Y3D4NT5KZJPWOZGJRZGMUHH243YS2QCNYA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.320733332 (6h17m59.910078319s) [noise=true , address=104.36.82.5:4849 , piece_id=JPNMX6MSMTXRZOJERFQ56Z5DTRAZVGENT5AX5WX3DPIZEBME34WA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.323232033 (6h17m59.91257701s) [noise=true , address=104.36.82.55:4206 , piece_id=QFSCVEEEMATXZWSY44VQ4I7HMGHJ63W2ULCGSATOGDUAGU3FHWGA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.329549890 (6h17m59.918894877s) [noise=true , address=104.36.82.23:4447 , piece_id=X5R2UHGAP62QXFD6SNOLKKECMJCSQCAKSOQL3B3AMC7LLC3ATKMA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.403643443 (6h17m59.99298843s) [noise=true , address=104.36.82.9:4040 , piece_id=ONBH7LWEZGVYG5LFTGGPMYCOXUVJA72WJULSE3E2NB5M7KOXBW3A, seg_pos={0 0}, upload=6548]: Done uploading piece. err: <nil>
Aug 7 15:58:00.403667438 (6h17m59.993012425s) [seg_pos={0 0}, upload=6548]: Segment reached optimal threshold of 14 pieces.
Aug 7 15:58:00.403768472 (6h17m59.993113459s) [noise=true , address=104.36.82.60:4339 , piece_id=ODENHZJO7X5ZQTJ2JIXLEPCJJZAIIR3K2ZCUXB25EKISOPWEFCZA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R): protocol: expected piece hash; context canceled; EOF
Aug 7 15:58:00.403822285 (6h17m59.993167272s) [noise=true , address=104.36.82.44:4885 , piece_id=C7O5U5KHQHECH32AFFEV2WFYZHQIK6MNSBYW2E4OFUFJO7DVYW5A, seg_pos={0 0}, upload=6548]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:118qGQLFSNbsYsvFrKjTXXWbFYu2sRaRHrUNkKNCrNVpzVQRtG): protocol: expected piece hash; context canceled; EOF
Aug 7 15:58:00.403875677 (6h17m59.993220664s) [noise=true , address=104.36.82.58:4278 , piece_id=MFX2JYFRYTHSI5YPO2O7CIU2TKBLRCUABHRN6NCBOHMEFKNTHV5A, seg_pos={0 0}, upload=6548]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A56U9nehknX9qwMC9j4h2AjfAvEGbjwFFt648E7haU3jusW76): protocol: expected piece hash; context canceled; EOF
Aug 7 15:58:00.403941923 (6h17m59.9932869s) [noise=true , address=45.91.116.10:4101 , piece_id=ZGOGKEWHNMQ4TS6FPL4XPX6RPAO53ZDQLENVJ2C6WSD7USLPBHCA, seg_pos={0 0}, upload=6548]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CoUo7z4dqTHpL9eRTd8aetc3jvdgo8Xc6r2GozFTHkY6vqFjs): protocol: expected piece hash; context canceled; EOF
Aug 7 15:58:00.403989855 (6h17m59.993334842s) [seg_pos={0 0}, upload=6548]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 7 15:58:00.404093945 (6h17m59.993438932s) [upload=6548]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 7 15:58:00.404099565 (6h17m59.993444552s) [upload=6548]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 7 15:58:00.701205833 (6h18m0.29055082s) [upload=6548]: Done uploading segments...
Aug 7 15:58:00.701222535 (6h18m0.290567523s) [upload=6548]: Upload finished. err: <nil>
Duration: 5.775752679s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
2.046010861s
FAIL (2025-08-07 15:45:56 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 0c950c360f79058769477f5e701e3fda95fc8e0cb346ec9019ff64f7c3416e1b != before 2cdc378d7ef84049ec07bf6864d59358611769fe343c5ef7867468744bd826f9 ======LOGS====== h item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 14:52:11.554336215 (5h12m11.143681192s) [upload=5437]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 14:52:11.842624981 (5h12m11.431969968s) [upload=5437]: Done uploading segments... Aug 7 14:52:11.842643616 (5h12m11.431988603s) [upload=5437]: Upload finished. err: <nil> Aug 7 14:52:29.842451400 (5h12m29.431796387s) [upload=5442]: Starting upload Aug 7 14:52:29.842494922 (5h12m29.431839899s) [upload=5442]: Done starting upload Aug 7 14:52:29.842567301 (5h12m29.431912288s) [upload=5442]: Uploading segments... Aug 7 14:52:29.842618079 (5h12m29.431963066s) [upload=5442]: Got next segment. Inline: true Aug 7 14:52:29.842622377 (5h12m29.431967354s) [upload=5442]: Waiting for error group managing segments... Aug 7 14:52:29.842657043 (5h12m29.43200203s) [upload=5442]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 14:52:29.842661120 (5h12m29.432006097s) [upload=5442]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 14:52:29.842663386 (5h12m29.432008373s) [upload=5442]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 14:52:30.223558730 (5h12m29.812903717s) [upload=5442]: Done uploading segments... Aug 7 14:52:30.223582075 (5h12m29.812927082s) [upload=5442]: Upload finished. err: <nil> Aug 7 14:53:10.306580950 (5h13m9.895925927s) [upload=5455]: Starting upload Aug 7 14:53:10.306625405 (5h13m9.895970392s) [upload=5455]: Done starting upload Aug 7 14:53:10.306668007 (5h13m9.896012994s) [upload=5455]: Uploading segments... Aug 7 14:53:10.306715197 (5h13m9.896060184s) [upload=5455]: Got next segment. Inline: true Aug 7 14:53:10.306718493 (5h13m9.89606347s) [upload=5455]: Waiting for error group managing segments... Aug 7 14:53:10.306752358 (5h13m9.896097345s) [upload=5455]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 14:53:10.306767648 (5h13m9.896112635s) [upload=5455]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 14:53:10.306770353 (5h13m9.89611534s) [upload=5455]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 14:53:10.689930130 (5h13m10.279275127s) [upload=5455]: Done uploading segments... Aug 7 14:53:10.689953535 (5h13m10.279298552s) [upload=5455]: Upload finished. err: <nil> Aug 7 14:53:33.669638991 (5h13m33.258983968s) [upload=5460]: Starting upload Aug 7 14:53:33.669694297 (5h13m33.259039284s) [upload=5460]: Done starting upload Aug 7 14:53:33.669745955 (5h13m33.259090952s) [upload=5460]: Uploading segments... Aug 7 14:53:33.669775272 (5h13m33.259120249s) [upload=5460]: Got next segment. Inline: true Aug 7 14:53:33.669778608 (5h13m33.259123595s) [upload=5460]: Waiting for error group managing segments... Aug 7 14:53:33.669844874 (5h13m33.259189861s) [upload=5460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 14:53:33.669849273 (5h13m33.25919426s) [upload=5460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 14:53:33.669851847 (5h13m33.259196834s) [upload=5460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 14:53:33.947641263 (5h13m33.53698624s) [upload=5460]: Done uploading segments... Aug 7 14:53:33.947667363 (5h13m33.53701234s) [upload=5460]: Upload finished. err: <nil> Aug 7 14:54:08.475562955 (5h14m8.064907942s) [upload=5472]: Starting upload Aug 7 14:54:08.475591189 (5h14m8.064936176s) [upload=5472]: Done starting upload Aug 7 14:54:08.475631777 (5h14m8.064976764s) [upload=5472]: Uploading segments... Aug 7 14:54:08.475658518 (5h14m8.065003505s) [upload=5472]: Got next segment. Inline: true Aug 7 14:54:08.475661844 (5h14m8.065006831s) [upload=5472]: Waiting for error group managing segments... Aug 7 14:54:08.475693354 (5h14m8.065038342s) [upload=5472]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 14:54:08.475696761 (5h14m8.065041748s) [upload=5472]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 14:54:08.475698955 (5h14m8.065043932s) [upload=5472]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 14:54:08.773471083 (5h14m8.36281606s) [upload=5472]: Done uploading segments... Aug 7 14:54:08.773489398 (5h14m8.362834385s) [upload=5472]: Upload finished. err: <nil> Aug 7 14:55:23.389342434 (5h15m22.978687451s) [upload=5494]: Starting upload Aug 7 14:55:23.389429360 (5h15m22.978774397s) [upload=5494]: Done starting upload Aug 7 14:55:23.389493003 (5h15m22.97883803s) [upload=5494]: Uploading segments... Aug 7 14:55:23.389570009 (5h15m22.978914996s) [upload=5494]: Got next segment. Inline: true Aug 7 14:55:23.389575010 (5h15m22.978919987s) [upload=5494]: Waiting for error group managing segments... Aug 7 14:55:23.389610337 (5h15m22.978955324s) [upload=5494]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 14:55:23.389615887 (5h15m22.978960874s) [upload=5494]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 14:55:23.389618252 (5h15m22.978963229s) [upload=5494]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 14:55:23.741693456 (5h15m23.331038443s) [upload=5494]: Done uploading segments... Aug 7 14:55:23.741717722 (5h15m23.331062709s) [upload=5494]: Upload finished. err: <nil> Aug 7 14:55:47.730252989 (5h15m47.319597966s) [upload=5501]: Starting upload Aug 7 14:55:47.730301272 (5h15m47.319646259s) [upload=5501]: Done starting upload Aug 7 14:55:47.730350075 (5h15m47.319695062s) [upload=5501]: Uploading segments... Aug 7 14:55:47.730382697 (5h15m47.319727684s) [upload=5501]: Got next segment. Inline: true Aug 7 14:55:47.730386825 (5h15m47.319731812s) [upload=5501]: Waiting for error group managing segments... Aug 7 14:55:47.730419067 (5h15m47.319764054s) [upload=5501]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 14:55:47.730422804 (5h15m47.319767791s) [upload=5501]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 14:55:47.730425148 (5h15m47.319770135s) [upload=5501]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 14:55:48.028445502 (5h15m47.617790489s) [upload=5501]: Done uploading segments... Aug 7 14:55:48.028465410 (5h15m47.617810397s) [upload=5501]: Upload finished. err: <nil> Aug 7 14:56:46.695516721 (5h16m46.284861708s) [upload=5518]: Starting upload Aug 7 14:56:46.695559422 (5h16m46.284904409s) [upload=5518]: Done starting upload Aug 7 14:56:46.695606793 (5h16m46.28495178s) [upload=5518]: Uploading segments... Aug 7 14:56:46.695635067 (5h16m46.284980054s) [upload=5518]: Got next segment. Inline: true Aug 7 14:56:46.695640697 (5h16m46.284985684s) [upload=5518]: Waiting for error group managing segments... Aug 7 14:56:46.695672628 (5h16m46.285017615s) [upload=5518]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 14:56:46.695676767 (5h16m46.285021754s) [upload=5518]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 14:56:46.695679482 (5h16m46.285024469s) [upload=5518]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 14:56:46.970794841 (5h16m46.560139848s) [upload=5518]: Done uploading segments... Aug 7 14:56:46.970817584 (5h16m46.560162592s) [upload=5518]: Upload finished. err: <nil> Aug 7 14:57:05.705953271 (5h17m5.295298259s) [upload=5524]: Starting upload Aug 7 14:57:05.705999811 (5h17m5.295344798s) [upload=5524]: Done starting upload Aug 7 14:57:05.706046721 (5h17m5.295391708s) [upload=5524]: Uploading segments... Aug 7 14:57:05.706075556 (5h17m5.295420543s) [upload=5524]: Got next segment. Inline: true Aug 7 14:57:05.706078922 (5h17m5.295423909s) [upload=5524]: Waiting for error group managing segments... Aug 7 14:57:05.706109120 (5h17m5.295454107s) [upload=5524]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 14:57:05.706113097 (5h17m5.295458085s) [upload=5524]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 14:57:05.706115382 (5h17m5.295460359s) [upload=5524]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 14:57:05.991692098 (5h17m5.581037085s) [upload=5524]: Done uploading segments... Aug 7 14:57:05.991709140 (5h17m5.581054127s) [upload=5524]: Upload finished. err: <nil> Aug 7 14:58:13.955580031 (5h18m13.544925028s) [upload=5542]: Starting upload Aug 7 14:58:13.955662298 (5h18m13.545007285s) [upload=5542]: Done starting upload Aug 7 14:58:13.955748843 (5h18m13.54509384s) [upload=5542]: Uploading segments... Aug 7 14:58:13.955790593 (5h18m13.54513558s) [upload=5542]: Got next segment. Inline: true Aug 7 14:58:13.955795512 (5h18m13.545140509s) [upload=5542]: Waiting for error group managing segments... Aug 7 14:58:13.955864605 (5h18m13.545209592s) [upload=5542]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 14:58:13.955870396 (5h18m13.545215393s) [upload=5542]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 14:58:13.955873892 (5h18m13.545218879s) [upload=5542]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 14:58:14.317027547 (5h18m13.906372534s) [upload=5542]: Done uploading segments... Aug 7 14:58:14.317044950 (5h18m13.906389937s) [upload=5542]: Upload finished. err: <nil> Aug 7 14:58:30.407376497 (5h18m29.996721474s) [upload=5549]: Starting upload Aug 7 14:58:30.407431412 (5h18m29.996776399s) [upload=5549]: Done starting upload Aug 7 14:58:30.407480204 (5h18m29.996825191s) [upload=5549]: Uploading segments... Aug 7 14:58:30.407548826 (5h18m29.996893813s) [upload=5549]: Got next segment. Inline: true Aug 7 14:58:30.407552403 (5h18m29.99689738s) [upload=5549]: Waiting for error group managing segments... Aug 7 14:58:30.407586347 (5h18m29.996931324s) [upload=5549]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 14:58:30.407590645 (5h18m29.996935623s) [upload=5549]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 14:58:30.407592860 (5h18m29.996937837s) [upload=5549]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 14:58:30.829653516 (5h18m30.418998503s) [upload=5549]: Done uploading segments... Aug 7 14:58:30.829674065 (5h18m30.419019072s) [upload=5549]: Upload finished. err: <nil>
Duration: 2.046044155s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
1.88898189s
FAIL (2025-08-07 15:44:48 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 0c950c360f79058769477f5e701e3fda95fc8e0cb346ec9019ff64f7c3416e1b != before 2cdc378d7ef84049ec07bf6864d59358611769fe343c5ef7867468744bd826f9 ======LOGS====== h item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 14:52:11.554336215 (5h12m11.143681192s) [upload=5437]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 14:52:11.842624981 (5h12m11.431969968s) [upload=5437]: Done uploading segments... Aug 7 14:52:11.842643616 (5h12m11.431988603s) [upload=5437]: Upload finished. err: <nil> Aug 7 14:52:29.842451400 (5h12m29.431796387s) [upload=5442]: Starting upload Aug 7 14:52:29.842494922 (5h12m29.431839899s) [upload=5442]: Done starting upload Aug 7 14:52:29.842567301 (5h12m29.431912288s) [upload=5442]: Uploading segments... Aug 7 14:52:29.842618079 (5h12m29.431963066s) [upload=5442]: Got next segment. Inline: true Aug 7 14:52:29.842622377 (5h12m29.431967354s) [upload=5442]: Waiting for error group managing segments... Aug 7 14:52:29.842657043 (5h12m29.43200203s) [upload=5442]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 14:52:29.842661120 (5h12m29.432006097s) [upload=5442]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 14:52:29.842663386 (5h12m29.432008373s) [upload=5442]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 14:52:30.223558730 (5h12m29.812903717s) [upload=5442]: Done uploading segments... Aug 7 14:52:30.223582075 (5h12m29.812927082s) [upload=5442]: Upload finished. err: <nil> Aug 7 14:53:10.306580950 (5h13m9.895925927s) [upload=5455]: Starting upload Aug 7 14:53:10.306625405 (5h13m9.895970392s) [upload=5455]: Done starting upload Aug 7 14:53:10.306668007 (5h13m9.896012994s) [upload=5455]: Uploading segments... Aug 7 14:53:10.306715197 (5h13m9.896060184s) [upload=5455]: Got next segment. Inline: true Aug 7 14:53:10.306718493 (5h13m9.89606347s) [upload=5455]: Waiting for error group managing segments... Aug 7 14:53:10.306752358 (5h13m9.896097345s) [upload=5455]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 14:53:10.306767648 (5h13m9.896112635s) [upload=5455]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 14:53:10.306770353 (5h13m9.89611534s) [upload=5455]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 14:53:10.689930130 (5h13m10.279275127s) [upload=5455]: Done uploading segments... Aug 7 14:53:10.689953535 (5h13m10.279298552s) [upload=5455]: Upload finished. err: <nil> Aug 7 14:53:33.669638991 (5h13m33.258983968s) [upload=5460]: Starting upload Aug 7 14:53:33.669694297 (5h13m33.259039284s) [upload=5460]: Done starting upload Aug 7 14:53:33.669745955 (5h13m33.259090952s) [upload=5460]: Uploading segments... Aug 7 14:53:33.669775272 (5h13m33.259120249s) [upload=5460]: Got next segment. Inline: true Aug 7 14:53:33.669778608 (5h13m33.259123595s) [upload=5460]: Waiting for error group managing segments... Aug 7 14:53:33.669844874 (5h13m33.259189861s) [upload=5460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 14:53:33.669849273 (5h13m33.25919426s) [upload=5460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 14:53:33.669851847 (5h13m33.259196834s) [upload=5460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 14:53:33.947641263 (5h13m33.53698624s) [upload=5460]: Done uploading segments... Aug 7 14:53:33.947667363 (5h13m33.53701234s) [upload=5460]: Upload finished. err: <nil> Aug 7 14:54:08.475562955 (5h14m8.064907942s) [upload=5472]: Starting upload Aug 7 14:54:08.475591189 (5h14m8.064936176s) [upload=5472]: Done starting upload Aug 7 14:54:08.475631777 (5h14m8.064976764s) [upload=5472]: Uploading segments... Aug 7 14:54:08.475658518 (5h14m8.065003505s) [upload=5472]: Got next segment. Inline: true Aug 7 14:54:08.475661844 (5h14m8.065006831s) [upload=5472]: Waiting for error group managing segments... Aug 7 14:54:08.475693354 (5h14m8.065038342s) [upload=5472]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 14:54:08.475696761 (5h14m8.065041748s) [upload=5472]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 14:54:08.475698955 (5h14m8.065043932s) [upload=5472]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 14:54:08.773471083 (5h14m8.36281606s) [upload=5472]: Done uploading segments... Aug 7 14:54:08.773489398 (5h14m8.362834385s) [upload=5472]: Upload finished. err: <nil> Aug 7 14:55:23.389342434 (5h15m22.978687451s) [upload=5494]: Starting upload Aug 7 14:55:23.389429360 (5h15m22.978774397s) [upload=5494]: Done starting upload Aug 7 14:55:23.389493003 (5h15m22.97883803s) [upload=5494]: Uploading segments... Aug 7 14:55:23.389570009 (5h15m22.978914996s) [upload=5494]: Got next segment. Inline: true Aug 7 14:55:23.389575010 (5h15m22.978919987s) [upload=5494]: Waiting for error group managing segments... Aug 7 14:55:23.389610337 (5h15m22.978955324s) [upload=5494]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 14:55:23.389615887 (5h15m22.978960874s) [upload=5494]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 14:55:23.389618252 (5h15m22.978963229s) [upload=5494]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 14:55:23.741693456 (5h15m23.331038443s) [upload=5494]: Done uploading segments... Aug 7 14:55:23.741717722 (5h15m23.331062709s) [upload=5494]: Upload finished. err: <nil> Aug 7 14:55:47.730252989 (5h15m47.319597966s) [upload=5501]: Starting upload Aug 7 14:55:47.730301272 (5h15m47.319646259s) [upload=5501]: Done starting upload Aug 7 14:55:47.730350075 (5h15m47.319695062s) [upload=5501]: Uploading segments... Aug 7 14:55:47.730382697 (5h15m47.319727684s) [upload=5501]: Got next segment. Inline: true Aug 7 14:55:47.730386825 (5h15m47.319731812s) [upload=5501]: Waiting for error group managing segments... Aug 7 14:55:47.730419067 (5h15m47.319764054s) [upload=5501]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 14:55:47.730422804 (5h15m47.319767791s) [upload=5501]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 14:55:47.730425148 (5h15m47.319770135s) [upload=5501]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 14:55:48.028445502 (5h15m47.617790489s) [upload=5501]: Done uploading segments... Aug 7 14:55:48.028465410 (5h15m47.617810397s) [upload=5501]: Upload finished. err: <nil> Aug 7 14:56:46.695516721 (5h16m46.284861708s) [upload=5518]: Starting upload Aug 7 14:56:46.695559422 (5h16m46.284904409s) [upload=5518]: Done starting upload Aug 7 14:56:46.695606793 (5h16m46.28495178s) [upload=5518]: Uploading segments... Aug 7 14:56:46.695635067 (5h16m46.284980054s) [upload=5518]: Got next segment. Inline: true Aug 7 14:56:46.695640697 (5h16m46.284985684s) [upload=5518]: Waiting for error group managing segments... Aug 7 14:56:46.695672628 (5h16m46.285017615s) [upload=5518]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 14:56:46.695676767 (5h16m46.285021754s) [upload=5518]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 14:56:46.695679482 (5h16m46.285024469s) [upload=5518]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 14:56:46.970794841 (5h16m46.560139848s) [upload=5518]: Done uploading segments... Aug 7 14:56:46.970817584 (5h16m46.560162592s) [upload=5518]: Upload finished. err: <nil> Aug 7 14:57:05.705953271 (5h17m5.295298259s) [upload=5524]: Starting upload Aug 7 14:57:05.705999811 (5h17m5.295344798s) [upload=5524]: Done starting upload Aug 7 14:57:05.706046721 (5h17m5.295391708s) [upload=5524]: Uploading segments... Aug 7 14:57:05.706075556 (5h17m5.295420543s) [upload=5524]: Got next segment. Inline: true Aug 7 14:57:05.706078922 (5h17m5.295423909s) [upload=5524]: Waiting for error group managing segments... Aug 7 14:57:05.706109120 (5h17m5.295454107s) [upload=5524]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 14:57:05.706113097 (5h17m5.295458085s) [upload=5524]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 14:57:05.706115382 (5h17m5.295460359s) [upload=5524]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 14:57:05.991692098 (5h17m5.581037085s) [upload=5524]: Done uploading segments... Aug 7 14:57:05.991709140 (5h17m5.581054127s) [upload=5524]: Upload finished. err: <nil> Aug 7 14:58:13.955580031 (5h18m13.544925028s) [upload=5542]: Starting upload Aug 7 14:58:13.955662298 (5h18m13.545007285s) [upload=5542]: Done starting upload Aug 7 14:58:13.955748843 (5h18m13.54509384s) [upload=5542]: Uploading segments... Aug 7 14:58:13.955790593 (5h18m13.54513558s) [upload=5542]: Got next segment. Inline: true Aug 7 14:58:13.955795512 (5h18m13.545140509s) [upload=5542]: Waiting for error group managing segments... Aug 7 14:58:13.955864605 (5h18m13.545209592s) [upload=5542]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 14:58:13.955870396 (5h18m13.545215393s) [upload=5542]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 14:58:13.955873892 (5h18m13.545218879s) [upload=5542]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 14:58:14.317027547 (5h18m13.906372534s) [upload=5542]: Done uploading segments... Aug 7 14:58:14.317044950 (5h18m13.906389937s) [upload=5542]: Upload finished. err: <nil> Aug 7 14:58:30.407376497 (5h18m29.996721474s) [upload=5549]: Starting upload Aug 7 14:58:30.407431412 (5h18m29.996776399s) [upload=5549]: Done starting upload Aug 7 14:58:30.407480204 (5h18m29.996825191s) [upload=5549]: Uploading segments... Aug 7 14:58:30.407548826 (5h18m29.996893813s) [upload=5549]: Got next segment. Inline: true Aug 7 14:58:30.407552403 (5h18m29.99689738s) [upload=5549]: Waiting for error group managing segments... Aug 7 14:58:30.407586347 (5h18m29.996931324s) [upload=5549]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 14:58:30.407590645 (5h18m29.996935623s) [upload=5549]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 14:58:30.407592860 (5h18m29.996937837s) [upload=5549]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 14:58:30.829653516 (5h18m30.418998503s) [upload=5549]: Done uploading segments... Aug 7 14:58:30.829674065 (5h18m30.419019072s) [upload=5549]: Upload finished. err: <nil>
Duration: 1.889005596s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
1.933050004s
FAIL (2025-08-07 15:43:55 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 0c950c360f79058769477f5e701e3fda95fc8e0cb346ec9019ff64f7c3416e1b != before 2cdc378d7ef84049ec07bf6864d59358611769fe343c5ef7867468744bd826f9 ======LOGS====== h item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 14:52:11.554336215 (5h12m11.143681192s) [upload=5437]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 14:52:11.842624981 (5h12m11.431969968s) [upload=5437]: Done uploading segments... Aug 7 14:52:11.842643616 (5h12m11.431988603s) [upload=5437]: Upload finished. err: <nil> Aug 7 14:52:29.842451400 (5h12m29.431796387s) [upload=5442]: Starting upload Aug 7 14:52:29.842494922 (5h12m29.431839899s) [upload=5442]: Done starting upload Aug 7 14:52:29.842567301 (5h12m29.431912288s) [upload=5442]: Uploading segments... Aug 7 14:52:29.842618079 (5h12m29.431963066s) [upload=5442]: Got next segment. Inline: true Aug 7 14:52:29.842622377 (5h12m29.431967354s) [upload=5442]: Waiting for error group managing segments... Aug 7 14:52:29.842657043 (5h12m29.43200203s) [upload=5442]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 14:52:29.842661120 (5h12m29.432006097s) [upload=5442]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 14:52:29.842663386 (5h12m29.432008373s) [upload=5442]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 14:52:30.223558730 (5h12m29.812903717s) [upload=5442]: Done uploading segments... Aug 7 14:52:30.223582075 (5h12m29.812927082s) [upload=5442]: Upload finished. err: <nil> Aug 7 14:53:10.306580950 (5h13m9.895925927s) [upload=5455]: Starting upload Aug 7 14:53:10.306625405 (5h13m9.895970392s) [upload=5455]: Done starting upload Aug 7 14:53:10.306668007 (5h13m9.896012994s) [upload=5455]: Uploading segments... Aug 7 14:53:10.306715197 (5h13m9.896060184s) [upload=5455]: Got next segment. Inline: true Aug 7 14:53:10.306718493 (5h13m9.89606347s) [upload=5455]: Waiting for error group managing segments... Aug 7 14:53:10.306752358 (5h13m9.896097345s) [upload=5455]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 14:53:10.306767648 (5h13m9.896112635s) [upload=5455]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 14:53:10.306770353 (5h13m9.89611534s) [upload=5455]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 14:53:10.689930130 (5h13m10.279275127s) [upload=5455]: Done uploading segments... Aug 7 14:53:10.689953535 (5h13m10.279298552s) [upload=5455]: Upload finished. err: <nil> Aug 7 14:53:33.669638991 (5h13m33.258983968s) [upload=5460]: Starting upload Aug 7 14:53:33.669694297 (5h13m33.259039284s) [upload=5460]: Done starting upload Aug 7 14:53:33.669745955 (5h13m33.259090952s) [upload=5460]: Uploading segments... Aug 7 14:53:33.669775272 (5h13m33.259120249s) [upload=5460]: Got next segment. Inline: true Aug 7 14:53:33.669778608 (5h13m33.259123595s) [upload=5460]: Waiting for error group managing segments... Aug 7 14:53:33.669844874 (5h13m33.259189861s) [upload=5460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 14:53:33.669849273 (5h13m33.25919426s) [upload=5460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 14:53:33.669851847 (5h13m33.259196834s) [upload=5460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 14:53:33.947641263 (5h13m33.53698624s) [upload=5460]: Done uploading segments... Aug 7 14:53:33.947667363 (5h13m33.53701234s) [upload=5460]: Upload finished. err: <nil> Aug 7 14:54:08.475562955 (5h14m8.064907942s) [upload=5472]: Starting upload Aug 7 14:54:08.475591189 (5h14m8.064936176s) [upload=5472]: Done starting upload Aug 7 14:54:08.475631777 (5h14m8.064976764s) [upload=5472]: Uploading segments... Aug 7 14:54:08.475658518 (5h14m8.065003505s) [upload=5472]: Got next segment. Inline: true Aug 7 14:54:08.475661844 (5h14m8.065006831s) [upload=5472]: Waiting for error group managing segments... Aug 7 14:54:08.475693354 (5h14m8.065038342s) [upload=5472]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 14:54:08.475696761 (5h14m8.065041748s) [upload=5472]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 14:54:08.475698955 (5h14m8.065043932s) [upload=5472]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 14:54:08.773471083 (5h14m8.36281606s) [upload=5472]: Done uploading segments... Aug 7 14:54:08.773489398 (5h14m8.362834385s) [upload=5472]: Upload finished. err: <nil> Aug 7 14:55:23.389342434 (5h15m22.978687451s) [upload=5494]: Starting upload Aug 7 14:55:23.389429360 (5h15m22.978774397s) [upload=5494]: Done starting upload Aug 7 14:55:23.389493003 (5h15m22.97883803s) [upload=5494]: Uploading segments... Aug 7 14:55:23.389570009 (5h15m22.978914996s) [upload=5494]: Got next segment. Inline: true Aug 7 14:55:23.389575010 (5h15m22.978919987s) [upload=5494]: Waiting for error group managing segments... Aug 7 14:55:23.389610337 (5h15m22.978955324s) [upload=5494]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 14:55:23.389615887 (5h15m22.978960874s) [upload=5494]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 14:55:23.389618252 (5h15m22.978963229s) [upload=5494]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 14:55:23.741693456 (5h15m23.331038443s) [upload=5494]: Done uploading segments... Aug 7 14:55:23.741717722 (5h15m23.331062709s) [upload=5494]: Upload finished. err: <nil> Aug 7 14:55:47.730252989 (5h15m47.319597966s) [upload=5501]: Starting upload Aug 7 14:55:47.730301272 (5h15m47.319646259s) [upload=5501]: Done starting upload Aug 7 14:55:47.730350075 (5h15m47.319695062s) [upload=5501]: Uploading segments... Aug 7 14:55:47.730382697 (5h15m47.319727684s) [upload=5501]: Got next segment. Inline: true Aug 7 14:55:47.730386825 (5h15m47.319731812s) [upload=5501]: Waiting for error group managing segments... Aug 7 14:55:47.730419067 (5h15m47.319764054s) [upload=5501]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 14:55:47.730422804 (5h15m47.319767791s) [upload=5501]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 14:55:47.730425148 (5h15m47.319770135s) [upload=5501]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 14:55:48.028445502 (5h15m47.617790489s) [upload=5501]: Done uploading segments... Aug 7 14:55:48.028465410 (5h15m47.617810397s) [upload=5501]: Upload finished. err: <nil> Aug 7 14:56:46.695516721 (5h16m46.284861708s) [upload=5518]: Starting upload Aug 7 14:56:46.695559422 (5h16m46.284904409s) [upload=5518]: Done starting upload Aug 7 14:56:46.695606793 (5h16m46.28495178s) [upload=5518]: Uploading segments... Aug 7 14:56:46.695635067 (5h16m46.284980054s) [upload=5518]: Got next segment. Inline: true Aug 7 14:56:46.695640697 (5h16m46.284985684s) [upload=5518]: Waiting for error group managing segments... Aug 7 14:56:46.695672628 (5h16m46.285017615s) [upload=5518]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 14:56:46.695676767 (5h16m46.285021754s) [upload=5518]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 14:56:46.695679482 (5h16m46.285024469s) [upload=5518]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 14:56:46.970794841 (5h16m46.560139848s) [upload=5518]: Done uploading segments... Aug 7 14:56:46.970817584 (5h16m46.560162592s) [upload=5518]: Upload finished. err: <nil> Aug 7 14:57:05.705953271 (5h17m5.295298259s) [upload=5524]: Starting upload Aug 7 14:57:05.705999811 (5h17m5.295344798s) [upload=5524]: Done starting upload Aug 7 14:57:05.706046721 (5h17m5.295391708s) [upload=5524]: Uploading segments... Aug 7 14:57:05.706075556 (5h17m5.295420543s) [upload=5524]: Got next segment. Inline: true Aug 7 14:57:05.706078922 (5h17m5.295423909s) [upload=5524]: Waiting for error group managing segments... Aug 7 14:57:05.706109120 (5h17m5.295454107s) [upload=5524]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 14:57:05.706113097 (5h17m5.295458085s) [upload=5524]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 14:57:05.706115382 (5h17m5.295460359s) [upload=5524]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 14:57:05.991692098 (5h17m5.581037085s) [upload=5524]: Done uploading segments... Aug 7 14:57:05.991709140 (5h17m5.581054127s) [upload=5524]: Upload finished. err: <nil> Aug 7 14:58:13.955580031 (5h18m13.544925028s) [upload=5542]: Starting upload Aug 7 14:58:13.955662298 (5h18m13.545007285s) [upload=5542]: Done starting upload Aug 7 14:58:13.955748843 (5h18m13.54509384s) [upload=5542]: Uploading segments... Aug 7 14:58:13.955790593 (5h18m13.54513558s) [upload=5542]: Got next segment. Inline: true Aug 7 14:58:13.955795512 (5h18m13.545140509s) [upload=5542]: Waiting for error group managing segments... Aug 7 14:58:13.955864605 (5h18m13.545209592s) [upload=5542]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 14:58:13.955870396 (5h18m13.545215393s) [upload=5542]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 14:58:13.955873892 (5h18m13.545218879s) [upload=5542]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 14:58:14.317027547 (5h18m13.906372534s) [upload=5542]: Done uploading segments... Aug 7 14:58:14.317044950 (5h18m13.906389937s) [upload=5542]: Upload finished. err: <nil> Aug 7 14:58:30.407376497 (5h18m29.996721474s) [upload=5549]: Starting upload Aug 7 14:58:30.407431412 (5h18m29.996776399s) [upload=5549]: Done starting upload Aug 7 14:58:30.407480204 (5h18m29.996825191s) [upload=5549]: Uploading segments... Aug 7 14:58:30.407548826 (5h18m29.996893813s) [upload=5549]: Got next segment. Inline: true Aug 7 14:58:30.407552403 (5h18m29.99689738s) [upload=5549]: Waiting for error group managing segments... Aug 7 14:58:30.407586347 (5h18m29.996931324s) [upload=5549]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 7 14:58:30.407590645 (5h18m29.996935623s) [upload=5549]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 7 14:58:30.407592860 (5h18m29.996937837s) [upload=5549]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 7 14:58:30.829653516 (5h18m30.418998503s) [upload=5549]: Done uploading segments... Aug 7 14:58:30.829674065 (5h18m30.419019072s) [upload=5549]: Upload finished. err: <nil>
Duration: 1.933074321s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|